Sharing mediated reality content

A method, apparatus and computer program code are provided. The method comprises: causing display of mediated reality content by one or more displays associated with a device; identifying a further device with which at least a portion of the mediated reality content is to be shared; determining one or more parameters characterizing one or more displays associated with the further device; determining at least a portion of the mediated reality content to be shared with the further device based, at least in part, on the determined one or more parameters; and causing the portion of the mediated reality content to be shared with the further device.

RELATED APPLICATION

This application was originally filed as Patent Cooperation Treaty Application No. PCT/FI2016/050522 filed Jul. 15, 2016 which claims priority benefit to European Patent Application No. 15179736.2, filed Aug. 4, 2015.

TECHNOLOGICAL FIELD

Embodiments of the present invention relate to sharing mediated reality content such as augmented reality content or virtual reality content.

BACKGROUND

Mediated reality in this document refers to a user experiencing a fully or partially artificial/virtual environment.

Augmented reality is a form of mediated reality in which a user experiences a partially artificial, partially real environment. Virtual reality is a form of mediated reality in which a user experiences a fully artificial/virtual environment.

BRIEF SUMMARY

According to various, but not necessarily all, embodiments of the invention there is provided a method, comprising: causing display of mediated reality content by one or more displays associated with a device; identifying a further device with which at least a portion of the mediated reality content is to be shared; determining one or more parameters characterizing one or more displays associated with the further device; determining at least a portion of the mediated reality content to be shared with the further device based, at least in part, on the determined one or more parameters; and causing the at least a portion of the mediated reality content to be shared with the further device.

According to various, but not necessarily all, embodiments of the invention there is provided computer program code that, when performed by at least one processor, causes at least the following to be performed: causing display of mediated reality content by one or more displays associated with a device; identifying a further device with which at least a portion of the mediated reality content is to be shared; determining one or more parameters characterizing one or more displays associated with the further device; determining at least a portion of the mediated reality content to be shared with the further device based, at least in part, on the determined one or more parameters; and causing the at least a portion of the mediated reality content to be shared with the further device.

One or more computer programs may comprise the computer program code. The one or more computer programs may be stored on one or more non-transitory computer readable mediums.

According to various, but not necessarily all, embodiments of the invention there is provided an apparatus, comprising: means for causing display of mediated reality content by one or more displays associated with a device; means for identifying a further device with which at least a portion of the mediated reality content is to be shared; means for determining one or more parameters characterizing one or more displays associated with the further device; means for determining at least a portion of the mediated reality content to be shared with the further device based, at least in part, on the determined one or more parameters; and means for causing the at least a portion of the mediated reality content to be shared with the further device.

According to various, but not necessarily all, embodiments of the invention there is provided an apparatus, comprising: at least one processor; and at least one memory storing computer program code that is configured, working with the at least one processor, to cause the apparatus to perform at least: causing display of mediated reality content by one or more displays associated with a device; identifying a further device with which at least a portion of the mediated reality content is to be shared; determining one or more parameters characterizing one or more displays associated with the further device; determining at least a portion of the mediated reality content to be shared with the further device based, at least in part, on the determined one or more parameters; and causing the at least a portion of the mediated reality content to be shared with the further device.

According to various, but not necessarily all, embodiments of the invention there is provided a method, comprising: causing display of mediated reality content by one or more displays associated with a device; causing at least a portion of the mediated reality content to be shared with a further device; and causing the at least a portion of the mediated reality content to be demarcated by the one or more displays associated with the device prior to, during or after sharing the at least a portion of the mediated reality content with the further device.

According to various, but not necessarily all, embodiments of the invention there is provided computer program code that, when performed by at least one processor, causes at least the following to be performed: causing display of mediated reality content by one or more displays associated with a device; causing at least a portion of the mediated reality content to be shared with a further device; and causing the at least a portion of the mediated reality content to be demarcated by the one or more displays associated with the device prior to, during or after sharing the at least a portion of the mediated reality content with the further device.

One or more computer programs may comprise the computer program code. The one or more computer programs may be stored on one or more non-transitory computer readable mediums.

According to various, but not necessarily all, embodiments of the invention there is provided an apparatus, comprising: means for causing display of mediated reality content by one or more displays associated with a device; means for causing at least a portion of the mediated reality content to be shared with a further device; and means for causing the at least a portion of the mediated reality content to be demarcated by the one or more displays associated with the device prior to, during or after sharing the at least a portion of the mediated reality content with the further device.

According to various, but not necessarily all, embodiments of the invention there is provided an apparatus, comprising: at least one processor; and at least one memory storing computer program code that is configured, working with the at least one processor, to cause the apparatus to perform at least: causing display of mediated reality content by one or more displays associated with a device; causing at least a portion of the mediated reality content to be shared with a further device; and causing the at least a portion of the mediated reality content to be demarcated by the one or more displays associated with the device prior to, during or after sharing the at least a portion of the mediated reality content with the further device.

According to various, but not necessarily all, embodiments of the invention there is provided an apparatus, comprising: at least one processor; and at least one memory storing computer program code that is configured, working with the at least one processor, to cause the apparatus to perform at least: causing display of mediated reality content by one or more displays associated with a device; enabling a user to choose a starting perspective from which the mediated reality content is initially viewed by a further user of a further device, when the mediated reality content is shared with the further device.

According to various, but not necessarily all, embodiments of the invention there is provided examples as claimed in the appended claims.

DESCRIPTION

Embodiments of the invention relate to sharing of mediated reality content such as virtual reality content or augmented reality content. For example, mediated reality content may be displayed by one or more displays associated with a device. A further device may be identified with which at least a portion of the mediated reality content is to be shared.

One or more parameters characterizing one or more displays associated with the further device may be used by the device in order to determine a portion of the mediated reality content to be shared with the further device.

For instance, in one example, the device may be a personal computer or a games console connected to a head-mounted viewing device and the further device may be a mobile telephone. The games console/personal computer determines one or more parameters characterizing the one or more displays associated with the mobile telephone and uses the parameter(s) to determine an appropriate portion of the mediated reality content to share with the mobile telephone. For example, the parameter(s) could include a resolution, an aspect ratio, a parameter relating to color reproduction or a display type of the one or more displays of the mobile telephone.

A technical effect of embodiments of the invention is an improved, more efficient way of sharing mediated reality content between devices.

Definitions

In this document, the following definitions apply:

“field of view” refers to extent of the observable world that is visible to a user at a particular instance in time;

“virtual space” refers to fully or partially artificial environment, which may be three dimensional;

“virtual scene” refers to a representation of the virtual space viewed from a particular point of view within the virtual space;

“real space” refers to a real environment, which may be three dimensional;

“real scene” refers to a representation of the real space viewed from a particular point of view within the real space;

“mediated reality” in this document refers to a user visually experiencing a fully or partially artificial environment (a virtual space) as a virtual scene at least partially displayed by a computer to a user. The virtual scene is determined by a point of view within the virtual space and a field of view. Displaying the virtual scene means providing it in a form that can be seen by the user;
“mediated reality content” is content which enables a user to visually experience a fully or partially artificial environment (a virtual space) as a virtual scene. Mediated reality content could include interactive content such as a video game or non-interactive content such as motion video;
“augmented reality” in this document refers to a form of mediated reality in which a user visually experiences a partially artificial environment (a virtual space) as a virtual scene comprising a real scene of a physical real world environment (real space) supplemented by one or more visual elements displayed by an apparatus to a user;
“augmented reality content” is a form of mediated reality content which enables a user to visually experience a partially artificial environment (a virtual space) as a virtual scene. Augmented reality content could include interactive content such as a video game or non-interactive content such as motion video;
“virtual reality” in this document refers to a form of mediated reality in which a user visually experiences a fully artificial environment (a virtual space) as a virtual scene displayed by an apparatus to a user;
“virtual reality content” is a form of mediated reality content which enables a user to visually experience a fully artificial environment (a virtual space) as a virtual scene. Virtual reality content could include interactive content such as a video game or non-interactive content such as motion video;
“perspective-mediated” as applied to mediated reality, augmented reality or virtual reality means that user actions determine the point of view within the virtual space, changing the virtual scene;
“first person perspective-mediated” as applied to mediated reality, augmented reality or virtual reality means perspective mediated with the additional constraint that the user's real point of view determines the point of view within the virtual space;
“user interactive-mediated” as applied to mediated reality, augmented reality or virtual reality means that user actions at least partially determine what happens within the virtual space; and
“displaying” means providing in a form that is perceived visually by the user.

DETAILED DESCRIPTION

FIGS. 1A-1C and 2A-2Cillustrate examples of mediated reality. The mediated reality may be augmented reality or virtual reality.

FIGS. 1A, 1B, 1Cillustrate the same virtual space20comprising the same virtual objects21, however, each figure illustrates a different point of view24. The position and direction of a point of view24can change independently. The direction but not the position of the point of view24changes fromFIG. 1AtoFIG. 1B. The direction and the position of the point of view24changes fromFIG. 1BtoFIG. 1C.

FIGS. 2A, 2B, 2Cillustrate a virtual scene22from the perspective of the different points of view24of respectiveFIGS. 1A, 1B, 1C. The virtual scene22is determined by the point of view24within the virtual space20and a field of view26. The virtual scene22is at least partially displayed to a user.

The virtual scenes22illustrated may be mediated reality scenes, virtual reality scenes or augmented reality scenes. A virtual reality scene displays a fully artificial virtual space20. An augmented reality scene displays a partially artificial, partially real virtual space20.

The mediated reality, augmented reality or virtual reality may be user interactive-mediated. In this case, user actions at least partially determine what happens within the virtual space20. This may enable interaction with a virtual object21such as a visual element28within the virtual space20.

The mediated reality, augmented reality or virtual reality may be perspective-mediated. In this case, user actions determine the point of view24within the virtual space20, changing the virtual scene22. For example, as illustrated inFIGS. 1A, 1B, 1Ca position23of the point of view24within the virtual space20may be changed and/or a direction or orientation25of the point of view24within the virtual space20may be changed. If the virtual space20is three-dimensional, the position23of the point of view24has three degrees of freedom e.g. up/down, forward/back, left/right and the direction25of the point of view24within the virtual space20has three degrees of freedom e.g. roll, pitch, yaw. The point of view24may be continuously variable in position23and/or direction25and user action then changes the position and/or direction of the point of view24continuously. Alternatively, the point of view24may have discrete quantised positions23and/or discrete quantised directions25and user action switches by discretely jumping between the allowed positions23and/or directions25of the point of view24.

FIG. 3Aillustrates a real space10comprising real objects11that partially corresponds with the virtual space20ofFIG. 1A. In this example, each real object11in the real space10has a corresponding virtual object21in the virtual space20, however, each virtual object21in the virtual space20does not have a corresponding real object11in the real space10. In this example, one of the virtual objects21, the computer-generated visual element28, is an artificial virtual object21that does not have a corresponding real object11in the real space10.

A linear mapping exists between the real space10and the virtual space20and the same mapping exists between each real object11in the real space10and its corresponding virtual object21. The relative relationship of the real objects11in the real space10is therefore the same as the relative relationship between the corresponding virtual objects21in the virtual space20.

FIG. 3Billustrates a real scene12that partially corresponds with the virtual scene22ofFIG. 1B, it includes real objects11but not artificial virtual objects. The real scene is from a perspective corresponding to the point of view24in the virtual space20ofFIG. 1A. The real scene12content is determined by that corresponding point of view24and the field of view26.

FIG. 2Amay be an illustration of an augmented reality version of the real scene12illustrated inFIG. 3B. The virtual scene22comprises the real scene12of the real space10supplemented by one or more visual elements28displayed by an apparatus to a user. The visual elements28may be a computer-generated visual element. In a see-through arrangement, the virtual scene22comprises the actual real scene12which is seen through a display of the supplemental visual element(s)28. In a see-video arrangement, the virtual scene22comprises a displayed real scene12and displayed supplemental visual element(s)28. The displayed real scene12may be based on an image from a single point of view24or on multiple images from different points of view24at the same time, processed to generate an image from a single point of view24.

FIG. 4Aillustrates an apparatus4in the form of a chip or a chipset. The illustrated apparatus4comprises at least one processor40and at least one memory46. The processor(s)40might be or include a central processing unit (CPU) and/or a graphics processing unit (GPU). The processor40is configured to read from and write to the memory46. The processor40may also comprise an output interface via which data and/or commands are output by the processor40and an input interface via which data and/or commands are input to the processor40.

The memory46stores a computer program148comprising computer program instructions (computer program code)48that controls the operation of the apparatus4/30when loaded into the processor40. The computer program instructions48, of the computer program148, provide the logic and routines that enables the apparatus4/30to perform the methods illustrated inFIGS. 6A & 6B and 8. The processor40by reading the memory46is able to load and execute the computer program48.

The computer program148may arrive at the apparatus4/30via any suitable delivery mechanism. The delivery mechanism may be, for example, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a compact disc read-only memory (CD-ROM) or digital versatile disc (DVD), an article of manufacture that tangibly embodies the computer program148. The delivery mechanism may be a signal configured to reliably transfer the computer program148. The signal could, for example, be sent over a wireless connection (such as a radio frequency connection) or a wired connection in accordance with one or more protocols. The apparatus4/30may cause the computer program148to be transmitted as a computer data signal.

Although the processor40is illustrated as a single component/circuitry it may be implemented as multiple processors, such as one or more separate components/circuitry some or all of which may be integrated/removable. The processor(s)40may be single core or multi-core.

FIG. 4Billustrates a schematic of an apparatus30that is operable to enable mediated reality and/or augmented reality and/or virtual reality.

In some embodiments, the apparatus30might be a single electronic device such as a head-mounted viewing device. An example of such a head-mounted viewing device is illustrated inFIG. 5.

In other embodiments, the apparatus30could be distributed across multiple devices, which may be formed from a combination of a head-mounted viewing device, a games console/personal computer and/or one or more hand-held controllers. Where the apparatus30is formed at least in part by a games console or a personal computer, the processor40and the memory46(or, where multiple processors and/or multiple memories are provided, one or more of either or both) may be provided in the games console/personal computer.

In the illustrated example, the apparatus30comprises the apparatus4illustrated inFIG. 4A, one or more displays32, display optics33, one or more transceivers42and user input circuitry44.

The one or more displays32are for providing at least parts of the virtual scene22to a user in a form that is perceived visually by the user. Such a virtual scene may form part of mediated reality content, such as virtual reality content or augmented reality content. The display(s)32may be one or more visual displays that provides light which displays at least parts of the virtual scene22to a user. Examples of visual displays include liquid crystal displays, organic light emitting displays, emissive, reflective, transmissive and transflective displays, direct retina projection display, near eye displays etc. The display(s)32is controlled in this example but not necessarily all examples by the processor40.

If the apparatus30comprises a head-mounted viewing device, the display(s)32may be positioned close to the user's eyes when the user is wearing the head-mounted viewing device, with the display optics33positioned between the user's eyes and the display(s)32.

The display optics33are configured, in conjunction with the display(s)32, to cause the user to perceive that virtual objects displayed by the display(s)32are positioned at a different distance from the user than the distance between the user's eyes and the display(s)32itself.

The display optics33may be, or include, a fisheye lens for each eye of the user. Each fisheye lens may be configured to produce visual distortion to create a wide panoramic/hemispherical image.

When the head-mounted viewing device is worn by a user, the display optics and the display(s)32are such that a first display portion of the display(s)32forms images that are conveyed to a first eye of the user and not a second eye, and a second display portion of the display(s)32forms images that are conveyed to the second eye of the user and not the first eye.

In some embodiments, a single display32is provided for both of the user's eyes. In such embodiments, one portion/half of the display32is viewed by the first eye of the user (and not the second eye of the user) when the head-mounted viewing device is worn and the other portion/half32is viewed by the second eye of the user (and not the first eye of the user).

In other embodiments, the first display portion is a first display and the second display portion is a second, different, display. In such embodiments, the first display32is viewed by the first eye of the user (and not the second eye of the user) when the head-mounted viewing device is worn and the second display32is viewed by the second eye of the user (and not the first eye of the user) when the head-mounted viewing device is worn.

The processor40may cause the user to perceive a stereoscopic effect by displaying content on the first display portion that is offset in space relative to content displayed on the second display portion. The geometry of virtual objects displayed on the display(s)32(if they were viewed, for example, without the aid of the display optics33) may be different from the geometry of the virtual objects that is perceived by the user when the head-mounted viewing device is worn and the virtual objects are viewed through the display optics33, due to the optical properties of the display optics33.

A head-mounted viewing device moves when the head of the user moves. The head-mounted viewing device may be a see-through arrangement for augmented reality that enables a live real scene12to be viewed while one or more visual elements28are displayed by the display(s)32to the user to provide in combination the virtual scene22. In this case a visor, if present, is transparent or semi-transparent so that the live real scene12can be viewed through the visor.

A head-mounted viewing device may be operated as a see-video arrangement for augmented reality that enables a live or recorded video of a real scene12to be displayed by the display(s)32for viewing by the user while one or more visual elements28are simultaneously displayed by the display(s)32for viewing by the user. The combination of the displayed real scene12and displayed one or more visual elements28provides the virtual scene22to the user. In this case the visor is opaque and may be used as the display(s)32.

The one or more transceivers42are configured to receive inputs from the processor40to provide outputs to the processor40. For example, the one or more transceivers42may receive data from the processor40and transmit it, and provide received data to the processor40.

The one or more transceivers42may include one or more wireless transceivers and/or one or more wired transceivers. Such wireless transceivers may, for example, include radio frequency receivers in the form of one or more long range cellular transceivers or short range wireless transceivers (which, for example, may operate in accordance with an Institute of Electrical and Electronic Engineers wireless local area network 802.11 protocol or a Bluetooth protocol). Such wired transceivers may, for example, include a Universal Serial Bus (USB) transceiver.

In the illustrated example, the user input circuitry44may comprise one or more tactile sensors43, one or more point of view sensors45one or more image sensors47for imaging real space10and one or more depth sensors49.

The one or more tactile sensors43may include, for example, one or more joysticks and one or more keys/buttons. The joystick(s) and/or the key(s)/button(s) may form part of a physical, hand-held controller. If the apparatus30is head-mounted, at least some of the one or more tactile sensors43may be positioned on the head-mounted apparatus.

The apparatus30may enable user-interactive mediation for mediated reality and/or augmented reality and/or virtual reality. The user input circuitry44detects user actions using user input, such as via the tactile sensor(s)43. These user actions are used by the processor40to determine what happens within the virtual space20. This may enable interaction with a visual element28within the virtual space20.

The apparatus30may enable perspective mediation for mediated reality and/or augmented reality and/or virtual reality. The user input circuitry44detects user actions. These user actions are used by the processor40to determine the point of view24within the virtual space20, changing the virtual scene22. The point of view24may be continuously variable in position and/or direction and user action changes the position and/or direction of the point of view24. Alternatively, the point of view24may have discrete quantised positions and/or discrete quantised directions and user action switches by jumping to the next position and/or direction of the point of view24.

The apparatus30may enable first person perspective for mediated reality, augmented reality or virtual reality. The user input circuitry44detects the user's real point of view14using user point of view sensor45. The user's real point of view is used by the processor40to determine the point of view24within the virtual space20, changing the virtual scene22. Referring back toFIG. 3A, a user18has a real point of view14. The real point of view may be changed by the user18. For example, a real location13of the real point of view14is the location of the user18and can be changed by changing the physical location13of the user18. For example, a real direction15of the real point of view14is the direction in which the user18is looking and can be changed by changing the real direction of the user18. The real direction15may, for example, be changed by a user18changing an orientation of their head or view point and/or a user changing a direction of their gaze. A head-mounted apparatus30may be used to enable first-person perspective mediation.

The apparatus30may comprise as part of the input circuitry44point of view sensors45for determining changes in the real point of view.

For example, positioning technology such as GPS, triangulation (trilateration) by transmitting to multiple receivers and/or receiving from multiple transmitters, acceleration detection and integration may be used to determine a new physical location13of the user18and real point of view14.

For example, accelerometers, electronic gyroscopes or electronic compasses may be used to determine a change in an orientation of a user's head or view point and a consequential change in the real direction15of the real point of view14.

For example, pupil tracking technology, based for example on computer vision, may be used to track movement of a user's eye or eyes and therefore determine a direction of a user's gaze and consequential changes in the real direction15of the real point of view14.

The apparatus30may comprise as part of the input circuitry44image sensors47for imaging the real space10.

An example of an image sensor47is a digital image sensor that is configured to operate as a camera. Such a camera may be operated to record static images and/or video images In some, but not necessarily all embodiments, cameras may be configured in a stereoscopic or other spatially distributed arrangement so that the real space10is viewed from different perspectives. This may enable the creation of a three-dimensional image and/or processing to establish depth, for example, via the parallax effect.

In some, but not necessarily all embodiments, the input circuitry44comprises depth sensors49. A depth sensor49may comprise a transmitter and a receiver. The transmitter transmits a signal (for example, a signal a human cannot sense such as ultrasound or infrared light) and the receiver receives the reflected signal. Using a single transmitter and a single receiver some depth information may be achieved via measuring the time of flight from transmission to reception. Better resolution may be achieved by using more transmitters and/or more receivers (spatial diversity). In one example, the transmitter is configured to ‘paint’ the real space10with light, preferably invisible light such as infrared light, with a spatially dependent pattern. Detection of a certain pattern by the receiver allows the real space10to be spatially resolved. The distance to the spatially resolved portion of the real space10may be determined by time of flight and/or stereoscopy (if the receiver is in a stereoscopic position relative to the transmitter).

The apparatus30may enable mediated reality and/or augmented reality and/or virtual reality, for example using the method60illustrated inFIG. 6Aor a similar method. The processor40stores and maintains a model50of the virtual space20. The model may be provided to the processor40or determined by the processor40. For example, sensors in input circuitry44may be used to create overlapping depth maps of the virtual space from different points of view and a three dimensional model may then be produced.

At block62it is determined whether or not the model of the virtual space20has changed. If the model of the virtual space20has changed the method moves to block66. If the model of the virtual space20has not changed the method moves to block64.

At block64it is determined whether or not the point of view24in the virtual space20has changed. If the point of view24has changed the method moves to block66. If the point of view24has not changed the method returns to block62.

At block66, a two-dimensional projection of the three-dimensional virtual space20is taken from the location23and in the direction25defined by the current point of view24. The projection is then limited by the field of view26to produce the virtual scene22. The method then returns to block62.

Where the apparatus30enables augmented reality, the virtual space20comprises objects11from the real space10and also visual elements28not present in the real space10. The combination of such visual elements28may be referred to as the artificial virtual space.FIG. 5Billustrates a method70for updating a model of the virtual space20for augmented reality.

At block72it is determined whether or not the real space10has changed. If the real space10has changed the method moves to block76. If the real space10has not changed the method moves to block74. Detecting a change in the real space10may be achieved at a pixel level using differentiating and may be achieved at an object level using computer vision to track objects as they move.

At block74it is determined whether or not the artificial virtual space has changed. If the artificial virtual space has changed the method moves to block76. If the artificial virtual space has not changed the method returns to block72. As the artificial virtual space is generated by the controller42changes to the visual elements28are easily detected.

At block76, the model of the virtual space20is updated.

In some but not necessarily all embodiments, the input circuitry44may comprise communication circuitry41in addition to or as an alternative to one or more of the image sensors47and the depth sensors49. Such communication circuitry41may communicate with one or more remote image sensors47in the real space10and/or with remote depth sensors49in the real space10. The communication circuitry41may form part of the transceiver(s)42.

Referring back toFIG. 4B, the apparatus30may enable user-interactive mediation for mediated reality and/or augmented reality and/or virtual reality. The user input circuitry44detects user actions from user input. These user actions are used by the processor40to determine what happens within the virtual space20. This may enable interaction with a visual element28within the virtual space20.

The detected user actions may, for example, be gestures performed in the real space10. Gestures may be detected in a number of ways. For example, depth sensors49may be used to detect movement of parts a user18and/or or image sensors47may be used to detect movement of parts of a user18and/or positional/movement sensors attached to a limb of a user18may be used to detect movement of the limb.

Object tracking may be used to determine when an object or user moves. For example, tracking the object on a large macro-scale allows one to create a frame of reference that moves with the object. That frame of reference can then be used to track time-evolving changes of shape of the object, by using temporal differencing with respect to the object. This can be used to detect small scale human motion such as gestures, hand movement, facial movement. These are scene independent user (only) movements relative to the user.

The apparatus30may track a plurality of objects and/or points in relation to a user's body, for example one or more joints of the user's body. In some examples, the apparatus30may perform full body skeletal tracking of a user's body.

The tracking of one or more objects and/or points in relation to a user's body may be used by the apparatus30in gesture recognition and so on.

FIG. 7illustrates a schematic of the apparatus30in communication with one or more devices304,305,306,308. By way of example, inFIG. 7the apparatus30is formed of a first device in the form of a games console301, a second device302in the form of a head mounted viewing device302and a third device303in the form of a hand-held controller303inFIG. 7.

In this example, the processor40, the memory46and the transceiver(s)42of the apparatus30are provided in the games console301, the display(s)32and the display optics33of the apparatus30are provided in the head-mounted viewing device302and the one or more tactile sensors43of the apparatus30are provided in the hand-held controller303.

FIG. 7illustrates a fourth device in the form of a first mobile telephone304orientated in a landscape orientation, a fifth device in the form of a second mobile telephone305in a portrait orientation, sixth and seventh devices in the form of a personal computer306connected to a head-mounted viewing device307and an eighth device in the form of a laptop computer308. The games console301may be in wired or wireless communication with any or all of the devices304,305,307,308via the transceiver(s)42.

A method according to embodiments of the invention will now be described in relation toFIG. 8. In block801ofFIG. 8, the processor40of the games console301causes the display(s)32of the head-mounted viewing device302to display mediated reality content90.

FIG. 9illustrates a user82viewing (visual) mediated reality content90displayed by the display(s)32of the head-mounted viewing device302. A virtual scene is displayed which includes virtual objects. The head-mounted viewing device302enables the user82to view the mediated reality content90in a first person perspective-mediated manner.

In the example described below and illustrated in the figures, the mediated reality content90is (visual) virtual reality content that enables the user to experience a fully artificial/virtual environment. In other examples, the mediated reality content90could be augmented reality content that enables the user to experience a partially artificial/virtual, partially real environment.

The mediated reality content90extends around the user82and beyond the field of view of the user82. This means that it is not possible for the user82to see the whole of the displayed mediated reality content90at any one instance in time. The mediated reality content90may extend 360 degrees around the user82in both a horizontal plane and a vertical plane, but this is not shown inFIG. 9for clarity purposes.

The whole of the mediated reality content90need not be displayed at the same time. That is, the angular range over which the mediated reality content90may be viewed by the user82by moving his head and/or eyes may be greater than the angular range over which the mediated reality content90is displayed by the display(s)32at a particular instance in time.

As the user82experiences the mediated reality content90, the processor40continually cycles through the method illustrated inFIG. 6Aand, in the case of augmented reality content, orFIG. 6B.

In block802ofFIG. 8, the games console301identifies a further device with which a least a portion of the mediated reality content90is to be shared. In this example, the further device is the first mobile telephone304in landscape orientation.

The identification of the further device need not occur after the commencement of the display of the mediated reality content90by the head-mounted viewing device302. It could instead occur before the commencement of the display of the mediated reality content90by the head-mounted viewing device302.

In some circumstances, the further device may send a request to the games console301to receive shared content, which is received at the transceiver(s)42and processed by the processor40. In such circumstances, content is being “pulled” from the games console301. In other circumstances, the processor40may control the transceiver(s)42to scan for devices, such as nearby devices, in order to identify the further device. The transceiver(s)42of the games console301, under the control of the processor40, may transmit a request to share content with the further device.

The identification of the further device that takes place in block802inFIG. 8may be identification of a device that is already in communication with the games console301via the transceiver(s)42. For example, a communication link may already have been established, but the further device may not have been identified for sharing purposes until block802inFIG. 8. Alternatively, the further device may not yet be in communication with the games console301when a request to receive shared content is received. In these circumstances, content is being “pushed” by the games console301.

In block803, the processor40of the games console301determines one or more parameters characterizing one or more displays associated with the further device. In this example, the further device is the first mobile telephone304which has a single display. The display of the first mobile telephone304may, for example, be characterized by parameters such as: a resolution of the display, an aspect ratio of the display, a parameter relating to color reproduction of the display and/or a display type of the display. These parameters may characterize permanent properties of the display, such as those described above, or one or more transient properties, such as a current orientation of the display.

For example, a parameter defining the resolution of the display of the further device may indicate that the resolution of the display is 1280×720 (high definition), 1920×1080 (full high definition), 2560×1440 (quad high definition) or any other resolution. A parameter defining the aspect ratio of the display may indicate that the aspect ratio of the display is 16:9, 4:3 or any other aspect ratio. A parameter relating to color reproduction may indicate whether the display is a color display or a black and white display (such as an electronic-ink display). A parameter defining a display type of the display may indicate that the display is a flat display, a curved display, a non-stereoscopic display, a stereoscopic display, etc.

The one or more parameters may be received in a signal which is transmitted by the first mobile telephone304, via a wired or wireless connection, to the transceiver(s)42of the games console301. In this example, the processor40of the games console301determines from the parameters that the display of the first mobile telephone304has a 16:9 aspect ratio, a 1280×720 resolution, the display is a color display, the display is flat and the display is currently in a landscape orientation (or that content that is in a landscape orientation is requested by the first mobile telephone304).

In block804inFIG. 8, the processor40of the games console301determines at least a portion of the mediated reality content90to be shared with the further device based on the determined parameters.

In this example, the mediated reality content90is rendered for display by the display(s)32of the head-mounted viewing device302by the processor40of the games console301. The rendering may depend, at least in part, on the resolution(s) of the display(s)32of the head-mounted viewing device302.

The display(s)32of the head-mounted viewing device302could be of any resolution. By way of example, in this instance the display(s)32will be considered to be a single display with a resolution of 7680×2160. A first display portion of the display32, having a resolution of 3840×2160 is viewed by a first eye of the user82(and not the second eye of the user82) and a second display portion of the display32, having a resolution of 3840×2160, is viewed by the second eye of the user82(and not the first eye of the user82).

In order to achieve a stereoscopic effect, the images that are displayed by the first display portion of the display32are offset in space relative to the images that are displayed by the second display portion of the display32. Some information/content that is displayed by the first display portion is not displayed by the second display portion. That is, some information/content can be seen by the first eye of the user and not the second eye of the user, and vice versa.

The processor40of the games console301processes the mediated reality content90to identify a portion of the mediated reality content90to be shared with the first mobile telephone304in dependence upon the parameters which characterize the display of the mobile telephone304. For example, the processor40may identify a portion of the mediated reality content90that has the same resolution as the resolution of the display of the first mobile telephone304.

The determination of the portion of the mediated reality content to be shared with the further device may depend upon the implementation of the apparatus30. In some implementations, the processor40may determine an area, in pixels, of the mediated reality content90which is displayed on the display(s)32that is of the same resolution as the resolution of the display associated with the further device. The portion of the mediated reality content90that is selected may depend on inputs provided by the user via the user input circuitry44.

In this example, the user provides inputs via the tactile user input sensors43of the hand-held controller303and selects a portion/area of the mediated reality content90on the basis of those inputs. The processor40might only obtain content/information that is provided for display on one of the display portions in order to share that content with the first mobile telephone304. In this example, much of the content that is displayed on the first display portion of the display32is replicated on the second display portion of the display32to obtain a stereoscopic effect, so it might only be needed to obtain information/content from that which is being provided to one display portion in order to share the selected determined portion of content.

In some implementations, the processor40may process the mediated reality content90based on the optical properties of the display optics33(for example, based on the distortion effect of one or more fisheye lenses), so the mediated reality content90is suitable for viewing through display optics33.

If so, the determined portion of the mediated reality content90may be obtained by the processor40prior to this processing being performed on the mediated reality content90, or, alternatively, post processing may be performed on the selected portion of the mediated reality content90which is output to the display(s)32to account for the optical effects of the display optics33, prior to the selected portion of mediated reality content90being shared with the first mobile telephone304. Otherwise, the geometry of virtual objects displayed by the display32may appear distorted if they were viewed, for instance, without the aid of the display optics33on the first mobile telephone304.

In block805inFIG. 8, the processor40of the games console301causes the portion of mediated reality content90to be shared with the further device. For example, the processor40of the games console301may control the transceiver(s)42to transmit the portion of the mediated reality content90directly to the first mobile telephone304or via one or more networks, such as the internet.

The portion of the mediated reality content90that is shared with the further device/first mobile telephone304is derived from the mediated reality content90that is rendered for display on the head-mounted viewing device302by the games console301.

The portion of the mediated reality content90that is shared could be a static image. Alternatively, it could be motion video taken from an interactive video game or non-interactive motion video. In such instances, the games console301may continue to provide the portion of the mediated reality content90to the first mobile telephone304over a period of time.

In some implementations, the user82that is viewing the mediated reality content90via the head-mounted viewing device302may be provided with an indication101that demarcates the shared portion of the mediated reality content90. Such an indication101could be provided before, during and/or after sharing. The indication is illustrated inFIG. 10A. InFIG. 10A, the indication101has an area which corresponds with the portion of the mediated reality content90that is being/will be/has been shared based on the resolution of the display associated with the further device. In this example, the indication101illustrated inFIG. 10Acorresponds with the portion of the mediated reality content90that is being/will be/has been shared based on the resolution of the display of the first mobile telephone304.

It will be understood by those skilled in the art that the games console301may also share a portion of the mediated reality content90with the second mobile telephone305, the personal computer306and/or the laptop computer308in the manner described above in relation to the sharing of a portion of mediated reality content with the first mobile telephone304.

FIG. 10Brelates to an example in which a portion of mediated reality content90is being shared with the laptop computer308. In this example, the resolution of the display of the laptop computer is 1920×1080. Since the size of indication102of the portion of mediated reality content90being shared depends on the resolution of the display of the laptop computer308, the indication102illustrated inFIG. 10Bis larger than the indication101illustrated inFIG. 10A.

FIG. 10Cillustrates an example in which the games console301is sharing a portion of mediated reality content90with the second mobile telephone305. The second mobile telephone305is in portrait orientation, so the indication103of the portion of mediated reality content90being shared with the second mobile telephone305has a different orientation from the indication101illustrated inFIG. 10A. In this example, however, the first mobile telephone304and the second mobile telephone305have the same resolution, so the size of the indication103illustrated inFIG. 10Cis the same as the size of the indication101illustrated inFIG. 10A.

The personal computer306illustrated inFIG. 7is associated with (in this case, electrically connected to) a head-mounted viewing device307. The head-mounted viewing device307enables a user/wearer to view mediated reality content in a first person perspective-mediated manner. The games console301may, for example, share the whole of the mediated reality content90with the personal computer306, such that the mediated reality content90is displayed by the head-mounted viewing device307in the same manner that is displayed by the head-mounted viewing device302. The shared mediated reality content90that is displayed on the head-mounted viewing device307connected to the personal computer306is, however, rendered by the games console301.

FIG. 11Aillustrates a reticule104that the user82of the games console301and the head-mounted viewing device302may guide, for instance, using the one or more tactile sensors43, to choose a starting perspective/point of view for another user when the mediated reality content90is shared. This could be done by selecting a particular object of interest in the mediated reality content90. In the embodiment illustrated inFIG. 11A, the whole of the mediated reality content90may be shared.

If, for example, the mediated reality content90is shared with the personal computer306and displayed by the head-mounted viewing device307, the perspective/point of view of the mediated reality content90that the user of the head-mounted viewing device307is initially provided with is that which was chosen by the user82of the head mounted viewing device302. The user of the head-mounted viewing device307may therefore initially see an object of interest in the mediated reality content90that was chosen by the user82.

In some embodiments, it may be possible for the user to provide inputs, for example, via the one or more tactile sensors43, to cause the processor40to zoom into the mediated reality content90before a portion of the mediated reality content90is selected for sharing. For instance, the user82may, for instance, use the one or more tactile sensors43, to choose a section of the mediated reality content90to zoom into. This magnifies the mediated reality content90to the user82. A magnified portion of the mediated reality content90may then be selected for sharing. Selection of a magnified portion of the mediated reality content90for sharing with a further device may result in the shared portion having a lower resolution than the resolution of the display associated with the further device.

FIG. 11Billustrates an example where the dotted box103indicates the resolution of the display associated with the further device, relative to the display(s)32being used to display the mediated reality content90. In this example, the user has zoomed into the area labeled with the reference numeral104. The processor40causes the display(s)32to display a magnified version105of the area104to demonstrate to the user82how the magnified area would appear on the display associated with the further device.

The processor40may demonstrate how the portion of the mediated reality content90would appear when it is displayed by the display of the further device in other ways. For instance, if the display associated with the further device were a black and white display, the portion of the mediated reality content90that is to be shared, is being shared or has been shared may be displayed in black and white to the user by the display(s)32of the head-mounted viewing device302, under the control of the processor40of the games console301.

In some embodiments of the invention, it is possible for the apparatus30to share different portions of the mediated reality content90with different devices at the same time. The different portions being shared may be demarcated by the display(s)32.FIG. 12Aillustrates an example in which different portions of mediated reality content are being shared with different devices at the same time. The indicator labeled with the reference numeral106indicates a portion of the mediated reality content90that is being shared with a device associated with a user named “Jim”. The indicator labeled with the reference numeral107indicates a portion of the mediated reality content90that is being shared with a device associated with a user named “Ashley”. The indicator labeled with the reference numeral108indicates a portion of the mediated reality content90that is being shared with a device associated with a user named “Matthew”.

As can be seen inFIG. 12A, there is some overlap between the content that is being shared with each of the devices in the example illustrated inFIG. 12A.

In some instances, different portions of the mediated reality content90may be shared with different devices that are associated with the same user. This is illustrated inFIG. 12B, where an indicator108illustrates content being shared with a mobile telephone associated with the user “Matthew” and the indicator109indicates a portion of content being shared with a smart watch that is also associated with the user “Matthew”.

The blocks illustrated in theFIGS. 6A, 6B and 8may represent steps in a method and/or sections of code in the computer program148. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block may be varied. Furthermore, it may be possible for some blocks to be omitted.

As used in this application, the term ‘circuitry’ refers to all of the following:

Although embodiments of the present invention have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the invention as claimed. For example, it will be apparent to those skilled in the art that the apparatus30need not be a combination of a games console301, a head-mounted viewing device302and a hand-held controller303as described above. It could, for example, be a combination of a personal computer and a head-mounted viewing device, or merely a head-mounted viewing device.

In some embodiments, the display(s)32might be holographic. In such embodiments, a head-mounted viewing device might not be required.