Patent Description:
When mediated reality is rendered to a user the user experiences, for example visually and/or or aurally, a fully or partially artificial environment as a virtual scene at least partially rendered by an apparatus to a user. The virtual scene is determined by a point of view of a virtual user within a virtual space.

Augmented reality is a form of mediated reality in which a user experiences a partially artificial environment as a virtual scene comprising a real scene, for example a real visual scene and/or real sound scene, of a physical real environment supplemented by one or more visual or audio elements rendered by an apparatus to a user. The term augmented reality implies a mixed reality or hybrid reality. Virtual reality is a form of mediated reality in which a user experiences a fully artificial environment as a virtual scene rendered by an apparatus to a user.

First person perspective-mediated, as applied to mediated reality means that the user's real point of view (location and/or orientation) determines the point of view (location and/or orientation) within the virtual space of a virtual user.

When mediated reality is rendered to a user as a sound scene, a three-dimensional sound field may be created.

<CIT> discloses embodiments related to outputting spatial audio via a portable computing device. For example, one disclosed embodiment provides a portable computing device for wirelessly detecting a presence of a remote speaker external to the portable computing device, determining a location of the remote speaker, and establishing a wireless connection with the remote speaker. The embodiment further includes presenting visual <NUM> content via a display of the portable computing device, the visual content comprising an object positionally associated with a location in a real-world environment around the portable computing device, generating an audio signal corresponding to the object, and sending the audio signal to the remote speaker.

<CIT> discloses a method comprising: enabling display to a user of a virtual visual scene; in dependence upon detection of an occurrence of a predetermined action by a user, starting display to the user of one or more control visual elements in the virtual visual scene while the virtual visual scene is being displayed to the user; and enabling interaction by the user with the one or more control visual elements within the displayed virtual visual scene to enable control while the virtual visual scene is being displayed to the user, wherein the one or more control visual elements displayed in the virtual visual scene represent a controller device physically used by the user, within the virtual visual scene, wherein the predetermined action by the user is in relation to a first portion of the physical controller device physically used by the user, and wherein the one or more control visual elements displayed in the virtual visual scene represent at least the first portion of the physical controller device within the virtual visual scene.

<CIT> discloses a method comprising: dividing a virtual space using virtual partitions that affect perception of the virtual space by a user within the virtual space; in response to a first action in the virtual space relative to a first virtual partition by a user making a first change to how the first virtual partition affects the virtual space perceived by the user.

<CIT> discloses a head mounded display that outputs an audio signal of an external digital device according to whether the audio signal is directional output signal.

According to various, but not necessarily all, embodiments there is provided an apparatus, a hand portable device, a method and a computer program according to the independent claims.

Three degrees of freedom (3DoF) describes mediated reality where the virtual position is determined by orientation only (e.g. the three degrees of three-dimensional orientation). An example of three degrees of three-dimensional orientation is pitch, roll and yaw. In relation to first person perspective-mediated reality 3DoF, only the user's orientation determines the virtual position.

Six degrees of freedom (6DoF) describes mediated reality where the virtual position is determined by both orientation (e.g. the three degrees of three-dimensional orientation) and location (e.g. the three degrees of three-dimensional location). An example of three degrees of three-dimensional orientation is pitch, roll and yaw. An example of three degrees of three-dimensional location is a three-dimensional coordinate in a Euclidian space spanned by orthogonal axes such as left -to-right (x), front to back (y) and down to up (z) axes. In relation to first person perspective-mediated reality 6DoF, both the user's orientation and the user's location in the real space determine the virtual position. In relation to third person perspective-mediated reality 6DoF, the user's location in the real space does not determine the virtual position. The user's orientation in the real space may or may not determine the virtual position.

Three degrees of freedom 'plus' (3DoF+) describes an example of six degrees of freedom where a change in location (e.g. the three degrees of three-dimensional location) is a change in location relative to the user that can arise from a postural change of a user's head and/or body and does not involve a translation of the user through real space by, for example, walking.

<FIG> illustrate first person perspective mediated reality. In this context, mediated reality means the rendering of mediated reality for the purposes of achieving mediated reality for a remote user, for example augmented reality or virtual reality. It may or may not be user interactive. The mediated reality may support 3DoF, 3DoF+ or 6DoF. Alternatively, the mediated reality may support 3DoF or 6DoF, not 3DoF+.

<FIG> illustrate at a first time a real space <NUM>, a sound space <NUM> and a visual space <NUM>. There is correspondence between the sound space <NUM> and the virtual visual space <NUM>. A user <NUM> in the real space <NUM> has a point of view (a position) <NUM> defined by a location <NUM> and an orientation <NUM>. The location is a three-dimensional location and the orientation is a three-dimensional orientation.

In an example of 3DoF mediated reality, the user's real point of view <NUM> (orientation) determines the point of view <NUM> (virtual position) within the virtual space of a virtual user. An orientation <NUM> of the user <NUM> controls a virtual orientation <NUM> of a virtual user <NUM>. There is a correspondence between the orientation <NUM> and the virtual orientation <NUM> such that a change in the orientation <NUM> produces the same change in the virtual orientation <NUM>.

The virtual orientation <NUM> of the virtual user <NUM> in combination with a virtual field of view <NUM> defines a virtual visual scene <NUM> within the virtual visual space <NUM>. In some examples, it may additionally or alternatively define a virtual sound scene <NUM>.

A virtual visual scene <NUM> is that part of the virtual visual space <NUM> that is displayed to a user. A virtual sound scene <NUM> is that part of the virtual sound space <NUM> that is rendered to a user. The virtual sound space <NUM> and the virtual visual space <NUM> correspond in that a position within the virtual sound space <NUM> has an equivalent position within the virtual visual space <NUM>. In 3DoF mediated reality, a change in the location <NUM> of the user <NUM> does not change the virtual location <NUM> or virtual orientation <NUM> of the virtual user <NUM>.

In the example of 6DoF mediated reality, the user's real point of view <NUM> (location and/or orientation) determines the point of view <NUM>(virtual position) within the virtual space of a virtual user <NUM>. The situation is as described for 3DoF and in addition it is possible to change the rendered virtual sound scene <NUM> and the displayed virtual visual scene <NUM> by movement of a location <NUM> of the user <NUM>. For example, there may be a mapping between the location <NUM> of the user <NUM> and the virtual location <NUM> of the virtual user <NUM>. A change in the location <NUM> of the user <NUM> produces a corresponding change in the virtual location <NUM> of the virtual user <NUM>. A change in the virtual location <NUM> of the virtual user <NUM> changes the rendered sound scene <NUM> and also changes the rendered visual scene <NUM>.

This may be appreciated from <FIG> which illustrate the consequences of a change in location <NUM> and orientation <NUM> of the user <NUM> on respectively the rendered sound scene <NUM> (<FIG>) and the rendered visual scene <NUM> (<FIG>). The change in location may arise from a postural change of the user and/or a translation of the user by walking or otherwise.

First person perspective mediated reality may control only a virtual sound scene <NUM>, a virtual visual scene <NUM> and both a virtual sound scene <NUM> and virtual visual scene <NUM>, depending upon implementation.

In some situations, for example when the sound scene is rendered to a listener through a head-mounted audio output device, for example headphones using binaural audio coding, it may be desirable for a portion of the rendered sound space to remain fixed in real space when the listener turns their head in space. This means that the rendered sound space needs to be rotated relative to the audio output device by the same amount in the opposite sense to the head rotation. The orientation of the portion of the rendered sound space tracks with the rotation of the listener's head so that the orientation of the rendered sound space remains fixed in space and does not move with the listener's head.

A sound 'locked' to the real world may be referred to as a diegetic sound.

A sound 'locked' to the user's head may be referred to as a non-diegetic sound.

The rendering of a virtual sound scene <NUM> may also be described as providing spatial audio or providing immersive audio. The virtual sound scene <NUM> comprises one or more sound sources at different positions in the sound space <NUM>. The audio rendered to the user depends upon the relative position of the virtual user from the positions of the sound sources. Perspective mediated virtual reality, for example first person perspective mediated reality enables the user <NUM> to change the position of the virtual user <NUM> within the sound space <NUM> thereby changing the positions of the sound sources relative to the virtual user which changes the virtual sound scene <NUM> rendered to the user <NUM>.

Channel-based audio, for example,. n,m surround sound (e.g. <NUM>, <NUM> or <NUM> surround sound) or binaural audio, can be used or scene-based audio, including spatial information about a sound field and sound sources, can be used.

Audio content may encode spatial audio as audio objects. Examples include but are not limited to MPEG-<NUM> and MPEG SAOC. MPEG SAOC is an example of metadata-assisted spatial audio.

Audio content may encode spatial audio as audio objects in the form of moving virtual loudspeakers.

Audio content may encode spatial audio as audio signals with parametric side information or metadata. The audio signals can be, for example, First Order Ambisonics (FOA) or its special case B-format, Higher Order Ambisonics (HOA) signals or mid-side stereo. For such audio signals, synthesis which utilizes the audio signals and the parametric metadata is used to synthesize the audio scene so that a desired spatial perception is created.

The parametric metadata may be produced by different techniques. For example, Nokia's spatial audio capture (OZO Audio) or Directional Audio Coding (DirAC) can be used. Both capture a sound field and represent it using parametric metadata. The parametric metadata may for example comprise: direction parameters that indicate direction per frequency band; distance parameters that indicate distance per frequency band; energy-split parameters that indicate diffuse-to-total energy ratio per frequency band. Each time-frequency tile may be treated as a sound source with the direction parameter controlling vector based amplitude panning for a direct version and the energy-split parameter controlling differential gain for an indirect (decorrelated) version.

The audio content encoded may be speech and/or music and/or generic audio.

3GPP IVAS (3GPP, Immersive Voice and Audio services), which currently under development, is expected to support new immersive voice and audio services, for example, mediated reality.

In some but not necessarily all examples amplitude panning techniques may be used to create or position a sound object. For example, the known method of vector-based amplitude panning (VBAP) can be used to position a sound source.

A sound object may be re-positioned by mixing a portion direct form of the object (an attenuated and directionally-filtered direct sound) with an indirect form of the object (e.g. positioned directional early reflections and/or diffuse reverberant).

<FIG> illustrates an example of a system <NUM> for controlling rendering of spatial audio content <NUM>.

In this example, the system <NUM> comprises an apparatus <NUM> that receives the spatial audio content <NUM> and produces a first part <NUM> of the spatial audio content <NUM> (first audio content) for a first device <NUM> and a second part <NUM> of the spatial audio content <NUM> (second audio content) for a second device <NUM>. In this example, the first device <NUM> is a hand portable device. A hand portable device is a device that has a size and mass that allows it to ported by hand. It some examples, it has a size and mass that allows it to moved relative to a user's body, for example lifted towards a user's ear by the user easily using one hand. It some examples, it has a size and mass that allows it carried in a user's pocket. In this example, the second device <NUM> is a spatial audio device and the second part <NUM> of the spatial audio content <NUM> is spatial audio content- the spatial audio device <NUM> is configured to render the second part <NUM> of the spatial audio content <NUM> as a rendered sound scene <NUM> with positioned rendered sound sources.

The spatial audio device <NUM> may be configured to render only spatial audio or may be configured to render spatial audio and also a virtual visual scene <NUM>.

The stereo loudspeakers are configured to render binaural audio content or other spatial audio content. The loudspeakers position sound sources <NUM> (not illustrated in <FIG>) within a sound space <NUM>. The rendered sound scene <NUM> is dependent upon the received second part <NUM> of the spatial audio content <NUM> and, in some examples, personal rendering settings of the spatial audio device <NUM>. The second part <NUM> of the spatial audio content <NUM> has or is converted to have a format suitable for rendering by the second device <NUM>.

The spatial audio device <NUM> may be a head-mounted audio output device, for example headphones. The headphones may use in-ear loudspeakers, over-ear bins with loudspeakers or adjacent-ear loudspeakers that are, for example, part of the 'temples' of a frame for eye glasses.

The spatial audio device <NUM> may be passthrough headphones that allow the user to hear not only audio rendered by one or more loudspeakers of the spatial audio device <NUM> but to also hear clearly ambient audio such as audio output from the hand portable device <NUM>.

The spatial audio device <NUM> is configured to enable first person perspective mediated reality. For example, the spatial audio device <NUM> may include circuitry that is capable of tracking movement of a user's head while they are wearing the spatial audio device <NUM>.

The spatial audio device <NUM> may, in some but not necessarily all examples, include a head-mounted display for one or both eyes of the user <NUM>.

The hand portable device <NUM> comprises one or more loudspeakers and is configured to cause rendering of the first part <NUM> of the spatial audio content <NUM> via the one or more loudspeakers.

Although the hand portable device <NUM> is, in some examples, capable of combined playback from multiple loudspeakers, it is not necessarily capable of spatial audio output. The output audio <NUM> is therefore heard as if emitted from the hand portable device <NUM>.

In contrast, the spatial audio device <NUM> is capable of placing sound sources <NUM> within the sound space <NUM>.

In some, but not necessarily all examples, the hand portable device <NUM> is a mobile cellular telephone.

The apparatus <NUM> decodes the received spatial audio content <NUM> and causes rendering to a user <NUM> of a first part <NUM> of the spatial audio content <NUM> via the hand portable device <NUM> and causes rendering to the user <NUM>, of a second part <NUM> of the spatial audio content <NUM> via the spatial audio device <NUM>. The apparatus <NUM> comprises a decoder <NUM> for decoding the spatial audio content <NUM>. The decoding produces the spatial audio content <NUM> in a format that can be used to identify and separately process sound sources <NUM>. The decoded spatial audio content is provided to rendering control block <NUM> and renderer <NUM>. The rendering control block <NUM> determines how the sound sources <NUM> within the spatial audio content <NUM> will be rendered. This determination is based upon received allocation information <NUM>. The rendering control block <NUM> provides a control output to the renderer <NUM>.

The renderer <NUM> under the control of the control signals from the rendering control block <NUM> renders the spatial audio content received from the decoder <NUM> as a first part <NUM> for rendering by the hand portable device <NUM> and as a second part <NUM> for rendering by the spatial audio device <NUM>. The second part <NUM> is spatial audio content.

The renderer <NUM> is configured to enable first person perspective mediated reality with respect to the second part <NUM> of the spatial audio content <NUM>. Consequently, the renderer <NUM> takes into account the point of view <NUM> of the user <NUM>.

The second part <NUM> of the spatial audio content <NUM> may be communicated over any suitable link to the spatial audio device <NUM>. In some but not necessarily all example the link is a wireless link such as a radio link. The radio link may be provided by any suitable protocol such as Bluetooth or WLAN.

The first part <NUM> of the spatial audio content <NUM> may be communicated over any suitable link to the hand portable device <NUM>. In some but not necessarily all example the link is a wireless link such as a radio link. The radio link may be provided by any suitable protocol such as Bluetooth or WLAN.

In some but not necessarily all examples, the renderer <NUM> could comprise different renderers for the first part <NUM> and the second part <NUM> of the spatial audio content <NUM>.

The rendering control block <NUM> is configured to identify and control each sound source separately if required. It is capable of controlling the renderer <NUM> to combine one or more sound sources within a rendered sound scene <NUM> in response to the second part <NUM> of the spatial audio content <NUM>.

In some examples, the allocation information <NUM> is optional. For example, in an embedded codec, the lowest layer (monovoice) may be automatically transferred as the first part <NUM> of the spatial audio content <NUM>, and the other layers (spatial enhancement layers) may be provided as the second part <NUM> of the spatial audio content <NUM>.

In other examples, the form of the rendered sound scene <NUM> can be controlled by the allocation information <NUM>.

In this example the rendering control block <NUM> and the renderer <NUM> are housed within the same apparatus <NUM>, in other examples, the rendering control block <NUM> and the renderer <NUM> may be housed in separate devices. In either implementation, there is provided an apparatus <NUM> comprising means for simultaneously controlling content <NUM> rendered by a hand portable device <NUM> and content <NUM> rendered by a spatial audio device <NUM>; and providing for rendering to a user <NUM>, in response to an action by the user <NUM>, of a first part <NUM>, not a second part <NUM>, of a spatial audio content <NUM> via the hand portable device <NUM> not the spatial audio device <NUM>.

The apparatus <NUM> may comprise means, such as the rendering control block <NUM>, for determining the first part <NUM> of the spatial audio content <NUM> and the second part <NUM> of the spatial audio content <NUM>, in response to the user action.

The apparatus <NUM> also comprises means for causing simultaneous rendering to the user, in response to the action of the user of the first part <NUM> of the spatial audio content <NUM> via the hand portable device <NUM> and also the second part <NUM> of the spatial audio content <NUM> via the spatial audio device <NUM>. The spatial audio device <NUM> renders the second part <NUM>, not the first part <NUM> and the hand portable device <NUM> renders the first part <NUM>, not the second part <NUM>. In some examples, the hand portable device <NUM> renders only the first part <NUM> and the spatial audio device <NUM> renders only the second part <NUM>.

The spatial audio content <NUM>, in some but not necessarily all examples, is received from a communication network by the apparatus <NUM>. The spatial audio content <NUM> may, for example, define a sound scene or may, for example, be an immersive call. An immersive call is provided in a downlink communication link from a remote person and, in this example, but not necessarily all examples, the first part <NUM> of the spatial audio content <NUM> can comprise a voice of the remote person and/or ambient sound.

The rendering control module <NUM> may be configured to control whether a sound source <NUM> is rendered as diegetic content or as non-diegetic content. Diegetic content has a position in the sound space <NUM> that corresponds to a position in the real space <NUM>. The location of the sound source relative to the user <NUM> is controlled by first person perspective mediated reality. In contrast, non-diegetic content has no position in the real space <NUM> and is not subject to modification as a consequence of first person perspective mediated reality. Examples of non-diegetic content include, for example, ambient sound or a "narrator" voice.

Whether or not to render a sound source <NUM> as diegetic content or non-diegetic content can be dependent upon whether the sound source <NUM> is to be rendered on the hand portable device <NUM> or the spatial audio device <NUM>. For example, the first part <NUM> of the spatial audio content <NUM> may be a non-diegetic sound source <NUM>. The second part <NUM> of the spatial audio content <NUM> may be diegetic content and/or non-diegetic content.

In the examples of <FIG>, the action by the user that causes rendering to the user of the first part <NUM> of the spatial audio content <NUM> via the hand portable device <NUM> and the rendering of the second part <NUM> of the spatial audio content <NUM> via the spatial audio device <NUM> is the lifting of the hand portable device <NUM> towards an ear of the user <NUM>. It should of course be appreciated that different user actions, including for example interaction and/or indication, may also cause such rendering.

<FIG> illustrates an example in which the hand portable device <NUM> renders <NUM> the first part <NUM> of the spatial audio content <NUM>.

<FIG> illustrates an example in which the spatial audio device <NUM> renders the second part <NUM> of the spatial audio content <NUM> as a rendered sound scene <NUM> comprising sound sources <NUM>. Rendering the sound scene <NUM> comprises rendering one or more sound sources at respective positions in the sound space <NUM>.

<FIG> illustrates the simultaneous rendering to the user <NUM>, in response to the action of the user <NUM>, of the first part <NUM> of the spatial audio content <NUM> via the hand portable device <NUM> and the second part <NUM> of the spatial audio content <NUM> via the spatial audio device <NUM>. In this example, the action performed by the user to cause rendering to the user <NUM>, of the first part <NUM> of the spatial audio content <NUM> via the hand portable device <NUM>, is lifting the hand portable device <NUM> to the left ear of the user <NUM>.

In this example, the apparatus <NUM> comprises means for communicating with the hand portable device <NUM> and/or the spatial audio device <NUM> to determine when the action of the user has occurred. This may, for example, be detected by using a proximity detector for the hand portable device <NUM> that detects that the hand portable device <NUM> is adjacent an ear of the user <NUM> by, for example, measuring acoustic impedance or detects that the hand portable device <NUM> is adjacent the spatial audio device <NUM> by detecting the spatial audio device <NUM>. Alternatively, the hand portable device <NUM> may use inertial measurement units to measure the movement of the hand portable device <NUM> and detect a movement that corresponds to an upward lift of the hand portable device <NUM> in combination with an upright orientation of the hand portable device <NUM>.

In other examples, the spatial audio device <NUM> may comprise a proximity detector for detecting proximity of the hand portable device <NUM>.

The use of the lifting of the hand portable device <NUM> to an ear of the user <NUM> to cause the splitting of the spatial audio data <NUM> into a first part <NUM> for the hand portable device <NUM> and a second part <NUM> for the spatial audio device <NUM> is particularly useful when the spatial audio content <NUM> relates to an incoming immersive telephone call. An immersive telephone call is a telephone call that includes spatial audio content.

Where the incoming spatial audio content <NUM> is an immersive telephone call, the first part <NUM> of the spatial audio content <NUM> may be content that renders a voice of a caller. The voice of the caller may then be removed from the spatial sound scene defined by the spatial audio content of the incoming immersive telephone call to produce the second part <NUM> of the spatial audio content <NUM> which is rendered by the spatial audio device <NUM> as a rendered sound scene <NUM>.

In some examples, the spatial audio content <NUM> may be entirely rendered by the spatial audio device <NUM> before the user action causes the splitting of the spatial audio content <NUM> into the first part <NUM> for the hand portable device <NUM> and the second part <NUM> for the spatial audio device <NUM>. Consequently, before the action of the user, the first part <NUM> and the second part <NUM> of the spatial audio content <NUM> may be simultaneously rendered by the spatial audio device <NUM>.

The following is an example of a use case of the system <NUM>. The user <NUM> is wearing the spatial audio device <NUM> and has the hand portable device <NUM> in his pocket. A friend of the user calls him on an immersive telephone call, for example using IVAS. The user <NUM> is alerted to the incoming immersive call via a ringtone of the hand portable device <NUM>, via content rendered by the spatial audio device <NUM> or by both. The user <NUM> wants to receive the incoming immersive call on his hand portable device <NUM> but also wishes to hear an ambient signal of his friend. The user <NUM> answers the incoming immersive call by touching the hand portable device <NUM> against the spatial audio device <NUM> at his ear. The rendering control module <NUM>, which may be part of the call software of the hand portable device <NUM>, allocates the caller's voice (the friend's voice) to the hand portable device <NUM> as the first part <NUM> of the spatial audio content <NUM> and the ambient signal to the spatial audio device <NUM> as the second part <NUM> of the spatial audio content <NUM>. The spatial audio device <NUM> allows the user to hear not only audio rendered by one or more loudspeakers of the spatial audio device <NUM> but to also hear clearly ambient audio such as audio output from the hand portable device <NUM>. The user hears the transmitted ambient signal around him as a rendered sound scene <NUM>, rendered by the spatial audio device <NUM>, and uses the hand portable device <NUM> in a traditional way to hear the caller's voice. The user <NUM> can, for example, switch the hand portable device <NUM> from ear to ear and, may, in some examples, be able to use the hand portable device <NUM> in a hands-free mode. A user interface on the hand portable device <NUM> and/or the spatial audio device <NUM> enables the user <NUM> to control allocation information <NUM>. The allocation information <NUM> may, for example, be varied to control a volume of the second part <NUM> of the spatial audio content rendered by the spatial audio device <NUM> and/or to control a volume of the first part <NUM> of the spatial audio content <NUM> rendered by the hand portable device <NUM>.

In the examples illustrated in <FIG>, the hand portable device <NUM> is used to render a first part <NUM> of the spatial audio content <NUM> and the spatial audio device <NUM> is used to render a second part <NUM> of the spatial audio content <NUM>. The hand portable device <NUM> and the spatial audio device <NUM> are therefore used simultaneously for the rendering of audio content. It is therefore possible for the rendered audio content from one device to affect the user's perception of the audio content rendered by the other device. In the examples illustrated the second part <NUM> of the spatial audio content is adapted to move one or more sound sources <NUM> within the second part <NUM> of the spatial audio content to avoid this problem. The sound source <NUM> that is moved is an additional sound source. It may, for example, be an internal diegetic sound source from within the immersive call content other than the primary voice of the caller or it may be an external additional sound source from content other than the immersive call.

The internal additional sound source may, for example, be another person talking (not ambient background noise). This may, for example, arise if the immersive call is a multi-person teleconference call. In some examples, when the user of the hand portable device <NUM> and the spatial audio device <NUM> is in a multi-party teleconference call with a 2nd remote party, a 3rd remote party, a 4th remote party etc, then user can select which remote party's voice is rendered from the hand portable device <NUM> and the other two other remote parties would continue to be rendered in the spatial audio output of the spatial audio device <NUM> The user can individually select which ones of the multiple remote parties participating in the multi-person teleconference call to respectively separate to hand portable device <NUM> and the spatial audio device <NUM>.

The external additional sound source may, for example, arise from on-going content that is not stopped or paused and which continues (at least partially) when the immersive telephone call is received, or, may be, new content, separate to the immersive call, started after receiving the immersive call.

In some, but not necessarily all examples, the additional sound source may be selected by the user, for example using a user interface of the hand portable device <NUM> or of the spatial audio device <NUM>.

Examples of external additional sound sources include, for example, music playback, messages, advertisements and contextual guidance.

In <FIG>, a spatial audio scene <NUM> defined by the second part <NUM> of the spatial audio content <NUM> is modified in dependence upon a position <NUM> of the hand portable device <NUM>.

For example, a position of an additional source <NUM>' is moved <NUM> based on user action <NUM>, for example movement of the hand portable device <NUM> to a new position.

<FIG> illustrates a position of a particular sound source <NUM>' in the spatial audio content <NUM> before the spatial audio content <NUM> has been split, in response to the user action <NUM>, into a first part <NUM> for the hand portable device <NUM> and a second part <NUM> for the spatial audio device <NUM>.

<FIG> illustrates the situation after the user <NUM> has performed the action <NUM> that causes the spatial audio content <NUM> to be split into the first part <NUM> and the second part <NUM>. The first part <NUM> is or will be rendered by the hand portable device <NUM> as audio content <NUM> in the left ear of the user, where the hand portable device <NUM> is located. As illustrated in <FIG>, the rendering control module <NUM> causes the renderer <NUM> to adapt the second part <NUM> of the spatial audio content <NUM> to change a position of the spatial audio source <NUM>'. In this example the sound source <NUM>' is moved <NUM> to be adjacent a right ear of the user <NUM>.

<FIG> are similar to <FIG>, however, there are now two original sound sources A, B and initial position of the particular sound source B <NUM> that is moved is different. In the example of <FIG>, the particular sound source <NUM>' of the rendered sound scene <NUM> is directly adjacent the left ear of the user and it is moved <NUM> away from the left ear of the user to create spatial separation between the additional sound source <NUM>' and the hand portable device <NUM>.

The allocation information <NUM> may provide information identifying a location of the hand portable device <NUM> in a three-dimensional space. This may, for example, indicate that it is proximal to a left ear or a right ear of the user or may provide a three-dimensional position in the real space <NUM>.

The rendering control module <NUM> is configured to determine a distance separation between the position of the hand portable device <NUM> and the additional sound source <NUM>'. If the separation distance is below a threshold value, the rendering control module <NUM> determines that the additional sound source <NUM>' is sub-optimally placed. It may, for example, determine that there is spatial conflict as illustrated in <FIG> or co-location as illustrated in <FIG>.

The rendering control module <NUM> modifies the spatial audio sound scene <NUM> by moving at least the additional sound source <NUM>'. In some, but not necessarily all examples, only the additional sound source <NUM>' that is determined to be sub-optimally placed is moved. In other examples, the whole of the rendered sound scene <NUM> is rotated relative to the user <NUM> to remove the spatial conflict or co-location.

In the examples of <FIG>, before the user action <NUM> causes modification of the spatial audio content <NUM> by splitting it into the first part <NUM> for the hand portable device <NUM> and the second part <NUM> for the spatial audio device <NUM>, the spatial audio content <NUM> is rendered to the user <NUM> using first person respective mediated reality. For example, the additional sound source <NUM>' may have a fixed position in real space as the user <NUM> changes their point of view <NUM>.

In the examples of <FIG>, after the user action <NUM> causes modification of the spatial audio content <NUM> by splitting it into the first part <NUM> for the hand portable device <NUM> and the second part <NUM> for the spatial audio device <NUM>, the spatial audio content <NUM>, including the re-positioned additional sound source <NUM>', is rendered to the user <NUM> using first person respective mediated reality. For example, the repositioned additional sound source <NUM>' may have a fixed position in real space as the user <NUM> changes their point of view <NUM>.

In some, but not necessarily all examples, the first part <NUM> of the spatial audio content <NUM> relates to a first voice associated with a first ear of the user <NUM>, at which the hand portable device <NUM> is positioned. The second part <NUM> of the spatial audio content relates to at least a second different voice, wherein the spatial audio scene <NUM> defined by the second part <NUM> of the spatial audio content <NUM> is modified to place the second voice <NUM>' at a second ear of the user, opposite the first ear.

In some, but not necessarily all examples, the first voice and the second different voice, may relate to different sound sources within an immersive telephone call. In such an example, the examples of <FIG>may occur in pre-processing before any rendering of the spatial audio content <NUM>. The spatial audio content <NUM>, of the immersive telephone call, may only be rendered, for the first time, after the modification to the second part <NUM> of the spatial audio content <NUM> illustrated in <FIG> has occurred.

<FIG> illustrate an example in which the user <NUM> edits the second part <NUM> of the spatial audio content <NUM>. The second part <NUM> of the spatial audio content <NUM> is edited in response to movement of the hand portable device <NUM>, in this example.

In some, but not necessarily all examples, the user <NUM> can control allocation of sound sources <NUM> between the first part <NUM> for the hand portable device <NUM> and the second part <NUM> for the spatial audio device <NUM>. This can be achieved by modifying the allocation information <NUM>.

In some, but not necessarily all examples, the user <NUM> can control allocation of a sound source <NUM> by moving the sound source <NUM> from being rendered as part of the rendered scene <NUM> by the spatial audio device <NUM> to being rendered by the hand portable device <NUM> and/or control reallocation of the sound source <NUM> from being rendered by the hand portable device <NUM> to being rendered by the spatial audio device <NUM>.

In other examples, the user can control the balance between the audio output by the hand portable device <NUM> and the spatial audio device <NUM> for example by separately controlling the volume of a device or by silencing a sound source <NUM> rendered by either device.

In the example of <FIG>, a primary sound source Y is rendered <NUM> by the hand portable device <NUM>, which is adjacent the left ear of the user <NUM>. Simultaneously the spatial audio device <NUM> renders, as a rendered sound scene <NUM>, an additional sound source <NUM>'. This additional sound source <NUM>' is a secondary sound source X. The primary sound source Y may, for example, be a voice in an immersive telephone call. The additional sound source <NUM>' may, for example, be a different voice in the immersive call or other, different content.

As illustrated in <FIG>, the system <NUM> is configured to enable the user <NUM> to control allocation of sound sources <NUM> between the rendering devices <NUM>, <NUM>. In this example, the user uses a user interface on the hand portable device <NUM> to switch the primary source Y with the secondary source X. The primary sound source Y is now rendered as a sound source <NUM> in the rendered sound scene <NUM> that is rendered by the spatial audio device <NUM>. This is achieved by modifying the second part <NUM> of the spatial audio content <NUM> in response to control signals from the rendering control module <NUM>. The secondary sound source X is now rendered by the hand portable device <NUM>. This is achieved by modifying the first part <NUM> of the spatial audio content <NUM> under the control of the rendering control module <NUM>.

As illustrated in <FIG>, the user is now able to lower the hand portable device <NUM> so that the user can no longer hear the secondary sound object X. The user is then able to listen to the primary sound object Y as a sound source <NUM> in the rendered sound scene <NUM> that is rendered by the spatial audio device <NUM>.

In some, but not necessarily all examples, the action <NUM> of lowering the hand portable device <NUM> may result in the removal of the secondary sound source X from the second part <NUM> of the spatial audio content <NUM>. A subsequent raising of the hand portable device to the user's ear may then cause the primary sound source Y to be transferred back to the first part <NUM> of the spatial audio content <NUM>. The user has consequently returned to a situation similar to that illustrated in <FIG> except that the additional sound source <NUM>' has been removed from the sound scene <NUM> rendered by the spatial audio device <NUM>.

In the example of <FIG>, if the user <NUM> is to speak during the immersive call then either the spatial audio device <NUM> will need to have a microphone or an additional microphone <NUM> needs to be provided either separately, for example a Lavalier microphone, or by the hand portable device <NUM>.

<FIG> illustrates an example in which the spatial audio device <NUM> is rendering spatial audio content <NUM> to the user <NUM>. The rendered spatial audio content defines a sound scene <NUM> that comprises multiple sound sources <NUM>. <FIG> illustrate that the user may perform an action that causes a sound source <NUM> to be moved from the rendered sound scene <NUM> so that it is instead rendered from the hand portable device <NUM>. Referring back to <FIG>, this corresponds to a change in the allocation information <NUM> such that the rendering control block <NUM> controls the renderer <NUM> to place a particular sound source <NUM> into the first part <NUM> of the spatial audio content that is rendered by the hand portable device <NUM>. The remaining spatial audio content, minus the particular sound source <NUM>, is provided as a second part <NUM> of the spatial audio content <NUM> to the spatial audio device <NUM> to be rendered as the sound scene <NUM>.

The apparatus <NUM> consequently comprises a means for simultaneously controlling content rendered by the hand portable device <NUM> and content rendered by the spatial audio device <NUM> and providing for rendering to the user <NUM>, in response to an action by the user, of a first part <NUM>, not a second part <NUM>, of a spatial audio content <NUM> via the hand portable device <NUM> not the spatial audio device <NUM>.

In this example, there is simultaneous rendering before the action of the user of the first part of the spatial audio content and the second part of the second audio content via the spatial audio device <NUM> (<FIG>). The apparatus <NUM> comprises means for enabling selection of the first part <NUM> of the spatial audio content <NUM> via the user <NUM> using first-person perspective mediated reality. This is illustrated in <FIG>. In the examples <FIG>, the user can select <NUM> a desired sound source <NUM> by varying the point of view <NUM> of the user <NUM> that is used to select the desired sound source <NUM>. The selected first part <NUM> of the spatial audio content <NUM> may be modified before being rendered by the hand portable device <NUM>, for example, to emphasize a particular audio source in a mono downmix.

The user action in this example is lifting the hand portable device <NUM> to the user's head. As previously described, the apparatus <NUM> renders to the user <NUM>, the first part <NUM> of the spatial audio content <NUM> via the hand portable device <NUM> and the second part <NUM> of the spatial audio content <NUM> via the spatial audio device <NUM>, simultaneously.

In some, but not necessarily all examples, the first part <NUM> comprises a single sound source <NUM> selected by the user <NUM>. In other examples, the first part <NUM> comprises multiple sound sources <NUM> that are separately and sequentially selected by the user <NUM>.

<FIG> illustrates an example of a method <NUM> that may be performed by the apparatus <NUM>. At block <NUM>, the apparatus <NUM> receives an immersive call. At block <NUM>, the immersive audio <NUM> is rendered by the spatial audio device <NUM>. At block <NUM>, the apparatus <NUM> detects that the user <NUM> has raised the hand portable device <NUM> to be adjacent to the user's ear. At block <NUM>, the first part <NUM> of the spatial audio content <NUM> is created by the renderer <NUM> in response to rendering control block <NUM>. In some examples, all of the spatial audio content <NUM> of the immersive call is down-mixed to form a single first part <NUM> that is rendered by the hand portable device <NUM>. In this example, the renderer <NUM> may not provide any second part <NUM> of the spatial audio content <NUM> to the spatial audio device <NUM>. In other examples, the renderer <NUM> under the control of the rendering control block <NUM> controls the second part <NUM> of the spatial audio content to comprise one or more particular sound sources <NUM>. In this example, the second part <NUM> of the spatial audio content <NUM> comprises that part of the spatial audio content <NUM> remaining after the subtraction of the first part <NUM>.

At block <NUM>, the apparatus <NUM> responds to a change in the point of view <NUM> of the user <NUM>. This change of point of view is provided as allocation information <NUM> and causes a change in the selection of the audio content for rendering by the hand portable device <NUM>.

For example, the mono downmix created from the spatial audio content <NUM> may be created with an emphasis on one or more sound sources in a particular direction of arrival that corresponds to the point of view <NUM> of the user <NUM>. Likewise, the selection of a particular sound source <NUM> may correspond to a sound source that is aligned with the point of view <NUM> of the user <NUM>. Therefore, as the point of view <NUM> of the user <NUM> changes, then the content of the first part <NUM> of the spatial audio content <NUM> also changes.

At block <NUM>, the first part <NUM> of the spatial audio content <NUM> is delivered, for rendering, to the hand portable device <NUM>.

<FIG> illustrate, in detail, how the apparatus <NUM> enables selection of the first part of the spatial audio content by the user using first-person perspective mediated reality. <FIG> illustrates the virtual user <NUM> who has a point of view <NUM> within a sound space <NUM> that comprises sound sources <NUM>. As previously described, in the first-person perspective mediated reality, there is a correspondence between the virtual user <NUM> and the user <NUM> and the point of view <NUM> of the virtual user <NUM> and the point of view <NUM> of the user <NUM>. The user <NUM> varies the point of view <NUM> by changing an orientation of the user's head. This changes the virtual point of view <NUM>.

In this immersive audio scene, the rendered sound scene <NUM> comprises multiple sound sources <NUM>. These sound sources are fixed in space, in this example, so that when the user <NUM> changes his point of view <NUM>, the positions of the sound sources <NUM> relative to the user <NUM> also change.

In the example of <FIG>, the user has raised the hand portable device <NUM> to his left ear or performed any other suitable gesture to cause rendering of the first part <NUM> of the spatial audio content <NUM> via the hand portable device <NUM> and rendering of the second part <NUM> of the spatial audio content <NUM> via the spatial audio device <NUM>. The first part <NUM> of the spatial audio content emphasises the sound source <NUM> that is aligned with or closest to the point of view <NUM> of the virtual user <NUM>. In the example of a mono downmix, the mono downmix is emphasised in the direction of the virtual point of view <NUM>. In the example of sound objects, or the selection of particular sound sources <NUM>, the point of view <NUM> selects the sound source <NUM> that is aligned with or closest to the virtual point of view <NUM>. The selected sound source <NUM> is included in the first part <NUM> but is not included in the second part <NUM> of the spatial audio content <NUM>. In the example of <FIG>, the sound source <NUM> labelled "<NUM>" is selected and emphasised in the first part <NUM> of the spatial audio content <NUM>.

In the example of <FIG>, the user <NUM> has changed his point of view <NUM> by changing an orientation of his head. This causes a consequent change in the point of view <NUM> of the virtual user <NUM>. The change in the point of view <NUM> of the virtual user changes which part of the sound scene is emphasised in the first part <NUM> of the spatial audio content <NUM>. The selection is as described for <FIG>, the only difference being the change in the point of view <NUM>. In the example of <FIG>, the sound source <NUM> labelled "<NUM>" is selected and emphasised in the first part <NUM> of the spatial audio content <NUM>.

In the example of <FIG>, the user <NUM> has removed the spatial audio device <NUM>. The selected sound source <NUM> is now frozen. In the example of <FIG>, the sound source <NUM> labelled "<NUM>" is selected and emphasised. Subsequent movement of the user's head and a change in the point of view <NUM> of the user <NUM> does not change the point of view <NUM> of the virtual user <NUM>. There is consequently no change in the selection of the emphasised sound source <NUM>, as illustrated in <FIG>. In the example of <FIG>, the sound source labelled "<NUM>" remains the emphasised sound source <NUM> in the first part <NUM> of the spatial audio content <NUM>.

It will therefore be appreciated that while the user wears the spatial audio device <NUM>, which has head tracking capabilities, first-person perspective mediated reality is enabled. The point of view <NUM> of the user <NUM> and the virtual point of view <NUM> of the virtual user <NUM> correspond and track each other. The virtual point of view <NUM> is used to emphasise particular sound sources <NUM> within the spatial audio content that are aligned with the virtual point of view <NUM>. The particular sound sources <NUM> are emphasised in the first part <NUM> of the spatial audio content <NUM>. The emphasised sound source <NUM> can be varied by varying the user's point of view <NUM>, which changes the virtual user's point of view <NUM>. However, once the spatial audio device <NUM> is removed, it is no longer possible to track the head movements of the user <NUM> and point of view <NUM> of the user and virtual point of view <NUM> of the virtual user are decoupled. Consequently, a change in the point of view <NUM> of the user <NUM> does not cause a change in the virtual point of view <NUM> of the virtual user <NUM> and does not cause a change in the selected sound source <NUM> for emphasis in the first part <NUM> of the spatial audio content <NUM>.

In the examples of <FIG> and <FIG>, a user <NUM> selects a particular sound source <NUM> and this sound sources is emphasised irrespective of subsequent relative changes in alignment between the virtual point of view <NUM> and the selected sound source <NUM>. This lack of alignment may, for example, arise because the selected sound source <NUM> is moving and/or because the user <NUM> changes his point of view <NUM>. A consequence of this is that a selected moving sound source <NUM> is rendered as if the moving sound source were stationary.

In <FIG>, the point of view <NUM> of the virtual user <NUM> is aligned with a particular sound object <NUM>. This alignment selects the particular sound object <NUM> which is then included in the first part <NUM> of the spatial audio content and is rendered by the hand portable device <NUM>. The selection of the sound object <NUM> may be as a consequence of an action by the user <NUM>.

In <FIG>, the selected sound source <NUM> has moved but the point of view <NUM> of the virtual user <NUM> has not moved. Despite the point of view <NUM> of the virtual user <NUM> no longer pointing towards the selected sound object <NUM>, the selected sound object <NUM> remains selected and is included in the first part <NUM> of the spatial audio content rendered by the hand portable device <NUM>. The selected sound object <NUM> remains selected based on tracking, rather than user point of view, and is included in the first part <NUM> of the spatial audio content rendered by the hand portable device <NUM>. The position of the selected sound object <NUM> can be tracked and the manner in which it is included in the first part <NUM> of the spatial audio content rendered by the hand portable device <NUM> can change with a changing tracked position of the selected sound object 140Consequently, the sound objects "<NUM>", "<NUM>", "<NUM>" are rendered by the spatial audio device <NUM>, controlled by the second part <NUM> of the spatial audio content <NUM> in both <FIG> and <FIG>. Also, the sound object "<NUM>" is rendered by the hand portable device <NUM>, under the control of the first part <NUM> of the spatial audio content <NUM>, in both <FIG> and <FIG>.

In the example of 10C, the spatial audio device <NUM> has been removed. Despite this, the selected spatial sound object <NUM> remains selected and continues to be rendered by the hand portable device <NUM>, under the control of the first part <NUM> of the spatial audio content <NUM>, even if it is moving.

<FIG> illustrates that even if the user <NUM> changes his point of view <NUM>, the selected sound object "<NUM>" continues to be rendered by the hand portable device <NUM>, under the control of the first part <NUM> of the spatial audio content <NUM>, as previously in <FIG>.

Referring back to the examples illustrated in <FIG> and <FIG>, it will be appreciated that when the user <NUM> changes his point of view <NUM>, he changes the point of view <NUM> of the virtual user, which changes the sound source <NUM> that is emphasised in the first part <NUM> of the spatial audio content <NUM> rendered by the hand portable device <NUM>. In the example where the sound source <NUM> that is emphasised is a sound object, the remaining sound objects of the sound scene are rendered by the spatial audio device <NUM>. That is, the first part <NUM> of the spatial audio content <NUM> includes the sound object "<NUM>" and the second part <NUM> of the spatial audio object includes the sound objects "<NUM>", "<NUM>", "<NUM>". When the user changes orientation, changing the point of view <NUM>, then the virtual point of view <NUM> of the user also changes. This is illustrated in <FIG>. As previously described, this changes the sound object selected. The first part <NUM> of the spatial audio content <NUM> includes the sound object "<NUM>". Consequently, the content of the second part <NUM> of the spatial audio content now includes the sound objects "<NUM>", "<NUM>", "<NUM>". In addition, as the spatial audio device <NUM> enables first-person perspective mediated reality, the positions of the sound objects "<NUM>" and "<NUM>" rendered to the user via the spatial audio device <NUM> have been rotated relative to the user comparing <FIG> and <FIG> as a consequence of the change in the point of view of the virtual user. Similar considerations also apply to <FIG>.

It will therefore be appreciated that the apparatus <NUM> enables adaptation of the second part <NUM> of the spatial audio content <NUM> using first-person perspective mediated reality, after the user action that determines the first part <NUM> of the audio content <NUM>, while using at least the spatial audio device <NUM>. The apparatus <NUM> also enables rendering of the adapted second part <NUM> of the spatial audio content <NUM> via at least the spatial audio device <NUM>.

In some, but not necessarily all, examples, the user <NUM> may be able to lock the selected object without locking the spatial audio scene. As a consequence, the apparatus <NUM> is capable of rendering a second part <NUM> of the spatial audio content <NUM> via the spatial audio device <NUM> where the second part <NUM> has been adapted in accordance with first-person perspective mediated reality.

The user may also be capable of locking the rendered sound scene <NUM> in response to a subsequent user action. The apparatus <NUM> is then configured to disable, after this subsequent user action, adaptation of the second part <NUM> of the spatial audio content in accordance with first-person perspective mediated reality and cause rendering of the second part <NUM> of the spatial audio content <NUM> in accordance with a fixed first-person perspective via the spatial audio device <NUM>.

In the example of <FIG>, the removal of the spatial audio device <NUM> simultaneously locks the object and locks the scene.

<FIG> illustrates an example of an implementation of the system <NUM> illustrated in <FIG>, the operation of which has been described in the previous description. In this example, the allocation information <NUM> is provided from the hand portable device <NUM> to a remote server <NUM> that comprises the rendering control block <NUM>. In this example, the server <NUM> also comprises the renderer <NUM>. However, in other examples, it may be located in the hand portable device <NUM>.

The renderer <NUM> under the control of the rendering control block <NUM> provides the first part <NUM> and the second part <NUM> of the spatial audio content <NUM> to the hand portable device <NUM> in a manner that allows the hand portable device <NUM> to render the first part <NUM> and to provide the second part <NUM> to the spatial audio device <NUM> for rendering. The hand portable device <NUM> provides allocation information <NUM> to the server <NUM>. This may, for example, include indications of user actions that are associated with control commands.

<FIG> illustrates an example of an implementation of the system <NUM> illustrated in <FIG>, the operation of which has been described in the previous description. In this example, the allocation information <NUM> is provided internally within the hand portable device <NUM> that comprises the rendering control block <NUM>. In this example, the hand portable device <NUM> comprises the renderer <NUM>. The server <NUM> provides the spatial audio content <NUM>.

The renderer <NUM> under the control of the rendering control block <NUM> provides the first part <NUM> and the second part <NUM> of the spatial audio content <NUM> to the hand portable device <NUM> in a manner that allows the hand portable device <NUM> to render the first part <NUM> and to provide the second part <NUM> to the spatial audio device <NUM> for rendering. The allocation information <NUM> may, for example, include indications of user actions that are associated with control commands.

<FIG> illustrates an example of a controller <NUM>. Such a controller may be used to control performance of any or all of the functions of the apparatus <NUM>, control performance of any or all of the functions of the hand portable device <NUM>, control performance of any or all of the functions of the spatial audio device <NUM>.

Implementation of a controller <NUM> may be as controller circuitry. The controller <NUM> may be implemented in hardware alone, have certain aspects in software including firmware alone or can be a combination of hardware and software (including firmware).

The memory <NUM> stores a computer program <NUM> comprising computer program instructions (computer program code) that controls the operation of the apparatus <NUM> when loaded into the processor <NUM>. The computer program instructions, of the computer program <NUM>, provide the logic and routines that enables the apparatus to perform the methods illustrated in FIGS <NUM> to <NUM>. The processor <NUM> by reading the memory <NUM> is able to load and execute the computer program <NUM>.

The apparatus <NUM> can therefore comprises:.

As illustrated in <FIG>, the computer program <NUM> may arrive at the apparatus <NUM> via any suitable delivery mechanism <NUM>. The delivery mechanism <NUM> may be, for example, a machine readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a Compact Disc Read-Only Memory (CD-ROM) or a Digital Versatile Disc (DVD) or a solid state memory, an article of manufacture that comprises or tangibly embodies the computer program <NUM>. The delivery mechanism may be a signal configured to reliably transfer the computer program <NUM>. The apparatus <NUM> may propagate or transmit the computer program <NUM> as a computer data signal.

In at least some examples, the computer program instructions are configured to cause an apparatus to perform at least the following:.

The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit for a hand portable device or a similar integrated circuit in a server, a cellular network device, or other computing or network device.

The blocks illustrated in the FIGS <NUM> to <NUM> may represent steps in a method and/or sections of code in the computer program <NUM>. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block may be varied. Furthermore, it may be possible for some blocks to be omitted.

The above described examples find application as enabling components of: automotive systems; telecommunication systems; electronic systems including consumer electronic products; distributed computing systems; media systems for generating or rendering media content including audio, visual and audio visual content and mixed, mediated, virtual and/or augmented reality; personal systems including personal health systems or personal fitness systems; navigation systems; user interfaces also known as human machine interfaces; networks including cellular, non-cellular, and optical networks; ad-hoc networks; the internet; the internet of things; virtualized networks; and related software and services.

Although embodiments have been described in the preceding paragraphs with reference to various examples, it should be appreciated that modifications to the examples given can be made without departing from the scope of the claims.

The term 'a' or 'the' is used in this document with an inclusive not an exclusive meaning. That is any reference to X comprising a/the Y indicates that X may comprise only one Y or may comprise more than one Y unless the context clearly indicates the contrary. If it is intended to use 'a' or 'the' with an exclusive meaning then it will be made clear in the context. In some circumstances the use of 'at least one' or 'one or more' may be used to emphasis an inclusive meaning but the absence of these terms should not be taken to infer and exclusive meaning.

The presence of a feature (or combination of features) in a claim is a reference to that feature) or combination of features) itself and also to features that achieve substantially the same technical effect (equivalent features). The equivalent features include, for example, features that are variants and achieve substantially the same result in substantially the same way. The equivalent features include, for example, features that perform substantially the same function, in substantially the same way to achieve substantially the same result.

Claim 1:
An apparatus comprising means for:
simultaneously controlling content rendered by a hand portable device (<NUM>) and content rendered by a spatial audio device (<NUM>);
in response to an action by a user, splitting a spatial audio content (<NUM>) into a first part (<NUM>) for the hand portable device (<NUM>) and a second part (<NUM>) for the spatial audio device (<NUM>), wherein rendering of the first part (<NUM>) and the second part (<NUM>), before the action of the user causes the splitting of the spatial audio content (<NUM>) into the first part (<NUM>) for the hand portable device (<NUM>) and the second part (<NUM>) for the spatial audio device (<NUM>), is simultaneous and via the spatial audio device (<NUM>); and
causing simultaneous rendering to the user, in response to the action of the user, of the first part (<NUM>) of the spatial audio content (<NUM>) via the hand portable device (<NUM>), not the spatial audio device (<NUM>), and the second part (<NUM>) of the spatial audio content (<NUM>) via the spatial audio device (<NUM>), not the hand portable device (<NUM>).