Patent Description:
Spatial audio rendering is a process used for presenting audio within an extended reality (XR) scene (e.g., a virtual reality (VR), augmented reality (AR), or mixed reality (MR) scene) in order to give a listener the impression that sound is coming from physical sources within the scene at a certain position and having a certain size and shape (i.e., extent). The presentation can be made through headphone speakers or other speakers. If the presentation is made via headphone speakers, the processing used is called binaural rendering and uses spatial cues of human spatial hearing that make it possible to determine from which direction sounds are coming. The cues involve inter-aural time delay (ITD), inter-aural level difference (ILD), and/or spectral difference.

The most common form of spatial audio rendering is based on the concept of point-sources, where each sound source is defined to emanate sound from one specific point. Because each sound source is defined to emanate sound from one specific point, the sound source doesn't have any size or shape. In order to render a sound source having an extent (size and shape), different methods have been developed.

One such known method is to create multiple copies of a mono audio element at positions around the audio element. This arrangement creates the perception of a spatially homogeneous object with a certain size. This concept is used, for example, in the "object spread" and "object divergence" features of the MPEG-H 3D Audio standard (see references [<NUM>] and [<NUM>]), and in the "object divergence" feature of the EBU Audio Definition Model (ADM) standard (see reference [<NUM>]). This idea using a mono audio source has been developed further as described in reference [<NUM>], where the area-volumetric geometry of a sound object is projected onto a sphere around the listener and the sound is rendered to the listener using a pair of head-related (HR) filters that is evaluated as the integral of all HR filters covering the geometric projection of the object on the sphere. For a spherical volumetric source this integral has an analytical solution. For an arbitrary area-volumetric source geometry, however, the integral is evaluated by sampling the projected source surface on the sphere using what is called a Monte Carlo ray sampling.

Another rendering method renders a spatially diffuse component in addition to a mono audio signal, which creates the perception of a somewhat diffuse object that, in contrast to the original mono audio element, has no distinct pin-point location. This concept is used, for example, in the "object diffuseness" feature of the MPEG-H 3D Audio standard (see reference [<NUM>]) and the "object diffuseness" feature of the EBU ADM (see reference [<NUM>]).

Combinations of the above two methods are also known. For example, the "object extent" feature of the EBU ADM combines the creation of multiple copies of a mono audio element with the addition of diffuse components (see reference [<NUM>]).

In many cases the actual shape of an audio element can be described well enough with a basic shape (e.g., a sphere or a box). But sometimes the actual shape is more complicated and needs to be described in a more detailed form (e.g., a mesh structure or a parametric description format).

In the case of heterogeneous audio elements, as are described in reference [<NUM>], the audio element comprises at least two audio channels (i.e., audio signals) to describe a spatial variation over its extent.

In some XR scenes there may be an object that blocks at least part of an audio element in the XR scene. In such a scenario the audio element is said to be at least partially occluded.

That is, occlusion happens when, from the viewpoint of a listener at a given listening position, an audio element is completely or partly hidden behind some object such that no or less direct sound from the occluded part of the audio element reaches the listener. Depending on the material of the occluding object, the occlusion effect might be either complete occlusion (e.g. when the occluding object is a thick wall), or soft occlusion where some of the audio energy from the audio element passes through the occluding object (e.g., when the occluding object is made of thin fabric such as a curtain).

<CIT> discloses an audio signal processing device. "The processor can acquire information related to an input audio signal and a virtual space in which the input audio signal is simulated, can determine whether a blocking object, which performs blocking between a sound source and a listener, exists among a plurality of objects, on the basis of the position of each of the plurality of objects included in the virtual space and the position of the sound source corresponding to the input audio signal, with respect to the listener in the virtual space, and can binaurally render the input audio signal on the basis of the determination result so as to generate an output audio signal. (abstract)" <CIT> discloses an audio engine for acoustically rendering a three-dimensional virtual environment. The audio engine uses geometric volumes to represent sound sources and any sound occluders. A volumetric response is generated based on sound projected from a volumetric sound source to a listener, taking into consideration any volumetric occluders in-between. <CIT> discloses a method of synthesizing an audio signal in two speaker systems or headphones. <CIT> discloses a method and apparatus for controlling a sound to be provided to a user based on a multipole sound object.

Certain challenges presently exist. For example, available occlusion rendering techniques deal with point sources where the occurrence of occlusion can be detected easily using raytracing between the listener position and the position of the point source, but for an audio element with an extent, the situation is more complicated since an occluding object may occlude only a part of the extended audio element. Therefore, a more elaborate occlusion detection technique is needed (e.g., one that determines which part of the extended audio element is occluded). For a heterogeneous extended audio element (i.e., an audio element with an extent which has non-homogeneous spatial audio information distributed over its extent (e.g. an extended audio element that is represented by a stereo signal)), the situation is even more complicated because the rendering of a partly occluded object of this type should take into account what would be the expected result of the partly occlusion on the spatial audio information that reaches the listener. A special version of the latter problem appears when a heterogeneous extended audio element is rendered by means of a discrete number of virtual loudspeakers. If using traditional occlusion, operating on individual virtual loudspeakers, and one or more of the virtual loudspeakers are occluded, which, for example, in the case of using two virtual loudspeakers (e.g. a left (L) and right (R) speaker) would mean that basically all spatial information is lost whenever either the L or R virtual loudspeaker is occluded. More generally in the case of extended objects that are rendered using a discrete number of virtual loudspeakers (so also including non-heterogeneous audio elements, e.g. homogeneous or diffuse extended audio elements), there is a problem with the amount of occlusion changing in a step-wise manner when the audio element, the occluding object, and/or listener are moving relative to each other.

Accordingly, in one aspect there is provided a method for rendering an audio element that is partially occluded, where the audio element has an extent and is represented using a set of two or more virtual loudspeakers, the set comprising a first virtual loudspeaker. A projection of the audio element is divided into sub-areas, where each sub-area is associated with one virtual loudspeaker. In one embodiment, the method includes determining a first occlusion amount, O1, for a first sub-area and modifying a first virtual loudspeaker signal for the first virtual loudspeaker based on O1 such that the modified loudspeaker signal is equal to: g1 * VS1, where g1 is a gain factor that is calculated using O1 and VS1 is the first virtual loudspeaker signal, thereby producing a first modified virtual loudspeaker signal. The method also includes determining a second occlusion amount, O2, for a second sub-area and modifying a second virtual loudspeaker signal for the second virtual loudspeaker based on O2 such that the second modified loudspeaker signal is equal to: g2 * VS2, where g2 is a gain factor that is calculated using O2 and VS2 is the second virtual loudspeaker signal, thereby producing a second modified virtual loudspeaker signal. The method also includes using the first and second modified virtual loudspeaker signals to render the audio element (e.g., generate an output signal using the first modified virtual loudspeaker signal). In another embodiment the method includes moving the first virtual loudspeaker from an initial position to a new position. The method also includes generating a first virtual loudspeaker signal for the first virtual loudspeaker based on the new position of the first virtual loudspeaker. The method also includes using the first virtual loudspeaker signal to render the audio element.

In another aspect there is provided a rendering apparatus that is configured to perform either of the above described methods. The rendering apparatus may include memory and processing circuitry coupled to the memory.

An advantage of the embodiments disclosed herein is that the rendering of an audio element that is at least partially occluded is done in a way that preserves the quality of the spatial information of the audio element.

The occurrence of occlusion may be detected using raytracing methods where the direct path between the listener position and the position of the audio element is searched for any occluding objects. <FIG> shows an example of two point sources (S1 and S2), where one is occluded by an object (O) (which is referred to as the "occluding object") and the other is not. In this case the occluded audio element should be muted in a way that corresponds to the acoustic properties of the material of the occluding object. If the occluding object is a thick wall, the rendering of the direct sounds from the occluded audio element should be more or less completely muted. In the case of an audio element (E) with an extent, as shown in <FIG>, the audio element (E) may be only partly occluded. This means that the rendering of the audio element needs to be altered in a way that reflects what part of the extent is occluded and what part is not occluded.

One strategy to for solving the occlusion problem for an audio element having an extent (see audio element <NUM> of <FIG>) is to represent the audio element <NUM> with a large number of point sources spread out over the extent (as shown in <FIG>) and calculate the occlusion effect individually for each point source using one of the known methods for point sources. This strategy, however, is highly inefficient due to the large number of point sources that need to be used in order to get a good enough resolution of the occlusion effect. And even if many point sources are used so that the resolution for a static case is good enough, there would still be a stepwise behavior where the effect of the occlusion changes in discrete steps as the individual point sources are either occluded or not occluded in a dynamic scene. Another disadvantage with using many point sources to represent a heterogeneous (multi-channel) audio element is that it is not trivial how to up-mix from a few audio channels to a large number of point sources without causing spatial and/or spectral distortions in the resulting listener signals (due to the fact that neighboring point sources would be highly correlated).

Accordingly, this disclosure describes additional embodiments that do not suffer these drawbacks discussed in the preceding paragraph. In one aspect, a method according to one embodiment comprises the steps of:.

Given the knowledge of what sub-areas of the audio element (more precisely a projection of the audio element) are at least partially occluded and given knowledge about the occluding object (e.g., a parameter indicating the amount of audio energy from the audio element that passes through the occluding object), an amount of occlusion is calculated for each said sub-area. In a scenario where the parameter indicates that no energy from the audio element passes through the occluding object, then the amount of occlusion is calculated as the percentage of the sub-area that is occluded from the listening position.

The sub-areas of the projection of the audio element can be defined in many different ways. There are as many sub-areas as there are virtual loudspeakers used for the rendering, and each sub-area corresponds to one virtual loudspeaker. In another embodiment not falling under the scope of the appended claims, the sub-areas are defined independently from the number and/or positions of the virtual loudspeakers used for the rendering. The sub-areas may be equal in size. The sub-areas may be directly adjacent to each other. The sub-areas together may completely fill the surface area of the projected extent of the audio element, i.e. the total size of the projected extent is equal to the sum of the surface areas of all the sub-areas.

For each sub-area, a gain factor is calculated depending on the amount of occlusion for that area. For example, in some scenarios where the occluding object is a thick, brick wall or the like, a sub-area that is completely occluded (amount is <NUM>%) by the occluding brick wall may be completely muted and the gain factor should therefore be set to <NUM>. For a sub-area where the occlusion amount is <NUM>, the gain factor should be set to <NUM>. For other amounts of occlusion, the gain factor should be somewhere in-between <NUM> and <NUM>, but the exact behavior may depend on the spatial character of the audio element. In one embodiment the gain factor is calculated as:<MAT> where O is the occlusion amount in percent.

In one embodiment, O for a given sub-area is a function of a frequency dependent occlusion factor (OF) and a value P, where P is the percentage of the sub-area that is covered by the occluding object (i.e., the percentage of the sub-area that cannot be seen by the listener due to the fact that the occluding object is located between the listener and the sub-area). For example, O = OF * P, where OF = Of1 for frequencies below f1, OF=Of2 for frequencies between f1 and f2, and OF=Of3 for frequencies above f2. That is, for a given frequency, different types of occluding objects may have a different occlusion factor. For instance, for a first frequency, a brick wall may have an occlusion factor of <NUM>, whereas a thin curtain of cotton may have an occlusion factor of <NUM>, and for a second frequency, the brick wall may have an occlusion factor of <NUM>, whereas a thin curtain of cotton may have an occlusion factor of <NUM>.

In another embodiment, the gain factor is calculated using the assumption that the audio element is mostly diffuse in spatial information and a <NUM>% occlusion amount should give a -3dB reduction in audio energy from that sub-area. The gain factor can then be calculated as: <MAT> or as <MAT>.

The embodiments are not limited to the above examples as other gain functions for calculating the gain of a sub-area are possible. As exemplified by the two embodiments described above, the effect of the occlusion can be a gradual one when the audio element is partly occluded, so that the signal from a virtual loudspeaker is not necessarily completely muted whenever the virtual loudspeaker is occluded for the listener. This prevents that, for example, in the case of a stereo rendering with two virtual loudspeakers, no sound at all is received from, for example, the left half of the audio element whenever the left virtual loudspeaker is occluded. Additionally, it prevents the undesirable "step-wise" occlusion effect when the occluding object, the audio element and/or the listener are moving relative to each other.

When a part of the audio element is occluded, the positions of the virtual loudspeakers representing the audio element can be moved so that they better represent the non-occluded part. If one of the edges of the extent of the audio element is occluded, the virtual loudspeaker(s) representing this edge should be move to the edge where the occlusion is happening as illustrated in <FIG> and <FIG>.

In the case where an occluding object is covering the middle of the audio element, as shown in <FIG>, the speaker positions are kept intact and the effect of the occlusion is only represented by the gain factors of the signals going to the respective virtual loudspeaker.

In the case that the audio element is only represented by virtual loudspeakers in the horizontal plane, an occlusion that covers either the bottom or top part can be rendered by changing the vertical position of the virtual loudspeakers so that their vertical position corresponds to the middle of the non-occluded part of the extent.

In another embodiment, the vertical position of each virtual loudspeaker is controlled by the ratio of occlusion amount in the upper sub-area and the lower sub-area. An example of how this position can be calculated is given by: <MAT> where PY is the vertical coordinate of the loudspeaker, OU and OL are the occlusion amount of the upper part and the lower part of the extent. PYT and PYB are the vertical coordinate of the top and bottom edges of the extent.

<FIG> is a flowchart illustrating a process <NUM>, according to an embodiment, for rendering an at least partially occluded audio element represented using a set of two or more virtual loudspeakers, the set comprising a first virtual loudspeaker. Process <NUM> may begin in step s402. Step s402 comprises modifying a first virtual loudspeaker signal for the first virtual loudspeaker, thereby producing a first modified virtual loudspeaker signal. Step s404 comprises using the first modified virtual loudspeaker signal to render the audio element (e.g., generate an output signal using the first modified virtual loudspeaker signal).

In some embodiments, the process further includes obtaining information indicating that the audio element is at least partially occluded, wherein the modifying is performed as a result of obtaining the information.

In some embodiments, the process further includes detecting that the audio element is at least partially occluded, wherein the modifying is performed as a result of the detection.

In some embodiments, modifying the first virtual loudspeaker signal comprises adjusting the gain of the first virtual loudspeaker signal.

In some embodiments, the process further includes moving the first virtual loudspeaker from an initial position (e.g., default position) to a new position and then generating the first virtual loudspeaker signal using information indicating the new position.

In some embodiments, the process further includes determining an occlusion amount (O) associated with the first virtual loudspeaker and the step of modifying the first virtual loudspeaker signal for the first virtual loudspeaker comprises modifying the first virtual loudspeaker signal based on O. In some embodiments, modifying the first virtual loudspeaker signal based on O comprises modifying the first virtual loudspeaker signal VS1 such that the modified loudspeaker signal equals (g * VS1), where g is a gain factor that is calculated using O and VS1 is the first virtual loudspeaker signal. In one embodiment, g = <NUM> -. <NUM> * O or g = sqrt(<NUM> -. <NUM> * O). In one embodiment determining O comprises obtaining a particular occlusion factor (Of) for the occluding object and determining a percentage of a sub-area of a projection of the audio element that is covered by the occluding object, where the first virtual loudspeaker is associated with the sub-area.

<FIG> is a flowchart illustrating a process <NUM>, according to an embodiment, for rendering an at least partially occluded audio element represented using a set of two or more virtual loudspeakers, the set comprising a first virtual loudspeaker. Process <NUM> may begin in step s452. Step s452 comprises moving the first virtual loudspeaker from an initial position to a new position. Step s454 comprises generating a first virtual loudspeaker signal for the first virtual loudspeaker based on the new position of the first virtual loudspeaker. Step s456 comprises using the first virtual loudspeaker signal to render the audio element. In some embodiments, the process further includes obtaining information indicating that the audio element is at least partially occluded, wherein the moving is performed as a result of obtaining the information. In some embodiments, the process further includes detecting that the audio element is at least partially occluded, wherein the moving is performed as a result of the detection.

<FIG> is a flowchart illustrating a process <NUM>, according to an embodiment, for rendering an occluded audio element. Process <NUM> may begin in step s502. Step s502 comprises obtaining metadata for an audio element and metadata for an object occluding the audio element (the metadata for the occluding object may include information specifying the occlusion factors for the object at different frequencies). Step s504 comprises, for each sub-area of the audio element, determining the amount of occlusion. Step s506 comprises calculating a gain factor for each virtual loudspeaker signal based on the amount of occlusion. Step s508 comprises, for each virtual loudspeaker, determining whether the virtual loudspeaker should be positioned in a new location and position the virtual loudspeaker in the new location. Step s510 comprises generating the virtual loudspeaker signals based on the locations of the virtual speakers. Step s512 comprises, based on the gain factors, adjusting the gains of one or more of the virtual loudspeaker signals.

<FIG> is an example of where audio element <NUM> (or, more precisely, the projection of the audio element <NUM> as seen from the listener position) is logically divided into six parts (a. , six sub-areas), where parts <NUM> & <NUM> represents the left area of the audio element <NUM>, parts <NUM> & <NUM> represents the right area, and parts <NUM> & <NUM> represents the center. Also, parts <NUM>, <NUM> & <NUM> together represent the upper area of the audio element and parts <NUM>, <NUM> & <NUM> represent the lower area of the audio element.

<FIG> shows an example scenario where audio element <NUM> as seen by the listener is partially occluded by an occluding object <NUM>, which, in this example and the other examples, has an occlusion factor of <NUM>. By calculating how much of each part of audio element <NUM> is covered by occluding object <NUM>, the relative gain balance of the left, center and right parts can be calculated. Likewise, a relative gain balance of the upper area as compared to the lower area can be calculated. In the example shown in <FIG>, the right area of the audio element should be completely muted as it is completely covered by object <NUM>, the center area should have slightly lower gain and the left area is unaffected. There is no difference in occlusion of the upper area as compared to the lower area.

<FIG> shows an example scenario where audio element <NUM> is partially occluded by an occluding object <NUM>. In this example, the center and right area should be partly muted. The lower part should be more muted than the upper part.

<FIG> shows an example where audio element <NUM> is represented by three virtual loudspeakers, SpL, SpC, SpR. <FIG> shows how the positions of the virtual loudspeakers are modified to reflect the occlusion of audio element <NUM> by object <NUM>. The speaker SpR, representing the right edge of the extent, is moved to the edge where the occlusion is happening. Speaker SpC is moved to the center of the part that is not occluded. <FIG> shows how the positions of the virtual loudspeakers are modified to reflect the occlusion of audio element <NUM> by object <NUM>. The speaker SpR, representing the right edge of the extent, is moved upward to a new position and speaker SpC is also moved upward.

<FIG> shows an example where the right sub-areas of audio element <NUM> are partly occluded. In this case the virtual loudspeaker representing the right edge is moved so that it lines up with the edge where the occlusion happens. The center speaker may be moved to the position representing the center of the non-occluded part of the audio element.

<FIG> shows an example of an audio element <NUM> that is represented by six virtual loudspeakers, where the lower part of the audio element is occluded. In this case the virtual loudspeakers representing the bottom edge are moved so that they line up with the edge where the occlusion happens.

<FIG> shows an example where the middle of the audio element <NUM> is occluded. In this case the positions of the loudspeakers are kept as they are since neither the left or the right edges are occluded and need to be represented. The occlusion in this case is only affecting the gain of the signals to each speaker. In this case the middle speaker would be completely muted (i.e., gain factor = <NUM>) and the gain to the left and right speakers slightly lowered to reflect that also sub-areas <NUM>,<NUM>,<NUM> and <NUM> are partly occluded.

<FIG> shows an example where the center and right areas of audio element <NUM> are partly occluded. The positions of the virtual loudspeakers are modified in elevation so that the greater amount of occlusion of these lower parts is reflected. The gain of the signals should also be lowered in order to reflect that the center and right areas are partly occluded.

<FIG> illustrates an XR system <NUM> in which the embodiments may be applied. XR system <NUM> includes speakers <NUM> and <NUM> (which may be speakers of headphones worn by the listener) and a display device <NUM> that is configured to be worn by the listener. As shown in <FIG>, XR system <NUM> may comprise an orientation sensing unit <NUM>, a position sensing unit <NUM>, and a processing unit <NUM> coupled (directly or indirectly) to an audio render <NUM> for producing output audio signals (e.g., a left audio signal <NUM> for a left speaker and a right audio signal <NUM> for a right speaker as shown). Audio renderer <NUM> produces the output signals based on input audio signals, metadata regarding the XR scene the listener is experiencing, and information about the location and orientation of the listener. The metadata for the XR scene may include metadata for each object and audio element included in the XR scene, and the metadata for an object may include information about the dimensions of the object and the occlusion factors for the object (e.g., the metadata may specify a set of occlusion factors where each occlusion factor is applicable for a different frequency or frequency range). Audio renderer <NUM> may be a component of display device <NUM> or it may be remote from the listener (e.g., renderer <NUM> may be implemented in the "cloud").

Orientation sensing unit <NUM> is configured to detect a change in the orientation of the listener and provides information regarding the detected change to processing unit <NUM>. In some embodiments, processing unit <NUM> determines the absolute orientation (in relation to some coordinate system) given the detected change in orientation detected by orientation sensing unit <NUM>. There could also be different systems for determination of orientation and position, e.g. a system using lighthouse trackers (lidar). In one embodiment, orientation sensing unit <NUM> may determine the absolute orientation (in relation to some coordinate system) given the detected change in orientation. In this case the processing unit <NUM> may simply multiplex the absolute orientation data from orientation sensing unit <NUM> and positional data from position sensing unit <NUM>. In some embodiments, orientation sensing unit <NUM> may comprise one or more accelerometers and/or one or more gyroscopes.

<FIG> shows an example implementation of audio renderer <NUM> for producing sound for the XR scene. Audio renderer <NUM> includes a controller <NUM> and a signal modifier <NUM> for modifying audio signal(s) <NUM> (e.g., the audio signals of a multi-channel audio element) based on control information <NUM> from controller <NUM>. Controller <NUM> may be configured to receive one or more parameters and to trigger modifier <NUM> to perform modifications on audio signals <NUM> based on the received parameters (e.g., increasing or decreasing the volume level). The received parameters include information <NUM> regarding the position and/or orientation of the listener (e.g., direction and distance to an audio element), metadata <NUM> regarding an audio element in the XR scene (e.g., audio element <NUM>), and metadata regarding an object occluding the audio element (e.g., object <NUM>) (in some embodiments, controller <NUM> itself produces the metadata <NUM>). Using the metadata and position/orientation information, controller <NUM> may calculate one more gain factors (g) for an audio element in the XR scene that is at least partially occluded as described above.

<FIG> shows an example implementation of signal modifier <NUM> according to one embodiment. Signal modifier <NUM> includes a directional mixer <NUM>, a gain adjuster <NUM>, and a speaker signal producer <NUM>.

Directional mixer <NUM> receives audio input <NUM>, which in this example includes a pair of audio signals <NUM> and <NUM> associated with an audio element (e.g. audio element <NUM>), and produces a set of k virtual loudspeaker signals (VS1, VS2,. , VSk) based on the audio input and control information <NUM>. In one embodiment, the signal for each virtual loudspeaker can be derived by, for example, the appropriate mixing of the signals that comprise the audio input <NUM>. For example: VS1 = α × L + β × R, where L is input audio signal <NUM>, R is input audio signal <NUM>, and α and β are factors that are dependent on, for example, the position of the listener relative to the audio element and the position of the virtual loudspeaker to which VS1 corresponds.

In the example where audio element <NUM> is associated with three virtual loudspeakers (SpL, SpC, and SpR), then k will equal <NUM> for the audio element and VS1 may correspond to SpL, VS2 may correspond to SpC, and VS3 may correspond to SpR. The control information <NUM> used by directional mixer to produce the virtual loudspeaker signals may include the positions of each virtual loudspeaker relative to the audio element. In some embodiments, controller <NUM> is configured such that, when the audio element is occluded, controller <NUM> may adjust the position of one or more of the virtual loudspeakers associated with the audio element and provide the position information to directional mixer <NUM> which then uses the updated position information to produce the signals for the virtual loudspeakers (i.e., VS1, VS2,.

Gain adjuster <NUM> may adjust the gain of any one or more of the virtual loudspeaker signals based on control information <NUM>, which may include the above described gain factors as calculated by controller <NUM>. That is, for example, when the audio element is at least partially occluded, controller <NUM> may control gain adjuster <NUM> to adjust the gain of one or more of the virtual loudspeaker signals by providing one or more gain factors to gain adjuster <NUM>. For instance, if the entire left portion of the audio element is occluded, then controller <NUM> may provide to gain adjuster <NUM> control information <NUM> that causes gain adjuster <NUM> to reduce the gain of VS1 by <NUM>% (i.e., gain factor = <NUM> so that VS1' = <NUM>). As another example, if only <NUM>% of the left portion of the audio element is occluded and <NUM>% of the center portion is occluded, then controller <NUM> may provide to gain adjuster <NUM> control information <NUM> that causes gain adjuster <NUM> to reduce the gain of VS1 by <NUM>% (i.e., VS1' = <NUM>% VS1) and to not reduce the gain of VS2 at all (i.e., gain factor = <NUM> so that VS2' = VS2).

Using virtual loudspeaker signals VS1', VS2',. , VSk', speaker signal producer <NUM> produces output signals (e.g., output signal <NUM> and output signal <NUM>) for driving speakers (e.g., headphone speakers or other speakers). In one embodiment where the speakers are headphone speakers, speaker signal producer <NUM> may perform conventional binaural rendering to produce the output signals. In embodiments where the speakers are not headphone speakers, speaker signal producer <NUM> may perform conventional speaking panning to produce the output signals.

<FIG> is a block diagram of an audio rendering apparatus <NUM>, according to some embodiments, for performing the methods disclosed herein (e.g., audio renderer <NUM> may be implemented using audio rendering apparatus <NUM>). As shown in <FIG>, audio rendering apparatus <NUM> may comprise: processing circuitry (PC) <NUM>, which may include one or more processors (P) <NUM> (e.g., a general purpose microprocessor and/or one or more other processors, such as an application specific integrated circuit (ASIC), field-programmable gate arrays (FPGAs), and the like), which processors may be co-located in a single housing or in a single data center or may be geographically distributed (i.e., apparatus <NUM> may be a distributed computing apparatus); at least one network interface <NUM> comprising a transmitter (Tx) <NUM> and a receiver (Rx) <NUM> for enabling apparatus <NUM> to transmit data to and receive data from other nodes connected to a network <NUM> (e.g., an Internet Protocol (IP) network) to which network interface <NUM> is connected (directly or indirectly) (e.g., network interface <NUM> may be wirelessly connected to the network <NUM>, in which case network interface <NUM> is connected to an antenna arrangement); and a storage unit (a. , "data storage system") <NUM>, which may include one or more non-volatile storage devices and/or one or more volatile storage devices. In embodiments where PC <NUM> includes a programmable processor, a computer program product (CPP) <NUM> may be provided. CPP <NUM> includes a computer readable medium (CRM) <NUM> storing a computer program (CP) <NUM> comprising computer readable instructions (CRI) <NUM>. CRM <NUM> may be a non-transitory computer readable medium, such as, magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory, flash memory), and the like. In some embodiments, the CRI <NUM> of computer program <NUM> is configured such that when executed by PC <NUM>, the CRI causes audio rendering apparatus <NUM> to perform steps described herein (e.g., steps described herein with reference to the flow charts). In other embodiments, audio rendering apparatus <NUM> may be configured to perform steps described herein without the need for code. That is, for example, PC <NUM> may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and/or software.

While various embodiments are described herein, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of this disclosure should not be limited by any of the above described exemplary embodiments. Moreover, any combination of the above-described objects in all possible variations thereof is encompassed by the disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

Claim 1:
A method (<NUM>) for rendering a partially occluded audio element (<NUM>, <NUM>), the audio element having an extent and being represented using a set of two or more virtual loudspeakers (SpL, SpC, SpR), the set comprising a first and a second virtual loudspeaker, a projection of the audio element being divided into sub-areas, wherein each sub-area is associated with one virtual loudspeaker, the method comprising:
determining a first occlusion amount, O1, for a first sub-area;
modifying (s402) a first virtual loudspeaker signal for the first virtual loudspeaker based on O1 such that the modified loudspeaker signal is equal to: g1 * VS1, where g1 is a gain factor that is calculated using O1 and VS1 is the first virtual loudspeaker signal, thereby producing a first modified virtual loudspeaker signal;
determining a second occlusion amount, O2, for a second sub-area;
modifying a second virtual loudspeaker signal for the second virtual loudspeaker based on O2 such that the second modified loudspeaker signal is equal to: g2 * VS2, where g2 is a gain factor that is calculated using O2 and VS2 is the second virtual loudspeaker signal, thereby producing a second modified virtual loudspeaker signal; and
using (s404) the first and second modified virtual loudspeaker signals to render the audio element.