Patent Description:
Parametric spatial audio processing is a field of audio signal processing where the spatial aspect of the sound is described using a set of parameters. For example, in parametric spatial audio capture from microphone arrays, it is a typical and an effective choice to estimate from the microphone array signals a set of parameters such as directions of the sound in frequency bands, and the ratios between the directional and non-directional parts of the captured sound in frequency bands. These parameters are known to well describe the perceptual spatial properties of the captured sound at the position of the microphone array. These parameters can be utilized in synthesis of the spatial sound accordingly, for headphones binaurally, for loudspeakers, or to other formats, such as Ambisonics.

The directions and direct-to-total energy ratios in frequency bands are thus a parameterization that is particularly effective for spatial audio capture.

A parameter set consisting of a direction parameter in frequency bands and an energy ratio parameter in frequency bands (indicating the directionality of the sound) can be also utilized as the spatial metadata (which may also include other parameters such as surround coherence, spread coherence, number of directions, distance, etc.) for an audio codec. For example, these parameters can be estimated from microphone-array captured audio signals, and for example a stereo or mono signal can be generated from the microphone array signals to be conveyed with the spatial metadata. The stereo signal could be encoded, for example, with an AAC encoder and the mono signal could be encoded with an EVS encoder. A decoder can decode the audio signals into PCM signals and process the sound in frequency bands (using the spatial metadata) to obtain the spatial output, for example a binaural output.

The aforementioned solution is particularly suitable for encoding captured spatial sound from microphone arrays (e.g., in mobile phones, VR cameras, stand-alone microphone arrays). However, it may be desirable for such an encoder to have also other input types than microphone-array captured signals, for example, loudspeaker signals, audio object signals, or Ambisonic signals.

Analysing first-order Ambisonics (FOA) inputs for spatial metadata extraction has been thoroughly documented in scientific literature related to Directional Audio Coding (DirAC) and Harmonic planewave expansion (Harpex). This is since there exist microphone arrays directly providing a FOA signal (more accurately: its variant, the B-format signal), and analysing such an input has thus been a point of study in the field. Furthermore, the analysis of higher-order Ambisonics (HOA) input for multi-direction spatial metadata extraction has also been documented in the scientific literature related to higher-order directional audio coding (HO-DirAC).

A further input for the encoder is also multi-channel loudspeaker input, such as <NUM> or <NUM> channel surround inputs and audio objects.

However, with respect to the components of the spatial metadata the compression and encoding of the spatial audio parameters (such as the direct-to-total energy ratios) is of considerable interest in order to minimise the overall number of bits required to represent the spatial audio parameters.

It is proposed in <CIT> to encode ER (= "energy ratios") parameters using a fixed predefined number of bits or using a variable number of bits in dependence of the value of the ER parameter. This patent application further exemplifies the use of a variable bit-rate quantiser that assigns shorter codewords for those ER parameter values that represent relatively high values of the ER parameter (e.g. relatively high values of the direct-to- total-energy ratio) and assigns longer codewords for those ER parameter values that represent relatively low values of the ER parameter (e.g. relatively low values of the direct-to-total-energy ratio) or that assigns shorter codewords for those ER parameter values that occur more frequently and assigns longer codewords for those ER parameter values that occur less frequently using ER quantisation table that maps a plurality of table entries that each store a pair of a quantised ER parameter value and a codeword (e.g. a bit-pattern) assigned thereto. It is otherwise proposed in <CIT> to apply a scalar non-uniform quantisation using <NUM> bits for each sub-band to encode energy ratios.

There is according to a first aspect an apparatus for spatial audio encoding comprising means for: converting two or more energy ratios associated with a time frequency tile of one or more audio signals to a further energy ratio parameter which is related to the two or more energy ratios; quantizing the further energy ratio parameter using a first quantizer; determining a distribution factor of energy ratios dependent on a ratio of a first of the two or more energy ratios to the sum of the two or more energy ratios; selecting a further quantizer from a plurality of further quantizers using the quantized further energy ratio parameter; and quantizing the distribution factor of energy ratios using the selected further quantizer.

The two or more energy ratios may be two direct-to-total energy ratios;
The further energy ratio parameter may be a diffuse-to-total energy ratio.

The diffuse-to-total energy ratio may comprise one minus the sum of the two direct-to-total energy ratios.

The further energy ratio parameter may be the sum of the two direct-to-total energy ratios.

The distribution factor of energy ratios may comprise the ratio of a first of the two direct-to-total energy ratios to the sum of the two direct-to-total energy ratios.

The means for selecting a further quantizer from a plurality of further quantizers using the quantized further energy ratio parameter may comprise means for: comparing the quantized further energy ratio parameter to a threshold value; and selecting the further quantizer from a plurality of further quantizers based on the comparison.

A first of the two direct-to-total energy ratios may be associated with a first direction of a sound wave and a second of the two direct-to-total energy ratio may be associated with a second direction of a sound wave, wherein the apparatus may further comprise proceeding means for: determining that a second of the two direct-to-total energy ratios is greater than a first of the two direct-to-total energy ratios; swapping the first of the two direct-to-total energy ratios to be associated with the second direction; and swapping the second of the two direct-to-total energy ratios to be associated with the first direction.

A first direction index, a first spread coherence and a first distance associated with the time frequency tile may each be associated with a first direction of the sound wave, and a second direction index, a second spread coherence and a second distance associated with the time frequency tile may each be associated with the second direction of the sound wave, if is determined that the second of the two direct-to-total energy ratios is greater than the first of the two direct-to-total energy ratios, the apparatus may further comprise the means for at least one of the following: swapping the first direction index to be associated with the second direction and swapping the second direction index to be associated with the first direction; swapping the first distance to be associated with the second direction and swapping the second distance to be associated with the first direction; and swapping the first spread coherence to be associated with the second direction and swapping the second spread coherence to be associated with the first direction.

There is according to a second aspect a method for spatial audio encoding comprising: converting two or more energy ratios associated with a time frequency tile of one or more audio signals to a further energy ratio parameter which is related to the two or more energy ratios; quantizing the further energy ratio parameter using a first quantizer; determining a distribution factor of energy ratios dependent on a ratio of a first of the two or more energy ratios to the sum of the two or more energy ratios; selecting a further quantizer from a plurality of further quantizers using the quantized further energy ratio parameter; and quantizing the distribution factor of energy ratios using the selected further quantizer.

The two or more energy ratios may be two direct-to-total energy ratios;
The distribution factor of energy ratios may comprise the ratio of a first of the two direct-to-total energy ratios to the sum of the two direct-to-total energy ratios.

Selecting a further quantizer from a plurality of further quantizers using the quantized further energy ratio parameter may comprise comparing the quantized further energy ratio parameter to a threshold value; and selecting the further quantizer from a plurality of further quantizers based on the comparison.

A first of the two direct-to-total energy ratios may be associated with a first direction of a sound wave and a second of the two direct-to-total energy ratio may be associated with a second direction of a sound wave, wherein the method further comprises the preceding processing steps of: determining that a second of the two direct-to-total energy ratios is greater than a first of the two direct-to-total energy ratios; swapping the first of the two direct-to-total energy ratios to be associated with the second direction; and swapping the second of the two direct-to-total energy ratios to be associated with the first direction.

A first direction index, a first spread coherence and a first distance associated with the time frequency tile may also be each associated with a first direction of the sound wave, and wherein a second direction index, a second spread coherence and a second distance associated with the time frequency tile may also each be associated with the second direction of the sound wave, wherein if is determined that the second of the two direct-to-total energy ratios is greater than the first of the two direct-to-total energy ratios, the method may further comprise at least one of the following: swapping the first direction index to be associated with the second direction and swapping the second direction index to be associated with the first direction; swapping the first distance to be associated with the second direction and swapping the second distance to be associated with the first direction; and swapping the first spread coherence to be associated with the second direction and swapping the second spread coherence to be associated with the first direction.

The following describes in further detail suitable apparatus and possible mechanisms for the provision of effective spatial analysis derived metadata parameters. In the following discussions multi-channel system is discussed with respect to a multi-channel microphone implementation. However as discussed above the input format may be any suitable input format, such as multi-channel loudspeaker, ambisonic (FOA/HOA), etc. Furthermore, the output of the example system is a multi-channel loudspeaker arrangement. However, it is understood that the output may be rendered to the user via means other than loudspeakers. Furthermore, the multi-channel loudspeaker signals may be generalised to be two or more playback audio signals. Such a system is currently being standardised by the 3GPP standardization body as the Immersive Voice and Audio Service (IVAS). IVAS is intended to be an extension to the existing 3GPP Enhanced Voice Service (EVS) codec in order to facilitate immersive voice and audio services over existing and future mobile (cellular) and fixed line networks. An application of IVAS may be the provision of immersive voice and audio services over 3GPP fourth generation (<NUM>) and fifth generation (<NUM>) networks. In addition, the IVAS codec as an extension to EVS may be used in store and forward applications in which the audio and speech content is encoded and stored in a file for playback. It is to be appreciated that IVAS may be used in conjunction with other audio and speech coding technologies which have the functionality of coding the samples of audio and speech signals.

The metadata may consist of at least of spherical directions (elevation, azimuth), at least one energy ratio of a resulting direction, a spread coherence, and surround coherence independent of the direction, for each considered time-frequency (TF) block or tile, in other words a time/frequency sub band. In total IVAS may have a number of different types of metadata parameters for each time-frequency (TF) tile. The types of spatial audio parameters which can make up the metadata for IVAS are shown in Table <NUM> below.

This data may be encoded and transmitted (or stored) by the encoder in order to be able to reconstruct the spatial signal at the decoder.

Moreover, in some instances metadata assisted spatial audio (MASA) may support up to <NUM> directions for each TF tile which would require the above parameters to be encoded and transmitted for each direction on a per TF tile basis. Thereby potentially doubling the required bit rate according to Table <NUM> below.

The bitrate allocated for metadata in a practical immersive audio communications codec may vary greatly. Typical overall operating bitrates of the codec may leave only <NUM> to 10kbps for the transmission/storage of spatial metadata. However, some further implementations may allow up to 30kbps or higher for the transmission/storage of spatial metadata. The encoding of the direction parameters and energy ratio components has been examined before along with the encoding of the coherence data. However, whatever the transmission/storage bit rate assigned for spatial metadata there will always be a need to use as few bits as possible to represent these parameters especially when a TF tile may support multiple directions corresponding to different sound sources in the spatial audio scene.

The concept as discussed hereafter is to quantize, the direct-to-total energy ratio for all directions in the form of the diffuse-to-total energy ratio for the TF tile and a ratio based on the direct-to-total energy ratios.

Accordingly, the invention proceeds from the consideration that the bit rate required for transmitting the MASA data (or spatial metadata spatial audio parameters) may be reduced by quantizing, on a TF tile basis, the direct-to-total energy ratio corresponding to each direction by using as few bits as possible in order to facilitate transmission and storage of the encoded audio signal.

In this regard, <FIG> depicts an example apparatus and system for implementing embodiments of the application. The system <NUM> is shown with an 'analysis' part <NUM> and a 'synthesis' part <NUM>. The 'analysis' part <NUM> is the part from receiving the multi-channel signals up to an encoding of the metadata and downmix signal and the 'synthesis' part <NUM> is the part from a decoding of the encoded metadata and downmix signal to the presentation of the re-generated signal (for example in multi-channel loudspeaker form).

The input to the system <NUM> and the 'analysis' part <NUM> is the multi-channel signals <NUM>. In the following examples a microphone channel signal input is described, however any suitable input (or synthetic multi-channel) format may be implemented in other embodiments. For example, in some embodiments the spatial analyser and the spatial analysis may be implemented external to the encoder. For example, in some embodiments the spatial metadata associated with the audio signals may be provided to an encoder as a separate bit-stream. In some embodiments the spatial metadata may be provided as a set of spatial (direction) index values. These are examples of a metadata-based audio input format.

The multi-channel signals are passed to a transport signal generator <NUM> and to an analysis processor <NUM>.

In some embodiments the transport signal generator <NUM> is configured to receive the multi-channel signals and generate a suitable transport signal comprising a determined number of channels and output the transport signals <NUM>. For example, the transport signal generator <NUM> may be configured to generate a <NUM>-audio channel downmix of the multi-channel signals. The determined number of channels may be any suitable number of channels. The transport signal generator in some embodiments is configured to otherwise select or combine, for example, by beamforming techniques the input audio signals to the determined number of channels and output these as transport signals.

In some embodiments the transport signal generator <NUM> is optional and the multi-channel signals are passed unprocessed to an encoder <NUM> in the same manner as the transport signal are in this example.

In some embodiments the analysis processor <NUM> is also configured to receive the multi-channel signals and analyse the signals to produce metadata <NUM> associated with the multi-channel signals and thus associated with the transport signals <NUM>.

The analysis processor <NUM> may be configured to generate the metadata which may comprise, for each time-frequency analysis interval, direction parameters <NUM> and energy ratio parameters <NUM> (comprising a direct-to-total energy ratio per direction and a diffuse-to-total energy ratio) and a coherence parameter <NUM>. The direction, energy ratio and coherence parameters may in some embodiments be considered to be spatial audio parameters. In other words, the spatial audio parameters comprise parameters which aim to characterize the sound-field created/captured by the multi-channel signals (or two or more audio signals in general).

In some embodiments the parameters generated may differ from frequency band to frequency band. Thus, for example in band X all of the parameters are generated and transmitted, whereas in band Y only one of the parameters is generated and transmitted, and furthermore in band Z no parameters are generated or transmitted. A practical example of this may be that for some frequency bands such as the highest band some of the parameters are not required for perceptual reasons. The transport signals <NUM> and the metadata <NUM> may be passed to an encoder <NUM>.

The encoder <NUM> may comprise an audio encoder core <NUM> which is configured to receive the transport (for example downmix) signals <NUM> and generate a suitable encoding of these audio signals. The encoder <NUM> can in some embodiments be a computer (running suitable software stored on memory and on at least one processor), or alternatively a specific device utilizing, for example, FPGAs or ASICs. The encoding may be implemented using any suitable scheme. The encoder <NUM> may furthermore comprise a metadata encoder/quantizer <NUM> which is configured to receive the metadata and output an encoded or compressed form of the information. In some embodiments the encoder <NUM> may further interleave, multiplex to a single data stream or embed the metadata within encoded downmix signals before transmission or storage shown in <FIG> by the dashed line. The multiplexing may be implemented using any suitable scheme.

In the decoder side, the received or retrieved data (stream) may be received by a decoder/demultiplexer <NUM>. The decoder/demultiplexer <NUM> may demultiplex the encoded streams and pass the audio encoded stream to a transport extractor <NUM> which is configured to decode the audio signals to obtain the transport signals. Similarly, the decoder/demultiplexer <NUM> may comprise a metadata extractor <NUM> which is configured to receive the encoded metadata and generate metadata. The decoder/demultiplexer <NUM> can in some embodiments be a computer (running suitable software stored on memory and on at least one processor), or alternatively a specific device utilizing, for example, FPGAs or ASICs.

The decoded metadata and transport audio signals may be passed to a synthesis processor <NUM>.

The system <NUM> 'synthesis' part <NUM> further shows a synthesis processor <NUM> configured to receive the transport and the metadata and re-creates in any suitable format a synthesized spatial audio in the form of multi-channel signals <NUM> (these may be multichannel loudspeaker format or in some embodiments any suitable output format such as binaural or Ambisonics signals, depending on the use case) based on the transport signals and the metadata.

Therefore, in summary first the system (analysis part) is configured to receive multi-channel audio signals.

Then the system (analysis part) is configured to generate a suitable transport audio signal (for example by selecting or downmixing some of the audio signal channels) and the spatial audio parameters as metadata.

The system is then configured to encode for storage/transmission the transport signal and the metadata.

After this the system may store/transmit the encoded transport signal and metadata.

The system may retrieve/receive the encoded transport signal and metadata.

Then the system is configured to extract the transport signal and metadata from encoded transport signal and metadata parameters, for example demultiplex and decode the encoded transport signal and metadata parameters.

The system (synthesis part) is configured to synthesize an output multi-channel audio signal based on extracted transport audio signals and metadata.

With respect to <FIG> an example analysis processor <NUM> and Metadata encoder/quantizer <NUM> (as shown in <FIG>) according to some embodiments is described in further detail.

<FIG> and <FIG> depict the Metadata encoder/quantizer <NUM> and the analysis processor <NUM> as being coupled together. However, it is to be appreciated that some embodiments may not so tightly couple these two respective processing entities such that the analysis processor <NUM> can exist on a different device from the Metadata encoder/quantizer <NUM>. Consequently, a device comprising the Metadata encoder/quantizer <NUM> may be presented with the transport signals and metadata streams for processing and encoding independently from the process of capturing and analysing.

The analysis processor <NUM> in some embodiments comprises a time-frequency domain transformer <NUM>.

In some embodiments the time-frequency domain transformer <NUM> is configured to receive the multi-channel signals <NUM> and apply a suitable time to frequency domain transform such as a Short Time Fourier Transform (STFT) in order to convert the input time domain signals into a suitable time-frequency signals. These time-frequency signals may be passed to a spatial analyser <NUM>.

Thus, for example, the time-frequency signals <NUM> may be represented in the time-frequency domain representation by
si(b, n),
where b is the frequency bin index and n is the time-frequency block (frame) index and i is the channel index. In another expression, n can be considered as a time index with a lower sampling rate than that of the original time-domain signals. These frequency bins can be grouped into sub bands that group one or more of the bins into a sub band of a band index k = <NUM>,. Each sub band k has a lowest bin bk,low and a highest bin bk,high, and the subband contains all bins from bk,low to bk,high. The widths of the sub bands can approximate any suitable distribution. For example, the Equivalent rectangular bandwidth (ERB) scale or the Bark scale.

A time frequency (TF) tile (or block) is thus a specific sub band within a subframe of the frame.

It can be appreciated that the number of bits required to represent the spatial audio parameters may be dependent at least in part on the TF (time-frequency) tile resolution (i.e., the number of TF subframes or tiles). For example, a <NUM> audio frame may be divided into <NUM> time-domain subframes of <NUM> a piece, and each time-domain subframe may have up to <NUM> frequency subbands divided in the frequency domain according to a Bark scale, an approximation of it, or any other suitable division. In this particular example the audio frame may be divided into <NUM> TF subframes/tiles, in other words <NUM> time-domain subframes with <NUM> frequency subbands. Therefore, the number of bits required to represent the spatial audio parameters for an audio frame can be dependent on the TF tile resolution. For example, if each TF tile were to be encoded according to the distribution of Table <NUM> above then each TF tile would require <NUM> bits (for one sound source direction per TF tile) and <NUM> bits (for two sound source directions per TF tile, taking into account parameters which are independent of the sound source direction).

In embodiments the analysis processor <NUM> may comprise a spatial analyser <NUM>. The spatial analyser <NUM> may be configured to receive the time-frequency signals <NUM> and based on these signals estimate direction parameters <NUM>. The direction parameters may be determined based on any audio based 'direction' determination.

For example, in some embodiments the spatial analyser <NUM> is configured to estimate the direction of a sound source with two or more signal inputs.

The spatial analyser <NUM> may thus be configured to provide at least one azimuth and elevation for each frequency band and temporal time-frequency block within a frame of an audio signal, denoted as azimuth ϕ(k, n), and elevation θ(k, n). The direction parameters <NUM> for the time sub frame may be also be passed to the spatial parameter set encoder <NUM>.

The spatial analyser <NUM> may also be configured to determine an energy ratio parameters <NUM>. The energy ratio may be considered to be a determination of the energy of the audio signal which can be considered to arrive from a direction. The direct-to-total energy ratio r(k,n) can be estimated, e.g., using a stability measure of the directional estimate, or using any correlation measure, or any other suitable method to obtain a ratio parameter. Each direct-to-total energy ratio corresponds to a specific spatial direction and describes how much of the energy comes from the specific spatial direction compared to the total energy. This value may also be represented for each time-frequency tile separately. The spatial direction parameters and direct-to-total energy ratio describe how much of the total energy for each time-frequency tile is coming from the specific direction. In general, a spatial direction parameter can also be thought of as the direction of arrival (DOA).

In embodiments the direct-to-total energy ratio parameter can be estimated based on the normalized cross-correlation parameter cor'(k,n) between a microphone pair at band k, the value of the cross-correlation parameter lies between -<NUM> and <NUM>. The direct-to-total energy ratio parameter r(k, n) can be determined by comparing the normalized cross-correlation parameter to a diffuse field normalized cross correlation parameter <MAT> as <MAT>. The direct-to-total energy ratio is explained further in <CIT>.

The energy ratio may be passed to the spatial parameter set encoder <NUM>.

The spatial analyser <NUM> may furthermore be configured to determine a number of coherence parameters <NUM> which may include surrounding coherence (γ(k, n)) and spread coherence (ζ(k, n)), both analysed in time-frequency domain.

The term audio source may relate to dominant directions of the propagating sound wave, which may encompass the actual direction of the sound source.

Therefore, for each sub band k there will be collection (or set) of spatial audio parameters associated with the sub band and sub frame n. In this instance each sub band k and sub frame n (in other words a TF tile) may have the following spatial audio parameters associated with it on a per audio source direction basis; at least one azimuth and elevation denoted as azimuth ϕ(k, n), and elevation θ(k, n), and a spread coherence (ζ(k, n) and a direct-to-total-energy ratio parameter r(k,n). Obviously if there is more than one direction per TF tile, then the TF tile can have each of the above listed parameters associated with each sound source direction. Additionally, the collection of spatial audio parameters may also comprise a surrounding coherence (γ(k, n)). Parameters may also comprise a diffuse-to-total energy ratio rdiff(k, n).

In embodiments the diffuse-to-total energy ratio rdiff(k, n) is the energy ratio of non-directional sound over surrounding directions and there is typically a single diffuse-to-total energy ratio (as well as surrounding coherence (γ(k, n)) per TF tile. The diffuse-to-total energy ratio may be considered to be the energy ratio remaining once the direct-to-total energy ratios (for each direction) have been subtracted from one. Going forward, the above parameters may be termed a set of spatial audio parameters (or a spatial audio parameter set) for a particular TF tile.

In embodiments the spatial parameter set encoder <NUM> can be arranged to quantize the energy ratio parameters <NUM> in addition to the direction parameters <NUM> and coherence parameters <NUM>. The energy ratio parameters <NUM> comprising direct-to-total-energy ratio parameters r(k,n) for each direction may be quantised based on the diffuse-to-total energy ratio rdiff(k, n) and a further parameter. The further parameter may comprise a ratio of one of the direct-to-total-energy ratio parameters to the sum of the direct-to-total energy ratios for all directions, the further parameter may be termed dr(k, n).

In some alternative embodiments, sum of direct-to-total energy ratios may be quantized instead of diffuse-to-total energy ratio rdiff(k, n), where the sum of direct-to-total energy ratios may be expressed as: <MAT>.

For TF tiles which have two audio source directions assigned to them, the direct-to-total-energy ratio parameter of the first direction r<NUM>(k, n) and the direct-to-total-energy ratio parameter of the second direction r<NUM>(k, n) for the TF tile (k, n) can be quantized in the form of the diffuse-to-total energy ratio rdiff(k, n) and dr(k, n) for the TF tile.

In embodiments the first direct-to-total-energy ratio parameter r<NUM>(k, n) and the second direct-to-total-energy ratio parameter r<NUM>(k, n) may be quantized by determining the diffuse-to-total energy ratio rdiff(k, n) as <MAT>.

In some alternative embodiments the diffuse-to-total energy ratio rdiff(k, n) may be provided as part of the MASA input metadata, rather than being calculated on the fly as outlined above. In this case the spatial parameter set encoder <NUM> may obtain a further energy ratio parameter (or diffuse-to-total energy ratio) associated with two or more energy ratios of a time frequency tile.

The step of determining the diffuse-to-total energy ratio rdiff(k, n) is shown as processing step <NUM> in <FIG>.

The value of rdiff(k, n) may then be scalar quantized to give r̂diff(k, n). In embodiments this may be performed using a non-uniform scalar quantizer.

The step of quantizing rdiff(k, n) is shown as processing step <NUM> in <FIG>.

In some embodiments the value of diffuse-to-total energy ratio parameter rdiff(k, n) can be used to determine the size of the quantizer to be used subsequently in the process. For instance, if rdiff(k, n) is above a selection value then a first sized quantizer may be selected, however if rdiff(k, n) is less that the selection value then a second sized quantizer may be selected. In embodiments this step may be written as
<IMG>.

In other words if rdiff(k, n) > Nq,(where Nq is the selection value) then the quantizer size Q<NUM> is selected, otherwise quantizer size Q<NUM> is selected. Q<NUM> and Q<NUM> may express the quantizer size in terms of the number of bits.

In embodiments Nq is found to lie between the values of <NUM> and <NUM>. For instance one operating point for Nq was found to be <NUM>.

In a particular example of an embodiment the above step may have the following numerical values
<IMG>.

In some embodiments the quantised diffuse-to-total energy ratio parameter r̂diff(k, n) may be used in the above processing step. This can have the advantage that the quantizer size (Quant_size) is not required to be signalled as part of the bitstream. Instead, the quantizer size may be determined at the decoder by inspecting the value of r̂diff(k, n).

The step of determining the size of the quantizer using r̂diff is shown as the processing step <NUM> in <FIG>.

Embodiments may then determine the ratio of the first direct-to-total-energy ratio parameter to the sum of the first and second direct-to-total-energy ratio parameters, in other words a distribution factor of energy ratios.

This distribution factor of energy ratios may be expressed as <MAT>.

The step of determining the above ratio dr is depicted as the processing step <NUM> in <FIG>.

For the case of three direct-to-total-energy ratio parameter per TF tile the diffuse-to-total energy ratio rdiff(k, n) may be expressed as <MAT> and the distribution factor of energy ratios may be given as <MAT> and <MAT>.

Naturally, the above scheme can be extended to a general number of direct-to-total-energy ratio parameter per TF tile.

The value of the ratio dr(k, n) may now be quantized using a scalar quantizer. In embodiments one of a number of quantizers may be selected to quantize dr(k, n). As alluded to above, the quantizer used to quantize the ratio dr may be selected based on the results of the above processing step <NUM>. In other words, the processing step <NUM> may be used to determine the size of the scalar quantizer used to quantize dr(k,n) to give <MAT>.

The processing step of selecting the quantizer for quantizing dr(k, n) is shown as step <NUM> in <FIG>.

In some embodiments dr(k, n) can be quantized using a quantizer selected from a number of uniform scalar quantizers. In the above example dr can be quantized to <MAT> using one of two uniform scalar quantizers as signified by Quant_size bits. Taking the above particular example of an embodiment either a <NUM> bit or <NUM> bit scalar quantizer may be used to quantize dr(k, n).

The processing step of quantizing dr(k, n) is shown as step <NUM> in <FIG>.

The indices corresponding to the two quantized parameters dr(k, n) and r̂diff(k, n) may be encoded using either a fixed or variable rate coding scheme.

Alternatively, the indices corresponding to the two quantized parameters <MAT> and r̂diff(k, n) may be jointly encoded by forming a master index and then use entropy encoding (such as Golomb Rice or Huffman encoding) to encode the master index.

In some embodiments the above quantization of the direct-to-total energy ratio parameters may comprise an additional pre-processing step in which for each TF tile it is checked whether there are actually two direct-to-total energy ratios r<NUM>(k, n), r<NUM>(k, n) (associated with the first and second directions). The presence of a second direct-to-total energy ratio would indicate that the TF tile (k,n) has at least two concurrent directions.

If it is determined that the TF tile has two concurrent directions then spatial audio parameters associated with each of the two directions may be swapped if the direct-to-total energy ratio r<NUM>(k, n) of the first direction is less than the direct-to-total energy ratio r<NUM>(k, n) of the second direction. In embodiments the spatial audio parameters associated with a particular audio direction may comprise the parameters (from above Table <NUM>) ; direction index, Direct-to-total energy ratio, spread coherence and distance. In other words, the pre-processing step may have the following form.

The above procedure effectively orders the directions such that the direction with the larger direct-to-total energy ratio is always the first direction, and the direction with the smaller direct-to-total energy ratio is always the second direction.

The above pre-processing step can have the advantage of allowing more efficient quantizers, such that dr is always between <NUM> and <NUM> (in comparison to having the values between <NUM> and <NUM> in case the above swapping mechanism is not performed). Hence, the same accuracy may be obtained with roughly half the number of codewords.

Any further processing undertaken by the spatial parameter set encoder <NUM> may use the quantized direct-to-total energy ratios obtained from r̂diff and <MAT>.

The above quantization scheme has been described in the terms of energy ratios for a TF tile. However, it would be appreciated by the skilled person that the above could be equally applied to other parameters which quantify a signal, such as magnitude ratios, amplitude ratios and power ratios.

The metadata encoder/quantizer <NUM> may also comprise a direction encoder. The direction encoder is configured to receive direction parameters (such as the azimuth ϕ and elevation θ )(and in some embodiments an expected bit allocation) and from this generate a suitable encoded output. In some embodiments the encoding is based on an arrangement of spheres forming a spherical grid arranged in rings on a 'surface' sphere which are defined by a look up table defined by the determined quantization resolution. In other words, the spherical grid uses the idea of covering a sphere with smaller spheres and considering the centres of the smaller spheres as points defining a grid of almost equidistant directions. The smaller spheres therefore define cones or solid angles about the centre point which can be indexed according to any suitable indexing algorithm. Although spherical quantization is described here any suitable quantization, linear or non-linear may be used.

Similarly, the metadata encoder/quantizer <NUM> may also comprise a coherence encoder which is configured to receive the surround coherence values γ and spread coherence values ζ and determine a suitable encoding for compressing the surround and spread coherence values.

The encoded direction, energy ratios and coherence values may be passed to a combiner. The combiner may be configured to receive the encoded (or quantized/compressed) directional parameters, energy ratio parameters and coherence parameters and combine these to generate a suitable output (for example a metadata bit stream which may be combined with the transport signal or be separately transmitted or stored from the transport signal).

In some embodiments the encoded datastream is passed to the decoder/demultiplexer <NUM>. The decoder/demultiplexer <NUM> demultiplexes the encoded the quantized spatial audio parameter sets for the frame and passes them to the metadata extractor <NUM> and also the decoder/demultiplexer <NUM> may in some embodiments extract the transport audio signals to the transport extractor for decoding and extracting.

In embodiments the metadata extractor <NUM> may be arranged to extract the indices for <MAT> and r̂diff(k, n) for each TF tile.

The index associated with r̂díff(k, n) can be read to give the corresponding quantized value.

The value of r̂díff(k, n) may then be used to determine the particular quantizer (or quantisation table) (from a plurality of quantizers) which can be used at the decoder to dequantize the value of dr(k, n). In other words, r̂diff(k, n) is used to select the quantization table (from a plurality of quantization tables) at the decoder. The value of <MAT> may then be read from the selected quantization table by using the index associated with <MAT>. The values of the direct-to-total energy ratios may then be determined by using the reverse process to that applied at the encoder. From the above example the quantized values of r<NUM>(k, n), r<NUM>(k, n) can be obtained as: <MAT> and<MAT>.

The decoded spatial audio parameters may then form the decoded metadata output from the metadata extractor <NUM> and passed to the synthesis processor <NUM> in order to form the multi-channel signals <NUM>.

With respect to <FIG> an example electronic device which may be used as the analysis or synthesis device is shown. The device may be any suitable electronics device or apparatus. For example, in some embodiments the device <NUM> is a mobile device, user equipment, tablet computer, computer, audio playback apparatus, etc..

In some embodiments the device <NUM> comprises a memory <NUM>. In some embodiments the at least one processor <NUM> is coupled to the memory <NUM>. The memory <NUM> can be any suitable storage means. In some embodiments the memory <NUM> comprises a program code section for storing program codes implementable upon the processor <NUM>. Furthermore, in some embodiments the memory <NUM> can further comprise a stored data section for storing data, for example data that has been processed or to be processed in accordance with the embodiments as described herein. The implemented program code stored within the program code section and the data stored within the stored data section can be retrieved by the processor <NUM> whenever needed via the memory-processor coupling.

In some embodiments the device <NUM> comprises a user interface <NUM>. The user interface <NUM> can be coupled in some embodiments to the processor <NUM>. In some embodiments the processor <NUM> can control the operation of the user interface <NUM> and receive inputs from the user interface <NUM>. In some embodiments the user interface <NUM> can enable a user to input commands to the device <NUM>, for example via a keypad. In some embodiments the user interface <NUM> can enable the user to obtain information from the device <NUM>. For example, the user interface <NUM> may comprise a display configured to display information from the device <NUM> to the user. The user interface <NUM> can in some embodiments comprise a touch screen or touch interface capable of both enabling information to be entered to the device <NUM> and further displaying information to the user of the device <NUM>. In some embodiments the user interface <NUM> may be the user interface for communicating with the position determiner as described herein.

In some embodiments the device <NUM> comprises an input/output port <NUM>. The input/output port <NUM> in some embodiments comprises a transceiver. The transceiver in such embodiments can be coupled to the processor <NUM> and configured to enable a communication with other apparatus or electronic devices, for example via a wireless communications network. The transceiver or any suitable transceiver or transmitter and/or receiver means can in some embodiments be configured to communicate with other electronic devices or apparatus via a wire or wired coupling.

The transceiver can communicate with further apparatus by any suitable known communications protocol. For example in some embodiments the transceiver can use a suitable universal mobile telecommunications system (UMTS) protocol, a wireless local area network (WLAN) protocol such as for example IEEE <NUM>. X, a suitable short-range radio frequency communication protocol such as Bluetooth, or infrared data communication pathway (IRDA).

The transceiver input/output port <NUM> may be configured to receive the signals and in some embodiments determine the parameters as described herein by using the processor <NUM> executing suitable code. Furthermore, the device may generate a suitable downmix signal and parameter output to be transmitted to the synthesis device.

In some embodiments the device <NUM> may be employed as at least part of the synthesis device. As such the input/output port <NUM> may be configured to receive the downmix signals and in some embodiments the parameters determined at the capture device or processing device as described herein and generate a suitable audio signal format output by using the processor <NUM> executing suitable code. The input/output port <NUM> may be coupled to any suitable audio output for example to a multichannel speaker system and/or headphones or similar.

Programs can route conductors and locate components on a semiconductor chip using well established rules of design as well as libraries of pre-stored design modules. Once the design for a semiconductor circuit has been completed, the resultant design, in a standardized electronic format may be transmitted to a semiconductor fabrication facility or "fab" for fabrication.

Claim 1:
An apparatus for spatial audio encoding comprising means for:
converting two or more energy ratios associated with a time frequency tile of one or more audio signals to a further energy ratio parameter which is related to the two or more energy ratios;
quantizing the further energy ratio parameter using a first quantizer;
determining a distribution factor of energy ratios dependent on a ratio of a first of the two or more energy ratios to the sum of the two or more energy ratios;
selecting a further quantizer from a plurality of further quantizers using the quantized further energy ratio parameter; and
quantizing the distribution factor of energy ratios using the selected further quantizer.