Patent Description:
Parametric spatial audio processing is a field of audio signal processing where the spatial aspect of the sound is described using a set of parameters. For example, in parametric spatial audio capture from microphone arrays, it is a typical and an effective choice to estimate from the microphone array signals a set of parameters such as directions of the sound in frequency bands, and the ratios between the directional and non-directional parts of the captured sound in frequency bands. These parameters are known to well describe the perceptual spatial properties of the captured sound at the position of the microphone array. These parameters can be utilized in synthesis of the spatial sound accordingly, for headphones binaurally, for loudspeakers, or to other formats, such as Ambisonics.

"<NPL> describes a spatial parameters compression approach to decrease the bit rates of spatial parameters for 3D audio.

<CIT>, discloses an encoding and decoding method and system for improving three-dimensional audio spatial parameter compression ratio. The audio signal of a three-dimensional audio, the spatial side information of the three-dimensional audio and the number of the audio object of spatial parameters are input when encoding is carried out; when encoding is carried out, clustering, quantization, intra-frame encoding and inter-frame differential encoding are successively carried out on the spatial parameters; when decoding is carried out, inter-frame differential decoding, intra-frame decoding, inverse quantization and spatial parameter mapping are successively carried out. Based on the characteristic that different sub-band spatial parameters in the same sound source and the same frame have similarities, a spatial parameter clustering method is adopted to improve the compression ratio of the three-dimensional audio spatial parameters, and the compression ratio of the three-dimensional audio spatial parameters is high.

<CIT>, discloses a scalable audio coding system and method based on sub-band spatial attention measures, including comprehensively calculating and sorting the sub-band importance measures of each sub-band based on energy, frequency and spatial information, and sorting the bit according to the sorting results of the comprehensive measures. It uses energy, frequency and space information as a subband priority allocation strategy, which has more obvious guiding significance than simply using energy or frequency as a perception measure.

<CIT>, discloses a method based on the finding that parameters including a first set of parameters of a representation of a first portion of an original signal and including a second set of parameters of a representation of a second portion of the original signal can be efficiently encoded, when the parameters are arranged in a first sequence of tuples and in a second sequence of tuples, wherein the first sequence of tuples comprises tuples of parameters having two parameters from a single portion of the original signal and wherein the second sequence of tuples comprises tuples of parameters having one parameter from the first portion and one parameter from the second portion of the original signal. An efficient encoding can be achieved using a bit estimator to estimate the number of necessary bits to encode the first and the second sequence of tuples, wherein only the sequence of tuples is encoded, that results in the lower number of bits.

<CIT> discloses an apparatus comprising: a parameter determiner configured to determine for a frame of at least one audio signal a set of frame audio signal multi-channel parameters; a parameter selector configured to select for the frame a sub-set of the set of frame audio signal multi-channel parameters based on an error value associated with the sub-set of the set of frame audio signal multi-channel parameters; and parameter encoder configured to generate an encoded frame audio signal multi-channel parameter based on the selected sub-set of the set of frame audio signal multi-channel parameters.

<CIT> discloses a parametric encoder for encoding a multi-channel audio signal having a first audio signal and a second audio signal is provided. The parametric encoder has a transformer (<NUM>) for transforming the first audio signal into frequency domain to obtain a first transformed audio signal and for transforming the second audio signal into frequency domain to obtain a second transformed audio signal, a parameter generator (<NUM>) for generating a first encoding parameter from the first transformed audio signal and from the second transformed audio signal at a first frequency and for generating a second encoding parameter from the first transformed audio signal and from the second transformed audio signal at a second frequency, and a parameter combiner (<NUM>) for combining the first encoding parameter and the second encoding parameter to obtain a combined encoding parameter.

<CIT> describes an audio decoder provides a combination of decoding components including components implementing base band decoding, spectral peak decoding, frequency extension decoding and channel extension decoding techniques. The audio decoder decodes a compressed bitstream structured by a bitstream syntax scheme to permit the various decoding components to extract the appropriate parameters for their respective decoding technique.

"<NPL>, discloses perceptual compression methods for metadata in directional audio coding.

The invention provides an apparatus according to claim <NUM> and a method according to claim <NUM>.

The following describes in further detail suitable apparatus and possible mechanisms for the provision of effective spatial analysis derived metadata parameters associated with energy ratios for microphone array and other input format audio signals.

Apparatus has been designed to transmit a spatial audio modelling of a sound field using Q (which is typically <NUM>) transport audio signals and spatial metadata. The transport audio signals are typically compressed with a suitable audio encoding scheme (for example advanced audio coding - AAC or enhanced voice services - EVS codecs). The spatial metadata may contain parameters such as Direction (for example azimuth, elevation) in time-frequency domain.

Furthermore other parameters which may be determined and signalled to a renderer or receiver is one or more direct-to-total energy ratios (in the time-frequency domain) which represents the distribution of energy between each specific direction and the total audio energy. Another parameter may be one (or more where practical) diffuse-to-total energy ratio (in the time-frequency domain) which represents distribution of energy between ambient or diffuse signal (i.e., non-directional signal such as reverberation) and total energy.

The parametric spatial audio signals may be represented as Q channels + metadata. This format can be compressed in encoding to efficiently store it for later retrieval or transmit it over a suitable transmission channel. Various methods can be used depending on how the channels are configured and what the metadata contains.

A common procedure is to define a constant bitrate budget for the whole bitstream that contains audio channels and the metadata. This bitrate budget can then be divided statically or adaptively (dynamically) between audio channels and metadata.

For example, a bitrate budget of <NUM> kb/s for <NUM>-channels + metadata could be used in various ways. Using the full <NUM> kb/s for the <NUM> audio channels would offer very good quality for encoding the stereo signal (for example using an EVS codec), but in this example the metadata would not be transmitted. In using <NUM> kb/s for the audio and <NUM> kb/s for metadata would usually provide a higher overall quality as the difference in audio coding quality is not large but the signalled metadata can provide full 3d surround reproduction.

With lower bitrates, dividing the bitrate budget becomes even more difficult. For example with a <NUM> kb/s budget, there may be the following coding modes:.

Optimizing between these example modes may require listening experiments. However, previous experiments have shown that with such low bitrates offering more bitrate to the raw audio quality over multiple channels tends to offer better perceived quality. The effect of metadata bitrate budgeting is that reducing the metadata bitrate such that the audio signal receives at least <NUM>% of the total bitrate budget is believed to be a good target.

However the amount of metadata generated and therefore the amount of data defining spatial parameters is frequency band related. For example, for B (e.g., <NUM>, <NUM>, <NUM>, or <NUM>) frequency bands and two parameters (direction and energy ratio) for each time frame, there may be at minimum <NUM>*B*K (K is number of bits per parameter) bits of metadata per time frame. Assuming the common number of <NUM> frames per second, B=<NUM>, and K=<NUM> there may be <NUM> kb/s metadata generated. With low bitrate applications (such as IVAS) the total target bitrate with audio can be so low as <NUM> kb/s so the metadata would take a big portion of the bitrate budget even after entropy coding (which may reduce the bitrate to half of the generated total).

Currently, attempts to reduce the generated include reducing bit accuracy per parameter or even removing less important parameters when the bitrate budget is low. Another approach is to reduce the number of frequency bands for metadata, for example generating just one parameter per timeframe and thus producing a reduction of generated metadata by B. One method for achieving this is to perform a wideband analysis (in other words assume only one frequency band for the full audible frequency range) and encode this wideband group.

The concept as discussed herein attempts to improve on these methods and in particular, instead of wideband analysis, attempts to:.

Thus, the concept as discussed in further detail in the embodiments herein implements an analysis system with multiple bands and then selects the best frequency band to represent the current time frame.

The embodiments discussed herein therefore attempt to reduce the bitrate by selecting one frequency band from the analysed metadata to represent all frequency bands. This reduces bitrate usage by factor of B (where B is the original number of frequency bands). The selection process in some embodiments relates to audio encoding and may relate (in aspects not covered by the claims) to decoding using a sound-field related parametrization (e.g., direction(s) and direct-to-total energy ratio(s) in frequency bands) where a solution is provided for automatically reducing the bitrate of the direction parameters by transmitting only one direction value for all frequency bands and where the transmitted one direction value is determined by:.

This may be further expanded or detailed as analysis apparatus configured to:.

With respect to the synthesis apparatus it is then configured to:.

The directions and the direct-to-total energy ratios can be estimated using any suitable method (e.g., SPAC), and depends on the type of the audio signals (e.g., microphone-array, Ambisonics, multichannel audio signals).

The normalized energy can be estimated as discussed in the embodiments herein in a suitable manner. For example by computing the sum of squares of the frequency-domain samples and dividing with the largest energy.

The threshold value may in some embodiments be determined for example by multiplying the average normalized energy by a factor.

In addition to the direction, also all other parameters (e.g., direct-to-total energy ratios) may be encoded using the same scheme. In other words transmitting only one parameter value for all frequency bands. The value to be transmitted can be selected using the same procedure.

The decoding can be performed using any suitable method for example by using the same parameter value at all frequency bands.

In some embodiments, in encoding, the selected frequency band can be used as a reference band and a very low bitrate difference coding related to it determined for other bands.

With respect to <FIG> an example apparatus and system for implementing embodiments of the application are shown. The system <NUM> is shown with an 'analysis' part <NUM> and a 'synthesis' part <NUM>. The 'analysis' part <NUM> is the part from receiving the input (multichannel loudspeaker, microphone array, ambisonics, or mobile device capture) audio signals <NUM> up to an encoding of the metadata and transport signal <NUM> which may be transmitted or stored <NUM>. The 'synthesis' part <NUM> may be the part from a decoding of the encoded metadata and transport signal <NUM> to the presentation of the synthesized signal (for example in multi-channel loudspeaker form <NUM> via loudspeakers <NUM> or binaural or ambisonic formats).

The input to the system <NUM> and the 'analysis' part <NUM> is therefore audio signals <NUM>. These may be suitable input multichannel loudspeaker audio signals, microphone array audio signals, ambisonic audio signals, or mobile captured audio signals.

The input audio signals <NUM> may be passed to an analysis processor <NUM>. The analysis processor <NUM> may be configured to receive the input audio signals and generate a suitable data stream <NUM> comprising suitable transport signals. The transport audio signals may also be known as associated audio signals and be based on the audio signals. For example in some embodiments the transport signal generator <NUM> is configured to downmix or otherwise select or combine, for example, by beamforming techniques the input audio signals to a determined number of channels and output these as transport signals. In some embodiments the analysis processor is configured to generate a <NUM>-audio-channel output of the microphone array audio signals. The determined number of channels may be two or any suitable number of channels.

In some embodiments the analysis processor is configured to pass the received input audio signals <NUM> unprocessed to an encoder in the same manner as the transport signals. In some embodiments the analysis processor <NUM> is configured to select one or more of the microphone audio signals and output the selection as the transport signals <NUM>. In some embodiments the analysis processor <NUM> is configured to apply any suitable encoding or quantization to the transport audio signals.

In some embodiments the analysis processor <NUM> is also configured to analyse the input audio signals <NUM> to produce metadata associated with the input audio signals (and thus associated with the transport signals). The analysis processor <NUM> can, for example, be a computer (running suitable software stored on memory and on at least one processor), mobile device, or alternatively a specific device utilizing, for example, FPGAs or ASICs. As shown herein in further detail the metadata may comprise, for each time-frequency analysis interval, at least one direction parameter and at least one energy ratio parameter. The at least one direction parameter and the at least one energy ratio parameter may in some embodiments be considered to be spatial audio parameters. In other words the spatial audio parameters comprise parameters which aim to characterize the sound-field of the input audio signals.

In some embodiments the parameters generated may differ from frequency band to frequency band and may be dependent on the transmission bit rate. Thus for example in band X all of the parameters are generated and transmitted, whereas in band Y only one of the parameters is generated and transmitted, and furthermore in band Z any other number of parameters are generated or transmitted. A practical example of this may be that for some frequency bands such as the highest band some of the parameters are not required for perceptual reasons.

The transport signals and the metadata <NUM> may be transmitted or stored, this is shown in <FIG> by the dashed line <NUM>. Before the transport signals and the metadata are transmitted or stored they may in some embodiments be coded in order to reduce bit rate, and multiplexed to one stream. The encoding and the multiplexing may be implemented using any suitable scheme.

In the decoder side <NUM>, the received or retrieved data (stream) may be input to a synthesis processor <NUM>. The synthesis processor <NUM> may be configured to demultiplex the data (stream) to coded transport and metadata. The synthesis processor <NUM> may then decode any encoded streams in order to obtain the transport signals and the metadata.

The synthesis processor <NUM> may then be configured to receive the transport signals and the metadata and create a suitable multi-channel audio signal output <NUM> (which may be any suitable output format such as binaural, multi-channel loudspeaker or Ambisonics signals, depending on the use case) based on the transport signals and the metadata. In some embodiments with headphone or loudspeaker reproduction, an actual physical sound field is reproduced (using the output device <NUM> for example loudspeakers/headphones etc) having the desired perceptual properties. In other embodiments, the reproduction of a sound field may be understood to refer to reproducing perceptual properties of a sound field by other means than reproducing an actual physical sound field in a space. For example, the desired perceptual properties of a sound field can be reproduced over headphones using the binaural reproduction methods as described herein. In another example, the perceptual properties of a sound field could be reproduced as an Ambisonic output signal, and these Ambisonic signals can be reproduced with Ambisonic decoding methods to provide for example a binaural output with the desired perceptual properties.

The synthesis processor <NUM> can in some embodiments be a computer (running suitable software stored on memory and on at least one processor), mobile device, or alternatively a specific device utilizing, for example, FPGAs or ASICs.

With respect to <FIG> an example flow diagram of the overview shown in <FIG> is shown.

First the system (analysis part) is configured to receive input audio signals or suitable multichannel input as shown in <FIG> by step <NUM>.

Then the system (analysis part) is configured to generate a transport signal channels or transport signals (for example downmix/selection/beamforming based on the multichannel input audio signals) as shown in <FIG> by step <NUM>.

Also the system (analysis part) is configured to analyse the audio signals to generate metadata: Directions; Energy ratios as shown in <FIG> by step <NUM>.

The system is then configured to (optionally) encode for storage/transmission the transport signals and metadata as shown in <FIG> by step <NUM>.

After this the system may store/transmit the transport signals and metadata as shown in <FIG> by step <NUM>.

The system may retrieve/receive the transport signals and metadata as shown in <FIG> by step <NUM>.

Then the system is configured to extract from the transport signals and metadata as shown in <FIG> by step <NUM>.

The system (synthesis part) is configured to synthesize an output spatial audio signals (which as discussed earlier may be any suitable output format such as binaural, multi-channel loudspeaker or Ambisonics signals, depending on the use case) based on extracted audio signals and metadata as shown in <FIG> by step <NUM>.

With respect to <FIG> an example analysis processor <NUM> is shown where the input audio signal is provided from an audio source <NUM> which in this example is a spatial capture device configured to generate multichannel audio signals from multiple microphones. The multichannel audio signals in this example are passed to a transport (audio) signal generator <NUM>. The transport signal generator <NUM> is configured to generate the transport audio signals according to any of the options described previously. For example the transport signals may be downmixed from the input signals. The number of the transport audio signals may be any number and may be <NUM> or more or fewer than <NUM>.

In the example shown in <FIG> the multichannel audio signals are also input to a time frequency transform <NUM>. The time frequency transform <NUM> may be configured to generate suitable time-frequency representations of the multichannel audio signals and pass these to a frequency band processor <NUM>.

The frequency band processor <NUM> is configured to generate spatial metadata outputs such as shown as the directions, direct-to-total energy ratios, and in some embodiments other types of energy ratios such as diffuse-to-total energy ratio(s) and remainder-to-total energy ratio(s).

The implementation of the analysis may be any suitable implementation that produces the described metadata outputs.

Thus for example in some embodiments the frequency band processor <NUM> comprises a direction analyser <NUM> configured to generate the direction metadata and an energy ratio analyser <NUM> configured to generate the energy ratio metadata.

The direction and energy ratio metadata for all of the analysed frequency bands may then be passed to a transmission/storage encoder <NUM>. The transmission/storage encoder <NUM> may be configured to combine and encode the transport signals, the directions, and the energy ratios to generate the data stream <NUM>.

For example in some embodiments the transmission/storage encoder <NUM> may comprise a suitable transport signal compressor/encoder configured to compress the audio signals using a suitable codec (e.g., AAC or EVS).

With respect to <FIG> is shown a flow diagram of the operation of the analysis processor.

The first operation is one of receiving the (multichannel loudspeaker or other) audio signals as shown in <FIG> by step <NUM>.

In some embodiments the audio signals are processed in some form to generate the transport audio signals as shown in <FIG> by step <NUM>.

The following operation may be one of spatially analysing the (multichannel loudspeaker) signals in order to determine direction metadata as shown in <FIG> by step <NUM>.

Then the energy ratios (for example the direct, diffuse and remainder energy ratios) are determined as shown in <FIG> by step <NUM>.

In some embodiments the metadata and transport audio signals are processed (compressed/encoded). For example the number of the directions and ratios are furthermore controlled (and may be selected and/or combined). The processing of the metadata/transport audio signals is shown in <FIG> by step <NUM>.

The processed transport audio signals and the metadata may then be furthermore be combined to generate a suitable data stream as shown in <FIG> by step <NUM>.

With respect to <FIG> there is shown an example analysis processor <NUM> suitable for implementing some embodiments with additions over the example provided in <FIG>.

The example analysis processor <NUM> is shown again with the input audio signal provided from an audio source <NUM> which also in this example is a spatial capture device configured to generate multichannel audio signals from multiple microphones. For example capturing a spatial audio signal can be performed with any known capture device. For example an Eigenmike or Nokia <NUM> mobile phone are suitable. As described previously the multichannel (spatial) audio signal may be any format such as mixed content (e.g., a multichannel audio format such as <NUM>) and Ambisonics content that may produce the relevant spatial audio parameters.

The multichannel audio signals in this example are passed to a transport (audio) signal generator <NUM>.

The transport signal generator <NUM> similar to the example in <FIG> is configured to generate the transport audio signals according to any of the options described previously. For example the transport signals may be downmixed from the input signals. The number of the transport audio signals may be any number and may be <NUM> or more or fewer than <NUM>.

These may be determined by performing spatial analysis on the time-frequency transformed multichannel audio signal.

An example of spatial analysis may be for example DirAC (Directional Audio Coding) spatial analysis.

DirAC may estimate the directions and diffuseness ratios (equivalent information to a direct-to-total ratio parameter) from a first-order Ambisonic (FOA) signal, or its variant the B-format signal.

The signals of <MAT> are transformed into frequency bands for example by STFT, resulting in time-frequency signals w(k,n), x(k,n), y(k,n), z(k,n) where k is the frequency bin index and n is the time index. DirAC estimates the intensity vector by <MAT> where Re means real part, and asterisk * means complex conjugate.

The direction parameter is opposite of the direction of the real part of the intensity vector. The intensity vector may be averaged over several time and/or frequency indices prior to the determination of the direction parameter.

DirAC determines the diffuseness as <MAT>.

Diffuseness is a ratio value that is <NUM> when the sound is fully ambient, and <NUM> when the sound is fully directional. Again, all parameters in the equation are typically averaged over time and/or frequency. The expectation operator E[ ] can be replaced with an average operator in practical systems.

When averaged, the diffuseness (and direction) parameters typically are determined in frequency bands combining several frequency bins k, for example, approximating the Bark frequency resolution.

DirAC, as determined above, is only one of the options to determine the directional and ratio metadata, and clearly one may utilize other methods to determine the metadata, for example, using a spatial audio capture (SPAC) algorithm with microphone-array signals (real or simulated). Furthermore, there are also many variants of DirAC analysis in the literature. For example where the input content is not FOA, a suitable modification can be done to convert the signal into FOA-format to perform analysis. Other analysis methods are also applicable as long as they produce the directional and energy ratio metadata.

The direction and energy ratio metadata for all of the analysed frequency bands are then passed to a metadata selector <NUM>.

Furthermore the output of the energy ratio analyser <NUM> is output to a weight factor determiner <NUM>.

Furthermore the frequency band processor <NUM> comprises a normalised energy determiner <NUM> configured to generate a normalised energy determination and pass this to a weight factor determiner <NUM> and to a weight limit determiner <NUM>.

In some embodiments the normalised energy determination may be performed as a two step operation. A first step being to calculate the average energy for each frequency band in this time instant for example with the following equation: <MAT> where N is number of time samples in this time frame, Kb and Kt are the current frequency band bottom and top frequency bins, and I is the number of input channels in the signal. S(i,k,n) is the time-frequency domain representation of the transport signal.

The second step may be to normalize the average energies of each frequency band so that the largest energy of any frequency band is found and then divide all energies with the largest energy value. This may be seen as the largest energy of a frequency band is (always) <NUM> and other frequency bands have less energy or represented as an equation as: <MAT>.

In some embodiments any suitable alternative normalization methods may be employed (e.g., normalizing with total energy instead of largest energy) and can be used but the limit parameter (as discussed hereafter) is appropriately tuned. In addition, in some embodiments unnormalized energy may be employed but the limit parameter requires even more careful tuning.

The frequency band processor <NUM> in some embodiments further comprises a weight factor determiner <NUM> configured to receive the normalised energy and the energy ratios and determine at least one weighting factor which is output to the metadata selector <NUM>.

With the normalized energy known, the weight factor may be determined by based on the product of energy ratio and the normalized energy in the frequency band. The weight factor may therefore be determined by the equation: <MAT> where r is the energy ratio parameter.

This weight factor is a number between <NUM> and <NUM>. It will be a very high value when there is a directional impulsive onset present in the scene as both energy ratio and normalized energy will be high. Likewise, if there is no onset present, these values tend to be lower for higher frequencies. The use of the product ensures that, for example, high normalized energy but low energy ratio (i.e., loud reverberation) does not produce high weight values as the direction and the metadata in this case is not the best representative.

In some embodiments, this weight factor can be any other suitable weight factor such as only the energy ratio parameter r.

The analysis processor <NUM> in some embodiments comprises a weight limit determiner <NUM> configured to receive the normalised energy determination and output a weight limit value to the metadata selector <NUM>.

The weight limit can be a constant value (e.g., <NUM>) or it can be based on the average normalized energy of all frequency bands in the time frame (e.g., average normalized energy multiplied with a constant like <NUM>). The latter option is preferred and is formed as: <MAT> where c is tuned threshold constant such as <NUM> and B is the total number of frequency bands.

In some embodiments, this weight limit can be any other suitable value.

The analysis processor <NUM> in some embodiments comprises a metadata selector <NUM> configured to receive the output of the direction analyser <NUM> (direction metadata for each band), energy ratio analyser <NUM> (energy ratio metadata for each band), weight factor determiner <NUM> (weight factors) and weight limit determiner <NUM>. The metadata selector <NUM> is then configured to select one of the directions and energy ratios based on the weight factor and weight factor limit and pass the selected metadata to a transmission/storage encoder <NUM>.

The metadata selector may be configured to choose or select the highest frequency band that has a weight factor over the weight limit. If for some reason no band has weight over the limit, the metadata selector in some embodiments is configured to select the lowest frequency band.

In some embodiments once the metadata selector determines the selected frequency band, it may be configured to discard metadata associated with the other bands.

In some embodiments the metadata selector is configured to prioritize and only discard part of the metadata. For example, in some embodiments the direction information for the other bands are discarded but the energy ratio parameters are kept for all frequency bands.

In some embodiments, two or more frequency bands (but fewer than the total number of frequency bands) are selected to represent the other frequency bands. For example, two frequency bands can be selected such that two (or N where N is less than the total number of frequency bands) highest frequency bands with weights over the threshold (or weight limit) are selected. The parameters associated with the selected higher frequency band is then used to represent parameters for frequency bands above it, and parameters associated with the lower frequency band is used to represent parameters for frequency bands below it, and both are used to represent frequency bands between them.

In some embodiments the 'best' frequency band is selected but a difference coding technique is employed to represent the other frequency bands.

In some embodiments, a few bits are used to signal which frequency band is the reference band for the difference coding. Using this method still significantly reduces the bitrate but offers more accurate representation.

In some embodiments the highest frequency band is selected and the metadata associated with the highest frequency band is used to 'represent' all frequency bands. This is less optimal in quality but is computationally more efficient to implement.

The analysis processor <NUM> may further comprise a transmission/storage encoder <NUM>. The transmission/storage encoder <NUM> may be configured to combine and encode the transport signals, the selected direction, and the energy ratio to generate the data stream <NUM>.

For example in some embodiments the transmission/storage encoder <NUM> may comprise a suitable transport signal compressor/encoder configured to compress the audio signals using a suitable codec (e.g., AAC or EVS) and encoding metadata using entropy coding methods (e.g., codebook coding).

With respect to <FIG> is shown a flow diagram of the operation of the analysis processor shown in <FIG> (and additionally the synthesis processor shown in <FIG>).

The first operation is one of obtaining the (multichannel loudspeaker or other) audio signals as shown in <FIG> by step <NUM>.

The audio signals may be processed by the application of a time-frequency transform as shown in <FIG> by step <NUM>.

In some embodiments the time-frequency domain audio signals are processed in some form to generate the transport signals as shown in <FIG> by step <NUM>.

Furthermore in some embodiments the time-frequency domain audio signals are processed and spatial analysis performed to determine parameters such as direction(s) (and/or distance) and energy ratio(s) for each band as shown in <FIG> by step <NUM>.

Additionally in some embodiments the time-frequency domain audio signals are processed and a normalised energy per band calculated as shown in <FIG> by step <NUM>.

Having determined the normalised energy per band and spatial analysis then in some embodiments the weight factor per band is formed or determined as shown in <FIG> by step <NUM>.

Also having determined the normalised energy per band in some embodiments the weight factor limit is formed or determined as shown in <FIG> by step <NUM>.

Based on the weight factor per band and the weight factor limit a highest band with a weight over the limit is chosen as shown in <FIG> by step <NUM>.

The other metadata is then discarded and the chosen band metadata saved as shown in <FIG> by step <NUM>.

The selected metadata and transport signals are then compressed/encoded (and combined) before being stored and/or transmitted as shown in <FIG> by step <NUM>.

With respect to the synthesis processor operations the transmitted/retrieved signal is decoded and metadata replicated for all frequency bands as shown in <FIG> by step <NUM>.

Then a suitable spatial synthesis is performed as shown in <FIG> by step <NUM>.

As described previously the audio signal input format may be any suitable format. For example with respect to <FIG> is shown a flow diagram of the operation of an encoder suitable to encoding an obtained transport audio signal and metadata. In such an embodiment the frequency band processor may comprise only the normalised energy determiner and weight factor determiner as the direction and energy ratios have been determined.

The first operation is one of obtaining the transport audio signals and metadata as shown in <FIG> by step <NUM>.

In this example the parameters such as direction(s) (and/or distance) and energy ratio(s) for each band have been obtained and a normalised energy per band calculated as shown in <FIG> by step <NUM>.

With respect to <FIG> an example operation of the metadata selector is shown in further detail. The first operation is to start and receive the inputs such as weight factors, weight limits, and parameters as shown in <FIG> by step801.

The next operation is setting an index i=B as shown in <FIG> by step <NUM>.

The next operation is testing the index weight factor wi against the weight limit wthr as shown in <FIG> by step <NUM>.

If wi>wthr then the next operation is determining i is the selected frequency band as shown in <FIG> by step <NUM> and then ending the operation as shown in <FIG> by step <NUM>.

If wi is not > wthr then the next operation is decrementing i by <NUM> as shown in <FIG> by step <NUM>.

Having decremented i by <NUM> then the next operation is checking whether i=<NUM> as shown in <FIG> by step <NUM>.

Where i=<NUM> then the next operation is determining i is the selected frequency band as shown in <FIG> by step <NUM> and then ending the operation as shown in <FIG> by step <NUM>.

Where i is not =<NUM> then the operation may then test the new index, index weight factor wi against the weight limit wthr as shown in <FIG> by step <NUM> and the process may continue until wi>wthr, for the index or the index =<NUM>.

The above assumes that frequency band indexing starts from <NUM>. The above can be modified to accommodate any other indexing system (such as starting from <NUM>).

With respect to the synthesis processor the single band metadata values may be obtained and then replicated for all frequency bands. This results in a normal full set of metadata that can be used in further synthesis.

The synthesis operation may then use the transport signals and replicated metadata to generate a suitable rendering of the audio signals. This procedure can be performed using any suitable means, for example, with methods such as DirAC based spatial audio signal synthesis. An example procedure for synthesising audio signals for loudspeakers is that the directions are synthesized into specific directions using 3D panning techniques such as vector-base amplitude panning (VBAP) multiplied with <MAT>, and non-directional ambient signal is decorrelated with a phase-scrambling filter and reproduced to all directions multiplied with <MAT>, where r is the energy ratio parameter and C is the number of loudspeaker channels.

In such a manner some embodiments may be implemented which reduce bitrate usage while offering quality that is in many cases at least reasonable and can be almost transparent (and in many cases is) to full metadata transmission with many signals. Bitrate reduction with the primary method is by factor of B, where B is the original number of frequency bands. , if original metadata bitrate is <NUM> kb/s and B=<NUM> then this method achieves bitrate of <NUM> kb/s.

Furthermore such embodiments may be able to produce a signal which is at least as good or better than using single wideband parametric analysis.

Additionally in some embodiments the implementation is computationally efficient method to reduce bitrate as it only requires a determination of the energies (this is often part of the analysis already) and weight factors and then discard data.

In some embodiments spatial sound transmission storage can be achieved even at very low bitrates.

For example a teleconference system may use a parametric spatial audio, e.g., DirAC, as the main analysis and synthesis method. Spatial capture may be obtained with an Eigenmike that produces first-order Ambisonics for this use. The spatial audio is analysed in time-frequency (<NUM> frame and <NUM> frequency bands) domain and produces direction parameters as azimuth and elevation, and energy ratio parameter in form of diffuseness. Rather than encoding these parameters using a determined number of bits per parameter, i.e., <NUM> bits, to produce metadata at a bitrate of <NUM> kb/s (before other compression) the application of some embodiments may result in a bitrate of just <NUM> kb/s for the metadata (before other compression). This leaves more bits to use for the coding of the audio signal which directly results in better perceived audio quality.

A further example would be using time-frequency resolution such as <NUM> time frame and <NUM> frequency bands would result in following comparison bitrates. <NUM> kb/s compared to <NUM> kb/s according to some embodiments.

As the reduction in bitrate of metadata is quite large, it especially benefits the use case where the bitrate budget is very low. For example, <NUM> kb/s is usually in the domain of mono downmix or very compressed stereo if only raw audio encoding is used. If spatial metadata is introduced using, for example, the second time-frequency resolution above, the full spatial metadata would be hard to fit to the bitrate budget even after expected <NUM>% entropy coding for it (metadata would take <NUM> kb/s of <NUM> kb/s available). However, using the presented embodiments it may be possible to reduce the metadata down to a fifth and in this case we achieve very reasonable division of bitrate after entropy coding (<NUM> kb/s for metadata, <NUM> kb/s for audio) thus offering full spatial audio even at low bitrates instead of mono or stereo. This means that at low bitrates, it may be possible to achieve a significant sound quality increase compared to sending full metadata.

With respect to <FIG> an example electronic device which may be used as the analysis or synthesis device is shown. The device may be any suitable electronics device or apparatus. For example in some embodiments the device <NUM> is a mobile device, user equipment, tablet computer, computer, audio playback apparatus, etc..

In some embodiments the device <NUM> comprises a memory <NUM>. In some embodiments the at least one processor <NUM> is coupled to the memory <NUM>. The memory <NUM> can be any suitable storage means. In some embodiments the memory <NUM> comprises a program code section for storing program codes implementable upon the processor <NUM>. Furthermore in some embodiments the memory <NUM> can further comprise a stored data section for storing data, for example data that has been processed or to be processed in accordance with the embodiments as described herein. The implemented program code stored within the program code section and the data stored within the stored data section can be retrieved by the processor <NUM> whenever needed via the memory-processor coupling.

In some embodiments the device <NUM> comprises a user interface <NUM>. The user interface <NUM> can be coupled in some embodiments to the processor <NUM>. In some embodiments the processor <NUM> can control the operation of the user interface <NUM> and receive inputs from the user interface <NUM>. In some embodiments the user interface <NUM> can enable a user to input commands to the device <NUM>, for example via a keypad. In some embodiments the user interface <NUM> can enable the user to obtain information from the device <NUM>. For example the user interface <NUM> may comprise a display configured to display information from the device <NUM> to the user. The user interface <NUM> can in some embodiments comprise a touch screen or touch interface capable of both enabling information to be entered to the device <NUM> and further displaying information to the user of the device <NUM>.

In some embodiments the device <NUM> comprises an input/output port <NUM>. The input/output port <NUM> in some embodiments comprises a transceiver. The transceiver in such embodiments can be coupled to the processor <NUM> and configured to enable a communication with other apparatus or electronic devices, for example via a wireless communications network. The transceiver or any suitable transceiver or transmitter and/or receiver means can in some embodiments be configured to communicate with other electronic devices or apparatus via a wire or wired coupling.

The transceiver can communicate with further apparatus by any suitable known communications protocol. For example in some embodiments the transceiver or transceiver means can use a suitable universal mobile telecommunications system (UMTS) protocol, a wireless local area network (WLAN) protocol such as for example IEEE <NUM>. X, a suitable short-range radio frequency communication protocol such as Bluetooth, or infrared data communication pathway (IRDA).

The transceiver input/output port <NUM> may be configured to receive the loudspeaker signals (or other input format audio signals) and in some embodiments determine the parameters as described herein by using the processor <NUM> executing suitable code. Furthermore the device may generate a suitable transport signal and parameter output to be transmitted to the synthesis device.

In some embodiments the device <NUM> may be employed as at least part of the synthesis device. As such the input/output port <NUM> may be configured to receive the transport signals and in some embodiments the parameters determined at the capture device or processing device as described herein, and generate a suitable audio signal format output by using the processor <NUM> executing suitable code. The input/output port <NUM> may be coupled to any suitable audio output for example to a multichannel speaker system and/or headphones or similar.

of Mountain View, California and Cadence Design, of San Jose, California automatically route conductors and locate components on a semiconductor chip using well established rules of design as well as libraries of pre-stored design modules.

Claim 1:
An apparatus comprising means configured for:
obtaining (<NUM>) at least one audio signal;
obtaining (<NUM>) at least one parameter for respective at least two frequency bands associated with the at least one audio signal;
the apparatus is characterized in that the means are further configured for:
selecting a frequency band of the at least two frequency bands based on comparing at least one further parameter for respective the at least two frequency bands wherein the at least one further parameter is determined from respective at least two frequency bands, wherein the means for selecting the frequency band of the at least two frequency bands is for:
selecting the at least one parameter for the selected frequency band of the at least two frequency bands; and
discarding any other of the at least one parameter for the at least two frequency bands; and
generating (<NUM>) an output comprising a selection of the at least one parameter associated with the selected frequency band of the at least two frequency bands, wherein the output comprises the selected at least one parameter for the selected frequency band of the at least two frequency bands and not the discarded other of the at least one parameter for the at least two frequency bands, such that the selection of the at least one parameter associated with the selected frequency band is configured to reduce a bitrate or size of the output and wherein the at least one parameter of the selected frequency band is configured to represent respective parameters of the at least two frequency bands.