Patent Description:
This document discusses methods on discontinuous transmission mode (DTX) and comfort noise generation (CNG) for audio scenes for which the spatial image was parametrically coded by the directional audio coding (DirAC) paradigm or transmitted in Metadata-Assisted Spatial Audio (MASA) format.

Embodiments relate to Discontinuous Transmission of Parametrically Coded Spatial Audio such as a DTX mode for DirAC and MASA.

Embodiments of the present invention are about efficiently transmitting and rendering conversational speech e.g. captured with soundfield microphones. The thus captured audio signal is in general called three-dimension (3D) audio, since sound events can be localized in the three dimensional space, which reinforces the immersivity and increases both intelligibility and user experience.

Transmitting an audio scene e.g. in three dimensions requires handling multiple channels which usually engenders a large amount of data to transmit. For example Directional Audio Coding (DirAC) technique [<NUM>] can be used for reducing the large original data rate. DirAC is considered an efficient approach for analyzing the audio scene and representing it parametrically. It is perceptually motivated and represents the sound field with the help of a direction of arrival (DOA) and diffuseness measured per frequency band. It is built upon the assumption that at one time instant and for one critical band, the spatial resolution of the auditory system is limited to decoding one cue for direction and another for inter-aural coherence. The spatial sound is then reproduced in frequency domain by cross-fading two streams: a non-directional diffuse stream and a directional non-diffuse stream.

Moreover, in a typical conversation, each speaker is silent for about sixty percent of the time. By distinguishing frames of the audio signal that contain speech ("active frames") from frames containing only background noise or silence ("inactive frames"), speech coders can save significant data rate. Inactive frames are typically perceived as carrying little or no information, and speech coders are usually configured to reduce their bit-rate for such frames, or even transmitting no information. In such case, coders run in so-called Discontinuous Transmission (DTX) mode, which is an efficient way to drastically reduce the transmission rate of a communication codec in the absence of voice input. In this mode, most frames that are determined to consist of background noise only are dropped from transmission and replaced by some Comfort Noise Generation (CNG) in the decoder. For these frames, a very low-rate parametric representation of the signal is conveyed by Silence Insertion Descriptor (SID) frames sent regularly but not at every frame. This allows the CNG in the decoder to produce an artificial noise resembling the actual background noise.

Embodiments of the present invention relate to a DTX system and especially an SID and CNG for 3D audio scenes, captured for example by a soundfield microphone and which may be coded parametrically by a coding scheme based on the DirAC paradigm and alike. Present invention allows drastic reduction of the bit-rate demand for transmitting conversational immersive speech.

<CIT> is concerned with stereo DTX encoding and decoding. The procedure switches between a stereo coder and a DTX stereo coder. When using the DTX encoder, data can be saved by not transmitting the data for all the frames.

<CIT> discloses how to implement discontinuous encoding of a downmixed signal and stereo parameter of a multichannel audio.

<CIT> proposes to encode an averaged side gain parameter for an inactive segment and a downmix plus side gain parameters for an active segment in the context of generating a comfort noise parameter.

Useful embodiments are defined in the depend claims.

In particular, the invention relates with an apparatus for generating an encoded audio scene from an audio signal having a first frame and a second frame, comprising:.

At first, some discussion of known paradigms (DTX, DirAC, MASA, etc.) is provided, with the description of techniques some of which may be, at least in some cases, implemented in examples of the invention.

Immersive speech communication is a new domain of research and very few systems exist, moreover no DTX systems were designed for such application.

However, it can be straightforward to combine existing solutions. One can for example apply independently DTX on each individual multi-channel signal. This simplistic approach faces several problems. For this, one needs to transmit discretely each individual channel which is incompatible with the low bit-rate communication constraints and therefore hardly compatible with DTX, which is designed for low bit-rate communication cases. Moreover it is then required to synchronize the VAD decision across the channels to avoid oddities and unmasking effects and also to fully exploit the bit-rate reduction of the DTX system. Indeed for interrupting the transmission and profit from it, one needs to make sure that Voice Activity Decisions are synchronized across all channels.

Another problem arises on the receiver side, when generating the missing background noise during inactive frames by the comfort noise generator(s). For immersive communications, especially when directly applying DTX to individual channels, one generator per channel is required. If these generators, which typically sample a random noise, are used independently, the coherence between channels will be zero or close to zero and may deviate perceptually from the original sound-scape. On the other hand, if only one generator is used and the resulting comfort noise copied to all output channels, the coherence will be very high and immersivity will be drastically reduced.

These problems can be partially solved by applying DTX not directly to the input or output channels of the system, but instead after a parametric spatial audio coding scheme, like DirAC, on the resulting transport channels, which are usually a downmixed or reduced version of the original multi-channel signal. In this case, it is necessary to define how inactive frames are parameterized and then spatialized by the DTX system. This is not trivial and is the subject of embodiments of the present invention. The spatial image must be consistent between active and inactive frames, and must be as faithful perceptually as possible to the original background noise.

<FIG> shows an encoder <NUM> according to an example of the claimed invention. The encoder <NUM> generates an encoded audio scene <NUM> from an audio signal <NUM>.

The audio signal <NUM> or the audio scene <NUM> (and also other audio signals disclosed below) may be divided into frames (e.g. it may be a sequence of frames). The frames may be associated to time slots, which may be defined subsequently one with another (in some examples, a preceding aspect may overlap with a subsequent frame). For each frame, values in the time domain (TD) or frequency domain (FD) may be written in the bitstream <NUM>. In TD, values may be provided for each sample (each frame having e.g. a discrete a sequence of samples). In FD, values may be provided for each frequency bin. As will be explained later, each frame may be classified (e.g. by an activity detector) either as an active frame <NUM> (e.g., non-void frame) or inactive frame <NUM> (e.g., void frames, or silence frames, or only-noise frames). Different parameters (e.g. active spatial parameters <NUM> or inactive spatial parameters <NUM>) may also be provided in association to the active frame <NUM> and inactive frame <NUM> (in case of no data, reference numeral <NUM> shows that no data is provided).

The audio signal <NUM> may be, for example, a multi-channel audio signal (e.g. with two channels or more). The audio signal <NUM> may be, for example, a stereo audio signal. The audio signal <NUM> may be, for example, an Ambisonics signal, e.g., in A-format or B-format. The audio signal <NUM> may have, for example, a MASA (metadata assisted spatial audio) format. The audio signal <NUM> may have an input format being a first order Ambisonics format, a higher order Ambisonics format, a multi-channel format associated with a given loudspeaker setup, such as <NUM> or <NUM> or <NUM> + <NUM>, or one or more audio channels representing one or several different audio objects localized in a space as indicated by information included in associated metadata, or an input format being a metadata associated spatial audio representation. The audio signal <NUM> may comprise a microphone signal as picked up by real microphones or virtual microphones. The audio signal <NUM> may comprise a synthetically created microphone signal (e.g. being in a first order Ambisonics format, or a higher order Ambisonics format).

The audio scene <NUM> comprises at least one or a combination of:.

Active frames <NUM> (first frames) may be those frames that contain speech (or, in some examples, also other audio sounds different from pure noise). Inactive frames <NUM> (second frames) may be understood as being those frames that do not comprise speech (or, in some examples, also other audio sounds different from pure noise) and may be understood as containing uniquely noise.

An audio scene analyzer (soundfield parameter generator) <NUM> is provided, for example, to generate a transport channel version <NUM> (subdivided among <NUM> and <NUM>) of the audio signal <NUM>. Here, we may refer to transport channel(s) <NUM> of each first frame <NUM> and/or transport channel(s) <NUM> of each second frame <NUM> (transport channel(s) <NUM> may be understood as providing a parametric description of silence or noise, for example). The transport channel(s) <NUM> (<NUM>, <NUM>) may be a downmix version of the input format <NUM>. In general terms, each of the transport channels <NUM>, <NUM> may be, for example, one single channel if the input audio signal <NUM> is a stereo channel. If the input audio signal <NUM> has more than two channels, the downmix version <NUM> of the input audio signal <NUM> may have less channels than the input audio signal <NUM>, but still more than one channel in some examples (e.g., if the input audio signal <NUM> has four channels, the downmix version <NUM> may have one, two, or three channels).

The audio signal analyzer <NUM> additionally provides soundfield parameters (spatial parameters), indicated with <NUM>. In particular, the soundfield parameters <NUM> include active spatial parameters (first spatial parameters or first spatial parameter representation) <NUM> associated to the first frame <NUM>, and inactive spatial parameters (second spatial parameters or second spatial parameter representation) <NUM> associated to the second frame <NUM>. Each active spatial parameter <NUM> (<NUM>, <NUM>) may comprise (e.g. be) a parameter indicating a spatial characteristic of the audio signal (<NUM>) e.g. with respect to a listener position. In some other examples, the active spatial parameter <NUM> (<NUM>, <NUM>) may comprise (e.g. be) at least partially a parameter indicating a characteristic of the audio signal <NUM> with respect to the position of the loudspeakers. In some examples, the active spatial parameter <NUM> (<NUM>, <NUM>) may comprise (e.g. be) may at least partially comprise characteristics of the audio signal as taken from the signal source.

For example, the spatial parameters <NUM> (<NUM>, <NUM>) can include diffuseness parameters: e.g. one or more diffuseness parameter(s) indicating a diffuse to signal ratio with respect to the sound in the first frame <NUM> and/or in the second frame <NUM>, or one or more energy ratio parameter(s) indicating an energy ratio of a direct sound and a diffuse sound in the first frame <NUM> and/or in the second frame <NUM>, or an inter-channel/surround coherence parameter(s) in the first frame <NUM> and/or in the second frame <NUM>, or a Coherent-to-Diffuse Power ratio(s) in the first frame <NUM> and/or in the second frame <NUM>, or a signal-to-diffuse ratio(s) in the first frame <NUM> and/or in the second frame <NUM>.

In examples, the active spatial parameter(s) (first soundfield parameter representation) <NUM> and/or the inactive spatial parameter(s) <NUM> (second soundfield parameter representation) may be obtained from the input signal <NUM> in its full-channel version, or a subset of it, like the first order component of a higher order Ambisonics input signal.

The apparatus <NUM> includes an activity detector <NUM>. The activity detector <NUM> analyzes the input audio signal (either in its input version <NUM> or in its downmix version <NUM>), to determine, depending on the audio signal (<NUM> or <NUM>) whether a frame is an active frame <NUM> or an inactive frame <NUM>, hence performing a classification on the frame. As can be seen from <FIG>, the activity detector <NUM> can be assumed as controlling (e.g. through the control <NUM>) a first deviator <NUM> and a second deviator 322a. The first deviator <NUM> may select between the active spatial parameter <NUM> (first soundfield parameter representation) and the inactive spatial parameters <NUM> (second soundfield parameter representation). Therefore, the activity detector <NUM> may decide whether the active spatial parameters <NUM> or the inactive spatial parameters <NUM> are to be outputted (e.g. signalled in the bitstream <NUM>). The same control <NUM> may control the second deviator 322a, which may select between outputting the first frame <NUM> (<NUM>) in the transport channel <NUM>, or the second frame <NUM> (<NUM>) (e.g. parametric description) in the transport channel <NUM>. The activities of the first and second deviators <NUM> and 322a are coordinated with each other: when the active spatial parameters <NUM> are outputted, then the transport channels <NUM> of the first frame <NUM> are also outputted, and when the inactive spatial parameters <NUM> are outputted, then the transport channels <NUM> of the first frame <NUM> the transport channels are outputted. This is because the active spatial parameters <NUM> (first soundfield parameter representation) describe spatial characteristics of the first frame <NUM>, while the inactive spatial parameters <NUM> (second soundfield parameter representation) describes spatial characteristics of the second frame <NUM>.

The activity detector <NUM> may therefore basically decide which one among the first frame <NUM> (<NUM>, <NUM>), and its related parameters (<NUM>), and the second frame <NUM> (<NUM>, <NUM>), and its related parameters (<NUM>), are to be outputted. The activity detector <NUM> may also control the encoding of some signalling in the bitstream which signals whether the frame is an active or an inactive (other techniques may be used).

The activity detector <NUM> may perform processing on each frame <NUM>/<NUM> of the input audio signal <NUM> (e.g., by measuring energy in the frame, e.g., in all, or at least a plurality of, the frequency bins of the particular frames of the audio signal) and may classify the particular frame as being a first frame <NUM> or a second frame <NUM>. In general terms, the activity detector <NUM> may decide one single classification result for one single, whole frame, without distinguishing between different frequency bins and different samples of the same frame. For example, one classification result could be "speech" (which would amount to the first frame <NUM>, <NUM>, <NUM>, spatially described by the active spatial parameters <NUM>) or "silence" (which would amount to second frame <NUM>, <NUM>, <NUM>, spatially described by the inactive spatial parameters <NUM>). Therefore, according to the classification exerted by the activity detector <NUM>, the deviators <NUM> and 322a may perform their switching, and their result is in principle valid for all the frequency bins (and samples) of the classified frame.

The apparatus <NUM> includes an audio signal encoder <NUM>. The audio signal encoder <NUM> generates an encoded audio signal <NUM>. The audio signal encoder <NUM>, in particular, provides an encoded audio signal <NUM> for the first frame (<NUM>, <NUM>), e.g. generated by a transport channel encoder <NUM> which may be part of the audio signal encoder <NUM>. The encoded audio signal <NUM> may be or include a parametric description <NUM> of silence (e.g., parametric description of noise) and may be generated, by a transport channel Sl descriptor <NUM>, which may be part of the audio signal encoder <NUM>. The generated second frame <NUM> may correspond to at least one second frame <NUM> of the original audio input signal <NUM> and to at least one second frame <NUM> of the downmix signal <NUM>, and may be spatially described by the inactive spatial parameters <NUM> (second soundfield parameter representation). Notably, the encoded audio signal <NUM> (whether <NUM> or <NUM>) may also be in the transport channel (and may therefore be a downmix signal <NUM>). The encoded audio signal <NUM> (whether <NUM> or <NUM>) may be compressed, so as to reduce its size.

The apparatus <NUM> includes an encoded signal former <NUM>. The encoded signal former <NUM> may write an encoded version of at least the encoded audio scene <NUM>. The encoded signal former <NUM> operates by bringing together the first (active) soundfield parameter representation <NUM> for the first frame <NUM>, the second (inactive) soundfield parameter representation <NUM> for the second frame <NUM>, the encoded audio signal <NUM> for the first frame <NUM>, and the parametric description <NUM> for the second frame <NUM>. Accordingly, the audio scene <NUM> may be a bitstream, which may either be transmitted or stored (or both) and used by a generic decoder for generating an audio signal to be output, which is a copy of the original input signal <NUM>. In the audio scene (bitstream) <NUM>, sequence of "first frames"/"second frames" may therefore be obtained, for permitting a reproduction of the input signal <NUM>.

<FIG> shows an example of an encoder <NUM>, and a decoder <NUM> (the decoder <NUM> being useful to understand the invention). The encoder <NUM> may be the same of (or a variation of) that of <FIG> in some examples (in some other examples, they can be different embodiments). The encoder <NUM> has in input the audio signal <NUM> (which may, for example, be in B-format) and may have a first frame <NUM> (which can be, for example, be an active frame) and a second frame <NUM> (which can be, for example, an inactive frame). The audio signal <NUM> may be provided, as signal <NUM> (e.g., as encoded audio signal <NUM> for the first frame and encoded audio signal <NUM>, or parametric representation, for the second frame), to the audio signal encoder <NUM> after a selection internal in the selector <NUM> (which may include audio associated to the deviators <NUM> and 322a). Notably, the block <NUM> can also have the capabilities of forming the downmix from the input signal <NUM> (<NUM>, <NUM>) onto the transport channels <NUM> (<NUM>, <NUM>). Basically, the block <NUM> (beamforming/signal-selection block) may be understood as including functionalities of the activity detector <NUM> of <FIG>, but some other functionalities (such as the generation of the spatial parameters <NUM> and <NUM>) which in <FIG> are performed by block <NUM> may be performed by "DirAC analysis block" <NUM> of <FIG>. Therefore, the channel signal <NUM> (<NUM>, <NUM>) may be a downmixed version of the original signal <NUM>. In some cases, however, it could also be possible that no downmixing is performed on the signal <NUM>, and a signal <NUM> is simply a selection between the first and second frames. The audio signal encoder <NUM> may include at least one of the blocks <NUM> and <NUM> as explained above. The audio signal encoder <NUM> may output the encoded audio signal <NUM> either for the first frame <NUM> or for the second frame <NUM>. <FIG> does not show the encoded signal former <NUM>, which may notwithstanding be present.

As shown, block <NUM> may include a DirAC analysis block (or more in general, soundfield parameter generator <NUM>). The block <NUM> (soundfield parameter generator) may include a filterbank analysis <NUM>. The filterbank analysis <NUM> subdivides each frame of the input signal <NUM> onto a plurality of frequency bins, which may be the output <NUM> of the filterbank analysis <NUM>. A diffuseness estimation block 392a may provide diffuseness parameters 314a (which may be one diffuseness parameter of the active spatial parameter(s) <NUM> for an active frame <NUM> or one diffuseness parameter in of the inactive spatial parameter(s) <NUM> for an inactive frame <NUM>), e.g. for each frequency bin of the plurality of frequency bins <NUM> outputted by the filterbank analysis <NUM>. The soundfield parameter generator <NUM> may include a direction estimation block 392b, whose output 314b may be a direction parameter (which may be one direction parameter of the active spatial parameter(s) <NUM> for an active frame <NUM> or one direction parameter in of the inactive spatial parameter(s) <NUM> for an inactive frame <NUM>), e.g. for each frequency bin of the plurality of frequency bins <NUM> outputted by the filterbank analysis <NUM>.

<FIG> shows an example of block <NUM> (soundfield parameter generator). The soundfield parameter generator <NUM> may be the same of that of <FIG> and/or may be the same or at least implement functionalities of block <NUM> of <FIG>, despite the fact that block <NUM> of <FIG> is also capable of performing a downmix of the input signal <NUM>, while this is not shown (or not implemented) in the soundfield parameter generator <NUM> of <FIG>.

The soundfield parameter generator <NUM> of <FIG> may include a filterbank analysis block <NUM> (which may be the same of the filterbank analysis block <NUM> of <FIG>). The filterbank analysis block <NUM> may provide frequency domain information <NUM> for each frame and for each bin (frequency tile). The frequency domain information <NUM> may be provided to a diffuseness analysis block 392a and/or a direction analysis block 392b, which may be those shown in <FIG>. The diffuseness analysis block 392a and/or direction analysis block 392b may provide diffuseness information 314a and/or direction information 314b. These can be provided for each first frame <NUM> (<NUM>) and for each second frame <NUM> (<NUM>). Complexively, the information provided by the block 392a and 392b is considered soundfield parameters <NUM> which encompass both first soundfield parameters <NUM> (active spatial parameters) and second soundfield parameters <NUM> (inactive spatial parameters). The active spatial parameters <NUM> may be provided to an active spatial metadata encoder <NUM> and the inactive spatial parameters <NUM> may be provided to an inactive spatial metadata encoder <NUM>. The resulting are first and second soundfield parameter representations (<NUM>, <NUM>, complexively indicated with <NUM>) which may be encoded in the bitstream <NUM> (e.g., through the encoder signal former <NUM>) and stored for being subsequently played back by a decoder. Whether the active spatial metadata encoder <NUM> or the inactive spatial parameters <NUM> is to encode a frame, this may be controlled by a control such as the control <NUM> in <FIG> (the deviator <NUM> is not shown in <FIG>), e.g. thorough the classification operated by the activity detector. (It is to be noted that the encoders <NUM>, <NUM> may also perform a quantization, in some examples).

<FIG> shows another example of possible soundfield parameter generator <NUM>, which is alternative to that of <FIG>, and which may also be implemented in the examples of <FIG> and <FIG>. In this example, the input audio signal <NUM> is already in MASA format, in which spatial parameters are already part of the input audio signal <NUM> (e.g., as spatial metadata), e.g. for each frequency bin of a plurality of frequency bins. Accordingly, there is no need for having a diffuseness analysis block and/or a directional block, but they can be substituted by a MASA reader <NUM>. The MASA reader <NUM> may read specific data fields in the audio signal <NUM>, which already contain information such as the active spatial parameter(s) <NUM> and the inactive spatial parameter(s) <NUM> (according to the fact whether the frame of the signal <NUM> is a first frame <NUM> or a second frame <NUM>). Examples of parameters that may be encoded in the signal <NUM> (and which may be read by the MASA reader <NUM>) may include at least one of a direction, energy ratio, surround coherence, spread coherence, and so on. Downstream to the MASA reader <NUM>, an active spatial metadata encoder <NUM> (e.g., like the one of <FIG>) and an inactive spatial metadata encoder <NUM> (e.g., like the one of <FIG>) may be provided, to output the first soundfield parameter representation <NUM> and the second soundfield parameter representation <NUM>, respectively. If the input audio signal <NUM> is a MASA signal, then the activity detector <NUM> may be implemented as an element which reads a determined data field in the input MASA signal <NUM>, and classifies as active frame <NUM> or inactive frame <NUM> based on the value encoded in the data field. The example of <FIG> can be generalized for an audio signal <NUM> which has already encoded therein spatial information which can be encoded as active spatial parameter <NUM> or inactive spatial parameter <NUM>.

Embodiments of the present invention are applied in a spatial audio coding system, e.g. illustrated in <FIG>, where a DirAC-based spatial audio encoder and decoder are depicted. A discussion thereof follows here.

The encoder <NUM> may usually analyze the spatial audio scene in B-format. Alternatively, DirAC analysis can be adjusted to analyze different audio formats like audio objects or multichannel signals or the combination of any spatial audio formats.

The DirAC analysis (e.g. as performed at any of stages 392a, 392b) may extract a parametric representation from the input audio scene <NUM> (input signal). A direction of arrival (DOA) 314b and/or a diffuseness 314a measured per time-frequency unit form the parameter(s) <NUM>, <NUM>. The DirAC analysis (e.g. as performed at any of stages 392a, 392b) may be followed by a spatial metadata encoder (e.g. <NUM> and/or <NUM>), which may quantize and/or encode the DirAC parameters to obtain a low bit-rate parametric representation (in the figures, the low bit-rate parametric representations <NUM>, <NUM> are indicated with the same reference numerals of the parametric representations upstream to the spatial metadata encoders <NUM> and/or <NUM>).

Along with the parameters <NUM> and/or <NUM>, a down-mix signal <NUM> (<NUM>) derived from the different source(s) (e.g. different microphones) or audio input signal(s) (e.g. different components of a multichannel signal) <NUM> may be coded (e.g. for transmission and/or for storage) by a conventional audio core-coder. In the preferred embodiment, an EVS audio coder (e.g. <NUM>, <FIG>) may be preferred for coding the down-mix signal <NUM> (<NUM>, <NUM>), but embodiments of the invention are not limited to this core-coder and can be applied to any audio core-coder. The down-mix signal <NUM> (<NUM>, <NUM>) may consist, for example, of different channels, also called transport channels: the signal <NUM> can be, e.g., or comprise, the four coefficient signals composing a B-format signal, a stereo pair or a monophonic down-mix depending on the targeted bit-rate. The coded spatial parameters <NUM> and the coded audio bitstream <NUM> may be multiplexed before being transmitted over the communication channel (or stored).

In the decoder (see below), the transport channels <NUM> are decoded by a core-decoder, while the DirAC metadata (e.g., spatial parameters <NUM>, <NUM>) may be first decoded before being conveyed with the decoded transport channels to the DirAC synthesis. The DirAC synthesis uses the decoded metadata for controlling the reproduction of the direct sound stream and its mixture with the diffuse sound stream. The reproduced sound field can be reproduced on an arbitrary loudspeaker layout or can be generated in Ambisonics format (HOA/FOA) with an arbitrary order.

It is here explained a non-limiting technique for estimating the spatial parameters <NUM>, <NUM> (e.g. diffuseness 314a, direction 314b). The example of B-format is provided.

In each frequency band (e.g., as obtained from the filterbank analysis <NUM>), the direction of arrival 314a of sound together with the diffuseness 314b of the sound may be estimated. From the time-frequency analysis of the input B-format components wi(n),xi(n),yi(n),zi(n), pressure and velocity vectors can be determined as: <MAT> <MAT> where i is the index of the input <NUM> and, k and n time and frequency indices of the time-frequency tile, and ex, ey, ez represent the Cartesian unit vectors. P(n, k) and U(n, k) may be necessary, in some examples, to compute the DirAC parameters (<NUM>, <NUM>), namely DOA 314a and diffuseness 314a through, for example, the computation of the intensity vector: <MAT> where (·) denotes complex conjugation. The diffuseness of the combined sound field is given by: <MAT> where E{. } denotes the temporal averaging operator, c the speed of sound and E(k, n) the sound field energy given by: <MAT>.

The diffuseness of the sound field is defined as the ratio between sound intensity and energy density having values between <NUM> and <NUM>.

The direction of arrival (DOA) is expressed by means of the unit vector direction(n, k), defined as <MAT>.

The direction of arrival 314b can be determined by an energetic analysis (e.g., at 392b) of the B-format input signal <NUM> and can be defined as opposite direction of the intensity vector. The direction is defined in Cartesian coordinates but can e.g. be easily transformed in spherical coordinates defined by a unity radius, the azimuth angle and elevation angle.

In the case of transmission, the parameters 314a, 314b (<NUM>, <NUM>) needed to be transmitted to the receiver side (e.g. decoder side) via a bitstream (e.g. <NUM>). For a more robust transmission over a network with limited capacity, a low bit-rate bitstream is preferable or even necessary, which can be achieved by designing an efficient coding scheme for the DirAC parameters 314a, 314b (<NUM>, <NUM>). It can employ for example techniques such as frequency band grouping by averaging the parameters over different frequency bands and/or time units, prediction, quantization and entropy coding. At the decoder, the transmitted parameters can be decoded for each time/frequency unit (k,n) in case no error occurred in the network. However, if the network conditions are not good enough to ensure proper packet transmission, a packet may be lost during transmission. Embodiments of the present invention aim to provide a solution in the latter case.

<FIG> shows an example of a decoder apparatus <NUM> useful to understand the invention. It may be an apparatus for processing an encoded audio scene (<NUM>) comprising, in a first frame (<NUM>), a first soundfield parameter representation (<NUM>) and an encoded audio signal (<NUM>), wherein a second frame (<NUM>) is an inactive frame. The decoder apparatus <NUM> may comprise at least one of:.

Notably, the activity detector (<NUM>) may exert a command <NUM>' which may determine whether the input frame is classified as an active frame <NUM> or an inactive frame <NUM>. The activity detector <NUM> may determine the classification of the input frame, for example, from an information <NUM> which is either signalled, or determined from the length of the obtained frame.

The synthetic signal synthesizer (<NUM>) may, for example, generate noise <NUM> e.g. using the information (e.g. parametric information) obtained from the parametric representation <NUM>. The spatial renderer <NUM> may generate the output signal <NUM> in such a way that the inactive frames <NUM> (obtained from the encoded frames <NUM>) are processed through the inactive spatial parameter(s) <NUM>, to obtain that a human listener has a 3D spatial impression of the provenience of the noise.

It is noted that in <FIG> the numerals <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are the same of the numerals of <FIG>, since they correspond as being obtained from the bitstream <NUM>. Notwithstanding, it may be that some slight differences (e.g., due to quantization) are present.

<FIG> also shows a control <NUM>' which may control a deviator <NUM>', so that the signal <NUM> (outputted by the synthetic signal synthesizer <NUM>) or the audio signal <NUM> (outputted by the audio decoder <NUM>) may be selected, e.g. through the classification operated by the activity detector <NUM>. Notably, the signal <NUM> (either <NUM> or <NUM>) may still be a downmix signal, which may be provided to the spatial renderer <NUM> so that the spatial renderer generates the output signal <NUM> through the active or inactive spatial parameters <NUM> (<NUM>, <NUM>). In some examples, the signal <NUM> (either <NUM> or <NUM>) can notwithstanding be upmixed, so that the number of channels of the signal <NUM> is increased with respect to the encoded version <NUM> (<NUM>, <NUM>). In some examples, despite being upmixed, the number of channels of the signal <NUM> may be less than the number of channel of the output signal <NUM>.

Here below, other examples of the decoder apparatus <NUM> are provided. <FIG> show examples of decoder apparatus <NUM>, <NUM>, <NUM>, <NUM> which may embody the decoder apparatus <NUM>.

Even though in <FIG> some elements are shown as being internal to the spatial renderer <NUM>, they may be notwithstanding outside of the spatial renderer <NUM> in some examples. For example, the synthetic synthesizer <NUM> may either be partially or totally external to the spatial renderer <NUM>.

In those examples, a parameter processor <NUM> (which may be either internal or external to the spatial renderer <NUM>) may be included. The parameter processor <NUM> may also be considered to be present in the decoder of <FIG>, despite not being shown.

The parameter processor <NUM> of any of <FIG> may include, for example, an inactive spatial parameter decoder <NUM> for providing the inactive frames may be parameters <NUM> (e.g., as obtained from the signaling in the bit stream <NUM>) and/or a block <NUM> ("recover spatial parameters in non-transmitted frames decoder") which provides inactive spatial parameters which are not read in the bitstream <NUM>, but which are obtained (e.g. recovered, reconstructed, extrapolated, inferred, etc.), e.g., by extrapolation or are synthetically generated.

Therefore, the second soundfield parameter representation may also be a generated parameter <NUM>, which was not present in the bitstream <NUM>. As will be explained later, the recovered (reconstructed, extrapolated, inferred, etc.) spatial parameters <NUM> may be obtained, for example, through a "hold strategy", to an "extrapolation of the direction strategy" and/or through a "dithering of the direction" (see below). The parameter processor <NUM> may, therefore, extrapolate or anyway obtain the spatial parameters <NUM> from the previous frames. As can be seen in <FIG>, a switch <NUM>' may select between the inactive spatial parameters <NUM> as signaled in the bitstream <NUM> and the recovered spatial parameters <NUM>. As explained above, the encoding of the silence frames <NUM> (SID) (and also of the inactive spatial parameters <NUM>) is updated at a lower bitrate than the encoding of the first frames <NUM>: the inactive spatial parameters <NUM> are updated with lower frequency with respect to the active spatial parameters <NUM>, and some strategies are performed by the parameter processor <NUM> (<NUM>) for recovering non-signaled spatial parameters <NUM> for non-transmitted inactive frames. Accordingly, the switch <NUM>' may select between the signaled inactive spatial parameters <NUM> and the non-signaled (but recovered or otherwise reconstructed) inactive spatial parameters <NUM>. In some cases, the parameter processor <NUM>' may store one or more soundfield parameter representations <NUM> for several frames occurring before the second frame or occurring in time subsequent to the second frame, to extrapolate (or interpolate) the soundfield parameters <NUM> for the second frame. In general terms, the spatial renderer <NUM> may use, for the rendering of the synthetic audio signal <NUM> for the second frame <NUM>, the one or more soundfield parameters <NUM> for the second frame <NUM>. In addition or alternatively, the parameter processor <NUM> may store soundfield parameter representations <NUM> for the active spatial parameters (shown in <FIG>) and synthesize the soundfield parameters <NUM> for the second frame (inactive frame) using the stored first soundfield parameter representation <NUM> (active frames) to generate the recovered spatial parameter <NUM>. As shown in <FIG> (but also implementable in any of <FIG>), it is also possible to also include an active spatial parameter decoder <NUM> from which active spatial parameters <NUM> can be obtained from the bitstream <NUM>. This may perform a dithering with directions included in the at least two soundfield parameter representations occurring in time before or after the second frame (<NUM>), when extrapolating or interpolating to determine the one or more soundfield parameters for the second frame (<NUM>).

The synthetic signal synthesizer <NUM> may be internal to the spatial renderer <NUM> or may be external or, in some cases, it may have an internal portion and an external portion. The synthetic synthesizer <NUM> may operate on the downmix channels of the transport channels <NUM> (which are less than the output channels) (it is noted here that M is a number of downmix channels and N is the number of output channels). The synthetic signal generator <NUM> (other name for the synthetic signal synthesizer) may generate, for the second frame, a plurality of synthetic component audio signals (in at least one of the channels of the transport signal or in at least one individual component of the output audio format) for individual components related to an outer format of the spatial renderer as the synthetic audio signal. In some cases, this may be in the channels of the downmix signal <NUM> and in some cases it may be in one of the internal channels of the spatial rendering.

<FIG> shows an example in which at least K channels 228a obtained from the synthetic audio signal <NUM> (e.g., in its version 228b downstream to a filterbank analysis <NUM>) may be decorrelated.

This is obtained, for example, when the synthetic synthesizer <NUM> generates the synthetic audio signal <NUM> in at least one of the M channels of the synthetic audio signal <NUM>. This correlating processing <NUM> may be applied to the signal 228b (or at least one or some of its components), downstream to the filterbank analysis block <NUM>, so that at least K channels (with K ≥ M and/or K ≤ N, with N the number of output channels) may be obtained. Subsequently, the K decorrelated channels 228a and/or M channels of the signal 228b may be provided to a block <NUM> for generating mixing gains/matrices which, through the spatial parameters <NUM>, <NUM> (see above), may provide a mixed signal <NUM>. The mixed signal <NUM> may be subjected to a filterbank synthesis block <NUM>, to obtain the output signal in N output channels <NUM>. Basically, reference numeral 228a of <FIG> may be an individual synthetic component audio signal which is decorrelated from the individual synthetic component audio signal 228b, so that the spatial renderer (and the block <NUM>) makes use of a combination of the component 228a and the component 228b. <FIG> shows an example in which the whole channels <NUM> are generated in K channels.

Furthermore, in <FIG>, the decorrelator <NUM> applied to K decorrelated channels 228b downstream to the filterbank analysis block <NUM>. This may be performed, for example, for the diffuse field. In some cases, M channels of the signal 228b downstream to the feedback analysis block <NUM> and may be provided to the block <NUM> generating mixing gain/matrices. A covariance method may be used for reducing the issues of the decorrelators <NUM>, e.g. by scaling the channels 228b by a value associated with a value complementary to the covariance between the different channels.

<FIG> shows an example of synthetic signal synthesizer <NUM> which is in the frequency domain. A covariance method may be used for the synthetic synthesizer <NUM> (<NUM>) of <FIG>. Notably, the synthetic audio synthesizer <NUM> (<NUM>) provides its output 228c in K channels (with K ≥ M), while the transport channel <NUM> would be in M channels.

<FIG> shows an example of decoder <NUM> (embodiment of the decoder <NUM>) which may be understood as making use of a hybrid technique of the decoder <NUM> of <FIG> and the decoder <NUM> of <FIG>. As can be seen here, the synthetic signal synthesizer <NUM> includes a first portion <NUM> (<NUM>) which generates a synthetic audio signal <NUM> in the M channels of the downmix signal <NUM>. The signal <NUM> may be inputted to a filterbank analysis block <NUM> which may provide an output 228b in which plural filter bands are distinguished from each other. At this time channels 228b may be decorrelated to obtain the decorrelated signal 228a in K channels. Meanwhile, the output 228b of the filterbank analysis in M channels is provided to a block <NUM> for generating mixing gain matrices which may provide a mixed version of the mixed signal <NUM>. The mixed signal <NUM> may keep into account the inactive spatial parameters <NUM> and/or the recovered (reconstructed) spatial parameters for the inactive frames <NUM>. It is to be noted that the output 228a of the decorrelator <NUM> may also be added, at an adder <NUM>, to an output 228d of a second portion <NUM> of the synthetic signal synthesizer <NUM>, which provides a synthetic signal 228d in K channels. The signal 228d may be summed, at addition block <NUM>, to the decorrelated signal 228a to provide summed signal 228e to the mixing block <NUM>. Therefore, it is possible to render the final output signal <NUM> by using a combination of the component 228b and the component 228e which brings into account both decorrelated components 228a and the generated components 228d. The components 228b, 228a, 228d, 228e (is present) of <FIG> and <FIG> maybe understood, for example, as diffuse and non-diffuse components of the synthetic signal <NUM>. In particular, with reference to the decoder <NUM> of <FIG>, basically the low frequency bands of the signal 228e can be obtained from the transport channel <NUM> (and are obtained from 228a) and the high frequency bands of the signal 228e can be generated in the synthesizer <NUM> (and are in the channels 228d), their addition at the adder <NUM> permitting to have both in the signal 228e.

Notably, in <FIG> above there is not shown the transport channel decoder for the active frames.

<FIG> shows an example of decoder <NUM> (embodiment of the decoder <NUM>) in which both the audio decoder <NUM> (which provides the decoded channels <NUM>) and the synthetic signal synthesizer <NUM> (here considered to be divided between a first, external portion <NUM> and a second, internal portion <NUM>) are shown. A switch <NUM>' is shown which may be analogous to that of <FIG> (e.g., controlled by the control or command <NUM>' provided by the activity detector <NUM>). Basically, it is possible to select between a mode in which the decoded audio scene <NUM> is provided to the spatial renderer <NUM> and another mode which the synthetic audio signal <NUM> is provided. The downmix signal <NUM> (<NUM>, <NUM>) is in M channels, which are in general less than the N output channels of the output signal <NUM>.

The signal <NUM> (<NUM>, <NUM>) may be inputted to a filterbank analysis block <NUM>. The output 228b of the filterbank analysis <NUM> (in a plurality of frequency bins) may be inputted onto an upmix addition block <NUM>, which may be also inputted by a signal 228d provided by the second portion <NUM> of the synthetic signal synthesizer <NUM>. The output 228f of the upmix addition block <NUM> may inputted to the correlator processing <NUM>. The output 228a of the decorrelator processing <NUM> may be provided, together to the output 228f of the upmix addition block <NUM>, to the block <NUM> for generating the mixing gain and matrices. The upmix addition block <NUM> may, for example, increase the number of the channels from M to K (and, in some cases, it can scale them, e.g. by multiplication by constant coefficients) and may add the K channels with the K channels 228d generated by the synthetic signal synthesizer <NUM> (e.g., second, internal portion <NUM>). In order to render a first (active) frame, the mixing block <NUM> may consider at least one of the active spatial parameters <NUM> as provided in the bit stream <NUM>, the recovered (reconstructed) spatial parameters <NUM> as extrapolated or otherwise obtained (see above).

In some examples, the output of the filterbank analysis block <NUM> may be in M channels but may take into consideration different frequency bands. For the first frames (and the switch <NUM>' and the switch <NUM>' being positioned as in <FIG>), the decoded signal <NUM> (in at least two channels) may be provided to the filterbank analysis <NUM> and may therefore be weighted at the upmix addition block <NUM> through K noise channels 228d (synthetic signal channels) to obtain the signal 228f in K channels. It is remembered that K ≥ M and may comprise, for example, diffuse channel and a directional channel. In particular, the diffuse channel may be decorrelated by the decorrelator <NUM> to obtain a decorrelated signal 228a. Accordingly, the decoded audio signal <NUM> may be weighted (e.g. at block <NUM>) with the synthetic audio signal 228d which can mask the transition between active and inactive frames (first frames and second frames). Then, the second part <NUM> of the synthetic signal synthesizer <NUM> is used not only for active frames but also for inactive frames.

<FIG> shows another example of the decoder <NUM> which may comprise in a first frame (<NUM>), a first soundfield parameter representation (<NUM>) and an encoded audio signal (<NUM>), wherein a second frame (<NUM>) is an inactive frame, the apparatus comprising an activity detector (<NUM>) for detecting that the second frame (<NUM>) is the inactive frame and for providing a parametric description (<NUM>) for the second frame (<NUM>); a synthetic signal synthesizer (<NUM>) for synthesizing a synthetic audio signal (<NUM>) for the second frame (<NUM>) using the parametric description (<NUM>) for the second frame (<NUM>); an audio decoder (<NUM>) for decoding the encoded audio signal (<NUM>) for the first frame (<NUM>); and a spatial renderer (<NUM>) for spatially rendering the audio signal (<NUM>) for the first frame (<NUM>) using the first soundfield parameter representation (<NUM>) and using the synthetic audio signal (<NUM>) for the second frame (<NUM>), or a transcoder for generating a meta data assisted output format comprising the audio signal (<NUM>) for the first frame (<NUM>), the first soundfield parameter representation (<NUM>) for the first frame (<NUM>), the synthetic audio signal (<NUM>) for the second frame (<NUM>), and a second soundfield parameter representation (<NUM>) for the second frame (<NUM>).

With reference to the synthetic signal synthesizer <NUM> in the examples above, as explained above, it may comprise (or even be) a noise generator (e.g. comfort noise generator). In examples, the synthetic signal generator (<NUM>) may comprise a noise generator and the first individual synthetic component audio signal is generated by a first sampling of the noise generator and the second individual synthetic component audio signal is generated by a second sampling of the noise generator, wherein the second sampling is different from the first sampling.

In addition or alternatively, the noise generator comprises a noise table, and wherein the first individual synthetic component audio signal is generated by taking a first portion of the noise table, and wherein the second individual synthetic component audio signal is generated by taking a second portion of the noise table, wherein the second portion of the noise table is different from the first portion of the noise table.

In examples, the noise generator comprises a pseudo noise generator, and wherein the first individual synthetic component audio signal is generated by using a first seed for the pseudo noise generator, and wherein the second individual synthetic component audio signal is generated using a second seed for the pseudo noise generator.

In general terms, the spatial renderer <NUM>, in the examples of <FIG>, <FIG>, <FIG>, <FIG> and <FIG>, may operate in a first mode for the first frame (<NUM>) using a mixing of a direct signal and a diffuse signal generated by a decorrelator (<NUM>) from the direct signal under a control of the first soundfield parameter representation (<NUM>), and in a second mode for the second frame (<NUM>) using a mixing of a first synthetic component signal and the second synthetic component signal, wherein the first and the second synthetic component signals are generated by the synthetic signal synthesizer (<NUM>) by different realizations of a noise process or a pseudo noise process.

As explained above, the spatial renderer (<NUM>) may be configured to control the mixing (<NUM>) in the second mode by a diffuseness parameter, an energy distribution parameter, or a coherence parameter derived for the second frame (<NUM>) by a parameter processor.

Examples above also regard a method of generating an encoded audio scene from an audio signal having a first frame (<NUM>) and a second frame (<NUM>), comprising: determining a first soundfield parameter representation (<NUM>) for the first frame (<NUM>) from the audio signal in the first frame (<NUM>) and a second soundfield parameter representation (<NUM>) for the second frame (<NUM>) from the audio signal in the second frame (<NUM>); analyzing the audio signal to determine, depending on the audio signal, that the first frame (<NUM>) is an active frame and the second frame (<NUM>) is an inactive frame; generating an encoded audio signal for the first frame (<NUM>) being the active frame and generating a parametric description (<NUM>) for the second frame (<NUM>) being the inactive frame; and composing the encoded audio scene by bringing together the first soundfield parameter representation (<NUM>) for the first frame (<NUM>), the second soundfield parameter representation (<NUM>) for the second frame (<NUM>), the encoded audio signal for the first frame (<NUM>), and the parametric description (<NUM>) for the second frame (<NUM>).

Examples above also regard a method of processing an encoded audio scene comprising, in a first frame (<NUM>), a first soundfield parameter representation (<NUM>) and an encoded audio signal, wherein a second frame (<NUM>) is an inactive frame, the method comprising: detecting that the second frame (<NUM>) is the inactive frame and for providing a parametric description (<NUM>) for the second frame (<NUM>); synthesizing a synthetic audio signal (<NUM>) for the second frame (<NUM>) using the parametric description (<NUM>) for the second frame (<NUM>); decoding the encoded audio signal for the first frame (<NUM>); and spatially rendering the audio signal for the first frame (<NUM>) using the first soundfield parameter representation (<NUM>) and using the synthetic audio signal (<NUM>) for the second frame (<NUM>), or generating a meta data assisted output format comprising the audio signal for the first frame (<NUM>), the first soundfield parameter representation (<NUM>) for the first frame (<NUM>), the synthetic audio signal (<NUM>) for the second frame (<NUM>), and a second soundfield parameter representation (<NUM>) for the second frame (<NUM>).

There is also provided an encoded audio scene (<NUM>) comprising: a first soundfield parameter representation (<NUM>) for a first frame (<NUM>); a second soundfield parameter representation (<NUM>) for a second frame (<NUM>); an encoded audio signal for the first frame (<NUM>); and a parametric description (<NUM>) for the second frame (<NUM>).

In the examples above, it may be that the spatial parameters <NUM> and/or <NUM> are transmitted for each frequency band (subband).

According to some examples, this silence parametric description <NUM> may contain this partial parameter <NUM> which may therefore be part of the SID <NUM>.

The spatial parameter <NUM> for the inactive frames may be valid for each frequency subband (or band or frequency).

The spatial parameters <NUM> and/or <NUM> discussed above, transmitted or encoded, during the active phase <NUM> and in the SID <NUM> may have different frequency resolution and in addition or alternatively the spatial parameters <NUM> and/or <NUM> discussed above, transmitted or encoded, during the active phase <NUM> and in the SID <NUM> may have different time resolution and in addition or alternatively the spatial parameters <NUM> and/or <NUM> discussed above, transmitted or encoded, during the active phase <NUM> and in the SID <NUM> may have different quantization resolution.

It is noted that the decoding device and an encoding device may be devices like CELP or DCX or bandwidth extension modules.

It is also possible to make use and an MDCT-based coding scheme (modified discrete cosine transform).

In the present examples of the decoder apparatus <NUM> (in any of its embodiments, e.g. those of <FIG>), it is possible to substitute the audio decoder <NUM> and the spatial renderer <NUM> with a transcoder for generating a meta data assisted output format comprising the audio signal for the first frame, the first soundfield parameter representation for the first frame, the synthetic audio signal for the second frame, and a second soundfield parameter representation for the second frame.

Embodiments of the present invention propose a way to extend DTX to parametric spatial audio coding. It is therefore proposed to apply a conventional DTX/CNG on the downmix/transport channels (e.g. <NUM>, <NUM>) and to extend it with spatial parameters (called afterward spatial SID) e.g. <NUM>, <NUM> and a spatial rendering on the inactive frames (e.g. <NUM>, <NUM>, <NUM>, <NUM>) at the decoder side. For restituting the spatial image of the inactive frames (e.g. <NUM>, <NUM>, <NUM>, <NUM>), the transport channel SID <NUM>, <NUM> is amended with some spatial parameters (spatial SID) <NUM> (or <NUM>) specially designed and relevant for immersive background noises. Embodiments of the present invention (discussed below and/or above) cover at least two aspects:.

<FIG> depicts an overview of embodiments of the encoder apparatus <NUM>. At the encoder side, the signal can be analyzed by the DirAC analysis. DirAC can analyze signals like B-format or first order Ambisonics (FOA). However it is also possible to extend the principle to higher order Ambisonics (HOA), and even to multi-channel signals associated with a given loudspeaker setup like <NUM>, or <NUM> or <NUM> + <NUM> as proposed in [<NUM>]. The input format <NUM> can also be individual audio channels representing one, or several different audio objects localized in the space by information included in associated metadata. Alternatively, the input format <NUM> can be Metadata associated Spatial Audio (MASA). In this case spatial parameters and transport channels are directly conveyed to the encoder apparatus <NUM>. The audio scene analysis (e.g. as shown in <FIG>) can be then skipped, and only an eventual spatial parameter (re-)quantization and resampling has to be performed for the inactive set of spatial parameters <NUM> or for both the active and inactive sets of spatial parameters <NUM>, <NUM>.

The audio scene analysis may be done for both active and inactive frames <NUM>, <NUM> and produce two sets of spatial parameters <NUM>, <NUM>. A first set <NUM> in case of active frame <NUM> and another (<NUM>) in case of inactive frame <NUM>. It is possible to have no inactive spatial parameters, but in the preferred embodiment of the invention the inactive spatial parameters <NUM> are fewer and/or quantized coarser than the active spatial parameters <NUM>. After that two versions of the spatial parameters (also called DirAC metadata) may be available. Importantly embodiments of the present invention can be mainly directed to spatial representations of the audio scene from the listener's perspective. Therefore spatial parameters, like DirAC parameters <NUM>, <NUM> including one or several direction(s) along with an eventual diffuseness factor or energy ratio(s), are considered. Unlike inter-channel parameters, these spatial parameters from the listener's perspective have the great advantage of being agnostic of the sound capture and reproduction system. This parametrization is not specific to any particular microphone array or loudspeaker layout.

The Voice Activity Detector (or more in general an activity detector) <NUM> may then be applied on the input signal <NUM> and/or the transport channels <NUM> produced by the audio scene analyzer. The transport channels are less than the number of input channels; usually a mono-downmix, a stereo downmix, an A-format, or a First Order Ambisonics signal. Based on the VAD decision the current frame under process is defined as active (<NUM>, <NUM>) or inactive (<NUM>, <NUM>). In case of active frames (<NUM>, <NUM>), a conventional speech or audio encoding of the transport channels is performed. The resulting code data are then combined with the active spatial parameters <NUM>. In case of inactive frames (<NUM>, <NUM>), a silence information description <NUM> of the transport channels <NUM> is produced episodically, usually at regular frame intervals during inactive phase, for example at every <NUM> active frames (<NUM>, <NUM>, <NUM>). The transport channel SID (<NUM>, <NUM>) may then be amended in the multiplexer (encoded signal former) <NUM> with the inactive spatial parameters. In case the inactive spatial parameters <NUM> are null, only the transport channel SID <NUM> is then transmitted. The overall SID can usually be a very low bit-rate description, which is for example as low as <NUM> or <NUM> kbps. The average bit-rate is even more reduced in the inactive phase since most of the time no transmission is done and no data are sent.

In the preferred embodiment of the invention the transport channel SID <NUM> has a size of <NUM>. 4kbps and the overall SID including spatial parameters has a size of <NUM>. The computation of the inactive spatial parameters are described in <FIG> for DirAC having as input a multi-channel signal like FOA, which could directly derived from a higher order of Ambisonics (HOA), in <FIG> for MASA input format. As described earlier, the inactive spatial parameters <NUM> can be derived in parallel to the active spatial parameters <NUM>, averaging and/or requantizing the already coded active spatial parameters <NUM>. In case of multi-channel signal like FOA as input format <NUM>, a filterbank analysis of the multi-channel signal <NUM> may be performed before computing the spatial parameters, direction and diffuseness, for each time and frequency tile. The metadata encoders <NUM>, <NUM> could average the parameters <NUM>, <NUM> over different frequency bands and/or time slots before applying a quantizer and a coding of the quantized parameters. Further inactive spatial metadata encoder can inherit from some of the quantized parameters derived in the active spatial metadata encoder to use them directly in the inactive spatial parameters or to requantize them. In case of MASA format (e.g. <FIG>), first the input metadata may be read and provided the metadata encoders <NUM>, <NUM> at a given time-frequency and bit depth resolution. The metadata encoder(s) <NUM>, <NUM> will process then further by eventually converting some parameters, adapting their resolution (i.e. lowering the resolution for example averaging them) and requantizing them before coding them by an entropy coding scheme for example.

At the decoder side as depicted e.g. in <FIG>, the VAD information <NUM> (e.g. whether the frame is classified as active or inactive) is first recovered, either by detecting the size of the transmitted packet (e.g. frame) or by detecting the non-transmission of a packet. In active frames <NUM>, the decoder runs in the active mode and the transport channel coder payload is decoded as well as the active spatial parameters. The spatial renderer <NUM> (DirAC synthesis) then upmixes/spatial-izes the decoded transport channels using the decoded spatial parameters <NUM>, <NUM> in the output spatial format. In inactive frames, a comfort noise may be generated in the transport channels by the transport channel CNG portion <NUM> (e.g. in <FIG>). The CNG is guided the transport channel SID for adjusting usually the energy and the spectral shape (through for example scale factors applied in frequency domain or Linear Predictive Coding Coefficients applied through a time domain synthesis filter). The comfort noise(s) 228d, 228a, etc. are then rendered/spatialized in the spatial renderer (DirAC synthesis) <NUM> guided this time by the inactive spatial parameters <NUM>. The output spatial format <NUM> can be a binaural signal (<NUM> channels), multi-channel for a given loudspeaker layout, or a multi-channel signal in Ambisonic format. In an alternative embodiment, the output format can be Metadata assisted spatial audio (MASA), that means that the decoded transport channels or the transport channel comfort noises are directly output along with the active or inactive spatial parameters, respectively, for rendering by an external device.

The inactive spatial parameters <NUM> can consist of one of multiple directions in frequency bands and associated energy ratios in frequency bands corresponding to the ratio of one directional component over the total energy. In case of one direction, as in a preferred embodiment, the energy ratio can be replaced by the diffuseness, which is complementary to the ratio of energy and then follow the original DirAC set of parameters. Since the directional component(s) is(are) in general expected to be less relevant than the diffuse part in inactive frames, it can be also transmitted on fewer bits using a coarser quantization scheme such as in active frames and/or by averaging the direction over time or frequency for getting a coarser time and /or frequency resolution. In a preferred embodiment, the direction may be sent every <NUM> instead of <NUM> for active frames but using the same frequency resolution of <NUM> non-uniform bands.

In a preferred embodiment, diffuseness 314a may be transmitted with same time/frequency as in active frames but on fewer bits, forcing a minimum quantization index. For example, if diffuseness 314a is quantized on <NUM> bits in active frames, it is then transmitted only on <NUM> bits, avoiding the transmission of original indices from <NUM> to <NUM>. The decoded index will be then added with an offset of +<NUM>.

It is also possible to completely avoid sending the direction 314b or alternatively avoid sending the diffuseness 314a and replace it at the decoder by a default or an estimated value, in some examples.

Moreover, one can consider to transmit an inter-channel coherence if input channels correspond to channels positioned the spatial domain. Inter-channel level differences are also an alternative to the directions.

More relevant is to send a surround coherence which is defined as the ratio of diffuse energy which is coherent in the sound field. It can be the exploited at the spatial renderer (DirAC synthesis) for example by redistributing the energy between direct and diffuse signals. The energy of surround coherent components is removed from the diffuse energy to be redistributed to the directional components which will be then panned more uniformly in the space.

Naturally, any combinations of the previously listed parameters could be considered for the inactive spatial parameters. It could be also envisioned for bit saving purposes, to not send any parameters in the inactive phase.

An exemplary pseudo code of the inactive spatial metadata encoder is given below:
<IMG>
<IMG>
<IMG>
<IMG>.

An exemplary pseudo code of the inactive spatial metadata decoder is given below:
<IMG>
<IMG>.

In case of SID during inactive phase, spatial parameters can be fully or partially decoded and then used for the subsequent DirAC synthesis.

In case of no data transmission or if no spatial parameters <NUM> are transmitted along with the transport channel said <NUM>, the spatial parameters <NUM> could need to be restituted. This can be achieved by synthetically generating the missing parameters <NUM> (e.g. <FIG>) by considering the past-received parameters (e.g. <NUM> and7or <NUM>). An unstable spatial image can be perceived has unpleasant, especially on background noise considered steady and not rapidly evolving. On the other hand, a strictly constant spatial image may be perceived as unnatural. Different strategies can be applied:.

Some examples, provided above, are now discussed.

In a first embodiment the Comfort Noise Generator <NUM> (<NUM>) is done in the core decoder as depicted in <FIG>. The resulting comfort noises are injected in the transport channels and then spatialized in the DirAC synthesis with the help of the transmitted inactive spatial parameters <NUM> or in case of non-transmission, using the spatial parameters <NUM> deduced as previously described. The spatialization may then be realized the way as described earlier, e.g. by generating two streams, a directional and a non-directional, which are derived from the decoded transport channels, and in case of inactive frames from the transport channel comfort noises. The two streams are then upmixed and mixed together at block <NUM> depending on the spatial parameters <NUM>.

Alternatively the comfort noise or a part of it, could be directly generated within the DirAC Synthesis in the filterbank domain. Indeed DirAC may control the coherence of the restituted scene with the help of the transport channels <NUM>, the spatial parameters <NUM>, <NUM>, <NUM>, and some decorrelators (e.g. <NUM>). The decorrelators <NUM> may reduce the coherence of the synthesized sound field. The spatial image is then perceived with more width, depth, diffusion, reverberation or externalization in case of headphone reproduction. However, decorrelators are often prone to typical audible artefacts, and it is desirable to reduce their use. This can be achieved for example by the so-called co-variance synthesis method [<NUM>] by exploiting the already existing incoherent component of the transport channels. However, this approach may have limitations, especially in case of a monophonic transport channel.

In case of comfort noise generated by random noise, it is advantageous to generate for each output channels, or at least a subset of them, a dedicated comfort noise. More specifically, it is advantageous to apply the comfort noise generation not only on the transport channels but also to the intermediate audio channels used in the spatial renderer (DirAC synthesis) <NUM> (and in the mixing block <NUM>). The decorrelation of the diffuse field will then be directly given by using different noise generators, rather than using the decorrelators <NUM>, which can lower the amount of artefacts but also the overall complexity. Indeed different realizations of a random noise are by definition decorrelated. <FIG> and <FIG> illustrates two ways of achievement this, by generating the comfort noise completely or partly within the spatial renderer <NUM>. In <FIG>, the CN is done in frequency domain as described in [<NUM>], it can be directly generated with the filterbank domain of the spatial renderer avoiding both the filterbank analysis <NUM> and the decorrelators <NUM>. Here, K the number of channels for which a comfort noise is generated is the equal or greater than M, the number of transport channels, and lower or equal than N the number of output channels. In the simplest case, K=N.

<FIG> illustrates another alternative to include comfort noise generation <NUM> in the renderer.

The comfort noise generation is split between inside (at <NUM>) and outside (at <NUM>) the spatial renderer <NUM>. The comfort noise 228d within the renderer <NUM> is added (at adder <NUM>) to eventual decorrelator output 228a. For example, low band can be generate outside in the same domain as in the core coder in order to be able to update easily the necessary memories. On the other hand, the comfort noise generation can be performed directly in the renderer for high frequencies.

Further, the comfort noise generation can be also apply during active frames <NUM>. Instead of switching off completely the comfort noise generation during active frames <NUM>, it can be kept active by reducing its strength. It serves then masking the transition between active and inactive frames, also masking artefacts and imperfections of both the core coder and the parametric spatial audio model. This was proposed in [<NUM>] for monophonic speech coding. Same principle can be extend to spatial speech coding. <FIG> illustrates an implementation. This time the comfort noise generations in the spatial renderer <NUM> is switched on both active and inactive phase. In inactive phase <NUM>, it is complementary to the comfort noise generation performed in the transport channels. In the renderer, the comfort noise is done on K channels equal or greater the M transport channels aiming to reduce the use of the decorrelators. The comfort noise generation in the spatial renderer <NUM> are added to upmixed version 228f of the transport channels, which can be achieved by a simple copy of the M channels into the K channels.

Embodiments of the present invention allow extending DTX to parametric spatial audio coding in an efficient way. It can restitute with a high perceptual fidelity the background noise even for inactive frames for which the transmission can be interrupted for communication bandwidth saving.

For this, the SID of the transport channels is extended by inactive spatial parameters relevant for describing the spatial image of the background noise. The generated comfort noise is applied in the transport channels before being spatialized by the renderer (DirAC synthesis). Alternatively, for an improvement in quality the CNG can be applied to more channels than the transport channels within the rendering. It allows complexity saving and reducing the annoyance of the decorrelator artefacts.

It is to be mentioned here that all alternatives or aspects as discussed before and all aspects as defined by independent aspects in the following aspects can be used individually, i.e., without any other alternative or object than the contemplated alternative, object or independent aspect. However, in other embodiments, two or more of the alternatives or the aspects or the independent aspects can be combined with each other and, in other embodiments, all aspects, or alternatives and all independent aspects can be combined to each other.

An inventively encoded signal can be stored on a digital storage medium or a non-transitory storage medium or can be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier or a non-transitory storage medium.

The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent aspects and not by the specific details presented by way of description and explanation of the embodiments herein.

Claim 1:
Apparatus (<NUM>) for generating an encoded audio scene (<NUM>) from an audio signal (<NUM>) having a first frame (<NUM>) and a second frame (<NUM>), comprising:
a soundfield parameter generator (<NUM>) for determining a first soundfield parameter representation (<NUM>) for the first frame (<NUM>) from the audio signal (<NUM>) in the first frame (<NUM>) and a second soundfield parameter representation (<NUM>) for the second frame (<NUM>) from the audio signal (<NUM>) in the second frame (<NUM>); and
an activity detector (<NUM>) for analyzing the audio signal (<NUM>) to determine, depending on the audio signal (<NUM>), that the first frame is an active frame (<NUM>) and the second frame is an inactive frame (<NUM>),
wherein the soundfield parameter generator (<NUM>) is configured to determine, from the second frame (<NUM>) of the audio signal, a plurality of individual sound sources and to determine, for each sound source, a parametric description (<NUM>) for the second frame,
wherein the soundfield parameter generator (<NUM>) is configured to decompose the second frame (<NUM>) into a plurality of frequency bins, each frequency bin representing an individual sound source of the plurality of individual sound sources, and to determine, for each frequency bin, at least one inactive spatial parameter as the second soundfield parameter representation (<NUM>) for the second frame (<NUM>), the at least one inactive spatial parameter comprising a direction parameter, a direction of arrival parameter, a diffuseness parameter, or an energy ratio parameter,
the apparatus further comprising:
an audio signal encoder (<NUM>) for generating an encoded audio signal (<NUM>), the encoded audio signal (<NUM>) providing an encoded audio signal (<NUM>) for the first frame being the active frame (<NUM>) and the parametric description (<NUM>) for the second frame being the inactive frame (<NUM>); and
an encoded signal former (<NUM>) for composing the encoded audio scene (<NUM>) by bringing together the first soundfield parameter representation (<NUM>) for the first frame (<NUM>), the second soundfield parameter representation (<NUM>) for the second frame (<NUM>), the encoded audio signal (<NUM>) for the first frame (<NUM>), and the parametric description (<NUM>) for the second frame (<NUM>).