Patent Description:
A spatial audio signal decoder performs operations to convert spatial audio signals from an input spatial audio format to an output spatial audio format. Known spatial audio signal format decoding techniques include passive decoding and active decoding. A passive signal decoder performs decoding operations that are based upon the input spatial audio signal format and the output spatial audio signal format and, in some examples, can use external parameters such as frequency but may not depend upon spatial characteristics of the audio input signal itself, such as the direction of arrival of audio sources in the audio input signal, for example. In other words, a passive signal decoder performs one or more operations independent of the spatial characteristics of the input signal.

An active signal decoder, on the other hand, performs decoding operations that are based upon the input spatial audio signal format, the output spatial audio signal format and perhaps external parameters such as frequency, for example, as well as spatial characteristics of the audio input signal. An active signal decoder can perform operations that are adapted to the spatial characteristics of the audio input signal.

Active and passive signal decoders lack universality. Passive signal decoders often blur directional audio sources. For example, passive signal decoders sometimes render a discrete point source in an input audio signal format to all of the channels of an output spatial audio format (corresponding to an audio playback system) instead of to a subset localized to the point-source direction. Active signal decoders, on the other hand, often focus diffuse sources by modeling such sources as directional, for example, as a small number of acoustic plane waves. As a result, an active signal decoder sometimes imparts directionality to nondirectional audio signals. For example, an active signal decoder sometimes renders nondirectional reverberations from a particular direction in an output spatial audio format (corresponding to an audio playback system) such that the spatial characteristics of the reverberation are not preserved by the decoder.

<NPL> discusses B-format audio having four microphone channels with different directional characteristics: one omnidirectional, and three channels directed towards orthogonal axis. The document further discusses analyzing diffuseness and direction of the B-formal signal for a plurality of subbands. Diffuseness is discussed as being a fraction of the total sound field energy that is due to the diffuse sound. The remaining portion, that is, the non-diffuse part, is approximated as a plane wave from the analyzed direction using the B-formal signal. The non-diffuse and the diffuse sound streams are calculated by multiplying each virtual microphone signal by specific coefficients. Synthesis of the loudspeakers signals is performed by distributing the omnidirectional microphone signal to the loudspeakers according to the analyzed direction and diffuseness in frequency bands.

Document <CIT> discloses methods and apparatus that optimally represent full 3D audio mixes (e.g., azimuth, elevation, and depth) as "sound scenes" in which the decoding process facilitates head tracking. Sound scene rendering can be modified for the listener's orientation (e.g., yaw, pitch, roll) and 3D position (e.g., x, y, z). The document also discusses ambisonics.

The invention provides for a method with the features of claim <NUM>, a computer-readable storage medium with the features of claim <NUM> and a computing apparatus with the features of claim <NUM>. Preferred embodiments of the invention are identified in the dependent claims.

The present inventors have recognized, among other things, that a problem to be solved includes optimizing audio signal rendering for omnidirectional components of an ambisonic mix, such as using active and passive decoders. In an example, some decoders may not correctly handle directionless or zero-distance information in ambisonic signals, such as "in-head" or "voice-of-god" type directionless signals. Some examples of potentially problematic signals include a 0th (zeroth) order (e.g., W-component only) ambisonic signals, or ambisonics that exhibit symmetry about one or more axes in three-dimensional space. In an example, ambisonic signals where the same signal is symmetrically panned in multiple directions or across nulls of the steering signals can be decoded improperly, such as when an active decoder makes assumptions about directionality. The present inventors have recognized that a solution can include, among other things, separating omnidirectional or middle components from an ambisonic signal, and using separate active and passive decoders for directional and omnidirectional components, respectively.

In an example, a 0th order or directionless signal, such as can be intended for diffuse or middle-panned rendering, can be added to a W-component of an ambisonic signal. To later decode the signal and recover the 0th order component, information about an energy ratio in the X, Y, and Z components can be used to renormalize the W component, such as with the remainder extracted as the 0th order or omnidirectional component. The renormalized directional part can then be decoded, such as using Harpex-like analysis or other techniques. In an example, the extracted part (which, for example, may have been intentionally added to the W-component) can be passively decoded or can be rendered using a volumetric method to diffuse and distribute the sound energy. In an example, directional but volumetric sources can be included in a mix by applying regular ambisonic panning and processing on multiple decorrelated signals.

This Brief Summary is intended to provide a summary of some of the present subject matter. The detailed description includes further information and explanation of the present subject matter.

Terms such as spatial encoding, spatial coding, or spatial audio coding can refer to representing a sound scene or soundfield in terms of audio signals and side information. The terms spatial format or spatial audio format or spatial audio signal can refer to audio signals and side information that represent a sound scene or soundfield. In an example, side information can include a definition of a format, such as directional characteristics corresponding to each of the audio channels in the format, and in some cases, can include signal-dependent information such as directions of sources present in audio signals. A spatial audio signal includes one or more constituents that may be referred to as audio signal components, or audio channels or sources. In some examples, a spatial audio signal may be referred to as an audio signal in a spatial format.

Spatial decoding, or spatial audio decoding, can include processing an input spatial audio signal in a specified spatial audio format to generate an output spatial audio signal in a specified spatial audio format. In an example, decoding can correspond to transcoding from an input spatial audio format to a different spatial audio format or decoding can include generating signals for playback over a specified audio reproduction system, such as a multichannel loudspeaker layout. An audio reproduction system can, in some examples, correspond to a spatial audio format or immersive format, such as a surround sound format.

In an example, active or parametric spatial audio systems can be configured to make determinations or assumptions about a direction of one or more audio signal sources. Some sources, however, can be intended to be rendered or reproduced with substantially no distance, that is, they can be considered to originate from essentially all directions simultaneously. In some examples, such signals or sources can be sourced from various panned objects. Examples discussed herein can encode such sources into an ambisonic mix such that the sources can be properly decoded downstream by active or parametric decoders.

In an example, information about one or more of an omnidirectional, "middle," or non-diegetic component can be added to an ambisonic source signal, or ambisonic source mix. That is, such signals or components can be encoded together with other directional components in an ambisonic mix, such as without using extra channels or metadata. A decoder receiving such a mix can identify residual energy, or 0th (zeroth) order energy, such as before or after accounting for diffuse or directional components in the same mix. In this example, the residual energy can be rendered as either a middle channel (e.g., having non-directional components, having zero distance, or being a "voice-of-god" type channel) or as an oiniii-voluinetric signal.

In another example, information from an ambisonic source can be pre-rendered as a volumetric source. In this example, omnidirectional components can be extracted by a parametric decoder as a diffuse component in a desired or intended number of different directions.

<FIG> illustrates generally a first block diagram <NUM> that includes a representation of operation of a spatial audio decoder <NUM> to decode an input spatial audio signal <NUM> in an input spatial audio format <NUM>. The spatial audio decoder <NUM> can be configured to decode the input spatial audio signal <NUM> to an output spatial audio signal <NUM> such as in an output spatial audio format suitable for a multichannel audio reproduction system <NUM>.

In an example, the spatial audio decoder <NUM> can transform an input signal, such as can be received in a first-order ambisonics B-format or other spatial audio format, to an output signal in a multichannel audio format suitable for playback using the multichannel audio reproduction system <NUM>. In an example, the spatial audio decoder <NUM> can be implemented as an active or passive decoder.

In the case of a passive decoder, the decoder can perform transformation operations from the input spatial format to the output spatial format independent of spatial characteristics of the audio input signal, such as direction of arrival of the audio input signal. In an example, the spatial audio decoder <NUM> can be implemented as an active decoder that performs transformation operations from the input spatial format to the output spatial format based at least in part upon spatial characteristics of the audio input signal, or the input spatial audio signal <NUM>. In an example, a rendered output using the multichannel audio reproduction system <NUM> can include information about a point source <NUM>, such as can be decoded using an active decoder, can include information about a volumetric source <NUM>, such as can be decoded using an active or passive decoder, or can include information about a middle source (e.g., at the reference listening position), such as can be decoded using an active or passive decoder.

<FIG> illustrates generally a spatial audio signal decoder example <NUM>. The decoder example <NUM> can represent a configuration of the spatial audio decoder <NUM> from the example of <FIG>. In an example, the decoder can be configured to map an input spatial audio signal in an input spatial format to an output spatial audio signal in an output spatial format. As further described below with refence to <FIG>, one example decoder can be configured as an active signal decoder <NUM>, and another example decoder can be configured as a passive signal decoder <NUM>. It can be appreciated that each input spatial audio signal can include multiple audio signal components and that each output spatial audio signal can include multiple audio signal components. The respective audio signal components may be referred to as channels. The example decoder of <FIG> can include or use one or more mapping operations to map M input spatial audio signal components to N spatial audio output signal components. More particularly, an example mapping operation includes an M-by-N spatial decoder matrix to map M input spatial audio signal components in an input spatial format to N spatial audio output signal components in an output spatial format. The mapping operations can be used as a basis to configure the decoder as an active signal decoder or a passive signal decoder.

In the example of the spatial audio decoder <NUM>, the value of M is four since the input spatial format is a first-order ambisonics B-format signal that has four signal components, and the value of N depends, for example, on the number of speakers in the multichannel audio reproduction system. The spatial format of the input spatial audio signal received by the example signal decoder can include B-format audio input signal components W, X, Y, and Z, such as having directivity patterns given by the respective elements in the vector (d)(Q) defined as <MAT> where Ω corresponds to an angular pair consisting of an azimuth angle θ and an elevation angle ϕ with respect to a reference point for measurement. A spatial audio scene or soundfield can be encoded in the W, X, Y, and Z components in accordance with the directivity patterns defined in the above vector. For instance, a point source S at azimuth angle θ and elevation angle ϕ is encoded in B-format components as <MAT>.

Thus ambisonics, such as B-format ambisonics, can be used to represent a soundfield by capturing or encoding a fixed set of signals corresponding to a single point in the soundfield. Each of the fixed set of signals in an ambisonic representation has a defined directivity pattern. The directivity patterns are designed such that ambisonic-encoded signals carry directional information for all of the sounds in an entire soundfield. An ambisonic encoder (not shown) encodes a soundfield in an ambisonic format, and the format can be independent from a specific loudspeaker layout which may be used to reconstruct an encoded soundfield. An ambisonic decoder decodes ambisonic format signals for a specific loudspeaker layout. <NPL>, provides a general explanation of ambisonics.

In some examples, a signal decoder transforms an input audio signal in an input spatial format to an output audio signal in an output spatial format suitable for a five-loudspeaker layout as depicted in <FIG>. The examples are not limited to the multichannel loudspeaker layout depicted in <FIG>, however. Example signal decoders can be configured to decode to a <NUM> loudspeaker layout, a <NUM> loudspeaker layout, an <NUM> loudspeaker layout, or other loudspeaker layout, for example. In other examples, the signal decoder transforms an input audio signal in an input spatial format to an output audio signal in a two-channel binaural format. The examples are not limited to input audio signals in the first-order ambisonics B-format. In other examples, the signal decoder transforms an input audio signal in a higher-order ambisonics format to an output audio signal in an output spatial format. In an example, a decoder or other processor can include or use headtracking information to further enhance a playback experience for a listener.

<FIG> illustrates generally a schematic block diagram of a spatial audio signal decoder system <NUM>. The spatial audio signal decoder system <NUM> can include a computer or processor system that includes one or more processor devices configured to be operatively coupled to one or more computer memory storage devices (e.g., a non-transitory storage device) that store instructions to configure the processor devices to provide the processing blocks described with reference to <FIG>. In the example of <FIG>, the spatial audio signal decoder system <NUM> can include one or more of a time-frequency transformation block <NUM>, an active/passive decomposition block <NUM>, an active signal decoder <NUM>, a passive signal decoder <NUM>, a combiner block <NUM>, and an inverse time-frequency transformation block <NUM>.

The time-frequency transformation block <NUM> can be configured to receive a time-domain input spatial audio signal, such as the input signal <NUM> in an input spatial audio format, and can convert the input signals to a time-frequency domain input signal <NUM>. Subsequent processing is carried out in a corresponding time-frequency domain. An alternative example first spatial audio signal decoder system (not shown) omits the time-frequency transformation block so that subsequent processing is carried out in the time domain.

The active/passive decomposition block <NUM> can be configured to decompose the time-frequency domain input signal <NUM>, such as in the time-frequency domain, to produce an active input component <NUM> and passive input component <NUM>. In an example, the active input component <NUM> and the passive input component <NUM> can sum or add up to the time-frequency domain input signal <NUM>. In an example, each of the active input component <NUM> and the passive input component <NUM> can have respective multiple components or component signals. In the spatial audio signal decoder system <NUM>, the components can be in the same spatial audio format as the input signal <NUM> as received in the time domain.

In an example, the input signal <NUM> can include an ambisonic signal such as can include X, Y, Z, and W components. In an example, the input signal <NUM> can comprise an ambisonic signal with an excess W component, such as can result from a multidirectional source signal having symmetry along one or more axes (e.g., axes X, Y, and Z of a three-dimensional space). For example, a phantom center signal constructed from identical sources that are evenly panned on either side of a listener can be such a source. Such a phantom center signal can have a primary direction but can have a more ambiguous or less certain position.

In the example spatial audio signal decoder system <NUM>, the active signal decoder <NUM> and the passive signal decoder <NUM> can transform their respective inputs from an input spatial format to respective decoder output spatial audio signals, such as including active decoder output signals <NUM> and passive decoder output signals <NUM>, that can have a common output spatial format such as a common multichannel loudspeaker layout. In another example decoder (not shown), different respective ones of the decoders can transform respective decoder input audio signals to respective decoder output audio signals having different spatial formats.

In an example, the active signal decoder <NUM> can receive the active input component <NUM> and, in response, provide the active decoder output signals <NUM>. The output format of the active decoder output signals <NUM> can be determined by a configuration of the active signal decoder <NUM>. For example, a feature of ambisonics and other spatial audio encoding methods is to be agnostic to the output format, meaning the input spatial audio signal can be decoded to whatever format the decoder is configured to provide. The active signal decoder <NUM> can transform the active input component <NUM>, such as can have a respective input spatial format, to active decoder output signals <NUM> having a particular or designated active signal output spatial format. The passive signal decoder <NUM> can receive the passive input component <NUM> and, in response, provide the passive decoder output signals <NUM>. The output format of the passive decoder output signals <NUM> can be determined by a configuration of the passive signal decoder <NUM>. The passive signal decoder <NUM> can transform the passive input component <NUM>, such as having a respective input spatial format, to the passive decoder output signals <NUM>, such as can have a specified passive signal output spatial format.

In an example, the passive signal decoder <NUM> can partition the passive input component <NUM> into one or more frequency bands such that different processing can be applied to each frequency band. For instance, an example passive signal decoder <NUM> can be configured to perform a lower frequency range transformation operation for a frequency range of the passive input component <NUM> below a cutoff frequency and is configured to perform an upper frequency range transformation operation for a frequency range of the passive input component <NUM> above the cutoff frequency.

In an example, the passive signal decoder <NUM> can be configured to apply decorrelation processing or filtering to at least a portion of the passive input component <NUM> to thereby provide a decorrelated passive decoder output signals <NUM>. In an example, the passive signal decoder <NUM> can render the decorrelated signals as an envelopment source. An envelopment source can include sound source information that originates, or is perceived by a listener to originate, from essentially all directions, and may or may not change with listener movement in any direction. In an example, a reverb signal can be considered to be enveloping source or an envelopment source.

In an example, the combiner block <NUM> can include a summation circuit to sum the respective output spatial audio signals from the decoders, such as the active decoder output signals <NUM> and the passive decoder output signals <NUM>. The combiner block <NUM> can provide summed output signals <NUM> representing the combination of the active decoder output signals <NUM> and the passive decoder output signals <NUM>. In an example, the combiner block <NUM> can perform other processing such as filtering or decorrelation. Filters, such as all-pass filters or others, can be applied to one or more channels of the passive decoder output signals <NUM> to decorrelate the channels prior to the combination with the active decoder output signals <NUM> by the combiner block <NUM>. Decorrelation of the channels can lead to a more diffuse and less directional rendering, which is generally preferable for the passive or omnidirectional components. In an example, additional processing of the decoded signal components can be carried out before combining the decoded signal components. For example, different filters may be applied to the active and passive components. In another example, additional processing of the decoded signal components can be carried out after combining the decoded signal components, for example, equalization or other filtering can be performed or applied.

In an example, the inverse time-frequency transformation block <NUM> can receive the summed output signals <NUM> from the combiner block <NUM>. The inverse time-frequency transformation block <NUM> can be configured to convert the summed output signals <NUM>, such as in the time-frequency domain, to a time-domain output, or time-domain spatial audio output signals <NUM>. The time-domain spatial audio output signals <NUM> can be provided to a sound reproduction system, such as the multichannel audio reproduction system <NUM> from the example of <FIG>, or can be provided to another signal processor or network for distribution.

<FIG> illustrates generally an example <NUM> of a first decomposition block <NUM>. In an example, the active/passive decomposition block <NUM> from the example of <FIG> comprises or corresponds to the first decomposition block <NUM>. The first decomposition block <NUM> can include a 0th order determination block <NUM> and a normalization block <NUM>.

In an example, the 0th order determination block <NUM> is configured to receive the time-frequency domain input signal <NUM> such as comprising an ambisonic signal. The 0th order determination block <NUM> can be configured to perform various processing operations, as discussed elsewhere herein, to provide a directional component <NUM> and an omnidirectional component <NUM>. The directional component <NUM> can include, for example, a first order ambisonic signal such as comprising X, Y, Z, and W components. In some examples, the W component of the directional component <NUM> can be unbalanced relative to the other components of the directional component <NUM>. The normalization block <NUM> can be configured to analyze the X, Y, and Z components and re-scale or normalize the W component to thereby provide a normalized output <NUM> that comprises balanced W, X, Y, and Z components. That is, the normalization block <NUM> can be used to prepare an ambisonic signal for output by correcting a perceived imbalance in energy with respect to the X, Y, Z, and W components of the signal.

In an example, the normalized output <NUM> includes or comprises the active input component <NUM> such as can be provided to the active signal decoder <NUM>. In another example, the normalization block <NUM> can be configured to prepare or provide an auxiliary ambisonic signal at its output by subtracting the omnidirectional component <NUM> from the directional component <NUM>. In this example, the resulting auxiliary ambisonic signal can include one or more non-zero directional components.

The omnidirectional component <NUM> can include or represent information about an omnidirectional signal, or less-directional signal, that is present in the time-frequency domain input signal <NUM>. The omnidirectional component <NUM> can include or comprise the passive input component <NUM> such as can be provided to the passive signal decoder <NUM>. In an example, the omnidirectional component <NUM> can include a middle channel, or in-head channel, or "voice of god" channel that can be rendered by the passive signal decoder <NUM>.

In an example, the omnidirectional component <NUM> can comprise an ambisonic signal with a W component and, in some examples, can omit X, Y, or Z components. In an example, a multidirectional source with perfect symmetry about all directional axes can comprise an ambisonic signal with only a W component. Generally, in the discussions herein, a "W-only" signal can be considered a 0th order ambisonic signal.

<FIG> illustrates generally an example <NUM> of a second decomposition block <NUM>. In an example, the active/passive decomposition block <NUM> from the example of <FIG>, or the first decomposition block <NUM> from the example of <FIG>, comprises or corresponds to the second decomposition block <NUM>.

The second decomposition block <NUM> can be configured to receive the time-frequency domain input signal <NUM> and route its various components to one or more of a direction estimation block <NUM>, a subspace determination block <NUM>, or the 0th order determination block <NUM>. The direction estimation block <NUM> can provide an estimate <NUM> of a number and direction of arrival (DOA) of directional audio sources in the time-frequency domain input signal <NUM>, in accordance with an input spatial audio format. The subspace determination block <NUM> can determine the active input component <NUM> based upon the estimate <NUM> of the number and DOAs of directional sound sources in the time-frequency domain input signal <NUM>. An example subspace determination block <NUM> determines the active input component <NUM> by projecting the active signal component onto a subspace that can be determined based on the number and DOAs of directional sound sources and the time-frequency domain input signal <NUM>.

The 0th order determination block <NUM> can be used to determine a value or values that can be used to scale a residual, passive, or omnidirectional component of the time-frequency domain input signal <NUM>. In an example, the 0th order determination block <NUM> can determine or provide a scale factor for scaling information between the active input component <NUM> and the passive input component <NUM>. The residual determination block <NUM> can determine the passive input component <NUM>, for example, based on a difference between the time-frequency domain input signal <NUM> and the active input component <NUM> as-determined by the subspace determination block <NUM>.

In an alternative example decomposition block, the passive input component <NUM> can be determined first, and the active input component <NUM> can be determined thereafter based upon a difference between the received time-frequency domain input signal <NUM> and the passive input component <NUM>.

As a result of processing using the second decomposition block <NUM>, active signals or the active input component <NUM> can include functionally directional source signals. The passive input component <NUM> can include a combination of non-directional or omnidirectional sources, such as can be detected using the 0th order determination block <NUM>. In an example, the passive input component <NUM> can include diffuse or residual error-based signals as identified by the subspace determination block <NUM> and the residual determination block <NUM>. Other examples of determining residual error and allocating residual signals are discussed by <NPL>," <CIT>'').

<FIG> illustrates generally an example of a first method <NUM> that can include processing an input ambisonic signal to separate omnidirectional components that can be rendered using a passive decoder from other, directional components that can be rendered using an active decoder. At block <NUM>, the example first method <NUM> can include receiving a B-format primary ambisonic signal such as including W, X, Y, and Z components. In an example, block <NUM> can include receiving the input spatial audio signal <NUM> such as comprising the input signal <NUM>. In an example, block <NUM> can include receiving the input signal <NUM> using one or more of the active/passive decomposition block <NUM>, the first decomposition block <NUM>, or the second decomposition block <NUM>.

At block <NUM>, the first method <NUM> can include determining, such as for a first signal band or frequency band of the primary ambisonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal. In an example, the active/passive decomposition block <NUM> can be configured to receive the input signal <NUM> and analyze the X, Y, Z, and W components of the input signal <NUM> such as to identify any imbalance of the W component relative to the X, Y, and Z components. If a mismatch or imbalance exists in the input signal <NUM> then the input signal <NUM> can be considered to have an omnidirectional component <NUM> that can be extracted from the input signal <NUM> such as for subsequent processing that can be different than the processing used for the other, directional component <NUM>.

At block <NUM>, the first method <NUM> can include determining a secondary ambisonic signal using information about a difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal. In an example, the active/passive decomposition block <NUM> can include a signal processor configured to determine or generate the secondary ambisonic signal to represent the omnidirectional component <NUM>. In an example, the secondary ambisonic signal can include a 0th order ambisonic signal, or can include a single channel signal that includes exclusively omnidirectional signal information.

In an example, block <NUM> can include determining a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the input signal <NUM>. For example, determining the total energy can include determining whether a sum of squared magnitudes of the X, Y, and Z components is substantially equal to a square magnitude of the W component. In other words, (X^<NUM> + Y^<NUM> + Z^<NUM>)/2w^<NUM> should be identically <NUM>. In another example, determining the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the input signal <NUM> can include determining whether component attributes of the input signal <NUM> are mathematically equivalent to solutions of assumed panning equations and normalizations for the input signal <NUM>, such as can have different values or requirements depending on a format used.

At block <NUM>, the first method <NUM> can include generating an output signal that is based on the secondary ambisonic signal, such as using the passive signal decoder <NUM>. In an example, the output can be a middle channel, or in-head type signal, sometimes referred to a "voice of god" signal because it can be interpreted by a listener as either originating inside the listener's head or as originating from all directions simultaneously.

At block <NUM>, the first method <NUM> can include determining or generating a re-balanced primary ambisonic signal based on a difference between the primary ambisonic signal as-received and the secondary ambisonic signal such as determined at block <NUM>. In an example, block <NUM> can include using the normalization block <NUM> to scale the primary ambisonic signal or to generate a W component that corresponds to the X, Y, and Z components of the primary ambisonic signal, such as received at block <NUM>. At block <NUM>, the first method <NUM> can include generating an output signal that is based on the re-balanced primary ambisonic signal, such as using the active signal decoder <NUM>.

<FIG> illustrates generally an example of a second method <NUM> that can include generating a secondary ambisonic signal, such as using the active/passive decomposition block <NUM>. The second method <NUM> can begin at block <NUM>, which is described above in the discussion of <FIG>. At block <NUM>, the second method <NUM> can include identifying one or more first-order or higher-order ambisonic steering components for the secondary ambisonic signal. The one or more directional steering components can be based on respective steering components from the primary ambisonic signal.

For example, at block <NUM>, the second method <NUM> can include computing a residual error projection matrix for the W, X, Y, and Z components of the primary ambisonic signal, such as received at block <NUM>. A residual error projection matrix can be computed or determined according to the systems and methods discussed by Goodwin et al. The residual can represent a passive signal component as a difference between an input signal and the primary ambisonic signal.

At block <NUM>, the second method <NUM> can include determining an omni-signal return factor. The omni-signal return factor can be based on the determined difference between the total energy of X, Y, and Z components and the total energy of the W component of the input signal.

At block <NUM>, the second method <NUM> can include updating the residual error projection matrix using the determined omni-signal return factor from block <NUM>. That is, the omni-signal return factor from block <NUM> can indicate a total energy difference (e.g., an excess) of the W component relative to the X, Y, and Z components. Updating the projection matrix can include rebalancing the values such that the residual or excess W component is removed. In an example, the matrix can be rebalanced such that any excess W component is allocated to the residual or passive component.

At block <NUM>, the second method <NUM> can include applying the updated residual error projection matrix to the primary ambisonic signal to thereby generate the secondary ambisonic signal. The generated secondary ambisonic signal can be provided to the passive signal decoder <NUM> for further processing or rendering.

Goodwin et al. explains generally various systems and methods for determining a residual signal, or portions of an input signal that may not be adequately represented by directional components that can be detected in the input. For example, an input signal including a front-left source with some volumetric attribute (e.g., not a pure point source) may have a residual. The decoder described by Goodwin et al. can, in an example, detect a center of the source and designate directionality based on the center (e.g., <NUM> degrees left). The decoder can then test the directionality hypothesis by forming a pure point source at the given direction (e.g., <NUM> degrees left). A difference between the pure point source and the original volumetric signal can, in theory, be designated as the residual and can thus be passively decoded (or, non-point source rendered), while the center or point source information can be actively decoded.

In an example that includes a phantom center signal (e.g., a source having substantially equal parts at -<NUM> degrees and +<NUM> degrees from front center), an ambisonic signal that includes the phantom signal will cancel for left-right balance. Since the source is intended to be front center, the decoder of Goodwin et al. can produce a strong front center signal. The residual can be minimized because the left-right balance was cancelled out. Any information that the source was a center image made of respective sources on the left and right can be hidden in the relationship to the magnitude, rather than the direction, of the signal. In an example, the present systems and methods can help identify or determine a spaciousness or breadth of the source and can move a different portion of the source to the passive decoder stream, which in turn can render the source more as a phantom center (as in the original input signal).

<FIG> illustrates generally an example of a third method <NUM> that can include preparing different ambisonic or other signals for further processing by an active or passive decoder, such as by the active signal decoder <NUM> or the passive signal decoder <NUM>. The third method <NUM> can be performed at least in part using the active/passive decomposition block <NUM> and can begin at block <NUM> with receiving a primary ambisonic signal having information about a volumetric source. The primary ambisonic signal can include or comprise the input signal <NUM>. The volumetric source can comprise a source with various directional cues and can have ambiguous or less-directional characteristics.

At block <NUM>, the third method <NUM> can include determining a secondary ambisonic signal having one or more non-zero directional components. The non-zero directional components can be based on the directional cues associated with volumetric source. In an example, the secondary ambisonic signal can represent predominantly the less-directional characteristics from the volumetric source.

At block <NUM>, the third method <NUM> can include generating a tertiary ambisonic signal. In an example, the tertiary ambisonic signal can be generated by subtracting the secondary ambisonic signal from the primary ambisonic signal such that the tertiary ambisonic signal includes one or more of the non-zero directional components.

At block <NUM>, the third method <NUM> can include outputting the one or more non-zero directional components of the tertiary ambisonic signal to an active decoder, such as to the active signal decoder <NUM>. Block <NUM> can include outputting the secondary ambisonic signal to a passive decoder, such as the passive signal decoder <NUM>.

One or more of the methods discussed herein can be used to process other-order ambisonic signals, or to process single-plane ambisonic signals. For example, an example of the input signal <NUM> can include or comprise a horizontal-only ambisonic signal comprising W, X, and Y components. In this case, any extracted or determined omnidirectional imbalance can be attributed to the "missing" Z component. A secondary ambisonic signal can be generated for W and Z, which could in turn be rendered as the missing height component.

<FIG> is a diagrammatic representation of a machine <NUM> within which instructions <NUM> (e.g., software, a program, an application, an applet, an app, or other executable code) for causing the machine <NUM> to perform any one or more of the methodologies discussed herein can be executed. For example, the instructions <NUM> can cause the machine <NUM> to execute any one or more of the methods described herein. The instructions <NUM> can transform the general, non-programmed machine <NUM> into a particular machine <NUM> programmed to carry out the described and illustrated functions in the manner described.

In an example, the machine <NUM> can operate as a standalone device or can be coupled (e.g., networked) to other machines or devices or processors. In a networked deployment, the machine <NUM> can operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine <NUM> can comprise a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions <NUM>, sequentially or otherwise, that specify actions to be taken by the machine <NUM>. Further, while only a single machine <NUM> is illustrated, the term "machine" can be taken to include a collection of machines that individually or jointly execute the instructions <NUM> to perform any one or more of the methodologies discussed herein. In an example, the instructions <NUM> can include instructions stored using a memory circuit, and the machine <NUM> can include or use a processor circuit such as can be associated with any one or more of the various blocks, modules, processors, or other processing hardware or software discussed herein.

The machine <NUM> can include various processors and processor circuitry, such as represented in the example of <FIG> as processors <NUM>, memory <NUM>, and I/O components <NUM>, which can be configured to communicate with each other via a bus <NUM>. In an example, the processors <NUM> (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an ASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) can include, for example, a processor <NUM> and a processor <NUM> that execute the instructions <NUM>. The term "processor" is intended to include multi-core processors that can comprise two or more independent processors (sometimes referred to as "cores") that can execute instructions contemporaneously. Although <FIG> shows multiple processors, the machine <NUM> can include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.

The memory <NUM> can include a main memory <NUM>, a static memory <NUM>, or a storage unit <NUM>, such as can be accessible to the processors <NUM> via the bus <NUM>. The memory <NUM>, the static memory <NUM>, and storage unit <NUM> can store the instructions <NUM> embodying any one or more of the methods or functions or processes described herein. The instructions <NUM> can also reside, completely or partially, within the main memory <NUM>, within the static memory <NUM>, within the machine-readable medium <NUM> within the storage unit <NUM>, within at least one of the processors (e.g., within a processor's cache memory), or any suitable combination thereof, during execution thereof by the machine <NUM>.

The I/O components <NUM> can include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components <NUM> that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones can include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components <NUM> can include many other components that are not shown in <FIG>. In various example embodiments, the I/O components <NUM> can include output components <NUM> and input components <NUM>. The output components <NUM> can include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The input components <NUM> can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and/or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

In an example, the I/O components <NUM> can include biometric components <NUM>, motion components <NUM>, environmental components <NUM>, or position components <NUM>, among a wide array of other components. For example, the biometric components <NUM> include components configured to detect a presence or absence of humans, pets, or other individuals or objects, or configured to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram-based identification), and the like. The motion components <NUM> can include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth.

The environmental components <NUM> can include, for example, illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors to detection concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components <NUM> include location sensor components (e.g., a GPS receiver component, an RFID tag, etc.), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude can be derived), orientation sensor components (e.g., magnetometers), and the like.

The I/O components <NUM> can include communication components <NUM> operable to couple the machine <NUM> to a network <NUM> or devices <NUM> via a coupling <NUM> and a coupling <NUM>, respectively. For example, the communication components <NUM> can include a network interface component or another suitable device to interface with the network <NUM>. In further examples, the communication components <NUM> can include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. The devices <NUM> can be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).

Moreover, the communication components <NUM> can detect identifiers or include components operable to detect identifiers. For example, the communication components <NUM> can include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information can be derived via the communication components <NUM>, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, or location via detecting an NFC beacon signal that can indicate a particular location, and so forth.

The various memories (e.g., memory <NUM>, main memory <NUM>, static memory <NUM>, and/or memory of the processors <NUM>) and/or storage unit <NUM> can store one or more instructions or data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions <NUM>), when executed by processors or processor circuitry, cause various operations to implement the embodiments discussed herein.

The instructions <NUM> can be transmitted or received over the network <NUM>, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components <NUM>) and using any one of a number of well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions <NUM> can be transmitted or received using a transmission medium via the coupling <NUM> (e.g., a peer-to-peer coupling) to the devices <NUM>.

This disclosure has been described in detail and with reference to exemplary embodiments thereof, it will be apparent to one skilled in the art that various changes and modifications can be made.

The drawings show specific embodiments by way of illustration. Moreover, the subject matter may include any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments can be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, the subject matter may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment, and it is contemplated that such embodiments can be combined with each other in various combinations or permutations.

Claim 1:
A method comprising:
receiving (<NUM>) a B-format primary ambisonic signal including W, X, Y, and Z components;
determining (<NUM>), for a first signal band of the primary ambisonic signal, a difference between a total energy of the X, Y, and Z components and a total energy of the W component of the primary ambisonic signal;
determining (<NUM>) a secondary ambisonic signal using information about the difference between the total energy of the X, Y, and Z components and the total energy of the W component of the primary ambisonic signal;
generating (<NUM>) an output signal based on the secondary ambisonic signal, the generating including scaling the secondary ambisonic signal for one or more of independent output, decoding, and spatial rendering;
wherein the method is characterised by:
generating (<NUM>) a re-balanced primary ambisonic signal based on a difference between the primary ambisonic signal as-received and the secondary ambisonic signal; and
generating (<NUM>) an output signal based on the re-balanced primary ambisonic signal.