Extended-range coarse-fine quantization for audio coding

A method of encoding audio data includes determining an energy level of a first subband of frequency domain audio data, determining a bit allocation for a coarse quantization process and a fine quantization process, determining that the energy level of the first subband of frequency domain audio data is outside a predetermined range of energy levels for the coarse quantization process, reallocating bits assigned to the fine quantization process to an extended-range coarse quantization process, the extended-range coarse quantization process using an extended range of energy levels, wherein the extended range of energy levels is larger than the predetermined range of energy levels for the coarse quantization process, and quantizing the energy level of the first subband of frequency domain audio data using the extended-range coarse quantization process to produce a quantized extended-range coarse energy level.

TECHNICAL FIELD

This disclosure relates to audio encoding and decoding.

BACKGROUND

Wireless networks for short-range communication, which may be referred to as “personal area networks,” are established to facilitate communication between a source device and a sink device. One example of a personal area network (PAN) protocol is Bluetooth®, which is often used to form a PAN for streaming audio data from the source device (e.g., a mobile phone) to the sink device (e.g., headphones or a speaker).

In some examples, the Bluetooth® protocol is used for streaming encoded or otherwise compressed audio data. In some examples, audio data is encoded using gain-shape vector quantization audio encoding techniques. In gain-shape vector quantization audio encoding, audio data is transformed into the frequency domain and then separated into subbands of transform coefficients. A scalar energy level (e.g., gain) of each subband is encoded separately from the shape (e.g., a residual vector of transform coefficients) of the subband.

SUMMARY

In general, this disclosure relates to techniques for performing extended-range fine-coarse quantization on a scalar energy level of a subband of frequency domain audio data. In some examples, the energy level of a subband is quantized in a two-step process. First, a certain number of bits are allocated for performing a coarse quantization process. The coarse quantization process is performed within a predetermined range of energy values (e.g., defined by maximum and minimum energy values). The quantized coarse energy is compared to the original energy of the subband and an error is computed. Then, using another number of bits, a fine quantization process is performed on the error to produce quantized fine energy. Together, the quantized coarse energy and the quantized fine energy represent the total energy level of the subband.

In some examples, the energy level of a subband may be outside of the predetermined range of energy levels for performing coarse quantization. In accordance with the techniques of this disclosure, an audio encoder may be configured to determine when the energy level of a subband is outside the predetermined range of energy levels for coarse quantization. If so, the audio encoder may reallocate bits assigned to the fine quantization process to an extended-range coarse quantization process. The extended-range coarse quantization process may be performed with an extended range of energy values in order to more accurately encode the energy level of the subband. The techniques of this disclosure may be used when quantizing the energy levels of one or more subbands of a frame of audio. That is, in some examples, the techniques of this disclosure may be used for a subset of subbands of a frame of audio data. In other examples, the techniques of this disclosure may be used with every subband of frame of audio data.

In this respect, the techniques may include a method of encoding audio data, the method comprising determining an energy level of a first subband of frequency domain audio data, determining a bit allocation for a coarse quantization process and a fine quantization process, determining that the energy level of the first subband of frequency domain audio data is outside a predetermined range of energy levels for the coarse quantization process, reallocating bits assigned to the fine quantization process to an extended-range coarse quantization process, the extended-range coarse quantization process using an extended range of energy levels, wherein the extended range of energy levels is larger than the predetermined range of energy levels for the coarse quantization process, and quantizing the energy level of the first subband of frequency domain audio data using the extended-range coarse quantization process to produce a quantized extended-range coarse energy level.

In another aspect, this disclosure describes an apparatus configured to encode audio data, the apparatus comprising a memory configured to store the audio data, and one or more processors in communication with the memory, the one or more processors configured to determine an energy level of a first subband of frequency domain audio data, determine a bit allocation for a coarse quantization process and a fine quantization process, determine that the energy level of the first subband of frequency domain audio data is outside a predetermined range of energy levels for the coarse quantization process, reallocate bits assigned to the fine quantization process to an extended-range coarse quantization process, the extended-range coarse quantization process using an extended range of energy levels, wherein the extended range of energy levels is larger than the predetermined range of energy levels for the coarse quantization process, and quantize the energy level of the first subband of frequency domain audio data using the extended-range coarse quantization process to produce a quantized extended-range coarse energy level.

In another aspect, this disclosure describes a method for decoding audio data comprising receiving a quantized coarse energy level for a subband of frequency domain audio data, receiving a syntax element that indicates if the quantized coarse energy level was quantized using an extended-range coarse quantization process, determining a scaling factor for performing an inverse quantization process based on the syntax element, and performing inverse quantization on the quantized coarse energy level with the determined scaling factor.

In another aspect, this disclosure describes an apparatus configured to decode audio data, the apparatus comprising a memory configured to store the audio data, and one or more processors in communication with the memory, the one or more processors configured to receive a quantized coarse energy level for a subband of frequency domain audio data, receive a syntax element that indicates if the quantized coarse energy level was quantized using an extended-range coarse quantization process, determine a scaling factor for performing an inverse quantization process based on the syntax element, and perform inverse quantization on the quantized coarse energy level with the determined scaling factor.

The details of one or more aspects of the techniques are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of these techniques will be apparent from the description and drawings, and from the claims.

DETAILED DESCRIPTION

FIG. 1is a diagram illustrating a system10that may perform various aspects of the techniques described in this disclosure for extended-range coarse-fine quantization of audio data. As shown in the example ofFIG. 1, the system10includes a source device12and a sink device14. Although described with respect to the source device12and the sink device14, the source device12may operate, in some instances, as the sink device, and the sink device14may, in these and other instances, operate as the source device. As such, the example of system10shown inFIG. 1is merely one example illustrative of various aspects of the techniques described in this disclosure.

In any event, the source device12may represent any form of computing device capable of implementing the techniques described in this disclosure, including a handset (or cellular phone), a tablet computer, a so-called smart phone, a remotely piloted aircraft (such as a so-called “drone”), a robot, a desktop computer, a receiver (such as an audio/visual—AV—receiver), a set-top box, a television (including so-called “smart televisions”), a media player (such as s digital video disc player, a streaming media player, a Blue-Ray Disc™ player, etc.), or any other device capable of communicating audio data wirelessly to a sink device via a personal area network (PAN). For purposes of illustration, the source device12is assumed to represent a smart phone.

The sink device14may represent any form of computing device capable of implementing the techniques described in this disclosure, including a handset (or cellular phone), a tablet computer, a smart phone, a desktop computer, a wireless headset (which may include wireless headphones that include or exclude a microphone, and so-called smart wireless headphones that include additional functionality such as fitness monitoring, on-board music storage and/or playback, dedicated cellular capabilities, etc.), a wireless speaker (including a so-called “smart speaker”), a watch (including so-called “smart watches”), or any other device capable of reproducing a soundfield based on audio data communicated wirelessly via the PAN. Also, for purposes of illustration, the sink device14is assumed to represent wireless headphones.

As shown in the example ofFIG. 1, the source device12includes one or more applications (“apps”)20A-20N (“apps20”), a mixing unit22, an audio encoder24, and a wireless connection manager26. Although not shown in the example ofFIG. 1, the source device12may include a number of other elements that support operation of apps20, including an operating system, various hardware and/or software interfaces (such as user interfaces, including graphical user interfaces), one or more processors, memory, storage devices, and the like.

Each of the apps20represent software (such as a collection of instructions stored to a non-transitory computer readable media) that configure the system10to provide some functionality when executed by the one or more processors of the source device12. The apps20may, to provide a few examples, provide messaging functionality (such as access to emails, text messaging, and/or video messaging), voice calling functionality, video conferencing functionality, calendar functionality, audio streaming functionality, direction functionality, mapping functionality, gaming functionality. Apps20may be first party applications designed and developed by the same company that designs and sells the operating system executed by the source device12(and often pre-installed on the source device12) or third-party applications accessible via a so-called “app store” or possibly pre-installed on the source device12. Each of the apps20, when executed, may output audio data21A-21N (“audio data21”), respectively. In some examples, the audio data21may be generated from a microphone (not pictured) connected to the source device12.

The mixing unit22represents a unit configured to mix one or more of audio data21A-21N (“audio data21”) output by the apps20(and other audio data output by the operating system—such as alerts or other tones, including keyboard press tones, ringtones, etc.) to generate mixed audio data23. Audio mixing may refer to a process whereby multiple sounds (as set forth in the audio data21) are combined into one or more channels. During mixing, the mixing unit22may also manipulate and/or enhance volume levels (which may also be referred to as “gain levels”), frequency content, and/or panoramic position of the audio data21. In the context of streaming the audio data21over a wireless PAN session, the mixing unit22may output the mixed audio data23to the audio encoder24.

The audio encoder24may represent a unit configured to encode the mixed audio data23and thereby obtain encoded audio data25. In some examples, the audio encoder24may encode individual ones of the audio data21. Referring for purposes of illustration to one example of the PAN protocols, Bluetooth® provides for a number of different types of audio codecs (which is a word resulting from combining the words “encoding” and “decoding”) and is extensible to include vendor specific audio codecs. The Advanced Audio Distribution Profile (A2DP) of Bluetooth® indicates that support for A2DP requires supporting a subband codec specified in A2DP. A2DP also supports codecs set forth in MPEG-1 Part 3 (MP2), MPEG-2 Part 3 (MP3), MPEG-2 Part 7 (advanced audio coding—AAC), MPEG-4 Part 3 (high efficiency-AAC—HE-AAC), and Adaptive Transform Acoustic Coding (ATRAC). Furthermore, as noted above, A2DP of Bluetooth® supports vendor specific codecs, such as aptX™ and various other versions of aptX (e.g., enhanced aptX—E-aptX, aptX live, and aptX high definition—aptX-HD).

The audio encoder24may operate consistent with one or more of any of the above listed audio codecs, as well as, audio codecs not listed above, but that operate to encode the mixed audio data23to obtain the encoded audio data25. The audio encoder24may output the encoded audio data25to one of the wireless communication units30(e.g., the wireless communication unit30A) managed by the wireless connection manager26. In accordance with example techniques of this disclosure that will be described in more detail below, the audio encoder24may be configured to encode the audio data21and/or the mixed audio data23using an extended-range coarse-fine quantization technique.

The wireless connection manager26may represent a unit configured to allocate bandwidth within certain frequencies of the available spectrum to the different ones of the wireless communication units30. For example, the Bluetooth® communication protocols operate over within the 2.5 GHz range of the spectrum, which overlaps with the range of the spectrum used by various WLAN communication protocols. The wireless connection manager26may allocate some portion of the bandwidth during a given time to the Bluetooth® protocol and different portions of the bandwidth during a different time to the overlapping WLAN protocols. The allocation of bandwidth and other is defined by a scheme27. The wireless connection manager40may expose various application programmer interfaces (APIs) by which to adjust the allocation of bandwidth and other aspects of the communication protocols so as to achieve a specified quality of service (QoS). That is, the wireless connection manager40may provide the API to adjust the scheme27by which to control operation of the wireless communication units30to achieve the specified QoS.

In other words, the wireless connection manager26may manage coexistence of multiple wireless communication units30that operate within the same spectrum, such as certain WLAN communication protocols and some PAN protocols as discussed above. The wireless connection manager26may include a coexistence scheme27(shown inFIG. 1as “scheme27”) that indicates when (e.g., an interval) and how many packets each of the wireless communication units30may send, the size of the packets sent, and the like.

The wireless communication units30may each represent a wireless communication unit30that operates in accordance with one or more communication protocols to communicate encoded audio data25via a transmission channel to the sink device14. In the example ofFIG. 1, the wireless communication unit30A is assumed for purposes of illustration to operate in accordance with the Bluetooth® suite of communication protocols. It is further assumed that the wireless communication unit30A operates in accordance with A2DP to establish a PAN link (over the transmission channel) to allow for delivery of the encoded audio data25from the source device12to the sink device14.

More information concerning the Bluetooth® suite of communication protocols can be found in a document entitled “Bluetooth Core Specification v 5.0,” published Dec. 6, 2016, and available at: www.bluetooth.org/en-us/specification/adopted-specifications. More information concerning A2DP can be found in a document entitled “Advanced Audio Distribution Profile Specification,” version 1.3.1, published on Jul. 14, 2015.

The wireless communication unit30A may output the encoded audio data25as a bitstream31to the sink device14via a transmission channel, which may be a wired or wireless channel, a data storage device, or the like. While shown inFIG. 1as being directly transmitted to the sink device14, the source device12may output the bitstream31to an intermediate device positioned between the source device12and the sink device14. The intermediate device may store the bitstream31for later delivery to the sink device14, which may request the bitstream31. The intermediate device may comprise a file server, a web server, a desktop computer, a laptop computer, a tablet computer, a mobile phone, a smart phone, or any other device capable of storing the bitstream31for later retrieval by an audio decoder. This intermediate device may reside in a content delivery network capable of streaming the bitstream31(and possibly in conjunction with transmitting a corresponding video data bitstream) to subscribers, such as the sink device14, requesting the bitstream31.

Alternatively, the source device12may store the bitstream31to a storage medium, such as a compact disc, a digital video disc, a high definition video disc or other storage media, most of which are capable of being read by a computer and therefore may be referred to as computer-readable storage media or non-transitory computer-readable storage media. In this context, the transmission channel may refer to those channels by which content stored to these mediums are transmitted (and may include retail stores and other store-based delivery mechanism). In any event, the techniques of this disclosure should not therefore be limited in this respect to the example ofFIG. 1.

As further shown in the example ofFIG. 1, the sink device14includes a wireless connection manager40that manages one or more of wireless communication units42A-42N (“wireless communication units42”) according to a scheme41, an audio decoder44, and one or more speakers48A-48N (“speakers48”). The wireless connection manager40may operate in a manner similar to that described above with respect to the wireless connection manager26, exposing an API to adjust scheme41by which operation of the wireless communication units42to achieve a specified QoS.

The wireless communication units42may be similar in operation to the wireless communication units30, except that the wireless communication units42operate reciprocally to the wireless communication units30to decapsulate the encoded audio data25. One of the wireless communication units42(e.g., the wireless communication unit42A) is assumed to operate in accordance with the Bluetooth® suite of communication protocols and reciprocal to the wireless communication protocol28A. The wireless communication unit42A may output the encoded audio data25to the audio decoder44.

The audio decoder44may operate in a manner that is reciprocal to the audio encoder24. The audio decoder44may operate consistent with one or more of any of the above listed audio codecs, as well as, audio codecs not listed above, but that operate to decode the encoded audio data25to obtain mixed audio data23′. The prime designation with respect to “mixed audio data23” denotes that there may be some loss due to quantization or other lossy operations that occur during encoding by the audio encoder24. The audio decoder44may output the mixed audio data23′ to one or more of the speakers48.

Each of the speakers48represent a transducer configured to reproduce a soundfield from the mixed audio data23′. The transducer may be integrated within the sink device14as shown in the example ofFIG. 1or may be communicatively coupled to the sink device14(via a wire or wirelessly). The speakers48may represent any form of speaker, such as a loudspeaker, a headphone speaker, or a speaker in an earbud. Furthermore, although described with respect to a transducer, the speakers48may represent other forms of speakers, such as the “speakers” used in bone conducting headphones that send vibrations to the upper jaw, which induces sound in the human aural system.

As noted above, the apps20may output audio data21to the mixing unit22. Prior to outputting the audio data21, the apps20may interface with the operating system to initialize an audio processing path for output via integrated speakers (not shown in the example ofFIG. 1) or a physical connection (such as a mini-stereo audio jack, which is also known as 3.5 millimeter—mm —minijack). As such, the audio processing path may be referred to as a wired audio processing path considering that the integrated speaker is connected by a wired connection similar to that provided by the physical connection via the mini-stereo audio jack. The wired audio processing path may represent hardware or a combination of hardware and software that processes the audio data21to achieve a target quality of service (QoS).

To illustrate, one of the apps20(which is assumed to be the app20A for purposes of illustration) may issue, when initializing or reinitializing the wired audio processing path, one or more request29A for a particular QoS for the audio data21A output by the app20A. The request29A may specify, as a couple of examples, a high latency (that results in high quality) wired audio processing path, a low latency (that may result in lower quality) wired audio processing path, or some intermediate latency wired audio processing path. The high latency wired audio processing path may also be referred to as a high quality wired audio processing path, while the low latency wired audio processing path may also be referred to as a low quality wired audio processing path.

FIG. 2is a block diagram illustrating an example of an audio encoder24configured to perform various aspects of the techniques described in this disclosure. The audio encoder24may be configured to encode audio data for transmission over a PAN (e.g., Bluetooth®). However, the techniques of this disclosure performed by the audio encoder24may be used in any context where the compression of audio data is desired. In some examples, the audio encoder24may be configured to encode the audio data21in accordance with as aptX™ audio codec, including, e.g., enhanced aptX—E-aptX, aptX live, and aptX high definition. However, the techniques of this disclosure may be used in any audio codec configured to perform coarse-fine quantization of an energy value of frequency domain audio data. As will be explained in more detail below, the audio encoder24may be configured to perform various aspects of an extended-range coarse-fine quantization process in accordance with techniques of this disclosure.

In the example ofFIG. 2, the audio encoder24may be configured to encode the audio data21(or the mixed audio data23) using a gain-shape vector quantization encoding process that includes an extended-range coarse-fine quantization process. In a gain-shape vector quantization encoding process, the audio encoder24is configured to encode both a gain (e.g., an energy level) and a shape (e.g., a residual vector defined by transform coefficients) of a subband of frequency domain audio data. Each subband of frequency domain audio data represents a certain frequency range of a particular frame of the audio data21.

The audio data21may be sampled at a particular sampling frequency. Example sampling frequencies may include 48 kHz or 44.1 kHZ, though any desired sampling frequency may be used. Each digital sample of the audio data21may be defined by a particular input bit depth, e.g., 16 bits or 24 bits. In one example, the audio encoder24may be configured operate on a single channel of the audio data21(e.g., mono audio). In another example, the audio encoder24may be configured to independently encode two or more channels of the audio data21. For example, the audio data21may include left and right channels for stereo audio. In this example, the audio encoder24may be configured to encode the left and right audio channels independently in a dual mono mode. In other examples, the audio encoder24may be configured to encode two or more channels of the audio data21together (e.g., in a joint stereo mode). For example, the audio encoder24may perform certain compression operations by predicting one channel of the audio data21with another channel of the audio data21.

Regardless of how the channels of the audio data21are arranged, the audio encoder24recited the audio data21and sends that audio data21to a transform unit100. The transform unit100is configured to transform a frame of the audio data21from the time domain to the frequency domain to produce frequency domain audio data112. A frame of the audio data21may be represented by a predetermined number of samples of the audio data. In one example, a frame of the audio data21may be 1024 samples wide. Different frame widths may be chosen based on the frequency transform being used and the amount of compression desired. The frequency domain audio data112may be represented as transform coefficients, where the value of each the transform coefficients represents an energy of the frequency domain audio data112at a particular frequency.

In one example, the transform unit100may be configured to transform the audio data21into the frequency domain audio data112using a modified discrete cosine transform (MDCT). An MDCT is a “lapped” transform that is based on a type-IV discrete cosine transform. The MDCT is considered “lapped” as it works on data from multiple frames. That is, in order to perform the transform using an MDCT, transform unit100may include a fifty percent overlap window into a subsequent frame of audio data. The overlapped nature of an MDCT may be useful for data compression techniques, such as audio encoding, as it may reduce artifacts from coding at frame boundaries. The transform unit100need not be constrained to using an MDCT but may use other frequency domain transformation techniques for transforming the audio data21into the frequency domain audio data112.

A subband filter102separates the frequency domain audio data112into subbands114. Each of the subbands114includes transform coefficients of the frequency domain audio data112in a particular frequency range. In some examples, subband filter102may be configured to separate the frequency domain audio data112into subbands114of uniform frequency ranges. In other examples, subband filter102may be configured to separate the frequency domain audio data112into subbands114of non-uniform frequency ranges.

For example, subband filter102may be configured to separate the frequency domain audio data112into subbands114according to the Bark scale. In general, the subbands of a Bark scale have frequency ranges that are perceptually equal distances. That is, the subbands of the Bark scale are not equal in terms of frequency range, but rather, are equal in terms of human aural perception. In general, subbands at the lower frequencies will have fewer transform coefficients, as lower frequencies are easier to perceive by the human aural system. As such, the frequency domain audio data112in lower frequency subbands of the subbands114is less compressed by the audio encoder24, as compared to higher frequency subbands. Likewise, higher frequency subbands of the subbands114may include more transform coefficients, as higher frequencies are harder to perceive by the human aural system. As such, the frequency domain audio112in data in higher frequency subbands of the subbands114may be more compressed by the audio encoder24, as compared to lower frequency subbands.

The audio encoder24may be configured to process each of subbands114using a subband processing unit128. That is, the subband processing unit128may be configured to process each of subbands separately. The subband processing unit128may be configured to perform a gain-shape vector quantization process with extended-range coarse-fine quantization in accordance with techniques of this disclosure.

A gain-shape analysis unit104may receive the subbands114as an input. For each of subbands114, the gain-shape analysis unit104may determine an energy level116of each of the subbands114. That is, each of subbands114has an associated energy level116. The energy level116is a scalar value in units of decibels (dBs) that represents the total amount of energy (also called gain) in the transform coefficients of a particular one of subbands114. The gain-shape analysis unit104may separate energy level116for one of subbands114from the transform coefficients of the subbands to produce residual vector118. The residual vector118represents the so-called “shape” of the subband. The shape of the subband may also be referred to as the spectrum of the subband.

A vector quantizer108may be configured to quantize the residual vector118. In one example, the vector quantizer108may quantize the residual vector using a pyramid vector quantization (PVQ) process to produce the residual ID124. Instead of quantizing each sample separately (e.g., scalar quantization), the vector quantizer108may be configured to quantize a block of samples included in the residual vector118(e.g., a shape vector). In some examples, the vector quantizer108may use a Linde-Buzo-Gray (LBG) algorithm to perform the vector quantization. A Linde-Buzo-Gray (LBG) algorithm typically results in less distortion with a fixed available bit-rate compared to scalar quantization. However, any vector quantization techniques method can be used along with the extended-range coarse-fine energy quantization techniques of this disclosure.

For example, the vector quantizer108may use structured vector quantization algorithms reduce storage and computational complexity LGB algorithms. A structured vector quantization may involve performing the quantization based upon a set of structured code-vectors that do not need to be stored explicitly and can be identified functionally. Examples of the structured vector quantizers include Lattice vector quantizers and Pyramid Vector Quantizers (PVQ). Using PVQ, the vector quantizer108may be configured to map the residual vector118to a hyperpyramid (with constant L1 norm) or a hypersphere (with constant L2 norm) and quantize the residual vector118upon the underlying structured codebook. The quantization code-vectors are then enumerated and assigned an ID (e.g., the residual ID124) to be encoded and transmitted. The quality of the mapping drives the accuracy of the quantization, while the number of enumeration code-vectors specifies the shape transmission rate.

In some examples, the audio encoder24may dynamically allocate bits for coding the energy level116and the residual vector118. That is, for each of subbands114, the audio encoder24may determine the number of bits allocated for energy quantization (e.g., by the energy quantizer106) and the number of bits allocated for vector quantization (e.g., by the vector quantizer108). As will be explained in more detail below, the total number of bits allocated for energy quantization may be referred to as energy-assigned bits. These energy-assigned bits may then be allocated between a coarse quantization process and a fine quantization process.

An energy quantizer106may receive the energy level116of the subbands114and quantize the energy level116of the subbands114into a coarse energy120and a fine energy122. This disclosure will describe the quantization process for one subband, but it should be understood that the energy quantizer106may perform energy quantization on one or more of the subbands114, including each of the subbands114. In general, the energy quantizer106may perform a two-step quantization process. Energy quantizer106may first quantize the energy level116with a first number of bits for a coarse quantization process to generate the coarse energy120. The energy quantizer106may generate the coarse energy using a predetermined range of energy levels for the quantization (e.g., the range defined by a maximum and a minimum energy level. The coarse energy120approximates the value of the energy level116. The energy quantizer106may then determine a difference between the coarse energy120and the energy level116. This difference is sometimes called a quantization error. The energy quantizer106may then quantize the quantization error using a second number of bits in a fine quantization process to produce the fine energy122. The number of bits used for the fine quantization bits is determined by the total number of energy-assigned bits minus the number of bits used for the coarse quantization process. When added together, the coarse energy120and the fine energy122represent a total quantized value of the energy level116.

The audio encoder24may be further configured encode the coarse energy120, the fine energy122, and the residual ID124using a bitstream encoder110to create the encoded audio data25. The bitstream encoder110may be configured to further compress the coarse energy120, the fine energy122, and the residual ID124using one or more entropy encoding techniques. Entropy encoding techniques may include Huffman coding, arithmetic coding, context-adaptive binary arithmetic coding (CABAC), and other similar encoding techniques. The encoded audio data25may then be transmitted to the sink device14and/or stored in a memory for later use.

In one example of the disclosure, the quantization performed by the energy quantizer106is a uniform quantization. That is, the step sizes (also called “resolution) of each quantization are equal. In some examples, the steps sizes may be in units of decibels (dBs). The step size for the coarse quantization and the fine quantization may be determined, respectively, from a predetermined range of energy values for the quantization and the number of bits allocated for the quantization. In one example, the energy quantizer106performs uniform quantization for both coarse quantization (e.g., to produce the coarse energy120) and fine quantization (e.g., to produce the fine energy122).

Performing a two-step, uniform quantization process is equivalent to performing a single uniform quantization process. However, by splitting the uniform quantization into two parts, the bits allocated to coarse quantization and fine quantization may be independently controlled. This may allow for more flexibility in the allocation of bits across energy and vector quantization and may improve compression efficiency. Consider an M-level uniform quantizer, where M defines the number of levels (e.g., in dB) into which the energy level may be divided. M may be determined by the number of bits allocated for the quantization. For example, the energy quantizer106may use M1 levels for coarse quantization and M2 levels for fine quantization. This equivalent to a single uniform quantizer using M1*M2 levels.

FIG. 3is a conceptual diagram showing an example of a two-step uniform quantization process. As shown inFIG. 3, a coarse quantization process may use M1 levels (e.g., as determined by a number of bits allocated to the coarse quantization process) to quantize an input energy level. The difference between the output of the coarse quantization process and the input energy level is the error. The error may then be quantized by in a fine quantization process using M2 levels (e.g., as determined by a number of bits allocated to the fine quantization process).FIG. 4is a conceptual diagram showing how a two-step uniform coarse-fine quantization is equivalent to a single uniform quantization.

In one example, the minimum and maximum energy levels that determine the predetermined range for coarse quantization may be determined by analysis a statistical distribution of energy scalars over a give set of data. The minimum and maximum energy levels may be determined such that a desired level of compression is achieved without sacrificing an undesirable amount of audio fidelity. However, in some example audio signals, the energy level of a particular subband may be outside the predetermined range of energy values for coarse quantization. That is, the energy level of a particular subband may be above or below the predetermined maximum and minimum energy levels for coarse quantization. This problem may occur more often at higher frequency subbands, where the number of bits assigned for coarse quantization leads to narrow levels for coarse energy quantization. When the energy levels of a subband are outside of a predetermined range for coarse quantization, encoding errors may increase. Such encoding errors may lead to increased harmonic distortion, more perceptible quantization noise, high-frequency content attenuation, poor transient tracking, and potentially lower audio perceptual quality.

FIG. 5is a graph illustrating audio signal error for different audio coding techniques.FIG. 5shows the root means square (RMS) of the error in audio data across a spectrum of fundamental frequencies. The different plots inFIG. 5represent audio data encoded using different coding modes. As shown inFIG. 5, the plot270represents audio encoded using a 16-bit quantization that does not use the extended-range coarse quantization techniques of this disclosure. In general, audio data encoded using 16-bit quantization exhibits a relatively high RMS of error. The plot274represents audio data encoded using a 24-bit quantization on the residual vector, with fewer bits assigned to energy quantization. The RMS of error on audio data encoding using a 24-bit quantization on the residual vector remains relatively poor. The plot272represents audio data encoded using a 24-bit quantization on the energy level, with fewer bits assigned to residual vector quantization. The RMS of error on audio data encoding using a 24-bit quantization on the energy level is generally improved. However, low-frequency performance shown for plot272is relatively poor due to fewer bits assigned for residual ID quantization. Accordingly, merely assigning more bits to energy quantization may not address problems with subbands having energy levels that are outside the predetermined range for coarse quantization.

This disclosure describes techniques for an extended-range coarse quantization process that may improve the encoding quality of subbands for frequency domain audio data having abnormally higher or low energy values. The audio encoder24may determine the number of bits to use for both coarse and fine quantization using a predetermined energy range for coarse quantization. The energy quantizer106of the audio encoder24may determine if the energy level for a subband is outside the predefined range for coarse quantization. If so, the energy quantizer106may reallocate the bits assigned for fine quantization to an extended-range coarse quantization process. This ensures that the resolution of the coarse energy remains accurate. The energy quantizer106may perform extended range coarse quantization using an extended (i.e., larger) range of energy values compared to the initial coarse quantization process.

FIG. 6is a conceptual diagram illustrating an extension of a range of energy values for performing an extended-range coarse quantization process. As shown in FIG.6, the energy quantizer106may perform a coarse quantization process using a predetermined range of energy values Emaxand Emin. If the energy quantizer106determines that the energy level of a subband is outside this range, the energy quantizer may reassign bits assigned for fine quantization process to use in an extended-range coarse quantization process. The extended-range coarse quantization process may use an extended-range of energy levels defined by Emax, Extand Emin, Ext. As can be seen inFIG. 6, the extended range of energy levels is larger than the predetermined range of energy levels for the initial coarse quantization process.

After performing the extended-range coarse quantization process, the energy quantizer106may then compute the difference between the original total energy of the subband and the extended range coarse energy. This difference may then be coded using an extended-range fine quantization process with any remaining bits that are available.

This process can address subbands having out-of-range energy values, such as transients, tones, and sudden frequency content variations, seamlessly. Using a two-step approach of quantization, allows the system to be able to control coarse and fine quantization energy levels independently. The extended-range quantization improves reconstruction errors, and encodes tones, sharp transients, abrupt frequency content variations robustly. It also allows high flexibility to extend in different levels of accuracy, depending on the available bit budget.

FIG. 7is a block diagram illustrating an example energy quantizer106configured to perform various aspects of the techniques described in this disclosure. In particular, the energy quantizer106may be configured to perform one or more extended-range coarse-fine quantization techniques of this disclosure.

The energy level116is input to the energy quantizer106. As discussed above, the energy level116is an energy level of one of subbands114of frequency domain audio data. A predictive and differential computation unit148may perform frame-wise and/or subband-wise prediction of the energy level116to produce conditioned energy116. Frame-wise prediction may refer to subtracting the energy level of the same subband in another frame of audio data from the energy level116of the current subband. Subband-wise prediction may refer to subtracting the energy level of another subband in the same frame of audio data from the energy level116of the current subband. In some examples, both frame-wise and subband-wise prediction may be performed, where some function of frame and subband predictors may be used as the predictor for the current energy level116. The prediction process performed by predictive and differential computation unit148may be used to create a generally smaller energy level for performing the quantization.

A coarse quantization analysis unit150may be configured to determine the coarse resolution188and the coarse-assigned bits182for the coarse quantizer154. The coarse resolution188defines the step size (e.g., in units of decibels) used by the coarse quantizer154. Coarse quantization analysis unit may determine the coarse resolution based on the energy range180and the number of coarse-assigned bits182. In some examples, the energy range180is a predetermined energy range defined by a maximum (max) and minimum (min) energy value (e.g., Emaxand EmininFIG. 6). In some examples, the energy range180is the same for every subband of a frame. In other examples, the energy range180is dependent on the frequency range of the subband being processed.

In some examples, the coarse quantization analysis unit150unit may determine the same number of coarse-assigned bits182for every subband of frame. That is, in some examples, the number of coarse-assigned bits is fixed. In other examples, the coarse quantization analysis unit150may determine the number of coarse-assigned bits based on the particular subband being encoded and/or based on the energy level116of the current subband. The energy quantizer may determine the number of fine-assigned bits to use for fine quantizer158by subtracting the coarse-assigned bits182from the energy-assigned bits184. The energy-assigned bits184are the number of bits the audio encoder24allocates for the entire energy quantization process of the energy quantizer106. The fine quantization analysis unit152may be configured to determine the fine resolution190for the fine quantizer158based on the fine-assigned bits186. The fine resolution190defines the step size (e.g., in units of decibels) used by the fine quantizer158.

A coarse quantizer154may be configured to quantize the conditioned energy172in accordance with the coarse resolution188and the energy range180using the coarse-assigned bits182. Through quantization, the coarse quantizer154creates the coarse energy120. In one example, during the quantization process, the coarse quantizer may determine if the energy level of the conditioned energy172is out of range of the energy range180. In other examples of the disclosure, the energy quantizer106may determine if the original input energy level116is outside of the range of energy range180.

If the energy level of the quantized coarse energy120is not outside of the energy range180, the two-step quantization process continues to the fine quantizer158. First, the energy quantizer106subtracts the quantized coarse energy120from the conditioned energy172. As discussed above, this result may be referred to as the quantization error176. The fine quantizer158may then perform a fine quantization process, based on the fine resolution190and the fine-assigned bits186, on the error176to produce the quantized fine energy122. At this point, the energy quantization process for the subband ends.

If, instead, the energy level of the quantized coarse energy120is determined to be outside of the energy range180, the energy quantizer106may be configured to perform an extended-range coarse quantization process on the conditioned energy172. Coarse quantizer154, or another unit of energy quantizer106, may be configured to signal an extended-range indication196(e.g., as a syntax element) that indicates to an audio decoder (e.g., the audio decoder44ofFIG. 1) whether or not the extended-range coarse quantization process was performed for a particular subband. The audio decoder44may then be configured to perform an inverse quantization process on any received encoded energy levels based on the indications. For example, depending on whether or not extended-range coarse quantization was used, the audio decoder44may determine different scaling factors to use in an inverse quantization process.

For example, audio decoder44may determine the energy range to use for coarse quantization based on the indication. If the indication indicates that extended-range coarse quantization was not used, a regular energy range (e.g., energy range180) may be used to map the quantized coarse energy120to an actual conditioned energy level based on the energy range. If the indication indicates that extended-range coarse quantization was used, an extended energy range (e.g., energy range194) may be used to map the quantized extended-range coarse energy220to an actual conditioned energy level based on the energy range. In this context, the scaling factor may refer to how the quantized coarse energy120is mapped to a conditioned energy level given the use of either a normal energy range (e.g., energy range180) or an extended energy range (e.g., extended energy range194). The energy range180and the extended energy range194may be predetermined and stored at both audio encoder24and audio decoder44. However, it is not necessary to use the same energy range180and the extended energy range194for each of the subbands114. That is, audio encoder24and audio decoder44may store a different energy range180and extended energy range194for each of subbands114.

In response to a determination that the quantized coarse energy120is outside of the energy range180, a bit reallocation unit170may be configured to reallocate some of the fine-assigned bits186to an extended-range coarse quantization process. The reallocated bits198may be used by extended-range coarse quantizer164to quantize the conditioned energy172. In some examples, the bit reallocation unit170may be configured to reallocate a fixed number of the reallocated bits198to the extended-range coarse quantizer164. In other examples, the bit reallocation unit170may be configured to reallocate a dynamic number of the reallocated bits198to the extended-range coarse quantizer164. In one example, the number of reallocated bits198may be based on energy level of quantized energy120(e.g., based on how far out of range the energy was). In other examples, the number of reallocated bits198may be based on the frequency range of the particular subband being encoded. The number of reallocated bits198added to the coarse-assigned bits182results in the extended-range coarse-assigned bits.

Also in response to a determination that the quantized coarse energy120is outside of the energy range180, a range extender160may be configured to determine an extended energy range194to use in an extended-range coarse quantization process. Referring back toFIG. 6, the extended energy range194may be defined by new maximum and minimum energy values (e.g., Emax, Extand Emin, Ext). The extended energy range194is larger than the initial energy range180. In some examples, the range extender160may determine the extended energy range194based on an extension level192. The extension level192may define an increased number of steps to use in quantization and/or an increase decibel range to use for extended energy range194(e.g., 3 dB, 5 dB, 7 dB, etc.). In some examples, the extension level192may be based on the particular frequency range of the subband.

An extended-range coarse quantization analysis unit162may determine an extended-range coarse resolution200based on the extended energy range194and the extended-range coarse-assigned bits. Again, the extended-range coarse-assigned bits are the coarse-assigned bits182plus the reallocated bits198.

An extended-range coarse quantizer164may be configured to quantize the conditioned energy172in accordance with the extended-range coarse resolution200and the extended energy range194using the extended-range coarse-assigned bits. Through quantization, the extended-range coarse quantizer164creates the extended-range coarse energy220.

An extended-range fine quantization analysis unit166may determine the number of extended-range fine-assigned bits by subtracting the number of reallocated bits198form the fine-assigned bits186. Based on the extended-range fine-assigned bits, extended-range fine quantization analysis unit166may determine an extended-range fine resolution202for extended-range fine quantizer168.

Energy quantizer106subtracts the quantized extended-range coarse energy220from the conditioned energy172. This result may be referred to as the quantization error204. The extended-range fine quantizer168may then perform a fine quantization process, based on the extended-range fine resolution202and the extended-range fine-assigned bits, on the error204to produce the quantized extended-range fine energy222. At this point, the energy quantization process for the subband ends.

FIG. 8is a block diagram illustrating an implementation of the audio decoder44ofFIG. 1in more detail. In the example ofFIG. 8, the audio decoder44includes an extraction unit232, a subband reconstruction unit234, and a reconstruction unit236. The extraction unit232may represent a unit configured to extract the coarse energy120, the fine energy122, and the residual ID124from the encoded audio data25. The extraction unit232may extract, based on an energy bit allocation, one or more of the coarse energy120/220, the fine energy122/222, and the residual ID124. The coarse energy received may either be the coarse energy120quantized by the normal coarse energy quantization process or the extended-range coarse energy220quantized by the extended-range coarse energy quantization process. Likewise, the fine energy received may either be the fine energy122quantized by the normal fine energy quantization process or the extended-range fine energy222quantized by the extended-range fine energy quantization process. The extraction unit232may output the coarse energy120/220, the fine energy122/222and the residual ID124to the subband reconstruction unit234.

The subband reconstruction unit234may represent a unit configured to operate in a manner that is reciprocal to the operation of the subband processing unit128of the audio encoder24shown in the example ofFIG. 2. The subband reconstruction unit234may, in other words, reconstruct the subband s from the coarse energy120, the fine energy122, and the residual ID124. The subband reconstruction unit234may include an energy dequantizer238, a vector dequantizer240, and a subband composer242.

The energy dequantizer238may represent a unit configured to perform dequantization in a manner reciprocal to the quantization performed by the energy quantizer106illustrated inFIG. 2andFIG. 7. The energy dequantizer238may perform dequantization (also called inverse quantization) with respect to the coarse energy120and the fine energy122to obtain the predicted/difference energy levels, which the energy dequantizer238may perform inverse prediction or difference calculations to obtain the energy level116. The energy dequantizer238may output the energy level116to the subband composer242.

If the encoded audio data25includes a syntax element (e.g., the extended-range indication196) set to a value indicating that the coarse energy120was quantized using an extended range, then the energy dequantizer238may dequantize the coarse energy120and the fine energy122in accordance with an extended-range dequantization process. If the encoded audio data25includes a syntax element (e.g., the extended-range indication196ofFIG. 7) set to a value indicating that the coarse energy120was not quantized using an extended range, then the energy dequantizer238may dequantize the coarse energy120and the fine energy122in accordance with a normal range dequantization process. The extended-range indication196may be received for each of the subbands114.

The vector dequantizer240may represent a unit configured to perform vector dequantization in a manner reciprocal to the vector quantization performed by the vector quantizer108. The vector dequantizer240may perform vector dequantization with respect to the residual ID124to obtain the residual vector118. The vector dequantizer240may output the residual vector118to the subband composer242.

The subband composer242represents a unit configured to operate in a manner reciprocal to the gain-shape analysis unit104. As such, the subband composer242may perform inverse gain-shape analysis with respect to the energy level116and the residual vector118to obtain the subband s114. The subband composer242may output the subband s114to the reconstruction unit236.

The reconstruction unit236may represent a unit configured to reconstruct, based on the subband s114, the audio data21′. The reconstruction unit236may, in other words, perform inverse subband filtering in a manner reciprocal to the subband filtering applied by the subband filter102to obtain the frequency domain audio data112. The reconstruction unit236may next perform an inverse transform in a manner reciprocal to the transform applied by the transform unit100to obtain the audio data21′.

FIG. 9is a block diagram showing an example of the energy dequantizer238ofFIG. 8in more detail. The energy dequantizer238may be configured to perform one or more extended-range coarse-fine quantization techniques of this disclosure. The energy dequantizer238may be configured to receive the quantized coarse energy120/220and the quantized fine energy122/22that were quantized by the audio encoder24. In accordance with the techniques of this disclosure, the audio encoder24is configured to quantize the coarse energy120/220and the fine energy122/22using a predetermined coarse energy range for quantization or an extended range for coarse energy quantization. As an example, coarse energy quantization may be performed using a predetermined energy range defined by a maximum (max) and minimum (min) energy value (e.g., Emaxand EmininFIG. 6). Extended-range coarse energy quantization may be defined by new maximum and minimum energy values (e.g., Emax, Extand Emin, Ext) that is a larger range than the coarse energy quantization.

In order to determine what range of quantization was used, the energy dequantizer238may be configured to receive an extended-range indication196at an extended-range controller310. The extended-range indication196may be a 1-bit syntax element that indicates whether or not an extended-range coarse energy quantization process was used at the audio encoder24. For example, a value of “1” for the extended-range indication196indicates that the extended-range coarse energy quantization process was used by the audio encoder24and a value of “0” for the extended-range indication196indicates that the extended-range coarse energy quantization process was not used by the audio encoder24, or vice versa.

Depending on the value the extended-range indication196, extended-range controller310may route the received coarse energy120/220and the fine energy122/22to the appropriate dequantization units. If extended-range coarse quantization was not used, the extended-range controller310routes coarse energy120and fine energy122to the coarse dequantizer353and the fine dequantizer358, respectively. The coarse dequantizer353and the fine dequantizer358convert the quantized values of coarse energy120and fine energy122into energy values based on a predetermined range for coarse quantization (e.g., Emaxand EmininFIG. 6). The dequantized energy values are added together to produce energy level116′.

If extended-range coarse quantization was used, the extended-range controller310routes coarse energy220and fine energy222to the extended-range coarse dequantizer364and the extended-range fine dequantizer368, respectively. The extended-range coarse dequantizer364and the extended-range fine dequantizer368convert the quantized values of coarse energy220and fine energy222into energy values based on an extended-range for coarse quantization (e.g., Emax, Extand Emin, ExtinFIG. 6). This process may be repeated for each of subbands114.

FIG. 10is a graph illustrating audio signal error for different audio coding techniques. As shown inFIG. 10, Model (280) represents audio encoded using a 16-bit quantization that does not use the extended-range coarse quantization techniques of this disclosure. The other plot (282) onFIG. 10show the RMS of the error for audio data coded using a 3-level extension for the extended-range coarse quantization process (e.g., for energy quantization (EQ)) of this disclosure. In general, the RMS of error for audio data encoded using the 3-level extended-range coarse quantization techniques of this disclosure is improved (i.e., has a lower error across more frequency ranges) relative to not using an extended-range coarse quantization process.

FIG. 11is a graph illustrating audio signal error for different audio coding techniques with different extension levels. As shown inFIG. 11, Model is a 16-bit quantization that does not use the extended-range coarse quantization techniques of this disclosure. The RMS of the error for Mode1is generally higher than the other modes shown inFIG. 1. The other plots onFIG. 11show the RMS of the error for audio data coded using different extension levels for the extended-range coarse quantization process of this disclosure. In general, the RMS of error for audio data encoded using the techniques of this disclosure is improved (i.e., has a lower error across more frequency ranges) relative to not using an extended-range coarse quantization process.

FIG. 12is a flowchart illustrating example operation of the source device12ofFIG. 1in performing various aspects of the techniques described in this disclosure. As shown in the example ofFIG. 12, the audio encoder24of the source device12may be configured to encode audio data in accordance with the techniques of this disclosure. The techniques ofFIG. 12are described with reference to a single subband of a frame audio data. However, it should be understood that the techniques ofFIG. 12may be applied to any number of subbands of a frame of audio data, including all subbands of a frame of audio data.

The audio encoder24may be configured to determine an energy level of a first subband of frequency domain audio data (300). In one example of the disclosure, the audio encoder24may first be configured to perform a frequency domain transformation on the audio data to create frequency domain audio data and filter the frequency domain audio data into a plurality of subbands of frequency domain audio data, the plurality of subbands of frequency domain audio data including the first subband of frequency domain audio data.

The audio encoder24may be further configured to determine a bit allocation for a coarse quantization process and a fine quantization process (302). In one example, in order to determine the bit allocation for the coarse quantization process and the fine quantization process, the audio encoder24may be further configured to determine a total number of bits to use to quantize the energy level of the first subband of frequency domain audio, determine a first number of bits to use to perform the coarse quantization process, and determine a second number of bits to use to perform the fine quantization process based on the determined total number of bits and the determined first number of bits.

The audio encoder24may determine that the energy level of the first subband of frequency domain audio data is outside a predetermined range of energy levels for the coarse quantization process (304). In one example, the audio encoder may first quantize the energy level of the first subband of frequency domain audio data using the coarse quantization process to create a quantized coarse energy level. Then, in order to determine that the energy level of the first subband of frequency domain audio data is outside the predetermined range of energy levels, the audio encoder24may be further configured to determine that the quantized coarse energy level is outside the predetermined range of energy levels.

The audio encoder24may reallocate bits assigned to the fine quantization process to an extended-range coarse quantization process, wherein the extended-range coarse quantization process using an extended-range of energy levels, and wherein the extended-range of energy levels is larger than the predetermined range of energy levels for the coarse quantization process (306). In another example, the audio encoder24may be configured to determine the extended-range of energy levels for the extended-range coarse quantization process based on a frequency range of the first subband of frequency domain audio data.

In one example, in order to reallocate bits assigned to the fine quantization process to the extended-range coarse quantization process, the audio encoder24may be further configured to reallocate one or more of the determined second number of bits to the extended-range coarse quantization process, wherein the extended-range coarse quantization process uses a third number of bits. In another example, the audio encoder24may be configured to determine the extended-range of energy levels for the extended-range coarse quantization process based on a frequency range of the first subband of frequency domain audio data.

In another example, the audio encoder24may be configured to determine a fourth number of bits to use to perform the fine quantization process based on the determined total number of bits and the third number of bits for the extended-range coarse quantization process in response to reallocating one or more of the determined second number of bits to the extended-range coarse quantization process. The audio encoder24may be further configured to determine a difference between the energy level of the first subband of frequency domain audio data and the quantized extended-range coarse energy level, and quantize the difference using the fine quantization process and the further number of bits to create a quantized fine energy level.

The audio encoder24may quantize the energy level of the first subband of frequency domain audio data using the extended-range coarse quantization process to produce a quantized extended-range coarse energy level (308). The audio encoder24may be configured to signal the quantized extended-range coarse energy level and the quantized fine energy level in an encoded audio bitstream.

In one example, the audio encoder24may be configured to generate a syntax element that indicates that the extended-range coarse quantization process is being used for the first subband of frequency domain audio data in response to determining that the energy level of the first subband of frequency domain audio data is outside the predetermined range of energy levels for the coarse quantization process.

In another example, the audio encoder24may be configured to transmit the syntax element in an encoded audio bitstream. In another example, the audio encoder24may be configured to entropy encode the quantized extended-range coarse energy level. In another example, the audio encoder24may be configured to transmit the quantized extended-range coarse energy level over a PAN using a PAN communication protocol. In one example, the PAN communication protocol is a Bluetooth communication protocol.

FIG. 13is a flowchart illustrating example operation of the sink device14ofFIG. 1in performing various aspects of the techniques described in this disclosure. As shown in the example ofFIG. 13, the audio decoder44of the sink device14may be configured to decode audio data in accordance with the techniques of this disclosure.

The audio decoder44may be configured to receive a quantized coarse energy level for a subband of frequency domain audio data (350) and receive a syntax element that indicates if the quantized coarse energy level was quantized using an extended-range coarse quantization process (352). The audio decoder44may be configured to determine a scaling factor for performing an inverse quantization process based on the syntax element (354).

For example, the audio decoder44may be configured to determine a first scaling factor if the syntax element indicates that an extended-range coarse quantization process is not used. The audio decoder44may be configured to determine a second, different scaling factor if the syntax element indicates that an extended-range coarse quantization process is used. The different first and second scaling factors may be based on the difference between the predetermined range of energy values used for a regular coarse quantization process and the extend range of energy values used for the extended-range coarse quantization process. The audio decoder44may then perform inverse quantization on the quantized coarse energy level with the determined scaling factor (356).

For example, audio decoder44may be configured to determine a first scaling factor based on a predetermined range of energy levels for a coarse quantization process in the case that the syntax element indicates that the coarse energy level was not quantized using the extended-range coarse quantization process, and determine a second scaling factor based on an extended range of energy levels, wherein the extended range of energy levels is larger than the predetermined range of energy levels for the coarse quantization process, in the case that the syntax element indicates that the coarse energy level was quantized using the extended-range coarse quantization process. Audio decoder44may be further configured to determine the extended range of energy levels for the extended-range coarse quantization process based on a frequency range of the subband of frequency domain audio data.

Audio decoder44may be further configured to receive a quantized fine energy level for the subband of frequency domain audio data, perform inverse quantization on the quantized fine energy level based on the syntax element, and add the inverse quantized coarse energy level to the inverse quantized fine energy level to determine an energy level for the subband of frequency domain audio data. Audio decoder44may then reconstruct decoded audio data using the energy level for the subband of frequency domain audio data, as described above with reference toFIG. 8.

FIG. 14is a block diagram illustrating example components of the source device12shown in the example ofFIG. 1. In the example ofFIG. 14, the source device12includes a processor412, a graphics processing unit (GPU)414, system memory416, a display processor418, one or more integrated speakers105, a display103, a user interface420, antenna421, and a transceiver module422. In examples where the source device12is a mobile device, the display processor418is a mobile display processor (MDP). In some examples, such as examples where the source device12is a mobile device, the processor412, the GPU414, and the display processor418may be formed as an integrated circuit (IC).

For example, the IC may be considered as a processing chip within a chip package and may be a system-on-chip (SoC). In some examples, two of the processors412, the GPU414, and the display processor418may be housed together in the same IC and the other in a different integrated circuit (i.e., different chip packages) or all three may be housed in different ICs or on the same IC. However, it may be possible that the processor412, the GPU414, and the display processor418are all housed in different integrated circuits in examples where the source device12is a mobile device.

Examples of the processor412, the GPU414, and the display processor418include, but are not limited to, one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. The processor412may be the central processing unit (CPU) of the source device12. In some examples, the GPU414may be specialized hardware that includes integrated and/or discrete logic circuitry that provides the GPU414with massive parallel processing capabilities suitable for graphics processing. In some instances, GPU414may also include general purpose processing capabilities, and may be referred to as a general-purpose GPU (GPGPU) when implementing general purpose processing tasks (i.e., non-graphics related tasks). The display processor418may also be specialized integrated circuit hardware that is designed to retrieve image content from the system memory416, compose the image content into an image frame, and output the image frame to the display103.

The processor412may execute various types of the applications20. Examples of the applications20include web browsers, e-mail applications, spreadsheets, video games, other applications that generate viewable objects for display, or any of the application types listed in more detail above. The system memory416may store instructions for execution of the applications20. The execution of one of the applications20on the processor412causes the processor412to produce graphics data for image content that is to be displayed and the audio data21that is to be played (possibly via integrated speaker105). The processor412may transmit graphics data of the image content to the GPU414for further processing based on and instructions or commands that the processor412transmits to the GPU414.

The processor412may communicate with the GPU414in accordance with a particular application processing interface (API). Examples of such APIs include the DirectX® API by Microsoft®, the OpenGL® or OpenGL ES® by the Khronos group, and the OpenCL®; however, aspects of this disclosure are not limited to the DirectX, the OpenGL, or the OpenCL APIs, and may be extended to other types of APIs. Moreover, the techniques described in this disclosure are not required to function in accordance with an API, and the processor412and the GPU414may utilize any technique for communication.

The system memory416may be the memory for the source device12. The system memory416may comprise one or more computer-readable storage media. Examples of the system memory416include, but are not limited to, a random-access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), flash memory, or other medium that can be used to carry or store desired program code in the form of instructions and/or data structures and that can be accessed by a computer or a processor.

In some examples, the system memory416may include instructions that cause the processor412, the GPU414, and/or the display processor418to perform the functions ascribed in this disclosure to the processor412, the GPU414, and/or the display processor418. Accordingly, the system memory416may be a computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors (e.g., the processor412, the GPU414, and/or the display processor418) to perform various functions.

The system memory416may include a non-transitory storage medium. The term “non-transitory” indicates that the storage medium is not embodied in a carrier wave or a propagated signal. However, the term “non-transitory” should not be interpreted to mean that the system memory416is non-movable or that its contents are static. As one example, the system memory416may be removed from the source device12and moved to another device. As another example, memory, substantially similar to the system memory416, may be inserted into the source device12. In certain examples, a non-transitory storage medium may store data that can, over time, change (e.g., in RAM).

The user interface420may represent one or more hardware or virtual (meaning a combination of hardware and software) user interfaces by which a user may interface with the source device12. The user interface420may include physical buttons, switches, toggles, lights or virtual versions thereof. The user interface420may also include physical or virtual keyboards, touch interfaces—such as a touchscreen, haptic feedback, and the like.

The processor412may include one or more hardware units (including so-called “processing cores”) configured to perform all or some portion of the operations discussed above with respect to one or more of the mixing unit22, the audio encoder24, the wireless connection manager26, and the wireless communication units30. The antenna421and the transceiver module422may represent a unit configured to establish and maintain the wireless connection between the source device12and the sink device14. The antenna421and the transceiver module422may represent one or more receivers and one or more transmitters capable of wireless communication in accordance with one or more wireless communication protocols. The antenna421and the transceiver422may be configured to receive encoded audio data that has been encoded according to the techniques of this disclosure. Likewise, the antenna421and the transceiver422may be configured to transmit encoded audio data that has been encoded according to the techniques of this disclosure. The transceiver module422may perform all or some portion of the operations of one or more of the wireless connection manager26and the wireless communication units30.

FIG. 15is a block diagram illustrating exemplary components of the sink device14shown in the example ofFIG. 1. Although the sink device14may include components similar to that of the source device12discussed above in more detail with respect to the example ofFIG. 14, the sink device14may, in certain instances, include only a subset of the components discussed above with respect to the source device12.

In the example ofFIG. 15, the sink device14includes one or more speakers502, a processor512, a system memory516, a user interface520, an antenna521, and a transceiver module522. The processor512may be similar or substantially similar to the processor412. In some instances, the processor512may differ from the processor412in terms of total processing capacity or may be tailored for low power consumption. The system memory516may be similar or substantially similar to the system memory416. The speakers502, the user interface520, the antenna521, and the transceiver module522may be similar to or substantially similar to the respective speakers402, user interface420, and transceiver module422. The sink device14may also optionally include a display500, although the display500may represent a low power, low resolution (potentially a black and white LED) display by which to communicate limited information, which may be driven directly by the processor512.

The processor512may include one or more hardware units (including so-called “processing cores”) configured to perform all or some portion of the operations discussed above with respect to one or more of the wireless connection manager40, the wireless communication units42, and the audio decoder44. The antenna521and the transceiver module522may represent a unit configured to establish and maintain the wireless connection between the source device12and the sink device14. The antenna521and the transceiver module522may represent one or more receivers and one or more transmitters capable of wireless communication in accordance with one or more wireless communication protocols. The antenna521and the transceiver522may be configured to receive encoded audio data that has been encoded according to the techniques of this disclosure. Likewise, the antenna521and the transceiver522may be configured to transmit encoded audio data that has been encoded according to the techniques of this disclosure. The transceiver module522may perform all or some portion of the operations of one or more of the wireless connection manager40and the wireless communication units28.

The foregoing techniques may be performed with respect to any number of different contexts and audio ecosystems. A number of example contexts are described below, although the techniques should be limited to the example contexts. One example audio ecosystem may include audio content, movie studios, music studios, gaming audio studios, channel-based audio content, coding engines, game audio stems, game audio coding/rendering engines, and delivery systems.

The movie studios, the music studios, and the gaming audio studios may receive audio content. In some examples, the audio content may represent the output of an acquisition. The movie studios may output channel-based audio content (e.g., in 2.0, 5.1, and 7.1) such as by using a digital audio workstation (DAW). The music studios may output channel-based audio content (e.g., in 2.0, and 5.1) such as by using a DAW. In either case, the coding engines may receive and encode the channel-based audio content based one or more codecs (e.g., AAC, AC3, Dolby True HD, Dolby Digital Plus, and DTS Master Audio) for output by the delivery systems. The gaming audio studios may output one or more game audio stems, such as by using a DAW. The game audio coding/rendering engines may code and or render the audio stems into channel-based audio content for output by the delivery systems. Another example context in which the techniques may be performed comprises an audio ecosystem that may include broadcast recording audio objects, professional audio systems, consumer on-device capture, high-order ambisonics (HOA) audio format, on-device rendering, consumer audio, TV, and accessories, and car audio systems.

The broadcast recording audio objects, the professional audio systems, and the consumer on-device capture may all code their output using HOA audio format. In this way, the audio content may be coded using the HOA audio format into a single representation that may be played back using the on-device rendering, the consumer audio, TV, and accessories, and the car audio systems. In other words, the single representation of the audio content may be played back at a generic audio playback system (i.e., as opposed to requiring a particular configuration such as 5.1, 7.1, etc.), such as audio playback system16.

Other examples of context in which the techniques may be performed include an audio ecosystem that may include acquisition elements, and playback elements. The acquisition elements may include wired and/or wireless acquisition devices (e.g., microphones), on-device surround sound capture, and mobile devices (e.g., smartphones and tablets). In some examples, wired and/or wireless acquisition devices may be coupled to mobile device via wired and/or wireless communication channel(s).

In accordance with one or more techniques of this disclosure, the mobile device may be used to acquire a soundfield. For instance, the mobile device may acquire a soundfield via the wired and/or wireless acquisition devices and/or the on-device surround sound capture (e.g., a plurality of microphones integrated into the mobile device). The mobile device may then code the acquired soundfield into various representations for playback by one or more of the playback elements. For instance, a user of the mobile device may record (acquire a soundfield of) a live event (e.g., a meeting, a conference, a play, a concert, etc.), and code the recording into various representation, including higher order ambisonic HOA representations.

The mobile device may also utilize one or more of the playback elements to playback the coded soundfield. For instance, the mobile device may decode the coded soundfield and output a signal to one or more of the playback elements that causes the one or more of the playback elements to recreate the soundfield. As one example, the mobile device may utilize the wireless and/or wireless communication channels to output the signal to one or more speakers (e.g., speaker arrays, sound bars, etc.). As another example, the mobile device may utilize docking solutions to output the signal to one or more docking stations and/or one or more docked speakers (e.g., sound systems in smart cars and/or homes). As another example, the mobile device may utilize headphone rendering to output the signal to a headset or headphones, e.g., to create realistic binaural sound.

In some examples, a particular mobile device may both acquire a soundfield and playback the same soundfield at a later time. In some examples, the mobile device may acquire a soundfield, encode the soundfield, and transmit the encoded soundfield to one or more other devices (e.g., other mobile devices and/or other non-mobile devices) for playback.

Yet another context in which the techniques may be performed includes an audio ecosystem that may include audio content, game studios, coded audio content, rendering engines, and delivery systems. In some examples, the game studios may include one or more DAWs which may support editing of audio signals. For instance, the one or more DAWs may include audio plugins and/or tools which may be configured to operate with (e.g., work with) one or more game audio systems. In some examples, the game studios may output new stem formats that support audio format. In any case, the game studios may output coded audio content to the rendering engines which may render a soundfield for playback by the delivery systems.

The mobile device may also, in some instances, include a plurality of microphones that are collectively configured to record a soundfield, including 3D soundfields. In other words, the plurality of microphone may have X, Y, Z diversity. In some examples, the mobile device may include a microphone which may be rotated to provide X, Y, Z diversity with respect to one or more other microphones of the mobile device.

A ruggedized video capture device may further be configured to record a soundfield. In some examples, the ruggedized video capture device may be attached to a helmet of a user engaged in an activity. For instance, the ruggedized video capture device may be attached to a helmet of a user whitewater rafting. In this way, the ruggedized video capture device may capture a soundfield that represents the action all around the user (e.g., water crashing behind the user, another rafter speaking in front of the user, etc.).

The techniques may also be performed with respect to an accessory enhanced mobile device, which may be configured to record a soundfield, including a 3D soundfield. In some examples, the mobile device may be similar to the mobile devices discussed above, with the addition of one or more accessories. For instance, a microphone, including an Eigen microphone, may be attached to the above noted mobile device to form an accessory enhanced mobile device. In this way, the accessory enhanced mobile device may capture a higher quality version of the soundfield than just using sound capture components integral to the accessory enhanced mobile device.

Example audio playback devices that may perform various aspects of the techniques described in this disclosure are further discussed below. In accordance with one or more techniques of this disclosure, speakers and/or sound bars may be arranged in any arbitrary configuration while still playing back a soundfield, including a 3D soundfield. Moreover, in some examples, headphone playback devices may be coupled to a decoder via either a wired or a wireless connection. In accordance with one or more techniques of this disclosure, a single generic representation of a soundfield may be utilized to render the soundfield on any combination of the speakers, the sound bars, and the headphone playback devices.

A number of different example audio playback environments may also be suitable for performing various aspects of the techniques described in this disclosure. For instance, a 5.1 speaker playback environment, a 2.0 (e.g., stereo) speaker playback environment, a 9.1 speaker playback environment with full height front loudspeakers, a 22.2 speaker playback environment, a 16.0 speaker playback environment, an automotive speaker playback environment, and a mobile device with ear bud playback environment may be suitable environments for performing various aspects of the techniques described in this disclosure.

In accordance with one or more techniques of this disclosure, a single generic representation of a soundfield may be utilized to render the soundfield on any of the foregoing playback environments. Additionally, the techniques of this disclosure enable a rendered to render a soundfield from a generic representation for playback on the playback environments other than that described above. For instance, if design considerations prohibit proper placement of speakers according to a 7.1 speaker playback environment (e.g., if it is not possible to place a right surround speaker), the techniques of this disclosure enable a render to compensate with the other 6 speakers such that playback may be achieved on a 6.1 speaker playback environment.

Moreover, a user may watch a sports game while wearing headphones. In accordance with one or more techniques of this disclosure, the soundfield, including 3D soundfields, of the sports game may be acquired (e.g., one or more microphones and/or Eigen microphones may be placed in and/or around the baseball stadium). HOA coefficients corresponding to the 3D soundfield may be obtained and transmitted to a decoder, the decoder may reconstruct the 3D soundfield based on the HOA coefficients and output the reconstructed 3D soundfield to a renderer, the renderer may obtain an indication as to the type of playback environment (e.g., headphones), and render the reconstructed 3D soundfield into signals that cause the headphones to output a representation of the 3D soundfield of the sports game.

In each of the various instances described above, it should be understood that the source device12may perform a method or otherwise comprise means to perform each step of the method for which the source device12is described above as performing. In some instances, the means may comprise one or more processors. In some instances, the one or more processors may represent a special purpose processor configured by way of instructions stored to a non-transitory computer-readable storage medium. In other words, various aspects of the techniques in each of the sets of encoding examples may provide for a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause the one or more processors to perform the method for which the source device12has been configured to perform.

In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which corresponds to a tangible medium such as data storage media. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and/or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

Likewise, in each of the various instances described above, it should be understood that the sink device14may perform a method or otherwise comprise means to perform each step of the method for which the sink device14is configured to perform. In some instances, the means may comprise one or more processors. In some instances, the one or more processors may represent a special purpose processor configured by way of instructions stored to a non-transitory computer-readable storage medium. In other words, various aspects of the techniques in each of the sets of encoding examples may provide for a non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause the one or more processors to perform the method for which the sink device14has been configured to perform.

By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but are instead directed to non-transitory, tangible storage media. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

Various aspects of the techniques have been described. These and other aspects of the techniques are within the scope of the following claims.