Patent Description:
Sound is one of main ways for human beings to obtain information. With the rapid development of highperformance computers and signal processing technologies, immersive audio technologies attract more attention. An immersive three-dimensional audio (3D audio) technology provides better three-dimensional sound experience for users by expanding audio representation to high-dimensional space. The three-dimensional audio technology does not simply perform representation by using a plurality of sound channels on a playback side. Instead, an audio signal is reconstructed in three-dimensional space, and audio is represented in the three-dimensional space by using a rendering technology.

<CIT> generally discloses a method which includes receiving, at an audio encoder, multiple streams of audio data, where N is the number of the received multi streams. The method includes determining a similarity value for each stream of the multiple streams and comparing the similarity value for each stream of the multiple streams with a threshold. The method also includes identifying, based on the comparison, L (L<N) number of streams to be encoded among the N number of the multiple streams. The method includes encoding the identified L number of streams to generate an encoded bitstream. <CIT> generally discloses a system and method for generating one or more scaled compressed bitstreams from a single encoded plenary file. The plenary file contains multiple audio object files that were encoded separately using a scalable encoding process having fine-grained scalability. Activity in the data frames of the encoded audio object files at a time period are compared with each other to obtain a data frame activity comparison. Bits from an available bitpool are assigned to all of the data frames based on the data frame activity comparison and corresponding hierarchical metadata. <CIT> generally discloses an input signal includes a channel-based audio signal and an object-based audio signal, and an audio encoding device includes an audio scene analysis unit configured to determine an audio scene from the input signal and detect audio scene information; a channel-based encoder that encodes the channel-based audio signal output from the audio scene analysis unit; an object-based encoder that encodes the object-based audio signal output from the audio scene analysis unit; and an audio scene encoding unit configured to encode the audio scene information. <CIT> generally discloses an encoder for spatially directional multichannel audio signals, using a mixture of individual-channel subband coding with subband-steered composite-channel signal coding that combines the spectral components of the individual channel subbands selected for steering. The composite channel representation includes a subband steering control signal which either conveys the levels of the spectral components from all the steered channels, or represents the apparent directions (net directional vectors) of the spectral components from all the steered channels.

In three-dimensional audio encoding and decoding standards in and outside China, a quantity of bits that are allocated to each audio signal and that are used for encoding and decoding cannot reflect a difference of the audio signals based on a spatial feature of the audio signals on the playback side, and cannot adapt to a feature of the audio signals. This reduces encoding and decoding efficiency of the audio signals.

In the following, parts of the description and drawings referring to former embodiments which do not necessarily comprise all features to implement embodiments of the claimed invention are not represented as embodiments of the invention but as examples useful for understanding the embodiments of the invention. Embodiments of this application provides a bit allocation method and apparatus for an audio signal, to adapt to a feature of audio signals. In addition, different audio signals match different quantities of bits for encoding. This improves encoding and decoding efficiency of the audio signals.

According to a first aspect, this application provides a bit allocation method for an audio signal according to claim <NUM>.

In this application, priorities of a plurality of audio signals are determined based on a feature of the plurality of audio signals included in the current frame and related information of the audio signals in metadata, and a quantity of bits to be allocated to each audio signal is determined based on the priorities, to adapt to a feature of the audio signals. In addition, different audio signals may match different quantities of bits for encoding. This improves encoding and decoding efficiency of the audio signals.

In a possible implementation, the obtaining a scene grading parameter of each of the M audio signals includes: obtaining one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, a diffuseness grading parameter, a status grading parameter, a priority grading parameter, and a signal grading parameter of a first audio signal, where the first audio signal is any one of the M audio signals; and obtaining a scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in a spatial scene, the loudness grading parameter describes loudness of the first audio signal in the spatial scene, the spread grading parameter describes a spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

A priority of the audio signal with respect to information in a plurality of dimensions may be obtained based on a plurality of parameters of an audio signal.

In a possible implementation, when the obtaining T audio signals in a current frame, the method further includes: obtaining S groups of metadata in the current frame, where S is a positive integer, T ≥ S, the S groups of metadata correspond to the T audio signals, and the metadata describes a status of a corresponding audio signal in a spatial scene.

The metadata is used as description information of the status of the corresponding audio signal in the spatial scene, and may provide a reliable and effective basis for subsequently obtaining a scene grading parameter of the audio signal.

In a possible implementation, the obtaining a scene grading parameter of each of the M audio signals includes: obtaining one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, a diffuseness grading parameter, a status grading parameter, a priority grading parameter, and a signal grading parameter of a first audio signal based on metadata corresponding to the first audio signal or based on the first audio signal and the metadata corresponding to the first audio signal, where the first audio signal is any one of the M audio signals; and obtaining a scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in the spatial scene, the loudness grading parameter describes loudness of the first audio signal in the spatial scene, the spread grading parameter describes a spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

With reference to a plurality of parameters of an audio signal and metadata of the audio signal, a reliable priority of the audio signal with respect to information in a plurality of dimensions may be obtained.

In a possible implementation, the obtaining a scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter includes: performing weighed averaging on the obtained more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter to obtain the scene grading parameter; performing averaging on the obtained more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter to obtain the scene grading parameter; or using, as the scene grading parameter, the obtained one of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter.

In a possible implementation, the determining the M priorities of the M audio signals based on the scene grading parameter of each of the M audio signals includes: determining a priority corresponding to the scene grading parameter of the first audio signal as a priority of the first audio signal based on a specified first correspondence, where the first correspondence includes correspondences between a plurality of scene grading parameters and a plurality of priorities, one or more scene grading parameters correspond to one priority, and the first audio signal is any one of the M audio signals; using the scene grading parameter of the first audio signal as a priority of the first audio signal; or determining a range of the scene grading parameter of the first audio signal based on a plurality of specified range thresholds, and determining a priority corresponding to the range of the scene grading parameter of the first audio signal as a priority of the first audio signal.

In a possible implementation, the performing bit allocation on the M audio signals based on the M priorities of the M audio signals includes: performing bit allocation based on a currently available bit quantity and the M priorities of the M audio signals, where a higher quantity of bits are allocated to an audio signal with a higher priority.

In a possible implementation, the performing bit allocation based on a currently available bit quantity and the M priorities of the M audio signals includes: determining a bit quantity ratio of the first audio signal based on the priority of the first audio signal, where the first audio signal is any one of the M audio signals; and obtaining a bit quantity of the first audio signal based on a product of the currently available bit quantity and the bit quantity ratio of the first audio signal.

In a possible implementation, the performing bit allocation based on a currently available bit quantity and the M priorities of the M audio signals includes: determining a bit quantity of the first audio signal from a specified second correspondence based on the priority of the first audio signal, where the second correspondence includes correspondences between a plurality of priorities and a plurality of bit quantities, one or more priorities correspond to one bit quantity, and the first audio signal is any one of the M audio signals.

In a possible implementation, the determining a first audio signal set based on the T audio signals includes: adding a pre-specified audio signal of the T audio signals to the first audio signal set.

In a possible implementation, the determining a first audio signal set based on the T audio signals includes: adding, to the first audio signal set, an audio signal that is in the T audio signals and that corresponds to the S groups of metadata; or adding, to the first audio signal set, an audio signal that corresponds to a priority parameter greater than or equal to a specified participation threshold, where the metadata includes the priority parameter, and the T audio signals include the audio signal that corresponds to the priority parameter.

In a possible implementation, the obtaining a scene grading parameter of each of the M audio signals includes: obtaining one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, and a diffuseness grading parameter of a first audio signal, where the first audio signal is any one of the M audio signals; obtaining a first scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, and the diffuseness grading parameter; obtaining one or more of a status grading parameter, a priority grading parameter, and a signal grading parameter of the first audio signal; obtaining a second scene grading parameter of the first audio signal based on the obtained one or more of the status grading parameter, the priority grading parameter, and the signal grading parameter; and obtaining a scene grading parameter of the first audio signal based on the first scene grading parameter and the second scene grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in a spatial scene, the loudness grading parameter describes playback loudness of the first audio signal in the spatial scene, the spread grading parameter describes a playback spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

In a possible implementation, the obtaining a scene grading parameter of each of the M audio signals includes: obtaining one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, and a diffuseness grading parameter of a first audio signal based on metadata corresponding to the first audio signal or based on the first audio signal and the metadata corresponding to the first audio signal, where the first audio signal is any one of the M audio signals; obtaining a first scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, and the diffuseness grading parameter; obtaining one or more of a status grading parameter, a priority grading parameter, and a signal grading parameter of the first audio signal based on the metadata corresponding to the first audio signal or based on the first audio signal and the metadata corresponding to the first audio signal; obtaining a second scene grading parameter of the first audio signal based on the obtained one or more of the status grading parameter, the priority grading parameter, and the signal grading parameter; and obtaining a scene grading parameter of the first audio signal based on the first scene grading parameter and the second scene grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in the spatial scene, the loudness grading parameter describes playback loudness of the first audio signal in the spatial scene, the spread grading parameter describes a playback spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

In this application, for different features of an audio signal, a plurality of scene grading parameters related to the audio signal are obtained by using a plurality of methods, and then a priority of the audio signal is determined based on the plurality of scene grading parameters. The priority obtained in this way may refer to the plurality of features of the audio signal, and may also be compatible with implementation solutions corresponding to the different features.

According to the invention, the determining the M priorities of the M audio signals based on the scene grading parameter of each of the M audio signals includes: obtaining a first priority of the first audio signal based on the first scene grading parameter; obtaining a second priority of the first audio signal based on the second scene grading parameter; and obtaining the priority of the first audio signal based on the first priority and the second priority.

In this application, for different features of an audio signal, a plurality of priorities related to the audio signal are obtained by using a plurality of methods, and then compatible combination is performed on the plurality of priorities to obtain a final priority of the audio signal. The priority obtained in this way may refer to the plurality of features of the audio signal, and may also be compatible with implementation solutions corresponding to the different features.

After the bit allocation method for an audio signal according to any one of the implementations of the first aspect is performed, the method further includes: encoding the M audio signals based on a quantity of bits allocated to the M audio signals to obtain an encoded bitstream.

In a possible implementation, the encoded bitstream includes a bit quantity of the M audio signals.

After the bit allocation method for an audio signal according to any one of the implementations of the first aspect is performed, the method further includes: receiving an encoding bitstream; obtaining a bit quantity of each of the M audio signals by performing the bit allocation method for an audio signal according to any one of the implementations of the first aspect; and reconstructing the M audio signals based on the bit quantity of each of the M audio signals and the encoded bitstream.

According to another aspect, this application provides a device according to claim <NUM>.

According to another aspect, this application provides a computer-readable storage medium according to claim <NUM>.

To make the objectives, technical solutions, and advantages of this application clearer, the following clearly describes the technical solutions in this application with reference to accompanying drawings in this application. Obviously, described embodiments are a part rather than all of embodiments of this application. All other embodiments obtained by a person of ordinary skill in the art based on embodiments of this application without creative efforts shall fall within the protection scope of this application.

In embodiments, claims, and accompanying drawings of the specification of this application, the terms "first", "second", and the like are merely intended for distinguishing and description, and shall not be understood as an indication or implication of relative importance or an indication or implication of an order. In addition, the terms "include", "have", and any variant thereof are intended to cover non-exclusive inclusion, for example, include a series of steps or units. Methods, systems, products, or devices are not necessarily limited to those steps or units that are literally listed, but may include other steps or units that are not literally listed or that are inherent to such processes, methods, products, or devices.

It should be understood that in this application, "at least one (item)" refers to one or more and "a plurality of" refers to two or more. The term "and/or" is used to describe an association relationship between associated objects, and represents that three relationships may exist. For example, "A and/or B" may represent the following three cases: Only A exists, only B exists, and both A and B exist, where A and B may be singular or plural. The character "/" generally indicates an "or" relationship between the associated objects. "At least one of the following items (pieces)" or a similar expression thereof means any combination of these items, including a single item (piece) or any combination of a plurality of items (pieces). For example, at least one item (piece) of a, b, or c may indicate a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c may be singular or plural.

Explanations of related terms in this application are as follows:.

The following is a system architecture to which this application is applied.

<FIG> is an example of a schematic block diagram of an audio encoding and decoding system <NUM> applied in this application. As shown in <FIG>, the audio encoding and decoding system <NUM> may include a source device <NUM> and a destination device <NUM>. The source device <NUM> generates encoded audio data, and therefore the source device <NUM> may be referred to as an audio encoding apparatus. The destination device <NUM> may decode the encoded audio data generated by the source device <NUM>, and therefore the destination device <NUM> may be referred to as an audio decoding apparatus. The source device <NUM>, the destination device <NUM>, or various implementation solutions of the source device <NUM> or the destination device <NUM> may include one or more processors and a memory coupled to the one or more processors. The memory may include but is not limited to a random access memory (random access memory, RAM), a read-only memory (read-only memory, ROM), a flash memory, or any other medium that may be used to store desired program code in a form of instructions or a data structure accessible by a computer. The source device <NUM> and the destination device <NUM> may include various apparatuses, including a desktop computer, a mobile computing apparatus, a notebook (for example, a laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display apparatus, a digital media player, an audio game console, a vehicle-mounted computer, a wireless communication device, or the like.

Although <FIG> depicts the source device <NUM> and the destination device <NUM> as separate devices, a device embodiment may alternatively include both the source device <NUM> and the destination device <NUM> or functionalities of both the source device <NUM> and the destination device <NUM>, namely, the source device <NUM> or a corresponding functionality and the destination device <NUM> or a corresponding functionality. In such embodiments, the source device <NUM> or the corresponding functionality and the destination device <NUM> or the corresponding functionality may be implemented by using same hardware and/or software, separate hardware and/or software, or any combination thereof.

A communication connection between the source device <NUM> and the destination device <NUM> may be implemented through a link <NUM>. The destination device <NUM> may receive encoded audio data from the source device <NUM> through the link <NUM>. The link <NUM> may include one or more media or apparatuses capable of moving the encoded audio data from the source device <NUM> to the destination device <NUM>. In an example, the link <NUM> may include one or more communication media that enable the source device <NUM> to directly transmit the encoded audio data to the destination device <NUM> in real time. In this example, the source device <NUM> may modulate the encoded audio data according to a communication standard (for example, a wireless communication protocol), and may transmit modulated audio data to the destination device <NUM>. The one or more communication media may include a wireless communication medium and/or a wired communication medium, for example, a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may constitute a part of a packet-based network, and the packet-based network is, for example, a local area network, a wide area network, or a global network (for example, the internet). The one or more communication media may include a router, a switch, a base station, or another device that facilitates communication from the source device <NUM> to the destination device <NUM>.

The source device <NUM> includes an encoder <NUM>. Optionally, the source device <NUM> may further include an audio source <NUM>, an audio preprocessor <NUM>, and a communication interface <NUM>. In a specific implementation form, the encoder <NUM>, the audio source <NUM>, the audio preprocessor <NUM>, and the communication interface <NUM> may be hardware components in the source device <NUM>, or may be software programs in the source device <NUM>. Descriptions are as follows.

The audio source <NUM> may include or may be any type of audio capture device, for example, configured to capture real-world sound, and/or any type of audio generation device, for example, a computer audio processor, or any type of device configured to obtain and/or provide real-world audio, computer animation audio (for example, screen content and audio in virtual reality (VR)), and/or any combination thereof (for example, audio in augmented reality (AR)). The audio source <NUM> may be a microphone for capturing audio or a memory for storing audio. The audio source <NUM> may further include any type of (internal or external) interface for storing previously captured or generated audio and/or obtaining or receiving audio. When the audio source <NUM> is a microphone, the audio source <NUM> may be, for example, a local audio collection apparatus or an audio collection apparatus integrated into the source device. When the audio source <NUM> is a memory, the audio source <NUM> may be, for example, a local memory or a memory integrated into the source device. When the audio source <NUM> includes an interface, the interface may be, for example, an external interface for receiving audio from an external audio source. The external audio source is, for example, an external audio capturing device, such as a speaker, a microphone, an external memory, or an external audio generation device. The external audio generation device is, for example, an external computer graphics processor, a computer, or a server. The interface may be any type of interface, for example, a wired or wireless interface or an optical interface, according to any proprietary or standardized interface protocol.

Audio may be considered as a one-dimensional vector of a pixel (picture element). A pixel in the vector may also be referred to as a sample. A quantity of samples on the vector or audio defines a size of the audio. In this application, audio transmitted by the audio source <NUM> to an audio processor may also be referred to as original audio data <NUM>.

The audio preprocessor <NUM> is configured to receive the original audio data <NUM> and perform preprocessing on the original audio data <NUM> to obtain preprocessed audio <NUM> or preprocessed audio data <NUM>. For example, the preprocessing performed by the audio preprocessor <NUM> may include trimming, tuning, or denoising.

The encoder <NUM> (or referred to as an audio encoder <NUM>) is configured to receive the preprocessed audio data <NUM>, and process the preprocessed audio data <NUM> to provide encoded audio data <NUM>. In some embodiments, the encoder <NUM> may be configured to perform various embodiments described below, to implement application of the bit allocation method for an audio signal described in this application to an encoder side.

The communication interface <NUM> may be configured to receive the encoded audio data <NUM>, and transmit the encoded audio data <NUM> to the destination device <NUM> or any other device (for example, a memory) through the link <NUM> for storage or direct reconstruction. The any other device may be any device for decoding or storage. The communication interface <NUM> may be, for example, configured to encapsulate the encoded audio data <NUM> into an appropriate format, for example, a data packet, for transmission through the link <NUM>.

The destination device <NUM> includes a decoder <NUM>. Optionally, the destination device <NUM> may further include a communication interface <NUM>, an audio post-processor <NUM>, and a playing device <NUM>. Descriptions are as follows.

The communication interface <NUM> may be configured to receive the encoded audio data <NUM> from the source device <NUM> or any other source. The any other source is, for example, a storage device. The storage device is, for example, an encoded audio data storage device. The communication interface <NUM> may be configured to transmit or receive the encoded audio data <NUM> through the link <NUM> between the source device <NUM> and the destination device <NUM> or through any type of network. The link <NUM> is, for example, a direct wired or wireless connection. The any type of network is, for example, a wired or wireless network or any combination thereof, or any type of private or public network, or any combination thereof. The communication interface <NUM> may be, for example, configured to decapsulate the data packet transmitted through the communication interface <NUM>, to obtain the encoded audio data <NUM>.

Both the communication interface <NUM> and the communication interface <NUM> may be configured as unidirectional communication interfaces or bidirectional communication interfaces, and may be configured to, for example, send and receive messages to establish a connection, and acknowledge and exchange any other information related to a communication link and/or data transmission such as encoded audio data transmission.

The decoder <NUM> (or referred to as an audio decoder <NUM>) is configured to receive the encoded audio data <NUM>, and provide decoded audio data <NUM> or decoded audio <NUM>. In some embodiments, the decoder <NUM> may be configured to perform various embodiments described below, to implement application of the bit allocation method for an audio signal described in this application to a decoder side.

The audio post-processor <NUM> is configured to perform post-processing on the decoded audio data <NUM> (also referred to as reconstructed audio data) to obtain post-processed audio data <NUM>. The post-processing performed by the audio post-processor <NUM> may include trimming or resampling, or any other processing, and may be further configured to transmit the post-processed audio data <NUM> to the playing device <NUM>.

The playing device <NUM> is configured to receive the post-processed audio data <NUM> to play audio to, for example, a user or a listener. The playing device <NUM> may be or may include any type of player configured to present reconstructed audio, for example, an integrated or external speaker or speaker.

A person skilled in the art clearly knows, based on the description, that existence and (accurate) division of functionalities of different units or the functionalities of the source device <NUM> and/or the destination device <NUM> shown in <FIG> may vary with an actual device and application. The source device <NUM> and the destination device <NUM> may be any one of a wide range of devices, including any type of handheld or stationary device, for example, a notebook or laptop computer, a mobile phone, a smartphone, a pad or a tablet computer, a video camera, a desktop computer, a set-top box, a television set, a camera, a vehicle-mounted device, a playing device, a digital media player, a game console, a media streaming transmission device (such as a content service server or a content distribution server), a broadcast receiver device, or a broadcast transmitter device, and may not use or may use any type of operating system.

The encoder <NUM> and the decoder <NUM> each may be implemented as any one of various appropriate circuits, for example, one or more microprocessors, digital signal processors (digital signal processors, DSPs), application-specific integrated circuits (application-specific integrated circuits, ASICs), field programmable gate arrays (field programmable gate arrays, FPGAs), discrete logic, hardware, or any combinations thereof. If the technologies are implemented partially by using software, a device may store software instructions in an appropriate and non-transitory computer-readable storage medium and may execute instructions by using hardware such as one or more processors, to perform the technologies of this disclosure. Any of the foregoing content (including hardware, software, a combination of hardware and software, and the like) may be considered as one or more processors.

In some cases, the audio encoding and decoding system <NUM> shown in <FIG> is merely an example and the technologies of this application may be applied to audio coding settings (for example, audio encoding or audio decoding) that do not necessarily include any data communication between an encoding device and a decoding device. In another example, data may be retrieved from a local memory, transmitted in a streaming manner through a network, or the like. An audio encoding device may encode data and store data into the memory, and/or an audio decoding device may retrieve and decode data from the memory. In some examples, the encoding and the decoding are performed by devices that do not communicate with one another, but simply encode data to the memory and/or retrieve and decode data from the memory.

<FIG> is an illustrative diagram of an example of an audio coding system <NUM> according to an example embodiment. The audio coding system <NUM> can implement a combination of various technologies in embodiments of this application. In the illustrated implementation, the audio coding system <NUM> may include a microphone <NUM>, the encoder <NUM>, the decoder <NUM> (and/or an audio encoder/decoder implemented by using a logic circuit <NUM> of a processing unit <NUM>), an antenna <NUM>, one or more processors <NUM>, one or more memories <NUM>, and/or a playing device <NUM>.

As shown in <FIG>, the microphone <NUM>, the antenna <NUM>, the processing unit <NUM>, the logic circuit <NUM>, the encoder <NUM>, the decoder <NUM>, the processor <NUM>, the memory <NUM>, and/or the playing device <NUM> can communicate with each other. As described, although the audio coding system <NUM> is illustrated with the encoder <NUM> and the decoder <NUM>, the audio coding system <NUM> may include only the encoder <NUM> or only the decoder <NUM> in different examples.

In some examples, the antenna <NUM> may be configured to transmit or receive an encoded bitstream of audio data. In addition, in some examples, the playing device <NUM> may be configured to play audio data. In some examples, the logic circuit <NUM> may be implemented by using the processing unit <NUM>. The processing unit <NUM> may include application-specific integrated circuit (application-specific integrated circuit, ASIC) logic, a graphics processing unit, a general-purpose processor, or the like. The audio coding system <NUM> may also include the optional processor <NUM>. The optional processor <NUM> may similarly include application-specific integrated circuit (application-specific integrated circuit, ASIC) logic, a graphics processing unit, or the like. In some examples, the logic circuit <NUM> may be implemented by using hardware, for example, audio coding dedicated hardware. The processor <NUM> may be implemented by using general-purpose software, an operating system, or the like. In addition, the memory <NUM> may be any type of memory, for example, a volatile memory (for example, a static random access memory (Static Random Access Memory, SRAM) or a dynamic random access memory (Dynamic Random Access Memory, DRAM)) or a non-volatile memory (for example, a flash memory). In a non-limitative example, the memory <NUM> may be implemented by using a cache memory. In some examples, the logic circuit <NUM> may access the memory <NUM>. In other examples, the logic circuit <NUM> and/or the processing unit <NUM> may include a memory (for example, a cache) for implementation of a buffer or the like.

In some examples, the encoder <NUM> implemented by using the logic circuit may include a buffer (for example, implemented by using the processing unit <NUM> or the memory <NUM>) and an audio processing unit (for example, implemented by using the processing unit <NUM>). The audio processing unit may be communicatively coupled to the buffer. The audio processing unit may include the encoder <NUM> implemented by using the logic circuit <NUM>, to implement various modules of any other encoder system or subsystem described in this specification. The logic circuit may be configured to perform various operations described in this specification.

In some examples, the decoder <NUM> may be implemented by using the logic circuit <NUM> in a similar manner, to implement various modules of any other decoder system or subsystem described in this specification. In some examples, the decoder <NUM> implemented by using the logic circuit may include a buffer (implemented by using the processing unit <NUM> or the memory <NUM>) and an audio processing unit (for example, implemented by using the processing unit <NUM>). The audio processing unit may be communicatively coupled to the buffer. The audio processing unit may include the decoder <NUM> implemented by using the logic circuit <NUM>, to implement various modules of any other decoder system or subsystem described in this specification.

In some examples, the antenna <NUM> may be configured to receive an encoded bitstream of audio data. As discussed, the encoded bitstream may include audio signal data, metadata, and the like that are related to an audio frame and that are described in this specification. The audio coding system <NUM> may further include the decoder <NUM> that is coupled to the antenna <NUM> and that is configured to decode the encoded bitstream. The playing device <NUM> is configured to play an audio frame.

It should be understood that, in this application, for the example described with reference to the encoder <NUM>, the decoder <NUM> may be configured to perform an inverse process. With regard to metadata, the decoder <NUM> may be configured to receive and parse such metadata, and correspondingly decode related audio data. In some examples, the encoder <NUM> may entropy encode the metadata into an encoded audio bitstream. In such examples, the decoder <NUM> may parse such metadata and correspondingly decode related audio data.

<FIG> is a schematic diagram of a structure of an audio coding device <NUM> (for example, an audio encoding device or an audio decoding device) according to this application. The audio coding device <NUM> is suitable for implementing embodiments described in this application. In an embodiment, the audio coding device <NUM> may be an audio decoder (for example, the decoder <NUM> in <FIG>) or an audio encoder (for example, the encoder <NUM> in <FIG>). In another embodiment, the audio coding device <NUM> may be one or more components of the decoder <NUM> in <FIG> or the encoder <NUM> in <FIG>.

The audio coding device <NUM> includes an ingress port <NUM> and a receiver unit (Rx) <NUM> for receiving data, a processor, a logic unit, or a central processing unit (CPU) <NUM> for processing the data, a transmitter unit (Tx) <NUM> and an egress port <NUM> for transmitting the data, and a memory <NUM> for storing the data. The audio coding device <NUM> may further include optical-to-electrical conversion components and electrical-to-optical (EO) components coupled to the ingress port <NUM>, the receiver unit <NUM>, the transmitter unit <NUM>, and the egress port <NUM> for egress or ingress of optical or electrical signals.

The processor <NUM> is implemented by using hardware and software. The processor <NUM> may be implemented as one or more CPU chips, cores (for example, a multi-core processor), FPGAs, ASICs, and DSPs. The processor <NUM> is in communication with the ingress port <NUM>, the receiver unit <NUM>, the transmitter unit <NUM>, the egress port <NUM>, and the memory <NUM>. The processor <NUM> includes a coding module <NUM> (for example, an encoding module <NUM> or a decoding module <NUM>). The encoding/decoding module <NUM> implements embodiments disclosed in this specification, to implement the bit allocation method for an audio signal provided in this application. For example, the encoding/decoding module <NUM> implements, processes, or provides various coding operations. Therefore, the encoding/decoding module <NUM> provides a substantial improvement to functions of the audio coding device <NUM> and affects a switching of the audio coding device <NUM> to a different state. Alternatively, the encoding/decoding module <NUM> is implemented as instructions stored in the memory <NUM> and executed by the processor <NUM>.

The memory <NUM> includes one or more disks, tape drives, and solid-state drives and may be used as an over-flow data storage device to store programs when such programs are selectively executed, and to store instructions and data that are read during program execution. The memory <NUM> may be volatile and/or non-volatile, and may be a read-only memory (ROM), a random access memory (RAM), a ternary content-addressable memory (ternary content-addressable memory, TCAM), and/or a static random access memory (SRAM).

<FIG> is a simplified block diagram of an apparatus <NUM> according to an example embodiment. The apparatus <NUM> can implement technologies of this application. In other words, <FIG> is a schematic block diagram of an implementation of an encoding device or a decoding device (briefly referred to as a coding device <NUM>) according to this application. The apparatus <NUM> may include a processor <NUM>, a memory <NUM>, and a bus system <NUM>. The processor and the memory are connected through the bus system. The memory is configured to store instructions. The processor is configured to execute the instructions stored in the memory. The memory of the coding device stores program code. The processor may invoke the program code stored in the memory to perform the method described in this application. To avoid repetition, details are not described herein again.

In this application, the processor <NUM> may be a central processing unit (Central Processing Unit, "CPU" for short), or the processor <NUM> may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or another programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.

The memory <NUM> may include a read-only memory (ROM) device or a random access memory (RAM) device. Any other proper type of storage device may also be used as the memory <NUM>. The memory <NUM> may include code and data <NUM> that are accessed by the processor <NUM> through the bus <NUM>. The memory <NUM> may further include an operating system <NUM> and an application <NUM>.

In addition to a data bus, the bus system <NUM> may further include a power bus, a control bus, a status signal bus, and the like. However, for clear description, various types of buses in the figure are marked as the bus system <NUM>.

Optionally, the coding device <NUM> may further include one or more output devices, for example, a speaker <NUM>. In an example, the speaker <NUM> may be a headset or a loudspeaker. The speaker <NUM> may be connected to the processor <NUM> through the bus <NUM>.

Based on the descriptions of the foregoing embodiments, this application provides a bit allocation method for an audio signal. <FIG> is a schematic flowchart of a bit allocation method for an audio signal for implementing this application. A process <NUM> may be executed by the source device <NUM> or the destination device <NUM>. The process <NUM> is described as a series of steps or operations. It should be understood that steps or operations of the process <NUM> may be performed in various sequences and/or simultaneously, not limited to an execution sequence shown in <FIG>.

Step <NUM>: Obtain T audio signals in a current frame.

T is a positive integer. The current frame is an audio frame obtained at a current moment in a process of performing the method in this application. To create immersive stereo sound effect, in a three-dimensional audio technology, different sounds are no longer simply represented by using a plurality of channels, but are represented by using different audio signals. For example, an environment includes a human sound, a music sound, and a vehicle sound, and three audio signals are separately used to represent the human sound, the music sound, and the vehicle sound. Then, each sound is reconstructed in three-dimensional space based on the three audio signals, to represent a plurality of sounds in the three-dimensional space. In other words, the audio frame may include a plurality of audio signals, and one audio signal represents voice, music, or sound effect in reality. It should be noted that any technology for extracting an audio signal from an audio frame may be used in this application. This is not specifically limited.

In a possible implementation, S groups of metadata in the current frame are obtained, where the S groups of metadata correspond to the T audio signals. For example, each of the T audio signals corresponds to one group of metadata. In this case, S = T. For another example, only some of the T audio signals correspond to the metadata. In this case, T > S. This is not specifically limited.

In this application, audio data and metadata are separately generated in this process on an encoder side based on preprocessing of an original voice, music, sound effect, or the like. The encoder side may select, based on a principle of an audio frame and corresponding to a start time (sample) and an end time (sample) of the current frame, metadata in a corresponding time range as metadata of the current frame. A decoder side may parse a received bitstream to obtain the metadata of the current frame.

In this application, the metadata describes a status of an audio signal in a spatial scene. For example, Table <NUM> describes an example of the metadata. Parameters included in the metadata include an object index (object_index), an azimuth (position_azimuth), an elevation (position_elevation), a position radius (position_radius), a gain factor (gain_factor), a uniform spread degree (spread_uniform), a spread width (spread_width), a spread height (spread_height), a spread depth (spread_depth), diffuseness (diffuseness), a priority (priority), divergence (divergence), and a speed (speed). The metadata records a value range and a quantity of bits of the foregoing parameters. It should be noted that the metadata may further include another parameter and a parameter recording form. This is not specifically limited in this application.

Step <NUM>: Determine a first audio signal set based on the T audio signals.

The first audio signal set includes M audio signals, where M is a positive integer, T audio signals include the M audio signals, and T ≥ M. In this application, an audio signal that is in the T audio signals and that corresponds to metadata may be added to the first audio signal set. In other words, if all the foregoing T audio signals correspond to metadata, all the T audio signals may be added to the first audio signal set. If only some of the foregoing T audio signals correspond to metadata, only these audio signals need to be added to the first audio signal set. In this application, a pre-specified audio signal in the T audio signals may be further added to the first audio signal set. Some or all of the T audio signals may be added to the first audio signal set through high-layer signaling or in a manner specified by a user. Optionally, an index of the audio signal to be added to the first audio signal set is directly configured through the high-layer signaling. Alternatively, the user specifies voice, music, or sound effect, and adds an audio signal of a specified object to the first audio signal set. In this application, reference may be further made to a priority parameter of an audio signal recorded in metadata. The priority parameter indicates importance of a corresponding audio signal in three-dimensional audio. When the priority parameter is greater than or equal to a specified participation threshold, the audio signal that is in the T audio signals and that corresponds to the priority parameter is added to the first audio signal set.

It should be noted that the foregoing provides several methods for classifying the T audio signals in the current frame (namely, adding all or some of the T audio signals to the first audio signal set). It should be understood that the methods cannot constitute all limitations in this application. Other methods, including another designation manner that refers to high-layer signaling, another parameter in metadata, and the like, may be further used in this application.

Step <NUM>: Determine M priorities of the M audio signals in the first audio signal set.

In this application, a scene grading parameter of each of the M audio signals may be first obtained, and then the M priorities of the M audio signals is determined based on the scene grading parameter of each of the M audio signals.

The scene grading parameter may be an importance indicator that is of the audio signal and that is obtained based on a related parameter of the audio signal. The related parameter may include one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, a diffuseness grading parameter, a status grading parameter, a priority grading parameter, and a signal grading parameter. These parameters may be obtained based on a signal feature of the audio signal, or may be obtained based on metadata of the audio signal. The movement grading parameter describes a movement speed of a first audio signal in a unit time in the spatial scene. The loudness grading parameter describes playback loudness of the first audio signal in the spatial scene. The spread grading parameter describes a playback spread range of the first audio signal in the spatial scene. The diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene. The status grading parameter describes sound source divergence of the first audio signal in the spatial scene. The priority grading parameter describes a priority of the first audio signal in the spatial scene. The signal grading parameter describes energy of the first audio signal in an encoding process.

The following uses an ith audio signal as an example to describe a method for obtaining the foregoing parameters. The ith audio signal is any one of the M audio signals. It should be noted that the following several parameters are examples for description, and the scene grading parameter may alternatively be calculated based on another parameter or feature of the audio signal. This is not specifically limited in this application.

The movement grading parameter may be calculated according to the following equation: <MAT>.

Herein, speedRatioi indicates a movement grading parameter of the ith audio signal. f(di) indicates a mapping relationship between a movement status of the ith audio signal in the spatial scene and metadata. d i indicates a movement distance of the ith audio signal in a unit time. θ i indicates an azimuth of the ith audio signal relative to a rendering center point after the ith audio signal is moved. φ i indicates an elevation of the ith audio signal relative to the rendering center point after the ith audio signal is moved. ri indicates a distance of the ith audio signal relative to the rendering center point after the ith audio signal is moved. θ <NUM> indicates an azimuth of the ith audio signal relative to the rendering center point before the ith audio signal is moved. φ<NUM> indicates an elevation of the ith audio signal relative to the rendering center point before the ith audio signal is moved. r<NUM> indicates a distance of the ith audio signal relative to the rendering center point before the ith audio signal is moved. As shown in <FIG>, it is assumed that spherical coordinates indicate a location of three-dimensional audio in the spatial scene, a sphere center is used as the rendering center point, a sphere radius is a distance between a location of the ith audio signal in the spatial scene and the sphere center, an included angle between the location of the ith audio signal in the spatial scene and a horizontal plane is the elevation of the ith audio signal, an included angle between a projection of the location of the ith audio signal in the spatial scene on the horizontal plane and a front of the rendering center point is the azimuth of the ith audio signal, and <MAT> indicates a sum of mapping relationships between movement statuses of the M audio signals in the spatial scene and the metadata.

Alternatively, the movement grading parameter may be calculated according to the following equation: <MAT>.

Herein, <MAT> indicates a sum of movement distances of the M audio signals in a unit time.

It should be noted that the movement grading parameter may alternatively be calculated by using another method. This is not specifically limited in this application.

The loudness grading parameter may be calculated according to the following equation: <MAT>.

Herein, loudRatioi indicates a loudness grading parameter of the ith audio signal. f(Ai, gaini, ri) indicates a mapping relationship between playback loudness of the ith audio signal in the spatial scene and both of a signal feature and the metadata. Ai indicates a sum or an average value of amplitudes of samples of the ith audio signal in the current frame. The amplitudes of the samples may be obtained based on metadata of the ith audio signal. gaini indicates a gain value of the audio signal in the current frame, and may be obtained based on the metadata of the ith audio signal. ri indicates a distance from the ith audio signal to the rendering center point in the current frame, and may be obtained based on the metadata of the ith audio signal. <MAT> indicates a sum of mapping relationships between playback loudness of the M audio signals in the spatial scene and both of the signal feature and the metadata.

Alternatively, the loudness grading parameter may be calculated according to the following equation: <MAT>.

Herein, mean(Ai) indicates a sum or an average value of amplitudes of samples of the ith audio signal in the current frame. The amplitudes of the samples may be obtained based on metadata of the ith audio signal. <MAT> indicates a sum or an average value of amplitudes of samples of the M audio signals in the current frame.

Herein, ri indicates a distance between the ith audio signal and the rendering center point, and may be obtained based on metadata of the ith audio signal. <MAT> indicates a sum of reciprocals of distances between the M audio signals and the rendering center point.

Herein, gaini indicates a gain of the ith audio signal in rendering. The gain may be obtained by a user by customizing the ith audio signal, or may be generated by a decoder according to a specified rule. <MAT> indicates a sum of gains of the M audio signals in rendering.

It should be noted that the loudness grading parameter may alternatively be calculated by using another method. This is not specifically limited in this application.

The spread grading parameter describes a spread degree of the ith audio signal in the current frame, and may be obtained based on spread-related metadata of the ith audio signal. It should be noted that the spread grading parameter may alternatively be calculated by using another method. This is not specifically limited in this application.

The diffuseness grading parameter describes diffuseness of the ith audio signal in the current frame, and may be obtained based on diffuseness-related metadata of the ith audio signal. It should be noted that the diffuseness grading parameter may alternatively be calculated by using another method. This is not specifically limited in this application.

The status grading parameter describes divergence of the ith audio signal in the current frame, and may be obtained based on divergence-related metadata of the ith audio signal. It should be noted that the status grading parameter may alternatively be calculated by using another method. This is not specifically limited in this application.

The priority grading parameter describes a priority of the ith audio signal in the current frame, and may be obtained based on priority-related metadata of the ith audio signal. It should be noted that the priority grading parameter may alternatively be calculated by using another method. This is not specifically limited in this application.

The signal grading parameter describes energy of the ith audio signal in an encoding process of the current frame, and may be obtained based on original energy of the ith audio signal, or may be obtained based on signal energy that is obtained after the ith audio signal is preprocessed. It should be noted that the signal grading parameter may alternatively be calculated by using another method. This is not specifically limited in this application.

After the foregoing one or more of the parameters of the ith audio signal are obtained, a scene grading parameter sceneRatioi of the ith audio signal may be calculated based on the one or more of the parameters. In other words, the scene grading parameter sceneRatioi of the ith audio signal may be a function about the one or more of the parameters, and may be expressed as: <MAT>.

The function may be linear or non-linear. This is not specifically limited in this application.

In a possible implementation, weighted averaging may be performed on the foregoing one or more of the parameters of the ith audio signal, for example, the plurality of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter, to obtain the scene grading parameter of the ith audio signal, that is,<MAT>.

Herein, α<NUM> - α<NUM> are separately weight factors of corresponding parameters. A value of the weight factor may be any value from <NUM> to <NUM>, inclusive. A sum of the weight factors is <NUM>. A larger value of the weight factor indicates higher importance and a higher ratio of the corresponding parameter during calculation of the scene grading parameter. If the value is <NUM>, it indicates that the corresponding parameter does not participate in the calculation of the scene grading parameter. In other words, a feature of an audio signal that corresponds to the parameter is not considered during the calculation of the scene grading parameter. If the value is <NUM>, it indicates that only the corresponding parameter is considered during the calculation of the scene grading parameter. In other words, a feature of an audio signal that corresponds to the parameter is a unique basis for the calculation of the scene grading parameter. The value of the weight factor may be preset, or may be obtained through adaptive calculation in an execution process of the method in this application. This is not specifically limited in this application. Optionally, if only one of the foregoing one or more of the parameters of the ith audio signal is obtained, the parameter is used as the scene grading parameter of the ith audio signal.

In a possible implementation, averaging may be performed on the foregoing one or more of the parameters of the ith audio signal, for example, the plurality of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter, to obtain the scene grading parameter of the ith audio signal, that is, <MAT>.

It should be noted that, in the foregoing function, the scene grading parameter of the ith audio signal is calculated. The foregoing provides two function implementation methods for calculating the scene grading parameter of the ith audio signal. Another calculation method may alternatively be used in this application. This is not specifically limited.

In this application, based on the scene grading parameter of the ith audio signal, a priority of the ith audio signal may be obtained by using the following method. There is a linear relationship between the scene grading parameter and the priority of the ith audio signal. In other words, a larger scene grading parameter indicates a higher priority. As shown in <FIG>, a spatial scene uses a rendering center as a sphere center. An audio signal closer to the sphere center has a higher priority. An audio signal farther from the sphere center has a lower priority.

In a possible implementation, a priority corresponding to the scene grading parameter of the ith audio signal may be determined as the priority of the ith audio signal based on a specified first correspondence. The first correspondence includes correspondences between a plurality of scene grading parameters and a plurality of priorities. One or more scene grading parameters correspond to one priority.

Based on historical data and/or experience accumulation of audio signal encoding, a priority of an audio signal and a correspondence between a scene grading parameter and each priority may be preset. For example, Table <NUM> describes an example of the first correspondence between the scene grading parameters and the priorities.

In Table <NUM>, when the scene grading parameter of the ith audio signal is <NUM>, the corresponding priority is <NUM>. In this case, the priority of the ith audio signal is <NUM>. When the scene grading parameter of the ith audio signal is <NUM>, the corresponding priority is <NUM>. In this case, the priority of the ith audio signal is <NUM>. It should be noted that Table <NUM> is an example of the correspondence between the scene grading parameters and the priorities, and does not constitute a limitation on such a correspondence in this application.

In a possible implementation, the scene grading parameter of the ith audio signal may be used as the priority of the ith audio signal.

In this application, the priority may not be classified, and the scene grading parameter of the ith audio signal is directly used as the priority of the ith audio signal.

In a possible implementation, a range of the scene grading parameter of the ith audio signal may be determined based on a specified range threshold, and a priority corresponding to the range of the scene grading parameter of the ith audio signal is determined as the priority of the ith audio signal.

Based on historical data and/or experience accumulation of audio signal encoding, a priority of an audio signal and a correspondence between a range of a scene grading parameter and each priority may be preset. For example, Table <NUM> describes another example of the first correspondence between the scene grading parameters and the priorities.

In Table <NUM>, when the scene grading parameter of the ith audio signal is <NUM>, the range of the scene grading parameter is [<NUM>, <NUM>), and the corresponding priority is <NUM>. In this case, the priority of the ith audio signal is <NUM>. When the scene grading parameter of the ith audio signal is <NUM>, the range of the scene grading parameter is [<NUM>, <NUM>), and the corresponding priority is <NUM>. In this case, the priority of the ith audio signal is <NUM>. It should be noted that Table <NUM> is an example of the correspondence between the scene grading parameters and the priorities, and does not constitute a limitation on such a correspondence in this application.

Step <NUM>: Perform bit allocation on the M audio signals based on the M priorities of the M audio signals.

In this application, bit allocation may be performed based on a currently available bit quantity and the M priorities of the M audio signals. A higher quantity of bits are allocated to an audio signal with a higher priority. The currently available bit quantity refers to a total quantity of bits that can be allocated to the M audio signals in the first audio signal set in the current frame before a codec performs bit allocation.

In a possible implementation, a bit quantity ratio of the first audio signal may be determined based on the priority of the first audio signal. The first audio signal is any one of the M audio signals. A bit quantity of the first audio signal is obtained based on a product of the currently available bit quantity and the bit quantity ratio of the first audio signal. A correspondence is pre-established between the priority and the bit quantity ratio of the audio signal. One priority may correspond to one bit quantity ratio, or a plurality of priorities may correspond to one bit quantity ratio. A corresponding quantity of bits that can be allocated to the audio signal may be obtained through calculation based on the bit quantity ratio and the currently available bit quantity. For example, M is <NUM>, a priority of a first audio signal is <NUM>, a priority of a second audio signal is <NUM>, and a priority of a third audio signal is <NUM>. It is assumed that a ratio corresponding to the priority <NUM> is set to <NUM>%, a ratio corresponding to the priority <NUM> is set to <NUM>%, a ratio corresponding to the priority <NUM> is set to <NUM>%, and the currently available bit quantity is <NUM>. In this case, a quantity of bits allocated to the first audio signal is <NUM>, a quantity of bits allocated to the second audio signal is <NUM>, and a quantity of bits allocated to the third audio signal is <NUM>. It should be noted that, in different audio frames, a bit quantity corresponding to a priority may be adaptively adjusted. This is not specifically limited.

In a possible implementation, the bit quantity corresponding to the priority of the first audio signal may be determined as the bit quantity of the first audio signal based on a specified second correspondence. The second correspondence includes correspondences between a plurality of priorities and a plurality of bit quantities. One or more priorities correspond to one bit quantity. A correspondence is pre-established between the priority and the bit quantity of the audio signal. One priority may correspond to one bit quantity, or a plurality of priorities may correspond to one bit quantity. Based on the correspondence, when the priority of the audio signal is obtained, the corresponding bit quantity may be obtained. For example, M is <NUM>, a priority of a first audio signal is <NUM>, a priority of a second audio signal is <NUM>, and a priority of a third audio signal is <NUM>. It is assumed that a bit quantity corresponding to the priority <NUM> is set to <NUM>, a bit quantity corresponding to the priority <NUM> is set to <NUM>, and a bit quantity corresponding to the priority <NUM> is set to <NUM>.

In a possible implementation, when the scene grading parameter of the audio signal does not include the signal grading parameter, and when the scene grading parameter is small, it is considered that a scene grading difference between audio signals is quite small. In this case, bit allocation between the audio signals may be determined based on an absolute energy ratio between the audio signals in an encoding and decoding process. When the scene grading parameter of the audio signal does not include the signal grading parameter, and when the scene grading parameter of the audio signal is large, it is considered that a scene grading difference between audio signals is quite large. In this case, bit allocation between the audio signals may be determined based on the scene grading parameter of the audio signal. In other cases, bit allocation of the audio signal may be determined based on a bit allocation factor of the audio signal. Therefore, the following equations may exist. sceneRatioi indicates the scene grading parameter of the ith audio signal. bits _ available indicates the currently available bit quantity. bits_ objecti indicates the quantity of bits allocated to the ith audio signal.

When sceneRatioi ≤ δ , bits _ object, = nrgRatioi × bits _ available , where δ indicates an upper limit of the scene grading parameter, and nrgRatioi indicates an absolute energy ratio between the ith audio signal and another audio signal.

When sceneRatio, ≥ τ , bits _ objecti = sceneRatio, × bits _ available , where τ indicates a lower limit of the scene grading parameter.

In addition to the foregoing two cases, bits _ object, = objRatio, × bits _ available , where objRatioi indicates a bit allocation factor of the ith audio signal.

It should be noted that, in addition to the foregoing described method for determining the quantity of bits allocated to the audio signal, another method may be used for implementation. This is not specifically limited in this application.

In this application, a priority of a plurality of audio signals is determined based on a feature of the plurality of audio signals included in the current frame and related information of the audio signals in metadata, and a quantity of bits to be allocated to each audio signal is determined based on the priority, to adapt to a feature of the audio signals. In addition, different audio signals may match different quantities of bits for encoding. This improves encoding and decoding efficiency of the audio signals.

In this application, in step <NUM>, the M audio signals are determined from the T audio signals of the current frame and added to the first audio signal set. The method in step <NUM> and step <NUM> is used for the M audio signals. A priority of each audio signal is first determined, and then a quantity of bits allocated to each audio signal is determined based on the priority of the audio signal. When T > M, audio signals in the first audio signal set are not all audio signals in the current frame, and remaining audio signals may be added to a second audio signal set. The second audio signal set includes N audio signals, where N = T - M. For the N audio signals, a simple method may be used to determine a quantity of bits allocated to the N audio signals. For example, a total available bit quantity of the second audio signal set is averaged by N to obtain a bit quantity of each audio signal. In other words, a total quantity of available bits of the second audio signal set are evenly allocated to the N audio signals in the set. It should be noted that another method may alternatively be used to obtain the bit quantity of each audio signal in the second audio signal set. This is not specifically limited in this application.

In addition to the method for determining the priority of the audio signal described in step <NUM>, this application further provides a priority combination method based on a plurality of priority determining methods, namely, a method for determining a final priority of an audio signal whose priority may be obtained by using a plurality of methods. The following uses the first audio signal as an example for description. The first audio signal is any one of the M audio signals.

In a possible implementation, a first parameter set and a second parameter set of the first audio signal are obtained based on the first audio signal and/or metadata corresponding to the first audio signal. The first parameter set includes one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter in the foregoing related parameters of the first audio signal. The second parameter set also includes one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter in the foregoing related parameters of the first audio signal. The first parameter set and the second parameter set may include a same parameter, or may include different parameters. A first scene grading parameter of the first audio signal is obtained based on the first parameter set. Herein, refer to the method for determining the scene grading parameter of the M audio signals in the first audio signal set in step <NUM>, or use another method. A second scene grading parameter of the first audio signal is obtained based on the second parameter set. A method used herein is different from a method for calculating the first scene grading parameter. A scene grading parameter of the first audio signal is obtained based on the first scene grading parameter and the second scene grading parameter. In this application, for the scene grading parameters obtained through calculation by using the two methods for the same audio signal, a weighted averaging method may be used, or a direct averaging method may be used, or a method of obtaining a larger value or a smaller value may be used to determine the final scene grading parameter of the audio signal. This is not specifically limited. In this way, the scene grading parameter of the audio signal may be obtained in diversified manners, and compatible with calculation solutions in various policies.

According to the invention, after the first scene grading parameter and the second scene grading parameter of the first audio signal are obtained, a first priority of the first audio signal is obtained based on the first scene grading parameter. In this case, the priority may be obtained by using the method in step <NUM>, or may be obtained by using another method. A second priority of the first audio signal is obtained based on the second scene grading parameter. A method used herein is different from a method for calculating the first priority. The priority of the first audio signal is obtained based on the first priority and the second priority. In this application, for the priorities obtained through calculation by using the two methods for the same audio signal, a weighted averaging method may be used, or an averaging method may be used, or a method of obtaining a larger value or a smaller value may be used to determine the final priority of the audio signal. This is not specifically limited. In this way, the priority of the audio signal may be obtained in diversified manners, and compatible with calculation solutions in various policies.

In this application, after the quantity of bits allocated to the T audio signals of the current frame is determined by using the method in the foregoing embodiment, a bitstream may be generated based on the quantity of bits of the T audio signals. The bitstream includes T first identifiers, T second identifiers, and T third identifiers. The T audio signals separately correspond to the T first identifiers, the T second identifiers, and the T third identifiers. The first identifier indicates an audio signal set to which a corresponding audio signal belongs. The second identifier indicates a priority of a corresponding audio signal. The third identifier indicates a bit quantity of a corresponding audio signal. The bitstream is sent to a decoding device. After receiving the bitstream, the decoding device performs the foregoing bit allocation method for an audio signal based on the T first identifiers, the T second identifiers, and the T third identifiers that are carried in the bitstream, to determine the bit quantity of the T audio signals. Alternatively, the decoding device may directly determine the audio signal set to which the T audio signals belong, the priority, and the quantity of allocated bits based on the T first identifiers, the T second identifiers, and the T third identifiers that are carried in the bitstream, to decode the bitstream and obtain the T audio signals. The first identifier, the second identifier, and the third identifier are identifier information added on the basis of the method embodiment shown in <FIG>, so that an encoder side or a decoder side of an audio signal can encode or decode the audio signal based on the same method.

<FIG> is a schematic diagram of a structure of an apparatus according to an embodiment of this application. As shown in <FIG>, the apparatus may be applied to the encoding device or the decoding device in the foregoing embodiments. The apparatus in this embodiment may include a processing module <NUM> and a transceiver module <NUM>. The processing module <NUM> is configured to: obtain T audio signals in a current frame, where T is a positive integer; determine a first audio signal set based on the T audio signals, where the first audio signal set includes M audio signals, M is a positive integer, the T audio signals include the M audio signals, and T ≥ M; determine M priorities of the M audio signals in the first audio signal set; and perform bit allocation on the M audio signals based on the M priorities of the M audio signals.

In a possible implementation, the processing module <NUM> is specifically configured to: obtain a scene grading parameter of each of the M audio signals; and determine the M priorities of the M audio signals based on the scene grading parameter of each of the M audio signals.

In a possible implementation, the processing module <NUM> is specifically configured to: obtain one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, a diffuseness grading parameter, a status grading parameter, a priority grading parameter, and a signal grading parameter of a first audio signal, where the first audio signal is any one of the M audio signals; and obtain a scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in a spatial scene, the loudness grading parameter describes loudness of the first audio signal in the spatial scene, the spread grading parameter describes a spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

In a possible implementation, the processing module <NUM> is specifically configured to obtain S groups of metadata in the current frame, where S is a positive integer, T ≥ S, the S groups of metadata correspond to the T audio signals, and the metadata describes a status of a corresponding audio signal in the spatial scene.

In a possible implementation, the processing module <NUM> is specifically configured to: obtain one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, a diffuseness grading parameter, a status grading parameter, a priority grading parameter, and a signal grading parameter of a first audio signal based on metadata corresponding to the first audio signal or based on the first audio signal and the metadata corresponding to the first audio signal, where the first audio signal is any one of the M audio signals; and obtain a scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in a spatial scene, the loudness grading parameter describes loudness of the first audio signal in the spatial scene, the spread grading parameter describes a spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

In a possible implementation, the processing module <NUM> is specifically configured to: perform weighed averaging on the obtained more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter to obtain the scene grading parameter; perform averaging on the obtained more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter to obtain the scene grading parameter; or use, as the scene grading parameter, the obtained one of the movement grading parameter, the loudness grading parameter, the spread grading parameter, the diffuseness grading parameter, the status grading parameter, the priority grading parameter, and the signal grading parameter.

In a possible implementation, the processing module <NUM> is specifically configured to: determine a priority corresponding to the scene grading parameter of the first audio signal as a priority of the first audio signal based on a specified first correspondence, where the first correspondence includes correspondences between a plurality of scene grading parameters and a plurality of priorities, one or more scene grading parameters correspond to one priority, and the first audio signal is any one of the M audio signals; use the scene grading parameter of the first audio signal as a priority of the first audio signal; or determine a range of the scene grading parameter of the first audio signal based on a specified range threshold, and determining a priority corresponding to the range of the scene grading parameter of the first audio signal as a priority of the first audio signal.

In a possible implementation, the processing module <NUM> is specifically configured to perform bit allocation based on a currently available bit quantity and the M priorities of the M audio signals, where a higher quantity of bits are allocated to an audio signal with a higher priority.

In a possible implementation, the processing module <NUM> is specifically configured to: determine a bit quantity ratio of the first audio signal based on the priority of the first audio signal, where the first audio signal is any one of the M audio signals; and obtain a bit quantity of the first audio signal based on a product of the currently available bit quantity and the bit quantity ratio of the first audio signal.

In a possible implementation, the processing module <NUM> is specifically configured to determine a bit quantity of the first audio signal from a specified second correspondence based on the priority of the first audio signal, where the second correspondence includes correspondences between a plurality of priorities and a plurality of bit quantities, one or more priorities correspond to one bit quantity, and the first audio signal is any one of the M audio signals.

In a possible implementation, the processing module <NUM> is specifically configured to add a pre-specified audio signal of the T audio signals to the first audio signal set.

In a possible implementation, the processing module <NUM> is specifically configured to: add, to the first audio signal set, an audio signal that is in the T audio signals and that corresponds to the S groups of metadata; or add, to the first audio signal set, an audio signal that corresponds to a priority parameter greater than or equal to a specified participation threshold, where the metadata includes the priority parameter, and the T audio signals include the audio signal that corresponds to the priority parameter.

In a possible implementation, the processing module <NUM> is specifically configured to: obtain one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, and a diffuseness grading parameter of a first audio signal, where the first audio signal is any one of the M audio signals; obtain a first scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, and the diffuseness grading parameter; obtain one or more of a status grading parameter, a priority grading parameter, and a signal grading parameter of the first audio signal; obtain a second scene grading parameter of the first audio signal based on the obtained one or more of the status grading parameter, the priority grading parameter, and the signal grading parameter; and obtain a scene grading parameter of the first audio signal based on the first scene grading parameter and the second scene grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in a spatial scene, the loudness grading parameter describes playback loudness of the first audio signal in the spatial scene, the spread grading parameter describes a playback spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

In a possible implementation, the processing module <NUM> is specifically configured to: obtain one or more of a movement grading parameter, a loudness grading parameter, a spread grading parameter, and a diffuseness grading parameter of a first audio signal based on metadata corresponding to the first audio signal or based on the first audio signal and the metadata corresponding to the first audio signal, where the first audio signal is any one of the M audio signals; obtain a first scene grading parameter of the first audio signal based on the obtained one or more of the movement grading parameter, the loudness grading parameter, the spread grading parameter, and the diffuseness grading parameter; obtain one or more of a status grading parameter, a priority grading parameter, and a signal grading parameter of the first audio signal based on the metadata corresponding to the first audio signal or based on the first audio signal and the metadata corresponding to the first audio signal; obtain a second scene grading parameter of the first audio signal based on the obtained one or more of the status grading parameter, the priority grading parameter, and the signal grading parameter; and obtain a scene grading parameter of the first audio signal based on the first scene grading parameter and the second scene grading parameter, where the movement grading parameter describes a movement speed of the first audio signal in a unit time in a spatial scene, the loudness grading parameter describes playback loudness of the first audio signal in the spatial scene, the spread grading parameter describes a playback spread range of the first audio signal in the spatial scene, the diffuseness grading parameter describes a diffuseness range of the first audio signal in the spatial scene, the status grading parameter describes sound source divergence of the first audio signal in the spatial scene, the priority grading parameter describes a priority of the first audio signal in the spatial scene, and the signal grading parameter describes energy of the first audio signal in an encoding process.

In a possible implementation, the processing module <NUM> is specifically configured to: obtain a first priority of the first audio signal based on the first scene grading parameter; obtain a second priority of the first audio signal based on the second scene grading parameter; and obtain the priority of the first audio signal based on the first priority and the second priority.

In a possible implementation, the processing module <NUM> is further configured to encode the M audio signals based on a quantity of bits allocated to the M audio signals, to obtain an encoded bitstream.

In a possible implementation, the apparatus further includes the transceiver module <NUM>, configured to receive the encoded bitstream. The processing module <NUM> is further configured to obtain a bit quantity of each of the M audio signals and reconstruct the M audio signals based on the bit quantity of each of the M audio signals and the encoded bitstream.

The apparatus in this embodiment may be configured to execute the technical solution of the method embodiment shown in <FIG>. Implementation principles and technical effects thereof are similar, and details are not described herein again.

<FIG> is a schematic diagram of a structure of a device according to an embodiment of this application. As shown in <FIG>, the device may be applied to the encoding device or the decoding device in the foregoing embodiments. The device in this embodiment may include a processor <NUM> and a memory <NUM>. The memory <NUM> is configured to store one or more programs. When the one or more programs are executed by the processor <NUM>, the processor <NUM> is enabled to implement the technical solution of the method embodiment shown in <FIG>.

In an implementation process, the steps in the foregoing method embodiments can be implemented by a hardware integrated logical circuit in the processor, or by using instructions in a form of software. The processor may be a general-purpose processor, a digital signal processor (digital signal processor, DSP), an application-specific integrated circuit (application-specific integrated circuit, ASIC), a field programmable gate array (field programmable gate array, FPGA) or another programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like. The steps of the methods disclosed with reference to this application may be directly performed by a hardware encoding processor, or may be performed by a combination of hardware and a software module in an encoding processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The storage medium is located in the memory. The processor reads information in the memory and completes the steps in the foregoing methods in combination with hardware of the processor.

The memory in the foregoing embodiments may be a volatile memory or a nonvolatile memory, or may include both a volatile memory and a nonvolatile memory. The non-volatile memory may be a read-only memory (read-only memory, ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (electrically EPROM, EEPROM), or a flash memory. The volatile memory may be a random access memory (random access memory, RAM), used as an external cache. By way of example, and not limitation, many forms of RAMs may be used, for example, a static random access memory (static RAM, SRAM), a dynamic random access memory (dynamic RAM, DRAM), a synchronous dynamic random access memory (synchronous DRAM, SDRAM), a double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), an enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), a synchronous link dynamic random access memory (synchlink DRAM, SLDRAM), and a direct rambus random access memory (direct rambus RAM, DR RAM). It should be noted that the memory of the systems and methods described in this specification includes but is not limited to these and any memory of another proper type.

For example, division into the units is merely logical function division and may be other division in actual implementation.

When the functions are implemented in the form of a software functional unit and sold or used as an independent product, the functions may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of this application essentially, or the part contributing to the conventional technology, or some of the technical solutions may be implemented in a form of a software product. The computer software product is stored in a storage medium, and includes several instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) to perform all or some of the steps of the methods described in embodiments of this application. The foregoing storage medium includes various media that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (read-only memory, ROM), a random access memory (random access memory, RAM), a magnetic disk, or an optical disc.

Claim 1:
A bit allocation method for an audio signal (<NUM>), comprising:
obtaining T audio signals in a current frame, wherein T is a positive integer (<NUM>);
determining a first audio signal set based on the T audio signals, wherein the first audio signal set comprises M audio signals, M is a positive integer, the T audio signals comprise the M audio signals, and T ≥ M (<NUM>);
determining M priorities of the M audio signals in the first audio signal set (<NUM>); and
performing bit allocation on the M audio signals based on the M priorities of the M audio signals (<NUM>);
wherein the determining M priorities of the M audio signals in the first audio signal set comprises:
obtaining a scene grading parameter of each of the M audio signals; and
determining the M priorities of the M audio signals based on the scene grading parameter of each of the M audio signals;
the determining the M priorities of the M audio signals based on the scene grading parameter of each of the M audio signals includes: obtaining a first priority of the first audio signal based on the first scene grading parameter; obtaining a second priority of the first audio signal based on the second scene grading parameter; and obtaining the priority of the first audio signal based on the first priority and the second priority; wherein,
a method used for obtaining the second priority of the first audio signal is different from that used for obtaining the first priority of the first audio signal.