VIDEO ENCODING SYSTEM AND VIDEO ENCODING METHOD

In order to make it possible to perform compression encoding of video more quickly and by an easier method, processing relating to the compression encoding of the video that has a greater speed-increasing effect due to parallel processing is performed by a parallel processing device that can execute the parallel processing more rapidly than a CPU, processing with a high computational load and processing that requires high-speed processing based on sequential processing are performed by a sequential processing device that can execute the sequential processing more rapidly than the CPU, and processing that involves a higher frequency of alteration of content of an algorithm is performed by the CPU.

TECHNICAL FIELD

The present invention relates to a video encoding system and a video encoding method.

BACKGROUND ART

As the definition of a video is increased, a load of video encoding processing is increased.

As a method of achieving the video encoding processing, first, there is a method of using a hardware encoder such as a dedicated large scale integrated circuit (LSI). There are a method of achieving encoding processing using a central processing unit (CPU), a method of achieving encoding processing using a CPU and a field programmable gate array (FPGA), and a method of achieving encoding processing using a CPU and a graphics processing unit (GPU). For example, PTL 1 describes that processing with a heavy load is offloaded to a GPU, and encoding processing is achieved using a CPU and the GPU.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

The method of using a hardware encoder using a dedicated LSI has advantages of easy size reduction and high reliability. However, this method has a very high development cost and a long development period.

The method of achieving all the processing using the CPU has an advantage that development is easy as compared with the method of using a hardware encoder. However, this method is slower in processing speed than the method of using a hardware encoder.

The method of using the CPU and the GPU and the method of using the CPU and the FPGA can speed up compression encoding as compared with the method of achieving all the processing by using the CPU. Development is easier than the method of using a hardware encoder using a dedicated LSI.

However, with the advent of new compression encoding techniques such as versatile video coding (VVC), it is desired to further speed up the compression encoding processing by a simpler method. For example, the processing time of VVC is about ten times longer than that of H.265(high efficiency video coding (HEVC)). Therefore, it is desirable to further speed up such compression encoding processing by a simpler method (a method with low development cost, a method with short development period, or the like).

An object of the present invention is to provide a video encoding system and a video encoding method that enable compression encoding of a video to be performed at a higher speed by a simpler method.

Solution to Problem

According to an aspect of the present invention, a video encoding system includes a CPU, a parallel processing device capable of executing parallel processing at a higher speed than the CPU, and a sequential processing device capable of executing sequential processing at a higher speed than the CPU, in which the parallel processing device performs processing having a greater speed-increasing effect by performing the parallel processing among pieces of processing related to compression encoding of a video, the sequential processing device performs processing with a high computational load or processing that requires high-speed processing based on the sequential processing among the pieces of processing related to the compression encoding, and the CPU performs processing that involves a high frequency of alteration of content of an algorithm among the pieces of processing related to the compression encoding.

According to another aspect of the present invention, a video encoding method includes performing, among pieces of processing related to compression encoding of a video, processing having a greater speed-increasing effect by performing parallel processing by a parallel processing device capable of executing the parallel processing at a higher speed than a CPU, performing processing with a high computational load or processing that requires high-speed processing based on sequential processing by a sequential processing device capable of executing the sequential processing at a higher speed than the CPU, and performing processing that involves a high frequency of alteration of content of an algorithm by the CPU.

Advantageous Effects of Invention

According to the present invention, compression encoding of a video can be performed at a higher speed by a simpler method.

EXAMPLE EMBODIMENT

First Example Embodiment

The first example embodiment of the present invention is described.

FIG.1illustrates a configuration example of a video encoding system10of the present example embodiment. The video encoding system10of the present example embodiment includes a CPU11, a parallel processing device12, and a sequential processing device13.

It is assumed that the parallel processing device12can execute parallel processing at a higher speed than the CPU11. It is assumed that the sequential processing device13can execute sequential processing at a higher speed than the CPU11.

The parallel processing device12performs processing having a greater speed-increasing effect by performing the parallel processing among pieces of processing related to compression encoding of a video. The sequential processing device13performs processing with a high computational load or processing that requires high-speed processing based on sequential processing among pieces of processing related to compression encoding. The CPU11performs processing that involves a high frequency of alteration of content of an algorithm among pieces of processing related to compression encoding.

By configuring the video encoding system10in this manner, in the video encoding system10, the parallel processing device12capable of executing the parallel processing at a higher speed than the CPU11performs processing having a greater speed-increasing effect by performing the parallel processing. The sequential processing device13capable of executing the sequential processing at a higher speed than the CPU11performs the processing with a high computational load or the processing that requires high-speed processing based on the sequential processing. The CPU11performs the processing that involves a high frequency of alteration of content of an algorithm. This makes it possible to perform the compression encoding at a higher speed than the method of using only the CPU, the method of using the two: the CPU and the FPGA, and the method of using the two: the CPU and the GPU. The compression encoding can be performed by a simpler method than the method of using a hardware encoder using a dedicated LSI. Therefore, the compression encoding of a video can be performed at a higher speed by a simpler method.

Next,FIG.2illustrates an example of an operation of the video encoding system10of the present example embodiment.

The parallel processing device12performs processing having a greater speed-increasing effect by performing the parallel processing among pieces of processing related to compression encoding of a video (Step S101). The sequential processing device13performs processing with a high computational load or processing that requires high-speed processing based on sequential processing among pieces of processing related to compression encoding (Step S102). The CPU11performs processing that involves a high frequency of alteration of content of an algorithm among pieces of processing related to compression encoding (Step S103).

The video encoding system10operates as described above in such a way that the parallel processing device12capable of executing the parallel processing at a higher speed than the CPU11performs processing having a greater speed-increasing effect by performing the parallel processing. The sequential processing device13capable of executing the sequential processing at a higher speed than the CPU11performs the processing with a high computational load or the processing that requires high-speed processing based on the sequential processing. The CPU11performs the processing that involves a high frequency of alteration of content of an algorithm. This makes it possible to perform the compression encoding at a higher speed than the method of using only the CPU, the method of using the two: the CPU and the FPGA, and the method of using the two: the CPU and the GPU. The compression encoding can be performed by a simpler method than the method of using a hardware encoder using a dedicated LSI. Therefore, the compression encoding of a video can be performed at a higher speed by a simpler method.

As described above, in the first example embodiment of the present invention, in the video encoding system10, the parallel processing device12capable of executing the parallel processing at a higher speed than the CPU11performs processing having a greater speed-increasing effect by performing the parallel processing. The sequential processing device13capable of executing the sequential processing at a higher speed than the CPU11performs the processing with a high computational load or the processing that requires high-speed processing based on the sequential processing. The CPU11performs the processing that involves a high frequency of alteration of content of an algorithm. This makes it possible to perform the compression encoding at a higher speed than the method of using only the CPU, the method of using the two: the CPU and the FPGA, and the method of using the two: the CPU and the GPU. The compression encoding can be performed by a simpler method than the method of using a hardware encoder using a dedicated LSI. Therefore, the compression encoding of a video can be performed at a higher speed by a simpler method.

Second Example Embodiment

Next, a video encoding system20of the second example embodiment of the present invention is described. In the present example embodiment, an example of a case where the system of compression encoding of a video is VVC is described.

First,FIG.3illustrates a configuration example of the video encoding system20of the present example embodiment. The video encoding system20of the present example embodiment includes a CPU21, a parallel processing device22, and a sequential processing device23.

Although the CPU21can easily alter a processing algorithm, the processing speed is not faster than that of the FPGA. In the case of processing capable of parallel processing, the processing time of the CPU21is longer than that of the GPU. Therefore, although the necessity of high-speed processing is not so high, the CPU21is suitable for executing processing that involves a high frequency of alteration of content of an algorithm.

The parallel processing device22is a device capable of executing parallel processing at a higher speed than the CPU21, and is equipped with, for example, a GPU. The GPU can execute the parallel processing at high speed. Therefore, the GPU is suitable for executing processing having a greater speed-increasing effect by the parallel processing.

The sequential processing device23is a device capable of executing the sequential processing at a higher speed than the CPU21, and is equipped with, for example, an FPGA. The FPGA can execute the sequential processing at high speed. However, the FPGA takes time and effort to develop, and the processing content is restricted by the circuit scale. Therefore, the FPGA is suitable for executing processing with a high computational load or processing that requires high-speed processing based on sequential processing among pieces of processing that involve a low frequency of alteration of content of an algorithm. The FPGA is also suitable for performing processing that is difficult to perform in parallel processing due to a dependency relationship with other processing or the like.

In the video encoding system20of the present example embodiment, the parallel processing device22performs processing having a greater speed-increasing effect by performing the parallel processing among pieces of processing related to compression encoding of a video. The sequential processing device23performs processing with a high computational load or processing that requires high-speed processing based on sequential processing among pieces of processing related to compression encoding. The CPU21performs processing that involves a high frequency of alteration of content of an algorithm among pieces of processing related to compression encoding.

Data transfer between the sequential processing device23and the parallel processing device22may be performed via the CPU21or may be performed without the CPU21.

For example, the parallel processing device22includes a pre-filter201, a pre-analysis unit202, a loop filter213, and an inter-prediction unit214.

For example, the sequential processing device23includes a forward two-dimensional orthogonal conversion unit205, a quantization unit206, an inverse quantization unit207, an inverse two-dimensional orthogonal conversion unit208, an arithmetic encoding unit209, and an intra-prediction unit212.

For example, the CPU21includes a rate control unit218.

A block dividing unit203is connected to the pre-analysis unit202and a subtraction unit204. The subtraction unit204is connected to the block dividing unit203, the forward two-dimensional orthogonal conversion unit205, and a switching unit215. An addition unit210is connected to the inverse two-dimensional orthogonal conversion unit208, the switching unit215, the intra-prediction unit212, and the loop filter213. The switching unit215is connected to the intra-prediction unit212, the inter-prediction unit214, the addition unit210, and the subtraction unit204.

The pre-filter201is a filter applied to an input image for the purpose of reducing encoding complexity. The processing performed by the pre-filter201is processing having a great speed-increasing effect by the parallel processing. Therefore, the parallel processing device22performs the processing of the pre-filter201.

The pre-analysis unit202analyzes an image processed by the pre-filter201. The analysis result is used for determining a scene change or the like, and is used for determining various encoding parameters by the rate control unit218. The processing performed by the pre-analysis unit202is processing having a great speed-increasing effect by the parallel processing and that involves a low frequency of alteration of content of an algorithm. Therefore, the parallel processing device22performs the processing of the pre-analysis unit202.

The block dividing unit203divides the image output from the pre-analysis unit202into blocks of encoding processing units. The processing of the block dividing unit203may be performed by any of the CPU21, the parallel processing device22, and the sequential processing device23.

The subtraction unit204calculates a difference between the image output from the block dividing unit203and a prediction image output from the switching unit215.

The processing of the subtraction unit204may be performed by any of the CPU21, the parallel processing device22, and the sequential processing device23.

The forward two-dimensional orthogonal conversion unit205performs frequency conversion on the image output from the subtraction unit204. The quantization unit206performs quantization on the output of the forward two-dimensional orthogonal conversion unit205. The forward two-dimensional orthogonal conversion unit205and the quantization unit206adjust (increase or reduce) the value of the transmission bit rate by frequency conversion and quantization.

The inverse quantization unit207performs inverse quantization on the output of the quantization unit206. The inverse two-dimensional orthogonal conversion unit208performs inverse two-dimensional orthogonal conversion on the output of the inverse quantization unit207. The inverse quantization and the inverse two-dimensional orthogonal conversion are performed for prediction in the intra-prediction unit212and the inter-prediction unit214.

The processing performed by the forward two-dimensional orthogonal conversion unit205, the quantization unit206, the inverse quantization unit207, and the inverse two-dimensional orthogonal conversion unit208is processing having a moderate speed-increasing effect by the parallel processing and requiring a moderate amount of processing. Therefore, the parallel processing device22or the sequential processing device23performs the processing of the forward two-dimensional orthogonal conversion unit205, the quantization unit206, the inverse quantization unit207, and the inverse two-dimensional orthogonal conversion unit208.FIG.3is an example of the video encoding system20in a case where the sequential processing device23performs the processing of the forward two-dimensional orthogonal conversion unit205, the quantization unit206, the inverse quantization unit207, and the inverse two-dimensional orthogonal conversion unit208.FIG.4is an example of a video encoding system30in a case where the parallel processing device22performs the processing of the forward two-dimensional orthogonal conversion unit205, the quantization unit206, the inverse quantization unit207, and the inverse two-dimensional orthogonal conversion unit208.

The arithmetic encoding unit209encodes the output of the quantization unit206based on the generation frequency of information “0” and “1” to be encoded. The more the generation frequency is biased, the smaller the transmission capacity of an output bit stream is. The processing performed by the arithmetic encoding unit209is processing having a moderate speed-increasing effect by the parallel processing, a low amount of processing, and that involves a low frequency of alteration of content of an algorithm. Therefore, the sequential processing device23performs the processing of the arithmetic encoding unit209.

The addition unit210adds the output of the inverse two-dimensional orthogonal conversion unit208and the prediction image output from the switching unit215, and outputs the result to the intra-prediction unit212and the loop filter213. The processing of the addition unit210may be performed by any of the CPU21, the parallel processing device22, and the sequential processing device23.

The intra-prediction unit212performs the prediction on an encoding target block in a screen by using only an encoding target picture. The processing performed by the intra-prediction unit212is processing having a moderate speed-increasing effect by the parallel processing and requiring a moderate amount of processing. Therefore, the parallel processing device22or the sequential processing device23performs the processing of the intra-prediction unit212.FIG.3is a configuration example of the video encoding system20in a case where the processing of the intra-prediction unit212is performed by the sequential processing device23.FIG.4is a configuration example of the video encoding system20in a case where the processing of the intra-prediction unit212is performed by the parallel processing device22. The sequential processing device23may perform the processing of the arithmetic encoding unit209and the processing of the intra-prediction unit212, and the processing of the forward two-dimensional orthogonal conversion unit205, the quantization unit206, the inverse quantization unit207, and the inverse two-dimensional orthogonal conversion unit208may be performed by the parallel processing device22.

The loop filter213is a filter applied to an image output from the switching unit215to bring the image closer to the input image. The processing performed by the loop filter213is processing having a great speed-increasing effect by the parallel processing.

Therefore, the parallel processing device22performs the processing of the loop filter213.

The inter-prediction unit214performs inter-screen prediction on an encoding target block by also using a picture other than an encoding target with respect to the output of the loop filter213. The processing performed by the inter-prediction unit214is processing having a great speed-increasing effect by the parallel processing. Therefore, the parallel processing device22performs the processing of the inter-prediction unit214.

The switching unit215selects an appropriate image from an intra-prediction image supplied from the intra-prediction unit212and an inter-prediction image supplied from the inter-prediction unit214, and outputs the selected image as a prediction image. The processing of the switching unit215may be performed by any of the CPU21, the parallel processing device22, and the sequential processing device23.

The rate control unit218calculates the quantization granularity of a picture to be encoded next by using information regarding the input image obtained from the pre-analysis unit202and encoded information (bit number) obtained from the arithmetic encoding unit209. The rate control unit218controls the transmission rate of a bit stream output from the arithmetic encoding unit209by calculating the quantization granularity of a picture to be encoded next. The processing of the rate control unit218is processing that does not require the parallel processing and has a higher frequency of alteration of an algorithm. Therefore, the CPU21performs the processing of the rate control unit218.

By configuring the video encoding system20in this manner, in the video encoding system20, the parallel processing device22capable of executing the parallel processing at a higher speed than the CPU21performs processing having a greater speed-increasing effect by performing the parallel processing. The sequential processing device23capable of executing the sequential processing at a higher speed than the CPU21performs the processing with a high computational load or the processing that requires high-speed processing based on the sequential processing. The CPU21performs the processing that involves a high frequency of alteration of content of an algorithm. This makes it possible to perform the compression encoding at a higher speed than the method of using only the CPU, the method of using the two: the CPU and the FPGA, and the method of using the two: the CPU and the GPU. The compression encoding can be performed by a simpler method than the method of using a hardware encoder using a dedicated LSI. Therefore, the compression encoding of a video can be performed at a higher speed by a simpler method.

Next, an operation example of the video encoding system20of the present example embodiment is described with reference toFIG.2.

The parallel processing device22performs processing having a greater speed-increasing effect by performing the parallel processing among pieces of processing related to compression encoding of a video (Step S101). In the case of the present example embodiment, the processing performed by the parallel processing device22includes pre-filter processing, pre-analysis processing, loop filter processing, and inter-prediction processing. The processing performed by the parallel processing device22may further include processing that is not performed by the sequential processing device23among forward two-dimensional orthogonal conversion processing, quantization processing, inverse quantization processing, inverse two-dimensional orthogonal conversion processing, and intra-prediction processing.

The sequential processing device23performs processing with a high computational load or processing that requires high-speed processing based on sequential processing among pieces of processing related to compression encoding (Step S102). In the case of the present example embodiment, the processing performed by the sequential processing device23includes arithmetic encoding processing. The processing performed by the sequential processing device23may further include at least one of the forward two-dimensional orthogonal conversion processing, the quantization processing, the inverse quantization processing, the inverse two-dimensional orthogonal conversion processing, and the intra-prediction processing.

The CPU21performs processing that involves a high frequency of alteration of content of an algorithm among pieces of processing related to compression encoding (Step S103). In the case of the present example embodiment, the processing performed by the CPU includes rate control processing.

The video encoding system20operates as described above in such a way that the parallel processing device22capable of executing the parallel processing at a higher speed than the CPU21performs processing having a greater speed-increasing effect by performing the parallel processing. The sequential processing device23capable of executing the sequential processing at a higher speed than the CPU21performs the processing with a high computational load or the processing that requires high-speed processing based on the sequential processing. The CPU21performs the processing that involves a high frequency of alteration of content of an algorithm. This makes it possible to perform the compression encoding at a higher speed than the method of using only the CPU, the method of using the two: the CPU and the FPGA, and the method of using the two: the CPU and the GPU. The compression encoding can be performed by a simpler method than the method of using a hardware encoder using a dedicated LSI. Therefore, the compression encoding of a video can be performed at a higher speed by a simpler method.

As described above, in the second example embodiment of the present invention, in the video encoding system20, the parallel processing device22capable of executing the parallel processing at a higher speed than the CPU21performs processing having a greater speed-increasing effect by performing the parallel processing. The sequential processing device23capable of executing the sequential processing at a higher speed than the CPU21performs the processing with a high computational load or the processing that requires high-speed processing based on the sequential processing. The

CPU21performs the processing that involves a high frequency of alteration of content of an algorithm. This makes it possible to perform the compression encoding at a higher speed than the method of using only the CPU, the method of using the two: the CPU and the FPGA, and the method of using the two: the CPU and the GPU. The compression encoding can be performed by a simpler method than the method of using a hardware encoder using a dedicated LSI. Therefore, the compression encoding of a video can be performed at a higher speed by a simpler method.

In the present example embodiment, the case where the system of compression encoding of a video is VVC is described. However, the video encoding system20of the present example embodiment is also applicable to a case where the compression encoding system is H.265 (high efficiency video coding (HEVC)), H.264/moving picture experts group (MPEG)-4 advanced video coding (AVC), Windows (registered trademark) media video (WMV), alliance for open media video 1 (AV1), VP9, or the like.

A configuration example of hardware resources for achieving the video encoding system (10,20) according to each of the above-described example embodiments of the present invention using one information processing device (computer) is described. The video encoding system may be achieved by using a plurality of, at least two or more, information processing devices physically or functionally. The video encoding system may be achieved as a dedicated device, or a general-purpose device may be used. Only some of the functions of the video encoding system may be achieved by using the information processing device.

FIG.5is a diagram schematically illustrating a hardware configuration example of an information processing device capable of achieving the video encoding system of each of the example embodiments of the present invention. An information processing device90includes a communication interface91, an input/output interface92, a computing device93, a storage device94, a nonvolatile storage device95, and a drive device96.

For example, the CPU11, the parallel processing device12, and the sequential processing device13inFIG.1correspond to the computing device93.

The communication interface91is a communication means for the video encoding system of each of the example embodiments to communicate with an external device in at least one of wired and wireless manners. In a case where the video encoding system is achieved by using at least two information processing devices, these devices may be communicably connected via the communication interface91.

The input/output interface92is a man-machine interface such as a keyboard as an example of an input device or a display as an output device.

The computing device93is achieved by, for example, a computation processing device such as a central processing unit (CPU) or a microprocessor, or a plurality of electric circuits. For example, the computing device93can read various programs stored in the nonvolatile storage device95into the storage device94and execute processing according to the read programs.

The storage device94is a memory device such as random access memory (RAM) that can be referred to from the computing device93, and stores programs, various data, and the like. The storage device94may be a volatile memory device.

The nonvolatile storage device95is, for example, a nonvolatile storage device such as read only memory (ROM), flash memory, or the like, and can store various programs, data, and the like.

The drive device96is, for example, a device that processes reading and writing of data recorded in a recording medium97described below.

The recording medium97is any recording medium capable of recording data, for example, an optical disk, a magneto-optical disk, semiconductor flash memory, or the like.

Each of the example embodiments of the present invention may be achieved, for example, by configuring the video encoding system by the information processing device90illustrated inFIG.5and supplying a program capable of achieving the functions described in each of the example embodiments described above to the video encoding system.

In this case, the computing device93executes the program supplied to the video encoding system, and the example embodiments can be achieved. Not all but some of the functions of the video encoding system can be configured by the information processing device90.

The program described above may be recorded in the recording medium97, and the program described above may be appropriately configured to be stored in the nonvolatile storage device95at a shipping stage, an operation stage, or the like of the video encoding system. In this case, the method of supplying the program described above may employ a method of installing the program in the video encoding system by using an appropriate jig in a manufacturing stage before shipment, an operation stage, or the like. As a method of supplying the program described above, a general procedure such as a method of downloading the program from the outside via a communication line such as the Internet may be adopted.

This application is based upon and claims the benefit of priority from Japanese patent application No. 2020-052601, filed on Mar. 24, 2020, the disclosure of which is incorporated herein in its entirety by reference.

REFERENCE SIGNS LIST