Abstract:
A video encoding method with support for editing when scene changed. The video encoding method reads and stores the pictures by the display order and detects whether the scene change occurred. The method encodes the pictures by the coding order when there are not scenes changed and encodes the pictures by a special coding process when there are scenes changed. Because the video encoding method encodes the pictures with considering the states of scenes changed and generates a new GOP when a scene change occurred, the video sequence can be cut into two parts by an image editing process without re-encoding. Therefore, the video can be edited without any loss and the editing performance of the editing process can be better.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     1. Field of the Invention  
         [0002]     The invention relates to a video encoding method, especially to a video encoding method with support for editing when scene changed.  
         [0003]     2. Description of the Related Art  
         [0004]     In MPEG (Moving Pictures Experts Group), there is three picture types: I-picture, P-picture and B-picture. I-pictures are coded without reference to other pictures. They provide access points to the coded sequence where decoding can begin, but are coded with only moderate compression. P-pictures are coded more efficiently using motion compensated prediction from a past I-picture or P-picture and are generally used as a reference for further prediction. B-pictures provide the highest degree of compression but require both past and future reference pictures for motion compensation. B-pictures are never used as references for prediction. The organization of the three picture types in a sequence is very flexible. The choice is left to the encoder and will depend on the requirements of the application.  
         [0005]     Because the B-pictures reference the past and future reference pictures, so the encoding process of the B-pictures has to be delayed until the future reference picture is coded. Therefore, the display order is different to the coding order. This is called the reordering of B-pictures.  
         [0006]     In MPEG-1, there is a group-of-pictures (hereinafter called as GOP) structure used to enclose some pictures into a group for manipulation. A GOP contains one I-picture, some P-pictures and some B-pictures. A GOP begins with an I-picture, and ends before the next I-pictures, in the coding order. In MPEG-2, the GOP structure becomes an option.  
         [0007]     Generally, an encoder employs a fixed GOP structure. The size of a GOP is defined as N, and the distance between two reference pictures is defined as M.  FIG. 1  illustrates a GOP with N=15 and M=3.  
         [0008]     Typically, if the input signal for the encoder is in NTSC (National Television System Committee) format (29.97 fps), the GOP structure with N=15 and M=3 is used. If the input signal is in PAL (25 fps) or film format (24 fps), the GOP structure with N=12 and M=3 is used. These fixed default settings can achieve a good balance between the complexity of encoder and the coding performance for most videos.  
         [0009]     Typically, the editing process would cut the whole video sequence into pieces based on the scene, and then rearrange them to form a new video sequence. If a video sequence is coded with a fixed pattern composed with only I- and P-pictures, like IPPPPIPPPP . . . , the situation is pretty simple. If a scene change occurred in an I-picture, the video sequence can be cut into two parts without any loss. If a scene change occurred in a P-picture, the former part is no problem, but the remaining part has to be re-encoder. The first P-picture has to be decoded and then re-encode to an I-picture. However, because the re-encoded I-picture differs from the original P-picture, there will be some error propagations. Re-encode the whole remaining part of the GOP until the next I-picture would be a better solution, but we would remind that re-encoding degrades the image quality significantly.  
         [0010]     If there are B-pictures in the coded sequence, video editing becomes more complex. Please reference to  FIG. 2 . If a scene change occurred in the picture just after the I-picture in the coding order, like the picture B 4 , cutting from picture  16  can separate the two scenes easily. However, even the picture P 3  and picture B 4  are belong to different scenes, there would be some macroblocks in picture B 4  and B 5  reference the picture P 3 . Therefore the picture B 4  and B 5  have to be re-encoded with only referencing to the picture  16 . Discarding the pictures B 4  and B 5  is the easiest way, but losing the beginning some pictures of a scene would not be acceptable.  
         [0011]     If a scene change occurred in the picture B 5 , the former part and the remaining part of the GOP have some pictures to be re-encoded. The picture B 4  has to be re-encoded to a P-picture and then append to the former part. In the remaining part, the coded data of the picture B 4  is removed and the picture B 5  has to be re-encoded.  
         [0012]     If a scene change occurred in the picture I 6 , the remaining part has only to remove the coded data of the pictures B 4  and B 5 . However, the former part requires a complicate process. One solution is to re-encode the picture B 5  to a P-picture, and then re-encode the picture B 4  by referencing to the pictures P 3  and B 5 . Another solution is to change the two B-pictures to two P-pictures.  
         [0013]     If a scene change occurred in the picture B 7 , the former part needs no any process, but a new I-picture has to be generated for the remaining part. A choice is to change the picture B 7  to an I-picture, and then re-encode the remaining GOP. However, because the B-pictures usually coded with a lower quality than the I- and P-pictures, a better choice would be to change the picture P 9  to an I-picture, and re-encode the remaining GOP. The pictures B 5  and B 6  are B-pictures with only backward reference. This method can also reduce the number of P-pictures to reduce the error caused by referencing to a re-encoded picture.  
         [0014]     If a scene change occurred in the picture B 8 , the former part has only to re-encode the picture B 7  to a P-picture. The remaining part can change the picture P 9  to an I-picture and then re-encode the remaining GOP.  
         [0015]     Finally, if a scene change occurred in the picture P 9 , the former part is processed like the situation of picture  16 . For the remaining part, the picture P 9  has to be changed to an I-picture, and then re-encode the remaining GOP.  
         [0016]     Therefore, all the other situations can be processed like the methods described above, even if the number of B-pictures between two reference pictures increases to three or more.  
         [0017]     Generally, the I-pictures are designed for the purpose of random access and preventing of error propagation. The P-pictures use the motion compensation to remove the temporal redundancy between the current picture and the reference picture to improve the compression performance. However, if there is almost no temporal redundancy between the current picture and the reference picture, for example a scene change, coding a picture as a P-picture can&#39;t obtain any benefit. In this case, an I-picture can achieve the same coding quality with fewer bits. Therefore, an encoder has to detect the existence of a scene change and then start a new GOP. There is already many researches focus on the scene change detection and then how to adjust the rate control algorithm. A general idea is to detect the difference of the current picture and the reference picture from the result of motion estimation. If more than a percentage of macroblocks select the intra-coded mode, the encoder can decide that there is only few temporal redundancy existed, and therefore a scene change can be detected.  
         [0018]     However, if the encoder just start a new GOP when detect a scene change but with no other effort, the re-encoding of some pictures would be unavoidable when the video sequence is being editing, as we described above.  
       SUMMARY OF THE INVENTION  
       [0019]     In view of the above-mentioned problems, an object of the invention is to provide a video encoding method with support for editing when scene changed.  
         [0020]     To achieve the above-mentioned object, the video encoding method with support for editing when scene changed of the present invention encodes the pictures by the coding order when there are not scenes changed and encodes the pictures by a special coding process when there are scenes changed. Because the video encoding method encodes the pictures with considering the states of scenes changed and generates a new GOP when a scene change occurred, the video sequence can be cut into two parts by an image editing process without re-encoding.  
         [0021]     Therefore, the video can be edited without any loss and the editing performance of the editing process can be better. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0022]      FIG. 1  illustrates a GOP with N=15 and M=3.  
         [0023]      FIG. 2  illustrates a video sequence with a B-picture.  
         [0024]      FIG. 3  illustrates a GOP with a fixed structure.  
         [0025]      FIG. 4  illustrates a first example of GOP with display order and coding order.  
         [0026]      FIG. 5  illustrates a second example of GOP with display order and coding order.  
         [0027]      FIG. 6  illustrates a third example of GOP with display order and coding order.  
         [0028]      FIG. 7  shows the flowchart of the video encoding method with support for editing when scene changed of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0029]     The video encoding method with support for editing when scene changed of the invention will be described with reference to the accompanying drawings.  
         [0030]     First, an encoder using the video encoding method with support for editing when scene changed of the present invention also has to include the scene change detecting function. The scene change detecting function has to be applied in the display order. This is because the encoder has to know where scene changes and then encodes the pictures before and after the scene change into two GOP.  
         [0031]     Before a scene change is detected, the encoder encodes the video sequence with a fixed GOP structure. Once a scene change is detected, the encoder decides how to encode the following pictures based on the type and position in a GOP of the just coded pictures. Please note that because the B-pictures have to be coded just after the future reference picture being coded, a scene change can be detected far before the coding actually happen.  FIG. 2  depicts an example. The pictures are captured and stored into a buffer by the display order. The picture B 4  and B 5  are captured but can&#39;t be encode until the picture I 6  is coded. Assume that the encoder can encode a picture in each period of capturing a picture. The picture I 6  is captured and then encoded in the same period. In the next period, the picture B 4  is encoded while the picture B 7  is being captured. The picture B 5  is encoded in the same period that the picture B 7  is captured. The picture P 9  needs only the picture I 6  to reference, so it can be captured and encoded in the same period.  
         [0032]     An encoder encodes the video sequence with a fixed GOP structure that the distance between two reference pictures is defined as M and a reference picture (I- or P-picture) is represented as an R. The first B-picture (in the display order) after the forward reference picture R X  is called B X   1 , the second B-picture is called B X   2 , and so on. The final one before the backward reference picture is called B X   M−1 .  FIG. 3  illustrates an example of the GOP in the display order and coding order.  
         [0033]     The process methods for the scene change occurred at different location are described as following.  
         [0034]     A. A Scene Change Occurred in the First B-Picture  
         [0035]     If there is no scene change occurred in the pictures from B A   1  to R B , the picture B A   1 ˜B A   M−1  is captured and stored until the picture R B  is captured and coded. If the scene changed in the picture B B   1 , the pictures until R B  would belong to the former GOP and the pictures from B B   1  would belong to a new GOP. After coding the picture B A   M−1 , if the encoder starts a new GOP and encodes the following pictures without referencing to the picture R B , it can completely separate the video sequence into two parts. An editing process can cut the video sequence from the new GOP without any re-encoding.  
         [0036]     There are two strategies to start a new GOP. One is to start a fixed GOP structure from I-picture. In the above example, the original picture B B   1  is changed to an I-picture R C , the following M−1 pictures are B-picture B C   1 ˜B C   M−1 , the next is a P-picture R D , then are the M−1 B-pictures, and so on.  FIG. 4  illustrates an example of this case.  
         [0037]     However, a new GOP need not be started with an I-picture in the display order. By observing the coding order in  FIG. 4 , we can find that there is no B-pictures between the picture R C  and R D . B-pictures can be coded with lower quality and save the bit rate for the I-pictures and P-pictures. If there are too many reference pictures in a short duration, the result is that each reference picture can&#39;t obtain enough bits to achieve a higher quality. Therefore, the second strategy of starting a new GOP is trying to maintain the ratio on the number of B-pictures and reference pictures. The first M−1 pictures of the new GOP are B-pictures, the next is an I-pictures, following by other M−1 B-pictures, and then a P-picture, and so on.  FIG. 5  illustrates an example of this case.  
         [0038]     Seems that the picture type of each picture is remaining the same as no scene change occurred. Actually, the difference is that the picture B B   1 ˜B B   M−1  have only backward reference to the picture R C . In fact, there may not be M−1 B-pictures before the picture R C , and can be adjust freely.  
         [0039]     B. A Scene Change Occurred in the Second B-Picture  
         [0040]     Please reference to  FIG. 3 . If a scene change occurred in the picture B B   2 , the picture B B   1  belongs to the former GOP and the pictures from B B   2  form a new GOP. The new GOP can be encoded with the same method described in subsection A.  
         [0041]     A GOP can be ended by a reference picture. Therefore the picture B B   1  must be encoded as a reference picture. There is no reason to not encode the picture B B   1  as a P-picture but an I-picture.  FIG. 6  illustrates an example of this case.  
         [0042]     C. A Scene Change Occurred in the n-th B-Picture  
         [0043]     If a scene change occurred in the n-th B-picture after the reference picture R X , 2≦n≦M−1, the pictures until B X   n−1  belong to the former GOP and the pictures from B X   n  form a new GOP. The new GOP is encoded with the same method described in subsection A.  
         [0044]     Based on the method described in section B, the encoder will encode the picture B X   n−1  as a P-picture, and the pictures B X   1 ˜B X   n−2  (if any) are encoded as B-pictures by referencing to the picture R X  and the new generated P-picture.  
         [0045]     D. A Scene Change Occurred in a Reference Picture  
         [0046]     Please reference to  FIG. 3 . If a scene change occurred in the picture R B , the picture B A   M−1  belongs to the former GOP and the pictures from R B  form a new GOP. The former GOP can be coded by the method described in subsection C. The new GOP could be encoded with the same method described in subsection A.  
         [0047]     Finally, we can simplify an algorithm for an encoder to encode the pictures before and after a scene change separately to make the video editing of scenes without any re-encoding.  FIG. 7  shows the flowchart of the video encoding method with support for editing when scene changed of the present invention.  
         [0048]     Step  702 : Capture the picture PIC n  in the display order and detect the scene change.  
         [0049]     Step  704 : If there is no scene change in the picture PIC n , the flowchart jumps to step S 706 . If there is a scene change in the picture PIC n , the flowchart jumps to step S 708 .  
         [0050]     Step  706 : Code pictures in the coding order and jump back to step S 702 .  
         [0051]     Step  708 : If the picture PIC n−1  is not coded as a reference picture, jump to step S 710 . If the picture PIC n−1  is coded as a reference picture, jump to step S 716 .  
         [0052]     Step  710 : If there are B-pictures preceding a previous reference picture, finish coding the B-pictures.  
         [0053]     Step S 712 : Encode the picture PIC n−1  as a P-picture.  
         [0054]     Step S 714 : If there are B-pictures preceding the picture PIC n−1 , coding the B-pictures and jump to step S 718 .  
         [0055]     Step S 716 : If there are B-pictures preceding the picture PIC n−1 , coding the B-pictures and jump to step S 718 .  
         [0056]     Step S 718 : Start a new GOP and encode the picture P n+M−1  as an I-picture.  
         [0057]     Step S 720 : Encode the pictures P n ˜P n+M−2  as B-pictures with only referencing to the picture P n+M−1.    
         [0058]     Because the present invention starts a new GOP when the scene change occurred, the image editor can cut the video sequence directly without re-coding. Therefore, the performance of the image editor can be increased and the image quality will not be loss.  
         [0059]     While certain exemplary embodiments have been described and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not restrictive on the broad invention, and that this invention should not be limited to the specific construction and arrangement shown and described, since various other modifications may occur to those ordinarily skilled in the art.