MOVING PICTURES ENCODING WITH CONSTANT OVERALL BIT RATE

Abstract: A method and apparatus for controlling bit rates used in a moving pictures encoder, such as an MPEG standard encoder. A sequence of moving pictures is divided into segments each of which comprises one or more group of pictures. A constant overall bit rate is specified for the sequence of pictures, but variable bit rate encoding used within each segment. A difference between the number of bits allocated for encoding the segment and the actual bits used for encoding is determined, and the difference distributed over one or more subsequent segments.

Description: MOVING PICTURES ENCODING WITH CONSTANT OVERALL BIT-RATE Field of the invention The present invention relates to a method and apparatus for encoding moving pictures. In particular, the present invention relates to a method and apparatus for performing variable bit rate control in a digital video encoder while maintaining a particular overall bit-rate. Background of the Invention One of the main obstacles faced by industry dealing with digital video processing, storage and transmission is the large amount of data needed to represent analog video in the digital domain. Accordingly, digital compression is often applied to moving pictures to achieve a reduction in required transmission bandwidth or storage size. One variety of such compression techniques can be derived from the ISO/IEC MPEG Standards, the ISO/IEC 11172-3 (MPEG-1), the ISO/IEC 13818-2 (MPEG-2) and the MPEG-2 TM5 (test model 5), developed by the Moving Picture Experts Group of the International Organisation for Standardization. The disclosures of those standards documents are hereby expressly incorporated into this specification by reference. MPEG-1 is the compression standard used in VideoCD while MPEG-2 is the video compression standard used in DVD and many digital broadcasting systems. The MPEG standards specify only the syntax of the compressed bitstreams and method of decoding. The method of implementation in the encoder is left to the developer, and any form of encoder may be employed as long as the resulting bitstream conforms with the specified syntax. In certain applications such as video storage device (recorder), it is possible to use variable bit rate (VBR) encoding. A VBR encoder is able to vary its output bit-rate over a larger range than a CBR encoder, and this would generate an output which has a more constant WO 99/49664 -.,-,--„- PCT/SG98/00022 visual quality. An example of a VBR encoder is described in United States patent number 5,650,860, entitled "Adaptive Quantization". In order to maintain a maximum bit rate allowed by the target storage device as well as an overall bit-rate which enables input picture sequence to be stored into a defined storage space, such VBR encoders utilise multiple encoding passes. In the first encoding pass, the bit utilisation information is determined for each scene or each picture in the input sequence. This may be done by fixing the reference quantisation step size and disabling the VBV control. The determined bit utilisation information is then used to generate a bit budget for each scene or picture such that an overall target number of bits to code the sequence is fixed, and that the maximum bit rate is not violated. In cases that bit utilisation information obtained is not close to that required for generating the bit budget, steps from the first coding pass must be repeated with an adjusted reference quantisation step-size. The input sequence is coded in a final pass using the generated bit budget information to achieve the target bits or overall bit rate. Multiple-pass VBR encoders requires large storage memory for intermediate bit utilisation information, and large computation needs for the additional passes and bit budget generation. Furthermore, such a VBR encoder cannot process input sequences in real-time as required by certain applications. Summary of the Invention It is an object of the present invention to provide a single-pass real-time variable bit rate encoder for moving pictures. It is also an object of the present invention to provide variable bit rate encoding of moving p'ctures such that the change in encoded picture quality from one scene to another is minimised. A further object is to provide a real-time variable bit rate encoding algorithm which produces a constant overall bit rate. In particular, the present invention encodes an input moving pictures sequence one 3 - segment at a time according to a target encoding quality which is determined by a target segment bit rate. The target segment bit rate of a current segment is preferably derived from the differences between the target segment bit rates and the actual coding bit rates of previous or previous few encoded segments. To maintain consistent encoding quality for all pictures within a segment, the actual target bit rate for encoding the pictures is made variable according to their scene complexities as well as the target encoding quality of the segment. As the target encoding quality of each segment is modified based on the differences between the target and the actual bit rates of previous or previous few encoded segments, the change of target encoding quality from segment to segment is made relatively smooth compared to that of a Constant Bit-Rate Encoder, and furthermore, the overall bit rate of encoding is maintained constant. In accordance with the present invention, there is provided a method for use in a moving pictures encoder for encoding a sequence of segments each having at least one image, comprising the steps of: a) determining an overall target bit rate for encoding the sequence of images; b) determining a bit allocation and target quantisation step size for encoding a first segment on the basis of a segment target bit rate calculated using said overall target bit rate; c) encoding said first segment using a variable bit rate encoding method according to the target quantisation step size; d) determining a difference between the number of bits used to encode said first segment and said first segment bit allocation; e) distributing said difference for use in encoding at least one subsequent segment to determine a subsequent segment bit allocation; f) determining a new target quantisation step size for encoding a said subsequent segment on the basis of a new target segment bit rate calculated using 4 - said segment target bit rate and the distributed difference; and g) encoding said subsequent segment using a variable bit rate encoding method according to the new target quantisation step size; wherein variable bit rate encoding is employed for encoding pictures within a segment whilst maintaining a substantially constant bit rate over said sequence. The present invention also provides a method for encoding moving pictures in a moving pictures encoder wherein a sequence of images are provided as input, the sequence of images comprising a plurality of segments each having a plurality of images, the method including: a) determining an overall target bit rate for encoding the sequence of images; b) maintaining a distribution record of bits from at least one previously encoded segment allocated for use in encoding at least one segment to be encoded; c) deteιτrrj-αing a target segment bit rate for a segment of the sequence of images on the basis of the overall target bit rate and a bit rate change calculated from the corresponding allocated bits from the distribution record; d) determining a target segment encoding quality from the target segment bit rate, a preceding target segment bit rate and a preceding target segment encoding quality,; and e) encoding the images of the segment according to the target segment encoding quality using a variable bit rate encoding technique taking into account scene complexities of the images in the segment; wherein maintaining said distribution record includes determining a difference between the number of bits used to encode a particular segment and the number of bits allocated for encoding the particular segment on the basis of the target segment encoding quality. According to the current invention, a moving pictures sequence is divided into segments. The size of each segment may be suitably determined. Each segment is encoded with a target encoding quality derived from its target segment bit rate. A variable bit rate (VBR) - 5 encoder is utilised to encode the segment according to its target encoding quality. The target segment bit rate of an initial segment is obtained from a user defined target overall bit rate. After encoding the segment, the difference between the actual bit rate used and the target segment bit rate is obtained. This difference is propagated to the next or next few segments to be coded. This process is repeated for each segment; therefore, for each subsequent segment, a new target segment bit rate is determined from the user defined target overall bit rate and the differences between the target segment bit rates and actual bit rates of previous or previous few segments. The present invention further provides a method for controlling bit allocation in a moving pictures encoder for encoding a sequence of images comprising a plurality of segments each having a plurality of images, the method including, for each segment: determining a difference between a number of bits used for encoding a previous segment and a number of bits allocated for encoding the previous segment; calculating a bits distribution from the determined bits difference and a predetermined distribution function; calculating a bit rate change from the bits distribution and a predetermined number of images in the segment; calculating a target segment bit rate from the bit rate change and a predetermined target overall bit rate for the sequence of images; and determining a target segment encoding quality from the target segment bit rate. The present invention further provides an encoding quality adjustment processor for generating a target segment encoding quality value in a moving pictures encoder for encoding a series of segments each having at least one image using a variable bit rate encoding scheme whilst maintaining a substantially constant overall bit rate, comprising: a bits difference computation means coupled to receive a segment encoding bit utilisation value and a target segment bit rate and generate therefrom a bits difference value representing a difference in bits allocated and bits used for encoding a segment; a bits distribution means coupled to me bits difference computation means for computing at least one bits distribution value from the bits difference value and a predetermined distribution function; a bit rate difference computation means coupled to the bits distribution means for computing a segment bit rate difference from the at least one bits distribution value and a predetermined number of images in a segment; a target segment bit rate adjustment means coupled to the bit rate difference computation means and e bits difference computation means for computing said target segment bit rate from the segment bit rate difference and a predetermined target overall bit rate for the sequence of segments; and an encoding quality computation means coupled to the target segment bit rate adjustment means for computing a target segment encoding quality value from said target segment bit rate. The present invention further provides a moving pictures encoder comprising: a coding processor for encoding picture data based on macroblocks according to a quantisation step size; a virtual buffer processor coupled to the coding processor for tracking a number of bits used for encoding successive macroblocks in a picture and a number of bits used for encoding successive pictures in a group of pictures; a quantisation step size processor coupled to the coding processor for deteimir-ing said quantisation step size from a target number bits allocated for a picture and the number of bits already used for encoding macroblocks in that picture; a picture bit allocation processor coupled to the quantisation step size processor for determining said target number of bits allocated for a picture from a target bit rate and the number of bits already used for encoding pictures in a current group of pictures; a bit rate adjustment processor coupled to the picture bit allocation processor, the virtual buffer processor and the quantisation step size processor for determining said target bit rate from the number of bits already used for encoding successive pictures in the current group of pictures, a target encoding quantisation step size and an average quantisation step size for pictures in the current group of pictures; and a target encoding quantisation step size processor coupled to the bit rate adjustment processor and me virtual buffer processor for determining said target encoding quantisation step size from a predetermined target overall bit rate and the number of bits used for encoding a preceding group of pictures. Brief Description of the Drawings The invention is described in greater detail hereinafter, by way of example only, with reference to the accompanying drawings, wherein: Figure 1 is a block diagram of the constant bit-rate controller based on TM5; Figure 2 illustrates the difference between a target bit allocation and an actual bits consumption at macroblock level for one frame (a P-picture is chosen for this case), where d* is the virtual buffer fullness at a particular instance; Figure 3 is a block diagram of the variable bit-rate encoder with constant overall bit-rate control; Figure 4 is a block diagram of the constant overall bit-rate controller; Figure 5 illustrates how the difference in bit count for the last segment is redistributed over the next four segments (e.g. f(m) = l/4); and Figure 6 is a flow chart of a VBR algorithm with constant overall bit-rate control. Detailed Description of the Preferred Embodiments In a standard MPEG compliant video encoder, a sequence of moving pictures (e.g. video) is input to the encoder where it is compressed with a user defined target bitrate. The target bitrate is set according to the communication channel bandwidth in which the compressed video is to be transmitted, or the storage media capacity in which the compressed video sequence is to be stored. A typical MPEG encoder involves motion estimation/prediction, Inter/Intra classification, discrete cosine transform (DCT) computation, quantisation, zig-zag scanning, variable length coding and rate-control. - 8 - Several different forms of coding can be employed depending upon the character of the input pictures, referred to as I-pictures, P-pictures, or B-pictures. The I-pictures are intra- coded pictures used mainly for random access or scene update. The P-pictures use forward motion predictive coding with reference to previously coded I- or P- pictures (anchor pictures), and the B-pictures use both forward and backward motion predictive/interpolative coding with reference to previously coded I-or P-pictures. Furthermore, a group of pictures (GOP) is formed in encoded order starting with an I- picture and ending with the picture before the next I-picture in the sequence. The pictures are partitioned into smaller and non-overlapping blocks of pixel data called macroblocks (MBs) before encoding. Each MB from a P- or B-picture is subjected to a motion estimation process in which forward motion vectors, and backward motion vectors in the case of a B-picture MB, are determined using reference pictures from a frame buffer. The target macroblock in the current picture is matched with a set of displaced macroblocks in the reference picture, the macroblock that best matches the target macroblock is used as the predicted macroblock. The position of this predicted macroblock is specified by a set of vectors known as motion vectors, which describe the vertical and horizontal displacement between the target and predicted macroblock. For a B-picture, the process is similar except there are two reference pictures: anchor pictures immediately preceding and following the B-picture. With the determined motion vectors, motion compensation is performed where the intra-or inter-picture prediction mode of the MB is first determined according to the accuracy of the motion vectors found, followed by generating the necessary predicted MB. I-pictures are always intra coded while for P and B-pictures, a decision on whether intra or inter coding will be used (at macroblock level) is made depending on which method will give rise to a more efficient coding. Transformation of the macroblock using a DCT is then carried out on the 8x8 pixel blocks within the macroblock. For intra coding, the actual picture data is coded, while for inter 9 - coding the prediction error is coded. This is followed by a quantisation process of the DCT coefficients which involves a quantisation matrix and a quantisation step size. The quantized coefficients are then run-length encoded with variable length codes. The resultant bit usage and statistical data are passed onto a rate control module, which is used for allocating a target number of bits used to encode each picture and each macroblock within the picture. This is an important module in the encoder as it plays a major role in maintaining the quality of the encoded picture. The rate control in MPEG-2 Test Model 5 (TM5) comprises the steps of allocating the target amount of bits for coding each picture, deriving the reference quantisation parameter (Q to be used on each macroblock in a picture, and modulating the Q j based on the activity masking level of the surrounding blocks of d e corresponding macroblock to obtain the modulated quantisation step size (Mquant) used to quantise die macroblock. An objective of this rate controller is to ensure all pictures maintain a similar level of quality. It assumes that the subjective quality of a single coded picture can be qualified with a single number V, described by factor K/Q where AT is a constant particular to a picture type and Q is the quantisation step size for the picture being coded. That is, the rate controller will try to preserve the following equation: K, K n K b Q, Q p Q b The subscripts i, p, b refers to I, P, and B-picture types. The parameters K„ K p , and K„ are experimentally determined constants for all I, P, and B pictures respectively, and AT* usually is normalised to the value of 1. Figure 1 illustrates a block diagram of a known embodiment of the TM5 rate controller. A group-of-pictures (GOP) is a collection of one I picture, some P pictures and B pictures, and serves as a basic access unit with the I picture as the entry point to facilitate random - 10 access. The controller comprises three levels of processing: a GOP level 111, a picture level 110, and a macroblock (MB) level 109. At die start of every new GOP at the GOP level 111, a GOP bit allocation process 100 computes die total number of bits (R gσp ) allocated for the GOP. The value of R gop is given by: bit rate x N R g°P picture yate 0) where bityate is the target bit rate for encoding the picture sequence, picture yate is the number of pictures coded per second, and N is the total number of pictures coded in the GOP. The remaining bits (R) for the GOP is determined at the picture level 110 by a remaining bits determination process 106. The value of R is updated as: R = R - 5 where S is the number of bits used by previously coded pictures in the GOP, and R = R + R top for a new GOP. With the computed value of R, a picture bit allocation process 101 at the picture level 110 then computes a target bit value (7) allocated to die current picture according to the equations below for corresponding I, P or B picture type: R bit rate T. = max " JVX K x picture yate (2) x.κ„ xκ h R bit rate T_ = max K x picture yate (3) N. 11 T. = max) R bit rate ΗK C K x picture yate N,+ (4) 6 p where r*, T p , T b Represents target bits for the next picture (I,P,B) N b , N p Represents the remaining number of pictures (P,B) in the GOP Xj, X,, X b Represents global complexity measures and gives a measure of the actual number of bits required to represent d e picmre (without compression), and X i = Q i x S i X p = Q p x S p χ„ = x S b Sj, S p , S b are the actual number of bits used to code die previous I/P/B frame. K is a constant (e.g. 8) After computing the target bits for the picture, a reference quantisation step-size (Qj) computation 102 is carried out at die MB level 109 for each macroblock within the picmre. The value of Q j is computed based on the determined target bit allocation ( T t , T p , or T b ) and a buffer fullness value (dp. Each of the three picture types has a virtual buffer associated widi it and these buffers are updated after coding each macro block of the picture type according to the following equations: τ, χ (j- \) 4 - B. M A BO C / *"■ n--t/ (5) 12 - X ♦ Bj ,-ItALR (6) l MB Cm d > - 4 * B, T " x {J~l) (7) 1 ° " MB Cnt where d 0 is the initial virtual buffer fullness, d j is the virtual buffer fullness when coding the/" MB, B is the actual bits consumed up to and including (j-l) Λl MB as provided by a virtual buffer update process 105, and MB_Cnt is the number of MBs in the picture. Equations (5) to (7) effectively track the differences between the acmal number of bits used and the target bits allocated, and tiiese differences are then added to die respective virtual buffers which are used to compute Q,*. This allows the rate controller to control the bits allocation based on the consumption pattern of the picmre. <2y is computed from the buffer fullness via d e equation: Q j = (d j x 31) / r ■= d j x constant (8) where r is a reaction parameter = 2 x bit yate I picture jrate The reaction parameter, as the name implies, is a factor that can control the sensitivity of the algorithm from a change in d j . Large r will cause the reaction to be slower, which may cause the target bits and acmal bits to differ significantly, but it brings about a more gradual rate of change in bit consumption which is favourable for scene changes. On the other hand, a small r causes the controller to be more reactive to changes, giving rise to a closer target and actual bits value, but a faster response also means a less gradual change 13 - during scene change which is not desirable. A graph illustrating the difference between the target bit allocation and the acmal bits allocated within a frame is shown in Figure 2. The value of Q,* is further modified by an activity masking process 103 to give the modulated quantisation step size, Mquant. Basically, the spatial masking ability of the macroblock is used to place a correction factor onto die computed Q s value. Spatial masking ability is the ability to contain noise masked from human visual systems. Typically, an area with complex texture will have a larger correction factor d an an area with simple texture, hence, Mquant = N_act j x Q t (9) where N_αct ; * is d e normalised value of macroblock activity level and may have a value of between 0.5 and 2 Widi the computed Mquant, the macroblock coding process 104 is performed where die macro block is compressed according to die MPEG standards. At the end of coding a picture, the bit count S used for coding the picmre is used to perform a VBV check 107 and to compute die remaining bits R for the current GOP at the remaining bits determination process 106. The VBV (video buffer verifier) is a virtual buffer emulating the status of the decoding buffer. The VBV check (107) is performed to detect any overflow or underflow of the decoding buffer with reference to the target bit rate. When coding a picture sequence with varying scene complexity, a constant bit rate (CBR) encoder is unable to vary the bit-rate according to the scene complexity. This results in scenarios whereby simple scenes are allocated more bits than required while complex scenes are allocated insufficient bits, and therefore results in a variation of ouφut visual 14 quality between scenes of different complexities. An embodiment of a moving picmres encoding apparatus according to the present invention is illustrated in block diagram form in Figure 3. The encoding apparatus as illustrated is arranged in four levels corresponding to processing stages in the bit rate control process, namely a segment level 313, a GOP level 312, a picmre level 311 and a macroblock (MB) level 310. An input moving picmres sequence is encoded in such a way tiiat on die average, or at the end of encoding, the output bit rate is close to a user definable target overall bit rate (overall_BR) 320. The input sequence is divided into segments and groups of picmres (GOP). The purpose of a segment is for monitoring of output bit rate with reference to the overall_BR, and die purpose of the GOP is primarily to facilitate random access. A segment may be defined to include a number of GOP(s) depending on the need in terms of frequency of monitoring output bit rate. Each GOP comprises at least an I-picture and optionally one or more P-pictures and/or B-pictures. Each picmre is divided into macroblocks of pixels for encoding. Hence, a method of rate control according to the current invention may contain processes at the segment level 313, GOP level 312, picmre level 311, and finally macroblock level 310. With a new group of picmres, a GOP bit allocation processor 300 computes the number of bits allocated to the current GOP (R' gop ) as follows: N R = BR x g A (10) s p picture yate where: /V- op is the number of picmres in the GOP, pictureyate is the number of picmres coded per second, and BR is a target bit rate. The target bit rate BR is determined for each picmre or plurality of picmres by a bit rate - 15 adjustment processor 306 which is described hereinbelow. A remaining number of bits (R') for the GOP is determined at die picmre level 311 by a remaining bits determination processor 308, which carries out the steps of: (a) before encoding the first picmre in a GOP, adjusting R' with a new R' !°P~ R' + = R' g°P, and then set 5R„ M = BR (b) otherwise if the picmre to be coded is not die first picmre of a GOP, and a new target bit rate BR is determined, then adjusting R' with the new BR according to: R' += — x (BR - BR A, and then set BR old = BR pictureyate where N is remaining number of picmres in the current GOP (c) and removing number of bits used by die last coded picmre S from the remaining bits value R', hence: R' = R' - *S The computed R' is tiien passed to a picmre bit allocation processor 301 to compute the target bit (7) allocated to a current picmre to be coded. For example, equations (2), (3) and (4) described hereinabove may be used for d at purpose, with the parameters K p , K b , N p , N b and pictjype being supplied. Note that die lower limit of bit_rate/( *picture_rate) is only optional and may be adjusted if necessary. At the macroblock (MB) level 310, a reference quantisation step-size computation processor 302 computes a reference quantisation step-size Q for each MB using the computed target bit allocation T and a bit utilisation parameter B determined by a virtual buffer update processor 305. An example of a method for computing (2, or the/' MB of - 16 - the current picmre is represented by: K { x D x picture yate °- ' 'K κ 2. x BR ~ < U > where K { and K z are constants (eg. 31 and 2 respectively), BR is the determined target bit rate, picturejrate is the number of coded picmres per second, and D j is a virtual buffer fullness The virtual buffer fullness of die 1 MB is determined by: V>o B (12) MB Cnt where D 0 is an initial buffer fullness before coding die current picmre, ie. D ; at end of d e previous picmre, B is the bit utilisation information supplied by the virtual buffer update processor 305, ie. bits used to code die 1 st MB to j-k^T MB, MB_Cru is the total number of MB in the current picmre, and K z is a constant (eg. 1). Three independent current and initial virmal buffers fullness values (Dj, Df, Df, D 0 ', D 0 P , D 0 b ) may be maintained for the three picmre types (I-picmres, P-picmres, and B-picmres). The computed , may also be further scaled by an activity masking processor 303 according to a masking factor determined by die surrounding activity levels of the MB to form the final quantisation step-size, Mquant, for coding d e current MB. A metiiod of masking factor determination can be found in the aforementioned MPEG-2 TM5. Macroblock coding (304) is then performed to encode the current MB. A macroblock coding processor 304 may employ, for example, methods according to the MPEG-1 or MPEG-2 video encoding standards. Such encoding methods include necessary motion 17 compensation, discrete cosine transform, quantisation witii die determined quantisation step-size, and run-length encoding wiu variable length codes. The methods also include necessary decoding processes such diat motion compensation can be performed. The number of bits utilised by die macroblock coding processor 304 to code each MB is passed to die virmal buffer update processor 305. In applications where a maximum and/or minimum bit rate must be maintained for encoding, a VBV checking processor 307 is utilised. User definable maximum and/or minimum bit rates (max/min BR) are input at 321 to the VBV checker 307, a method of VBV checking according to the MPEG standards is used to examine bit utilisation information from d e virtual buffer 305, and necessary corrections are made to ensure compliance and diat the output bit rate is within die defined max/min BR. At the end of coding a picmre or a plurality of picmres, bit rate adjustment is applied by die bit rate adjustment processor 306 to provide any necessary correction to the current target bit rate BR. The correction is based on a target encoding quality provided by a target segment quality adjustment processor 309 and a resultant encoding quality of previously coded picmre(s) such that die overall encoding qualities of picmres witiiin a segment are relatively close to the target constant. The resultant encoding quality may be determined by die average value of the reference quantisation step size (ie. average Qp of the previously coded picture(s). A method of target bit rate (BR) determination can be derived from a rate-quantisation model as given by: K. x (average O, - target O) BR = current JR + -2 — 6 -^ ~ ~ (13) average O where current_BR is a current estimated bit rate, or current _BR = S, + N p x S p + N b x S b , S„ S p , S b are bits used by previously coded I, P, and B-picmres respectively, N p , N b are total number of P and B-pictures in the current GOP, average_Q, is the average value of Q : of previously coded picture(s) 18 target_Q j is the target encoding quality or target value of Q j , AT, may be a constant, or a factor of previous BR, maxJBR, or current_BR, and K 4 may also be separately determined for I, P, and B-picture types. Maximum and/or minimum bit rates can be applied to the determined BR according to application requirements. The max/min BR input at 322 are used as given by: if (BR > maxJBR), then BR = max_BR if (BR < min_BR), then BR = min_BR. It is also possible to make use of the target_Q t at die macro-block level to increase efficiency of encoding. Basically, the target _Q } is used by the reference quantisation step- size computation processor 302 as a lower limit of the ouφut reference quantisation step- size Q j such diat when tiiis target (quality) is reached at the MB level, bits are saved for fumre encoding immediately. In tiiis case, die ouφut reference quantisation step size may be set according to: if (Q j < target _Qp, then Q } = target _Q j . The target bit rate BR is adjusted continuously with reference to a fixed target encoding quality within each segment. In mm at the beginning of each segment, the target encoding quality is adjusted so diat a target overall bit rate is achieved. A segment may contain a few groups of picmres depending on die need in terms of frequency of monitoring ouφut bit rate. A new segment may also be defined by a scene change detectable by any conventional scene change detector wit a given range of time. At the onset of a new segment, the target segment quality adjustment processor 309 executes a check on the overall bit-rate based on bits usage and makes any necessary adjustments to the target encoding quality for the bit rate adjustment processor 306 so that the overall bit-rate converges to a user definable target overall bit-rate 320 (overall _BR). 19 An example embodiment of a constant overall bit rate controller implementing the target segment quality adjustment process according to the present invention is illustrated in block diagram form in Figure 4. The bit count difference between the acmal and target bit counts (bits_diff) for coding of a previous segment is calculated by a bits difference computation processor 401, utilising the steps of: a) obtaining the acmal bits (bits_segment) used for coding the previous segment according to the input bit utilisation of picmres (S) 400 in the previous segment, and b) computing the value of bits_diffbaseά on a corresponding target segment bit rate 406 (segment _BR) for the previous segment, according to: segment BR x N„„----,, bits diff = — = 2E=Ϊ - bits egment (14) ~ pictureyate where N segmem is the number of coded picmres in the previous segment. The bit count difference (bits_diff) is then redistributed by a bit difference distribution processor 402 over the next k number of segments using a bit distribution function, fim , expressed as: delta_bits m = βjn) x bits Jiff (15) where m = \,...,k, ∑ k Λ«) = i, delta_bits m is the bits difference distributed to next πf 1 segment. For an example case of k= and m) = \lk, Figure 5 illustrates how the bit count difference (bits_diff) of an encoded segment is distributed over the next 4 segments. The period of A: segments used for bits compensation is referred to as an adjustment interval. 20 A delta bit-rate computation processor 403 (Figure 4) accumulates all bit differences (delta_ bits) distributed from previously encoded segments to the current segment to be coded, and computes a delta segment bit-rate ( segment_BR) based on de accumulated delta_bits and die number of picmres in the current segment to be coded. The delta segment bit-rate and a user definable target overall bit-rate (overall_BR) input at 408 are used by the target segment BR adjustment processor 405 to derive die target segment bit rate (segment_BR) 406 for the current segment. The value of segment_BR can be expressed as: segment _BR - overall _BR + ^segment _BR (16) A new target encoding quality (target Ap 407 for encoding of die current segment is then determined from die target segment bit rate using a BR-Q* modelling processor 404. The BR-Q j modelling processor may operate, for example, according to: ΔO = target Q . x Se ^ ent - BR ' ^gment_BR') ~ J J K 5 x segment J3R' i /) target _Q S = target _Q } + ΔO (18) where target _Q '. is the target Q } of the previous segment, segment _BR' is the target segment bit rate of the previous segment, and K 5 is a constant which can be experimentally determined. A maximum limit may be placed on the determined ΔQ y * so that tiiere is no drastic change in ouφut quality from one segment to another. A flow chart for a process of variable bit rate (VBR) encoding with constant overall bit- 21 - rate control according to an embodiment of the invention is illustrated in Figure 6. Initialisation of predetermined parameters is first carried out at step 600. Then, at step 601 d e process determines whether processing is at die start of a new group-of-pictures (GOP) or not. Where a new GOP is determined, step 602 is carried out by updating the 5 bit allocation for the new GOP. This may involve, for example, computation of the GOP bit allocation R' gop according to Equation (10) described hereinabove and accumulating to the remaining bit value R' to give the R' for die new GOP. For each picmre in the current GOP, a target bit allocation value T is determined at step 10 603, and this may be computed according to Equations (2), (3) and (4), for example. Then, for each macroblock in the picmre, a reference quantisation step size Q is determined using the computed target bit allocation T, such as by the process represented by Equations (11) and (12). Activity masking may also be included in this step. The macroblock of the picmre is then encoded at step 605 using the computed quantisation step 15 size, and steps 604 and 605 are repeated for all of d e macroblocks in the picmre until the end of d e picmre is determined at step 606. When d e end of die sequence of moving picmres which is being encoded is reached, tiiis is determined at step 607, which terminates the process at step 611 upon that occurrence. 20 Otherwise, the process continues to step 608 where it is determined whether d e end of die current segment of picmres has been reached. When die end of a segment of picmres is reached and processing of a new segment about to begin, die target encoding quality is adjusted at step 609. This is performed by firstly 25 computing the difference between the number of bits allocated for coding the previous segment and the acmal number of bits used in coding mat segment, such as by die process represented by Equation (14). This quantity, bitsjiiff, represents extra bits which are left over from the previous segment of picmres, and may be distributed for use in encoding the ensuing picmre segments. The left over bits bitsjiiff ' are distributed for use over one or 30 more segments according to a bit distribution function, an example of which is described - 22 in connection with Equation (15). In essence, die left over bits from the previous segment are divided into a plurality of k, preferably equal, amounts and allocated to die following k segments for encoding with. The dividends allocated from all of the k previously processed segments are accumulated and used to determine a change in segment bit rate, Δsegment_BR, according to die accumulated bits difference and the number of picmres in die segment to be coded. This can dien be used to compute an allocated bit rate for die segment to be coded according to Equation (16). Finally, a target encoding quality, target _Q j , can be computed according to Equations (17) and (18), for example. If the end of a segment has not been reached at step 608, or after the target quality adjustment of step 609, the target quality and the acmal quality of encoded picmres is compared, and a new bit rate for encoding the next picmre is computed based on the difference. This step can be carried out as described in relation to Equation (13), for example. The procedure tiien returns to step 601, described above. In summary, embodiments of die present invention provide metiiods and apparatus for encoding moving picmres witii a variable bit rate whilst maintaining a consistent ouφut encoding quality according to a determined target quality. The target quality is adjusted at die beginning of every moving picmres segment according to a bit distribution function to ensure an overall bit rate that meets a defined target. Furthermore, the segment based bit distribution function provides smooth and flexible modulation of the encoding quality. The method can be implemented at low cost and witii the ability to encode an input moving picmres sequence in one-pass at real-time. It will be readily apparent to those of ordinary skill in the art that the foregoing detailed description of the present invention has been presented by way of example only, and is not intended to be considered limiting to the invention as defined in the claims appended hereto. In particular, it is envisaged that numerous variations to the embodiments as described can be made without departing from the spirit and scope of the invention. WO 99/49664 pcτ/SG98/ooo22 - 23 Throughout this specification and the claims which follow, unless die context requires odierwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not die exclusion of any other integer or step or group of integers or steps. - 24 Glossary: VBV video buffer verifier VBR variable bit rate CBR constant bit rate MPEG Moving Picmre Experts Group DCT discrete cosine transform GOP group of picmres MB macroblock Q quantisation parameter (picmre quality) quantisation parameter for macroblock y a , . Q P quantisation parameters for I, B and P type picmres number of bits allocated for encoding a group of picmres bit ate target bit rate for a picmre sequence picture ate number of picmres coded per second N total number of picmres in the group of picmres N b , N p remaining number of P,B picmres in the group of picmres R, R' remaining number of bits for encoding a group of picmres S number of bits already used in coding a group of picmres T„ T p , T b target number of bits for coding next I, P, B type picmre Xj, X p , X b global complexity measures for I, P, B type picmre d j , D j virtual buffer fullness when coding the/" macroblock d o , D 0 initial virtual buffer fullness i acmal bits used up to and including (j-l)" 1 macroblock MB Znt number of macroblocks in the picmre r reaction parameter ( = 2 x bitjate I picture jate) N ct j normalised macroblock activity level Mquant modulated quantisation step size ( = Njict j x Qp overall _BR definable target overall bit rate BR target bit rate 25 B bit utilisation information (eg. bits used to code previous MBs in picmre) current _BR current estimated bit ra average J2 j average value of quantisation parameter for previously coded picmres target_Q j quantisation parameter value for target encoding quality max_BR maximum permitted bit rate minJBR minimum permitted bit rate bitsjiiff difference between acmal and target bits used in coding previous segment segment j3R target segment bit rate bits egment acmal number of bits used in coding previous segment N 's.egment number of coded picmres in previous segment delta_bits m number of difference bits available to next nf segment fim) bits distribution function segment_BR available difference in segment bit rate

Claims: 26 Claims: 1. A method for use in a moving picmres encoder for encoding a sequence of segments each having at least one image, comprising the steps of: a) determining an overall target bit rate for encoding the sequence of images; b) determining a bit allocation and target quantisation step size for encoding a first segment on the basis of a segment target bit rate calculated using said overall target bit rate; c) encoding said first segment using a variable bit rate encoding method according to the target quantisation step size; d) determining a difference between the number of bits used to encode said first segment and said first segment bit allocation; e) distributing said difference for use in encoding at least one subsequent segment to determine a subsequent segment bit allocation; f) determining a new target quantisation step size for encoding a said subsequent segment on the basis of a new target segment bit rate calculated using said segment target bit rate and the distributed difference; and g) encoding said subsequent segment using a variable bit rate encoding metiiod according to the new target quantisation step size; wherein variable bit rate encoding is employed for encoding picmres within a segment whilst maintaining a substantially constant bit rate over said sequence. 2. A method as claimed in claim 1, wherein the steps d) to g) are carried out iteratively using the number of bits used to encode die subsequent segment and the subsequent segment bit allocation for determining said difference. 3. A method as claimed in claim 1 or 2, wherein each segment comprises at least one group of picmres having an I-picmre and optionally at least one P and/or B-picmre. 4. A method as claimed in claim 2 or 3, wherein the difference between the number W yy O j 9 y 9 y / / 4 < *9v6o6o44 PCT/SG98/00022 27 of bits used to encode the first segment and said first segment bit allocation is determined according to: overall BR x N „-,„„, bitsjiiff = = Z2Σ2L - bits egment picture ate where: bitsjiiff is said difference, overall _BR is the overall target bit rate, N segment •**■> me number of coded images in the first segment, and bits egment is die acmal number of bits used to encode the first segment. 5. A mediod as claimed in claim 4, wherein said difference is distributed according to: delta _bits m = f(m) x bitsjiiff where delta_bits m is the bits difference distributed to next m* segment in the sequence, m = l,...,k, k is a positive integer, and fi n) is a bit distribution function, where ∑ k _fl = !• 6. A method as claimed in claim 5, where f(m) — \lk. 7. A mediod as claimed in claim 5 or 6, wherein the steps d) to g) are carried out iteratively using the number of bits used to encode die subsequent segment and the subsequent segment bit allocation for determining said difference. 8. A method as claimed in claim 7, wherein the new target quantisation step size is calculated according to: ΔO = targets x &»"" ~ s*W«JR) K x segment _BR - 28 target J) - target _Q + ΔO where target _Q is the new target quantisation step size, segment JBR is the new target bit rate, target _Q' is the target quantisation step size of the preceding segment, segment _BR' is the target segment bit rate of die preceding segment, and AT is a constant. 9. A method as claimed in claim 7 or 8, wherein for each subsequent segment d e difference between the number of bits used to encode d e segment and d e segment bit allocation is determined according to: segment BR x N, m ,.„ bitsjiiff = — ZZOS. - bitsjegment picture jate where: bitsjiiff is said difference, segment _BR is the segment target bit rate, ^ segm t ■■•■-- the number of coded images in the segment, and bitsjegment is the acmal number of bits used to encode die segment. 10. A method as claimed in claim 9, wherein said difference is distributed according to: delta _bits m = f(m) x bitsjiiff where delta _bits m is the bits difference distributed to next rri h segment in the sequence. m = 1 k, A: is a positive integer, and fXm) is a bit distribution function, where ∑ f m) = 1. 11. A method as claimed in claim 10, where f(m) = \lk. 29 - 12. A method for encoding moving picmres in a moving picmres encoder wherein a sequence of images are provided as input, the sequence of images comprising a plurality of segments each having a plurality of images, the method including: a) determining an overall target bit rate for encoding die sequence of images; b) maintaining a distribution record of bits from at least one previously encoded segment allocated for use in encoding at least one segment to be encoded; c) determining a target segment bit rate for a segment of die sequence of images on the basis of the overall target bit rate and a bit rate change calculated from die corresponding allocated bits from the distribution record; d) determining a target segment encoding quality from the target segment bit rate, a preceding target segment bit rate and a preceding target segment encoding quality,; and e) encoding the images of the segment according to die target segment encoding quality using a variable bit rate encoding technique taking into account scene complexities of die images in the segment; wherein maintaiiiing said distribution record includes determining a difference between the number of bits used to encode a particular segment and the number of bits allocated for encoding die particular segment on the basis of the target segment encoding quality. 13. A method as claimed in claim 12, wherein steps b) to e) are carried out iteratively for said sequence of segments. 14. A method as claimed in claim 13, wherein each segment comprises at least one group of picmres having an I-picmre and optionally at least one P and/or B-picmre. 15. A method as claimed in claim 12, 13 or 14, wherein the target segment encoding quality is calculated according to: ΔO = t arget _Q x i^Sm^J^ ~ segment R') ~ K x segment _BR! 30 target J ) = target JJ + ΔO, where target _Q is the target segment encoding quality, segment_BR is the target segment bit rate, target Q' is the preceding target segment encoding quality, segment _BR' is the preceding target segment bit rate, and AT is a constant. 16. A method as claimed in claim 15, wherein the difference between the number of bits used to encode a particular segment and the number of bits allocated for encoding d e particular segment is determined according to: segment BR x N„ amιmt bits j iiff = — = ^SSSL - bitsjegment picture jate where: bitsjiiff is said difference, segment JBR is the segment target bit rate, S gme t ** s the number of coded images in the segment, and bitsjegment is the acmal number of bits used to encode die segment. 17. A mediod as claimed in claim 16, wherein the allocation of bits in the distribution record is calculated according to: delta Jits m = j\m) x bitsjiiff where delta Jits m is the bits difference allocated to next m' Λ segment in the sequence, m = l ,... ,k, k is a positive integer, and f(m) is a bit distribution function, where ∑*f m.) = 1. 18. A method as claimed in claim 17, where f(m) = \lk. 31 - 19. A method for controlling bit allocation in a moving picmres encoder for encoding a sequence of images comprising a plurality of segments each having a plurality of images, the method including, for each segment: determining a difference between a number of bits used for encoding a previous segment and a number of bits allocated for encoding the previous segment; calculating a bits distribution from the determined bits difference and a predetermined distribution function; calculating a bit rate change from the bits distribution and a predetermined number of images in the segment; calculating a target segment bit rate from the bit rate change and a predetermined target overall bit rate for the sequence of images; and determining a target segment encoding quality from the target segment bit rate. 20. A method as claimed in claim 19, wherein die difference between the number of bits used for encoding die previous segment and the number of bits allocated for encoding the previous segment is determined according to: segment BR x N, m , bits Jiff = — = iSSSL - bitsjegment picture jate where: bits Jiff is said difference, segment_BR is the previous segment target bit rate, l men , ** s the number of coded images in die previous segment, and bitsjegment is the acmal number of bits used to encode the previous segment. 21. A method as claimed in claim 20, wherein the bits distribution is calculated according to: delta Jits m = f(m) x bits Jiff where delta Jits m is the number of bits difference allocated to next m' h segment in the sequence, - 32 m — 1 k, k is a positive integer, and f(m) is a bit distribution function, where Σ v yT ) = 1. 5 22. A mediod as claimed in claim 21, where jn) = \lk. 23. A method as claimed in any one of claims 19 to 22, wherein the target segment encoding quality is calculated according to: ΔO = target^ x ^ se ^ ment ~ BR ~ *W∞*JβK ~ K x segment JR' target J) = target JJ + ΔO where target _Q is the target segment encoding quality, 0 segment JBR is the target segment bit rate, target J∑ is the previous segment target segment encoding quality, segment JR' is the previous segment target segment bit rate, and AT is a constant. 15 24. An encoding quality adjustment processor for generating a target segment encoding quality value in a moving picmres encoder for encoding a series of segments each having at least one image using a variable bit rate encoding scheme whilst maintaining a substantially constant overall bit rate, comprising: a bits difference computation means coupled to receive a segment encoding bit 20 utilisation value and a target segment bit rate and generate therefrom a bits difference value representing a difference in bits allocated and bits used for encoding a segment; a bits distribution means coupled to the bits difference computation means for computing at least one bits distribution value from e bits difference value and a predetermined distribution function; 33 a bit rate difference computation means coupled to the bits distribution means for computing a segment bit rate difference from the at least one bits distribution value and a predetermined number of images in a segment; a target segment bit rate adjustment means coupled to die bit rate difference computation means and die bits difference computation means for computing said target segment bit rate from the segment bit rate difference and a predetermined target overall bit rate for the sequence of segments; and an encoding quality computation means coupled to die target segment bit rate adjustment means for computing a target segment encoding quality value from said target segment bit rate. 25. A moving picmres encoder comprising: a coding processor for encoding picmre data based on macroblocks according to a quantisation step size; a virmal buffer processor coupled to die coding processor for tracking a number of bits used for encoding successive macroblocks in a picmre and a number of bits used for encoding successive picmres in a group of picmres; a quantisation step size processor coupled to die coding processor for determining said quantisation step size from a target number bits allocated for a picmre and d e number of bits already used for encoding macroblocks in diat picmre; a picmre bit allocation processor coupled to die quantisation step size processor for determining said target number of bits allocated for a picmre from a target bit rate and die number of bits already used for encoding picmres in a current group of picmres; a bit rate adjustment processor coupled to die picmre bit allocation processor, the virmal buffer processor and the quantisation step size processor for determining said target bit rate from the number of bits already used for encoding successive picmres in die current group of picmres, a target encoding quantisation step size and an average quantisation step size for picmres in the current group of picmres; and a target encoding quantisation step size processor coupled to the bit rate adjustment processor and the virmal buffer processor for determining said target encoding - 34 - quantisation step size from a predetermined target overall bit rate and the number of bits used for encoding a preceding group of picmres.

Inventors: HUI YAU WAI LUCAS, OH HONG LYE, HUI, YAU, WAI, LUCAS, OH, HONG, LYE
Applicants: HUI YAU WAI LUCAS, OH HONG LYE, SGS THOMSON MICROELECTRONICS A, HUI, YAU, WAI, LUCAS, OH, HONG, LYE, SGS-THOMSON MICROELECTRONICS ASIA PACIFIC (PTE) LTD.
Application year: 1998
Main classification: H04N7
IPC codes: G06T9, H04N7