# Basic Unit Layer Rate Control Algorithm for H.264 Based on Human Visual System

Authors: Xiao Chen; Haiying Liu
Journal: Mathematical Problems in Engineering
 (2013)

---

## Abstract

In the process of the video coding, special attention should be paid to the subjective quality of the image. In the JVT-G012 algorithm for H.264, the influence of the human visual characteristic in basic unit layer rate control was not taken into account. This paper takes the influence of the human visual characteristic into the full consideration and offers ways to improve the subjective quality of the image. The visual characteristic factor, which is constituted by the motion feature and edge feature, is used to reasonably allocate the target bits, and then its quantization parameter is adjusted by encoded frame information. The experimental results show that, in comparison to the original algorithm, the proposed algorithm can not only control the bit rate more accurately but also make the peak signal to noise ratio (PSNR) stable, so as to improve the stationarity of the video image. The subjective quality of the reconstructed video is more satisfying.


---

## Body

## 1. Introduction

With the rapid development and popularization of the internet, electronic devices gradually become an indispensable part of our daily life, such as online broadcasting, online advertising, e-commerce, VOD, distance education, telemedicine, real-time video conference, smart phones, 3D video [1], VCD, DVD, HDTV, and streaming of multimedia video [2]. However, the real-time data transmission and storage of multimedia data have become more difficult due to the limited communication bandwidth, especially in the video communication. Its high capacity of data has much difficulty in the process of transmission and storage. Thus with the limits of the bandwidth and the low storage capacity, the video coding aiming at using the least bits to represent image is very important.The rate control plays an important role in the process of video coding. Since 70% of the information is obtained from eyes and the eyes is the final receiver of video information, it is vital important to take full advantage of the human visual characteristics to get a higher subjective quality of images.According to the special structure of the human eyes, some scholars put forward some relevant algorithms [3–10]. An adaptive bit allocation method is presented in [3], based on the space and time perception functions. The work in [4, 5] presents a method based on the region of interest in the human eyes. A novel rate control algorithm is presented in [6], based on the visual perception characteristics. The work in [7] proposes a new digital video watermark method based on the human visual system (HVS). The work in [8] proposes a method based on the region of interest, aiming at distribution of the target bits. The work in [9] presents a video quality evaluation method based on the region of interest. The work in [10] presents an algorithm to distribute the target bits of the basic unit layer, which analyze the motion information and texture features.Although the JVT-G012 is by now the most acceptable rate control algorithm, it still has shortcomings. The work in [11] proposes a joint rate-distortion optimization for the H.264 rate control algorithm with a novel distortion prediction equation, which avoids linear regression employed in other distortion predictors and can considerably speed up rate estimation. Multiple quantization parameters determination algorithm based on the statistics of the deviation measure is proposed in [12], which can achieve accurately QP. The work in [13] proposes a rate control technique for H.264/AVC using subjective quality of video. The work in [14] presents a complexity coefficient to combine the target bits. This paper presents a reformative basic unit layer rate control algorithm based on the HVS. The HVS was not taken into account in the JVT-G012 algorithm. Since eyes are the final receiver of the video information, so it is vital important to take the HVS into account in the video coding process. The HVS is very sensitive to the brim part and motion part. However each pixel of JVT-G012 algorithm is treated equally in the basic unit layer. Though the work in [10] takes advantage of the HVS, it is not comprehensive. In this paper the visual characteristic factor is used to improve the rate control in the basic unit layer.


## 2. JVT-G012 Algorithm

The JVT-G012 basic unit layer rate control mainly consists of three steps. First, it predicts the MAD of the current basic unit in the current frame. Then, it computes the target bit for the current basic unit. Last, it calculates the quantization parameter of the current basic unit and performs RDO.


### 2.1. Predict the MAD of the Current Basic Unit in the Current Frame

Consider(1)MADcb=a1*MADpb+a2,
where MADcb is the predicted MAD of the current basic unit in the current frame. MADpb is the actual MAD of the co-located basic unit in the previous frame. a1 and a2 are two coefficients of the predictive model, whose initial values are 1 and 0, respectively; after finishing encoding every basic unit, the coefficients a1 and a2 are updated.


### 2.2. Compute the Target Bit for the Current Basic Unit

The target bit for the current basic unitb~l is
(2)b~l=Tr×MADcb2∑k=1NunitMADcb2(k),
where Tr and Nunit are the number of remaining bits for the all uncoded basic units in the current frame and the number of uncoded basic units, respectively. MADcb is the predicted MAD of the current basic unit in (1).


### 2.3. Compute the Quantization Parameter of the Current Basic Unit and Perform RDO

Consider(3)b~l=MADcb*(X1Q+X2Q2),
where Q is the quantization parameter of the current basic unit. b~l is the target bit of the current basic unit in (2). MADcb is the predicted MAD of current basic unit in (1). X1 and X2 are the first-order and the second-order model parameters of the quadratic rate-distortion model, respectively. They are updated in the process of encoding.


## 2.1. Predict the MAD of the Current Basic Unit in the Current Frame

Consider(1)MADcb=a1*MADpb+a2,
where MADcb is the predicted MAD of the current basic unit in the current frame. MADpb is the actual MAD of the co-located basic unit in the previous frame. a1 and a2 are two coefficients of the predictive model, whose initial values are 1 and 0, respectively; after finishing encoding every basic unit, the coefficients a1 and a2 are updated.


## 2.2. Compute the Target Bit for the Current Basic Unit

The target bit for the current basic unitb~l is
(2)b~l=Tr×MADcb2∑k=1NunitMADcb2(k),
where Tr and Nunit are the number of remaining bits for the all uncoded basic units in the current frame and the number of uncoded basic units, respectively. MADcb is the predicted MAD of the current basic unit in (1).


## 2.3. Compute the Quantization Parameter of the Current Basic Unit and Perform RDO

Consider(3)b~l=MADcb*(X1Q+X2Q2),
where Q is the quantization parameter of the current basic unit. b~l is the target bit of the current basic unit in (2). MADcb is the predicted MAD of current basic unit in (1). X1 and X2 are the first-order and the second-order model parameters of the quadratic rate-distortion model, respectively. They are updated in the process of encoding.


## 3. Improved Rate Control Algorithm

The reformative basic unit layer rate control algorithm mainly consists of two steps. First, it allocates the target bits based on the HVS. Then, it adjusts the quantization parameter and performs RDO.


### 3.1. Compute the Target Bit of the Current Basic Unit Based on the HVS

This paper takes the HVS perception mechanism into account because the human eyes are extremely sensitive to the edge part and the motion part of images. So the proposed algorithm assigns fewer bits to the unimportant region and assigns more bits to the region of interest in the process of allocating bits. It can achieve the goal of improving the overall video quality. This paper adjusts (2) with the visual characteristic factor V(i,j):
(4)b~l=Tr×V(i,j)×MADcb2∑k=1NunitMADcb2(k),V(i,j)=motion(i,j)+edge(i,j),
where the motion vision characteristics and the edge vision characteristics are denoted by motion(i,j) and edge(i,j), respectively.


#### 3.1.1. Motion Characteristics

In the real scene, there exist two major motion scenes as following.The whole scene changes a little while only parts of the objects move or change. At this time, human eyes are concerned much more about the moving and changing objects. That is, whenMVavg is less than 4.5,
(5)motion(i,j)={1MV<12-(6.5-MV)82<MV≤6.53MV>6.5.When the whole scene moves fast,MVavg is more than 4.5, there are two subordinate situations in this case.When there are many fast moving macroblocks, human eyes pay more attention to those objects that move little. This time MV is more than 2.5;motion(i,j) can be expressed as
(6)motion(i,j)={1+(6.5-MV)102.5<MV≤6.53MV>6.5.When most of the objects move inconspicuously while only some of them move fast in the scene, human eyes pay more attention to the fast-moving part. This time, MV is less than 2.5;motion(i,j) can be expressed as
(7)motion(i,j)={1MV≤1.52-(3.5-MV)101.5<MV<2.5,
where the magnitude of motion vector for the jth basic unit in the ith frame and the magnitude of average motion vector in the remaining basic units of current frame are denoted by MV(i,j) and MVavg, respectively. One has MV=MVX2+MVY2, where MVX represents the magnitude of the macroblock motion vector in horizontal direction and MVY represents the magnitude of the macroblock motion vector in vertical direction.


#### 3.1.2. Edge Characteristics

This paper describes the edge characteristics of images with the variance because there is high variance in the edge area of images:(8)edge(i,j)={1-D(i,j)max(D)D(i,j)<medium(D)D(i,j)max(D)other,
where D(i,j) represents the variance for the jth basic unit in the ith frame. Medium(D) and max(D) represent the midvalue and the maximum value of the variance, respectively.


### 3.2. Adjust the Quantization Parameter and Perform RDO

To consider the feedback information of the encoded frames, this paper adapts the quantization parameter adjustment coefficient. This paper usesη to adjust the quantization parameter, which is defined as the ratio of texture bits to the header bits:
(9)QPi(j)={QPi′(j)-11.0<η≤2.0QPi′(j)+10.6<η≤1.0QPi′(j)+2η<0.6Qcother,
where QPi′(j) is the average value of quantization parameters for all basic units in the previous frame. Qcis the quantitative parameter in the JVT-G012 algorithm.After the adjustment, the algorithm took into account the encoded frame information. The proposed algorithm achieves a good rate control. The algorithm performs RDO and updates the model parameters.


## 3.1. Compute the Target Bit of the Current Basic Unit Based on the HVS

This paper takes the HVS perception mechanism into account because the human eyes are extremely sensitive to the edge part and the motion part of images. So the proposed algorithm assigns fewer bits to the unimportant region and assigns more bits to the region of interest in the process of allocating bits. It can achieve the goal of improving the overall video quality. This paper adjusts (2) with the visual characteristic factor V(i,j):
(4)b~l=Tr×V(i,j)×MADcb2∑k=1NunitMADcb2(k),V(i,j)=motion(i,j)+edge(i,j),
where the motion vision characteristics and the edge vision characteristics are denoted by motion(i,j) and edge(i,j), respectively.


### 3.1.1. Motion Characteristics

In the real scene, there exist two major motion scenes as following.The whole scene changes a little while only parts of the objects move or change. At this time, human eyes are concerned much more about the moving and changing objects. That is, whenMVavg is less than 4.5,
(5)motion(i,j)={1MV<12-(6.5-MV)82<MV≤6.53MV>6.5.When the whole scene moves fast,MVavg is more than 4.5, there are two subordinate situations in this case.When there are many fast moving macroblocks, human eyes pay more attention to those objects that move little. This time MV is more than 2.5;motion(i,j) can be expressed as
(6)motion(i,j)={1+(6.5-MV)102.5<MV≤6.53MV>6.5.When most of the objects move inconspicuously while only some of them move fast in the scene, human eyes pay more attention to the fast-moving part. This time, MV is less than 2.5;motion(i,j) can be expressed as
(7)motion(i,j)={1MV≤1.52-(3.5-MV)101.5<MV<2.5,
where the magnitude of motion vector for the jth basic unit in the ith frame and the magnitude of average motion vector in the remaining basic units of current frame are denoted by MV(i,j) and MVavg, respectively. One has MV=MVX2+MVY2, where MVX represents the magnitude of the macroblock motion vector in horizontal direction and MVY represents the magnitude of the macroblock motion vector in vertical direction.


### 3.1.2. Edge Characteristics

This paper describes the edge characteristics of images with the variance because there is high variance in the edge area of images:(8)edge(i,j)={1-D(i,j)max(D)D(i,j)<medium(D)D(i,j)max(D)other,
where D(i,j) represents the variance for the jth basic unit in the ith frame. Medium(D) and max(D) represent the midvalue and the maximum value of the variance, respectively.


## 3.1.1. Motion Characteristics

In the real scene, there exist two major motion scenes as following.The whole scene changes a little while only parts of the objects move or change. At this time, human eyes are concerned much more about the moving and changing objects. That is, whenMVavg is less than 4.5,
(5)motion(i,j)={1MV<12-(6.5-MV)82<MV≤6.53MV>6.5.When the whole scene moves fast,MVavg is more than 4.5, there are two subordinate situations in this case.When there are many fast moving macroblocks, human eyes pay more attention to those objects that move little. This time MV is more than 2.5;motion(i,j) can be expressed as
(6)motion(i,j)={1+(6.5-MV)102.5<MV≤6.53MV>6.5.When most of the objects move inconspicuously while only some of them move fast in the scene, human eyes pay more attention to the fast-moving part. This time, MV is less than 2.5;motion(i,j) can be expressed as
(7)motion(i,j)={1MV≤1.52-(3.5-MV)101.5<MV<2.5,
where the magnitude of motion vector for the jth basic unit in the ith frame and the magnitude of average motion vector in the remaining basic units of current frame are denoted by MV(i,j) and MVavg, respectively. One has MV=MVX2+MVY2, where MVX represents the magnitude of the macroblock motion vector in horizontal direction and MVY represents the magnitude of the macroblock motion vector in vertical direction.


## 3.1.2. Edge Characteristics

This paper describes the edge characteristics of images with the variance because there is high variance in the edge area of images:(8)edge(i,j)={1-D(i,j)max(D)D(i,j)<medium(D)D(i,j)max(D)other,
where D(i,j) represents the variance for the jth basic unit in the ith frame. Medium(D) and max(D) represent the midvalue and the maximum value of the variance, respectively.


## 3.2. Adjust the Quantization Parameter and Perform RDO

To consider the feedback information of the encoded frames, this paper adapts the quantization parameter adjustment coefficient. This paper usesη to adjust the quantization parameter, which is defined as the ratio of texture bits to the header bits:
(9)QPi(j)={QPi′(j)-11.0<η≤2.0QPi′(j)+10.6<η≤1.0QPi′(j)+2η<0.6Qcother,
where QPi′(j) is the average value of quantization parameters for all basic units in the previous frame. Qcis the quantitative parameter in the JVT-G012 algorithm.After the adjustment, the algorithm took into account the encoded frame information. The proposed algorithm achieves a good rate control. The algorithm performs RDO and updates the model parameters.


## 4. Simulation Results and Discussion

In order to validate the effectiveness of our algorithm in this paper, this paper has implemented the proposed rate control algorithm by the JM10.1 test model software. Also the proposed algorithm is compared with the JVT-G012 algorithm and the Zheng algorithm in [10]. The tested sequences are in QCIF4:2:0 formats: highway, mother-daughter, foreman, claire, hall, silent, akiyo, news, and carphone. In the experiments, all sequences are coded as the IPPP structure, the frame rate is set to 30 frames per second, the total number of frames is set to 200, the length of GOP is set to 40, and the target rate is 32 kbps.Tables1 and 2 show the comparison of the bit rate and the PSNR. As summarized in Tables 1 and 2, the proposed algorithm can control more accurately the bit rates than the JVT-G012 algorithm and obtain much better PSNR for the different video sequences. In particular for the test sequence highway, the proposed algorithm achieves a PSNR gain of 0.74 dB.Table 1
Bit rate comparison for the proposed algorithm and the JVT-G012 algorithm.
Test sequences
Output bit rate (kbps)
JVT-G012
The proposed algorithm
Highway
32.41
32.06
Mother-daughter
32.10
32.04
Foreman
32.10
32.06
Claire
32.15
32.06
Hall
32.14
32.07
Silent
32.09
32.06
Akiyo
32.08
32.04
News
32.04
32.04
Carphone
32.13
32.06Table 2
PSNR comparison for the proposed algorithm and the JVT-G012 algorithm.
Test sequences
PSNR (dB)
JVT-G012
The proposed algorithm
Gain
Highway
34.60
35.34
0.74
Mother-daughter
34.79
34.90
0.11
Foreman
28.38
28.49
0.11
Claire
39.36
39.56
0.20
Hall
34.05
34.26
0.21
Silent
31.03
31.16
0.13
Akiyo
37.93
38.16
0.23
News
31.29
31.50
0.21
Carphone
31.56
31.70
0.14Tables3 and 4 show the comparison of the bit rate and the PSNR. The proposed algorithm gets much better PSNR than the Zheng algorithm for different video sequences and also controls accurately the bit rate. The proposed algorithm can improve the average PSNR for all test sequences, while the Zheng algorithm can improve PSNR for a part of test sequences. For example, Zheng algorithm gets less PSNR than the JVT-G012 algorithm for test sequences: mother-daughter, foreman, and hall.Table 3
Bit rate comparison for the proposed algorithm and the Zheng algorithm.
Test sequences
Output bit rate (kbps)
Zheng algorithm
The proposed algorithm
Highway
31.94
32.06
Mother-daughter
31.95
32.04
Foreman
31.93
32.06
Claire
32.03
32.06
Hall
32.04
32.07
Silent
32.06
32.06
Akiyo
32.01
32.04
News
32.04
32.04
Carphone
32.04
32.06Table 4
PSNR comparison for the proposed algorithm and the Zheng algorithm.
Test sequences
PSNR (dB)
Zheng algorithm
The proposed algorithm
Gain
Highway
35.23
35.34
0.11
Mother-daughter
34.75
34.90
0.15
Foreman
28.35
28.49
0.14
Claire
39.49
39.56
0.07
Hall
34.00
34.26
0.26
Silent
31.07
31.16
0.09
Akiyo
38.04
38.16
0.12
News
31.40
31.50
0.10
Carphone
31.59
31.70
0.11Figures1 and 2 show that the PSNR curve is flatter than the one obtained from the JVT-G012 algorithm. The proposed algorithm suppresses the sharp drop of the PSNR and improves the stability of the picture quality.Figure 1
PSNR curves of the highway sequence comparison for the JVT-G012 and the proposed algorithms.Figure 2
PSNR curves of the akiyo sequence comparison for the JVT-G012 and the proposed algorithms.Figure3 shows obviously that the highway keeps the edge part of images to which human eyes are sensitive, while the one obtained from the JVT-G012 algorithm distorts and influences the subjective feeling of eyes.Comparison of the subjective quality in the highway sequence for the JVT-G012 and the proposed algorithms.
(a)
(b)Figures4, 5, and 6 are the comparison of the subjective quality. For the test sequences claire, mother-daughter, and silent, their facial and body parts have a dramatic decline in image quality in the JVT-G012 algorithm. However, these features have better visual quality in the proposed algorithm.Comparison of the subjective quality in the claire sequence for the JVT-G012 and the proposed algorithms.
(a)
(b)Comparison of the subjective quality in the mother-daughter sequence for the JVT-G012 and the proposed algorithms.
(a)
(b)Comparison of the subjective quality in the silent sequence for the JVT-G012 and the proposed algorithms.
(a)
(b)Figure7 is the comparison of subjective quality for the test sequence carphone. It is found that the sensitive regions of eyes in the image, such as the face, clothing, and edge parts, have become blurred in the JVT-G012. However, these features have better visual quality in the proposed algorithm.Comparison of the subjective quality in the carphone sequence for the JVT-G012 and the proposed algorithms.
(a)
(b)


## 5. Conclusions

This paper proposes a reformative basic unit layer rate control algorithm by using the visual characteristic factor and the adjusted quantization parameter. The proposed algorithm allocates bits based on the HVS in basic unit layer and adjusts the quantization parameter with texture bits and the header bits of the current basic unit. The experimental results show that the proposed algorithm can control the bit rate more accurately and have a much better visual quality. Compared with the JVT-G012 algorithm, the PSNR can be improved by 0.2–0.7 dB. What is more, the PSNR has little fluctuation and the video image will become more stable. Compared with [10] algorithm, the PSNR for different test sequences has been improved obviously in this paper. The [10] algorithm has great effect on particular tests, while the proposed algorithm in this paper has universal applicability. In addition, the algorithm in this paper has great effect at low bit rates.



---

*Source: 270692-2013-11-06.xml*