905

Sensors and Materials, Vol. 31, No. 3 (2019) 905–922
MYU Tokyo
S & M 1824

Hardware Architecture Design for High-performance
H.264/AVC Deblocking Filter
Xiaodong Zheng,1 Wei Zhu,1* Shaoyong Yu,2 and Jinpeng Wu1
1

School of Software Engineering, Xiamen University of Technology,
No. 600, Ligong Road, Jimei District, Xiamen City 361024, China
2
School of Mathematics and Information Engineering, Longyan University,
No. 1 Dongxiaobei Road, Xinluo District, Longyan City, Fujian Province 364012, China(Received October 15, 2018; accepted January 30, 2019)

Keywords:
low cost

block-based discrete cosine transform (BDCT), block effect, deblocking filter, low memory,

In multimedia imaging/video compression systems, the block-based discrete cosine
transform (BDCT) is widely applied to conversion and compression. During the quantizationof compression, after processing with an inverse discrete cosine transform (IDCT), a certain
defect emerges in BDCT, i.e., two adjacent blocks fail to be perfectly joined together. This
block effect of the picture undermines the video quality and tends to become more pronounced
as the sampling factor increases. In image decoding, a deblocking filter requires an enormous
amount of computation. If the computation is carried out entirely by software, a large system
bandwidth will inevitably be occupied. To overcome this problem, the design of a suitablehardware architecture is urgently required. In this study, we proposed the use of the system bus
and a low-memory, low-cost, and effective deblocking filter architecture to reduce the operation
time of the deblocking filter. In contrast to existing hardware architectures, the design proposedin this paper adopts dual-port static random access memory (SRAM) and two-port SRAM
architectures, thereby storing data via a new data reading approach, to save transposed memory
and reduce the hardware size. By coupling the proposed architecture with parallel processing
units, the processing speed will be increased.

1.

Introduction

With the booming of information technology, various multimedia technologies are often used
via Internet applications, such as video conferencing, video on demand, and video surveillance.
Network technology has led to the introduction of broadband internet access; however, with theincreasing number of users and the higher demand for picture quality, the network bandwidth
is expected to be saturated soon. Therefore, multimedia video compression technology will
be heavily relied on to compress data, so as to store more multimedia data in existing storage
spaces, which in turn can reduce the time required for data transformation via the Internet.
The H.264/AVC standard for video compression has recently been jointly developed*

Corresponding author: e-mail: zhw@xmut.edu.cn
https://doi.org/10.18494/SAM.2019.2163
ISSN 0914-4935 © MYU K.K.
https://myukk.org/

906

Sensors and Materials, Vol. 31, No. 3 (2019)by ITU-T (Video Coding Experts Group, VCEG) and ISO/IEC (Moving Picture Experts
Group, MPEG).(1) The picture quality, compression efficiency, and error tolerance of H.264/
AVC significantly outperform those of previous video compression standards. Compared withother video compression standards,(2) in the case of low-data movies, H.264/AVC can provide
higher picture quality and compression efficiency. However, the significant increase in thecomputational complexity of the signal results in many obstacles for the use of H.264/AVC
in real-time systems. Similarly to other existing video compression systems, the block-based
discrete cosine transform (BDCT) and quantization based on block processing are commonly
used, whose signal is processed by dividing the entire picture into non-overlapping blocks. In
BDCT, each block is then converted from a space domain to a frequency domain. Then, theobtained coefficient is divided by a quantization-parameter-based quantization matrix. As aresult of this processing, the high-frequency signal invisible to human eyes is removed, thereby
achieving data compression. However, when a relatively large quantization parameter isused, the BDCT coefficient obtained after quantization and the amount of data are sufficient,indicating that the quantization parameter directly leverages the compression quality of a video
system.
A deblocking filter is involved in the next-generation video compression system based on
H.264/AVC and called the in-loop filter. Traditionally, a deblocking filter is integrated intoa video codec as a postfilter, then the built-in deblocking filter will process the picture into areference image.(3) In contrast, a H.264/AV deblocking filter is equipped with a mechanismthat is highly adaptable to different picture sources, which produces higher video quality.(4) In
the H.264/AVC standard, the deblocking filter is an essential mechanism for ensuring video
quality. Taking into account the fact that the prediction, conversion, quantization, and motion
compensation blocks have minimum sizes of 4 × 4, to remove the blocking effect in the picture,
the deblocking filter must filter every 4 × 4 matrix in the picture, that is, almost every pixel
must be processed by the deblocking filter. To this end, it is necessary to carry out complicatedreading and access of the memory storing the picture signals. In addition, H.264/AVC utilizes
various thresholds and conditions to determine and select modes, so as to work in concert withhighly adaptable and tunable pictures. Although H.264/AVC has been optimized in terms of the
deblocking algorithm, it still contributes one-third of the total computational complexity of thedecoder.(5)The hardware reported in Refs. 6–9 applies a traditional processing flow, i.e., firstly, all
the pixel signals necessary to process a macroblock (MB) are loaded into the design, andthen they are transferred back to the picture memory. Huang et al.(6) proposed a typicalarchitecture, wherein horizontal filtering can be processed perfectly, while in the verticaldirection, transposed memory must be introduced to avoid the possibility of a memory crash;
this process takes nearly twice the computing time in Ref. 7. However, it only considered thecalculation of the luminance part of the MB, while the color part was not considered, which isconsidered to be a limitation of this study. Our newly proposed memory architecture is basedon the concept of two-dimensional memory processing introduced by Li et al.,(8) wherein the
pixel data originally stored in the same memory module is interleaved, so as to avoid the abovementioned memory crash in Ref. 6. Indeed, a certain hardware cost is required, but suchprocessing is expected to reduce the use of transposed memory. Venkatraman et al.(9) suggested

Sensors and Materials, Vol. 31, No. 3 (2019)

907

that two sets of deblocking filter modules can be applied at the same time, which can enable theparallel operation of horizontal and vertical filtering, thus reducing the operation time. This
however, results in a higher cost in terms of the memory and hardware circuit in the system.
The hardware designs proposed in Refs. 10–15 attempted to adopt a process integrating reading,write-back, and processing, while increasing the hardware usage and reducing the waiting timeof the circuit. Cheng et al.(10) applied shift registers to preserve the temporary data that requires
continuous operation, so as to reduce the time required for data access. In the literature studyof Chang et al.,(11) pixels that must be filtered are first calculated, and the pixels that do notneed to be filtered are not transferred into the hardware, resulting in a variable operation time.
By using two-dimensional continuous operations, Sheng et al.(12) were able to retain more
temporary data and reduce the time required for reading and loading. The disadvantage of theirapproach is the high memory requirement;(13–15) slice memory was introduced to store the pixel
data of horizontal pictures, thereby reducing the time spent reading and loading data, which,however, increases the demand for memory. Zheng et al.(13) attempted to use more internalbuffers to reduce the times required for data reading and loading. Liu et al.(14) proposed to unify
external memory and internal transpose memory to reduce computation time, and Shih et al.(15)
proposed the design of a fifth-order pipeline filter with a parallel computation to increase theboundary strength (BS). Effective hardware architectures(6–15) have generally aimed to reducethe circuit size, memory cost, and operation time. However, these architectural designs failed toeffectively take into account the memory cost and processing speed. In contrast to the previous
designs in Refs. 6–15, in this paper, we propose a more effective hardware architecture thatenables the H.264/AVC deblocking filter to achieve a lower memory capacity, reduced accessrequirements, and a reduced number of operation cycles.
In Sect. 2, we outline the deblocking filter algorithm in accordance with H.264/AVC. In Sect. 3,
we thoroughly explain our newly proposed effective hardware architecture design. In Sect. 4,
we compare the efficiency of the proposed architecture with that of the previous architectures.
We conclude this study in Sect. 5.

2.

Deblocking Filter Algorithm for H.264/AVC

A deblocking filter is used to eliminate the blocking effect in pictures by applying its
algorithm,(5) thereby generating a smoother picture. In the block-based H.264/AVC standard,
the blocking effect originates from the use of conversions with 4 × 4 matrices and block
motion compensation. Therefore, a deblocking filter is considered as an effective tool for
removing the blocking effect. In theory, a deblocking filter can be separated from the system
by postfiltering,(5) which is only used to filter displayed pictures. By introducing a deblocking
filter in the loop at the encoding end, higher visual quality is expected to be achieved. This isbecause the reference picture used for motion compensation has been filtered and reconstructed.
Another benefit is that adding a deblocking filter to the system allows the image provider tospecify the picture quality at the transmission end.(16–20)
The deblocking filter used in the H.264/AVC standard has high adaptability. The large
number of thresholds can be used to adjust the strength of the filter in accordance with thepicture and image characteristics. Also, the thresholds can be fine-tuned in accordance with

908

Sensors and Materials, Vol. 31, No. 3 (2019)

quantization parameters (QPs), because the generation of the blocking effect is directly related
to QPs.(4)2.1Filtering order

The deblocking filter applied in H.264/AVC carries out processing with the MB as a unit.
There are many 4 × 4 blocks within an MB. The deblocking filter first processes the verticalsides of adjacent blocks in the horizontal direction before processing the horizontal sides ofadjacent blocks in the vertical direction. After deblocking filtering in both directions in an MB
is completed, the next MB is processed. As shown in Fig. 1, deblocking filtering is carried out
on all the MBs in the picture in a raster-scan manner until the MBs of the entire picture arefiltered. Deblocking filtering requires a maximum of eight pixel values. Taking into accountthe BS and QP, filtering may not be performed. If the pixel values of up to six points are to be
adjusted, the values of up to three pixels (p2, p1, p0, q0, q1, q2) can be changed on each side of
the boundary, as shown in Fig. 2.
2.2

BSFor two adjacent 4 × 4 luma component blocks, a parameter ranging between 0 and 4 called
the BS is specified in the standard. For two adjacent blocks, the BS is determined from theselection of the intrablock/interblock prediction mode, the motion vector error, and whether the
residual value is encoded. If two adjacent 4 × 4 blocks are subjected to intrablock coding, andthe two adjacent sides overlap the boundary of the MB, the strongest filtering mode is used andthe BS is set to 4. If two adjacent blocks are subjected to intrablock coding, without overlapping
the boundary of the MB, the BS is set to 3. If the above conditions have not yet been satisfied,the determination process is continued. If intrablock coding is used by one of the blocks andone block has added a residual value into the code, a medium-intensity filtering mode is applied
and the BS is set to 2. If the motion compensation of the two blocks refers to different pictures,or the difference between the moving coordinates of the two blocks is at least one luma, aFig. 1.

(Color online) Raster scan.

Fig. 2. (Color online) Filtering of vertical andhorizontal boundaries.

Sensors and Materials, Vol. 31, No. 3 (2019)909

relatively weak filtering mode is chosen and the BS is set to 1. When all the above-mentioned
conditions are not yet satisfied, the boundary remains unfiltered and the BS is set to 0, as shownin Fig. 3. Also, the BS of the chroma component is not to be recalculated; instead, the BS of thecorresponding luma component is directly copied to the boundary of the chroma component, asshown in Fig. 4.
2.2.1 Definition of thresholds α and β
The set of sample values to be filtered was mentioned in Sect. 2.2. If the decision of whether
or not to carry out filtering is made simply according to the BS, it may result in a blurred
picture. Therefore, only the edges with a blocking effect are deblocked, and the other edges
remain unchanged so as to preserve the original sharpness of the picture. The decision process
is shown in Fig. 5, which indicates that the sample value will be regarded as having a squareeffect only when the following conditions are met. Thereby, deblocking filtering will be carriedout on the sample in accordance with Eqs. (1)–(4).

Fig. 3.

Fig. 4.

Decision tree of BS.

(Color online) BS of MB at different positions.

910

Sensors and Materials, Vol. 31, No. 3 (2019)

Fig. 5.

(Color online) Block edge satisfying the filtering conditions.

BS = 1, 2, 3, or 4

(1)

| p 0 − q 0 | < α ( Index A )(2)| p 0 − p1| < β ( IndexB )(3)| q 0 − q1| < β ( IndexB )(4)

With increasing QP of blocks Q and P, the thresholds α and β increase, where α and β are
regarded as the criteria for judging whether the changes in the original image are sufficiently
large. If the QP shrinks, any changes at the block edge are regarded as the original feature of
the picture, not a false edge brought about by the blocking effect. On this basis, α and β are set
to small values, to keep the original picture as unchanged as possible. Given that the distortion
driven by the block effect becomes rather pronounced when the QP is increased, the inevitable
increases in α and β lead to more sampling points in the picture required to apply the deblocking
filter.
The slice level of H.264/AVC’s is defined by two parameters, which are to adjust offsets
at the encoding end, i.e., Offset A and Offset B. These offsets can be fine-tuned at thecompression end, thereby causing the filter to apply different thresholds α and β for the same
QPs.
=
Index A Min( Max(0, QP + Offset A ),51)(5)=
IndexB Min( Max(0, QP + Offset B ),51)(6)

Sensors and Materials, Vol. 31, No. 3 (2019)911

2.2.2 Filter mode utilized when BS = 1–3
A filter with a basic strength is applied when BS = 1–3. The boundary pixels p3, p2, p1, p0,
q0, q1, q2, and q3 are input to obtain P1, P0, Q0, and Q1 via its algorithm. Only the filtered P0
and Q0 are output, which replace the original p0 and q0 when they satisfy Eqs. (1)–(4). Only
the filtered P1 is output, which replaces the original p1 when it satisfies Eq. (9). Similarly, Q1
is output and replaces the original q1 if and only if Eq. (10) is satisfied. The equations used
to calculate P1, Q1, P0, and Q0 are Eqs. (11) to (14), respectively, where c1 is the BS and thecoefficient is related to IndexA. In the luma component, c0 is the number of true values of c1 +Eqs. (9) and (10). When considering the chroma component, c0 is fixed to c1 + 1.
| p 2 − p 0 | < β ( IndexB )(9)| q 2 − q 0 | < β ( IndexB )(10)

2.2.3 Filter mode utilized when BS = 4
The strongest deblocking filter is applied when BS = 4. The pixels p3, p2, p1, p0, q0, q1, q2,
and q3 are input to obtain P2, P1, P0, P0f, Q0f, Q0, Q1, and Q2 using the algorithm.
In addition to satisfying Eqs. (1)–(4), if the luma component blocks satisfy Eqs. (9)–(15), thefiltered P2, P1, and P0 are output and replace the original pixels. Similarly, Q2, Q1, and Q0
are only output when they are positioned in the luma component and satisfy Eqs. (10) and (15).
In the case of the chroma component, if Eq. (9) or (15) is not satisfied by the luma component,
then only P0f out of p2–p0 replaces p0 and is output, with p2 and p1 remaining unchanged.
Vice versa, when it comes to the chroma component, or one of Eqs. (10) and (15) in the luma
component is not true, then only Q0f out of q0–q2 will replace q0 and be output, while q1 andq2 remain unchanged. The formulas for calculating P2, P1, P0, P0f, Q0f, Q0, Q1, and Q2 aregiven as Eqs. (16)–(23).
P=
1 p1 + CLIP (−c1, c1, ( p 2 + (( p 0 + q 0 + 1) >> 1) − 2 p1) >> 1)

(11)

Q=
1 q1 + CLIP (−c1, c1, (q 2 + (( p 0 + q 0 + 1) >> 1) − 2q1) >> 1)

(12)

P=
0 p 0 + CLIP (−c0, c0, (4(q 0 − p 0) + ( p1 − q1) + 4) >> 3)

(13)

Q=
0 q 0 − CLIP (−c0, c0, (4(q 0 − p 0) + ( p1 − q1) + 4) >> 3)(14)| p 0 − q 0 | < ((α ( Index A )) >> 2) + 2(15)=
P 2 (2 p3 + 3 p 2 + p1 + p 0 + q 0 + 4) >> 3

(16)

912

Sensors and Materials, Vol. 31, No. 3 (2019)=
P1 ( p 2 + p1 + p 0 + q 0 + 2) >> 2

(17)=
P 0 ( p 2 + 2 p1 + 2 p 0 + 2q 0 + q1 + 4) >> 3

(18)

Q
=
0 (q 2 + 2q1 + 2q 0 + 2 p 0 + p1 + 4) >> 3

(19)

Q
=
1 (q 2 + q1 + q 0 + p 0 + 2) >> 2

(20)=
Q 2 (2q3 + 3q 2 + q1 + q 0 + p 0 + 4) >> 3

(21)

P=
0 f (2 p1 + p 0 + q1 + 2) >> 2

(22)

Q=
0 f (2q1 + q 0 + p1 + 2) >> 2

(23)

In these equations, CLIP stands for the reduction operation and >> represents the shiftoperation.

3.

Hardware Architecture Design

In Fig. 6, an architecture for the deblocking filter system is proposed. In the memory,interleaved methods are applied to store pixels, thereby addressing the problem of transposing
memory in previous designs. In addition, two parts of modularized memory with different
functions are proposed, equipping both memory modules with the capability of twodimensional storage. One of the memory modules is centered on the core of the dual-port static
random access memory (SRAM), storing the block data under real-time processing, which isreferred to as the RAM-0 module. The other memory module is in the mode of the two-port
SRAM, which stores the block data on the far right of the previous MB and is named the RAM1 module. A novel feature of this architecture is that a fourth-order pipeline filter is placed inFig. 6.

(Color online) Architecture for the proposed deblocking filter system.

Sensors and Materials, Vol. 31, No. 3 (2019)913

the deblocking filter module, which, coupled with the proposed recursive control, can reduce
the frequency of memory access. In the control unit, in addition to controlling the addressesof the basic memory components, the data stream selection, and the data input and output, theparameters required for the deblocking filter module must also be stored in a system on chip (SOC).
These parameters include IndexA, IndexB, and the BS. In the system architecture diagram
shown in Fig. 6, except for the control signal lines, the width of the entire internal data bus is 32
bits. The external bus input and output pixel data are asynchronous. Therefore, they can shareone 32-bit bidirectional channel, making this architecture ideal for an SOC.
Four 4 × 4 luma component blocks and four 4 × 4 color component blocks from an MB are
stored in the module RAM-1, while the data is composed of eight groups of 16 word × 8 bit
dual-port memory, for which the required storage space is 32 × 32 bits, equivalent to 1024 bits.
The other memory module RAM-0 temporarily stores the pixel data of the MB while it is being
processed. For the deblocking filter processing flow proposed in this paper, the maximum
temporary storage is sixteen 4 × 4 blocks; the RAM-0 module consists of eight groups of 32
word × 8 bit dual-port memory, for which the storage space required is 64 × 32 bits, equivalent
to 2048 bits.
For the deblocking filter architecture proposed in this paper, the processing of an MB only
takes 279 computation cycles, among which five cycles are required to load the calculated BS
and the parameters IndexA and IndexB, and the other 274 cycles are required for loading pixels
and restoring data. When the processing of an MB has reached the last MB in the picture, anadditional 32 cycles are required to store the pixel data remaining in RAM-1 in the memory of
the reconstructed picture, giving a total of 301 computation cycles.
3.1

Data processing flow

The deblocking filter signal processing proposed in this paper is shown in Figs. 7 and
8, in which B0–B39 are 4 × 4 pixel blocks. The circles numbered 1 to 48 all require fourFig. 7.

(Color online) Data processing sequence.

914

Sensors and Materials, Vol. 31, No. 3 (2019)

Fig. 8.

(Color online) Time sequence of deblocking filter.

computation cycles; ellipse H indicates horizontal filtering of the vertical edges and ellipse V
indicates vertical filtering of the horizontal edges. In the first stage, the data in block B5 isreceived from the outside, and the data in block B4 is extracted from the RAM-1 module andsent to the deblocking filter module. In the second stage, the block filter will successively
complete the B4 and B5 obtained from the first stage in a synchronized manner. At this point,
because B5 must be input in the deblocking filter again, it is not necessary to write B5 in thememory. Instead, it is necessary to send B5 and B6 received from the outside to the deblocking
filter module. At the same time, B4 is temporarily saved in the RAM-0 module. In the fifth
stage, the data in block B0 data is received from the outside, and the data in block B5 stored in
the RAM-0 module in the second stage is extracted and sent to the deblocking filter module.
Moreover, B7 and B8 are stored together in the RAM-0 module. After stage eight has been
completed, B0–B3, which are not used again in the current MB processing flow, are extracted
and sent to the outside of the deblocking filter module. After completing stages 16, 24, 32, 45,
46, 47, and 48, blocks B8, B13, B18, B23, B26, B29, B34, and B37 are respectively stored in
the RAM-1 module, because the next MB is soon applied again, thereby the time spent in theread and load stages is reduced, and the repeated loading and writing of the same block can beeliminated.
3.2

Internal memory planning

In Fig. 9, during the vertical access of the memory, memory crash occurs, which hinders theaccess from being completed in one execution cycle. This means that transposed memory must
be used, which can result in reduced efficiency. To solve this problem, we employ the twodimensional memory access design proposed in Ref. 8, which applies interleaved methods fordata placement in different memory modules, eliminating the origin of the memory crash in theprevious design. It also makes it possible to read and write in both the horizontal and verticaldirections.

Sensors and Materials, Vol. 31, No. 3 (2019)

(a)

915

(b)

Fig. 9. (Color online) Diagram of memory access. (a) Traditional memory storage and (b) shift transposition
storage method.

In a two-dimensional memory module, considering that vertical pixel data is placed indifferent modules, memory crash can be entirely avoided, allowing both horizontal and verticaloperations to be accelerated in two-dimensional memory without the occurrence of memory
crash.
To achieve two-dimensional memory access, compatible components are required for thecircuit to conduct data segmentation, address generation, and data combination, as shown in
Fig. 10. The address generator produces the address corresponding to the read or write of thememory row or column as required. Also, the module in which data segmentations occurs
is responsible for shifting the input pixel and determining the frequency of the data shift fordifferent input addresses, so as to enable vertical pixel signals to be stored in different memory
modules. The data combination module receives the shifted data and the output from thememory unit, and according to the output address, the data combination module reverses theshift by restoring the pixel data in the memory module back into the pixel data that has not beenpreviously shifted. Thereby, the correct sequence can be obtained, facilitating the subsequent
processing.
3.3

Pipeline-based deblocking filter module

To optimize the processing performance of the deblocking filter, a parallel and pipelined
circuit design is used as shown in Fig. 11. The parallel processing data includes eight pixel
inputs, eight pixel outputs, and a set of recursive inputs, and the reduced latency of the critical
path is largely avoided by employing a fourth-order pipeline design. Different signals are
selected as the input source to read pixels. The purpose of Stage 1 is to query the values of α, β,and Clip in line with the values of IndexA, IndexB, and BS, respectively, so as to conduct initialprocessing on the input pixels. In Stage 2, the output from Filter Stage 1 is processed and theprecalculation of the decision flag required by the final signal selector is conducted. Stage 3
performs the calculation in the last stage using the output result obtained from the part of Filter

916

Sensors and Materials, Vol. 31, No. 3 (2019)

Fig. 10. (Color online) Schematic diagram of memory modules.

Fig. 11.

(Color online) Architecture diagram of pipeline deblocking filter module.

Stage 2 with BS = 4. Filter Clip applies the reduction operation to the output result obtained
from the part of the Filter Clip with the BS ranging from 1 to 3. In the Filter Out stage, the final
result based on the previously calculated decision flag is selected and output.
3.3.1 Flag operation unit
On the basis of the absolute value obtained by subtracting the real-time BS from the pixel
values, a flag operation unit can determine whether the boundary on the picture is a false
boundary generated by the block effect or a real one sourced from an actual picture, from which
FLAG1 to FLAG6 and FLAG chroma can be obtained. In the case of different BSs, the flag isused for decision making [Eqs. (24)–(26) are all satisfied]. If a false boundary resulting fromthe blocking effect is spotted, the filtered pixels are output to remove the effect. If the result isthe boundary of a real picture’s boundary [one of Eqs. from (24)–(26) is not satisfied], or the BS
is 0, the original pixel can be left unfiltered. Output selection lists are detailed in Tables 1 and 2.
FLAG1 = | p 0 − q 0 | < α ( Index A )(24)

FLAG 2 =| p1 − p 0 | < β ( IndexB )(25)

Sensors and Materials, Vol. 31, No. 3 (2019)

917

Table 1
Output selection table when BS = 1–3.
Filtered
pixels
P1
P0
P0
P0
P0
P0

Flag1

Flag2

Flag3

Flag4

Flag5

True
True
True
True
True
True

True
True
True
True
True
True

True
True
True
True
True
True

True
False
True
False
True
X

X
False
False
True
True
X

Flag
chroma
False
False
False
False
False
True

Outputsymbolbs1p1
bs1p0
bs1p0a1
bs1p0a1
bs1p0a2bs1p0a1

Table 2
Output selection table when BS = 4.
Filtered
pixels
P2
P1
P0
P0
P0
P0

Flag1

Flag2

Flag3

Flag4

Flag5

Flag6

True
True
True
True
True
True

True
True
True
True
True
True

True
True
True
True
True
True

True
True
True
False
True
X

X
X
X
X
X
X

True
True
True
True
False
X

Flag
chroma
False
False
False
False
False
True

Outputsymbol
bs4p2
bs4p1
bs4p0
bs4p0fbs4p0fbs4p0fFLAG 3 =| q1 − q 0 | < β ( IndexB )(26)

FLAG 4 =| p 2 − p 0 | < β ( IndexB )(27)

FLAG 5 = | q 2 − q 0 | < β ( IndexB )

(28)

FLAG=
6 | p 0 − q 0 | < (((α ( Index A )) >> 2) + 2)

(29)

FLAG Chroma= (=y 0) or (u or =
v 1)

(30)

3.3.2 When BS = 1–3“True”, “False”, and “X” in the following tables represent true, false, and unaffected,
respectively. Taking P1 as an example, when FLAG1, FLAG2, FLAG3, and FLAG4 are all true
and FLAG Chroma is false, then the output of P1 is the value (bs1p1) processed by the filter;otherwise, it remains unchanged.
bs1 p=
1 p1 + clip (−c1, c1,(( p 2 + (( p 0 + q 0 + 1) >> 1) − 2 p1) >> 1))(31)

bs1 p=
0 p 0 + clip (−c0, c0,(((q 0 − p 0) << 2) + ( p1 − q1) + 4) >> 3))

(32)

918

Sensors and Materials, Vol. 31, No. 3 (2019)

3.3.3 When BS = 4bs=
4 p 2 ((2 p3 + 3 p 2 + p1 + p 0 + q 0 + 4) >> 3)

(33)

bs 4=
p1 (( p 2 + p1 + p 0 + q 0 + 2) >> 2)

(34)

bs 4=
p 0 (( p 2 + 2 p1 + 2 p 0 + 2q 0 + q1 + 4) >> 3)(35)

bs 4 =
p 0 f ((2 p1 + p 0 + q1 + 2) >> 2)(36)

3.3.4 Operation decomposition of operation for pipeline filter
According to the deblocking filter algorithm proposed in H.264/AVC, the operation equation
contains many common items. Therefore, we can further optimize the decomposition while
carrying out hardware design and implementation, which can also facilitate the segmentationof the pipeline filter. For example, Eqs. (33)–(36), for which BS = 4, can be replaced by thefollowing equations:=bs 4 p 2 ( Sp 0q 0 p1 p 2 + ( Sp 2 p3 << 1)) >> 3;(37)=
bs 4 p1 ( Sp 0q 0 p1 p 2) >> 2;(38)=
bs 4 p 0 ( Sp 0q 0 p1q1 + Sp 0q 0 p1 p 2) >> 3;

(39)

bs=
4 p 0 f ( Sp 0 p1 + Sp1q1) >> 2;(40)

Taking Eqs. (10) and (11) with the BSs of 1, 2, and 3 as an example, we derived the following
procedures:=
bs1 p1 ( Sp 0q 0 + {Sd 2 p1 p 2[8], Sd 2 p1 p 2,1' b0}) >> 2;(41)=
bs1 p 0 (( Sq 0dp 0 << 2) + {{Sp1dq1[9]}, Sp1dq1} + 11' d 4) >> 3;(42)=
bs1q1 ( Sp 0q 0 + {Sd 2q1q 2[8], Sd 2q1q 2, 1' b0}) >> 2;(43)

A number of common items are involved in Eqs. (37)–(43), which can be extracted andfurther simplified as shown below:
Sp 0q 0 = p 0 + q 0 + 1;

(44)

Sp 0 p1 = p 0 + p1 + 1;

(45)

Sensors and Materials, Vol. 31, No. 3 (2019)919

Sp1 p 2 = p1 + p 2 + 1;

(46)

Sp 2 p3 = p 2 + p3 + 1;

(47)

Sp 0q 0=
p1q1 Sp 0q 0 + Sp1q1;(48)

Sp 0q 0=p1 p 2 Sp 0q 0 + Sp1 p 2;

(49)

Sp 0q 0=
q1q 2 Sp 0q 0 + Sq1q 2;(50)

Sq 0dp=0 q 0 − p 0;(51)

Sp1dq=
1 p1 − q1;(52)

Sd 2 p1=
p 2 ( p 2 − ( p1 << 1));(53)

Sd 2q1q=
2 (q 2 − (q1 << 1));(54)

where Eqs. (44)–(47) and (51)–(54) can be completed in the Filter 1 stage, Eqs. (40)–(43) and (48)–(50) can be completed in Filter Stage 2, and Eqs. (37) and (39) can be carried through to Filter
Stage 3. The trimming functions required for Eqs. (8)–(11) can be completed during the Filter
Clip.

4.

Simulation Results and Comparison

The hardware circuit design for the proposed architecture was based on VerilogHDL andsynthesized using Synopsys Design Compiler under a TSMC CMOS 0.18 μm process withthe operating frequency set to 100 MHz. The number of synthesized logic gates was 19.4 K.
By comparing the proposed hardware architecture with those in the recent literature,(6–15) weavoided the disadvantages of the deblocking filter in terms of the memory cost and processing
speed, as shown in Table 3.
We hereby briefly expounded on the fields that were evaluated. Cycles/MB is defined as
the number of execution cycles required to fully filter an MB. Filter cycle/MB is the numberof operation cycles actually needed by the deblocking filter in the core. SRAM for pixels is
the memory applied in the design, and SRAM in bits is the memory in bits used to directly
convert the design via the capacity that can be stored, where the space involved in this part is
not included in the gate count. 4 × 4 registers consist of the internal memory of the circuit,
first-in, first-out (FIFO), the transposed memory, and space. Registers in bits are the storage
space that is converted to the corresponding bits; # of edge filters is the number of deblocking
filter cores utilized. The calculation of frames per second (FPS) can produce the number offrames of 1280 × 720 size that can be processed per second at the same frequency of 100 MHz;

920

Sensors and Materials, Vol. 31, No. 3 (2019)

Table 3Comparison of hardware circuit designs.
MethodRef. 10

Ref. 11

Ref. 12

Ref. 13

Ref. 14

Cycles (MB)

614

584

566

510

386

Filter cycle (MB)

240

214

192

SRAM for pixels

Dual 96 × 32
Dual 64 × 32

Dual 16 × 32

8 Dual 80 × 8

5120
4
512
1

512
6
768
1

5120
0
0
1

Ref. 15
Max 374
Min 50Avg. 86–244
0–374

136
336
Dual 88 × 32
Dual 72 × 32 Single 80 × 32 Single 96 × 32
Single 32 × 32
6144
2560
3072
11
2
2
1408
256
256
2
1
1

SRAM in bits
#4 × 4 registers
Registers in bits
# of edge filters
Capability@1280*720p@
100 MHz
Process (μm)
Gate count
Performance

45 FPS47.5 FPS

49.1 FPS

54.46 FPS

71.9 FPS

113.8–322.9
FPS.25
20.66k3458048

N/A(FPGA)
N/A(FPGA)
750080.35
9.35k2897920

N/A(FPGA)
N/A(FPGA)
3851520.18
21.8k1086976.18
20.9k1244672

MethodCycles (MB)Filter cycle (MB)Ref. 16446
192

Proposed
279
277

SRAM for pixels

Dual 64 × 32
2 Two 96 × 32

Ref. 17
Ref. 18
Ref. 19
238
250
214
236
250
200
Dual 96 × 32 2 Single 96 × Single 96 × 32
Two 32 × 32
32
Dual 64 × 32
SingleSingleSingle(1.5 × frame(2 × frame(2 × frame
width) × 32 width +20) × 32 width) × 32
5120 + 65536 6144 + 66176 4096 + 49152
9
4
2
1152
512
256
1
1
1

SRAM in bits
#4 × 4 registers
Registers in bits
# of edge filters
Capability@1280*720p@
100 MHz
Process (μm)
Gate count
Performance

8192
8
1024
1

8 Dual 32 × 8
8 Two 16 × 8
3072
0
0
1

62.3 FPS

97.1 FPS

111.1 FPS

128.6 FPS

98.15 FPS

.25
24k

.18
19.5k
180633620680704.1819.6k
1664000
18208000.1820.9k
93132811449856

.18
19.4k

4110336857088taking into account the design process, the cost of the circuit after synthesis, and the figure of
merit derived by the performance measurement system P (performance = cycle × memory), theobtained reference factors are in line with the standard. We carried out a comparison of the
proposed hardware architecture with those in the recent literature, as summarized in Table 3.

5.

Conclusion

In this paper, a new hardware architecture design was proposed, which employs a deblocking
filter with low cost and high efficiency. The distinguishing feature of the proposed hardware

Sensors and Materials, Vol. 31, No. 3 (2019)921

architecture is that it is based on a static random-access memory module, its application ofa fourth-order pipeline deblocking filter architecture, and its improved read/load capability,
reducing the computation time needed and the frequency of memory access. In terms ofintegrated circuits, the architecture design was carried out with Verilog HDL and the circuit
synthesis tool Synopsys Design Compiler under a TSMC CMOS 0.18 μm process.
In contrast to the architectures proposed in the recent literature, the deblocking filter
proposed in this study applies two-dimensional access memory for the interleaving of data,
which enables the design to be free from transposed memory. This approach is characterized bya low memory requirement (3072 bits) and as few as 279 operation cycles to calculate an MB.
The experimental results suggest that our newly designed hardware architecture is compatible
with the H.264/AVC video compression system and can satisfy the requirements for real-time
computation.

Acknowledgments
This work is supported by the young teachers’ education scientific research projectof Fujian province (Grant no. JT180455), Xiamen Science and Technology Foundation(Grant 3502Z20173035), and Fujian Provincial Natural Science Foundation of China (Grant
2018J01570).

References
1 T. Wiegand: ITU-T Rec. H. 264| ISO/IEC 14496-10 AVC, JVT-G050 (2003).
2 T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra: IEEE Trans. Circuits Syst. Video Technol. 13 (2003)
560.
3 JVT H.264/AVC Reference Software JM 9.5.
4 I. E. Richardson: H. 264 and MPEG-4 Video Compression (Wiley, 2004).
5 P. List, A. Joch, J. Lainema, G. Bjontegaard, and M. Karczewicz: IEEE Trans. Circuits Syst. Video Technol.
13 (2003) 614.
6 Y. Huang, T. Chen, B. Hsieh, T. Wang, T. Chang, and L. Chen: Proc. Int. Conf. Multimedia and Expo 1 (2003)
1–693.
7 M. Sima, Y. Zhou, and W. Zhang: IEEE Trans. Consum. Electron. 50 (2004) 292.
8 L. Li, S. Goto, and T. Ikenaga: IEICE Trans. Inf. Syst. 88 (2005) 1623.
9 V. Venkatraman, S. Krishnan, and N. Ling: Picture Coding Symp. (2004) 1623.
10 C. Cheng and T. Chang: IEEE Int. Conf. Consumer Electronics, Digest of Technical Papers (2005) 235.
11 S. Chang, W. Peng, S. Wang, and T. Chiang: IEEE Trans. Consum. Electron. 51.1 (2005) 249.
12 B. Sheng, W. Gao, and D. Wu: IEEE Int. Conf. Image Proc. 1 (2004) 665.
13 V. Venkatraman, S. Krishnan, and N. Ling: Proc. Picture Coding Symp., San Francisco, California, USA, Dec.
(2004).
14 T. Liu, W. Lee, T. Lin, and C. Lee: IEEE Int. Symp. Circuits and Systems (2005) 2140.
15 S.-C. Chang, W.-H. Peng, S.-H. Wang, and T. Chiang: IEEE Trans. Consum. Electron. 51 (2005) 249.
16 MPEG Video Group: MPEG-4 Video Verification Model Version 8.0, ISO/IEC JTC1/SC29/WG11 N1796, July
1997.
17 A. Luthra, G. J. Sullivan, and T. Wiegand: IEEE Trans. Circuits Syst. Video Technol. 13 (2003) 557.
18 J. Ostermann, J. Bormans, P. List, D. Marpe, M. Narroschke, F. Pereira, T. Stockhammer, and T. Wedi: IEEE
Circuits Syst. Mag. 4 (2004) 7.
19 K. Denolf, C. Blanch, G. Lafruit, and A. Bormans: IEEE Workshop Signal Proc. Systems (IEEE, 2002) 222.
20 M. Horowitz, A. Joch, F. Kossentini, and A. Hallapuro: IEEE Trans. Circuits Syst. Video Technol. 13 (2003)
704.

922

Sensors and Materials, Vol. 31, No. 3 (2019)

About the Authors
Xiaodong Zheng is a lecturer and teacher in the School of Software
Engineering at Xiamen University of Technology. His research interests lie in
the areas of software development, information security, and artificial
intelligence. (zxd@xmut.edu.cn)

Wei Zhu is a college lecturer in the School of Software Engineering at
Xiamen University of Technology. Her research interests lie in the areas ofsoftware development, information security, and artificial intelligence.
(zhw@xmut.edu.cn)

Shaoyong Yu received his Ph.D. degree from Xiamen University in 2017. He
is also a college lecturer in the School of Mathematics and Information
Engineering, Longyan University. His research interests lie in the areas ofcomputer vision and deep learning. (syyu@xmut.edu.cn)

Jinpeng Wu is a lecturer and teacher in the School of Software Engineering
at Xiamen University of Technology. His research interests lie in the areas ofprobability theory and mathematical statistics. (wjp@xmut.edu.cn)