SICE Annual Conference 2005 in Okayama, August 8-10, 2005 Okayama University, Japan

Noise Reduction of JPEG Imagesby Sampled-Data H ∞ Optimal ε Filters
H. Kakemizu†,1 , M. Nagahara‡,2 , A. Kobayashi‡,3 , Y. Yamamoto‡,4† Texas Instruments‡ Kyoto University, Kyoto, 606-8501, Japan
1 kakemizu@acs.i.kyoto-u.ac.jp
2 nagahara@i.kyoto-u.ac.jp
3 kobayashi@acs.i.kyoto-u.ac.jp
4 yy@i.kyoto-u.ac.jp
Abstract: We propose a new method to reduce artifacts of digital images decompressed via
Joint Photographic Experts Group (JPEG). JPEG images produce blocking and ringing artifacts,
which are distinctive especially at low bit rates. We introduce sampled-data H ∞ optimal filters,
which reduce such artifacts taking the analog characteristic of the original image into account,
combined with ε filters to preserve high frequency details. An example is presented to illustrate
the effectiveness of the proposed method.
Keywords: image processing, JPEG, sampled-data H ∞ optimization

1. Introduction
Artifacts in JPEG 9) images are caused mainly by thediscrete cosine transform and quantization. In particular, blocking and mosquito artifacts are well-known characteristics of JPEG images. Blocking artifacts are visible at low bit rates owing to discontinuities between
adjacent blocks. On the other hand, mosquitoes arefound around edges of decompressed images because oflack of high frequency data.
We treat these artifacts as high frequency noise added
to an image before compression, and propose to reduce
the noise via sampled-data H ∞ optimization 2) . By
sampled-data H ∞ optimization, we optimize the analog performance with an H ∞ optimality criterion. By
using that, we can take account of the analog characteristic of the original image (e.g., the 2D frequency
distribution of an object to be photographed by a digital camera). Moreover, we combine it with ε filters
3). They are nonlinear filters which can remove small
noise on images and preserve high frequency details. We
present an example to show that our method is superior
to conventional ones.

frequency-domain characteristic 1 of the analog signal
w. The signal y is then sampled by the sampler Sh
with sampling period h to become a discrete-time (ordiscrete-space) signal. Then, the signal is corrupted by
the additive noise d. The filter K(z) attempts to reduce
the noise and to reconstruct the original image. Thenthe filtered signal becomes an analog (i.e., continuoustime) signal by a hold device Hh and we obtain thereconstructed signal ye.
We thus consider the block diagram shown in Fig. 2that is the signal reconstruction error system for the
d
W (z)

w

+

F (s)

K(z)

Sh
y

Hh

ye

e
− +

e−Ls

2. Design Problem

Figure 2: Error system

Consider the block diagram shown in Fig. 1. The in-design. In the diagram the following points are taken
into account:

d
w

F (s)

y

+

K(z)

Sh

Hh

• The time delay e−Ls is introduced because we allow a certain amount of time delay for signal reconstruction.

ye• The noise obeys a frequency characteristic W (z).
Figure 1: Signal reconstruction

Our design problem is defined as follows:coming signal (image) w first goes through an analog
filter F (s) and the filtered signal y becomes nearly (butnot entirely) band-limited. The filter F (s) governs the- 1080 -1 In the conventional design, F (s) is considered to be an ideal
filter that has the cut-off frequency up to the Nyquist frequency

PR0001/05/0000-1080 ¥400 © 2005 SICE

Problem 1 Given a stable, strictly proper, continuoustime F (s), stable proper discrete-time W (z), sampling
period h, find the digital filter K(z) that minimizes
J(K) :=

kek2L22
2 .
w∈L2 ,d∈`2 kwkL2 + kdk`2

where φ is a hold function defined by(
1, θ ∈ [−h/2, h/2)
φ(θ) :=
0, otherwise.

sup

3.2 Reduction to a finite-dimensional
problem

3. H ∞ Optimal Filter
Problem 1 involves a continuous-time delay, and hence
it is an infinite-dimensional sampled-data H ∞ optimization problem. The problem is, however, reducible
to a finite-dimensional discrete-time one via fast sampling/hold approximation 5) or discretization preserving
the H ∞ norm 7) . We here discuss the norm preserving
discretization.

3.1 Non-causal hold for image reconstruction
By usual, a zero-order hold is used for converting
discrete-time signals into continuous-time ones. The
most common method is to causally hold the discretetime signals constant over the sampling intervals asshown in Fig. 3(a). The causal hold is appropriate forcausal signals such as audio ones, whereas non-causal
hold as shown in Fig. 3(b) is more suitable for noncausal signals such as images. The non-causal hold Hh

Let Tew denote the input/output operator of the error
system shown in Fig. 2, and {A, B, C, 0} be a minimal
realization of F (s). By the linear fractional transformation 11) , Tew is represented by F` (G0 , Hh KSh ), where
¸
· −Ls
e
F (s) −1.
G0 (s) :=F (s)0Assume L = mh, m ∈ Z+ := {0, 1, 2, . . .}. Then, forany γ > 0, we have 6)kF` (G0 , K)k∞ < γ ⇔ kF` (G, K̄)k∞ < γ,
where K̄(z) := z m K(z) and
·

¸



A
F (s) −1
G(s) :== C
F (s) 0
C

B
0
0


0
−1  .
0

By lifting G, we have the lifted system 2)A¯d B́d1
B̄d2
Ğ(z) =  C̀d1 D̆d11 D̀d12  ,
C̄d2
0
0

(2)

¸
· ¸
· ¸
·
Ā 0
0
B́1
, B́d1 =, B̄d2 =
,
0 0
1
0
£
¤
£
¤
C̀d1 = C̀1 C̀H , C̄d2 = C 0 ,
Ād =

h

2h

3h

4h

5h

D̆d11 = D̆11 ,(a) Causal holdD̀d12 = D̀H ,where {Ā, B́1 , C̀1 , D̆11 } is a state-space realization of thelifted system of F (s):(x[k + 1] = Āx[k] + (B́1 w)[k],y[k](θ) = (C̀1 x[k])(θ) + (D̆11 w)[k](θ),
Ā := eAh ,

h

2h

3h

4h

2

n

Z h

B́1 : L [0, h) → R : w 7→5heA(h−τ ) Bw(τ )dτ,0C̀1 : Rn → L2 [0, h) : x → CeAθ x,(b) Non-causal holdD̆11 : L2 [0, h) → L2 [0, h),Z θ: w 7→
CeA(θ−τ ) Bw(τ )dτ,Figure 3: Zero-order hold Hhis given as follows.
Hh : `2 3 ud 7→ u ∈ L2 ,u(kh + θ) := ud [k]φ(θ),θ ∈ [−h/2, h/2), k = 0, 1, 2, . . . ,θ ∈ [0, h),0(1)

and {C̀H , D̀H } is defined by lifting the non-causal holdHh (1):(
σ[k + 1] = ud [k],ye[k](θ) = (C̀H σ[k])(θ) + (D̀H ud [k])(θ),- 1081 -C̀H : R → L2 [0, h),2D̀H : R → L [0, h),: σ 7→ φ(θ + h)σ: ud 7→ φ(θ)ud ,θ ∈ [0, h).

For the sampled-data system F` (G, K̄), a normequivalent discrete-time system is obtained 1) assuming
γh := kD̆11 k < γ. However, since K̄ can be a non-causal
filter, it is possible that the optimal γ can be less thankD̆11 k by a non-causal K̄. Therefore, we adopt a discretization method which is independent of controller
causality constraints 7) .
Let Π̆B be the orthogonal projection of L2 [0, h) on
the null space N (B́d1 ). This is obtained by Π̆B =∗∗ †I − B́d1(B́d1 B́d1
) B́d1 where “†” indicates the pseudoinverse. For the system (2), define a discrete-time system
Ḡ by


Ā B̄1B̄2
Ḡ(z) :=  C̄1 D̄11 D̄12  ,C̄200
where B̄1 is a matrix which satisfies∗
B̄1 B̄10 = B́d1 B́d1,·
Γ(t) =

Γ11 (t)Γ21 (t)¸µ·Γ12 (t)A:= expΓ22 (t)−γ −2 C 0 C

¸ ¶
BB 0
t .
−A0
(4)

Then γ > γ̂h if and only if for any t ∈ (0, h],det (Γ12 (t)) 6= 0.
Moreover, we have·B̄1 B̄10 =¸
Φ12 (h)Φ−1
22 (h) 0 ,
0
0




  0 0
M11
C̄10
C̄1

 D̄a0   D̄a0  = M21
0
M13
0
0
D̄12
D̄12
M41

0
M21
M22
0
M23
0
M24

M13
M23
M33
M43

(5)

0
M41
M24 

0 ,
M43
M44

where each matrix Mij is given in the appendix.

and C̄1 , D̄11 , D̄12 are matrices which satisfy

4. Edge Preservation by ε Filters

D̄11 = D̄a B̄1 ,
0
C̄10
C̄1
³
´−1
∗
 D̄a0   D̄a0  = Òz∗ I − γ −2 D̆d11 Π̆B D̆d11
Òz ,
0
0
D̄12
D̄12
£
¤
∗
∗ †
Òz := C̀d1 D̆d11 B́d1
(B́d1 B́d1
) D̀d12 ,Lemma 1 For γ > 0, define
·
¸
µ·
¸ ¶
Φ11 (t) Φ12 (t)A BB 0Φ(t) =:= expt , (3)
Φ21 (t) Φ22 (t)
0 −A0


0

for γ > γ̂h := kD̆11 Π̆B k. Then we have the following
theorem.
Theorem 1 For any stable K̄, kF` (Ğ, K̄)k∞ ≥ γ̂h .
Moreover, for any stable K̄ and any γ > γ̂h , the followings are equivalent.

Filters designed by the method proposed above are linear ones. It is, however, difficult for linear filters to
distinguish original signals and noise in the same frequency bands. ε filters 3) are designed to remove small
noise on signals with sharp discontinuities. They distinguish edges and noise by setting a threshold ε on theamplitude of input signals. Since artifacts produced by
JPEG compression is relatively small, we effectively remove the noise by the filter proposed above with the εfilter.
The input/output relation of ε filter is given by

1. kF` (Ğ, K̄)k∞ < γ.
y[m] = x[m] −2. kF` (Ḡ, K̄)k∞ < γ.

NX

k[n] · f (x[m] − x[m − n]),n=−N

The proof is given by 4) . By using this theorem, we candesign the optimal filter K̄ as follows.
1. Give γ > γ̂h and compute the state-space realization of the discrete-time system Ḡ.
2. Define·Ḡaug :=z −m Ḡ11
Ḡ21

Ḡ12
0

¸

where x and y are input and output signals respectively,
and k is the impulse response of a filter which satisfies
NX

k[n] = 1.

n=−NThe function f is a nonlinear function such that for allx ∈ R,|f (x)| ≤ ε,where Ḡij (z) := C̄i (zI − Ā)−1 B̄j + D̄ij .
3. Solve a discrete-time H ∞ optimization problem

where ε > 0 is a given constant. Fig. 4 shows an example of f (x), which is defined bykF` (Ḡaug , K)k < γ.

(x,f (x) :=²,

m

Then K̄ = z K satisfies kF` (Ğ, K̄)k < γ.
For giving γ > γ̂h and computing Ḡ, the following
lemma is available.

- 1082 -|x| ≤ ε,|x| > ε.

By this nonlinear function, if the difference between

we set x[n − l] := x[n] to obtain a signal without edges,
as shown in the right of Fig. 5, which is to be filteredby k[n]. The threshold level ε can be chosen by the
amplitude of noise.

y

5. Design Examplex

In this section, we present a design example and compare our method with a conventional one. The design parameters are as follows: the sampling period
h = 1 and delay step m = 6. The analog filter is
F (s) = 1/(T s + 1)2 , T := 10/π. The weighting function for noise is a highpass filter−

Figure 4: Nonlinear function
samples is small (i.e., |x[m] − x[m − n]| ≤ ε), the input
is just filtered by the filter k, that is,NX

y[k] = x[k] −W (z) := 0.02 ·0.2929 − 0.5858z −1 + 0.2929z −2.
1 + 0.1716z −2

The Bode diagram of W (z) is shown in Fig. 6. The

k[n](x[k] − x[k − n])

n=−N

n=−N

On the other hand, if the difference is large (i.e., |x[m]−x[m − n]| > ε), we have
¯
¯
N
¯ X
¯
¯
¯
|y[m] − x[m]| = ¯k[n] · f (x[m] − x[m − n])¯ ≤ ².
¯
¯
n=−N

Consequently, ε filter smooths small-amplitude random
signals and keeps large amplitude signal components
unchanged. Therefore, without removing sharpness ofedges, this filter can suppress small-amplitude random
noises.
To make the ε filter suitable for JPEG noise reduction, we adopt the following ε filter.
y[m] =

N2
X

k[n] · fε (x[m − n], x[m]),

n=−N1

(
a,
fε (a, b) :=b,(6)|a − b| ≤ ε,|a − b| > ε,where the filter coefficients k[−N1 ], k[−N1 + 1], . . .,k[N2 ] satisfies|k[0]| ≥ |k[n]|,Magnitude (dB)0k[n]x[k − n].

−50

−100

−150

0

0.1

0.2

0.3
0.4
0.5
0.6
0.7
Normalized Frequency (×π rad/sample)

0.8

0.9

1

0

0.1

0.2

0.3
0.4
0.5
0.6
0.7
Normalized Frequency (×π rad/sample)

0.8

0.9

1

200
Phase (degrees)

=

N
X

150
100
50
0

Figure 6: Bode diagram of W (z)threshold level ε is set to be ε = 50.
The impulse response of the proposed filter is shownin Fig. 7. The impulse response is symmetrical aboutt = 6 since the filter is designed with the non-causal
hold (1). We process two images shown in Fig. 8(a)and Fig. 8(b). The images are first compressed by JPEG
compression, and then processed by the following filters:
1. sampled-data H ∞ optimal filtern = −N1 , −N1 + 1, . . . , N2 .

Fig. 5 shows this process. Consider the input shown in

2. sampled-data H ∞ optimal ε filter
3. median filter 10)

x[n]

x[n]

n

n

Figure 5: Nonlinear processing by (6)the left of Fig. 5. If |x[n] − x[n − l]| is larger than ε,- 1083 -Table 1 shows SNR (Signal-to-Noise Ratio) of processed
images. Since the image “Lena” contains few high frequency components, there is not much difference in SNR(we can see that the proposed filter is the best). However, in the case of the image “Characters” the proposed
method is much superior to the others. While medianfilters have good edge preservation properties, they remove impulses. Therefore, the median filter removes
thin lines in characters as shown in Fig. 9(a). On the

0.7
0.6

Amplitude

0.5
0.4
0.3
0.2
0.1
0
−0.1
0

2

4

6
Time [t]

8

10

12

Figure 7: Impulse response of K(z)(a) Lena

Table 1: Comparison of SNR [dB]
JPEG Compressed
H ∞ optimal linear filterH ∞ optimal ε filter
Median filter

Lena
25.6885
25.791726.0136
25.9288

Characters24.622919.9852
25.3001
16.4248other hand, Fig. 9(b) shows that our method preserves
details.

6. Conclusion
We have proposed a new method to reduce blocking andringing artifacts of JPEG images. We have introduced
the sampled-data H ∞ optimal filter, which reduces such
artifacts taking the analog characteristic of the original
image into account, combined with an ε filter to preserve high frequency details. We have also presented an
example to illustrate the effectiveness of the proposed
method.

References[1] B. Bamieh and J. B. Pearson, “A general framework for linear periodic systems with applications to
H ∞ sampled-data control,” IEEE Trans. Automat.
Contr., vol. 37, no. 4, pp. 418–435, 1992.
[2] T. Chen and B. A. Francis, Optimal Sampled-Data
Control Systems, Springer, 1995.
[3] H. Harashima, K. Odajima, Y. Shishikui and H.
Miyakawa, ε-separating nonlinear digital filter andits applications, IEICE Trans., vol. J65-A, no. 4,
pp. 297–304, 1982.

- 1084 -(b) Characters

Figure 8: Original images[4] H. Kakemizu, Digital Image Reconstruction via
Sampled-Data H ∞ Control Theory, Master Thesis,
Kyoto University, 2005. (in Japanese)[5] J. P. Keller and B. D. O. Anderson, A new approachto the discretization of continuous-time controllers,IEEE Trans. Autom. Control, AC-37, pp. 214–223,
1992.
[6] P. P. Khargonekar and Y. Yamamoto, “Delayed
signal reconstruction using sampled-data control,”in Proc. 35th IEEE Conf. Decision Control, Kobe,
pp. 1259–1263, 1996.
[7] L. Mirkin and G. Tadmor, Yet another H ∞ discretization, IEEE Trans. Autom. Control, AC-48,
pp. 891–894, 2003.

A. Formula for Mij©
M11 = γ 2 (I − Φ011 (h)Γ22 (h)) Γ−1
12 (h)× (Φ11 (h) − Γ11 (h)) − Φ011 (h)Γ21 (h)} ,M13 = γ 2 (I − Φ011 (h)Γ22 (h)) Γ−1
12 (h),¡¢−1
2
M33 = γ Φ22 (h)Φ12 (h) − Γ22 (h)Γ−1
12 (h) ,Z h/2
M21 = −C
Γ11 (t)dt
0

Z h/2
−C
0

Γ12 (t)dt · Γ−1
12 (h) (Φ11 (h) − Γ11 (h)) ,

·
¸−1
Z
£
¤ h/2
I
0
M22 = γ −2 C 0Γ(t)Γ11 (h) Γ12 (h)
0
½·
¸Z t
·
¸
I 0
0
×
Γ(−s)ds
0 0 0
−C 0
·
¸ Z h/2
·
¸)
h
0 0
0
−
dt +
Γ(h − s)ds
0
I 0 t
−C
2
Z h
Γ12 (t)dt · Γ−1
M23 = −C
12 (h),(a) Median filter0£
M24 = − γ −2 C

¤

·

Z h/2

I
0
0
Γ(t)dt
Γ
(h)
Γ
(h)
11
12
0
¸Z h
·
¸
0
0
Γ(h − s)ds
0 h/2
−C 0

·

¸−1

0
I
Z h
M41 = −C
Γ11 (t)dt
×

h/2

Z h
−C

h/2

Z h
M43 = −Ch/2

Figure 9: Filtered images[8] M. Nagahara and Y. Yamamoto, Design for digitalcommunication systems via sampled-data H ∞ control, IFAC Workshop on Periodic Control Systems,
pp. 211–216, 2001.
[9] W. Penebaker and J. Mitchell, JPEG Still Image Compression Standard, Van Nostrand Reinhold,1992.
[10] I. Pitas and A. N. Venetsanopoulos, Nonlinear Digital Filters: Principles and Applications, Kluwer
Academic Publishers, 1990.
[11] K. Zhou, J. Doyle and K. Glover, Robust and Optimal Control, Prentice Hall, 1996.

- 1085 -Γ12 (t)dt · Γ−1
12 (h),

¸−1
I
0
0
Γ(t)
Γ11 (h) Γ12 (h)
h/2
(·
¸Z t
·
¸
I 0
0
×
Γ(−s)ds
0 0 h/2
−C 0
·
¸Z h
·
¸)
h
0 0
0
−
Γ(h − s)ds
dt +
I 0 t
−C 0
2

£
M44 = γ −2 C(b) Sampled-data H ∞ optimal ε filterΓ12 (t)dt · Γ−1
12 (h) (Φ11 (h) − Γ11 (h)) ,¤Z h·