1

Nonlinear Camera Response Functions and Image
Deblurring: Theoretical Analysis and PracticeYu-Wing Tai1 Xiaogang Chen2,5 Sunyeong Kim1 Seon Joo Kim3 Feng Li4
Jie Yang2 Jingyi Yu5 Yasuyuki Matsushita6 Michael S. Brown7
1 KAIST,
2 Shanghai Jiao Tong University,
3 Yonsei University,
4 MERL,5 University of Delaware,
6 MSRA,
7 National University of Singapore

Abstract— This paper investigates the role that nonlinear
camera response functions (CRFs) have on image deblurring. We
present a comprehensive study to analyze the effects of CRFs on
motion deblurring. In particular, we show how nonlinear CRFs
can cause a spatially invariant blur to behave as a spatiallyvarying blur. We prove that such nonlinearity can cause large
errors around edges when directly applying deconvolution toa motion blurred image without CRF correction. These errors
are inevitable even with a known point spread function (PSF)
and with state-of-the-art regularization based deconvolution algorithms. In addition, we show how CRFs can adversely affect
PSF estimation algorithms in the case of blind deconvolution. To
help counter these effects, we introduce two methods to estimate
the CRF directly from one or more blurred images when the PSF
is known or unknown. Our experimental results on synthetic andreal images validate our analysis and demonstrate the robustness
and accuracy of our approaches.
Index Terms— Nonlinear Camera Response Functions (CRFs),
Motion Deblurring, CRF Estimation

I. I NTRODUCTION
Image deblurring is a long standing computer vision problem
for which the goal is to recover a sharp image from a blurred
image. Mathematically, the problem is formulated as:B=

I ⊗ K + n,(1)

where B is the captured blurred image, I is the latent image, K
is the point spread function (PSF), ⊗ is the convolution operator,and n represents image noise.
One common assumption in most previous algorithms that isoften overlooked is that the image B in Equation (1) responds
in a linear fashion with respect to irradiance, i.e., the final image
intensity is proportional to the amount of light received by thesensor. This assumption is valid when we capture an image in a
RAW format. However, when capturing an image using a common
consumer level camera or camera on a mobile device, there is anonlinear camera response function(CRF) that maps the scene
irradiance to intensity. CRFs vary among different camera manufacturers and models due to design factors such as compressing
the scene’s dynamic range or to simulate conventional irradianceresponses of film [15], [33]. Taking this nonlinear response into
account, the imaging process of Equation (1) can be considered
as:
B

=

f (I ⊗ K + n),(2)

where f (·) is the CRF. To remove the effect of the nonlinear CRF
from image deblurring, the image B has to be first linearized
by the inverse CRF, i.e., f −1 . After deconvolution, the CRF is

applied again to restore the original intensity which leads to thefollowing process:I=f (f −1 (B) ⊗ K −1 ),(3)where K −1 is inverse filter which denotes the deconvolution
process.
Contributions This paper offers two contributions with regards
to CRFs and their role in image deblurring. First, we providea systematic analysis of the effect that a CRF has on theblurring process and show how a nonlinear CRF can make aspatially invariant blur behave as a spatially varying blur aroundedges. We prove that such nonlinearity can cause abrupt ringing
artifacts around edges which are non-uniform. In addition, these
artifacts are difficult to eliminated by regularization. Our analysis
also shows that PSF estimation for various blind deconvolution
algorithms are adversely affected by the nonlinear CRF.
Along with the theoretical analysis, we further introduce twoalgorithms to estimate the CRF from one or more images: thefirst method is based on a least-square formation when the PSF is
known; the second method is formulated as a rank minimization
problem when the PSF is unknown. Both of these approaches
exploit the relationship between the blur profile about edges ina linearized image and the PSF. While our estimation methods
cannot compete with well-defined radiometric calibration methods
based on calibration patterns or multiple exposures, they areuseful to produce a sufficiently accurate CRF for improving
deblurring results.
Shorter versions of this work appeared in [22], [7]. This paper
unifies these two concurrent independent works with more indepth discussion and analysis of the role of CRF in deblurring, and additional experiments. In addition, a new sectionthat analyzes the effectiveness of gamma curve correction in
traditional deblurring methods, and the limitations of the proposed
algorithms are presented in Section VII-A.
The remainder of our paper is organized as follow: In Section II, we review related works in motion deblurring and CRF
estimation. Our theoretical analysis about the blur inconsistency
introduced by a nonlinear CRF is presented in Section III, followed by the analysis on the deconvolution artifacts and PSF estimation errors in Section IV. Our two algorithms which estimate
the nonlinear CRF from motion blurred image(s) with known
and unknown PSF are presented in Section V. In Section VI,
we present our experimental results. Section VII-A provides
additional discussion about the effectiveness of gamma curve
correction. Finally, we conclude our work in Section VIII.

2

II. R ELATED WORK
Image deblurring is a classic problem with well-studied approaches including Richardson-Lucy [35], [40] and Wiener deconvolution [48]. Recently, several different directions were introduced to enhance the performance of deblurring. These include
methods that use image statistics [14], [29], [18], [6], sparsity
or sharp edge prediction [30], [20], [8], [2], [50], [26], [45],multiple images or hybrid imaging systems [1], [39], [9], [51],[5], [42], [43], and new blur models that account for camera
motion [44], [46], [47], [19], [17]. The vast majority of thesemethods, however, do not consider the nonlinearity in the imaging
process due to CRFs.
The goal of radiometric calibration is to compute a CRF
from a given set of images or a single image. The most accurate radiometric calibration algorithms use multiple images withdifferent exposures [12], [4], [37], [13], [15], [21], [24], [34],[28]. Our work is more related to single-image based radiometriccalibration techniques [32], [33], [38], [36], [49]. In [32], the CRF
is computed by observing the color distributions of local edge
regions: the CRF is computed as the mapping that transforms
nonlinear distributions of edge colors into linear distributions.
This idea is further extended to deal with a single gray-scale
image using histograms of edge regions in [33]. In [49], a CRF
is estimated by temporally mixing of a step edge within a single
camera exposure by the linear motion blur of a calibration pattern.
Unlike [49], however, our method deals with uncontrolled blurred
images.
To the best of our knowledge, there are only a handful ofprevious works that consider CRFs in the context of image
deblurring. Examples include the work by Fergus et al. [14],where images are first linearized by an inverse gamma-correction
with γ = 2.2. Real world CRFs, however, are often drasticallydifferent from gamma curves [15], [31]. Another example by Lu etal. [34] involves reconstructing a high dynamic range image froma set of differently exposed and possibly motion blurred images.
Work by Cho et al. [10] discussed nonlinear CRFs as a cause
for artifacts in deblurring, but provided little insight into whysuch artifacts arise. Their work suggested to avoid this by using
a pre-calibrated CRF or the camera’s RAW output. While a precalibrated CRF is undoubtedly the optimal solution, the CRF may
not always be available. Moreover, work by Chakrabarti et al. [3]suggests that a CRF may be scene dependent when the camera isin “ auto mode”. Furthermore, work by Kim et al. [23] showedthat the CRF for a given camera may vary for different camera
picture styles (e.g., landscape, portrait, etc).
Our work aims to provide more insight into the effect that
CRFs have on the image deblurring process. In addition, we seek
to provide a method to estimate a CRF from a blurred input image
in the face of a missing or unreliable pre-calibrated CRF.
III. B LUR INCONSISTENCY DUE TO NONLINEAR CRF
We first study the role of the CRF in the image blurring process.
We denote by f as the nonlinear CRF, f −1 as the inverse of f ,I as the blur-free intensity image, I˜ as the irradiance image of
˜ , B as the observed motion blurred image with
I , i.e., I = f (I)B = f (I˜ ⊗ K + n). To focus our analysis, we follow previous
work by assuming that the PSF is spatially invariant and the image
noise is negligible (i.e., n ≈ 0).

We analyze the blur inconsistency introduced by a nonlinear
CRF by measuring:
b
Γ=B−B
(4)

b = I ⊗ K which denotes the conventional intensity based
where Bb. Our goal
blurring process. Note that I is used instead of I˜ in B
in here is to understand where the intensity based convolution
model would introduce errors when CRF correction is excluded.
Claim 1. In uniform intensity regions, Γ = 0 .
Proof: Since pixels within the blur kernel region have uniform
intensity, we have f −1 (I) ⊗ K = f −1 (I) = f −1 (I ⊗ K).
Therefore,b
B = f (f −1 (I) ⊗ K) = f (f −1 (I ⊗ K)) = B,(5)

f −1 (I + ∆I) ⊗ K ≈ f −1 (I) ⊗ K + (f ′−1 (I) · ∆I) ⊗ K,

(6)

thus Γ = 0.
Claim 1 applies to any CRF (both linear or nonlinear). Thisimplies that a nonlinear CRF will not affect deblurring quality
for uniform intensity regions.
Claim 2. If the blur kernel K is small and the CRF f issmooth, Γ ≈ 0 in low frequency regions .
Proof: Let I = I + ∆I be a local patch covered by the blurkernel K . The term I is the average intensity within the patchand ∆I is the deviation from I . In low frequency regions, ∆I issmall.
Next, we apply the first-order Taylor series expansion tof −1 (I) ⊗ K as:

where f ′−1 is the first order derivative of f −1 . Since I is uniform,
we have f −1 (I) ⊗ K = f −1 (I) and f ′−1 (I) is constant in thelocal neighborhood. Thus, Equation (6) can be approximated as:
f −1 (I) + f ′−1 (I) · ∆I ⊗ K.

(7)

Similarly, by using the first-order Taylor series expansion, wehave
f −1 (I ⊗ K)=

f −1 (I ⊗ K + ∆I ⊗ K)≈

f −1 (I ⊗ K) + f ′−1 (I ⊗ K) · (∆I ⊗ K)(8)=f −1 (I) + f ′−1 (I) · ∆I ⊗ K.

(9)

Therefore,b
B = f (f −1 (I) ⊗ K) ≈ f (f −1 (I ⊗ K)) = B,

(10)i.e., Γ ≈ 0.
Claim 2 holds only for small kernels. When the kernel sizeis large, e.g.80 × 80, the first-order Taylor series expansion is
not accurate. To illustrate this property, we simulate a 1D smoothsignal in Figure 1(a). A uniform motion blur kernel with different
size (10, 20, 30 and 80 pixels) is applied to the 1D signal. Web with f −1 being a gamma curve (γ = 2.2).
compute B and B
The plotting of Γ along the 1D signal with different sized K s areshown in Figure 1(a). Note that Γ ≈ 0 for the first three kernels.
Claim 3. Γ can be large at high frequency high contrast
regions.
Proof: Let us consider a sharp edge in I represented by a Step
Edge Function,
I(x) =

0,1,

if x < 0.5
otherwise

(11)

3

smooth signal
Γ(length=10)
Γ(length=20)
Γ(length=30)
Γ(length=80)

0.3

1.0

0

sharp signal
Γ(length=10)
Γ(length=20)
Γ(length=30)
Γ(length=80)

0

0

300

(a)

(b)

0

(c)

500

intensity

Fig. 1. Illustration of blur inconsistency Γ. A gamma curve with γ = 2.2 is used to simulate f −1 across all sub-figures. (a): Γ is computed along a
1D smooth signal with four different uniform motion blur kernels of size 10, 20, 30 and 80 pixels. (b) The first two rows show the latent pattern and thecorresponding irradiance-based motion blurred pattern respectively. The bottom row shows the measured Γ which varies across different edge strength. (c)Plotting of Γ in (b) with different kernel sizes.

0.9

from f −1 (I) since K contains only zero or positive values. If
f −1 is convex, we can use the Jensen’s inequality to obtain

0.75

f −1 (I) ⊗ K ≥ f −1 (I ⊗ K).

Further, since the CRF f is a monotonically increasing [12], [37],
we have:

0.6
0.45
0.3

0

0.15

0.3

0.45 0.6
irradiance

0.75

0.9

Fig. 2. The left panel shows 188 CRF curves of real cameras from DoRF
[15]. Nearly all curves appear concave.

Since f −1 has boundary f −1 (0) = 0 and f −1 (1) = 1 [15], wehave f −1 (I) = I . Therefore,
B = f (f−1b(I) ⊗ K) = f (I ⊗ K) = f (B),b
B = f (f −1 (I) ⊗ K) ≥ f (f −1 (I ⊗ K)) = B,

(14)

B = f (f −1 (I) ⊗ K) ≤ f (f −1 (Imax )) = Imax .

(15)i.e., Γ ≥ 0.
Next, we derive the upper bound of Γ. Since I ⊗K ≤ Imax andf −1 is monotonically increasing (as f −1 is inverse of f and f ismonotonically increasing), f −1 (I) ⊗ K ≤ f −1 (Imax ). Therefore,
we have,

0.15
0

(13)

(12)

b −B
b.
Hence, Γ = f (B)In this example, Γ(x) measures how f (x) deviates from thelinear function y(x) = x. In other words, Γ(x) → 0, iff f (x) → x.
However, practical CRFs f are highly nonlinear [15], [23].
To validate Claim 3, we simulate a 1D signal of sharp edges
with different gradient magnitudes and measure the blur inconsistency Γ as shown in Figure 1(b). The first row is the original clear
signal. The second row is the blur signal B where the blur kernel
size is 20 pixels. The third row shows the Γ which is non-zeroaround edge regions and the Γ increase as the gradient magnitudes
get larger. To further analyze the effect of kernel size on sharp
signals, we plot Γ with four different sized kernels in Figure 1(c).
Notice that the area of non-zero Γ increase with kernel size since
the blurry edge gets larger. Also, the magnitude of Γ depends onedge magnitude but it does not depends on kernel size.
Theorem 1. Let Imin and Imax be the local minimum andmaximum pixel intensities in a local neighborhood covered bykernel K in image I . If f −1 is convex, then the Blur Inconsistency
is bounded by 0 ≤ Γ ≤ Imax − Imin .
Proof: Consider f −1 (I) ⊗ K as a convex combination of pixels

b = I ⊗ K ≥ Imin . Combining thisLikewise, we can also derive Bb ≤with Equation (14) and Equation (15), we have: Imin ≤ B
B ≤ Imax . Therefore,b ≤ Imax − Imin .
0 ≤ Γ ≤ Imax − B(16)Theorem 1 explains the phenomenon in Figure 1: when thegradient magnitude of an edge is large, the upper-bound of Γ
will be large. On the other hand, in low contrast regions, theupper bound Imax − Imin is small and so is Γ. Thus, B can beb.
well approximated by B
It is important to note that Theorem 1 assumes a convex inverse
CRF f −1 . This property has been observed in many previous
results. For example, the widely used Gamma curves f −1 (x) =xγ , γ > 1, are convex. To better illustrate the convexity of f −1(or equally the concavity of f ), we plot 188 real camera CRF
curves (f ) collected in [15] in Figure 2. We compute the discrete
second-order derivatives of the 188 real camera CRF curves. Our
experiment shows that the majority (84.4%) of sample points arenegative and therefore the inverse CRF f −1 is largely convex.
Combining Claim 1, 2, 3, and theory 1, we see that the blurinconsistency introduced by nonlinear CRF mainly appears inhigh contrast high frequency regions in original signal. These
regions, however, are the regions that mostly cause ringing
artifacts in image deblurring. Next, we will analysis the effects
of CRF and how it affects the quality of image deblurring.

4Linear CRF (kernel A)

Nonlinear CRF (kernel A)

10Linear CRF (kernel B)

1

0

50

100

150

200

Nonlinear CRF (kernel B)

1

1

PSF

PSF

PSF

PSF

PSF

PSF

PSF

PSF

250

300

0

0

50

100

(a)

150

200

250

300

0

0

50

100

(b)

150

200

250

300

0

0

50

100

(c)

150

200

250

300

200

250

300

200

250

300

(d)

Wiener Filter Result

Wiener Filter Result

1

1

1

1
0

0

−1
0

−1

0
0

50

100

150

200

250

300

0

50

100

(e)

150

200

250

300

0

50

100

(f)

150

200

300

0

50

100

(g)

150

(h)

With Regularization

With Regularization

1

1

250

1
1

0
0

0
0

50

100

150

200

250

300

0

50

100

(i)

150

200

250

300

0
0

50

100

(j)

150

200

250

300

0

50

100

(k)

150

(l)

Fig. 3. This figure shows a simple step edge image that has been blurred by two different PSF kernels: a 1D motion PSF with uniform speed (kernel A)
and a 1D PSF with non-uniform speed (kernel B). These blurred images are transformed with a linear CRF (a)(c), and with a nonlinear CRF (b)(d). The 1D
PSF and the 1D slice of intensity values are also shown. (e)-(h) show non-blind deconvolution results of (a)-(d) using Wiener filter. (i)-(l) show non-blind
deconvolution results of (a)-(d) using an iterative re-weighting method [30] with sparsity regularization.

1

ge
ed

After CRF
Re-mapping

ge
ed
0

1

2

Before CRF
Re-mapping

0

1

Fig. 4. This figure illustrates the problem when the image intensities are
re-mapped according to different part of the camera response function. Two
identical edge profiles at different intensity ranges are re-mapped to havedifferent shapes.

IV. E FFECTS OF CRF IN IMAGE DEBLURRING
A. Effects on PSF and image deconvolution
We begin our analysis using synthetic examples shown in
Figure 3. An image with three different intensity levels (Black,Gray, and White) is blurred with 1D motion PSFs with uniform
speed (kernel A) and non-uniform speed (kernel B) to generate theobservations in Figure 3 (a)(b) and Figure 3 (c)(d) respectively.
A real CRF 1 is used to nonlinearly re-map the image intensity
in Figure 3 (b)(d) after convolution.
In this example, we can observe how the blur profiles of animage becomes spatially varying after the nonlinear re-mapping
1 Pre-calibrated CRF of a Cannon G5 camera in standard mode is used.

of image intensities even though the underlying motion PSF is
spatially invariant. With a linear CRF, the blur profile from theblack to gray region (first step edge) is the same as the blur profile
from the gray to white region (second step edge) as shown in
Figure 3 (a)(c). However, with a nonlinear CRF, the two blur
profiles become different as shown in Figure 3 (b)(d). This isbecause the slope and the curvature of the CRF are different fordifferent range of intensities. Figure 4 helps to illustrate this effect.
To examine the effect that these cases have on image deblurring, we performed non-blind deconvolution on the synthetic
images as shown in the second and third rows of Figure 3. The
second row shows the results of Wiener filtering [48] while thethird row shows the results of [30] with sparsity regularization.
Without using any regularization, the deblurring results of boththe linear and nonlinear CRF contain ringing artifacts due to thezero component of the PSF in the frequency domain. However, themagnitude of these ringing artifacts in the nonlinear CRF result
is significantly larger than the one with the linear CRF. The useof image regularization [30] helps to reduce the ringing artifacts,
but the regularization is less effective in the case of a nonlinear
CRF even with a very large regularization weight.
In addition, if we assume I is a step edge and f is a Gammabfunction, f (x) = xγ with γ < 1, we can re-write B = f (B)(Equation (12)) using Taylor series expansion:
where P =

∞
P

κ=1
κ

bγ = Bb + P,
B=B(γ−1)κ bb κ.
B(ln B)κ!

(17)

An important property offunction x(ln x) is that it approaches zero for x → 0+ or x → 1.

5

θ

+

+
P

PSF

B ⊗ K −1

⊗ K −1B̂

B

marginal
probability⊗ K −1

Fig. 5. Given an invertible filter, the ringing artifacts of a step edge is causedby the deconvolution of blur inconsistency P introduced by nonlinear CRF
in Equation (18).

CDF

θ

edge
profileFig. 7. This figure shows how the edge profile of a step edge in a motion
blurred image is equal to the cumulative distribution function of the marginal
probability of the motion blur kernel along the direction perpendicular to theedge.

Fergus et al. [14]Shan et al. [41]Xu and Jia [50]Cho et al. [11]Fig. 6. This figure shows examples of PSF kernels from several images withlinear CRF (left columns) or nonlinear CRF (right columns). PSF kernels inthe same row are from the same image with different estimated methods:
Fergus et al. [14]; Shan et al. [41]; Xu and Jia [50]; Cho et al. [11].

Therefore, P only has non-zero values in the blurred edge regions.
Assuming an invertible filter K which has non-zero entries infrequency domain, we can represent the deconvolution of B withK by B ⊗ K −1 , and thusb ⊗ K −1 + P ⊗ K −1 ,B ⊗ K −1 = B(18)

b ⊗ K −1 = I (Equation (4)). Thus, P ⊗ K −1 is thewhere Bdeconvolution artifacts introduced by the Blur Inconsistency Γ.
Notice that even the blur inconsistency appears only within
the edge areas, the deconvolution process can propagate theblur inconsistency errors from edge regions to smooth regions.
Figure 5 illustrates the effects of decomposition in Equation (18).
Again, the CRF plays a significant role in quality of the image
deblurring.
B. Effects on PSF estimation
We also analyze the effects of a CRF on the PSF estimationby comparing the estimated PSF between a linear and nonlinear
CRF. We show the estimated PSFs from Fergus et al. [14], Shanet al. [41], Xu and Jia [50] and Cho et al. [11] in Figure 6. The
raw images from [11] were used for our testing purpose.
Ideally, the CRF should only affect the relative intensity of a
PSF, but the shape of a PSF should remain the same since theshape of the PSF describes the trajectory of motion causing the
motion blur, and the intensity of the PSF describes the relative
speed of the motion. However, as we can observe in Figure 6,
the estimated PSFs are noticeably different, especially for the
results from [14] and [41]. This is because both [14] and [41]use alternating optimization for their non-blind deconvolution.
As previously discussed, the nonlinear CRF causes errors during
the deconvolution process which in turn propagates the errors

to the estimated PSF in an alternating optimization framework.
The results from [50] and [11] contains less errors because they
separate the process of PSF estimation and image deconvolution.
However, the shape of the estimated PSF are still different. The
method in [50] requires edge selection and sharpening, and themethod in [11] requires edge profiles to be aligned. Since thenonlinear CRF alters the edge profiles in the blurred image, theirestimated PSF also contains errors inherent from non-uniform
edge profiles. Note that small errors in the PSF estimation cancauses significant artifacts in subsequent deconvolution.
V. CRF ESTIMATION FROM A BLURRED IMAGE
In this section, we describe a method to estimate the CRF
from one or more blurred images. To begin, we examine therelationship between the PSF and edge profiles in a blurred image
under a linear CRF assumption. We then describe a method toestimate the CRF based on this relationship using least-squares
fitting assuming a known PSF. This method is then converted toa robust estimation of an unknown PSF and a CRF using rank
minimization.
A. PSF and edge profiles
We begin with the observation that the shape of a blur profile
with the linear CRF resembles the shape of the cumulative
distribution of the 1D PSF as shown in Figure 3 (a)(c). Our
analysis is similar in fashion to that in [18] which used alphamattes of blurred object to estimate the PSF. Our approach,however, works directly from the image intensities and requires
no matting or object extraction. Instead, we only need to identify
the step edges with homogeneous areas on both sides. For suchedges, the shape of the blur profile is equal to the shape of the
cumulative distribution of the 1D PSF.
Theorem 2. Under a 1D motion blur, if the CRF is linear
and the original edge in clear image is a step edge with uniform
intensity on both sides, the shape of the blur profile is equal to
the shape of the cumulative distribution of the 1D PSF.
Proof: Consider a simple case where the original step edge
has values [0, . . . , 0, 1, . . . , 1] and the values of the PSF is[α1 , α2 , . . . , αM ]. If the number of 0’s and 1’s are both larger
than M , the blurPprofile after the motion blur is equal toM[α1 , α1 + α2 , . . . , i=1 αi ], which is the cumulative distributionof the 1D PSF.
For any edges with intensities [I1 , I2 ], the value of the blur
profilePm at m ∈ [1, . . . , M ] after the blur is equal to I1 +i=1 αi (I2 − I1 ).

6Theorem 2 is valid under the assumptions that the motion
blurred image does not contain any noise or quantization error.
In the case of a 2D PSF, Theorem 2 still holds when the edge
is a straight line. In this case, the 1D PSF becomes the marginal
probability of the 2D PSF projected onto the line perpendicular
to the edge direction as illustrated in Figure 7.
B. CRF approximation with a known PSF
Considering that the shape of blurred edge profiles are equal to
the shape of the cumulative distribution of the PSF2 if the CRF
is linear. Given that we know the PSF, we can compute the CRF
as follows:arg min
g(·)

+

E1 X
M
X

m

g(Ij (m)) − lj X
−
αi
wj

wj

j=1 m=1

E2 X
M
X

i=1
M

!2

X
g(Ij (m)) − lj
−
αi
wj

wj

j=1 m=1i=m!2, (19)

where g(·) = f −1 (·) is the inverse CRF function, E1 and E2 are
the numbers of selected blurred edge profiles from dark to brightregions and from bright region to dark regions, respectively.
The variables lj and wj are the minimum intensity value andthe intensity range (intensity difference between the maximum
and the minimum intensity values) of the blurred edge profiles
after applying the inverse CRF. Blur profiles that span a wider
intensity range are weighted more because their wider dynamic
range covers a larger portion of g(·), and therefore provide more
information about the shape of g(·). We filtered out the edges
with wj < 0.1 since according to our blur inconsistency analysis
in Section III, these low contrast edges does not provide muchinformation for PSF estimation.
We follow the method in [49] and model the inverse CRF
g(·) using a polynomial of degree d = 5 with coefficients ap ,Pdi.e., g(I) = p=0 ap I p . The optimization is subject to boundary
constraints g(0) = 0 and g(1) = 1, and a monotonicity constraintthat enforces the first derivative of g(·) to be non-negative. Our
goal is to find the coefficients ap such that the following objectivefunction is minimized:arg min
ap

+

E1 X
M
X

wj

j=1 m=1

E2 X
M
X

wj

j=1 m=1+ λ1 a20 +
+ λ2

L
X
r=1

H

Pd

p
p=0 ap Ij (m) − ljPdwj

p=0 ap Ij (m)

d
X
p=0

d
X
p=0

p

− lj

wj

 r − 1 p
L

−

αi

i=1

M
X

−

!2

αi

i=m

!2 
ap − 1 

ap

m
X

−

!
 r p L,10, which control the boundary constraint and the monotonic

constraint, respectively. The solution of Equation (20) can beobtained by a simplex search method of Lagarias et al. [27]3 .
C. CRF estimation with unknown PSF
Using the cumulative distribution of the PSF can reliably
estimate the CRF under ideal conditions. However, the PSF isusually unknown in practice. As we have studied in Section IV-B,
nonlinear CRF affects the accuracy of the PSF estimation, which
in turn will affect our CRF estimation described in Section V-B.
In this section, we introduce a CRF estimation method without
explicitly computing the PSF.
As previously discussed, we want to find an inverse response
function g(·) that makes the blurred edge profiles have the same
shape after applying the inverse CRF. This can be achieved byminimizing the distance between each blur profile to the average
blur profile:arg min
g(·)

+

E1 X
M
X

wj

j=1 m=1

E2 X
M
X

wj

j=1 m=1

g(Ij (m)) − lj
− A1 (m)wj
g(Ij (m)) − lj
− A2 (m)wj2
2, (21)PEwk
1
g(Ik (m)) is the weighted average blurwhere A1 (m) = k=1
P E1 W
profile, and W = l=1 wl is a normalization factor.
Using the constraint in Equation (21), we can compute the
CRF, however, this approach is unreliable not only because theconstraint in Equation (21) is weaker than the constraint inEquation (19), but the nature of least-squares fitting is sensitive
to outliers. To avoid these problems, we generalize our method
to robust estimation via rank minimization.
Recall that the edge profiles should have the same shape after
applying the inverse CRF. This means that if the CRF is linear, theedge profiles are linearly dependent with each other, and hence
the observation matrix of edge profiles form a rank-1 matrix foreach group of edge profiles:g(M)= g(I1 (1)) − l1···g(I1 (M )) − l1

..
.

..

.

..
.

g(IE1 (1)) − lE1···

g(IE1 (M )) − lE1 , (22)

where M is length of edge profiles, and E1 is the number ofobserved edge profiles grouped according to the orientation ofedges. Now, we transform the problem into a rank minimization
problem which finds a function g(·) that minimizes the rank of
the observation matrix M of edge profiles. Since the CRF is thesame for the whole image, we define our objective function forrank minimization as follow:

!2

(20)

where H is the Heviside step function for enforcing the monotonicity constraint, i.e., H = 1 if g(r) < g(r − 1), or H = 0
otherwise and L is the maximum intensity level, e.g.255 for 8bit color depth. The weights are fixed to λ1 = 100 and λ2 =2 For simplicity, we assume that the PSF is 1D, and it is well aligned with theedge orientation. If the PSF is 2D, we can compute the marginal probabilityof the PSF.

arg ming(·)

KX

wk rank(g(Mk )),

(23)

k=1

where K is total number of observation matrix (total number ofgroup of edge profiles), wk is a weight given to each observation
matrix. We assign larger weight to the observation matrix that
contains more edge profiles. Note that Equation (23) is also
applicable to multiple images since the observation matrix is built
individually for each edge orientation and for each input image.
3 fminsearch function in Matlab.

7

1

1

1

PSF

0

0

50

100

150

200

250

300

0

0

50

100

150

(a)

200

250

300

0

0

50

100

150

200

250

PSF

300

0.8

0

50

100

150

200

250

PSF

300

0.8

0

0.8

0.8

0.7

0.7

0.7

0.7

0.6

0.6

0.6

0.6

0.5

0.5

0.5

0.5

0.5

0.4

0.4

0.4

0.4

0.4

0.3

0.3

0.3

0.3

0.3

0.2

0.2

0.2

0.2

0.2

0.1

0.1

0.1

0.1

0.2

0.4

0.6

0.8

1

0

0

0.2

0.4

(f)

0.6

0.8

1

0

0

0.2

(g)

0.4

0.6

(h)

150

0.8

1

0

200

250

300

0.8

1

Ground Truth
With known PSF (Eq. 4)
With Observation Matrix (Eq. 6) &Rank Minimization (Eq. 10)

0.9

0.6

0

100

1

Ground Truth
With known PSF (Eq. 4)
With Observation Matrix (Eq. 6) &Rank Minimization (Eq. 10)

0.9

0.7

0

50

(e)

1

Ground Truth
With known PSF (Eq. 4)
With Observation Matrix (Eq. 6) &Rank Minimization (Eq. 10)

0.9

0

(d)

1

Ground Truth
With known PSF (Eq. 4)
With Observation Matrix (Eq. 6) &Rank Minimization (Eq. 10)

0.9

0

(c)

1

Ground Truth
With known PSF (Eq. 4)
With Observation Matrix (Eq. 6) &Rank Minimization (Eq. 10)

0.8

1

PSF

(b)

1
0.9

1

PSF

0.1
0

0.2

0.4

0.6

(i)

0.8

1

0

0

0.2

0.4

0.6

(j)

Fig. 8. We test the robustness of our CRF estimation method under different configurations. (a) Blur profiles with different intensity ranges, (b) edges in theoriginal image contains mixed intensities (edge width is equal to 3 pixels), (c) Gaussian noise (σ = 0.02) is added according to Equation (2), (d) the unionof intensity range of all blur profiles does not cover the whole CRF curve. (e) Blur profiles of (a), (b), (c), (d). The original image (black lines), the blurred
image with linear CRF (red lines), and the blurred image with nonlinear CRF (blue lines) are shown on top of each figure in (a)-(d). (f)-(i) the corresponding
estimated inverse CRF using our methods with (a)-(d). (j) the corresponding estimated inverse CRF with multiple images in (e).

We evaluate the rank of matrix M by measuring the ratio of
its singular values:arg min
g(·)

K
X

wk

Ek
X
σkj
j=2

k=1

σk1

(24)

,

where σkj are the singular values of g(M)k . If the observation
matrix is rank-1, only the first singular values is nonzero andhence minimizes Equation (24). In our experiments, we foundthat Equation (24) can be simplified to just measuring the ratioof the first two singular values:arg min
g(·)

K
X

wk

k=1

σk2
.
σk1

(25)

Combining the monotonic constraint, the boundary constraint
and the polynomial function constraint from Equation (20), we
obtain our final objective function:arg min
ap

K
X

k=1

+ λ2



σ
wk k2 + λ1 a20 +
σk1

L
X
r=1

H

d
X

ap

p=0



d
X

!2 

ap − 1

p=0

r−1
L

p

−



 p 
r
L

!

(26)

.

Equation (26) can be solved effectively using nonlinear leastsquares fitting4 .
D. Implementation issues for 2D PSF
Our two proposed methods for CRF estimation are based on the
1D blur profile analysis. Since, in practice, PSFs are 2D in nature,
we need to group image edges with similar orientation and selectvalid edge samples for building the observation matrix. We use
the method in [11] to select the blurred edges. The work in [11]filtered edge candidates by keeping only high contrast straight-line
edges. A user parameter controls the minimum length of straightline edges which depends on the size of the PSF. We refer to [11]4 lsqnonlin function in Matlab.

Fig. 9. Our selected edges for CRF estimation. Edges are grouped according
to the edge orientation (color coded). The estimated CRF and the deblurred
images are shown in Figure 12.

for details of selecting high contrast straight-line edges in blurry
images.
After selecting valid edge profiles, they are grouped according
to edge orientations. Figure 9 shows examples of the selectededges and the grouping results. The selected edges were groupedby selecting partitioning thresholds such that the orientation
variation within each group is less than 2 degree. To increase thenumber of candidate edges within each group, edges in opposite
direction were grouped together. When building the observation
matrix in Equation (22), we reverse the CDF of the opposite
edges by computing {1 − [g(Ij (1)) − lj ] · · · 1 − [g(Ij (M )) − lj ]}.
To make the selected edge profiles more robust against noise andoutliers, we apply a 1D directional filter in a direction orthogonal
to the edge orientation to get the local average of edge profiles.
We found that this directional filtering improved the robustnessof our edge selection algorithm even when the candidate edges
were slightly curved.
After edge selection and grouping, edge profiles are aligned tobuild the observation matrix. In [11], edge profiles were aligned
according to the center of mass. In our case, however, due to thenonlinear CRF effects, alignment based on the center of mass is
not reliable. We instead align the two end points of the edges.
In cases where the projected 1D PSF contains discontinuities, thestarting and end points of the discontinuities are also consideredin the alignment. Since the amount of blur in different directions

8

(a)

1

Our estimated curve
Chart values
Curve fitted by Chart values
Inverse Gamma−Correction

0.6
0.4
0.2

1

0.4
0.6
image intensity

0.8

0.6
0.4

1

Canon 400D

00

Our estimated curve
Chart values
Curve fitted by Chart values
Inverse Gamma−Correction

0.8
0.6
0.4
0.2

0.2

0.2

ISO: 200
T: 1/10 s(f)

Our estimated curve
Chart values
Curve fitted by Chart values
Inverse Gamma−Correction

0.8
irradiance

0.8

(e)

ISO: 3200
T: 1/125 s

ISO: 200
T: 1/5 sirradiance1(d)irradiance

ISO: 3200
T: 1/80 s

ISO: 100
T: 1/4 s

ISO: 100
T: 1/4 s

0
0

(c)

(b)

0.2

0.4
0.6
image intensity

Canon 60D

0.8

1

0
0

0.2

0.4
0.6
image intensity

0.8

1

Nikon D3100

Fig. 10. CRF estimation on real images. The top row shows the sharp/blurred pairs. The bottom row shows our recovered the CRF (in blue) and the groundtruth CRF (in red) obtained by acquiring the MacBeth’s chart (in green).

varies according to the shape of the PSF, we give a larger weightto the directions with longer edge profiles as they provide more
information about the CRF than sharp edges. When dealing withmultiple images, the same weighting scheme applies where alarger weight is given to a more blurry edge profile.
Our rank minimization using nonlinear least-squares fitting issensitive to the initial estimation of the CRF. In our implementation, we use the average CRF profile from the database of
response functions (DoRF) created by Grossberg and Nayar [16]as our initial guess. The DoRF database contains 201 measured
functions which allows us to obtain a good local minima inpractice.
VI. E XPERIMENTAL RESULTS
In this section, we evaluate the performance of our CRF
estimation method using both synthetic and real examples. In
the synthetic examples, we test our algorithm under different
conditions in order to better understand the behaviors and thelimitations of our method. In the real examples, we evaluate ourmethod by comparing the amount of ringing artifacts with andwithout CRF intensity correction to demonstrate the effectivenessof our algorithm and the importance of CRF in the context ofimage deblurring.
A. Synthetic examples
Figure 8 shows the performance of our CRF estimation under
different conditions. We first test the effects of the intensity rangeof the blur profiles in Figure 8 (a). In a real application, it isuncommon that all edges will have a similar intensity range. These

intensity range variations can potentially affect the estimated CRF
as low dynamic range edges usually contain larger quantization
errors. As shown in Figure 8 (f), our method is reasonably robust
to these intensity range variations.
Our method assumes that the original edges are step edges.
In practice, there may be color mixing effect even for an edgethat is considered as a sharp edge. In our experiments, we findthat the performance of our approach degrades quickly if the stepedge assumption is violated. However, as shown in Figure 8 (g),our approach is still effective if the color mixing effects is lessthan 3 pixel wide given a PSF with size 15. The robustness of
our method when edge color mixing is present depends on thesize of the PSF with our approach being more effective for larger
PSFs.
Noise is inevitable even when the ISO of a camera is high.
We test the robustness of our method against image noise in
Figure 8 (c). We add Gaussian noise to Equation (2) where
the noise is added after the convolution process but before the
CRF mapping. As can be observed in Figure 8 (h), the noise
affects the accuracy of our method. In fact, using the model inEquation (2), we can observe that the noise has also captured
some characteristics of the CRF. The magnitude of noise in thedarker region is larger than the magnitude of noise in the brighter
region. Such information may even be useful and combined into
our framework to improve the performance of our algorithm as
discussed in [36].
We test the sensitivity of our algorithm when the union of blur
profiles does not cover the whole range of CRF. As can be seenin Figure 8 (i), our method still gives reasonable estimations. Thisis because the polynomial and monotonicity constraint assist in9Linear CRF

Gamma 2.2

Input

[ Krishnan and Fergus NIPS 2009 ]Ground Truth[ Levin et al. Siggraph 2007 ]Our CRF[ Xu and Jia ECCV 2010 ]Fig. 11. Qualitative comparison on non-blind deconvolution results with linear CRF, gamma curve, and our estimated CRF correction. We show the results
for three different state-of-the-art regularization based deconvolution method: Krishnan and Fergus [25], Levin et al. [30], and Xu and Jia [50]. The estimated
CRF is shown in Figure 10. Among all of the compared results, our estimated CRF correction consistently performs better than gamma curve correction.

maintaining the shape of CRF. Note that having limited range
of intensities will degrade the performance of all radiometriccalibration methods.
Finally, we show a result where we use all input images toestimate the CRF. As expected, more input images give a moreaccurate CRF estimation.
The synthetic experiments show that the performance of our
method depends on not only the quantity of the observed blur
profiles but also the quality of the profiles. For instance, a blur
profile with a large motion covering a wide intensity range isbetter than the combination of blur profiles that cover only aportion of intensity range. When combining the observations frommultiple images, our estimated CRF is more accurate as themutual information from different images increases the robustnessof our algorithm. The rank minimization also makes our algorithm
less sensitive to outliers.
B. Real examples
1) With Known PSF: To test our algorithm with a known PSF
in real world examples, we implemented the method from [51]which uses a pair of blurry/sharp image to estimate a reliablemotion PSF. Although we can use previous single image methods,
e.g. [14], [41], [50], [11], to get the PSF, we found that the PSF

from [51] is more reliable especially when the CRF is nonlinear(Section IV-B). We validated our approach on three different
camera models: Canon 60D, 400D, and Nikon D3100.
Figure 10(a),(b) and (c) show the captured sharp/blurry image
pairs. The ISO value and the relative exposure setting for each
captured image pair were also shown. The sharp images were
captured with a tripod whereas the blurry images were captured byholding the camera by hands with long exposure period. To obtain
the ground truth CRF for comparisons, we used the Macbeth color
checkerboard and applied the PCA-based method [15] to estimate
f −1 via curve fitting [38]. Figure 10 (d),(e) and( f) plot ourestimated CRFs against the ground truth CRFs. The green dots are
the sampled chart values from the Macbeth color checkerboard.
Finally, we tested the effectiveness of different regularizationbased deconvolution algorithms in Figure 11. In particular, we
compare the method from Krishnan and Fergus [25], Levin etal. [30], and Xu and Jia [50]. The work from Krishnan and Fergus
uses hyper-laplacian prior to regularize the deconvolved image,
Levin et al.use the sparsity regularization while Xu and Jia use
the L1-norm regularization. Among all the deconvolution results,
the gamma correction consistently shows better results than the
results using linear CRF correction. However, with our estimated
CRF correction, the deconvolution results for all three methods

10
1

Ground Truth
Our Estimated CRF
with Single Image
Our Estimated CRF
with multiple Images
Inverse Gamma-Correction

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

0.2

0.4

0.6

0.8

1

0.8

1

0.8

1

0.8

1

1

Ground Truth
Our Estimated CRF
with Single Image
Our Estimated CRF
with multiple Images
Inverse Gamma-Correction

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

0.2

0.4

0.6

1

Ground Truth
Our Estimated CRF
with Single Image
Our Estimated CRF
with multiple Images
Inverse Gamma-Correction

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

0.2

0.4

0.6

1

Ground Truth
Our Estimated CRF
with Single Image
Our Estimated CRF
with multiple Images
Inverse Gamma-Correction

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

(a)

0

0.2

0.4

0.6

(b)

(c)

(d)

Fig. 12. Our result with real examples: (a) input images; (b) estimated CRFs; (c) deblurred images with gamma correction; (d) deblurred images with CRF
correction.

can be further improved.
2) With Unknown PSF: When the PSF is not available, weestimate the inverse CRF using the method in Section V-C. Notethat since the CRF estimation step is separated from the PSF
estimation step, the accuracy of our estimated CRF for the realexamples does not depend on the PSF estimation algorithm. In
our implementation, we choose the method in [50] to estimate
the PSF from a single image and the method in [30] with thesparsity regularization for deconvolution.
Figure 12 shows the results from the input image in (a) thatis captured by a Canon EOS 400D (first row) and a Nikon D90
DSLR (second and third rows) camera with a nonlinear CRF. The
results in (b) are the estimated CRF using single and multiple
images. The ground truth CRFs were computed by using method
in [28] with a calibration pattern. For the reference, we have also
show the inverse gamma-correction curve with γ equal to 2.2.
This is a common method suggested by the previous deblurring
algorithm [14] when the CRF is unknown. Note the shape of theinverse gamma-correction curve is very different from the shapeof ground truth and our estimated CRF. We compare the resultswith gamma correction (γ = 2.2) and with our estimated CRF
corrections in (c) and (d) respectively. As expected, our resultswith CRF corrections are better – not only do the deblurring
results contain less artifacts, but the estimated PSFs are also moreaccurate after the CRF correction.

VII. D ISCUSSION
A. Effectiveness of Gamma Correction
When the ground truth CRF is unknown, a common method in
previous deblurring algorithms [14], [50] is to use gamma curve
correction with γ = 2.2. In this section, we provide additional
analysis to the effectiveness of this approach. Let the blurred
irradiance be B̃ = I˜⊗K , and the observed intensity be B = f (B̃).
When gamma curve is used, we have:fg −1 (f (B̃)) = f (B̃)2.2 ,

(27)

11

0.7

0.7

0.6

0.6

1
0.8

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0
0

0
0

Estimated curve
Ground truth curve
Gamma curve

0.6
0.4

0.2

0.4

0.6

0.8

1

0.2
0.2

0.4

(a)

0.6

0.8

1

(b)

0
0

(a) Blurred image

0.2

0.4

0.6

0.8

1

(b) inverse CRFs

Fig. 13. The error curves of DoRF CRFs when (a) gamma curve (γ = 2.2)and (b) linear CRF are used to approximate the ground truth CRFs. The
horizontal axis denotes the B̃ values, and the vertical axis denotes the errorτ (B̃). The colors of these curves represent different underlying CRFs. The
central dark curve shows the mean τ for all the 188 real CRFs.

1.2

1

1.2

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.8
0.6
0.4

0.2
0.2
0
0

100

200

300

400

500

0.2

0

0

−0.2
0

−0.2
0

100

200

300

400

500

1.2

1

1
0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2
0.2
0
0

200

(a)

300

400

500

−0.2
0

100

200

300

400

500

300

400

500

(c) deblurred result withgamma correction(d) deblurred result with
our estimated CRF correction(e) error map of (c)(f ) error map of (d)

0.2

0
100

200

1.2

1
0.8

0.8

100

0

100

200

300

400

500

(b)

−0.2
0

(c)

Fig. 14. We tested the effectiveness of gamma correction on synthetic example. (a) Input synthetic signal. Deconvolution with gamma curve correction
using (b) wiener filter, and (c) sparsity regularization [30]. As discussed in
Section IV, the ringing artifacts are mainly caused by the blur inconsistency.
The gamma curve correction cannot fully eliminate blur inconsistency especially for edges with high contrast.

where fg (x) = x1/2.2 denotes the gamma curve CRF. Thus, theabsolute error introduced by the curve can be measured by:τ (B̃) = |f (B̃)2.2 − B̃|.

(28)

As discussed previously in Section III and Section IV, errors in
the CRF estimation will be directly transferred to the deblurred
image leading to large ringing artifacts that cannot fully be
suppressed by regularization.
In order to show the magnitude of the approximation errors,
we measure the differences between the gamma curve and the
188 real CRF database [15] presented in Figure 2. Figure 13(a)shows the approximation errors of the gamma curve. The central
dark curve shown in the figure represents the mean of the errorτ (B̃) for all the 188 real CRF curves. Note that most part of theerror curves have errors larger than 0.1 which is 10% of intensity
range. Some curves even have errors as large as 0.6. For thereference, we also show the approximation errors if a linear CRF
is used in Figure 13(b). Although the gamma curve has smaller
errors compared to the linear CRF, gamma curve is insufficient
to represent the real-world CRFs as noted in previous work [15],[31], [23].
Next, we use the synthetic examples in Figure 3 to analyze
the effectiveness of gamma curve correction in the non-blind
deconvolution. Figure 14 shows the result. The gamma curve
correction can reduce errors compared to the results using the lin-

Fig. 15. (a) Input blurred Image. (b) The inverse CRFs. Note the gamma
curve with γ = 2.2 is closed to the inverse of mean CRFs. (c) Deblurred
image with gamma curve correction. (d) Deblurred image with our estimatedinverse CRF correction. (e) Error maps using gamma correction. (f) Error
maps using our estimated CRF correction.

ear correction, and the ringing artifacts are further reduced whena regularization-based deconvolution is used. Yet, as illustrated
in Figure 14, it cannot completely remove the ringing artifacts
caused by the blur inconsistency from nonlinear CRF.
Finally, we show an example in Figure 15, where the gamma
curve correction is “successful”. We use the mean CRF of the 188
real CRF database [15] as the ground truth CRF. As illustrated
in Figure 15(b), the gamma curve with γ = 2.2 is closed tothe inverse of mean CRFs. We use the method in Section V-B toestimate the CRF with known PSF. This example is considered as
successful in deblurring since the deconvolution artifacts are small
and unnoticeable after a regularization is used. However, if wecompare the error maps between the gamma curve correction andour estimated CRF correction, our approach can further reduce
errors with the same deconvolution algorithm [25].
B. Assumptions and Limitations
Our proposed algorithms algorithms work under several assumptions. In the following, we analyze each of the assumptions
and discuss the limitations of our approach in practise.
1) Robustness against image noise: Theorem 2 for our CRF
estimation algorithms assumes noise-free conditions. To analyze121Sigma 5Sigma 15
Sigma 25
Ground truth

0.8

0.6

0.6

0.4

0.4
0.2

0.2
0
0

0.02

Sigma 5Sigma 15
Sigma 25
Ground truth

RMSE of the estimated CRF

1
0.8

0.2

0.4

0.6

0.8

1

0
0

0.01

0.005

0
0

0.2

0.4

0.6

0.8

1

Canon400D CRF
NikonD3100 CRF

0.015

5

10
15
20
noise variance (sigma)

25

Fig. 16. The estimated CRF under different noise level. (a) Canon400D.
(b) Nikon D3100. (c) Plot of RMSE of the estimated CRF against different
amount of noise.

0

0.5

1

(a) Input
10.8(b) with gamma 2.2(c) with our CRF

Ground truth
Ours
Gamma 2.2

0.6
0.4
0.2
0
0

0.2

0.4

0.6

0.8

(d) Estimated CRF

better deblurring results than not performing linearization at all.
The blur inconsistency analysis in Section III assumes a convex
CRF. In practice, there are camera models where the CRF is
non-convex in shape especially for the blue channel. We tested
our algorithms for non-convex CRF using synthetic examples.
However, we found that our estimated CRFs for unknown PSF are
not as accurate as the convex cases. One possible explanation isthat the non-convex CRFs deviate enough from the mean CRF tocause the nonlinear least-squares fitting in the rank minimization
to converge to an incorrect local minima.
We have also tested our algorithms for highly textured images where straight-line edges are very limited. Both algorithms
failed since the edge detection algorithm fail to detect straightline edges. We consider this is a fundamental limitation to ouralgorithms.

1(e) error map of (b)(f ) error map of (c)Fig. 17. Failure case. Our algorithm fails to estimate accurate CRF when
the intensity range of input image is narrow. Yet, the deblurred result is still
better than the gamma curve with less ringing artifacts.

the robustness of our algorithms against image noise, we generate
a set of synthetic blurry and noisy images with different amount ofGaussian noise. The ground truth image in Figure 15 and the CRF
curves of Canon400D and Nikon D3100 were used to produce
the synthetic inputs. Figure 16 shows the plot of RMSE of theestimated CRF (with unknown PSF) against different amount ofnoise. Our CRF estimation algorithm is robust to image noise
thanks to the usage of rank minimization optimization and the
directional filtering in the edge selection step.
2) Spatially varying blur: Our CRF estimation algorithms
assume spatially invariant motion blur. When facing spatiallyvarying motion blur or defocus blur with different depth layers,
the blind CRF estimation algorithm with unknown PSF will break
down since the observation matrix is no longer a rank-1 matrix.
In such scenario, only the CRF estimation algorithm with known
PSF can be used providing that the local PSF can be obtained
accurately. The blur inconsistency analysis, however, should still
hold which claims that the irradiance of the blurry image should
be linearized before deblurring in order to avoid ringing artifacts
caused by non-linear CRF.
3) Failure Cases: We finally provide a failure case in Figure 17
where the CRF estimation fails due to the narrow intensity
range of the input image. The estimated CRF deviates from theground truth CRF especially for the area that is not covered by
the intensity range of input image. However, when comparing
deblurring results using a gamma curve, our result is still betterwith less ringing artifacts. As discussed earlier, the goal ofthis paper is to provide a method to handle nonlinear CRF in
deblurring problem when the ground truth CRF is not accessible.
In some cases, even an incorrectly estimated CRF may produce

VIII. C ONCLUSION
This paper offers two contributions targeting image deblurringin the face of nonlinear CRFs. First, we have presented an analysis
on the role that nonlinear CRFs play in image deblurring. We
prove that the blur inconsistency introduced by nonlinear CRF
can caused notable ringing artifacts in the deconvolution process
which cannot be completely ameliorated by image regularization.
Such blur inconsistency is spatially varying and it depends onedge sharpness, edge intensity range, and image brightness. We
have also demonstrated how the nonlinear CRF adversely affects
PSF estimation for several state-of-the-art techniques.
In Section V, we prove how the shape of edge projections
resemble the cumulative distribution function of 1D PSFs forlinear CRFs. This theorem was used to formulate two CRF
estimation strategies for when the motion blur PSF kernel isknown or not known. In the latter case, we show how rank
minimization can be used to provide a robust estimation. We
do note that our approach is sensitive to the quality of theselected blur profiles. The requirement of straight line sharp edges
also places a limitation to our current solution. To this end, we
have also provided a multiple image solution in case a motion
blurred image does not contains sufficient edge profiles for CRF
estimation.
Experimental results in real examples demonstrated the importance of intensity linearization in the context of image deblurring.
In addition, we have analyzed the effectiveness of gamma curve
correction which is commonly used in previous deblurring algorithms [14], [50]. The shape of gamma curve with γ = 2.2 is close
to the inverse of mean CRFs. Therefore, in most cases, gamma
curve correction is more effective than linear CRFs. However,when compared to the results with our estimated CRF corrections,gamma curve correction is still insufficient especially when the
CRF is largely deviate from the mean CRFs.
IX. ACKNOWLEDGEMENT
This work was funded in part by the National Research
Foundation (NRF) of Korea (2012-0003359), Microsoft Research
Asia under the KAIST-Microsoft Research Collaboration Center(KMCC), Natural Science Foundation (China) (Nos:61273258,
61105001), the Committee of Science and Technology, Shanghai(No. 11530700200), National Science Foundation (US) under
grants IIS-CAREER-0845268 and IIS-RI-1016395, the Air Force
Office of Science Research under the YIP Award, and the Singapore A*STAR PSF grant (Proj No. 1121202020).

13

R EFERENCES[1] M. Ben-Ezra and S. Nayar. Motion-based Motion Deblurring. IEEE
Trans. PAMI, 26(6):689–698, Jun 2004.
[2] J. Cai, H. Ji, C. Liu, and Z. Shen. Blind motion deblurring from a single
image using sparse approximation. In CVPR, 2009.
[3] A. Chakrabarti, D. Scharstein, and T. Zickler. ”an empirical camera
model for internet color vision”. In BMCV, 2009.
[4] Y. Chang and J. Reid. Rgb calibration for color image-analysis inmachine vision. IEEE Trans. Image Processing, 5(10):14141422, 1996.
[5] J. Chen and C. K. Tang. Robust dual motion deblurring. In CVPR,
2008.
[6] X. Chen, X. He, J. Yang, and Q. Wu. An effective document image
deblurring algorithm. In CVPR, 2011.
[7] X. Chen, F. Li, J. Yang, and J. Yu. A theoretical analysis of camera
response functions in image deblurring. In ECCV, 2012.
[8] S. Cho and S. Lee. Fast motion deblurring. In ACM SIGGRAPH ASIA,
2009.
[9] S. Cho, Y. Matsushita, and S. Lee. Removing non-uniform motion blurfrom images. In ICCV, 2007.
[10] S. Cho, J. Wang, and S. Lee. Handling Outliers in Non-blind Image
Deconvolution. In ICCV, 2011.
[11] T.-S. Cho, S. Paris, B. Freeman, and B. Horn. Blur kernel estimation
using the radon transform. In CVPR, 2011.
[12] P. Debevec and J. Malik. Recovering high dynamic range radiance maps
from photographs. In ACM SIGGRAPH, 1997.
[13] H. Farid. Blind inverse gamma correction. IEEE Trans. Image
Processing, 10(10):14281433, 2001.
[14] R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman.
Removing camera shake from a single photograph. ACM Trans. Graph.,25(3), 2006.
[15] M. Grossberg and S. Nayar. Modeling the space of camera response
functions. IEEE Trans. PAMI, 26(10), 2004.
[16] M. D. Grossberg and S. K. Nayar. What is the space of camera response
functions? In CVPR, 2003.
[17] M. Hirsch, C. Schuler, S. Harmeling, and B. Schölkopf. Fast removalof non-uniform camera shake. In ICCV, 2011.
[18] J. Jia. Single image motion deblurring using transparency. In CVPR,
2007.
[19] N. Joshi, S. Kang, L. Zitnick, and R. Szeliski. Image deblurring withinertial measurement sensors. ACM Trans. Graph., 29(3), 2010.
[20] N. Joshi, R. Szeliski, and D. Kriegman. Psf estimation using sharp edge
prediction. In CVPR, 2008.
[21] S. Kim, J. Frahm, and M. Pollefeys. Radiometric calibration with
illumination change for outdoor scene analysis. In CVPR, 2008.
[22] S. Kim, Y.-W. Tai, S. Kim, M. S. Brown, and Y. Matsushita. Nonlinear
camera response functions and image deblurring. In CVPR, 2012.
[23] S. J. Kim, H. T. Lin, Z. Lu, S. Susstrunk, S. Lin, and M. S. Brown. A new
in-camera imaging model for color computer vision and its application.
In IEEE Trans. PAMI, 2012.
[24] S. J. Kim and M. Pollefeys. Robust radiometric calibration andvignetting correction. IEEE Trans. PAMI, 30(4), 2008.
[25] D. Krishnan and R. Fergus. Fast image deconvolution using hyperlaplacian priors. In NIPS, 2009.
[26] D. Krishnan, T. Tay, and R. Fergus. Blind deconvolution using anormalized sparsity measure. In CVPR, 2011.
[27] J. Lagarias, J. Reeds, M. Wright, and P. Wright. Convergence propertiesof the nelder-mead simplex method in low dimensions. SIAM Journalof Optimization, 9(1):112–147, 1998.
[28] J.-Y. Lee, B. Shi, Y. Matsushita, I. Kweon, and K. Ikeuchi. Radiometriccalibration by transform invariant low-rank structure. In CVPR, 2011.
[29] A. Levin. Blind motion deblurring using image statistics. In NIPS, 2006.
[30] A. Levin, R. Fergus, F. Durand, and W. T. Freeman. Image and depth
from a conventional camera with a coded aperture. ACM Trans. Graph.,26(3), 2007.
[31] H. Lin, S. J. Kim, S. Susstrunk, and M. S. Brown. Revisiting radiometriccalibration for color computer vision. In ICCV, 2011.
[32] S. Lin, J. Gu, S. Yamazaki, and H.-Y. Shum. Radiometric calibrationfrom a single image. In CVPR, 2004.
[33] S. Lin and L. Zhang. Determining the radiometric response function
from a single grayscale image. In CVPR, 2005.
[34] P.-Y. Lu, T.-H. Huang, M.-S. Wu, Y.-T. Cheng, and Y.-Y. Chuang. Highdynamic range image reconstruction from hand-held cameras. In CVPR,
2009.
[35] L. Lucy. An iterative technique for the rectification of observed
distributions. Astron. J., 79, 1974.
[36] Y. Matsushita and S. Lin. Radiometric calibration from noise distributions. In CVPR, 2007.
[37] T. Mitsunaga and S. Nayer. Radiometric self calibration. In CVPR,
1999.

[38] T.-T. Ng, S.-F. Chang, and M.-P. Tsui. Using geometry invariants forcamera response function estimation. In CVPR, 2007.
[39] A. Rav-Acha and S. Peleg. Two motion blurred images are better thanone. PRL, 26:311–317, 2005.
[40] W. Richardson. Bayesian-based iterative method of image restoration.
J. Opt. Soc. Am., 62(1), 1972.
[41] Q. Shan, J. Jia, and A. Agarwala. High-quality motion deblurring froma single image. ACM Trans. Graph., 2008.
[42] Y.-W. Tai, H. Du, M. Brown, and S. Lin. Image/video deblurring usinga hybrid camera. In CVPR, 2008.
[43] Y.-W. Tai, H. Du, M. S. Brown, and S. Lin. Correction of spatiallyvarying image and video motion blur using a hybrid camera. IEEE
Trans. PAMI, 32(6):1012–1028, 2010.
[44] Y.-W. Tai, N. Kong, S. Lin, and S. Shin. Coded exposure imaging for
projective motion deblurring. In CVPR, 2010.
[45] Y.-W. Tai and S. Lin. Motion-aware noise filtering for deblurring of
noisy and blurry images. In CVPR, 2012.
[46] Y.-W. Tai, P. Tan, and M. Brown. Richardson-lucy deblurring for scenes
under projective motion path. IEEE Trans. PAMI, 33(8):1603–1618,
2011.
[47] O. Whyte, J. Sivic, A. Zisserman, and J. Ponce. Non-uniform deblurringfor shaken images. In CVPR, 2010.
[48] N. Wiener. Extrapolation, interpolation, and smoothing of stationary
time series. New York: Wiley, 1949.
[49] B. Wilburn, H. Xu, and Y. Matsushita. Radiometric calibration using
temporal irradiance mixtures. In CVPR, 2008.
[50] L. Xu and J. Jia. Two-phase kernel estimation for robust motion
deblurring. In ECCV, 2010.
[51] L. Yuan, J. Sun, L. Quan, and H. Shum. Image deblurring withblurred/noisy image pairs. ACM Trans. Graph., 26(3), 2007.

Yu-Wing Tai received the BEng (first class honors) and MS degrees in compute science from theHong Kong University of Science and Technology(HKUST) in 2003 and 2005 respectively, and the
PhD degree from the National University of Singapore (NUS) in June 2009. He joined the Korea Advanced Institute of Science and Technology(KAIST) as an assistant professor in Fall 2009. He
regularly serves on the program committees for the
major Computer Vision conferences (ICCV, CVPR,
and ECCV). His research interests include computer
vision and image/video processing. He is a member of the IEEE.

Xiaogang Chen is currently a PhD student in the Institute of Image Processing and Pattern Recognition,
Shanghai Jiao Tong University. His research interests
include image and video processing, and computer
vision. He is a student member of the IEEE.

Sunyeong Kim received the BS and MS degrees in
Computer Science from Korea Advanced Instituteof Science and Technology (KAIST) in 2010 and2012 respectively. She is currently pursuing the PhD
degree in KAIST. Her research interests include
computer vision and image/video processing, especially in computational photography. She is a student
member of the IEEE.

14

Seon Joo Kim received the BS and MS degrees fromYonsei University, Seoul, Korea, in 1997 and 2001.
He received the PhD degree in computer sciencefrom the University of North Carolina at Chapel
Hill in 2008. He is an assistant professor at the
Department of Computer Science, Yonsei University
since March 2013. His research interests include
computer vision, computer graphics/computational
photography, and HCI/visualization. He is a memberof the IEEE.

Michael S. Brown obtained his BS and PhD in
Computer Science from the University of Kentucky
in 1995 and 2001 respectively. He is currently an
Associate Professor and Assistant Dean (External
Relations) in the School of Computing at the National University of Singapore. Dr. Browns researchinterests include computer vision, image processing
and computer graphics. He has served as an area
chair for CVPR, ICCV, ECCV, and ACCV and iscurrently an associate editor for the IEEE Transactions on Pattern Analysis and Machine Intelligence(TPAMI).

Feng Li received his BE from the Department
of Electrical Engineering, Fuzhou University, in
2003, MS from the Institute of Pattern Recognition, Shanghai Jiaotong University, in 2006, and
his PhD from the Department of Computer andInformation Sciences, University of Delaware, in2011. He joined Mitsubishi Electric Research Labs
as a visiting research scientist in November 2011.
His research interests include multi-camera systemdesign and applications, fluid surface reconstruction
and medical imaging. He is a member of the IEEE.

Jie Yang received his PhD in computer science fromthe University of Hamburg, Germany. He is now
a professor and director of the Institute of Image
Processing and Pattern Recognition, Shanghai Jiao
Tong University. He has led more than 30 nationaland ministry scientific research projects in image
processing, pattern recognition, data amalgamation,
data mining, and artificial intelligence.

Jingyi Yu is an Associate Professor in the Department of Computer & Information Sciences and theDepartment of Electrical & Computer Engineering
at the University of Delaware. He received his B.S.
from Caltech in 2000 and Ph.D. from MIT in 2005.
His research interests span a range of topics in computer vision and computer graphics, especially on
computational cameras and displays. He has served
as a Program Chair of OMNIVIS 11, a General
Chair of Projector-Camera Systems 08, and an Area
and Session Chair of ICCV 11. He is a recipient ofboth the NSF CAREER Award and the AFOSR YIP Award.

Yasuyuki Matsushita received his B.S., M.S. and
Ph.D. degrees in EECS from the University of Tokyoin 1998, 2000, and 2003, respectively. He joined
Microsoft Research Asia in April 2003. He is a Lead
Researcher in Visual Computing Group. His areasof research are photometric methods in computer
vision. He is on the editorial board member of IEEE
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), International Journal of Computer
Vision (IJCV), IPSJ Journal of Computer Vision andApplications (CVA), The Visual Computer Journal,
and Encyclopedia of Computer Vision. He served/is serving as a Program CoChair of PSIVT 2010, 3DIMPVT 2011, and ACCV 2012. He is appointed
as a Guest Associate Professor at Osaka University (April 2010-), Visiting
Associate Professor at National Institute of Informatics (April 2011-) and
Tohoku University (April 2012-), Japan. He is a senior member of IEEE.