Estimating Camera Response Functions using Probabilistic Intensity Similarity
Jun Takamatsu
Microsoft Institute for Japanese Academic Research Collaboration (MS-IJARC)
Suginami-ku, Tokyo, Japanj-taka@cvl.iis.u-tokyo.ac.jp

Yasuyuki Matsushita
Microsoft Research Asia
Beijing, P.R. China, 100080
yasumat@microsoft.com

Katsushi Ikeuchi
Microsoft Institute for Japanese Academic Research Collaboration (MS-IJARC)
Suginami-ku, Tokyo, Japan
ki@cvl.iis.u-tokyo.ac.jp

Abstract

camera I as O = f (I). Assuming they are continuousand monotonic, response functions can be inverted to obtain the inverse response function g (= f −1 ). Measured
image intensities can be linearized using I = g(O). Usually only observed intensities O are available, so most estimation methods attempt to estimate the inverse response
function g.
Our method uses a small number of images of a static
scene captured from the same viewpoint with fixed camera
parameters. Due to noise, these images exhibit slight differences in intensity. Our method uses these intensity fluctuations to estimate camera response functions. More specifically, we use the probabilistic intensity similarity [13],which represents the likelihood that two intensity observations originated from the same scene radiance. The similarity varies with the underlying noise distributions and theshape of the response functions. Our method uses this relationship between the intensity similarity and the response
function to estimate response functions. As illustrated in
Figure 1, our method uses a physics-based noise model [6]to define a model-based intensity similarity. Taking a few
images captured from the same viewpoint with fixed camera parameters, our method estimates the response function
by maximizing the similarity among the input images. The
method outputs the estimate of the response function as wellas the data-specific intensity similarity measure. The contributions of this paper are as follows:

We propose a method for estimating camera response
functions using a probabilistic intensity similarity measure.
The similarity measure represents the likelihood of two intensity observations corresponding to the same scene radiance in the presence of noise. We show that the response
function and the intensity similarity measure are strongly
related. Our method requires several input images of astatic scene taken from the same viewing position with fixedcamera parameters. Noise causes pixel values at the same
pixel coordinate to vary in these images, even though theymeasure the same scene radiance. We use these fluctuations to estimate the response function by maximizing theintensity similarity function for all pixels. Unlike priornoise-based estimation methods, our method requires onlya small number of images, so it works with digital cameras as well as video cameras. Moreover, our method does
not rely on any special image processing or statistical priormodels. Real-world experiments using different cameras
demonstrate the effectiveness of the technique.

1. Introduction
Many computer vision algorithms rely on the assumption that image intensity is linearly related to scene radiancerecorded at the camera sensor. However, this assumption
does not hold with most cameras; the linearity is not maintained in the observation domain due to non-linear camera
response functions. Linearization of measured image intensity is important for many vision algorithms to work. To
linearize measured image intensities, we must estimate thecamera response function.
Camera response functions describe the relationship between the measured image intensity O and irradiance at a

• We show a theoretical relationship between intensity
similarity and response functions.
• We develop a practical noise-based method that only
requires a minimum of two images. We also extend
the method to handle a single-image case.
1ModelPhysics-basednoise model

Model-based
intensity similarity

0.35

0.3

Input

Estimation

A few of imagesof the same scene

Maximization of
input image similarity

Output
1

0.45

0.9

0.4

0.8

0.35

0.7

0.3

0.6

0.25

0.5

0.2

0.4

0.15

0.3

Intensity similarity

0.5

50

50

0.25

0.4
100

100

0.2

0.3
150

0.15

Inverse response function
0.5

150

0.5

50

0.4
100
0.3
150

0.2

0.2

0.1

200
0.1

200

200

0.05

250
0

50

100

150

200

250

50

100

150

200

250

0

0.1

0.2

0.05

0.1

0.1

250

250
50

100

150

200

250

0

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

50

100

150

200

250

0

Figure 1. Overview of the proposed method. Our method uses a probabilistic intensity similarity based on a physics-based image noisemodel. We estimate camera response functions by maximizing the similarity measure of images of the same scene computed from themodel-based intensity similarity. We produce estimates of the inverse response functions and data-specific intensity similarity measures.

Throughout this paper, we use the terms input signal andoutput signal of a response function. The input signal originates from the scene radiance and is considered the input tothe response function. This signal is converted into a digital
output signal by a series of circuits including a photodetector, amplifiers, and possibly signal processing blocks. This
output signal is the measured pixel intensity in images. We
normalize the input and output ranges of the response functions.
The rest of this paper is organized as follows: after reviewing prior work in Section 2, we describe the theoretical background and our estimation method in Section 3. In
Section 4, we describe implementation details. In Section 5
we show experimental results to verify the effectiveness ofthe method and present some of its applications. Section 6
concludes the paper.

2. Prior work
One class of methods for estimating response functions
uses knowledge of irradiance ratios for measured intensities. The Macbeth color checker is used for estimating the
response functions by several research groups such as [2].
Nayar and Mitsunaga [16] use an optical filter with spatiallyvarying transmittance; the variation corresponds to the irradiance ratio. To avoid using such special equipment, somemethods use a set of images of a static scene from a fixed
viewpoint, taken with different exposure times, so the irradiance ratio is known. These methods are classified into
parametric and non-parametric approaches.
In the parametric framework, Mann and Picard [12] propose a method that assumes the response functions can beapproximated by gamma correction functions. Mitsunaga
and Nayar use a polynomial function for the representation [15]. Grossberg and Nayar apply principal component
analysis (PCA) to a database of real-world response functions (DoRF) and show that the space of response functions
can be represented by a small number of basis functions [5].
Mann [11] uses a comparametric equation that relates two
input images taken with different exposure times.
Debevec and Malik estimate response functions witha non-parametric representation using a smoothness con-straint [3]. Tsin et al. robustly estimate non-parametricforms of response functions using the statistics of image
noise [19]. Pal et al. use a Bayesian network consistingof probabilistic imaging models and prior models of response functions [18]. While non-parametric approaches
have greater descriptive power, the large number of unknowns often lead to computationally expensive or unstablealgorithms.
A few authors have proposed more general estimation
methods that allow camera motion or scene motion between
input images. Grossberg and Nayar [4] propose a methodthat allows a small amount of motion in the scene. Theiruse of the brightness histograms efficiently avoids the problem of pixel-wise image registration. Kim and Pollefeys [7]propose a method that allows free camera motion by finding
the correspondences between images. While these methods
can estimate the function from less restricted setups, they
still require images taken with multiple different exposures.
Another class of the estimation methods is based on thephysics of the imaging process. Lin et al. [8] estimate theresponse function by analyzing the linear blending of colors along edges in images. For grayscale images, a 1D analogue of the 3D color method is presented in [9]. Whilethe use of spatial color mixtures has theoretical importance,these authors note that image noise can be a major problem.
Matsushita and Lin later proposed a noise-based method using the assumption that profiles of input noise distributions
are symmetric [14]. Although the method is robust against
noise, it generally requires a large number of images of astatic scene to collect noise distributions. If the scene contains many uniformly colored, uniformly illuminated, flat
surfaces, then one image can suffice. This limits its applicability only to videos in practice. Our method is a noisebased method as well, but in contrast to the prior work it
only requires a small number of images of a natural scene.

3. Method
In this section we first provide the theoretical relationship between the response function and probabilistic intensity similarity. We then describe the algorithm for estimating response functions.

Inverse response function

1

Intensity similarity

Normalized input I

0.9
0.5

50

0.8

I2

0.7

0.4
100

0.6
0.5

0.3
150

0.4

0.2

0.3
200

0.2

0.1
0.1
0

250
0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

50

100

150

200

250

0

I1

Normalized output O

1

Normalized input I

0.9
0.5

50

0.8

I2

0.7

0.4

0.5

0.3
150

0.4

200

0.2

0.1
0.1
250
0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

50

100

150

200

250

0

I1

Normalized output O

1
0.9

Normalized input I

p(I) = f 0 (I)p(O).

(2)

0.2

0.3

0.5

50

0.8

I2

0.7

0.4

100

0.6
0.5

0.3
150

0.4

0.2

0.3
200

0.2

0.1
0.1
0

Using dO = f 0 (I)dI, we obtain

100

0.6

0

probability of observing the intensity O when the noise-free
intensity is Õ. The similarity in the input domain SI (I1 , I2 )is defined in the same manner. Now we derive the relationship between the intensity similarity in the input andoutput domains. For the derivation, we use the transformation of pdf’s from output to input domain. For all outputs
O = f (I), the pdf in the output domain p(O) and its transformation p(I) in the input domain satisfy the following relationship:p(I)dI = p(O)dO.

250
0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Normalized output O

0.9

1

50

100

150

200

250

0

I1

Figure 2. Relationship between inverse response function andprobabilistic intensity similarity. The graphs in the left column
show inverse response function shapes. The plots in the right column show the intensity similarity measures for any two pixel values. The similarity measure is color-coded as indicated by the baron the right. These plots show how the intensity similarity varies
with the inverse response function.

3.1. Response functions and intensity similarity
Our method uses the probabilistic intensity similarity
proposed by Matsushita and Lin [13]. The intensity similarity represents the likelihood of two observations derivingfrom the same scene radiance. As pointed out in their work
as well as by Liu et al. [10], probability density functionsof image noise distributions vary with the scene radiance.
Nonlinear camera response functions also alter the intensity
similarity function. In fact response functions and the intensity similarity measure are closely related. Figure 2 shows
how the intensity similarity varies with the shape of the response function.
The intensity similarity SO (O1 , O2 ) between two measured intensities O1 and O2 is defined in [13] asZ
SO (O1 , O2 ) = p(O1 |Õ)p(O2 |Õ)p(Õ)dÕ,(1)
where Õ represents the noise-free output intensity, the probability density function (pdf) p(Õ) is the prior distributionof having a noise-free output intensity Õ in the scene, andthe conditional density function (cdf) p(O|Õ) describes the

˜ in the input domain can be transformed in
The cdf p(I|I)
a similar manner. Using Equation (2), Equation (1) can be
written as:Z
SO (O1 , O2 ) =p(O1 |Õ)p(O2 |Õ)p(Õ)dÕZ
˜ p(I)˜
˜ p(I2 |I)p(I1 |I)˜ I˜
f 0 (I)d=00˜f (I1 ) f (I2 ) f 0 (I)
Z
1
˜
˜ ˜ ˜
= 0
p(I1 |I)p(I
2 |I)p(I)dI
f (I1 )f 0 (I2 )1= 0
SI (I1 , I2 ),(3)
f (I1 )f 0 (I2 )where I1 = g(O1 ) and I2 = g(O2 ). This equation showsthat the intensity similarities in the input and output domains are related by the first-order derivatives of the response function.

3.2. Image similarity
As described above, our method estimates camera response functions from the similarity among several images.
Suppose we have as input a set of images of the same scenetaken from the same viewpoint with the same camera parameters. Using all pairs of pixels from different images
with the same pixel coordinates, we can compute the cdf’sof the observations and the image similarity measures in the
output domain.
We begin by defining the image similarity in the input
domain as follows:
ZZ
def 1
E1 =
p(I1 |I2 )SI (I1 , I2 )dI1 dI2 ,(4)
Z
where p(I1 |I2 ) is the likelihood of observing input I1 after
observing input I2 from the same scene radiance. Note thatp(I1 |I2 ) and SI (I1 , I2 ) cannot be directly computed fromthe observed images. We explain how the observed images
and the physics-based noise model are used in this equationin Section 3.3 and Section 3.4, respectively. In Equation (4),the normalization term Z is defined as
sZ Z
Z=

2

(SI (I1 , I2 )) dI1 dI2 .

By defining the similarity in the input domain, we can evaluate Equation (4), despite varying camera specifications.
Equation (4) can be viewed as the sum of the intensity
similarity SI over all inputs weighed by the cdf p(I1 |I2 ).
The weighting is derived by normalizing the joint distribution of I1 and I2 by the probability density of observing
either I1 or I2 . For example, in the case that p(I2 ) is used
for normalization, we havep(I1 , I2 ).
p(I1 |I2 ) =p(I2 )

3.3. Estimation method
Now we transform Equation (4) to the output domain.
p(O1 |O2 ) is more suitable for estimating response functions
because we can compute it from image measurements. Using Equation (2), Equation (4) becomes:
ZZ
1
1
E2 =
p(O1 |O2 )SI (g(O1 ), g(O2 ))(5)Z
g 0 (O1 )g 0 (O1 )g 0 (O2 )dO1 dO2
ZZ
1=g 0 (O2 )p(O1 |O2 )SI (g(O1 ), g(O2 ))dO1 dO2 .
Z
The image similarity measure E2 evaluates how similar images are. If the correct response function g is assumed, theimage similarity E2 of our input images is expected to become high because the only fluctuations are image noise.
In Equation (5), p(O1 |O2 ) is treated as a known parameter because it is directly measured from the input images.
Our goal is to estimate the two unknown parameters g and
SI . We formulate the estimation problem in an energy maximization framework as(ĝ, ŜI ) = argmax E2 (g, SI ),and readout noise, respectively. Without noise, the noisefree input I˜ equals aP . Assuming the noise sources are
˜ is the convolution of the noise
independent, the cdf p(I|I)
distributions.
Shot noise is known to obey a Poisson distribution. We
assume natural lighting conditions where a large number ofphotons are available. The profile of the Poisson distribution approaches a Gaussian distribution as the number of
events increases, so we can approximate the Poisson distribution by a Gaussian distribution with zero-mean and vari˜ where A is some positive real number. The disance AI,
tributions of the dark current noise and the readout noise
also follow Gaussian distributions. Assuming these noise
sources are independent, the combined noise model can be
represented as a single normal distribution whose average
is zero and variance is some real number B. As a result,˜ can be written asp(I|I)!
˜2(I − I)
1
˜
exp −
.
(6)
p(I|I) = q
2(AI˜ + B)
2π(AI˜ + B)
We compute the similarity SI (I1 , I2 ) is calculated using a
numerical approximation of the integral, such as the midpoint method, from this equation:
SI (g(O1 ), g(O2 ))= SI (I1 , I2 )
(7)
Z
˜
˜ ˜ ˜
=
p(I1 |I)p(I
2 |I)p(I)dI.

˜ is considered a uniform distriIn the above equation, p(I)bution [13]. In this way, SI (I1 , I2 ) can be parameterized byonly two variables A and B.

4. Implementation
This section describes two key points of our implementation: the parameterization of the response functions andthe solution algorithm.

g,SI

where ĝ and ŜI , respectively, are the estimates of the inverse
response function and data-specific intensity similarity inthe input domain.

3.4. Modeling intensity similarity
In practice, it is difficult to treat SI (I1 , I2 ) in Equation (5) in a nonparametric form.
To parameterize
SI (I1 , I2 ), we use the physics-based noise model describedby Healey and Kondepudy [6]. As described in [6], the input I with sensor noise can be written as
I = aP + NDC + NS + NR ,where a is a factor of photon-to-electron conversion efficiency with amplification, and P is the number of photons.
NDC , NS , and NR indicate dark current noise, shot noise,

4.1. Representation of inverse response functions
To reduce the computational cost, we represent the inverse response functions using the parametric model proposed by Grossberg and Nayar [5]. We chose this parameterization because of its compactness and ease of use. Although we use this parameterization, we do not use the priordistributions in our energy function for estimating the response function. Grossberg and Nayar use principal component analysis (PCA) of the database of real-world response
functions (DoRF) to obtain a small number of eigenvectorsthat represent the space of the response functions. Using
the principal components, we represent the inverse response
function g as g = g0 + Hc, where g0 is the mean vector of
all the inverse response functions, H is a matrix in which acolumn vector represents an eigenvector, and c is a vector

of PCA coefficients. Following the original paper [5], we
use the first five eigenvectors. With this representation, thenumber of unknown variables decreases significantly, e.g.,
from 256 to 5 in the case of 8-bit images. Our method is not
restricted to this parameterization. Other parameterization
methods such as polynomial representations [15] could also
be used.

4.2. Maximization of image similarity
We maximize the image similarity measure definedin Equation (5) using an iterative alternating algorithm. We continue the following steps until convergence:
Algorithm: Maximization of E2
Input: Images taken from the same viewpoint with fixedcamera parameters.
Output: Inverse response function ĝ and data-specific similarity ŜI .
Initialize the similarity measure in the input domain
SI (I1 , I2 ) using Equations (6)(7) with initial parameters A
and B.
Repeat until E2 converges do
1. Maximize E2 with the fixed input similarity function
SI (I1 , I2 ) to estimate the inverse response function gwith ĝ = argmaxg E2 .
2. Update g: g ← ĝ.
3. Fix the inverse response function g and maximize E2
according to ŜI = argmaxSI E2 . Note SI is parameterized by A and B.
4. Update SI : SI ← ŜI by updating A and B.
done
We use the Matlab fminsearch algorithm, an implementation of the Nelder-Mead Simplex method [17]to maximize the energy function. We empirically set
A = 0.06 and B = 1 as the initial values for 8-bit images.
The maximization usually converges within ten iterations.
Like all multi-dimensional non-linear optimization problems, our optimization can converge to a local maxima.
We try to avoid this by restarting the optimization fromneighborhoods of the current best estimate.

5. Experiments
We used two different conditions to evaluate the performance of the proposed algorithm. The first uses multiple
images taken by a fixed camera with the same camera parameter settings, and the second uses only a single image.
These two setups differ only in the way of computing the
cdf’s p(O1 |O2 ) of the input dataset.

5.1. Multiple-image caseIn this experiment, we run our algorithm using multiple shots of a static scene captured from a fixed view-

point with the same camera parameters. We compute a
histogram h(O1 , O2 ), which represents the co-occurrenceof two output intensity values O1 and O2 , by gathering allpairs of pixels with the same pixel coordinates. The cdf’sof the image intensities p(O1 |O2 ) are obtained from the histogram h by:h(O1 , O2 )p(O1 |O2 ) = P.
O2 h(O1 , O2 )Results We used the following four cameras for thisexperiment: Point Grey Dragonfly2 (Camera A), Sony
DCR-TRV9E Video camera (Camera B), Sony DCRTRV900NTSC Video camera (Camera C), and Canon EOS
Kiss Digital, the original model (Camera D). For estimating the response function of camera D, we used six images(3072 × 2048 pixels with ISO 400 gain level). For the othercameras, we used ten video frames (Camera A 640 × 480
pixels, Camera B 720×576 pixels, and Camera C 720×480
pixels). These are considerably fewer images than wereused by the prior noise-based estimation method [14].
To obtain ground truth inverse response functions for
Cameras A and D, we used Mitsunaga and Nayar’s
method [15]. Cameras B and C do not have exposure controls, so we computed their response functions using theMacbeth color checker-based method [2].
Figure 3 shows typical results of our method. The images in the first column are the scenes used for the experiment. The second column shows the cdf’s p(O1 |O2 )obtained from the input data. The third column showsthe estimated intensity similarity in the input domain, andthe fourth column shows the plot of the estimated inverse
response functions and the corresponding ground truth
curves. The horizontal and vertical axes represent normalized output and input, respectively. These results verify theaccuracy of our method; the estimated curves are very close
to the ground truth curves.
Table 1 summarizes the results of the experiment. For
each camera, three different scenes are used. Because thealgorithm is applied to the RGB channels independently, we
use 9 datasets for each camera. The disparity is the mean of
the maximum differences in the normalized input domain.
These results verify the stability as well as the accuracy of
our method.
Estimation accuracy and the number of images We
performed another experiment to observe the relationshipbetween the number of images and the estimation accuracy.
One would expect higher accuracy as more images are usedbecause of the increased statistical stability. We ran our algorithm using two, five, ten, and twenty frames from Camera C. For each run, we used 9 different datasets. Figure 4
shows the results. We see a slight improvement in accuracy
with increasing numbers of images, but the improvement is
not significant. Thus, our method works well with a small
number of images.

1

0.4

0.25

50
0.3

0.2
0.25

100

100
0.15

0.2
150

150
0.15
0.1
0.1

200

200

Normalized input

50

0.05

50

100

150

200

250

0

0.8
0.7
0.6
0.5
0.4
0.3

Max diff.: 0.0088
RMSE : 0.0066

0.2

0.05

250

Ground Truth
Our method

0.9

0.35

0.1
250
50

100

150

200

250

0

0

0

0.1

0.2

50

0.2
100
0.15

0.2
150

150
0.15
0.1
0.1

200

200

Normalized input

0.25

0.3

0.25

0.05
0.05
250
50

100

150

200

250

0

0.8

50

100

150

200

250

0

0.3

Max diff.: 0.018
RMSE : 0.011

0.2

0

0.1

0.2

100
0.15
150

0.15
0.1
200

Normalized input

0.25

0.2

0.05
0.05
250
50

100

150

200

250

0

50

100

150

200

250

0

0.9

1

0.5
0.4
0.3

Max diff.: 0.010
RMSE : 0.0053

0.2

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.9

1

1

100
0.15

0.2

150
0.15

0.1
200

Normalized input

0.25

0.2

0.1

0.8

0.6

Ground Truth
Our method

0.9
50

200

0.7

Normalized output

0.3

150

0.6

0.7

0

0.35

0.25

0.5

0.1
250

0.4

100

0.4

Ground Truth
Our method

0.8(c) Sony DCR-TRV900NTSC (B-Channel)
50

0.3

1

0.2

200

1

0.4

0.9
50

0.1

0.9

Normalized output

0.3

150

0.8

0.5

0

0.35

0.25

0.7

0.6

0.1
250

0.4

100

0.6

0.7

(b) Sony DCR-TRV9E (R-Channel)
50

0.5

Ground Truth
Our method

0.9

0.35

100

0.4

1

0.4

50

0.3

Normalized output(a) Point Grey Dragonfly2 (G-Channel)

0.8
0.7

Max diff.: 0.040
RMSE : 0.021

0.6
0.5
0.4
0.3
0.2

0.05
0.05
250
50

100

150

200

250

0

0.1
250
50

100

150

200

250

0

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Normalized output(d) Canon EOS Kiss Digital (B-Channel)
Figure 3. Results of estimating camera response functions from multiple images. From left to right, the first column shows an image fromthe dataset. The second column shows the computed cdf p(O1 |O2 ). The estimated intensity similarity and inverse response functions
are shown in the third and fourth column, respectively. The estimated inverse response function is plotted with a red solid line, while theground truth is plotted with a blue dotted line.
0.05

Disparity

0.04

Mean RMSE

0.03
0.02
0.01
0

2

5

10

20

Number of frames

Figure 4. Plot of the number of images versus estimation accuracy.
It shows our method performs well even with a small number ofimages.

5.2. Single-image caseWe further extend our method to handle a single-imagecase. The estimation algorithm is the same, but we use a
different means to compute the cdf p(O1 |O2 ) because we

do not have registered pixel pairs.
As in [14], we assume that the scene contains a sufficiently large portion of uniformly colored regions. In thisexperiment, histograms h are obtained from spatially pairing neighboring pixels. We used four neighbors; up, down,
left, and right. To efficiently gather pixel pairs, we evaluate
the pixel similarity by a normal distribution with standard
deviation σ in intensity differences for each color channel.
The values of σ in the images used for this experiment were
between 7 and 9, which tended to increase with increasing
the ISO gain level. We use only pixel pairs whose similarity is within 3σ in the distribution. While it may containoutliers in the histogram h, the frequency is expected to be
dominated by pixel pairs with the same scene radiance.
Results For this experiment, we used Camera D withthree different gain levels: ISO 400, 800, and 1600. A larger

1

1

Ground Truth
Our method

0.9

0.6
0.5

Max diff.: 0.064
RMSE : 0.047

0.4
0.3

0.7
0.6
0.5

0.3
0.2

0.1

0.1

Scene

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Max diff.: 0.042
RMSE : 0.030

0.4

0.2

0

0.8

Normalized input

0.7

0Ground Truth
Our method

0.9

0.8

Normalized input

Normalized input

0.8

0

1

Ground Truth
Our method

0.9

0.7
0.6
0.5

Max diff.: 0.057
RMSE : 0.019

0.4
0.3
0.2
0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Normalized output

Normalized output

Normalized output

ISO 400

ISO 800

ISO 1600

0.8

0.9

1

Figure 5. Result of estimating red channel inverse response functions from a single image. From left to right, we show the input image andthe results for ISO 400, ISO 800, and ISO 1600.
Table 1. Mean RMSE and disparity of the estimated inverse response functions in the the normalized input domain. Three different scenes were used for each camera.

CameraA. Dragonfly
B. DCR-TRV9E
C. DCR-TRV900NTSC
D. EOS Kiss Digital

Mean RMSE
0.0068
0.026
0.017
0.025

Disparity
0.011
0.049
0.032
0.046

0.1

0.06

0.09

RMSE

0.04
0.03
0.02
R
G
B

0.01
0

Max difference

0.05

0.08
0.07
0.06
0.05
0.04
0.03
R
G
B

0.02
0.01
0

ISO 400

ISO 800

ISO 1600

ISO 400

ISO 800

ISO 1600

Figure 6. The relationship between ISO level and RMSE (left) andbetween ISO level and maximum difference (right).

ISO setting corresponds to a higher noise level.
Figure 5 shows the red channel estimation results withdifferent ISO gain levels. We used the same scene to compare the different gains. At the ISO 1600 gain level, theestimation result is entirely satisfactory, even though only asingle image is used. In contrast to the other single-image
methods [8, 9, 14], our method does not use any special image processing such as edge detection and image segmentation, nor does it require a prior model of the inverse response functions.

5.3. Applications
In this sub-section, we introduce applications of our
method. We can estimate not only camera response functions but also the function of probabilistic intensity similarity. As a use of the estimated response function, we firstshow the result of linearization of image intensity. Next, weshow the result of edge detection using the estimated probabilistic intensity similarity.
Linearization of image intensity Figure 7 shows the result of intensity linearization for image taken by Camera D.

Before calibration

After calibration

Figure 7. The result of intensity linearization using the estimatedcamera response functions for Camera D.

For linearization, we estimated the inverse response functions of the RGB channels independently. We use I = g(O)to produce the linearized image shown on the right.
Edge detection We show in this application the effectiveness of our estimated intensity similarity measure for edge
detection. We used a 3 × 3 four-neighbor Laplacian filter,i.e., [0, −1, 0; −1, 4, −1; 0, −1, 0]. The Laplacian operation
at some pixel p is written asXd(O(p), O(q)),
q∈adj(p)where adj(p) is a set of neighboring pixels of p, O(p) is theintensity value at pixel p, and d(O1 , O2 ) is a signed distancemetric between O1 and O2 . The standard definition of themetric is their difference, i.e., d(O1 , O2 ) = O1 −O2 . Using
the probabilistic intensity similarity measure, it becomesdefd(O1 , O2 ) = − log(SO (O1 , O2 )),as described in [13]. We transform the similarity measurefrom the input to the output domain according to Equation (3). Thresholding after convolving with the Laplacian
filter produces a binary edge image.
Figure 8 shows the comparison between the two metrics
for edge detection using a noisy input image. The thresholdfor the binarization was carefully tuned to produce the visually best result; we used 22 for the standard metric and
40 for our metric. The intensity similarity measure produces a better result. Because our proposed method computes data-specific intensity similarity measures as well as
response functions, it enables edge detection that is robust
to noise. As in [10], we can perform this type of noiseadaptive image processing on any images that are suitablefor our single-image method.

InputDifference

Our metricFigure 8. Result of edge detection from a noisy image (ISO-1600) taken by Camera D. The Laplacian filter is used with two different
metrics: the standard intensity difference and probabilistic intensity similarity.

6. Conclusions
We have proposed a method for estimating camera response functions by maximizing the image similarity measure defined as the integral of the probabilistic intensity similarity. Using the image similarity, our method can estimate
inverse response functions from only a few images. We alsopresented an extension that handles the single-image case.
Our experimental results for both the multiple and single
image cases quantitatively demonstrate the effectiveness ofthe proposed algorithm. Our method does not require any
prior knowledge about the inverse response functions, unlike [8, 9, 14]. In the future, we plan to explore cases where
the physic-based noise model no longer holds, such as lowlight conditions.
Limitations Our method uses a physics-based image
noise model [6]. Therefore, if the noise distribution deviates from this model as described in [1], our approach may
produce inaccurate estimates. The single-image method described in this paper assumes a scene containing many local
planar patches of uniform colors. If the scene does not fit
this assumption, e.g., gradient-colored surfaces, the method
may produce an unsatisfactory result. Note that this assumption is not required for the multiple-image case.
Since our method uses noise observations, it is unable toperform the estimation from noise-free images. In practice,noise is always present. Our experimental result suggeststhat the method works well with noisier images, however,this tendency is expected to be capped at a certain noiselevel. For future work, we plan to explore the upper boundof the noise level where the method still works.

Acknowledgement
The authors would like to thank Bennett Wilburn for his
useful feedback on this research.

References[1] F. Alter, Y. Matsushita, and X. Tang. An intensity similarity measure
in low-light conditions. In Proc. of European Conf. on Comp. Vis.
(ECCV), pages 267–280, 2006. 8[2] Y.-C. Chang and J. F. Reid. Rgb calibration for color image analysis
in machine vision. IEEE Trans. on Image Processing, 5(10):1414–
1422, 1996. 2, 5

[3] P. E. Debevec and J. Malik. Recovering high dynamic range radiancemaps from photographs. In Proc. of ACM SIGGRAPH, pages 369–
378, 1997. 2[4] M. D. Grossberg and S. K. Nayar. What can be known about theradiometric response function from images? In Proc. of European
Conf. on Comp. Vis. (ECCV), volume 2, pages 189–205, 2002. 2[5] M. D. Grossberg and S. K. Nayar. What is the space of camera response functions? In Proc. of Comp. Vis. and Patt. Recog. (CVPR),
volume 2, pages 602–609, 2003. 2, 4, 5
[6] G. E. Healey and R. Kondepudy. Radiometric ccd camera calibrationand noise estimation. IEEE Trans. on Patt. Anal. and Mach. Intell.,16(3):267 – 276, 1994. 1, 4, 8
[7] S. J. Kim and M. Pollefeys. Radiometric alignment of image sequences. In Proc. of Comp. Vis. and Patt. Recog. (CVPR), pages
645–651, 2004. 2[8] S. Lin, J. Gu, S. Yamazaki, and H. Y. Shum. Radiometric calibrationfrom a single image. In Proc. of Comp. Vis. and Patt. Recog. (CVPR),
volume 2, pages 938–945, 2004. 2, 7, 8
[9] S. Lin and L. Zhang. Determining the radiometric response function from a single grayscale image. In Proc. of Comp. Vis. and Patt.
Recog. (CVPR), volume 2, pages 66–73, 2005. 2, 7, 8
[10] C. Liu, W. T. Freeman, R. Szeliski, and S. B. Kang. Noise estimationfrom a single image. In Proc. of Comp. Vis. and Patt. Recog. (CVPR),
pages 901–908, 2006. 3, 7
[11] S. Mann. Comparametric equations with practical applications inquantigraphic image processing. IEEE Trans. on Image Processing,9(8):1389–1406, 2000. 2[12] S. Mann and R. Picard. Being ‘undigital’ with digital cameras: Extending dynamic range by combining differently exposed pictures. In
Proc. of IS & T 48th Annual Conf., pages 422 – 428, 1995. 2[13] Y. Matsushita and S. Lin. A probabilistic intensity similarity measure
based on noise distributions. Proc. of Comp. Vis. and Patt. Recog.
(CVPR), 2007. 1, 3, 4, 7
[14] Y. Matsushita and S. Lin. Radiometric calibration from noise distributions. Proc. of Comp. Vis. and Patt. Recog. (CVPR), 2007. 2, 5, 6,
7, 8[15] T. Mitsunaga and S. K. Nayar. Radiometric self-calibration. In Proc.
of Comp. Vis. and Patt. Recog. (CVPR), volume 2, pages 374–380,
1999. 2, 5
[16] S. K. Nayar and T. Mitsunaga. High dynamic range imaging: Spatially varying pixel exposures. In Proc. of Comp. Vis. and Patt. Recog.
(CVPR), volume I, pages 472–479, 2000. 2[17] J. A. Nelder and R. Mead. A simplex method for function minimization. Computer Journal, 7:308–312, 1965. 5[18] C. Pal, R. Szeliski, M. Uyttendale, and N. Jojic. Probability models
for high dynamic range imaging. In Proc. of Comp. Vis. and Patt.
Recog. (CVPR), volume 2, pages 173–180, 2004. 2[19] Y. Tsin, V. Ramesh, and T. Kanade. Statistical calibration of ccd
imaging process. In Proc. of Int’l Conf. on Comp. Vis. (ICCV), volume 1, pages 480–487, 2001. 2