A Probabilistic Intensity Similarity Measure based on Noise Distributions
Yasuyuki Matsushita
Stephen Lin
Microsoft Research Asia
Beijing, P.R. China, 100080
Email: {yasumat,stevelin}@microsoft.com
Abstract

1
Similarity measure
1−Normalized Distance

0.9

We derive a probabilistic similarity measure between two
observed image intensities that is based on the noise properties of the camera. In many vision algorithms, the effect ofcamera noise is either neglected or reduced in a preprocessing stage. However, noise reduction cannot be performed
with high accuracy due to lack of knowledge about the true
intensity signal. Our similarity metric specifically represents the likelihood that two intensity observations correspond to the same unknown noise-free scene radiance. By
directly accounting for noise in the evaluation of similarity,
the proposed measure makes noise reduction unnecessaryand enhances many vision algorithms that involve matching of image intensities. Real-world experiments demonstrate the effectiveness of the proposed similarity measure
in comparison to the standard L2 norm.

1. Introduction
Imaging noise produces random perturbations of intensity values that degrade the visual quality of images andreduce the reliability of computer vision algorithms. There
has been much work on modeling and removing these variations; however, even when the noise characteristics of animage are known, it remains a difficult problem to remove
noise in a manner that preserves the actual scene information in the measured intensity signal. Since there in generaldoes not exist sufficient information in an image to extract
the original scene data, noise reduction cannot be performed
with high accuracy, and often leads to loss of image detailor introduction of image artifacts.
In this work, we present an alternative approach for dealing with the effects of imaging noise on vision algorithms.
Rather than attempt to remove the noise from an image, we
aim to directly account for its effects on intensity similarity
measurement, which is one of the most fundamental and important operations in computer vision. Similarity between
intensity observations is used to guide a broad range of algorithms including segmentation, object recognition, and optical flow. Conventionally, the similarity of two observed

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 1. Plot of the proposed similarity measure and the L2 distance, shown after normalization and with respect to intensity
level 0.5. The displayed similarity measure is computed basedon intensity-dependent Gaussian noise whose variance is proportional to the intensity level. The resulting similarity measure has
an asymmetric profile.

intensities I1 and I2 is measured as a function of their L2
norm ||I1 − I2 ||2 . While such a distance metric provides
a valid comparison in theory, the quality of this measurelessens in practice due to the presence of noise in real images.
Noise is introduced into intensity observations at multiple points in the imaging pipeline. Prominent components
of image noise include the random noise associated with
A/D conversion and uneven photon flow from the scene,
fixed pattern noise due to differences in sensitivity among
photon detectors in the imaging array, and dark current
noise that results from measurement of thermal radiation
within the camera. A detailed presentation of the various
noise sources can be found in [6]. Since noise arises froma series of stochastic processes, it is natural to treat intensity observations as samples from a probability distribution
defined by these imaging fluctuations. Such noise distributions may vary among pixels in the image, and can also be
dependent on intensity level.
Our proposed method captures the intensity dependentnoise distribution of a camera in a calibration step, and thenbased on these probability distributions, we derive in thispaper a probabilistic measure of the similarity between two
observed intensities. This metric specifically represents thelikelihood that two intensity observations correspond to thesame noise-free scene radiance, with respect to the probabilistic noise distributions.
The effect of considering image noise in similarity measurement is illustrated by the example in Figure 1. While
metrics based on the L2 norm do not account for the particular imaging fluctuations within a camera, our proposed intensity similarity measure provides statistically meaningfulvalues with respect to noise characteristics. The likelihoodthat two observations have the same noise-free value depends upon these noise characteristics, such that the similarity is high when the two intensities are both well within thenoise distributions of certain true intensities, and becomessignificantly lower otherwise. This is in contrast to conventional metrics which have a fixed structure that is intended
to measure distance between observations rather than theirsimilarity. These distance measures do not represent thelikelihood that two intensity observations correspond to thesame scene radiance value, since they do not consider how
intensities fluctuate in the imaging system. We contend that
it is the similarity of intensities that is generally importantfor many image understanding tasks, and that the use of asimilarity-based measure is especially beneficial in real applications where intensity perturbations exist.
The presented intensity similarity measure can be directly extended to handle color values, and can also be employed in subpixel and multiresolution applications. With
this proposed metric, we experimentally demonstrate improved performance on the common vision process of blockmatching.

1.1. Prior work
How to determine the similarity of a pair of R1 intensity
observations is a problem that has received little attention.
In fact, it has become a convention in computer vision tocompute the dissimilarity between two intensity observations as a function of their Euclidean distance. While these
metrics are valid measures of distance between two intensity observations, they do not represent their similarity inthe sense of how likely the two observations correspond to
the same scene radiance value. A likelihood measure should
be a function of intensity fluctuations in the imaging system,
which are not accounted for in these fixed distance metrics.
Omer and Werman [8] proposed an approach to modelcolor distributions in an image as lines. These distributions
are collected from regions that have homogeneous color.
The proposed color model enables comparison of color values that is invariant to shading, highlights and color distortions caused by the imaging process. Comaniciu andMeer [3] also took an image-specific clustering approachthat is based upon the mean shift algorithm. An obtained
affinity measure leads to good segmentation results. A
learning-based approach to modeling pixel affinity was presented by Fowlkes et al. [4], in which human-segmented
images are used as a training set. A similarity measure
called the bottleneck affinity was presented by Omer andWerman [10]. The similarity measure is defined as a likelihood that two features belong to the same cluster, basedon the histogram profiles along the straight-line path between the two points. Similar affinity measures based on
geodesic distances [12] or geodesic paths [9] have also been
proposed. These previous affinity-based methods all seek todetermine how likely two observations belong to the sameclass, based on modeling of feature distributions for eachclass. In contrast, our work aims to measure how likely two
observations correspond to the same scene radiance value,
based on models of intensity distributions due to noise.
Closer to our work, Alter et al. [2] established an intensity similarity measure for low-light conditions, which uses
Maximum Likelihood (ML) estimation to define the similarity between two intensity observations. While it moreaccurately measures similarity than other basic metrics, it
relies heavily on accurately estimating the true intensity
signal, which is difficult to achieve from only two observations. In our work, we derive a new intensity similarity
measure that avoids having to estimate the true intensity signal. Instead, our similarity measure is based on the marginaldensity of observations with respect to every possible true
intensity signal.

2. Probabilistic framework
The measured intensity of a pixel can be considered asa random variable that takes a value I in the space of observations Ω. Given an observed I, the probability that it
resulted from a true intensity I  of scene radiance is givenby the conditional probability density p(I|I  ).
Alter et al. [2] presents a similarity measure based on
ML estimation. From two observations, it first estimates
the true intensity I ∗ , then derives similarity as the productof conditional probabilities:
I ∗ = argmax p(I1 |I  )p(I2 |I  ),I(1)

SM L (I1 , I2 ) = p(I1 |I ∗ )p(I2 |I ∗ ).
The ML similarity SM L is defined as the likelihood of two
intensity observations resulting from the single true intensity I ∗ that gives the greatest likelihood.
The ML similarity gives a sound measure in theory if theestimated true intensity is correct. However, the true intensity generally cannot be estimated accurately from two observations in practice, especially under noisy imaging conditions. Our approach differs from ML similarity in that it

…
I1

I2

I1

…
I1

I2

I'

I'

(a)

−

I'

(n)

( n 1)

I'

(b)

+

I2

( n 1)(c)Figure 2. Comparison of similarity measures in the case of Gaussian noise. (a) Gaussian similarity measure, which assumes that one of thetwo observations (I1 or I2 ) is free of noise. (b) ML similarity measure, which tries to estimate the true intensity signal I  by ML estimationand then derives the similarity as the product of conditional probabilities p(I1 |I  )p(I2 |I  ). (c) Our probabilistic similarity measure, whichsums up the probability of observing I1 and I2 from every possible true intensity I  . I (n) represents the n-th possible true intensity value.
1
0.9

does not attempt to estimate the true intensity. We define thesimilarity of two observed intensities I1 and I2 as the likelihood that they each resulted from the same but unknown
true intensity I  , i.e., p(I1 |I  )p(I2 |I  ). Since the true intensity is unknown, all possible true intensities are taken into
account, rather than using a specific true intensity as donein the ML similarity measure. Our probabilistic similarity
measure is therefore formulated as


def
(2)
S(I1 , I2 ) = p t(I1 ) = t(I2 )p(I1 |I  )p(I2 |I  )p(I  )dI  ,=Ωwhere t(I) is the true intensity of observation I, and p(I  )is the prior density of the true intensity, which is uniform.
With this definition in the form of marginal densities, theproposed similarity measure is able to naturally avoid theuncertainty that exists in the estimation of a single true intensity.
The proposed intensity similarity measure is formulated
as a marginal density of the likelihood function over the parameter I  as shown in Equation (2). This formulation isequivalent to the marginal likelihood in Bayesian estimation [13], in which there is not a single parameter value (inour case of I  ) that characterizes the entire model, but rather
an entire set of possible likelihood functions is used. In theBayesian framework, the marginal likelihood is computed
using the likelihood functions with the associated prior distributions of the parameters. Our similarity measure alsouses all possible likelihood functions p(I1 |I  )p(I2 |I  ) instead of relying upon a specific parameter I ∗ as done in ML
similarity.

2.1. Case studies with different noise models
With different models of noise, we examine the proposed
similarity measure and explain its differences from priorsimilarity metrics.

50

50

50

100

100

100

0.6

I2

I2

I2

0.5

150

150

150

0.4

200

200

200

250

250

250

0.8
0.7

0.3
0.2
0.1
50

100

150

200

50

250

100

150

200

250

50

100

150

200

250

0
1
0.9

50

50

50

100

100

100

0.6

I2

I2

I2

0.5

150

150

150

0.4

200

200

200

250

250

250

0.8
0.7

0.3
0.2
0.1
50

100

I1

150

200

250

50

(a)

100

I1

150

(b)

200

250

50

100

I1

150

200

250

0

(c)

Figure 3. Visualization of different intensity similarity measuresin the case of intensity-independent Gaussian noise (top row :σ 2 = 100, bottom row : σ 2 = 250). (a) Gaussian similarity,(b) ML similarity, and (c) probabilistic similarity. I1 and I2 are
two observations, and brighter values indicate greater similarity ofthe two observations. Similarity values are mapped to range [0,1]for visualization purposes.

Intensity-independent Gaussian noise
Suppose that image noise follows a Gaussian distribution,and that this Gaussian noise is the same at all intensities.
The most common approach for determining the similarityof two observations (I1 and I2 ) in this case is to assume
that either I1 or I2 is the true intensity, and then evaluate a
Gaussian function as follows:(I1 −I2 )2
1
SG (I1 , I2 ) = p(I2 |I1 ) = p(I1 |I2 ) = √e− 2σ2 . (3)2πσA visualization of this metric for various values of I1 andI2 is shown in Figure 2 (a).
With the ML similarity measure defined in Equation (1),similarity in this case can be expressed as
SM L (I1 , I2 )==p(I1 |I ∗ )p(I2 |I ∗ )∗ 22 −I )
1 − (I1 −I ∗ )2 +(I
2σ 2
e
,
2πσ 2

(4)

1

0.9

60
50

50

0.8

50

Variance

0.7

40

30

100

100

I2

I2

150

150

200

200

250

250

0.6

0.5

0.4

20

0.3

0.2

10

0.1

0

0

50

100

150

200

250

Intensity

0
50

100

I1

150

200

250

50

100

I1

150

200

250

1
120

0.9
50

50

100

100

0.6

I2

I2

0.5

150

150

0.4

200

200

250

250

0.8

100

0.7
80

Variance

approach takes into account all possible true intensity signals, it theoretically gives a more accurate similarity estimate than previous approaches.

60

40

0.3
20

0.2
0.1

0

0

50

100

150

200

250

Intensity

50

100

I1

150

200

250

50

100

I1

150

200

250

0

1
120

0.9
50

50

Intensity-dependent Gaussian noise
As described in [7], noise level or variance is often a function of intensity in real images. In this case, it is unclear how
to define the Gaussian similarity measure of Equation (3),since p(I1 |I2 ) = p(I2 |I1 ). Variance σ now becomes σI  ,which is a function of I  . For ML similarity and our proposed measure, the similarity metrics become
SM L (I1 , I2 )

0.8

100

=

0.7

Variance

80

60

100

100

0.6

I2

I2

0.5

150

150

0.4

200

200

40

0.3
20

0.2
0.1

0

0

50

100

150

Intensity

(a)

200

250

250

250
50

100

I1

150

(b)

200

250

50

100

I1

150

200

250

0

(c)

Figure 4. Visualization of different intensity similarity measuresin the case of intensity-dependent Gaussian noise. (a) Noise level
function, (b) ML similarity, and (c) probabilistic similarity. From
top row to bottom row, the noise variances σ 2 are respectively alinear, sinusoidal, and step function with respect to intensity. I1
and I2 are two observations, and brighter values indicate greater
similarity of the two observations. Similarity values are mapped
to range [0,1] for visualization purposes.

where I ∗ = argmaxI  p(I1 |I  )p(I2 |I  ). Figure 2 (b) displays the form of this function. Since the true intensity I ∗is obtained through ML estimation, rather than by assuming
one of the observations to be noise-free, it should provide a
better similarity estimate than the basic Gaussian distancefunction. However, it still relies heavily on an accurate estimate of the true intensity value.
Our approach differs from these previous two in thatit does not attempt to estimate the true intensity. Rather,it sums up probabilities that these two observations result
from the same, but unknown, true intensity:S(I1 , I2 ) =p(I1 |I  )p(I2 |I  )p(I  )dI 
(5)
Ω


−(I1 −I2 )2
 I +I  2I −(I +I ) 
1
m
2
1
2
1
2
4σ
√ e
+ erf 2σ,=erf
2σ4σ|Ω| π
def

=

def

where Im = max I  , |Ω| = p(I  ) = const, and erf(·) isthe error function [1] defined as
 x
2
def 2
e−t dt.
erf(x) = √π 0
Derivation of Equation (5) is detailed in Appendix. Figure 2 (c) illustrates the form of this measure. Since thisp(I1 |I ∗ )p(I2 |I ∗ )(I −I ∗ )2 +(I2 −I ∗ )2
− 1
2σ 2∗
I

1
e
2πσI2∗(6),

where I ∗ = argmaxI  p(I1 |I  )p(I2 |I  ), and
S(I1 , I2 ) =p(I1 |I  )p(I2 |I  )p(I  )dI 
Ω

=

1
2π|Ω|



1 −
2 e
σ
Ω I

(I1 −I  )2 +(I2 −I  )2
2σ 2
I

(7)
dI  .

Visualizations of these similarity measures built upon
intensity-dependent Gaussian noise are shown in Figure 4.
We note that unlike the other similarity measures, our probabilistic metric may take an asymmetric form that arises
from the intensity dependency of noise, as also illustrated
in Figure 1.
Nonparametric noise model
When the noise cannot be modelled analytically, conditional pdfs can be expressed more generally in the form of
discrete histograms. Histograms representing noise distributions can be obtained through real-world measurements,
such as by capturing multiple shots of a static scene froma fixed viewpoint. In this setup, a histogram is constructed
for each pixel from all of its recorded intensity observations.
To relate a histogram to a conditional pdf p(·|I  ), it is necessary to estimate the true intensity signal I  from the histogram. Various criteria may be used for this purpose. For
zero-mean noise, the mean value of the distribution can beused to determine the true intensity signal. For an assumption that the true intensity signal should correspond to mostcommon value in the distribution, the intensity value at themode of the histogram is taken as the true intensity signal.
Once the true intensity signal I  of a histogram is established, the histogram is normalized such that it sums toone. In this way, the histogram provides a representation of
the joint probability density p(I, I  ). The conditional pdfs
p(I|I  ) are directly computed from the joint pdfs p(I, I  ) asp(I|I  ) = p(I, I  ).
p(I, I  )dI
Ω

(8)

0db

0db

1

1

0.9

50

50

50

100

100

100

I

I2

I2

150

150

150

0.8

0.9

50

50

50

100

100

100

I

I2

I2

150

150

150

200

200

200

250

250

250

0.8

0.7

0.6

0.5

0.7

0.6

0.5

0.4

0.4

0.3
200

200

200

0.3

0.2

0.2

0.1
250

250
50

100

I'

150

200

250

250
50

100

I1

150

200

250

50

100

I1

150

200

250

15db

0.1

0

50

100

I'

150

200

250

50

100

I1

150

200

250

50

100

I1

150

200

250

6db

1

1

0.9

50

50

50

100

100

100

I

I2

I2

150

150

150

200

200

200

0.8

0.9

50

50

50

100

100

100

I

I2

I2

150

150

150

200

200

200

250

250

250

0.8

0.7

0.6

0.5

0.4

0.7

0.6

0.5

0.4

0.3

0.3

0.2

0.2

0.1
250

250
50

100

I'

150

200

250

250
50

100

I1

150

200

250

50

100

I1

150

200

250

30db

0.1

0

50

100

I'

150

200

250

50

100

I1

150

200

250

50

100

I1

150

200

250

12db

1

50

50

100

100

100

I

I2

I2

150

150

150

0.8

0.9

50

50

50

100

100

100

I

I2

I2

150

150

150

200

200

200

250

250

250

0.8

0.7

0.6

0.5

0.4

0.7

0.6

0.5

0.4

0.3
200

200

200
0.2

0.3

0.2

0.1
250

250
50

100

I'

150

200

250

(a)

250
50

100

I1

150

200

250

(b)

50

100

I1

150

200

250

0.1

0

(c)

Figure 5. Nonparametric case with different gain parameters. (a)conditional pdf p(I|I  ), (b) ML similarity, and (c) probabilistic
similarity. From top to bottom, the results of gain parameters 0 db,
15 db, and 30 db are shown. The data is obtained from a realworld scene using the green color channel of a PointGrey Research
Dragonfly camera. For visualization purposes, similarity values
are mapped to range [0, 1] and are shown over the true intensity
range of [1, 254].

0

1

0.9

50

0

50

100

I'

150

200

250

50

100

(a)

I1

150

200

250

50

(b)

100

I1

150

200

250

0

(c)

Figure 6. Nonparametric case with different gain parameters. (a)conditional pdf p(I|I  ), (b) ML similarity, and (c) probabilistic
similarity. From top to bottom, the results of gain parameters 0 db,
6 db, and 12 db are shown. The data is obtained from a real-world
scene using the green color channel of a Sony DSR-PD190P camera. For visualization purposes, similarity values are mapped torange [0, 1] and are shown over the true intensity range of [1, 254].

2.3. Subpixel & multi-resolution similarity measure
Using the obtained conditional pdfs and Equation (2), thenonparametric probabilistic similarity measure is obtained.
Figures 5 and 6 show the intensity similarity measuresderived from observed pdfs at different camera gain levels,
which can affect noise characteristics. As seen in the figures, actual conditional pdfs have complex structures thatare hard to model with simple parametric functions, but canbe easily handled in our probabilistic similarity measure.

There are many computer vision algorithms that utilize
subpixel or multi-resolution matching, e.g., image alignment and optical flow. The proposed intensity similarity measure can be extended to handle subpixel/multiresolution cases.
As shown in Figure 7, a subpixel value and a lowerresolution pixel value Is can both be expressed using combination weights wi with the original pixels:

2.2. Color similarity measureThis similarity measure for pixel intensity can be directly extended to handle pixel features represented as vector quantities, such as color or local texture attributes. Using
RGB color as an example, the similarity between vectors
v1 = [r1 , g1 , b1 ] and v2 = [r2 , g2 , b2 ] can be expressed asS(v1 , v2 ) =p(v1 |v )p(v2 |v )dv
(9)
Ωv
  
=
p(r1 |r )p(g1 |g  )p(b1 |b )
Ωb

Ωg

Ωr


p(r2 |r )p(g2 |g  )p(b2 |b )p(r )p(g  )p(b )dr dg  db
= S(r1 , r2 ) · S(g1 , g2 ) · S(b1 , b2 ).

Is =



wi Ii .

(10)i

Accordingly, the conditional pdf ps (Is |I  ) can be derived
by linear interpolation of the original pdfs:
ps (Is , Θ|I  ) =wi p(Ii |I  ),

(11)

i

where parameter Θ indicates the combination of Ii and withat is used for interpolation. Inserting the conditional pdfs
ps into Equation (2), we obtain the probabilistic similarity
measure for the subpixel/multiresolution case.

A

B

A

B

D

C

D

wA wB

wC wD
C

(a) Subpixel case

wA

wB

wC

wD

(b) Multi-resolution case

Figure 7. Subpixel and multi-resolution cases. Similarity is recomputed based upon combination weights w, which are determinedby the new pixel’s coverage of the original pixels.

Scene #1 (0db)

Scene #2 (6db)

Scene #3 (18db)

Scene #4 (18db)

3. Application and Experiments
The proposed similarity metric provides the likelihoodthat two observations correspond to the same scene radiance. To evaluate the effectiveness of the proposed similarity measure, we compare it to the conventional L2 distance in the application of basic block matching [5]. Blockmatching is chosen as our example since this fundamental
image matching algorithm is employed in a wide range ofapplications.
Application formulation Block matching finds the mostsimilar image block by minimizing the sum of squared differences to a given block:2
F1 (x) − F0 (x − u) ,(12)
u = argminux∈B

where B is the given block, Fi (·) is a function at the i-thframe that maps the pixel coordinate to its intensity, x is a
2D vector in the image space, and u is a 2D motion vector tobe estimated. By minimizing the distance between blocks inconsecutive frames, a motion vector u is obtained. With theproposed similarity measure, Equation (12) can be rewrittenas
S(F1 (x), F0 (x − u))(13)
u = argmaxu= argmax
u

x∈B



x∈B

p(F1 (x)|I  )p(F0 (x − u)|I  )p(I  )dI  .

ΩTo avoid computational errors caused by tiny numericalvalues, the actual computation is performed in the log domain:u = argmin− log S(F1 (x), F0 (x − u)). (14)ux∈B

Experimental configuration Experiments were performed using a PointGrey DragonFly and a Sony DSRPD190P camera. For each camera, the similarity measure
is precomputed using sequences shot of static scenes. At

Figure 8. Scenes used in block matching experiments, with noiselevel given in parentheses.

Scene
Similarity
L2 norm
Mean dev.
Max dev.
Min dev.

#1
57.12 %
56.78 %
0.34 %
0.59 %-0.09 %

#2
57.84 %
57.53 %
0.31 %
3.64 %-1.69 %

#3
20.38 %
19.35 %
1.03 %
1.60 %
0.51 %#4
52.56 %
48.47 %
4.09 %
6.06 %
2.05 %

Table 1. Results of block matching. The top two rows show thecorrect matching rate using the proposed similarity measure andthe standard L2 norm. Mean dev. is the mean performance improvement of the probabilistic similarity measure in comparison
to the L2 norm. Max dev. and Min dev. represent the maximum
and minimum performance improvement, respectively.

each gain setting, 300 frames are recorded for the 708×564resolution DSR-PD190P and the 640×480-resolution DragonFly. Observed noise distributions that have the same true
intensity value are merged together, since more observations per intensity level bring greater statistical reliability
to the similarity measure. This results in roughly 450000
samples per intensity level for the DSR-PD190P and 36000
samples per intensity level for the DragonFly. Once theprobabilistic intensity similarity measure is computed, theresult is stored in a symmetric matrix (look-up table) of sizen × n, where n is the number of intensity levels.
To obtain datasets for block matching, we mounted avideo camera at a fixed position and simply captured an image sequence of a static scene. Since there do not exist intensity changes except for noise, the correct matching scorecan be computed by counting the number of times blocks
stay at their original positions, as done in [2]. The experimental datasets are of the four scenes shown in Figure 8.
Scene #1 was captured using the DragonFly, and Scenes

Sorted plot of matching score

Variation of matching score with L2 norm

Variation of matching score with similarity measure

1

1

1

Similarity measure
L2 norm

Matching score

0.9
0.8
0.7

Matching score

0.8

0.8

0.6
0.5

Matching score

Search window
Block window

0.6
0.4
0.2

0.6
0.4
0.2
0

0

0.4

−10

−10

−5

−5

0.3
0.2

y
0

0

0

y

0.1

5

5
0

50

100

150

200

10

250

Sorted block poisition
Sorted plot of matching score
Similarity measure
L2 norm

0.9

−6

−8

10

6

8

x

1

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.2

0.2

0
−20

0
−20

0.5

−4

−2

0

2

4

−6

−8

10

15

x

Variation of matching score with L2 norm

1

0.7

0.8

Matching score

−4

−2

0

2

4

Variation of matching score with similarity measure1

Search window
Block window

6

8

0.4
0.3

−10

0.2

x

0.1

−10
0

x
10

0

0

200

400

600

800

1000

Sorted block position

0
10

20

−15

−10

−5

5

0

10

15

y

20

−15

−10

−5

5

0

y

2

(b) Plot of matching scores(c) Score variation (similarity)(a) Scene(d) Score variation (L )Figure 9. Variations in block matching scores. Top row: 5 × 5 block size, 15 × 15 search window size. Bottom row: 10 × 10 block
size, 30 × 30 search window size. (a) shows the scene, one block and its search range. (b) shows a plot of sorted matching scores using
the proposed similarity measure in comparison to the L2 norm. (c) and (d) show the distribution of matching scores at different block
positions in the neighbouring frames. Matching scores using the similarity measure are expressed in the negative log domain as describedin Equation (14). Matching scores are mapped into the range [0, 1], with lower scores indicating better matches.

#2, 3, and 4 are from the DSR-PD190P.
For performance evaluation, the captured image sequences are cropped to a 320 × 240 spatial resolution. We
use a 3 × 3 block size and a 21 × 21 search range centeredat the original block position. Block matching is evaluatedusing each of the non-boundary pixels, and 100 frames ofeach scene are used to statistically evaluate performance.
Results The results of block matching are summarized inTable 1. The top two rows list the percentage of correct
matches using the proposed similarity measure and the standard L2 norm. The third row shows the performance difference of the proposed similarity measure in comparison to
the L2 norm. Max dev. and Min dev. indicate the maximum
and minimum performance difference of the proposed similarity measure over the L2 norm. As we can see from thetable, the probabilistic similarity measure leads to better results under noisy conditions (Scenes #3 and 4) in which thestandard L2 norm breaks down. At low noise levels (Scenes#1 and 2), the performance of the proposed similarity measure is similar to the L2 norm.
While the probabilistic similarity measure does not deviate substantially from the L2 norm in low-noise conditions,
it is nevertheless useful in evaluating the reliability of estimated matches [11]. Since our proposed metric represents a
measure of similarity rather than distance, it can better indicate the confidence of a match than the L2 norm, and does
so in a statistically meaningful way. Such a confidence measure can provide useful information to subsequent processes

Similarity
L2

Mean of RMSD
0.547
0.262

Table 2. Mean of root mean squared distance (RMSD) of the correct matching score to the set of incorrect matching scores. Thisvalue is computed after scaling the matching scores into the range[0,1]. A larger RMSD indicates greater discrimination power. The
left figure shows a frame from the test image sequence.

in which the estimated motion is used, e.g., eliminating unreliable estimates. In addition, the proposed similarity measure offers greater discrimination power than the standard
L2 norm. Figure 9 shows the variation of block matchingscores. In the figure, matching scores obtained within the
search range are plotted in (b), (c), and (d). The matching scores using the similarity measure are computed in the− log domain as described in Equation (14), and all the obtained matching scores are scaled into the range [0, 1], withsmaller matching scores corresponding to better matches.
In these examples, the best matches all correspond to correct matches. As we can see in the sorted matching scoresin Figure 9 (b), the proposed measure provides a steeper
distribution than the L2 norm. This sharper discrimination
between correct and incorrect matches brings greater confidence in matching results. Figure 9 (c) and (d) exemplify
the variations in matching scores using the proposed similarity measure and the L2 norm, respectively. It can beobserved that the proposed similarity measure more clearlydifferentiates correct from incorrect matches.
The same experiment is conducted using a sequence of
100 images at 320 × 240-resolution to measure statistical
performance. To characterize the discrimination power, theroot mean squared distance (RMSD) is computed, whichmeasures the distance from the correct-matching score toeach of the incorrect-matching scores, defined as:
RM SD =1  ∗(s − s)2 )
ns s

1
2

,

(15)

∗

where s is the matching score of the correct match, s is amatching score at any position in the search range, and ns
is the number of matches tested in the search range. Table 2
shows the mean of the RMSD scores obtained from all correct matching results in the sequence. The larger RMSD
of the proposed similarity measure in comparison to the L2
norm indicates its greater power in discriminating between
correct and incorrect matches.

4. Conclusion
Intensity similarity is a quantity that is difficult to define;
however, the consideration of imaging noise distributions
enables a statistical approach to this problem. In this paper,
we propose a new intensity similarity measure in a probabilistic form that directly embeds the effect of noise. The
similarity measure is formulated as the likelihood of two
intensity observations having the same scene radiance. For
the cases of intensity-independent and intensity-dependent
Gaussian noise, we derive analytic expressions of the similarity measure. We also describe a method to compute thesimilarity measure for a data-driven noise model. It is also
shown that the intensity similarity measure can be directly
extended to handle color values, subpixel shifts, and multiresolution applications.
The proposed similarity measure is evaluated on a basic
block matching technique in comparison to the standard L2
norm. Better performance is demonstrated, particularly under noisy conditions. Moreover, we show that the proposed
similarity measure naturally indicates the confidence of amatch, and provides greater discrimination power between
correct and incorrect matches.

References[1] M. Abramowitz and I. A. Stegun, editors. Handbook ofMathematical Functions with Formulas, Graphs, and Mathematical Tables, chapter 7. Error Function and Fresnel Integrals, pages 297–309. Dover, 1972. 4, 8
[2] F. Alter, Y. Matsushita, and X. Tang. An intensity similarity measure in low-light conditions. In Proc. of European
Conference on Computer Vision, pages 267–280, 2006. 2, 6
[3] D. Comaniciu and P. Meer. Robust analysis of feature spaces:
color image segmentation. In Proc. of Computer Vision and
Pattern Recognition, pages 750–755, 1997. 2[4] C. Fowlkes, D. R. Martin, and J. Malik. Learning affinity
functions for image segmentation: Combining patch-based
and gradient-based approaches. In Proc. of Computer Vision
and Pattern Recognition, volume 2, pages 54–64, 2003. 2[5] R. Gonzalez and R. Woods.
Addison-Wesley, 1992. 6

Digital Image Processing.

[6] G. E. Healey and R. kondepudy. Radiometric ccd camera
calibration and noise estimation. IEEE Trans. on PatternAnalysis and Machine Intelligence, 16(3):267–276, 1994. 1[7] C. Liu, W. T. Freeman, R. Szeliski, and S. B. Kang. Noiseestimation from a single image. In Proc. of Computer Vision
and Pattern Recognition, volume 1, pages 901–908, 2006. 4[8] I. Omer and M. Werman. Color lines: Image specific color
representation. In Proc. of Computer Vision and Pattern
Recognition, volume 2, pages 946–953, 2004. 2[9] I. Omer and M. Werman. The bottleneck geodesic: Computing pixel affinity. In Proc. of Computer Vision and Pattern
Recognition, volume 2, pages 1901–1907, 2006. 2[10] I. Omer and M. Werman. Image specific feature similarities. In Proc. of European Conference on Computer Vision,
volume 2, pages 321–333, 2006. 2[11] I. Patras, E. A. Hendriks, and R. L. Lagendijk. Confidence
measures for block matching motion estimation. In Proc.
of International Conference on Image Processing, volume 2,
pages 277–280, 2002. 7[12] M. Rattray. A model-based distance for clustering. In Proc.
of the IEEE-INNS-ENNS International Joint Conference on
Neural Networks, volume 4, pages 13–16, 2000. 2[13] T. A. Severini. Likelihood Methods in Statistics. Oxford
University Press, 2001. 3

Appendix
When the noise variance σ 2 is constant for all intensity
values, the similarity of two observations I1 and I2 is derived as follows:S(I1 , I2 ) =p(I1 |I  )p(I2 |I  )p(I  )dI Ω
1−(I1 −I  )2 /2σ 2 −(I2 −I  )2 /2σ 2
e
=
e
dI 
2
Ω 2πσ |Ω|2
1−(I12 +I22 )/2σ 2=e(I1 +I2 −I )I /σ dI e2πσ 2 |Ω| Ω
Independent of I(I1 −I2 )2 
 
 
1
√ e− 4σ2
erf 2I −(I2σ1 +I2 ) Ω
4σ|Ω| π(I1 −I2 )2 
 +I 
1
1 +I2 )
2√ e− 4σ2
erf 2Imax −(I+ erf I12σ,=
2σ
4σ|Ω| π

=

def

def

where |Ω| = p(I  ) = const, Imax = max I  , and erf(·) isan error function [1] defined as
 x
2
def 2
erf(x) = √e−t dt.
π 0