Descattering of Transmissive Observation
using Parallel High-frequency Illumination
Kenichiro Tanaka1
Yasuhiro Mukaigawa1
Yasuyuki Matsushita2
Yasushi Yagi1
1
Institute of Scientific and Industrial Research, Osaka University, Japan
2
Microsoft Research Asia
http://www.am.sanken.osaka-u.ac.jp/

Abstract
The inner structure of an object can be measured by capturing transmissive images. However, recorded images of atranslucent object tend to be unclear due to strong scattering of light inside the object. In this paper, we propose adescattering method based on Parallel High-frequency Illumination. We show in this paper that the original highfrequency illumination method and its extended techniques
can be uniformly explained as a separation of overlappedand non-overlapped light rays. Moreover, we show that theissue of overlapped transmissive rays can be addressed bya parallel projection-measurement system. We develop ameasurement system that consists of a camera and projector
with telecentric lenses and evaluate its descattering effects
by extracting transmissive light rays.

1. Introduction
When recording transmissive observations of a translucent object, captured images tend to be unclear due to strong
scattering. The scattering is caused by collisions of lightrays with tiny particles in the medium that diffuse lightrays, and is inevitable in various translucent objects. Therefore, for recording a clear transmissive images, an effective
descattering technique is needed.
Figure 1 shows examples of transmissive images. A nailin a piece of food is shown in (a). There is an urgent demand in the food industry to inspect products for detecting
and eliminating foreign objects, where transmissive imaging can help. Another example in (b) shows vein patterns
inside a hand. Transmissive imaging is also useful for living body visualization, measurement of blood oxygen levels [1], and security purposes. In such application domains,capability of recording clear transmissive images plays animportant role.
There are various approaches for descattering in the fieldof computational photography via software post-processing(a) Foreign object in food(b) Vein patternFigure 1. Examples of transmissive image.

and/or specially designed imaging setups. Gilvert and Pernicka [4] and Treibitz and Schechner [13] show that circularpolarizers can be used for reducing back-scattering from thescene. The techniques are shown to be useful; however, themethods are limited to scenes where scattered light rays arewell polarized. Narasimhan et al. [10]’s method sharpens
images of a scene in muddy liquid by precisely modeling of
single scattering, which works well for the scenes that do
not contain multiple scattering. Fuchs et al. [3] use a projector for descattering. Kim et al. [5] propose a method for
sharpening transmissive images using angular information
obtained by using a light field camera. Wu et al. [15] use
a femto-laser and a streak camera for image decomposition
to achieve a similar goal.
In this paper, we propose a Parallel High-frequency Illumination method which sharpens transmissive images by
separating the transmissive and scattered lights. We show
in this paper that transmissive and scattered lights can beseparated based on angular and positional clues. We thenshow that the original high-frequency illumination method
proposed by Nayar et al. [11] and several extended methods [6, 8, 9] can be uniformly explained as a separation of
the overlapped and non-overlapped rays. We finally evaluate the effectiveness of the proposed parallel projection approach to separating transmissive and scattered lights.

light source

light source

light source

light source

polarizer

probe
analyzer
camera(a) Polarization

camera(b) Angle

camera(c) Positiont(d) Time

Figure 2. Differences between transmissive and scattered lights.

2. Related Work
X-ray imaging is used for obtaining clear transmissive
images, because of its high-transmissive and low-scatteringcharacteristics. However, it is known that X-ray have a riskof radiation exposure, and alternative imaging techniques
are wanted. As a safe imaging technique, near infra-red
light is used for living body imaging because of its hightransmittance. Vein authentication for security purpose is
one example of this type of imaging. In the medical field,
the difference in the absorption properties of near infra-red
light between oxy-hemoglobin and deoxy-hemoglobin areused to measure blood oxygen levels [1]. Our goal is toobtain sharp images by removing scattering lights by using
visible or infra-red light and special optics.
In the optical measurement field, parallel light is traditionally used. Shadowgraphy [12] is a volumetric density distribution imaging method based on the detectionof slight directional change of light rays caused by differences of refractive index in the medium. Although schlieren
method [12] is similar to shadowgraphy, it cuts some parallel rays at their focal point with a knife edge and can visualize the density distributions with higher contrast. Hence,the schlieren method has been used for scattering measurements [2]. Recently, schlieren photography with light fieldprobes [14] has been proposed to produce ray-to-ray correspondences.
In this work, we also use the observation that a parallel
ray changes its direction because of scattering, and the optical setting is also similar. However, while shadowgraphy
and schlieren methods use only the directional clue, we use
both the directional and positional clues by combining themwith high-frequency illumination. Therefore, we can separate the transmissive and scattered lights more effectively.

3. Parallel High-frequency Illumination
3.1. Transmissive Images
Transmissive image such as those in Fig. 1 can be obtained by placing a camera and a light source on opposite sides of the translucent object. The lights observed by
the camera contain two components. One is the transmissive light, which passes straight through the scattering object. The other is the scattered light, which has various pathchanges caused by collisions with tiny particles. While theformer component shows the internal structures in a translucent object directly, the latter component disturbs the clarityof the observation. To obtain a clear transmissive image, weneed to eliminate the scattering lights and extract only thetransmissive lights.
Before separating the transmissive and scattered lights,
we must clarify the differences between the two components and determine the clues that can be used for our purposes. We summarize several clues for separation of thetwo components and introduce some conventional methods
in the following.
Polarization clue
When the incident light is polarized, the transmissive andscattered lights have different polarization characteristics,
as shown in Fig. 2 (a). While transmissive lights maintaintheir polarization because they are not affected by the object, the polarization can be gradually lost because of thecollisions with tiny particles. Therefore, scattered lights
which have bounced several times become only partiallypolarized, and the polarization is perfectly lost after multiple repeated bounces. Based on this difference, the scattered light can be reduced by the use of two co-linear polarizers [13].

Angle clueparaboloidal mirror

projector

Lights are emitted from an observed point to various directions, as shown in Fig. 2 (b). This emission angle is different for the transmissive and scattered lights. Because thepath of the transmissive light does not change when passing
through the object, the emitted light has the same direction
as the incident light. However, the scattered light path variously changes by collisions, and thus the emission direction also changes. Based on this difference, Kim et al. [5]have estimated the intensity of the transmissive light fromangular observation using a light field camera. Brogili etal. [2] have applied the schlieren method, which obscures
the transmitted ray by using a knife edge, to measure thescattering lights.
Position clue
Scattered and transmissive lights have difference about
emission positions, as shown in Fig. 2 (c). While the transmissive lights are emitted on the same line as the incident
light, the scattered light is emitted from various positions
because they are spread by the scattering objects. If the incident light is a narrow beam, then the transmissive light isobserved as a pointed peak. When the incident point of thebeam shifts slightly, the position of the observed transmissive light also shifts. However, the scattered lights spreadsmoothly, and a slight shift in the incident beam does notaffect to the observation.
Other clues
Other clues have been also used for the separation. For example, the path length of scattered light rays obviously becomes longer than that of transmissive light rays, as shownin Fig. 2 (d). Therefore, the transmissive lights arrive earlier than the scattered lights. By using an ultra high speed
sensor such as a streak tube, the two components can beseparated based on this time clue [15]. A wavelength clue
can also be used. Because the scattering and absorption depend on the wavelength, multi-band illumination is used for
deblurring [7].

3.2. Parallel projection system
Our aim is to sharpen the transmissive images effectively
based on the differences between the transmissive and scattered lights. As explained above, there are several clues
that can help to detect the separation of these lights. However, there is no single perfect way. For example, the angle clue cannot be used to distinguish transmissive lightsfrom straight-line scattered lights. Similarly, the position
clue cannot be used to distinguish transmissive lights fromscattered lights that are emitted from the same position.

projector

cameracameraparaboloidal mirror(a) Telecentric lenses(b) Paraboloidal mirrors

Figure 3. Optical design for parallel projection.

To overcome this problem, we use a combination of boththe angle and the position clues. To combine the different
types of clues, we have designed a special optical systemthat can illuminate and observe high-frequency patterns inparallel, as shown in Fig. 3 (a). This system can also easily
combine the polarization and wavelength clues by using apolarizer and a near infra-red light source, and thus we canuse many differences as possible.
Two optical designs are used to realize the parallel highfrequency illumination. The first uses telecentric lenses, as
shown in Fig. 3 (a). Although telecentric lenses are expensive and the viewing area becomes narrow, a variety oflenses are commercially available and this design is easy
to configure. The other design uses paraboloidal mirrors,
as shown in Fig. 3 (b). Although paraboloidal mirrors are
low cost and the viewing area can easily be extended, it isnecessary to precisely align the paraboloid’s focal point to
the optical centers of both the camera and the projector. In
practice, we can select the design according to the size ofthe scene, the cost, and the difficulty of the setup.

3.3. Decomposition of transmissive and scattered
lightsLet us assume that L[c] is the light intensity observed bya camera at pixel c. L can be expressed as
L[c] = Lt [c] + Ls [c],(1)where Lt [c] and Ls [c] are the intensities of the transmissive
and scattered lights at pixel c of the camera respectively.
Our method separates the transmissive and scattered
lights by projecting effectively small checkered pattern, asimilar way to the original method [11], but in parallel.
When the phase of the projected pattern changes slightly,
Lt also changes, but Ls rarely changes. Therefore, we canseparate the light components based on this difference. We
assume that the numbers of white and black pixels of theprojected pattern are same. When the projected pattern isshifted, the maximum intensity Lmax [c] and the minimum
intensity Lmin [c] of all observations are expressed as
1
Lmax [c] = Lt [c] + Ls [c],21
Lmin [c] = Ls [c].
2cameraprojector

camera(2)
object(3)(a) Direct reflection

projector(b) Specular reflection

From these relations, the transmissive and scattered
lights can be computed as
Lt [c] = Lmax [c] − Lmin [c],(4)

Ls [c] = 2Lmin [c].

(5)

We call this separation method the Parallel High-frequency
Illumination method.

cameraprojector(c) Planar single scattering

cameraprojector(d) Volumetric single scattering

4. Separated components
4.1. Overlapped and non-overlapped rays
Nayar et al. [11] have proposed the original highfrequency illumination method that separates the direct andglobal components. However, the photometric factors that
directly correspond to the direct and global components depend on the settings of illumination and observation. Every separation method has a different scheme for patternprojection. The separable components are also different.
However, we found that these methods share a commonality. In fact, these methods can be uniformly defined asa method for separation of overlapped and non-overlapped
light rays. If we need to separate certain components, then
we only have to design a non-overlapping projection system
for these components.

Figure 4. Light extraction by high-frequency illumination.

Separation of diffuse and specular reflections
Lamond et al. [6] show a separation method for diffuse andspecular reflections. They projected a stripe pattern on to ahemispherical screen surrounding the object and observed
its reflection, as shown in Fig. 4 (b). In this case, the directcomponent corresponds to the specular reflection and therays do not overlap with each other.
Separation of single and multiple scatterings in a planeFirst, we show how the direct and global componentschange when the illumination and observation settingschange in normal perspective projection.

Mukaigawa et al. [8] have separated single and multiple
scattering components in a thin planar translucent object.
They projected a stripe pattern from the side of the object
and observed vertically, as shown in Fig. 4 (c). In this case,the direct component corresponds to the single scattering.
The important point here is that the target is limited to aplane, and thus each ray is non-overlapped.

Separation of direct and indirect reflections

Separation of single and multiple scatterings in a volume

The direct components in the original method [11] are thediffuse and specular reflection, while the global components
are inter-reflection, subsurface scattering, and volumetricscattering. The red line in Fig. 4 (a) is an example of lightpath from the projector to the camera. This red line does
not overlap with the other paths indicated by the blue line.
Because these light paths do not overlap with each other,
the high frequency components remain.

Mukaigawa et al. [9] have extended their previous method
for volumetric object. They swept the stripe pattern andobserved each depth independently, as shown in Fig. 4 (d).
In this case, the direct component corresponds to the single
scattering. If a two-dimensional pattern is projected on a
volumetric scene, the single scattering light rays are overlapped in the captured image. This method avoids the overlapping problem by observing each depth independently.

4.2. Perspective projection

projector

projector

Q
r(Q)

Q
ro

Cp

Co

E

E
cameracamera(a) Perspective projection(b) Parallel projection

Figure 5. High frequency illumination to a translucent object.

4.3. Parallel projection
Let us assume a simple setting in which a projector anda camera are placed face to face, as shown in Fig. 5 (a). If
a normal projector and a normal camera are used for highfrequency illumination, the transmissive rays cannot be extracted as a direct component. This is because the transmissive rays are overlapped. Let us consider a light ray E,
which corresponds to a pixel Q in the projection pattern.
This ray can be expressed as
E = Q + t · r(Q),where ro denotes the direction of the ray, which is the sameas the projection direction. It should be noted that ro does
not depend on the pixel Q. The coordinates in the captured
image are expressed asCo(7)

where P denotes the perspective projection matrix of thecamera. This shows that a pixel in the projected pattern isobserved as a line in the captured image. Therefore, there
is no one-to-one correspondence between the projected pattern and the captured image. For example, red and bluelines in the projected pattern produce two rays which do not
overlap in the 3-D volume, as shown in Fig. 5 (a). However,
the two rays overlap with each other in the image capturedby the camera.
On the other hand, parallel projection produces different relationships. Let us assume that the projector and thecamera are arranged to produce a coaxial orthographic projection, as shown in Fig. 5 (b). In this case, a ray E can be
expressed as
E = Q + t · ro ,(8)=

O(Q + t · ro )=OQ + t · Oro ,where O denotes the orthographic projection matrix of thecamera. Here, Oro = 0 because ro is degenerated by the
projection. Therefore,
Co = OQ.

(6)

where r(Q) denotes the direction of the ray, which depends
on the pixel Q, and t is a parameter. This ray is captured bya camera, and the captured image Cp is expressed as
Cp = P(Q + t · r(Q)),

Figure 6. Parallel projection system.

(9)This shows that the ray is projected as a point in the captured image. For example, the red and blue pixels produce
two parallel rays, and they are observed as being different points without overlapping. Therefore, the transmissive
and scattered lights can be separated as non overlapped andoverlapped rays, respectively. This is the core essence of
our Parallel High-frequency Illumination method, which isbased on both the angle and position clues.

5. Experiment5.1. Measuring system
We have constructed a parallel projection system using
telecentric lenses, as shown in Fig. 6. We have used thedigital micro-mirror device (DMD) projector developmentkit (Texas Instruments LightCommander) with a telecentriclens (Edmund Optics). This projector has an infra-red (850nm) light source along with visible RGB light sources.
The CCD camera (Point Grey Grasshopper2) which has
sensitivity to the near infra-red light, also has a mounted
telecentric lens. Linear polarizers are placed in front ofground truth
visible light
infra-red
proposed method

Figure 8. Comparison of intensity profiles.
Table 1. Normalized cross correlation.

Normal illuminationin visible light-0.10

Normal illuminationin near infra-red
0.68

Proposedmethod0.95

both lenses to reduce scattered lights. Using this equipment,
we have realized parallel high-frequency illumination in thenear infra-red region.
For the high-frequency illumination, a 9[pix]×9[pix]checker pattern is projected. We shift the pattern for one
sixth phase and capture 18 images. This size provides a
threshold to determine the allowed spread of the transmissive lights. The appropriate size depends on the object size,
the resolutions of both the camera and the projector, and theoptical density. The automatic selection of the most appropriate size is a part of our future work.

5.2. Experiment with diluted milkFirst, we have confirmed the descattering effect by the
proposed method. The target object is a metallic wire, as
shown in Fig. 7 (a). We regard a clear transmissive image
which is captured when the target object is in pure water asa ground truth, as shown in Fig. 7(b). Then, a small amountof milk is poured into the water. The density of the diluted
milk is 2.2%.
We use two wavelengths of visible (525 nm) and nearinfra-red (850 nm) light for comparison purposes. Figure 7 (c, d, e) show the experimental results. The image in
Fig. 7(c) shows a normal transmissive image in visible light.
Because the light is strongly scattered, it is hard to recognize the object. The image in Fig. 7(d) is a normal transmissive image, but in the near infra-red region. The use ofnear infra-red light contributes to sharpening of the image,but it is insufficient. The image in Fig. 7(e) shows the resultof our proposed method. This confirms the image sharpening effect, while it also contains some artifacts due to pattern projection. Figure 8 shows a comparison of the inten-

sity profile of each image along the red line in Fig. 7 (b).
We can see that the image from the proposed method has
sharp edges and it is similar to the ground truth image. Table 1 shows the result of normalized cross correlation withthe ground truth image. The proposed method produces thehighest value, which confirms the nobility of it.
Next, we confirm the descattering effect for various milk
densities. we use 1.9%, 2.2%, 2.5%, 2.8%, and 3.1% milk
density values for our experiment. Figure 9 compares theresults of normal illumination with near infra-red light andthe proposed method. As the normalized cross correlation
with the ground truth is computed for each image. As thedensity of the solution increases, hence clarity of the transmissive images decreases. All images between 1.9% and2.8% are clearly sharpened by the proposed method and thecorrelation value increases. However, at 1.9% density, thenormal image is still clear but the result is actually worse,
because it contains noise caused by the pattern projection
process. On the other hand, for 3.1% density, it is difficult
to confirm good result because the level of the transmissive
light is the same as the level of the camera noise.

5.3. Combination with polarization analysis
Polarizers used in our system are fixed to co-linear.
However, our method can be combined with traditional polarization analysis methods. That is, two patterns are captured in co-linear and cross settings, and the difference isused to reduce scattering. Figure 10 is a comparison of
our static method and the combined method. The effect ofthe polarization analysis is much smaller than our proposed
method. Although the polarization analysis can be easily
combined, the quality is not improved and measuring time
becomes two times longer.

5.4. Experiments with several scenes
Finally, we apply our method to several scenes, as shownin Fig. 11. We can clearly see the pitch of the screw, thesmall veins of leaf, and smart card by using the proposed
method.

5.5. Discussions and limitations
We confirm that the proposed method is effective forsharpening of transmissive images via several experiments.
However, some problems still exist.
First, sometime the image contains large amount ofnoise. Depending on the particular object, the ratio of transmissive light and scattered light can be quite small. In sucha case, the level of the transmissive lights is nearly sameas the observation noise level. Therefore, we must have toapply a denoising technique or use a cooled CCD camera.
However, this problem is essentially difficult to solve.
Second, the results are degraded if the object surface is(a) Target object(b) Target object isin pure water(c) White patternof visible light(d) White patternof near infra-red(e) Proposed method

Figure 7. Experimental results.

densitycorrelation

1.9%
0.98

densitycorrelation

1.9%
0.95

2.2%
2.5%
2.8%
0.68
0.30
0.06
(a) Normal illumination of near infra-red

2.2%
2.5%
0.95
0.84
(b) Proposed method

2.8%
0.21

3.1%
0.01

3.1%
0.03

Figure 9. Results for various densities.

0.916
(a)

0.937
(b)

0.988
(c)

0.988
(d)

with the same refractive index as the object.
Finally, this method cannot be applied to scenes where
there are no transmissive lights. It is therefore impossible tovisualize the internal organs of the human body using nearinfra-red light.

6. Conclusion
Figure 10. Comparison with polarization analysis. Values under
the images show cross-correlations with the ground truth.

(a) Normal illumination (b) Traditional polarization analysis (c) Our proposed method (fixed polarizers) (d) Combined method
not planar. When the surface is round, the light rays refract and break their parallel path. To solve this problem,
we must have to devise a method to prevent refraction, e.g.
index matching by submerging the target object in a liquid

In this paper, we have proposed a new method for the
separation of transmissive and scattered lights to sharpen
the transmissive image. We have explained the principlesof the high-frequency illumination method and the importance of avoiding overlapping rays. We have proposed a
new optical design for Parallel High-frequency Illuminationand have developed a special illumination and observation
system using telecentric lenses. The effectiveness of the
sharpening process has been confirmed by several experiments and numerical evaluations.

Target objects

Infra-red image

Proposed method

Figure 11. Results for several different scenes.

Acknowledgment
This research is granted by the Japan Society for the Promotion of Science (JSPS) through the “Funding Program
for Next Generation World-Leading Researchers (NEXT
Program),” initiated by the Council for Science and Technology Policy (CSTP).

References[1] D. Boas and M. A. Franceschini. Near infrared imaging.
Scholarpedia, 4(4):6997, 2009.
[2] D. Brogioli, A. Vailati, and M. Giglio. A schlieren method
for ultra-low-angle light scattering measurements. Europhysics Letters, 63(2):220–225, 2003.
[3] C. Fuchs, M. Heinz, M. Levoy, H.-P. Seidel, and H. P. A.
Lensch. Combining confocal imaging and descattering.
Computer Graphics Forum, Special Issue for the Eurographics Symposium on Rendering 2008, 27(4):1245–1253, 6
2008.
[4] G. D. Gilbert and J. C. Pernicka. Improvement of underwater visibility by reduction of backscatter with a circularpolarization technique. Applied Optics, 6(4):741–746, 1967.
[5] J. Kim, D. Lanman, Y. Mukaigawa, and R. Raskar.
Descattering transmission via angular filtering. Proceeding
ECCV’10 Proceedings of the 11th European conference on
Computer vision, pages 86–99, 2010.
[6] B. Lamond, P. Peers, and P. Debevec. Fast image-based separation of diffuse and specular reflections. In ACM SIGGRAPH 2007 sketches, SIGGRAPH ’07, New York, NY,
USA, 2007. ACM.

[7] N. Miura and Y. Sato. Removing skin wrinkles and deblurring veins by using tri-band illumination. Asian Conference
on Computer Vision (ACCV2012), 2012.
[8] Y. Mukaigawa, R. Raskar, and Y. Yagi. Analysis of lighttransport in scattering media. Computer Vision and Pattern
Recognition (CVPR), 2010 IEEE Conference on, pages 153–
160, 2010.
[9] Y. Mukaigawa, R. Raskar, and Y. Yagi. Analysis of scattering
light transport in translucent media. IPSJ Transactions on
Computer Vision and Applications (CVA), 3:122–133, dec
2011.
[10] S. G. Narasimhan, S. K. Nayar, B. Sun, and S. J. Koppal.
Structured light in scattering media. Computer Vision, 2005
ICCV 2005 Tenth IEEE International Conference on, pages
420–427, 2005.
[11] S. K. Nayar, G. Krishnan, M. D. Grossberg, and R. Raskar.
Fast separation of direct ang global components of a sceneusing high frequency illumination. Proceeding SIGGRAPH’06 ACM SIGGRAPH 2006 Papers, pages 935–944, 2006.
[12] G. S. Settles. Schlieren and Shadowgraph Techniques: Visualizing Phenomena in Transparent Media. Springer, 2001.
[13] T. Treibitz and Y. Y. Schechner. Active polarization descattering. IEEE Transactions on Pattern Analysis and MachineIntelligence, 31(3):385–399, 2009.
[14] G. Wetzstein, R. Raskar, and W. Heidrich. Hand-heldschlieren photograpy with light field probes. IEEE Conference on Computational Photography (ICCP), 2011.
[15] D. Wu, M. O’Toole, A. Velten, A. Agrawal, and R. Raskar.
Decomposing global light transport using time of flight
imaging. Computer Vision and Pattern Recognition (CVPR),
2012 IEEE Conference on, pages 366–373, 2012.