Recovering Inner Slices of Translucent Objects by Multi-frequency Illumination
Kenichiro Tanaka1,2
Yasuhiro Mukaigawa2
Hiroyuki Kubo2
Yasuyuki Matsushita1,3Yasushi Yagi1
1
2
Osaka University
Nara Institute of Science and Technology 3 Microsoft Research Asia
http://www.am.sanken.osaka-u.ac.jp/˜tanaka/projects/multi-freq.html

Abstractnique [5], and sharper results can be obtained differently
from infrared reflectography [9].
Our method exploits the fact that the spread of light
due to scattering has dependency on the depth of inner
layer where light rays are emitted. By modeling the lightspreads as depth-dependent point spread functions (PSFs)
and utilizing their depth-dependent low-pass characteristics, we develop a method for recovering inner layer appearances from a set of images taken under a variant ofhigh-frequency illumination [26]. Specifically, our method
uses a spatial pattern projection with varying the pattern
pitch – we call this multi-frequency illumination. Our multifrequency illumination method allows us to separate direct(high-frequency) and global (low-frequency) components
as in [26], yet at various frequency levels that define highand low-frequencies. Our method uses the direct component observations for recovering the appearance of innerslices, which are related to the direct components via depthdependent PSFs.
The key contributions of this paper are twofold. First,
we describe the relationship between depth inside a translucent object and its PSF by a physically motivated scatteringmodel. Second, using the relationship, we develop a method
for recovering the appearance of inner slices using varying
pitch pattern projection. We implement a coaxial projectorcamera setup for measuring real-world objects, and show
the recovery results using oil painting, mural, and paper
scenes.

This paper describes a method for recovering appearance of inner slices of translucent objects. The outer appearance of translucent objects is a summation of the appearance of slices at all depths, where each slice is blurredby depth-dependent point spread functions (PSFs). By exploiting the difference of low-pass characteristics of depthdependent PSFs, we develop a multi-frequency illumination
method for obtaining the appearance of individual innerslices using a coaxial projector-camera setup. Specifically,by measuring the target object with varying the spatial frequency of checker patterns emitted from a projector, ourmethod recovers inner slices via a simple linear solution
method. We quantitatively evaluate accuracy of the proposed method by simulations and show qualitative recovery
results using real-world scenes.

1. Introduction
Translucent objects have complex appearance. It is a superposition of light rays emitted from inner slices at every
depths, blurred by subsurface scattering. Because seeing internal appearance of objects is of broad interest in medical
and art analyses and industry inspection, various imaging
techniques have been developed in the past. In particular,
since the translucency effect becomes significant for many
materials in near infrared (NIR) wavelengths, infrared photography is used as one of common techniques for achieving
this goal. For example, it is used for observing inner layersof oil paintings that tell us the drawing technique, growth,
history, and/or authenticity of old age painters.
One of the major challenges in observing inner layersof translucent objects is to separate inner appearances withproperly dealing with scattering. To overcome this problem,
we develop a multi-frequency illumination method, which
can recover sharp appearance of inner slices at a desired
depth with explicitly removing scattering blur. Comparedwith conventional techniques that aim at a similar goal, ourmethod is faster and safer than the X-ray fluorescence tech-

2. Related work
As our work aims to recover slices from superposed observation, layer decomposition techniques [29, 27, 19] fromsuper-imposed images are related. These methods are designed for scenes with semi-transparent layers that do notexhibit multiple scattering; thus, the image formation modelis represented by simple alpha blending. Our method isdesigned for translucent scenes where significant multiple
scattering is observed.
Our work is related to descattering methods in computational photography in that our goal is to recover sharp slices
1inside a scattering medium. Narasimhan et al. [25]’s and
Gu et al. [10]’s methods sharpen images of a target scene inmuddy liquid by precisely modeling single scattering. Theirmethods work well for those scenes that do not exhibit multiple scattering. Fuchs et al. [8] use multiple cameras andprojectors for descattering using a confocal imaging technique for the purpose of recording a solid target in scattering media. Duran et al. [6] descatters a target pattern inside translucent media by combining pattern projection andcompressive sensing approach. Our method aims at recovering slices inside translucent objects at arbitrary depths.
Our method falls in a class of active sensing techniquesthat use high-frequency pattern projection. The original Nayar et al. [26]’s method and its extended methods [18, 21,
22, 30, 2, 1] separate direct and global components by projecting multiple high-frequency patterns. Gupta et al. [11]acquire scene depths with direct-global separation by modeling both projector’s defocus and global light transport. We
also use a pattern projection technique not only for separating scattering effects but also for recovering outer and innerslices.
Methods for measuring transparent or translucent media,
such as smoke scanning, are also related to our work. Morris et al. [20] recover the shape of clear transparent materials
that refract light by analyzing light rays recorded from different viewpoints. Hawkins et al. [13] acquire the density
distribution of participating media by laser scanning, which
does not contain multiple scattering, such as smoke. Ihrke
and Magnor [15] reconstruct the volume of dynamic semitransparent media by a visual-hull based method. While related, our goal is to recover slices inside translucent objects
instead of reconstructing 3-D shape of an object’s surface.
In other fields, there have been independent developments of technologies for imaging internal structures of target objects. In art analysis, several techniques have beendeveloped for imaging hidden layers of paintings. Infrared
reflectography [9] and X-ray transmission radiography [5]can visualize internal layers of paintings, although the surface texture cannot be separated. X-ray fluorescence technique [5] uses spectroscopic information measured over a
couple of days and estimates the metallic atom distribution
for determining colored appearance of inner layers. Terahertz imaging [3] is an another technique that can see inner
layers of paints. In the medical field and its related areas,
optical coherence tomography [14, 7] techniques are widelyused, especially for visualizing retina. These techniques
enable visualizing inside translucent objects using interference of coherent light at the cost of expensive wavelengthorder optics and mechanics. In contrast to these method,
our method uses a commodity camera and projector for recovering slices inside translucent objects, resulting in an inexpensive setup. In microscopy, there are methods that use
pattern projection for visualizing inside substances, or re-

(a)

(b)

Figure 1: Appearance of translucent objects. (a) Captured
image contains all components from each depth. (b) Lights
spread depending on the depth.
constructing 3-D shape of a small sample, such as protein
structure [12, 17, 31]. They sharpen the microscopic image
by precisely taking into account both micro-scale opticalblur and scattering blur. Our aim is to develop a techniquethat is applicable to a standard scale, where multiple scattering is significant.

3. Appearance of translucent objects
When an image of a translucent layered object isrecorded, the observed intensity can be modeled as a summation of the appearance of multiple depth layers as illustrated in Fig. 1(a). Let Sd be the appearance slice of thelayer at depth d. The observed intensity Lo (c) at camera
pixel c can be expressed byX
Lo (c) =Sd (c)(1)din a discrete form.
The appearance slice Sd is generally blurry due to thescattering effect inside the medium. The spread of radianceat a scene point inside a translucent object varies depending
on its depth from the object surface [28]. In general, the spatial spread of light can be expressed using PSFs. Let us consider light rays emitted (or returned) from a specific depth
inside a translucent object. When the depth is shallower,
the PSF becomes sharper. On the other hand, it gradually
spreads as the depth d becomes deeper inside the medium
as illustrated in Fig. 1(b). In this manner, there is a close
relationship between the PSF and depth. By denoting hd asa PSF at depth d, the appearance slice Sd at depth d can be
expressed as
Sd (c) = (Rd ∗ hd )(c),(2)

where Rd is the sharp slice that we are interested in estimating, which we call a radiance slice, and ∗ denotes a convolution operator.

Since the appearance of the translucent object under normal illumination is a superposition of radiance of multiple
layers as Eq. (1), the observation Lo can be re-written asX
Lo (c) =(Rd ∗ hd )(c).
(3)
d

Our goal is to recover radiance slices Rd from the composite
observation Lo . Before introducing the solution method, we
describe a model of depth-dependent PSFs hd .
Depth-dependent PSFs We represent the depthdependent PSFs hd by a physically motivated scatteringmodel, the radiative transfer equation (RTE) [4, 16]. With
the RTE model, the intensity of an arbitrary light ray
in a homogeneous scattering medium can be iteratively
calculated [23] asI(d, θ) =∞X(gk (T ) + gk+1 (T )) Lk (cos θ),

(4)

k=0

where Lk is the Legendre polynomial of order k, g0 = 0,and
2k + 1k−1(1 − q)T − (k + 1) log T .
gk (T ) = I0 exp −k
The parameter T (= σd) represents optical thickness, whichis the product of extinction coefficient σ of scattering media
and distance d from the point light source to the scattering point. The forward scattering parameter q controls how
light rays spread; q = 0 corresponds to isotropic, while
positive and negative values of q indicate forward and backscattering, respectively. θ is the angle between the depth
axis and light ray’s direction as depicted in Fig. 2(a). k is
the number of light bounces, and I0 is the intensity of thepoint light source.
In our problem setting, a camera is placed outside themedia. When we consider the x-axis is on the surfaceof translucent object as illustrated in Fig. 2(b), the depthdependent PSF hd (x) can be written as
hd (x) = I (d0 , φ) ,0√

x2 + d2 and φ = tan−1

(5)
x
d



where d =. In this work,
we assume the camera is placed far enough from the object compared to both d and x; therefore, the direction ofemitting light rays from the surface becomes parallel to thedepth axis. Moreover, we ignore the refraction of the surface for simplicity.

4. Proposed Method
We are interested in recovering radiance slices Rd fromthe mixed observation Lo . To achieve this goal, we develop

(a)

(b)

Figure 2: Illustration of RTE model described in [23]. (a)Black arrow shows an scattering light ray, whose intensity
depends on d and θ. d is the distance from the point lightsource I0 inside scattering media and θ is the radial direction of the light ray. (b) Depth-dependent PSF hd can be
expressed using RTE model. We consider the x-axis is on
the surface of translucent object and assume the direction ofemitting light rays from the surface becomes parallel to thedepth axis.

a multi-frequency illumination measurement method, which
is built upon the high-frequency illumination (HFI) method
proposed by Nayar et al. [26]. To begin with, we briefly
review the original HFI method.
High-frequency illumination method [26] The HFI
method separates direct and global components by projecting small pitch checker patterns. When the phase of the projection pattern changes slightly, the direct component D(c)varies accordingly, but the global component G(c) remains
stable. Based on this observation, their method computes
direct and global components using the maximum Lmax (c)and minimum Lmin (c) intensities that are obtained by shifting the projector pattern asD(c) = Lmax (c) − Lmin (c),(6)G(c) = 2Lmin (c).
The direct component D(c) contains high-frequency components, while the global component G(c) contains only
lower frequency components than the frequency of projection pattern. Therefore, the HFI method can be viewed asa separation technique for high- and low-frequency components.
Pattern pitch of HFI In our case, when a translucent
object is measured under HFI with pattern pitch p, we
can obtain direct component D(p, c) and global componentG(p, c) asP D(p, c) =Pd Dd (p, c)G(p, c) =(7)
d Gd (p, c)
Sd (c) = Dd (p, c) + Gd (p, c),where Dd (p, c) and Gd (p, c) are the direct (high-frequency)
and global (low-frequency) components at depth d, respectively.

When the projected pattern pitch p changes slightly, intensities of direct and global components also change. In
fact, there is an important relationship between the pitch
size and strength of direct component. Suppose we measure the scene with two variations of pattern pitches pi andpj (pi < pj ) individually. When the pattern pitch becomes larger, wider spread of light rays are included in the
direct component, hence the corresponding direct components have the following relationship:
Dd (pi , c) < Dd (pj , c),pi < pj .

(8)This relation indicates that the separation frequency varies
with the pattern pitch – wider frequency band is regarded as
the direct component as the pattern pitch becomes larger.
As low-pass characteristics of depth-dependent PSFs arealso depth-dependent, the difference of direct components
Dd (pj , c) − Dd (pi , c) also varies with depth d. Our method
exploits these differences for recovering each inner slice bychanging the projection pattern pitch.

4.1. Multi-frequency illumination
By measuring the target object using multi-frequency
patterns, corresponding multiple direct components are obtained. Unfortunately, increasing the number of measurements does not make the problem easier as it also increases
the number of variables to solve for. To make the problem
tractable, we assume that the texture of direct components
does not vary drastically when the pattern frequency is highenough and the pitch variation ∆p (= pj − pi ) is sufficiently
small. Namely, we assume that direct component images
Dd (p, c) have linear relationship within a small pitch variations of pi and pj as
Dd (pi , c) ≈ α∆p Dd (pj , c),(9)where α∆p is the ratio of the mean brightnesses of two individual images. At the same time, these direct components
Dd at a certain depth d are supposed to have a similar texture with the original radiance; therefore, the following relationship holds:
Dd (p, c) ≈ α(hd , p)Rd (c),(10)

where α(hd , p) is a relative brightness of Dd (p, c) to Rd (c).
We call α(hd , p) the direct component ratio that represents
the ratio of direct component’s mean intensity to the radiance Rd (c)’s mean intensity. Hence, Eq. (7) can be rewritten asX
D(p, c) =α(hd , p)Rd (c).
(11)
dWith these assumptions, a set of direct component images D(p, c) taken under the multi-frequency illuminationof m pitch variations (p = p0 , p1 , . . . , pm−1 ) can be written
in a matrix form as
D(c) = AR(c),

(12)

where
D(c)

=





A = 
R(c)

=



D(p0 , c) D(p1 , c) · · ····..
.
α(hd0 , pm−1 ) · · ·α(hd0 , p0 )..
.

D(pm−1 , c)α(hdn−1 , p0 )
..
.

T

,



,

α(hdn−1 , pm−1 )T
Rd0 (c) Rd1 (c) · · · Rdn−1 (c).

Here, D is a vector of direct components measured under mvariations of the pattern pitches at pixel c, R is a vector of nlayers of radiance slices, and A is a matrix containing directcomponent ratios of direct components computed from the
projected pattern pitch and the depth-dependent PSF.
When the number of projected patterns m is no less than
the number of depth layers n (m ≥ n) and rank(A) = n,
the radiance slices R can be obtained in a least-squares
sense by computing the pseudo-inverse A+ as
R(c) = A+ D(c).

(13)

Computation of direct component ratio The directcomponent ratio α(hd , p) can be derived from the depthdependent PSF hd and the projected pattern pitch p. Whenchecker pattern is projected to a translucent object, it
reaches at the depth d with the blur effect, and returns tothe surface with the blur effect again. To obtain the directcomponent ratio, we simply consider the difference of maximum and minimum intensities (as original HFI) under simple conditions: the reflectance of the layer is constant, andillumination is normalized. The direct component ratio canthen be obtained asα(hd , p) = max ((lp ∗ hd ) ∗ hd )− min ((lp ∗ hd ) ∗ hd ) ,(14)

where lp is normalized projection patterns, whose pitch is p.
Normalized illumination is defined as that 1 to be white and
0 to be black. max and min operators return a maximum
and minimum value from all pixels, respectively.

4.2. Estimation of informative slices
Once we determine the depths to recover (or PSFs), we
can set up a matrix A using Eq. (14), and thus can recoverslices corresponding to the PSFs using Eq. (13). Selecting
a good set of depths is important for recovering informative slices. For example, if an arbitrary depth is chosen, it
may correspond to the middle of distinct texture layers. To
recover informative slices, we use a two step approach todifferences because of the following step 2. Therefore, forefficient parallelization, we choose the per-pixel implementation.

(a)

(b)

(c)

Figure 3: Selection of informative PSFs (corresponding todepths). We select useful PSFs from a lot of candidate PSFs.
(a) Results of the optimization problem (Eq. (15)). Nonzero pixels indicate the informative ones. (b) We find local
maxima of the non-zero counts for all pixels across depth.
(c) Selected informative PSFs.

determining the matrix A. The first step is the estimation of
a set of informative depths via optimization. This is equivalent to estimating the small number of useful PSFs out frommany possible PSFs. The second step is the recovery ofslices at the informative depths determined by step 1. The
flow of this strategy is illustrated in Fig. 3. Now, we explain
the details of each step.
Step 1: Estimation of informative depths Elements ofmatrix A depend on depth-dependent PSFs; hence estimation of informative depths corresponds to estimating theshape of PSFs. The depth-dependent PSFs depend not on
the absolute depth value but on the relative depth T , hence
we regard it as a depth value. The parameter T of the RTE
model are set from 1.01 to 10.01 step by 0.05. We add theoffset 0.01 because Eq. (4) does not converge if T ≤ 1.00.
Initially, we set up the matrix A using all parameters (thusm < n) in order to estimate informative slices.
Frequently, there are only small number of informativeslices inside translucent objects. Hence we can regard suchradiance slices exist sparsely along depth. Our method uses
this sparsity to determine informative slices by solving al1 regularized problem (as known as the lasso [32]) with a
non-negative constraint about R:
2

R̃(c) = argmin kAR(c) − D(c)k2 + λ kR(c)k1

(15)

R(c)

subject to R  0.
It becomes a quadratic programming (QP) problem and thus
can be efficiently solved in a polynomial time. We solve
the optimization in a per-pixel manner. Solving a similar
problem for an entire image is also a viable option; however, we have observed that these two does not make much

Step 2: Informative slice recovery We can regard thedepth d (= Tσ ), where R̃d (c) has a non-zero value, is informative while others are not. This step determines informative depth slices of the whole image by consolidating all the
pixel-wise selections. The informative depths d̃ are local
maximas of the sum of l0 norm of R̃d (c) for all pixels. The
evaluation function f (d) is defined asXf (d) =kR̃d (c)k0 ,(16)cand we find all local maxima of f (d) along d from 0 to
200 as shown in Fig. 3(b). Using the selected depths, we
can set up a small sized matrix Ã. Finally, we can recover
informative slices using the matrix Ã in a least-square senseasR(c) = Ã+ D(c)(17)

in the same way as Eq. (13).

5. Experiments
We evaluate our method numerically by simulation andalso show qualitative results using real-world scenes.

5.1. Evaluation by simulation
We first assess the validity of the approximation in
Eq. (10), and then evaluate the accuracy of the slice recovery.
Validity of approximation in Eq. (10) In this simulation,
we change the pattern pitch p from 3 to 20 pixels, the depth
d from 4 to 18 [mm] (in optical depths T , which correspondto 4σ to 18σ), and use 15 different slice textures. The scene
is one-layered for the purpose of assessing the approximation capability. The coefficient σ is set to 0.001 [mm−1 ],and forward scattering parameter q is set to 0.9. These parameters are chosen according to [24]. We generate the apϕpearance slices Sd,pusing checker pattern lpϕ as
ϕSd,p= ((lpϕ ∗ hd ) ◦ Rd ) ∗ hd .

(18)

p and ϕ are the pitch and phase of the pattern, and ◦ is the
Hadamard (element-wise) product operator. We compute
direct component Dd (p) from these synthetic images withchanging the phase ϕ.
We assess correlations between the recovered directcomponents Dd (p) and the ground truth slices Rd using
zero-mean cross correlation (ZNCC). The ZNCC value falls

Cross-correlation

1.000
0.995
0.990
0.985
0.980
0.975
T =4σ
T =12σ0.970
T=8
σT =18σ0.965
0.960
3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Pattern size (p, pixels)

Figure 4: Evaluation result of the approximation in Eq. (10).
We calculate the cross correlation between direct component slice Dd (p) and the ground truth slice Rd . Correlation
values are very high even with the worst case.

in the range from −1 to 1, where a negative value indicates negative correlation, and a greater value indicateshigher similarity. The evaluation results are summarized inFig. 4. The plots indicate mean correlations of all slice textures. ZNCC values decreases as the depth becomes deeper(greater T ) and pattern pitch becomes larger. However, they
are all consistently well correlated (minimum ZNCC value
is 0.982 and the mean value is 0.992), and it shows the accuracy of the approximation.

LayersTop layer
Bottom layer

ZNCC values
Max. Mean Min.
0.99
0.93
0.89
0.92
0.84
0.67

Table 1: 2-layer recovery result for 20 sets of scenes.
We compare recovered slices to the ground truth slices by
ZNCC. Higher ZNCC scores indicate more accurate recovery.

(a) Simulated scene. three star-shaped pillars with different
heights are placed in scattering media. An example slice is illustrated by the orange plane.
Ground truth slices

Recovered slices

Evaluation of slice recovery We also evaluate how accurately slices can be recovered by our method via simulation.
ϕ
We generate images Lϕ
p under projection patterns lp as
d=1

Lϕ
p =

X
((lpϕ ∗ hd ) ◦ Rd ) ∗ hd .

(19)

d=5

d = 15(b) Comparison of the recovered and ground truth slicesd

We use 20 different two-layered scenes that have distinct
textures R5 and R15 at two depth ranges 5 ≤ d < 15 and15 ≤ d. The textures are randomly paired from the ones
that are used in the previous experiment. We change thepitch p of checker pattern and shift the pattern with ϕ. Fromthe generated images, we compute D(p, c) for each pattern
pitch p, and apply our method to recover slices at d = 5 and15.
We again use ZNCC values between the recovered andground truth slices. The experimental results are shown in
Table 1. In all sets, the ZNCC values of upper layer is higher
than the lower layer as expected. The result of this simulation indicates that our method can recover slices of various
textures with high correlation.
Figure 5 shows a synthetic example of three-layer recovery, where slices at at d = 1, 5, and 15 are recovered. The
ZNCC scores for the recovery results are 0.98, 0.83, and0.51, respectively. While the result is generally consistent
with the ground truth, negative intensities and ringing artifacts are observed due to the discrete pitch variations andconvolution.

Figure 5: (a) Target scene (b) Result of three-layer recovery
at d = 1, 5, and 15. The ground truth radiances slices (upper) and recovered slices (lower) are shown. ZNCC scores
are 0.98, 0.83, and 0.51, respectively.

5.2. Real-world experiment
We develop a coaxial projector-camera setup for realizing the measurement setup as shown in Fig. 6. The coaxial
setup has a favorable property, i.e., correspondence between
projector and camera pixels becomes invariant with respect
to depths. Unlike non-coaxial settings, with which a illumination ray inside the translucent object forms a line in
the image coordinates [30], the coaxial setting allows us toeasily separate the direct rays. We use a “LightCommander” projector, which is a DMD projector development kit
by Texas Instruments, and use near infrared (NIR) light formeasurements. The lenses of both camera and projector are
set same (Ai Micro-Nikkor 105mm f/2.8S) for easy alignment. In the experiment, we use 18 variations of checker
patterns (3px to 20px with 1px interval), and shift the pattern for one-third of square size for each pattern.

cameraa pair of same lenses

beam splitter

projector(a) The scene(b) Draft layer(c) Oil painted

target object

Experimental results First, we use an oil painting as atarget scene as shown in Fig. 7(a), which has draft and surface pigment layers as depicted in Figs. 7(b) and 7(c). By
taking a standard photograph under the near infrared light(as done in infrared reflectography in the art field), we canvaguely observe the draft layer as shown in Fig. 7(d). Thisis, however, the superposition of draft and surface pigment
layers, hence it results in a blurry image. Therefore, a simple layer decomposition still suffers from the blur. We cannot see the shape of the draft tree clearly even if we manually adjust the contrast, which is essentially same as infrared
reflectography [9], as shown in Fig. 7(e). By applying our
method to this scene, two PSFs are estimated as depicted inFig. 7(h) and two slices are recovered as in Figs. 7(f) and
7(g). The upper surface layer corresponds to the surface
pigment layer. Because the yellow pigment is almost transparent in the infrared wavelength, the corresponding painting regions become dark in the surface slice. The lower
layer shows the inner layer, where the texture of the tree isclearly observed.
The second target scene is stacked papers that consistof layered translucent papers and printed texture films asshown in Fig. 8(a). In a normal NIR photograph, we canobserve mixed textures as shown in Fig. 8(b), where textures are blurry and appearances across depths are superpositioned. Our method correctly selects two informative
depths and recovers radiance slices as shown in Figs. 8(c)and 8(d). In the upper slice, only ‘ABCD’ texture is visible,and ‘1234’ texture appears in the lower slice. Due to theshadowing effect caused by the solid material on the upper
layer (the texture of ‘ABCD’), the lower slice is contains the‘ABCD’ texture as an unobserved shadowed region.
Additional results are shown in Fig. 9. The upper row isa piece of painting, where the painter’s signature is hidden
under pigment. Our method clearly recovers surface texture
and inner signature slices. The lower row is a mural painting covered by white mold. In this example, we used RGB(d) NIR photol0 -norm

Figure 6: Measurement setup. The coaxial system allows
us to maintain the correspondences between projector andcamera pixels.

10e4
8e4
6e4
4e4
2e4
0
0

(e) Baseline(f) Ours(surface slice)(g) Ours(inner slice)

2 selected PSFs
20σ

40σ

60σ

80σ

100σ

optical depth ( T )

120σ

140σ

(h) Selected PSFs corresponding to local maxima

Figure 7: Experimental result of oil painting. (a) Target
scene. We draw a colored round tree on top of the draft of
spiny tree. (b) Inner layer (draft) of the painting. (c) Painted
scene. Red rectangle region is measured. (d) Normal photousing infrared light. (e) Baseline result. Intensity range andcontrast of (d) are manually adjusted. (f, g) Results of our
method. Layer of surface texture and hidden drawing, respectively. Range of the intensities is adjusted for visualization. (h) Selected PSFs. There are two peaks in the plotof Eq. (16), hence two corresponding PSFs are selected torecover.

light sources instead of NIR, and record each color channel
separately. Our method can recover the slices in this example as well, and the inner appearance is clearly visible in the
result of Fig. 9(d).
Thickness estimation Our method can be applied to estimate the thickness of pigments by computing Eq. (16) ina patch-based manner instead of the entire image region.
As we do not know the scattering coefficient, the estimatedthickness is relative (but linear). Figure 10(a) is a scene,
where brown pigment is drawn on a white paper. We findthe relative depth of a pixel using a 9 × 9 image patch
centered at the pixel. The estimated thickness is shown in
Fig. 10(b), where the estimated relative thickness is shownin a pseudo color map.

(a) Target scene(b) Normal photo(c) Surface slice(d) Inner slice

Figure 9: Additional results for a painting and a mural. (a) Target scenes. Upper: author’s signature covered by red pigment.
Lower: A mural painting covered by white mold. Rectangle regions are measured. (b) Normal photograph. Both upper andlower slices, and global components are composed. (c) Recovered slices of surface. (d) Recovered inner slices. The signatureand inner painting are visible clearly.
85𝜎

65𝜎

(a) Non-smooth painting(a) The scene(c) Upper slice(b) NIR photo(d) Lower slice

Figure 8: Experimental result of layered scene. (a) Scene is
a composite object of texture and translucent sheets (tracing
paper). (b) Normal photo of the scene. Textures inside theobject can be seen, although blurry. (c) Recovered slice of
upper layer. (d) Recovered slice of lower layer. Textures in
sub-millimeter gap can be separated.

6. Discussions
This paper described a method for recovering inner slicesof translucent objects based on multi-frequency pattern projection. We have shown that inner radiance slices can beobtained by varying pitches of projection patterns. We also
developed a method for automatically selecting informativeslices via a sparse representation, i.e., determining sparse
coefficients for recovery of the observations. The effective-(b) Relative thickness

Figure 10: Pigment thickness estimation. By selecting
depth-dependent PSFs in a patch-based manner, our method
can estimate the relative thickness of the pigment. σ is an
unknown scattering parameter, which works as scale factor.
ness of the proposed method is shown by several experiments of simulation and real-world translucent objects.
There are future works that we are interested in exploring. First, the result is affected by an opaque object in upper
layers due to that the lower layers are shadowed. In otherwords, the lower layer’s reflectance is regarded to be zero,and as a result, we observe zero-reflectance textures in thelower layers. While this issue is difficult to directly addressin the proposed scheme, we are interested in resolving thisissue, or at least identify the shadowed regions, by assessing
the correlation between upper and lower textures and theirreflectance. Second, because our method requires multiple images under varying pitches, the measurement takes
approximately ten minutes in our experiments. We are interested in developing a more efficient measurement technique by exploiting multiplexing in space and wavelengthdomains.

References[1] S. Achar and S. G. Narasimhan. Multi Focus Structured
Light for Recovering Scene Shape and Global Illumination.

In European Conference on Computer Vision (ECCV), 2014.
2[2] S. Achar, S. T. Nuske, and S. G. Narasimhan. Compensatingfor Motion during Direct-Global Separation. In IEEE International Conference on Computer Vision (ICCV), 2013. 2[3] A. J. L. Adam, P. C. M. Planken, S. Meloni, and J. Dik. TeraHertz imaging of hidden paint layers on canvas. Optics Express, 17(5):3407, Feb. 2009. 2[4] S. Chandrasekhar. Radiative Transfer. Dover Publications,Inc., 1960. 3[5] J. Dik, K. Janssens, G. Van der Snickt, L. van der Loeff,
K. Rickers, and M. Cotte. Visualization of a lost painting
by Vincent van Gogh using synchrotron radiation based Xray fluorescence elemental mapping. Analytical chemistry,80(16):6436–42, Aug. 2008. 1, 2
[6] V. Durán, F. Soldevila, E. Irles, P. Clemente, E. Tajahuerce,
P. Andrés, and J. Lancis. Imaging at depth in tissue with asingle-pixel camera. arXiv, Nov. 2014. 2[7] A. F. Fercher. Optical coherence tomography. Journal of
Biomedical Optics, 1(2):157–173, 1996. 2[8] C. Fuchs, M. Heinz, M. Levoy, H.-P. Seidel, and H. P. A.
Lensch. Combining confocal imaging and descattering.
Computer Graphics Forum, Special Issue for the Eurographics Symposium on Rendering 2008, 27(4):1245–1253, 6
2008. 2[9] D. Gavrilov, E. Maeva, O. Grube, X. Maldague, and
R. Maev. Infrared Methods in Noninvasive Inspection of Artwork. In International Conference on NDT of Art, number
May, pages 1–5, 2008. 1, 2, 7
[10] J. Gu, S. K. Nayar, E. Grinspun, P. Belhumeur, and R. Ramamoorthi. Compressive Structured Light for Recovering Inhomogeneous Participating Media. IEEE Transactions on Pattern Analysis and Machine Intelligence(TPAMI),
35(3):555–567, 2013. 2[11] M. Gupta, Y. Tian, S. G. Narasimhan, and L. Zhang. A
Combined Theory of Defocused Illumination and Global
Light Transport. International Journal of Computer Vision,
98(2):146–167, Oct. 2011. 2[12] M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang,
I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat. Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophysical journal, 94(12):4957–70, June 2008. 2[13] T. Hawkins, P. Einarsson, and P. Debevec. Acquisitionof time-varying participating media. ACM Transaction on
Graphics, 24(3):812–815, July 2005. 2[14] D. Huang, E. Swanson, C. Lin, J. Schuman, W. Stinson, W. Chang, M. Hee, T. Flotte, K. Gregory, C. Puliafito, and A. Et. Optical coherence tomography. Science,
254(5035):1178–1181, Nov. 1991. 2[15] I. Ihrke and M. Magnor. Adaptive grid optical tomography.
Graphical Models, 68(5):484 – 495, 2006. Special Issue on
the Vision, Video and Graphics Conference 2005. 2[16] A. Ishimaru. Wave Propagation and Scattering in RandomMedia. IEEE Press, 1997. 3[17] P. Kner, B. B. Chhun, E. R. Griffis, L. Winoto, and M. G. L.
Gustafsson. Super-resolution video microscopy of live cells

by structured illumination. Nature methods, 6(5):339–42,
May 2009. 2[18] B. Lamond, P. Peers, and P. Debevec. Fast image-based
separation of diffuse and specular reflections. In ACM SIGGRAPH sketches, 2007. 2[19] Y. Li and M. S. Brown. Single Image Layer Separation using Relative Smoothness. In IEEE Conference on Computer
Vision and Pattern Recognition (CVPR), 2014. 1[20] N. J. W. Morris and K. N. Kutulakos. Reconstructing the Surface of Inhomogeneous Transparent Scenes by Scatter Trace
Photography. In International Conference on Computer Vision (ICCV), 2007. 2[21] Y. Mukaigawa, R. Raskar, and Y. Yagi. Analysis of lighttransport in scattering media. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 153–
160, 2010. 2[22] Y. Mukaigawa, R. Raskar, and Y. Yagi. Analysis of scattering
light transport in translucent media. IPSJ Transactions on
Computer Vision and Applications, 3:122–133, 2011. 2[23] S. Narasimhan and S. Nayar. Shedding light on the weather.
In Proceedings of the IEEE Conference on Computer Vision
and Pattern Recognition (CVPR), pages 665–672, 2003. 3[24] S. G. Narasimhan, M. Gupta, C. Donner, R. Ramamoorthi,
S. K. Nayar, and H. W. Jensen. Acquiring scattering properties of participating media by dilution. ACM Transactions
on Graphics, 25, 2006. 5[25] S. G. Narasimhan, S. K. Nayar, B. Sun, and S. J. Koppal. Structured light in scattering media. In IEEE International Conference on Computer Vision (ICCV), pages 420–
427, 2005. 2[26] S. K. Nayar, G. Krishnan, M. D. Grossberg, and R. Raskar.
Fast separation of direct and global components of a sceneusing high frequency illumination. In Proceeding ACM SIGGRAPH, pages 935–944, 2006. 1, 2, 3[27] B. Sarel and M. Irani. Separating transparent layers throughlayer information exchange. In European Conference on
Computer Vision (ECCV), 2004. 1[28] K. Shimizu, K. Tochio, and Y. Kato. Improvement of transcutaneous fluorescent images with a depth-dependent pointspread function. Applied Optics, 44(11):2154–2161, 2005. 2[29] R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images containing reflections and transparency. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 246–253, 2000. 1[30] K. Tanaka, Y. Mukaigawa, Y. Matsushita, and Y. Yagi.
Descattering of transmissive observation using Parallel
High-Frequency Illumination. In IEEE International Conference on Computational Photography (ICCP), 2013. 2, 6
[31] O. D. Therrien, B. Aubé, S. Pagès, P. D. Koninck, and
D. Côté. Wide-field multiphoton imaging of cellular dynamics in thick tissue by temporal focusing and patterned illumination. Biomedical optics express, 2(3):696–704, 2011.
2[32] R. Tibshirani. Regression shrinkage and selection via thelasso. Journal of the Royal Statistical Society, Series B,
58:267–288, 1994. 5