Depth-resolved image mapping spectrometer(IMS) with structured illumination
Liang Gao,1,2 Noah Bedard,1 Nathan Hagen,1 Robert T. Kester,1and Tomasz S.
Tkaczyk1,2,3,*
1

Department of Bioengineering, Rice University, Houston, Texas 77005, USA
2
Rice Quantum Institute, Rice University, Houston, Texas 77005, USA
3
Department of Electrical and Computer Engineering, Rice University, Houston, Texas 77005, USA*ttkaczyk@rice.edu

Abstract: We present a depth-resolved Image Mapping Spectrometer (IMS)
which is capable of acquiring 4D (x, y, z, λ) datacubes. Optical sectioning is
implemented by structured illumination. The device’s spectral imaging
performance is demonstrated in a multispectral microsphere and mouse
kidney tissue fluorescence imaging experiment. We also compare
quantitatively the depth-resolved IMS with a hyperspectral confocal
microscope (HCM) in a standard fluorescent bead imaging experiment. The
comparison results show that despite the use of a light source with fourorders of magnitude lower intensity in the IMS than that in the HCM, theimage signal-to-noise ratio acquired by the IMS is 2.6 times higher than that
achieved by the equivalent confocal approach.
©2011 Optical Society of America
OCIS codes: (110.4234) Multispectral and hyperspectral imaging; (180.2520) Fluorescencemicroscopy; (170.6280) Spectroscopy, fluorescence and luminescence.

References and links
J. W. Lichtman and J. A. Conchello, “Fluorescence microscopy,” Nat. Methods 2(12), 910–919 (2005).
Y. Hiraoka, T. Shimi, and T. Haraguchi, “Multispectral imaging fluorescence microscopy for living cells,” Cell
Struct. Funct. 27(5), 367–374 (2002).
3. T. Zimmermann, J. Rietdorf, and R. Pepperkok, “Spectral imaging and its applications in live cell microscopy,”
FEBS Lett. 546(1), 87–92 (2003).
4. V. Raicu, M. R. Stoneman, R. Fung, M. Melnichuk, D. B. Jansma, L. F. Pisterzi, S. Rath, M. Fox, J. W. Wells,
and D. K. Saldin, “Determination of supramolecular structure and spatial distribution of protein complexes inliving cells,” Nat. Photonics 3(2), 107–113 (2009).
5. S. Kumazaki, M. Hasegawa, M. Ghoneim, Y. Shimizu, K. Okamoto, M. Nishiyama, H. Oh-Oka, and M.
Terazima, “A line-scanning semi-confocal multi-photon fluorescence microscope with a simultaneous broadband
spectral acquisition and its application to the study of the thylakoid membrane of a cyanobacterium Anabaena
PCC7120,” J. Microsc. 228(Pt 2), 240–254 (2007).
6. H. R. Morris, C. C. Hoyt, and P. J. Treado, “Imaging spectrometers for fluorescence and Raman microscopy acoustooptic and liquid-crystal tunable filters,” Appl. Spectrosc. 48(7), 857–866 (1994).
7. B. K. Ford, C. E. Volin, S. M. Murphy, R. M. Lynch, and M. R. Descour, “Computed tomography-based spectral
imaging for fluorescence microscopy,” Biophys. J. 80(2), 986–993 (2001).
8. C. F. Cull, K. Choi, D. J. Brady, and T. Oliver, “Identification of fluorescent beads using a coded aperture
snapshot spectral imager,” Appl. Opt. 49(10), B59–B70 (2010).
9. A. Gorman, D. W. Fletcher-Holmes, and A. R. Harvey, “Generalization of the Lyot filter and its application tosnapshot spectral imaging,” Opt. Express 18(6), 5602–5608 (2010).
10. L. Gao, R. T. Kester, and T. S. Tkaczyk, “Compact image slicing spectrometer (ISS) for hyperspectral
fluorescence microscopy,” Opt. Express 17(15), 12293–12308 (2009).
11. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot image mapping spectrometer (IMS) with highsampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010).
12. R. T. Kester, N. Bedard, L. Gao, and T. S. Tkaczyk, “Real-time snapshot hyperspectral imaging endoscope,” J.
Biomed. Opt. 16(5), 056005 (2011).
13. M. A. A. Neil, R. Juskaitis, and T. Wilson, “Method of obtaining optical sectioning by using structured light in aconventional microscope,” Opt. Lett. 22(24), 1905–1907 (1997).
14. D. Karadaglić and T. Wilson, “Image formation in structured illumination wide-field fluorescence microscopy,”
Micron 39(7), 808–818 (2008).

1.
2.

#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17439

15. T. Zimmermann, “Spectral imaging and linear unmixing in light microscopy,” in Advances in Biochemical
Engineering Biotechnology, T. Scheper, ed. (Springer, 2005).
16. A. Hopt and E. Neher, “Highly nonlinear photodamage in two-photon fluorescence microscopy,” Biophys. J.
80(4), 2029–2036 (2001).
17. J. Xiao, “Single-molecule imaging in live cells,” in Handbook of Single-Molecule Biophysics, P. Hinterdorfer and
A. V. Oijen, eds. (Springer, 2009).
18. J. R. Lakowicz, Principles of fluorescence spectroscopy, 3rd ed. (Springer, 2006).
19. H. Photonics, “Characteristics of Photomultiplier Tubes” in Photomultiplier Tubes: Basics and Applications, 2nded. (1999), p. 71.
20. The Molecular Probes Handbook, Invitrogen, Inc.
21. P. J. Shaw, “Comparison of widefield/deconvolution and confocal microscopy for three-dimensional imaging,”
in Handbook of Biological Confocal Microscopy, 3rd. ed., J. B. Pawley, ed. (Springer, 2006)
22. R. Dixit and R. Cyr, “Cell damage and reactive oxygen species production induced by fluorescence microscopy:
effect on mitosis and guidelines for non-invasive fluorescence microscopy,” Plant J. 36(2), 280–290 (2003).
23. Y. Sako, A. Sekihata, Y. Yanagisawa, M. Yamamoto, Y. Shimada, K. Ozaki, and A. Kusumi, “Comparison of
two-photon excitation laser scanning microscopy with UV-confocal laser scanning microscopy in threedimensional calcium imaging using the fluorescence indicator Indo-1,” J. Microsc. 185(Pt 1), 9–20 (1997).
24. J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. Stelzer, “Optical sectioning deep inside live embryos
by selective plane illumination microscopy,” Science 305(5686), 1007–1009 (2004).
25. Fairchild, Inc., Andor, Inc., and PCO, Inc., “sCMOS data sheet,” http://www.scmos.com/.

1. Introduction
Fluorescence microscopy is an indispensable tool in biological research and has been used
extensively in widespread applications [1]. The continuing development of fluorescent probes
bringing new spectral characteristics and new functional properties for cellular labeling has
widened the possibilities of what biological features can be measured by a microscope.
Taking full advantage of their properties, however, often requires imaging systems have aspectral resolving power to discriminate those fluorescent probes which have significant
spectral overlaps.
Hyperspectral imaging is a hybrid imaging modality that combines the advantages of
traditional imaging cameras and spectrometers [2]. It measures sample’s spatial and spectral
information, and constructs a 3D (x, y, λ) datacube for spectral unmixing analysis [3].
However, most hyperspectral imagers are scanning-based systems. They either scan in thespatial domain, e.g. hyperspectral confocal microscope [4] and slit-scanning microscope [5],or scan in the spectral domain, e.g., liquid crystal tunable filters or acoustic optic tunablefilters [6]. Because scanning-based systems cannot collect light from all voxels of thedatacube in parallel, there is a loss of light throughput by a factor of N when measuring N scanelements. To overcome this limitation, snapshot hyperspectral imagers such as Computed
Tomography Imaging Spectrometer (CTIS) [7], Coded Aperture Snapshot Spectral Imager(CASSI) [8] and Image-Replicating Imaging Spectrometer (IRIS) [9] have been developed.
Although the spectral imaging capabilities of these snapshot hyperspectral imagers have been
demonstrated, they suffer many problems, e.g., CTIS and CASSI require extensive
computational cost, while IRIS is limited in the number of spectral bands which can acquire athigh spatial resolution.
The Image Mapping Spectrometer (IMS) is a novel snapshot hyperspectral imager that isdeveloped for full-throughput measurement of spectrally resolved scenes [10–12]. It replaces
the regular camera in a digital imaging system, allowing one to add spectrum acquisition
capability to a variety of imaging modalities, such as microscopy [10,11] and endoscopy [12].
The operation of the IMS is based on the image mapping principle [10]. Briefly, a customfabricated optical component – termed image mapper – is utilized to remap a sample’s 3D (x,y, λ) datacube onto a 2D CCD camera, so that a sample’s spatial and spectral information canbe measured in parallel. Since no scanning is employed, the IMS features high-speed(currently up to 7.2 fps [12]) (x, y, λ) datacube acquisition without sacrificing spatial andspectral resolution. In addition, since the IMS is a direct imaging device, little computational
cost is required to reconstruct a (x, y, λ) datacube.

#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17440

Previous IMS-based experiments were implemented in wide-field imaging, but for
imaging volumetric samples such as biological tissues, wide-field measurements suffer fromspatial-spectral crosstalk due to light contributed from out-of-focus layers. This crosstalkdecreases the ability to resolve all voxels in the datacube and compromises spectral unmixing
capability. To address this problem, here we add optical sectioning capability to the IMS withstructured illumination (SI). By acquiring three (x, y, λ) datacubes I1 , I 2 and I 3 with

2π / 3 phase-shifted sinusoid illumination patterns and subsequent demodulation, the IMS
achieves ~1 µm axial resolution for the acquired spectral channel images.
Although the throughput of the IMS is halved by the added sectioning hardware (seeSection 2), high contrast depth-resolved spectral channel images were still acquired in amouse kidney tissue fluorescence imaging experiment (see Section 3). In addition, due to theadded sectioning capability, a 4D (x, y, z, λ) datacube is also successfully acquired by the
IMS. The 4D imaging results demonstrate that no artifacts are introduced by integrating
traditional 3D (x, y, z) imaging with IMS’ multispectral imaging.
To show the advantages of depth-resolved IMS over other commercially available
multispectral imagers, we quantitatively compare the depth-resolved IMS with a hyperspectral
confocal microscope (HCM) (Zeiss Meta 510) in the context of photon collection and image
signal-to-noise ratio (SNR) (see Section 4). The experimental results indicate that, at a given
frame rate, the depth-resolved IMS surpasses the corresponding HCM approach 130 times in
the photon collection, and 2.6 times in the image SNR respectively.
2. System layout
The schematic of the depth-resolved IMS is shown in Fig. 1. The IMS is coupled to a side
image port of an inverted microscope (Axio Observer, Zeiss). The optical design and layout
for the IMS is detailed in [12]. Here the IMS is assembled in a robust ruggedized enclosure sothat the device is stable and functional in ambient room light. The 3D (x, y, λ) datacube that
the IMS acquires is of size 350 × 350 × 46. The spatial sampling and spectral sampling arebalanced in the IMS design so that over 90% IMS’ camera pixels can be effectively utilized[12]. When a Zeiss Plan-Apochromat 63 × oil immersion objective (NA = 1.4) is used on the
microscope, the IMS achieves ~0.44 µm lateral spatial resolution in the 160 µm × 160 µm
field-of-view (FOV).
The sample is illuminated by a 120W X-cite arc lamp. Structured illumination is
implemented by placing a 10 lp/mm Ronchi ruling grid (NT38-258, Edmund Optics) at a
conjugate plane of the microscope’s sample plane in the epi-illumination optical path. The
grid is mounted on a ceramic servo motor stage (ALIO industries, P/N: AI-HR4-10000E-XY).
The IMS is synchronized with the movement of the grid for automatic acquisition. The depthresolved (x, y, λ) datacube I is acquired from three (x, y, λ) datacubes I1 , I 2 and I 3 which arecaptured with 2π / 3 phase shifted sinusoid illumination patterns by the demodulation
algorithm [13,14]:

I = ( I1 − I 2 ) 2 + ( I1 − I 3 ) 2 + ( I 2 − I 3 )2 .

(1)#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17441

Fig. 1. Depth-resolved IMS with SI.

The axial resolution of depth-resolved IMS (with a 63 × oil immersion objective on the
microscope) was measured by imaging sub-resolution point sources – green fluorescent beads(Dia = 175 nm, Invitrogen PS-Speck Microscope Point Source Kit). By scanning the sample
in the z axis, the axial PSF was measured at the wavelength 530 nm (see the red line in Fig.
2). To provide a baseline reference, the theoretical axial PSF at this wavelength is also shown
as a blue dashed line in Fig. 2. The axial resolution res z is calculated as the full-width-halfmaximum (FWHM) of the measured axial PSF, which is:

res z = FWHM ( PSF ) = 1.08 µ m.

(2)

Fig. 2. Axial PSF measurement in the depth-resolved IMS.

This imaging result proves that the depth-resolved IMS has similar sectioning capability as
regular wide-field imaging modalities with SI. Although the raw intensity images are
remapped and reconstructed in the IMS, it does not reduce the effectiveness of SI in allowing
depth-resolved imaging.
3. Multispectral fluorescence microscopic imaging with depth-resolved IMS
To demonstrate the IMS’ sectioning and spectral imaging capabilities in fluorescencemicroscopy, we imaged a multispectral microsphere (FocalCheck test slide #2, Invitrogen).
The diameter of the fluorescent microsphere is 6 µm. Its core and shell are stained with two
different fluorescent dyes: The fluorescence emitted from the core has a peak emission at 524#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17442

nm; while the fluorescence emitted from the shell has a peak emission at 511 nm (See Fig. 2(b). The core and shell reference spectra were acquired from single dye stained control sample
on the same test slide). A Chroma filter set 41015 (485 LP) was used to separate fluorescencefrom excitation light.
The spectral imaging results from a sectioned depth layer are shown in Fig. 3. Due tosignificant spectral overlaps of the dyes, in the panchromatic display of the acquired datacube(Fig. 3(a)), the core and shell cannot be discriminated. After applying a linear spectral
unmixing algorithm [15] to the acquired datacube, the fluorescent emissions from core (peakat 524 nm) and shell (peak at 511 nm) are successfully separated (Fig. 3(c)–(e)). The spectral
unmixing accuracy is evaluated at the pixel level near the separating interface of the two
fluorophores. The result shows that the fitted spectrum matches well with the measured
spectrum (see Fig. 3 (g)). In addition, a dashed line is drawn across the bead section (Fig.
3(e)) to show the intensity changes of two fluorescent dyes (Fig. 3(f)). No fluorescenceemitted from the shell is detected in the core. This demonstrates the sectioning and spectral
imaging capabilities of depth-resolved IMS.

Fig. 3. Spectral imaging of a sectioned depth layer of a fluorescent microsphere by depthresolved IMS. (a) Panchromatic display of acquired datacube. (b) The spectra of the core andring of the microsphere. (c) – (d) Pseudo-colored core and ring components after spectral
unmixing. (e) A merged image of (c) and (d). (f) Intensity profile of two fluorescent dyes
across the microsphere section. (g) Measured spectrum vs. fitted spectrum at the interface ofthe two fluorophores.

Next, we imaged a mouse kidney tissue section with a combination of fluorescent dyes. In
the sample, the glomeruli and convoluted tubules were stained with Alexa Fluor 488 (exc: 493
nm, emi: 516 nm), and the filamentous actin was stained with Alexa Fluor 568 (exc: 578 nm,
emi: 604 nm). Chroma filter set 61001 (DAPI/FITC/PI) was used to separate fluorescencefrom excitation light. The integration time for each phase image was 0.1 sec. Selected spectral
channel images are shown in Fig. 4 (a). For comparison, the same FOV was also imaged by
the IMS in wide-field imaging and shown in Fig. 4 (b). Comparing Fig. 4 (a) with Fig. 4 (b),#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17443

the image contrast acquired by the depth-resolved IMS has been significantly increased due toreduced spatial-spectral crosstalk.

Fig. 4. Spectral channel images of mouse kidney tissue section acquired by (a) depth-resolved
IMS, and (b) IMS under wide-field imaging.

To evaluate the 4D (x, y, z, λ) imaging performance of depth-resolved IMS, the mouse
kidney tissue section was imaged along the z axis with 1 µm step. The captured 4D (x, y, z, λ)datacube is of size 350 × 350 × 35 × 46, and shown at four selected wavelengths in Fig. 5 (seeMedia 1 for the rotation view of (x, y, z) datacubes at four selected wavelengths, and see
Media 2 for a scan through of all acquired wavelengths). The x-y cross section views shown in
Fig. 5 are taken at the plane with z = 15 µm, and the x-z cross section views are taken at theplane with y = 80 µm. Due to the added sectioning capability, the hollow structure of microtubules and filamentous actin is unambiguously revealed by the IMS. This 4D imaging result
proves that no artifacts are introduced by integrating traditional 3D (x, y, z) imaging with
IMS’ multispectral imaging,
4. Quantitative comparison between the depth-resolved IMS and a hyperspectral
confocal microscope
Since the IMS has the sectioning capability due to the implementation of SI, here wequantitatively compare it with a commercially available hyperspectral confocal microscope(HCM) (Zeiss Meta 510) in a depth-resolved fluorescence imaging experiment.
Although the IMS and HCM have similar functionality, technically they behave very
differently both at light collection and illumination side. At the light collection side, the IMS
is a parallel acquisition system – the spectral information of all spatial sampling points iscollected simultaneously in the FOV; while the HCM is a scanning-based system – thespectral information of spatial points is collected in sequence. The HCM’s scanningmechanism causes a severe trade-off between the number of photons collected at each spatialsampling pixel and the whole frame acquisition time – the faster the imager scans, the shorter
dwelling time a pixel can have. To some extent, a HCM can compensate this throughput loss
at the illumination side by focusing a high-power laser onto the sample. The illumination
irradiance in the HCM is usually several orders of magnitude higher than that in the IMS.

#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17444

However, the high illumination irradiance may cause serious photo-damage and phototoxicity, which is of a particular concern in live cell imaging experiments [16,17]. Once thefluorophores have been boosted to their saturation excitation state [18] – a situationcommonly reached by existing confocal systems – even the method of using high power laser
falters because the fluorescent emission rate could not be further increased (see the discussion
in Section 4.3).
Additionally, the approaches that the IMS and HCM adopt to achieve optical sectioning
are also different. Whereas HCM gates the out-of-focus light by a physical pinhole prior todetection, IMS with SI rejects out-of-focus light by computational demodulation after
detection. Thus the image noise level acquired under SI is usually higher than that acquired in
the HCM.

Fig. 5. (Media 1 and Media 2) 4D (x, y, z, λ) imaging of a mouse kidney tissue section by thedepth-resolved IMS. The glomeruli and convoluted tubules were stained with Alexa Fluor 488
and the filamentous actin was stained with Alexa Fluor 568. The acquired 4D (x, y, z, λ)datacube is of size 350 × 350 × 35 × 46. A total of four out of 46 wavelength (x, y, z) datacubes
are shown.

To provide a quantitative comparison between the IMS and HCM, a standard fluorescentbead imaging experiment was implemented. Since the two imaging modalities use different
photon detectors – the IMS uses an interline CCD camera (Imperx, P/N: IPX-16M3-L) while
the HCM uses a photomultiplier tube (PMT) array, we first calibrated the photon detector’s#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17445

gain for each imaging modality (Section 4.1). Then, we measured the maximum number ofphotons that can be collected from a sectioned depth layer when two imaging modalities wereoperated at a given frame rate (Section 4.2). The illumination power (Arc lamp for the IMS,
Argon laser for the HCM) was maximized in both setups. We also investigated whether theHCM can compensate its scanning-caused throughput loss by using a more powerful laser
source in Section 4.3. Since noise amplification occurs in SI measurement, SNR can be amore useful quantity to compare than collected photon count. Thus, we estimated the noiselevel in the spectral channel images acquired by the depth-resolved IMS and HCM, andcompared the two imaging modalities in the context of acquired image SNR (Section 4.4).
4.1 Calibration of photon detector’s gain in the IMS and HCM
For a digital imaging system, the signal Sc one acquires from an image is in the unit of counts.
To calculate from this quantity the actual number of detected photons S p , we define thedetector’s gain as the conversion relation:g = photons / count.
(3)
Note that the detector’s gain defined here is not necessarily the same thing as the internal
gain value for a PMT, as it includes the amplifier gain and analog-to-digital conversion factor.
For shot noise limited systems, the measured image noise N c (in counts) in the IMS’ CCD
camera and HCM’ PMT are:N c , IMS ~ σ c = σ p / g ,N c , HCM ~ 1.21/ 2 σ c = 1.1σ p / g ,(4)where σ C is photon noise standard deviation in units of counts, while σ p is the photon noise inunits of photons. There is a factor 1.21/ 2 between the measured image noise N c , HCM andphoton noise σ c due to the fact that the PMT is working in the analog mode [19].
Since photon noise σ p obeys Poissonian statistics:

σ p2 = S p .

(5)

Combining Eq. (4) and Eq. (5), we have:N c2, IMS ~ S p / g 2 = gSc / g 2 = Sc / g ,N c2, HCM ~ 1.2S p / g 2 = 1.2 gSc / g 2 = 1.2 Sc / g .

(6)Equation (6) indicates that there is a linear relationship between the square of image noise
N c2 (termed signal variance) and image signal level Sc with a slope equal to 1/ g in IMS
imaging while 1.2 / g in HCM imaging.
To experimentally measure the IMS and HCM detector gains, we mounted an integration
sphere (FOIS-1, Ocean Optics) on the microscope’s sample stage. The integration sphere was
illuminated by a halogen lamp source. The uniform fields were imaged by the IMS and HCM
respectively. Specific detector gain settings were chosen on the two modalities based on thecriteria that full dynamic range was reached in the subsequent fluorescent bead imaging
experiment (Section 4.2). By applying a linear regression to the variables N c2 and Sc at
different illumination intensities (see Fig. 6), the IMS and HCM detector’s gains are obtained:g IMS = 17.6 photons/count,g HCM = 0.0152 photons/count.

(7)#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17446

Fig. 6. Signal variance vs. Intensity in (a) IMS and (b) HCM.

4.2 Measurement of collected photons in a standard fluorescent bead imaging experiment by
the depth-resolved IMS and HCM
In the comparison experiment, a standard green fluorescent bead (Diameter = 2 µm, emi
peak: 516 nm) sample was chosen to be imaged by both modalities. Since these fluorescentbeads are photo-stable [20], no photobleaching is expected to be observed in the present
study. The comparison experiment concept is shown in Fig. 7. A spatial depth layer (x, y, λ) ofa fluorescent bead is acquired by SI in the depth-resolved IMS, and by pinhole filtering in the
HCM. Subsequently, this depth-resolved spatial layer is dispersed and imaged into thespectral channels with nominal wavelengths λ1 , λ2 ⋯ λn . Since the spectral channel

λm = 516 nm is close to the peak of bead’s fluorescent emission and maximum of detectors’
quantum efficiency curve, it is chosen as the representative in the following comparisons.

Fig. 7. Experimental design concept for comparison study.

The depth-resolved IMS used a Chroma filter set 41015 (exc: 450/50 nm, emi: 485nm LP)
to separate fluorescence from excitation light. The HCM excited the sample with an Argonlaser (488 nm), and the fluorescence was collected by the same Zeiss objective (63 × /NA =1.40) that was used for the IMS. Lambda acquisition mode (8 parallel spectral acquisition
channels with spectral bin width ~10 nm) was chosen on the HCM’s operation software. The
HCM’s pinhole size was set to be 1.26 Airy disks so that the sectioning thickness (~0.9 µm)
was close to that of the depth-resolved IMS. The imaging parameters used in the comparison
experiment are summarized in Table 1.
Note that in the depth-resolved IMS, the pixel exposure time is equal to the frame time dueto the parallel acquisition of the IMS (although the illumination intensity is halved due to thegrid pattern under SI); while in the HCM, the pixel dwelling time is equal to the division of
the frame time to the frame sampling minus scanning time due to the mechanical scanning
across the field. To cancel out the differences in frame sampling, spectral bin width (assuming#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17447

spectral flatness over both bin widths), and sectioning thickness between the two modalities, asignal scaling factor is calculated for the depth-resolved IMS with respect to the HCM by:L=

2∆λHCM ∆z HCM M HCM/
×= 1.11.
2∆λIMS∆z IMS M IMS(8)Table 1. Imaging Parameters for Depth-resolved IMS and HCM in Comparison
Experiment

T: Frame time to acquire a
sectioned depth layer
t: Pixel exposure (dwelling) time
M × M: Frame sampling∆λ: Spectral bin width∆z: Sectioning thickness

Depth-resolved IMS

HCM

983 ms (327 ms for aphase image)983 ms350 × 350
~3.5 nm~1.08 µm

983 ms
1.6 µs
512 × 512
~10 nm~0.9 µm

In Eq. (8), the frame sampling is included in the signal scaling factor because in the HCM
imaging one can increase the pixel dwelling time by scanning a smaller area.
The fluorescence emitted by a bead was measured by summing all the pixel intensities (incounts) at the location of the bead. A total of 20 fluorescent beads were measured by thedepth-resolved IMS and HCM respectively (see Fig. 8). The mean fluorescent signals of beads
measured by the depth-resolved IMS are SC , IMS = 4.87 × 103 in counts; while those measured
by the HCM are SC , HCM = 4.84 ×104 in counts. By converting the measured signal counts tophotons and scaling the signal with factor L, the ratio of photons collected by the depthresolved IMS to the HCM at the given frame rate is:r=

S p , IMS L
S p , HCM=

SC , IMS g IMS L
SC , HCM g HCM=

4.87 × 103 × 17.6 × 1.11
= 130.
4.84 × 104 × 0.0152(9)

Note that the photon collection ratio calculated in Eq. (9) is consistent with previous
studies [21], which estimate the peak signal in wide-field deconvolution microscopy isnormally around 30,000 photons/pixel while in point-scanning confocal microscopy it isaround 20 – 100 photons/pixel. Since the throughput is halved in the depth-resolved IMS
setup due to the implementation of SI, the photon collection ratio r calculated in Eq. (9) issmaller than the result comparing wide-field deconvolution microscopy with confocal
microscopy.

#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17448

Fig. 8. (a) Depth-resolved IMS imaging vs. (b) HCM imaging in a standard fluorescent beadimaging experiment. The color bar is in the unit of converted photons from image intensity
counts.

4.3 Illumination irradiance in the depth-resolved IMS and HCM
In our microscope setups, the illumination power (Arc lamp for the IMS, Argon laser for theHCM) was maximized for both depth-resolved IMS and HCM. To show the difference
between the two modalities at the illumination side, we calculate the illumination irradianceon the microscope stage. In the HCM imaging, the power of the 488 nm excitation laser on thesample plane was measured to be 0.32mW. Assuming the illumination power is focused onto
a point-spread-function (PSF), the size of which is ~0.44 µm, the illumination irradiance isapproximately:

EHCM ~

0.32 mW
0.32 mW
0.32 mW=== 2.14 mW/µ m 2 . (10)
1
1
1
2
2
2
π × PSF
π × (1.22λ / NA)
π × (0.44 µ m)
44
4

In the depth-resolved IMS imaging, a total of 40 mW illumination power was measured;
however, the illumination area is over the entire microscope objective’s FOV (Dia ~400 µm).
Thus the illumination irradiance is approximately
EIMS ~

40 mW= 3.2 × 10−4mW/µ m 2 .
12
π × (400 µ m)
4(11)

The above results indicate that, at the illumination side, HCM can achieve much higher
illumination irradiance than the IMS. This approach is usually adopted by the HCM tocompensate its scanning-caused throughput loss at the light collection side. However, the highillumination irradiance may cause serious photodamage and phototoxicity, especially in live
cell imaging experiments. Since the generation rate of reactive oxygen species (ROS) – thechemical radical suggested as the main source of photodamage and phototoxicity – has anonlinear relationship with the illumination irradiance [22], the differences between short#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17449

exposure of high-intensity light (as in the HCM imaging) and continuous exposure to much
lower light level (as in the IMS imaging) may yield important differences in the photodamage
and phototoxicity in living cells [21]. This fact makes the IMS better suited than the HCM inlive cell imaging applications because the light-induced perturbation is minimized.
To investigate whether the photon collection ratio in Eq. (9) can be reduced if HCM use amore powerful laser source than the current setup, we measured the emissions of a fluorescentbead at different excitation laser powers (see Fig. 9). The maximal number of photons that thefluorophore can emit before reaching its saturation state [18] is calculated by fitting an
exponential curve I = a − ae − bP to the measured data (I is the measured fluorescence signal
and P is the excitation laser power). The fitting result shows the decay coefficient b is equal to2.61 mW −1 . The signal ratio of the fluorophore at the laser power P = ∞ to P = 0.32 mW is thus
obtained:ξ=I ( P = ∞)a=
= 1.77.
I ( P = 0.32 mW) a(1 − e −0.32b )(12)

Fig. 9. Excitation laser power vs. Measured signal in HCM imaging.

Consequently, if the HCM uses a more powerful laser source in the presented comparison
study, the measured photons by the depth-resolved IMS still remains:r * > r / ξ = 73

(13)times higher than what can be measured by the HCM.
4.4 Noise level estimate and image S/N comparison
Here we consider both systems to be shot noise limited. In HCM imaging, the out-of-focus
light is rejected prior to the detection, so the image noise is mainly contributed by the photon
noise from the sectioned depth layer. Thus the image SNR for a sectioned fluorescent bead in
the HCM imaging is:

SNRHCM =S p , HCM
N p , HCM=

S p , HCM
1/ 2

1.2 σ p=S p , HCM
1.2S p , HCM= S p , HCM / 1.2 = 24.7.

(14)In Eq. (14), S p , HCM is the fluorescent signal from a sectioned bead depth layer and is
calculated in Eq. (9).
In the depth-resolved IMS imaging, the out-of-focus light is rejected after detection, sothat the image noise is contributed by the photon noise both from in-focus and out-of-focus
depth layers. To measure the actual noise in the depth-resolved IMS imaging, we captured
1000 time-sequenced (x, y, z, λ, t) datacubes for the same fluorescent bead sample. Then the#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17450

mean fluorescent intensity I and its standard deviation σ I at a bead spatial sampling point( x1 , y1 , z1 ) in spectral channel λ = 516 nm are calculated as:
I=

1 1000∑ I ( x1 , y1 , z1 , t j , λ = 516nm),
1000 j =1

1 1000σI =∑ ( I ( x1 , y1 , z1 , t j , λ = 516nm) − I )2 .
1000 − 1 j =1

(15)

The measured actual noise σ I is 4.78 times higher than the sectioned layer’s photon
noise I . Assuming the ratio σ I / I is the same for all spatial sampling points (x, y, z) at thebead location, the image SNR for a sectioned fluorescent bead in the IMS imaging isestimated to be:∑∑ I ( x , y )i

SNRIMS =i∑∑ σ ( x , y )
2
I

i

j

∑∑ I ( x , y )

j

j
i

j

i

=

i

j

j

4.78 ∑∑ I ( xi , y j )i=1
LS p , IMS = 64.7, (16)4.78

jwhere S p , IMS is the fluorescent signal from a sectioned bead depth layer and is calculated in
Eq. (9). L is the signal scaling factor defined in Eq. (8).
By comparing Eq. (14) with Eq. (16), we have following conclusions: at a frame rate of ~1
fps, the image SNR acquire by the depth-resolved IMS is

SNRIMS~ 2.6
SNRHCM(17)

times higher than that acquired by the hyperspectral confocal approach. Note that this result isalso suggested by Fig. 8.

5. Conclusions
In this paper, we present a depth-resolved IMS which is capable of acquiring 4D (x, y, z, λ)datacubes. By implementing structured illumination, the IMS has the same sectioning
capability as the regular wide-field imaging modalities but with additional spectral resolving
power. The 4D imaging of a mouse kidney tissue section proves that no artifacts are
introduced by integrating traditional 3D (x, y, z) imaging with IMS’s multispectral imaging.
The depth-resolved IMS not only provides high-resolution sectioned (x, y, λ) datacubes,
but its light collection capability surpasses that of an equivalent confocal microscope by over
two orders of magnitude. This is achieved despite the use of a lower power excitation sourcein the IMS measurements relative to the HCM’s argon laser, a fact that is of value in the live
cell imaging because the light-induced photodamage and phototoxicity can be minimized.
Note that the presented comparison between depth-resolved IMS and HCM was implemented
under one-photon excitation. For two-photon excitation microscopy (TPEM), previous studies
have shown that the in-focus light collection efficiency of TPEM is comparable to theconfocal pinhole filtering approach [23]. So even if the pinhole filtering in the HCM isreplaced by the TPEM, the depth-resolved IMS is still expected to have an edge on the photon
collection.
SI is an effective way to add the sectioning capability to the IMS, especially due to its easy
integration with the IMS on a standard microscope setup. However, under SI the out-of-focus
light still uses the detector’s dynamic range and thus contributes to the shot noise, so that theacquired image noise level is usually higher than for HCM, a fact that decreases theadvantages of the IMS at the light collection side. However, in the presented study, the#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17451

acquired image SNR by the depth-resolved IMS is still ~2.6 times higher than that acquired by
the HCM. For the imaging applications in which the out-of-focus light dominates thedetector’s dynamic range, the alternative implementation of selective plane illumination [24]with the IMS may provide better image contrast due to the illumination’s intrinsic sectioning
capability.
In addition, in the comparison both IMS and HCM are considered to be shot noise limited
because the detectors’ full dynamic range is used. Generally speaking, the readout noise in the
CCD camera (e.g., ~16 electrons in the IMS’ camera) is higher than that in the HCM’s PMT(< 1 electron). For low light imaging experiments, the CCD camera’s relatively high readout
noise may become a concern. However, in the recently developed scientific CMOS (sCMOS)
cameras [25], readout noise has been significantly reduced to less than 2 electrons even atvery high frame rate (100 fps). Incorporation of such cameras into the IMS will not only
increase IMS’ acquisition speed, but also provide a background noise level comparable to thatin HCM imaging.

Acknowledgment
This work is supported by the National Institute of Health under Grant No. R21EB009186 andR21EB011598.

#150742 - $15.00 USD Received 11 Jul 2011; revised 17 Aug 2011; accepted 18 Aug 2011; published 19 Aug 2011

(C) 2011 OSA

29 August 2011 / Vol. 19, No. 18 / OPTICS EXPRESS 17452