# Multiband Polarization Imaging

Authors: Yongqiang Zhao; Qunnie Peng; Chen Yi; Seong G. Kong
Journal: Journal of Sensors
 (2016)

---

## Abstract

Multiband polarization imaging is an emerging sensing method that enables simultaneous acquisition of multiband spectral and multiangle polarization information of an object of interest in the scene. Spectral signatures of the light reflected from a target reveal the characteristics of the material composing the target while polarized light provides useful information on the surface features such as light scattering and specular reflection. In multiband spectral imaging, combined spectral and polarization information offers a comprehensive representation of an object utilizing complementary spectral and polarization information in visual sensing. Multiband polarization imaging has demonstrated a potential in the recognition of targets in challenging operating environments such as low-contrast and hazy conditions. This paper presents the concept and recent advances of multiband polarization imaging techniques, in particular, a bioinspired multiband polarization vision system. Applications of multiband polarization imaging in various fields include atmospheric observation, object detection and classification, medical diagnostics, surveillance, and 3D object reconstruction.


---

## Body

## 1. Introduction

An electromagnetic wave of light reflected from an object carries the signatures that characterize the object such as light intensity as a function of wavelength or spectrum, transmission directions, and the plane of polarization. Reflectance of absorption patterns of the spectrum describes material compositions of a target object, serving as a useful signature for target detection and classification. As a combined technique of spectroscopy and photography, spectral imaging observes spectral response of an object in different wavelengths at every location in an image plane. Spectral imaging acquires multiple images of a scene in different spectral bands. According to some criteria such as spectral resolution, number of spectral bands, and contiguousness of bands, spectral imaging techniques exist in several terms such as multispectral imaging and hyperspectral imaging, often in the visible and infrared regions of electromagnetic spectrum. Spectroscopic analysis has been used to identify the features inherent to the object of interest through measurement and analysis of electromagnetic spectra produced by reflection or absorption. Polarization is a property of transversal waves that can oscillate with more than one orientation. Light can be approximated as a plane wave and propagates as a transverse wave, in which both the electric and magnetic fields are perpendicular to the direction of propagation. In linear polarization, the oscillation of these fields takes place in a single direction. A polarizer is an optical filter that transmits only one polarization direction. Imaging in different polarization angles has been employed to capture unique surface features of an object. The amount and orientation information of polarization have been of great significance to enhance discriminating power of vision-based detection and classification systems.Recently there have been growing interests in multiband polarization imaging techniques to take advantage of both polarization and spectral signatures in various computer vision tasks. Multiband polarization imaging offers several advantages over conventional imaging techniques in sensing and analysis of objects in challenging environments. Multiband polarization imaging measures the intensity of the light reflected from the object in multiple spectral bands and multiple polarization angles to capture comprehensive optical characteristics of an object of interest in the scene. Combined spectral and polarization information gives a complete representation of an object in terms of the material compositions as well as surface characteristics. In multiband polarization imaging, spatial, spectral, and polarization information are simultaneously acquired [1]. Spatial, spectral, and polarization information reveal the different characteristics of a material. It has been demonstrated that spatial-spectral information (spectral imagery) or spatial-polarization information (polarization imagery) provides improved classification accuracies [2–4]. Multiband polarization imaging provides an effective means to observe surface characteristics as well as material properties of an object in the scene, which may not be readily obtained using conventional imaging techniques. Polarization information finds surface properties of an object such as diffuse or specular reflection, scattering, and refractive index. Spectral signatures of an object measured in different spectral bands are used to characterize the types of materials.Multiband polarization vision has gained significant interests in both biological and computer vision research communities [5] from the observations that some biological organisms in the nature demonstrate multiband polarization vision capabilities. Mantis shrimp and dragonfly are known to be able to easily detect and catch even transparent prey utilizing combined spectral and polarization information. They detect and recognize hidden or camouflaged objects or easily navigate through the water using the difference in polarization properties of the target and the background [2–4]. A compound eye of such organisms consists of tens of thousands of individual imaging units called ommatidia with effective imaging field-of-view of nearly 360 degrees. Each ommatidium responds to the light in different spectral bands as well as polarization angles. A unique structure of ommatidia in the eyes of mantis shrimp, squid, cuttlefish, and dragonfly is highly sensitive to polarized lights. In addition, the compound vision system demonstrates parallel processing capabilities that exist from retina to all ganglions [6, 7]. According to the studies on such organisms, a group of ommatidia are found in the dorsal rim area (DRA) of a compound eye. The DRAs of different organisms are known to have similar physiological characteristics. Every ommatidium contains two homochromatic photoreceptors that are orthogonal to each other. The photoreceptors in different ommatidia are sensitive to the light of different polarization angles [7–9]. Studies also indicate that the optic nerve systems of those organisms have particular polarization coding ability. From the research on retina in ommatidium of mantis shrimp and squid, the rhabdom and retinular cells have spectral perception ability from 300 nm to 700 nm with a perception bandwidth from 30 nm to 60 nm [10]. Mantis shrimp has 12 photoreceptors, each sampling a narrow set of wavelengths ranging from deep ultraviolet (300 nm) to far red (720 nm). Compared to three photoreceptors of the human visual system, those organisms can sense the light in a wider spectral region than humans do. Each type of photoreceptors is sensitive to a specific color. Mantis shrimp’s color vision system is based on temporal signaling combined with scanning eye movements, enabling color recognition rather than discrimination without brain-power-heavy comparisons [11]. This scheme probably gives the predatory shrimp a speed advantage in distinguishing prey with different color from cluttered background under changing light conditions [12]. In addition, the ommatidium in the hemisphere of the eye can identify the luminance, which acts as panchromatic imaging. Based on such physiological structures and functions, the organisms demonstrate multiband polarization vision in high resolution [13]. Multiband polarization vision system can be implemented based on the model of a cluster of ommatidia, where each ommatidium senses spectral or polarization information of the scene radiance in a particular spectral band or polarization angle.Applications of multiband polarization imaging in object classification and clustering have demonstrated the effectiveness of multiband polarization imaging techniques. The variations of polarization parameters with wavebands are closely linked to the physiochemical characteristics of materials. Different inherent properties can enhance object discrimination and classification even when no obvious intensity difference exists. The measurement of Fresnel reflection coefficients in multiple bands can quantitatively assess the conductive characteristics to retrieve the dielectric constants, which provides good detectability of conductors and insulators. Roughness and surface orientation can be reflected in the spectral polarization parameters, which is critical for inhomogeneous objects identification [14]. Multiband polarization imaging has demonstrated enhanced target detection and navigation in military applications [15, 16]. Medical diagnosis with cytometry imaging and tissue assessment [17, 18] and the other applications including the measurement of aerosol density in the atmosphere [19, 20], geological exploration of glacier and vegetation distribution [21, 22], image dehazing [23–25], land cover classification [26], pathologic diagnosis of epithelial tissues [27], and visualization of an object hidden in the shadow [28] are application examples to name a few. Section 4 introduces some of popular application examples such as atmospheric observation, earth remote sensing, medical diagnosis, surveillance and reconnaissance, dehazing, and 3D reconstruction of specular objects.


## 2. Multiband Polarization Imaging Principles

Multiband polarization imaging techniques capture spectral and polarization information inherent to the properties of a target. We analyze multiband polarization image data to distinguish the differences in spectropolarimetric information among different objects. Spectral imaging attempts to measure intensity distribution of the light over different wavelengths while polarization imaging finds intensity of the image in different polarization directions. Figure1 illustrates a hierarchy of multiband polarization imaging techniques. Two major steps to ensure precise multiband polarization imaging are spectral tuning and polarization adjustment. Multiband polarization imaging techniques obtain spectral information based on dispersion, channel tuning, or interference. Polarization information can be measured by rotating polarizer or micropolarization array. Images of different spectral bands and polarization angles are captured in sequence, which cannot satisfy time-sensitive imaging requirement.Figure 1
A hierarchy of multiband polarization imaging.


### 2.1. Spectral Tuning

Spectral imaging combines photography and spectroscopy to generate image data whose picture element (pixel) is associated with a spectral signature (spectrum). The spectral information provided by this pixel is valuable in the discrimination, detection, and classification of elements and structures within the image [29]. Each pixel in a spectral image is typically composed of narrow spectral bands of the electromagnetic spectrum. A spectral image I constitute a 3D data cube in two spatial coordinates (
x
,
y
) and one spectral dimension λ
(1)
I
x
,
y
,
λ
i
,
i
=
1,2
,
…
,
L
,where L denotes the number of spectral bands. Spectral imaging has the ability to exploit multiple regions of the electromagnetic spectrum to probe and analyze the composition of a material. The materials comprising various objects in a scene reflect, absorb, and emit electromagnetic radiation in amounts that vary with the wavelength. If the radiation arriving at the sensor is measured over a spectral range, the resulting spectral signature can be used to uniquely characterize and identify any given material. Spectral feature of a pixel in a spectral image is compared to a database of known materials to determine the type of the material of the pixel.In general, spectral components can be extracted using filters or devices with a function of spectral separation. Conventional spectral tuning methods include prism, mechanical filter wheel, diffraction grating, interference Fourier transform, and tomography [30, 31]. A flexible and programmable filtering elements such as acoustooptic tunable filter (AOTF) and liquid crystal tunable filter (LCTF) have been popular in spectral imaging. Multiband polarization imagers [2] collect spectral information using conventional methods such as diffraction grating, interference Fourier transform, and tunable filters and measure polarization information using optical polarizers. A multiband polarization imager using a single camera equipped with an LCTF and a polarizer wheel was proposed to capture simultaneously spectral and polarization information [21]. This imager obtains spectral information by sequentially tuning the center frequency of a bandpass characteristic of LCTF in different spectral bands and a set of component images of different polarization angles by rotating the polarizer in a filter wheel using a stepper motor. Such multiband polarization imaging techniques that capture spectral and polarization information separately and individually in sequence tend to be time consuming and require complicated optical, mechanical, and electronic devices. A multiband polarization imager was built for navigation with a constant-gain omnidirectional mirror, an UV camera, and a color camera [32]. The goal was to measure the polarization pattern of the sky. The images in different polarization angles are captured in sequence, with large distortion due to the omnidirectional mirror. Such types of polarization imagers may not capture moving objects due to slow response time. In the beam splitting method, the camera can only measure one-third of the input energy with a difficulty to acquire multiband polarization information.Compared with conventional optical imaging methods, spectral imaging techniques have access to spectral resolution, spatial resolution, radiative resolution, and time resolution of a target. The four resolutions enable reflecting the intrinsic spectral signatures of a target and distinguish the spectral differences among variable objects. Various spectral tuning methods being used in multiband polarization imaging systems are listed as Table1.Table 1
Spectral tuning methods in multiband polarization imaging systems.
Design features
Basic principles
Features
Dispersion
Prisminterferometer
Light dispersion
Easy to realizeLow spatial resolution Low spectral resolution
Interference
Interferometer
Fourier transformsSpectral pixel interferogram
Without calibrationHigh imaging efficiency Low spatial resolution
Spectral filtering
Optical filters
AOTF: light diffractionLCTF: electrical tuning
Easy to realizeSimple and compact Low imaging efficiency
Tomography
Tomography imager
Tomography projection
Complex structureFull-field High spectral resolution


### 2.2. Polarization Adjustment

Polarization is an important physical quantity describing the physicochemical properties of an object during the interaction of reflection, scattering, and transmission with solar radiation. Polarization imaging acquires multiple images in different polarization angles with abundant spectral information to improve the discrimination capability. Polarization imaging techniques have demonstrated such advantages as high signal-to-noise ratio and strong contrast over conventional optical imaging methods [33, 34]. Polarization imaging techniques have been successfully utilized in remote sensing and computer vision fields [34, 35].Polarization features of an object are usually represented using the Jones vector, Stokes vector, and Muller matrix [56]. Strategies such as rotating polarizer, splitter, and division of FPA have been utilized to modulate polarization states [57]. Instantaneous acquisition of the Stokes parameters is of great importance in polarization imaging research. Multiband polarization imagery represents spatial, spectral, and polarization information of an object using four Stokes parameters describing the state of polarization in multiple bands. Suppose an object was images at four different polarization directions, 0, 45, 90, and 135 degrees at a certain wavelength λ. Let I
0
(
x
,
y
), I
45
(
x
,
y
), I
90
(
x
,
y
), and I
135
(
x
,
y
) denote the four image components measured at different polarization angles by rotating the polarizer to different orientations. Then the Stokes vector at wavelength λ is given by(2)
S
λ
x
,
y
=
S
0
,
λ
x
,
y
S
1
,
λ
x
,
y
S
2
,
λ
x
,
y
S
3
,
λ
x
,
y
=
I
0
,
λ
x
,
y
+
I
90
,
λ
x
,
y
I
0
,
λ
x
,
y
-
I
90
,
λ
x
,
y
I
45
,
λ
x
,
y
-
I
135
,
λ
x
,
y
I
R
,
λ
x
,
y
-
I
L
,
λ
x
,
y
,where S
0
,
λ is the total intensity of light at wavelength λ, S
1
,
λ donates the difference between the light intensity at 0° and 90°, S
2
,
λ represents the difference between 45° and 135° linear components, and S
3
,
λ is the circular right to left polarization state. Circular polarization component S
3
,
λ of a ground scene tends to be small, and therefore is often negligible [57]. When a linear polarization analysis is taken into account, the degree of linear polarization (DoLP) can be represented by a fraction of intensity attributed to the polarized light state as [34](3)
D
o
L
P
λ
x
,
y
=
S
1
,
λ
2
x
,
y
+
S
2
,
λ
2
x
,
y
S
0
,
λ
x
,
y
.Polarization angle (Orient) indicates the angle of major axis of polarization ellipse with respect to the reference direction (x-axis) (4)
Orient
λ
⁡
x
,
y
=
1
2
tan
-
1
⁡
S
2
,
λ
x
,
y
S
1
,
λ
x
,
y
.Joint spatial, spectral, and polarization information can be represented by seven independent variables: spatial coordinates(
x
,
y
), wavelength (
λ
), and polarization angles (S
0, S
1, S
2, S
3). Based on the mathematic description of polarization, multiples images are required to characterize the polarization states of scenes. With the advance of photodetectors, there are a number of strategies to capture polarization signatures [21, 57]. Table 2 shows strategies of polarization adjustment.Table 2
Polarization adjustment strategies.
Design features
Integration issues
Imaging issues
Rotating polarizer
Simple and robustTime consuming Suitable for static scenes
InexpensiveEasy to implement
MisregistrationLow efficiency
Beam splitter
Large sizeLight splitting Simultaneous acquisition
ExpensiveComplex integration High mechanical flexibility
Easy to alignHigh efficiency Low contrast
Division of aperture
Small sizeSimultaneous acquisition
ExpensiveComplex optical elements
Easy to alignLow spatial resolution
Division of FPA
Small sizeCompact Simultaneous acquisition
ExpensiveComplex fabrication High mechanical flexibility
High efficiencyImaging blur Low spatial resolution


## 2.1. Spectral Tuning

Spectral imaging combines photography and spectroscopy to generate image data whose picture element (pixel) is associated with a spectral signature (spectrum). The spectral information provided by this pixel is valuable in the discrimination, detection, and classification of elements and structures within the image [29]. Each pixel in a spectral image is typically composed of narrow spectral bands of the electromagnetic spectrum. A spectral image I constitute a 3D data cube in two spatial coordinates (
x
,
y
) and one spectral dimension λ
(1)
I
x
,
y
,
λ
i
,
i
=
1,2
,
…
,
L
,where L denotes the number of spectral bands. Spectral imaging has the ability to exploit multiple regions of the electromagnetic spectrum to probe and analyze the composition of a material. The materials comprising various objects in a scene reflect, absorb, and emit electromagnetic radiation in amounts that vary with the wavelength. If the radiation arriving at the sensor is measured over a spectral range, the resulting spectral signature can be used to uniquely characterize and identify any given material. Spectral feature of a pixel in a spectral image is compared to a database of known materials to determine the type of the material of the pixel.In general, spectral components can be extracted using filters or devices with a function of spectral separation. Conventional spectral tuning methods include prism, mechanical filter wheel, diffraction grating, interference Fourier transform, and tomography [30, 31]. A flexible and programmable filtering elements such as acoustooptic tunable filter (AOTF) and liquid crystal tunable filter (LCTF) have been popular in spectral imaging. Multiband polarization imagers [2] collect spectral information using conventional methods such as diffraction grating, interference Fourier transform, and tunable filters and measure polarization information using optical polarizers. A multiband polarization imager using a single camera equipped with an LCTF and a polarizer wheel was proposed to capture simultaneously spectral and polarization information [21]. This imager obtains spectral information by sequentially tuning the center frequency of a bandpass characteristic of LCTF in different spectral bands and a set of component images of different polarization angles by rotating the polarizer in a filter wheel using a stepper motor. Such multiband polarization imaging techniques that capture spectral and polarization information separately and individually in sequence tend to be time consuming and require complicated optical, mechanical, and electronic devices. A multiband polarization imager was built for navigation with a constant-gain omnidirectional mirror, an UV camera, and a color camera [32]. The goal was to measure the polarization pattern of the sky. The images in different polarization angles are captured in sequence, with large distortion due to the omnidirectional mirror. Such types of polarization imagers may not capture moving objects due to slow response time. In the beam splitting method, the camera can only measure one-third of the input energy with a difficulty to acquire multiband polarization information.Compared with conventional optical imaging methods, spectral imaging techniques have access to spectral resolution, spatial resolution, radiative resolution, and time resolution of a target. The four resolutions enable reflecting the intrinsic spectral signatures of a target and distinguish the spectral differences among variable objects. Various spectral tuning methods being used in multiband polarization imaging systems are listed as Table1.Table 1
Spectral tuning methods in multiband polarization imaging systems.
Design features
Basic principles
Features
Dispersion
Prisminterferometer
Light dispersion
Easy to realizeLow spatial resolution Low spectral resolution
Interference
Interferometer
Fourier transformsSpectral pixel interferogram
Without calibrationHigh imaging efficiency Low spatial resolution
Spectral filtering
Optical filters
AOTF: light diffractionLCTF: electrical tuning
Easy to realizeSimple and compact Low imaging efficiency
Tomography
Tomography imager
Tomography projection
Complex structureFull-field High spectral resolution


## 2.2. Polarization Adjustment

Polarization is an important physical quantity describing the physicochemical properties of an object during the interaction of reflection, scattering, and transmission with solar radiation. Polarization imaging acquires multiple images in different polarization angles with abundant spectral information to improve the discrimination capability. Polarization imaging techniques have demonstrated such advantages as high signal-to-noise ratio and strong contrast over conventional optical imaging methods [33, 34]. Polarization imaging techniques have been successfully utilized in remote sensing and computer vision fields [34, 35].Polarization features of an object are usually represented using the Jones vector, Stokes vector, and Muller matrix [56]. Strategies such as rotating polarizer, splitter, and division of FPA have been utilized to modulate polarization states [57]. Instantaneous acquisition of the Stokes parameters is of great importance in polarization imaging research. Multiband polarization imagery represents spatial, spectral, and polarization information of an object using four Stokes parameters describing the state of polarization in multiple bands. Suppose an object was images at four different polarization directions, 0, 45, 90, and 135 degrees at a certain wavelength λ. Let I
0
(
x
,
y
), I
45
(
x
,
y
), I
90
(
x
,
y
), and I
135
(
x
,
y
) denote the four image components measured at different polarization angles by rotating the polarizer to different orientations. Then the Stokes vector at wavelength λ is given by(2)
S
λ
x
,
y
=
S
0
,
λ
x
,
y
S
1
,
λ
x
,
y
S
2
,
λ
x
,
y
S
3
,
λ
x
,
y
=
I
0
,
λ
x
,
y
+
I
90
,
λ
x
,
y
I
0
,
λ
x
,
y
-
I
90
,
λ
x
,
y
I
45
,
λ
x
,
y
-
I
135
,
λ
x
,
y
I
R
,
λ
x
,
y
-
I
L
,
λ
x
,
y
,where S
0
,
λ is the total intensity of light at wavelength λ, S
1
,
λ donates the difference between the light intensity at 0° and 90°, S
2
,
λ represents the difference between 45° and 135° linear components, and S
3
,
λ is the circular right to left polarization state. Circular polarization component S
3
,
λ of a ground scene tends to be small, and therefore is often negligible [57]. When a linear polarization analysis is taken into account, the degree of linear polarization (DoLP) can be represented by a fraction of intensity attributed to the polarized light state as [34](3)
D
o
L
P
λ
x
,
y
=
S
1
,
λ
2
x
,
y
+
S
2
,
λ
2
x
,
y
S
0
,
λ
x
,
y
.Polarization angle (Orient) indicates the angle of major axis of polarization ellipse with respect to the reference direction (x-axis) (4)
Orient
λ
⁡
x
,
y
=
1
2
tan
-
1
⁡
S
2
,
λ
x
,
y
S
1
,
λ
x
,
y
.Joint spatial, spectral, and polarization information can be represented by seven independent variables: spatial coordinates(
x
,
y
), wavelength (
λ
), and polarization angles (S
0, S
1, S
2, S
3). Based on the mathematic description of polarization, multiples images are required to characterize the polarization states of scenes. With the advance of photodetectors, there are a number of strategies to capture polarization signatures [21, 57]. Table 2 shows strategies of polarization adjustment.Table 2
Polarization adjustment strategies.
Design features
Integration issues
Imaging issues
Rotating polarizer
Simple and robustTime consuming Suitable for static scenes
InexpensiveEasy to implement
MisregistrationLow efficiency
Beam splitter
Large sizeLight splitting Simultaneous acquisition
ExpensiveComplex integration High mechanical flexibility
Easy to alignHigh efficiency Low contrast
Division of aperture
Small sizeSimultaneous acquisition
ExpensiveComplex optical elements
Easy to alignLow spatial resolution
Division of FPA
Small sizeCompact Simultaneous acquisition
ExpensiveComplex fabrication High mechanical flexibility
High efficiencyImaging blur Low spatial resolution


## 3. Multiband Polarization Imaging Systems

Inspired by the vision mechanism of some biological organisms, we developed a multiband polarization imaging system. The multiband polarization imager consists of a 3-by-3 array of CCD cameras, where each camera is equipped with either a color filter or an optical polarizer. An individual camera was set to capture an image in a certain spectral band or a polarization angle. This imager is to capture multiple images of the same scene in different spectral bands and polarization angles simultaneously for tasks such as target detection, so a large amount of geometrical distortion is not acceptable. Figure2 shows a schematic diagram of the proposed multiband polarization imager for the acquisition of multiband spectral and polarization information from the scene. We use an optical polarizer to simulate the photoreceptor that is sensitive to the polarized light. The rhabdom and retinular cells having spectral sensing ability are simulated by a spectral bandpass filter to obtain multiband spectral image data. A 9-way gigabit ethernet card processes multiband spectral and polarization image data in parallel. This imager is configured to have five cameras with five color filters in the spectral bands of red, green, blue, yellow, and orange and four cameras with four optical polarizers of 0, 45, 90, and 135 degrees.Figure 2
Acquisition of multiband spectral and polarization information of the scene using a camera array.Figure3 shows physical appearance of a prototype multiband polarization imaging system and the configuration of component cameras. For the sake of simplicity in implementation, nine (N
=
9) component CCD cameras are arranged in a 3-by-3 rectangular array rather than a hexagonal array as in the compound eye of an insect eye. Each camera is equipped with a color filter or a polarizer. Table 3 shows the arrangement of nine individual component cameras. To measure the polarization information with less noise or error, polarizers with four polarization angles of 0°, 45°, 90°, and 135° are used, mounted on the camera positions of (
1,2
), (
2,1
), (
2,3
), and (
3,2
). The spectral bandpass filters of red (600–700 nm), orange (590–610 nm), yellow (570–590 nm), green (490–570 nm), and blue (450–490 nm) measure the spectral information in five bands in the visible spectrum, mounted on the camera positions of (
1,1
), (
1,3
), (
2,2
), (
3,1
), and (
3,3
) respectively. An industrial CCD camera (Basler Ace 1300, 30 gm) of dimension 42 mm × 29 mm × 29 mm (W
×
D
×
H) was used for each component camera. A large amount of image data coming from nine CCD cameras is processed and transmitted in parallel using 9-channel gigabit ethernet. An optical lens with the focal length of 16 mm, viewing angles of 38° (diagonal), 30.8° (horizontal), and 23.4° (vertical) was used in all the cameras. For a cooling purpose, CCD cameras are arranged with a gap of 22 mm.Table 3
Placement of color filters and polarizers in a multiband polarization imager.
Camera number
Position
Polarization
Spectral band (nm)
1
(1, 1)
—
600–700  (red)
2
(1, 2)
0°
—
3
(1, 3)
—
590–610 (orange)
4
(2, 1)
45°
—
5
(2, 2)
—
570–590 (yellow)
6
(2, 3)
90°
—
7
(3, 1)
—
490–570 (green)
8
(3, 2)
135°
—
9
(3, 3)
—
450–490 (blue)Figure 3
Hardware configuration of a prototype multiband polarization imager. (a) Physical appearance of a 3-by-3 camera array and (b) arrangement of component cameras.
(a)
(b)Unlike sequential imaging process of conventional multiband polarization imagers, the proposed imager can capture simultaneously a set of images of the scene in multiple spectral bands as well as different polarization angles. Spectral and polarization information is extracted using color filters and optical polarizers. Multiband spectral and polarization information can be measured in the common FOV region that is viewed by all the component cameras. Due to the different viewing angles of each component camera and the resulting mismatch in the FOV, the spectral and polarization information is not complete in nonoverlapping, boundary region. This will reduce the effective FOV of the multiband polarization imager. To expand the FOV, missing spectral and polarization information in the boundary region must be recovered. An attempt has been made to estimate the missing spectral and polarization information in the expanded FOV using the low-rank matrix recovery method [43, 44] that exploits the redundancy and correlation of the measured data.


## 4. Applications of Multiband Polarization Imaging Techniques

Multiband polarization imaging offers simultaneous acquisition of spectral and polarimetric signatures of an object for the detection of targets camouflaged or hidden in cluttered background. This technique has demonstrated improved object detection capabilities in a wide variety of applications in optical metrology that range from atmospheric science and remote sensing to 3D reconstruction. Table4 lists some of popular application examples and the corresponding mainstream sensors and techniques used.Table 4
Application examples and mainstream sensors/techniques of multiband polarization imaging techniques.
Application examples
Mainstream sensors/techniques
Atmospheric observation
Electromechanical rotationDispersion, Spectral filtering [19, 20, 36, 37]  Multi-CCDs coordination [38]
Earth remote sensing
Interference, FPA-integrated CCDsSpectro-polarimetric retrieval [21, 22, 34]
Medical diagnosis
Multimodal sensors with tomographyEndoscopy with LCTF-based CCDs [39–42]
Surveillance and reconnaissance
CCDs with polarizer/apertureMulti-data fusion [15, 43–47]
Image dehazing
Spectral filteringCCDs with polarizer/splitter [23, 48–51]
3D reconstruction
Single CCD/multi-CCDs with polarizerMultiview/binocular image fusion [52–55]


### 4.1. Atmospheric Observation

Multiband polarization imaging can be used to acquire spectral and polarization features of suspended atmospheric particles, which is useful in the correction of atmospheric distortions, climate investigation, and astronomical sensing [19]. Due to inherent polarization effect caused by the atmospheric scattering and absorption of the light, the polarization states of aerosol particles in multiple spectral bands help to discriminate distribution, category, height, density, and size of suspended particles. Spectrum of metastable atomic oxygen in the upper atmosphere can be detected, which enables accurate measurements of the atmospheric properties, such as velocity, humidity, and temperature [36].Early atmospheric exploration using multiband polarization imaging techniques starts in 1980s. The Polarization and Directionality of the Earth’s Reflectances (POLDER) instrument has been used to obtain spectral and polarization information of aerosol to detect the atmospheric distribution, with the wavebands centered at 443 nm, 670 nm, and 865 nm [22]. Sano et al. [20] validated the accuracy of suspended particles distribution by extending the observation to six spectral bands. Airborne multiband polarimetric investigation [58] was conducted in the spectral range from 400 nm to 1000 nm. The polarization properties of aerosols with wavebands were analyzed. Li et al. [37] exploited ground-based multiband polarization data to retrieve the optical parameters of atmospheric aerosols, such as the optical density, size, polarization phase, and negative refractive index. Gartley et al. [59] explored multiband polarization detection by extending the wavebands from 200 nm to 250 nm, which enables achieving effective measurements of spectral aerosol absorption. Marbach et al. [38] incorporated multiview and multiangular schemes into multiband polarization imaging to resolve the directional anisotropy and the microphysical properties of aerosols. Redding et al. [60] developed an experimental multiband polarization apparatus. This setup extracts the polarization ratio and spectral features to identify the types and aggregations of aerosol particles. Distribution and density of aerosols between the two different weather conditions were successfully discriminated.


### 4.2. Earth Remote Sensing

Spectropolarimetric signatures of radiation energy reflected by ground targets have been applied in earth remote sensing. The joint spatial, spectral, and polarization information are more discriminative and accurate to describe the geometry and physicochemical properties of terrain targets, outperforming traditional sensing methods that capture intensity information only. The salient differences reflected in spectral polarization properties among various types of plants can be utilized to estimate the growth and the biomass of crops, plants species, and distribution and coverage area of forests [21, 22]. In earth remote sensing, exploiting spectropolarimetric differences reflected by various components like rocks and plants enables vegetation distributions. Microwave reflective and emissive characteristics of snow, water surface retrieved by the combination of spatial, spectral, and polarimetric detection, have potential applications in remote sensing, such as snowfall and rainfall parameters, sea salinity, wave height, marine pollution, oil spills, and coastline identification [61].Recent research on red tide detection with spectral polarized radiance measurements highlights the performance of multiband polarization imaging techniques, which gives incomparable detection accuracy than conventional detection methods. By investigating spectral variation and polarization effect of phytoplankton, the red tide species discrimination was implemented [62]. Multiband polarization imaging techniques are accessible for explorations of ice caves and glaciers [63]. Geological evolution can be recurred via detecting the polarized signatures of reflectance and fluorescence spectra. With the difference of spectral polarized information reflected by sand, soil, and water radiation, multiband polarization imaging techniques reveal significant information to observe landforms and geology. Mineral distribution and composition are approachable to be detected, with the inherent spectral polarized features. Maturity and dryness can be easily discriminated. Since multiband polarization imaging provides informative ground characteristics and thereby can be a useful indicator for evaluating the state of land desertification and erosion [64].


### 4.3. Medical Diagnosis

Pathological changes modify birefringence and structure of the tissue, which can be measured in terms of the polarization and spectral changes of scattered light. Multiband polarization imaging has been a powerful diagnostic tool to measure the features for quantitative pathology analysis. Absorption spectra in multiple wavebands are associated with molecular aggregations and provide information on the structure of biological tissues. Reflective index affected by the pathology of cell kernel and collagen can be retrieved by polarization analysis. Therefore, the compositional characteristics of organic tissues can be detected via spectral polarimetric diagnosis, such as the size and category of cell kernel and collagen [17, 18, 39, 65].Widely applied in medical diagnosis, multiband polarization imaging strategy is an emerging technology for cytometry imaging, due to the operational flexibility of fast spectra collections and blood density estimation [40, 41]. Complementary to traditional X-ray-based diagnosis, multiband polarization detection optimizes the assessment of bone tissue. As a nondestructive measurement of biochemical properties, multiband polarization imaging is sensitive to local changes in mineral maturity regarding the spectral polarized signatures. Those attributes make multiband polarization imaging as a potential indicator for clinical diagnosis, like fracture risk and bone damage assessment [42]. The severity of natural caries lesions on occlusal surfaces can also be resolved using the joint spectra and polarization tomography. The integrated reflectivity is acquired to monitor the mineral loss and dental decay in the occlusal pit and fissures [66]. Multiband polarization imaging is effective to diagnose organic characteristics of skin pathology. Efforts have been applied in the skin melanin cancerous melanotic nevus detection and chilblain tissue diagnosis [67].


### 4.4. Surveillance and Reconnaissance

Multiband polarization imaging detection proves to have prominent performance in military strikes. The identification and tracking of missiles is accomplished using spectral polarization analysis of exhaust plume and fume. A strong polarization effect caused by fume in multiple bands is likely to expose the missile routes [45, 46]. Conventional photoelectric detection often fails to distinguish man-made objects with low contrast. However, obvious differences in spectral and polarization features exist between military targets and natural objects. Multiband polarization imaging seeks to identify the geometric description of anomalous materials in cluttered background, even hidden or camouflage targets like tanks and armored vehicles under the trees [3, 4, 15, 43, 44, 47]. Multiband polarization imaging detection visually enhances the contrast of hidden targets compared to intensity-only images. With a high refractive index of the artifacts, spectral and polarization signatures from multiband polarization images fusion can be applied to ground reconnaissance and battle damage assessment [68]. On the other hand, multiband polarization detection impels the design of new type of camouflage coating materials [35]. By modulating the modality and roughness, the artifacts emit homothetic polarization spectra with natural background, which make it difficult to discriminate.Outperforming conventional optical detections, multiband polarization imaging has particular merits in surveillance and reconnaissance. Polarization images in different spectral bands are exploited to retrieve the sea regime, which ensures sailing safety [58, 69]. Aircraft aviation often encounters difficulties in overcast conditions, due to the severe erosion and turbulence caused by the ice crystal in cirrus. Such atmospheric disturbance is disruptive to the precision of navigation. However, crystal aerosols are strongly polarized of incident radiation, and the joint spectropolarimetric signatures are flexible for the detection of cirrus aggregation, which guarantees aircraft navigation and guidance [70, 71].


### 4.5. Image Dehazing

Outdoor imaging in poor weather conditions such as haze and mist remains a challenging task in practice. Captured images are pervasively plagued with nontrivial degradations caused by atmospheric particles. The airlight is partially polarized and dominates in the measured radiation. Taking into account the polarization parameters of atmospheric scattering and the refinement of transmission in three visible bands, the haze effect can be effectively eliminated using multiband polarization dehazing methods [23, 48, 49]. Image details and color fidelity of the scene are remarkably improved, which is closely comparable to the original haze-free scene. Compared to prior-based image dehazing methods, multiband polarization techniques provide more accurate visual effects, due to the unique physics-based analysis of intrinsic properties of targets [50, 51]. Multiband polarization scheme has performed physics-based effectiveness in descattering and visibility enhancement in turbid media, even liquids and solids. This unified spectral polarization imaging has remarkable advantages in specific applications, such as underwater inspection, torpedo navigation, and ecology evaluation [72, 73]. Overlapping cast shadow removal can be implemented based on spectral polarization analysis [74, 75]. Unexpected highlight and flare suppression is efficiently actualized by multiband polarimetric observation.


### 4.6.3D Reconstruction of Specular Objects

A three-dimensional (3D) reconstruction of less-textured objects with specular surfaces can be a challenging task. Due to the lack of features, classical photometric 3D reconstruction algorithms may fail to acquire dense disparity since these methods rely on precise alignments, like binocular stereo vision. A 3D shape reconstruction with multiband polarization imaging has demonstrated effectiveness in the inspection of specular objects, disregarding the texture information [52, 76]. Reflection of unpolarized light becomes partially polarized according to the dielectric index of the surface. Using the Fresnel laws, the geometrical parameters such as zenith angle and azimuth angle can be retrieved from the degree of polarization and the angle of polarization, respectively. Therefore, the normal vector is integrated to reconstruct the depth information. Refractive index differences in multiple spectra can be employed to resolve the ambiguity on the zenith angle [53, 77]. Multiband polarization technique can be employed as an alternative 3D inspection method of specular objects [54, 55]. Specular reflection component can be eliminated using polarization signatures and spectra coherence constraints, while maintaining high visual quality [78].


## 4.1. Atmospheric Observation

Multiband polarization imaging can be used to acquire spectral and polarization features of suspended atmospheric particles, which is useful in the correction of atmospheric distortions, climate investigation, and astronomical sensing [19]. Due to inherent polarization effect caused by the atmospheric scattering and absorption of the light, the polarization states of aerosol particles in multiple spectral bands help to discriminate distribution, category, height, density, and size of suspended particles. Spectrum of metastable atomic oxygen in the upper atmosphere can be detected, which enables accurate measurements of the atmospheric properties, such as velocity, humidity, and temperature [36].Early atmospheric exploration using multiband polarization imaging techniques starts in 1980s. The Polarization and Directionality of the Earth’s Reflectances (POLDER) instrument has been used to obtain spectral and polarization information of aerosol to detect the atmospheric distribution, with the wavebands centered at 443 nm, 670 nm, and 865 nm [22]. Sano et al. [20] validated the accuracy of suspended particles distribution by extending the observation to six spectral bands. Airborne multiband polarimetric investigation [58] was conducted in the spectral range from 400 nm to 1000 nm. The polarization properties of aerosols with wavebands were analyzed. Li et al. [37] exploited ground-based multiband polarization data to retrieve the optical parameters of atmospheric aerosols, such as the optical density, size, polarization phase, and negative refractive index. Gartley et al. [59] explored multiband polarization detection by extending the wavebands from 200 nm to 250 nm, which enables achieving effective measurements of spectral aerosol absorption. Marbach et al. [38] incorporated multiview and multiangular schemes into multiband polarization imaging to resolve the directional anisotropy and the microphysical properties of aerosols. Redding et al. [60] developed an experimental multiband polarization apparatus. This setup extracts the polarization ratio and spectral features to identify the types and aggregations of aerosol particles. Distribution and density of aerosols between the two different weather conditions were successfully discriminated.


## 4.2. Earth Remote Sensing

Spectropolarimetric signatures of radiation energy reflected by ground targets have been applied in earth remote sensing. The joint spatial, spectral, and polarization information are more discriminative and accurate to describe the geometry and physicochemical properties of terrain targets, outperforming traditional sensing methods that capture intensity information only. The salient differences reflected in spectral polarization properties among various types of plants can be utilized to estimate the growth and the biomass of crops, plants species, and distribution and coverage area of forests [21, 22]. In earth remote sensing, exploiting spectropolarimetric differences reflected by various components like rocks and plants enables vegetation distributions. Microwave reflective and emissive characteristics of snow, water surface retrieved by the combination of spatial, spectral, and polarimetric detection, have potential applications in remote sensing, such as snowfall and rainfall parameters, sea salinity, wave height, marine pollution, oil spills, and coastline identification [61].Recent research on red tide detection with spectral polarized radiance measurements highlights the performance of multiband polarization imaging techniques, which gives incomparable detection accuracy than conventional detection methods. By investigating spectral variation and polarization effect of phytoplankton, the red tide species discrimination was implemented [62]. Multiband polarization imaging techniques are accessible for explorations of ice caves and glaciers [63]. Geological evolution can be recurred via detecting the polarized signatures of reflectance and fluorescence spectra. With the difference of spectral polarized information reflected by sand, soil, and water radiation, multiband polarization imaging techniques reveal significant information to observe landforms and geology. Mineral distribution and composition are approachable to be detected, with the inherent spectral polarized features. Maturity and dryness can be easily discriminated. Since multiband polarization imaging provides informative ground characteristics and thereby can be a useful indicator for evaluating the state of land desertification and erosion [64].


## 4.3. Medical Diagnosis

Pathological changes modify birefringence and structure of the tissue, which can be measured in terms of the polarization and spectral changes of scattered light. Multiband polarization imaging has been a powerful diagnostic tool to measure the features for quantitative pathology analysis. Absorption spectra in multiple wavebands are associated with molecular aggregations and provide information on the structure of biological tissues. Reflective index affected by the pathology of cell kernel and collagen can be retrieved by polarization analysis. Therefore, the compositional characteristics of organic tissues can be detected via spectral polarimetric diagnosis, such as the size and category of cell kernel and collagen [17, 18, 39, 65].Widely applied in medical diagnosis, multiband polarization imaging strategy is an emerging technology for cytometry imaging, due to the operational flexibility of fast spectra collections and blood density estimation [40, 41]. Complementary to traditional X-ray-based diagnosis, multiband polarization detection optimizes the assessment of bone tissue. As a nondestructive measurement of biochemical properties, multiband polarization imaging is sensitive to local changes in mineral maturity regarding the spectral polarized signatures. Those attributes make multiband polarization imaging as a potential indicator for clinical diagnosis, like fracture risk and bone damage assessment [42]. The severity of natural caries lesions on occlusal surfaces can also be resolved using the joint spectra and polarization tomography. The integrated reflectivity is acquired to monitor the mineral loss and dental decay in the occlusal pit and fissures [66]. Multiband polarization imaging is effective to diagnose organic characteristics of skin pathology. Efforts have been applied in the skin melanin cancerous melanotic nevus detection and chilblain tissue diagnosis [67].


## 4.4. Surveillance and Reconnaissance

Multiband polarization imaging detection proves to have prominent performance in military strikes. The identification and tracking of missiles is accomplished using spectral polarization analysis of exhaust plume and fume. A strong polarization effect caused by fume in multiple bands is likely to expose the missile routes [45, 46]. Conventional photoelectric detection often fails to distinguish man-made objects with low contrast. However, obvious differences in spectral and polarization features exist between military targets and natural objects. Multiband polarization imaging seeks to identify the geometric description of anomalous materials in cluttered background, even hidden or camouflage targets like tanks and armored vehicles under the trees [3, 4, 15, 43, 44, 47]. Multiband polarization imaging detection visually enhances the contrast of hidden targets compared to intensity-only images. With a high refractive index of the artifacts, spectral and polarization signatures from multiband polarization images fusion can be applied to ground reconnaissance and battle damage assessment [68]. On the other hand, multiband polarization detection impels the design of new type of camouflage coating materials [35]. By modulating the modality and roughness, the artifacts emit homothetic polarization spectra with natural background, which make it difficult to discriminate.Outperforming conventional optical detections, multiband polarization imaging has particular merits in surveillance and reconnaissance. Polarization images in different spectral bands are exploited to retrieve the sea regime, which ensures sailing safety [58, 69]. Aircraft aviation often encounters difficulties in overcast conditions, due to the severe erosion and turbulence caused by the ice crystal in cirrus. Such atmospheric disturbance is disruptive to the precision of navigation. However, crystal aerosols are strongly polarized of incident radiation, and the joint spectropolarimetric signatures are flexible for the detection of cirrus aggregation, which guarantees aircraft navigation and guidance [70, 71].


## 4.5. Image Dehazing

Outdoor imaging in poor weather conditions such as haze and mist remains a challenging task in practice. Captured images are pervasively plagued with nontrivial degradations caused by atmospheric particles. The airlight is partially polarized and dominates in the measured radiation. Taking into account the polarization parameters of atmospheric scattering and the refinement of transmission in three visible bands, the haze effect can be effectively eliminated using multiband polarization dehazing methods [23, 48, 49]. Image details and color fidelity of the scene are remarkably improved, which is closely comparable to the original haze-free scene. Compared to prior-based image dehazing methods, multiband polarization techniques provide more accurate visual effects, due to the unique physics-based analysis of intrinsic properties of targets [50, 51]. Multiband polarization scheme has performed physics-based effectiveness in descattering and visibility enhancement in turbid media, even liquids and solids. This unified spectral polarization imaging has remarkable advantages in specific applications, such as underwater inspection, torpedo navigation, and ecology evaluation [72, 73]. Overlapping cast shadow removal can be implemented based on spectral polarization analysis [74, 75]. Unexpected highlight and flare suppression is efficiently actualized by multiband polarimetric observation.


## 4.6.3D Reconstruction of Specular Objects

A three-dimensional (3D) reconstruction of less-textured objects with specular surfaces can be a challenging task. Due to the lack of features, classical photometric 3D reconstruction algorithms may fail to acquire dense disparity since these methods rely on precise alignments, like binocular stereo vision. A 3D shape reconstruction with multiband polarization imaging has demonstrated effectiveness in the inspection of specular objects, disregarding the texture information [52, 76]. Reflection of unpolarized light becomes partially polarized according to the dielectric index of the surface. Using the Fresnel laws, the geometrical parameters such as zenith angle and azimuth angle can be retrieved from the degree of polarization and the angle of polarization, respectively. Therefore, the normal vector is integrated to reconstruct the depth information. Refractive index differences in multiple spectra can be employed to resolve the ambiguity on the zenith angle [53, 77]. Multiband polarization technique can be employed as an alternative 3D inspection method of specular objects [54, 55]. Specular reflection component can be eliminated using polarization signatures and spectra coherence constraints, while maintaining high visual quality [78].


## 5. Conclusion

Multiband polarization imaging is an emerging photoelectric detection technique with the ability to simultaneously acquire spectral and polarization information from an object of interest. This paper summarizes the principles and recent advances of multiband polarization imaging techniques. A multiband polarization imaging system implemented using a 2D camera array is introduced, which is inspired by a combined spectral and polarization vision mechanism of biological organisms. A number of application examples of multiband polarization imaging techniques are presented that include atmospheric observation, earth remote sensing, medical diagnosis, surveillance and reconnaissance, image dehazing, and 3D reconstruction of specular objects.



---

*Source: 5985673-2015-11-16.xml*