USOO962.5318B2(12) United States Patent(10) Patent No.:US 9.625,318 B2(45) Date of Patent:
Apr. 18, 2017

Kester et al.
(54)(58) Field of Classification Search

DIVIDED-APERTURE INFRA-RED
SPECTRAL MAGING SYSTEM FOR
CHEMICAL DETECTION

CPC .......... G01J 3/36; G01J 3/0232; G01J 3/2823;
GO1 3f2826

See application file for complete search history.
(71) Applicant: REBELLION PHOTONICS, INC.,
Houston, TX (US)(56)

References Cited
U.S. PATENT DOCUMENTS(72) Inventors: Robert T. Kester, Pearland, TX (US);Nathan A. Hagen, Houston, TX (US)4,134,683 A
4,963,963 A
5,127,742 A(73) Assignee: Rebellion Photonics, Inc., Houston, TX(US)(*) Notice: Subject to any disclaimer, the term of thispatent is extended or adjusted under 35
U.S.C. 154(b) by 0 days.
(21) Appl. No.: 14/543,692(22) Filed:

1/1979 Goetz et al.
10, 1990 Dorman
7, 1992 Fraden

5,877,500 A

3/1999 Braig et al.

5,920,066 A

7/1999 DiRenzo et al.

5,926,283 A

7/1999 Hopkins(Continued)
FOREIGN PATENT DOCUMENTS
CACA

2 365 866
9, 2000
WO 2009094782 A1 * 8, 2009

Nov. 17, 2014

(65)

(Continued)

Prior Publication Data

US 2015/OO69239 A1................ GO1J 3.02

OTHER PUBLICATIONS

Mar. 12, 2015
Williams et al. “Dual-Band MWIRLWIR Radiometer for Absolute

Related U.S. Application Data

Temperature Measurements”. Proc. of SPIE vol. 6205 62050, p.

ofapplication
No.
(63) Continuation
PCT/US2013/041278, filed on May 16, 2013.

(Continued)
Primary Examiner – Casey Bryant(74) Attorney, Agent, or Firm — Knobbe Martens

1-13.

(60) Provisional application No. 61/764,776, filed on Feb.
14, 2013, provisional application No. 61/688,630,
filed on May 18, 2012.
(51)

Int. C.

GOI 3/02
GOI 3/36
GOI 3/28(52)(2006.01)(2006.01)(2006.01)

U.S. C.

CPC ............... G0IJ 3/36 (2013.01); G0IJ 3/0232(2013.01); G0IJ 3/2823 (2013.01); G0IJ
2003/2826 (2013.01)

Olson & Bear LLP(57)

ABSTRACT

A divided-aperture infrared spectral imaging (DAISI) sys
tem that is structured to provide identification of target
chemical content in a single imaging shot based on spec
trally-multiplexed operation. The system is devoid of spec
tral scanning acquisition of infrared (IR) spectral signaturesof target content with an IR detector and does not require
COntent.

29 Claims, 11 Drawing Sheets
Moveable

400

temperatureN

controlled shuttersea less is440

Fisic effect his

460a, 460by

Pass, 454

438

ei:

1

as::::::stars. 424a

es: 3.328

HD- isc. 458
450

r

1E
cassessesssssf.

128a

-N444

US 9,625,318 B2
Page 2(56)

References Cited
U.S. PATENT DOCUMENTS

6,023,061 A *2/2000 Bodkin .................. B64D 47/08

6,268,883 B1

7/2001 Zehnder et al.

250/252.1

6,456,261 B1

9/2002 Zhang

6,680,778 B2
7,242.478 B1
7,315,377 B2
7,750,802 B1

1/20047/20071/2008
7, 2010

7,835,002 B2
7,888,624 B1

11/2010 Muhammed et al.
2/2011 Murguia et al.

8,159,568 B2
8,653,461 B1

4/2012 Ahdoot
2/2014 Benson et al.

8,654,328 B2

2/2014 Tkaczyk et al.

9,562,849 B2
2002fOO15151 A1
2002/O121370 A1
2002fO159101 A1
2003/0134426 A1
2005, 0103989 A1
2006, OO44562 A1
2008/0204744 A1
2008, 0231719 A1

Hinnrichs et al.
Dombrowski et al.
Holland et al.
Parish et al.

11.

Chu et al., “The NIST Quantitiative Infrared Database'. Journal of
Research of the National Institute of Standards and Technology,
Jan.-Feb. 1999, vol. 104, No. 1, pp. 59-81.
DiPietro et al., “Hyperspectral Matched Filter with False-Alarm
Mitigation”. Optical Engineering, Jan. 2012, vol. 51, No. 1, pp.

2/2017 Kester et al.

O162O2-1-0162O2-7.

2/2002 Gorin

Eriksson et al., “Radiative Cooling Computed for Model Atmo
spheres'. Applied Optics, Dec. 1, 1982, vol. 21, No. 23, pp.

9/2002 Kurkjian et al.
10, 2002 Alderson et al.

4381-4388.

7/2003 Jiang et al.
5, 2005 Watson et al.
3/2006 Hagene et al.

Flanigan, “Detection of Organic Vapors with Active and Passive
Sensors: A Comparison.” Applied Optics, 1986, vol. 25, No. 23, pp.
4253-4260.

8, 2008 Mir et al.
9, 2008 Benson et al.

2008/025 1724 A1* 10/2008 Baliga ....................... GO1J 3.02
250/338.5
2010/0211333 A1
2011/0261321 A1
2012fO273680 A1
2013, O181836 A1
2013,0321806 A1
2013,034.1509 A1
2015,0136981 A1
2015,0136982 A1
2015,0316473 A1
2016/009.7713 A1
2016,0349228 A1
2017, 0026588 A1

8, 2010 Pruet et al.
10/2011 Ramella-Roman et al.

11/2012 Furry
7, 2013
12/2013
12/2013
5, 2015
5, 2015
11/2015
4/2016
12/2016
1/2017

Cardoso et al.
Kester et al.
Nelson et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.

FOREIGN PATENT DOCUMENTS
CA
EP
EP
EP
EP
GB
WO
WO
WO
WO
WO
WO
WO
WO

Caulton et al., “Toward a Better Understanding and Quantificationof Methane Emissions from Shale Gas Development', PNAS, Apr.
29, 2014, vol. 111, No. 17, pp. 7.
Chen et al., “Quantitative Sectioning and Noise Analysis for Structured Illumination Microscopy: Erratum'. Optics Express, Oct. 19.
2015, vol. 23, No. 21, pp. 27633-27634.
Chidley et al., “Flow-Induced Birefringence: The Hidden PSF
Killer in High Performance Injection-Molded Plastic Optics'.
Endoscopic Microscopy, Proceedings of SPIE vol. 6082, 2006, pp.

2870 419
O 837 600
2871 4.52
2.942 615
2.955 496
1014769
WO 2004/097389
WO 2007/0O8826
WO 2008/109183
WO 2010/053979
WO 2012/078417
WO 2012/082366
WO 2013, 173541
WO 2016, 196224

5/2015
4f19985/2015
11/2015
12/2015
12/1965
11 2004
1, 20079, 2008
5, 2010
6, 2012
6, 2012
11 2013
12/2016

OTHER PUBLICATIONSAllen et al., “Measurements of Methane Emissions at Natural Gas
Production Sites in the United States', PNAS, Oct. 29, 2013, vol.
110, No. 44, pp. 7.

Alvarez et al., “Greater Focus Needed on Methane Leakage fromNatural Gas Infrastructure', PNAS, Apr. 24, 2012, vol. 109, No. 17.
pp. 12.
Bedard et al., “Image Mapping Spectrometry: Calibration andCharacterization”. Optical Engineering, Nov. 2012, vol. 51, No. 11,
pp. 111711-1-111711-13.
Ben-David et al., “Probability Theory for 3-Layer Remote Sensing
Radiative Transfer Model: Univariate Case,” Optics Express, Apr.
2012, vol. 20, No. 9, pp. 10004-10033.
Ben-David et al., “Probability Theory for 3-Layer Remote Sensing
Radiative Transfer Model: Errata,” Optics Express, May 20, 2013,vol. 21, No. 10, pp. 11852.
Brady et al., “Multiscale Lens Design”. Optics Express, Jun. 22.
2009, vol. 17, No. 13, pp. 10659-10674.

Gallagher et al., “Error Analysis for Estimation of Trace Vapor
Concentration Pathlength in Stack Plumes”. Applied Spectroscopy,
2003, vol. 57, No. 6, pp. 614-621.
Gallagher et al., “Estimation of Trace Vapor Concentration
Pathlength in Plumes for Remote Sensing Applications from
Hyperspectral Images'. Analytica Chimica Acta, 2003, vol. 490, pp.
139-152.

Gao et al., “Compact Image Slicing Spectrometer (ISS) for
Hyperspectral Fluorescence Microscopy”. Optics Express, Jul. 20.
2009, vol. 17, No. 15, pp. 12293-12308.
Gao et al., "Depth-Resolved Image Mapping Spectrometer (IMS)with Structured Illumination”. Optics Express, Aug. 29, 2011, vol.
19, No. 18, pp. 17439-17452.
Gao et al., “Optical Design of a Snapshot High-Sampling Image
Mapping Spectrometer (IMS) for Hyperspectral Microscopy”.
Three-Dimensional and Multidimensional Microscopy:Image
Acquisition and Processing XVII, Proceedings of SPIE vol. 7570,
2010, pp. 1-7.
Gao et al., “Quantitative Comparison Between Full-Spectrum andFilter-Based Imaging in Hyperspectral Fluorescence Microscopy'.
Journal of Microscopy, 2012, vol. 246, No. 2, pp. 113-123.
Gao et al., “Snapshot Image-Mapping Spectrometer for
Hyperspectral Fluorescence Microscopy”. Optics and Photonics
News, Nov. 2010, vol. 21, No. 12, p. 50.
Gao et al., “Snapshot Image Mapping Spectrometer (IMS) withHigh Sampling Density for Hyperspectral Microscopy”. Optics
Express, Jul. 5, 2010, vol. 18, No. 4, pp. 14330-14344.
Gerhart et al., “Detection and Tracking of Gas Plumes in LWIR
Hyperspectral Video Sequence Data.” Algorithms and Technologies
for Multispectral, Hyperspectral, and Ultraspectral Imagery XIX.
2013, SPIE Proceedings vol. 8743, pp. 1-14.
Gittins, Christopher M., “Detection and Characterization of Chemi
cal Vapor Fugitive Emissions by Nonlinear Optimal Estimation:
Theory and Simulation'. Applied Optics, Aug. 10, 2009, vol. 48,
No. 23, pp. 4545-4561.
Goldberg et al., “Dual Band MWIR/LWIR Focal Plane Array Test
Results.” Army Research Lab, Adelphi, MD, Aug. 1999, pp. 18.
Golowich et al., “Performance Limits of LWIR Gaseous Plume

Quantification'. Algorithms and Technologies for Multispectral,Hyperspectral, and Ultraspectral Imagery XVII, 2011, Proceedingsof SPIE vol. 8048, pp. 1-12.

Griffin et al., “The Herschel—SPIRE 1-15 Instrument and its

In-Flight Performance.” Astronomy and Astrophysics, Jul. 1, 2010,
vol. 518, pp. 7.
Gross et al., “Remote Identification and Quantification of Industrial
Smokestack Effluents via Imaging Fourier-Transform Spectros
copy”. Environmental Science & Technology, 2010, vol. 44, No. 24.
pp. 9390-9397.
Gupta et al., “Miniature Snapshot Multispectral Imager,” OpticalEngineering, 2011, vol. 50, pp. 033203-1-033203-9.

US 9,625,318 B2
Page 3(56)

References Cited
OTHER PUBLICATIONS

Hagen et al., “Analysis of Computed Tomographic Imaging Spec
trometers. I. Spatial and Spectral Resolution'. Applied Optics, Oct.
1, 2008, vol. 47, No. 28, pp. F85-F95.
Hagen et al., “Coded Aperture DUV Spectrometer for Standoff
Raman Spectoscopy'. Next-Generation Spectroscopic Technolo
gies II, Proceedings of SPIE vol. 7319, 2009, pp. 1-10.
Hagen et al., “Compound Prism Design Principles, I'. Applied
Optics, Sep. 1, 2011, vol. 50, No. 25, pp. 4998-5011.
Hagen et al., “Compound Prism Design Principles, II: Triplet andJanssen Prisms”. Applied Optics, Sep. 1, 2011, vol. 50, No. 25, pp.
5O12-5022.

Hagen et al., “Compound Prism Design Principles, III: Linear-in
Wavenumber and Optical Coherence Tomography Prisms”. Applied
Optics, Sep. 1, 2011, vol. 50, No. 25, pp. 5023-5030.
Hagen et al., “Fourier Methods of Improving Reconstruction Speed
for CTIS Imaging Spectrometers'. Imagin Spectrometry XII, Pro
ceedings of SPIE vol. 6661, 2007, pp. 11.
Hagen et al., “Foveated Endoscopic Lens”, Journal of Biomedical
Optics, Feb. 2012, vol. 17, No. 2, pp. 021104-1-021104-6.
Hagen et al., “Gaussian Profile Estimation in One Dimension'.
Applied Optics, Aug. 1, 2007, vol. 46, No. 22, pp. 5374-5383.
Hagen et al., “Gaussian Profile Estimation in Two Dimension'.
Applied Optics, Dec. 20, 2008, vol. 47, No. 36, pp. 6842-6851.
Hagen et al., “Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy”. Optics Express, Jan. 2, 2012,vol. 20, No. 1, pp. 403-413.
Hagen et al., “Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy: Errata'. Optics Express, Feb.
27, 2012, vol. 20, No. 5, pp. 5343.
Hagen et al., “Real-Time Quantitative Hydrocarbon Gas Imaging
with the Gas Cloud Imager (GCI)”. Proceedings of SPIE, vol. 8358,
Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE) Sensing XIII, May 1, 2012, pp. 7.
Hagen et al., “Review of Snapshot Spectral Imaging Technologies'.
Optical Engineering, Sep. 2013, vol. 52, No. 9, pp. 090901-1090901-23.

Hagen et al., “Snapshot Advantage: A Review of the Light Collec
tion Improvement for Parallel High-Dimensional Measurement
Systems.” Optical Engineering, Jun. 13, 2012, vol. 51, No. 11, p.
111702-1-111702-7.

Hagen et al., "Snapshot Mueller Matrix Spectropolarimeter' Optics
Letters, Aug. 1, 2007, vol. 32, No. 15, pp. 2100-2102.
Hagen et al., “Spectrally-Resolved Imaging of Dynamic TurbidMedia”. Multimodal Biomedical Imaging VI, Proceedings of SPIE
vol. 7892, 2011, pp. 1-7.
Hagen et al., “Video-Rate Spectral Imaging of Gas Leaks in the
Longwave Infrared,” Chemical, Biological, Radiological, Nuclear,and Explosives (CBRNE) Sensing XIV. May 29, 2013, SPIE
Proceedings vol. 8710, pp. 7.
Harley et al., “Remote Quantification of Smokestack Effluent Mass
Flow Rates Using Imaging Fourier Transform Spectrometry.”
Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE) Sensing XII, Apr. 25-29, 2011, SPIE Proceedings vol.
8018, pp. 1-13.
Hayden et al., “Determination of Trace-Gas Amounts in Plumes by
the Use of Orthogonal Digital Filtering of Thermal-Emission Spec
tra'. Applied Optics, Jun. 1, 1996, vol. 35. No. 16, pp. 2802-2809.
Hirsch et al., “Detection of Gaseous Plumes in IR Hyperspectral
Images Using Hierarchical Clustering”. Applied Optics, Sep. 1,
2007, vol. 46, No. 25, pp. 6368-6374.
Johnston et al., “A Real-Time FPGA Implementation of a BarrelDistortion Correction Aglorithm”. Projects, 2003, vol. 10, pp.
91-96.

Karion et al., “Methane Emissions Estimate from Airborne Mea
surements Over a Western United States Natural Gas Field', Geophysical Research Letters, 2013, vol. 40, pp. 4393-4397.
Keshava et al., “A Survey of Spectral Unmixing Algorithms'.
Lincoln Laboratory Journal, 2003, vol. 14, No. 1, pp. 55-78.

Kester et al., “A Real-Time Gas Cloud Imaging Camera for Fugitive
Emission Detection and Monitoring. Imaging and Applied Optics
Technical Digest, 2012, pp. 3.
Kester et al., “Development of Image Mappers for Hyperspectral
Biomedical Imaging Applications'. Applied Optics, Apr. 1, 2010,
vol. 49, No. 10, pp. 1886-1899.
Kester et al., “High Numerical Aperture Microendoscope Objective
for a Fiber Confocal Reflectance Microscope”. Optics Express, Mar.
5, 2007, vol. 15. No. 5, pp. 2409-2420.
Kester et al., “Low Cost, High Performance, Self-Aligning Minia
ture Optical Systems”. Applied Optics, Jun. 20, 2009, vol. 48, No.
18, pp. 3375-3384.
Kester et al., “Real-Time Snapshot Hyperspectral Imaging Endo
scope'. Journal of Biomedical Optics, May 2011, vol. 16, No. 5, pp.
O56005-1-05.6005-12.

Kudenov et al., “Fourier Transform Channeled Spectropolarimetry
in the MWIR”. Optics Express, Oct. 1, 2007, vol. 15, No. 20, pp.
12792-12805.

Kudenov et al., “Snapshot Imaging Mueller Matrix Polarimeter
Using Polarization Gratings'. Optics Letters, Apr. 15, 2012, vol. 37.
No. 8, pp. 1367-1369.
Landau et al., “Design and Evaluation of an Ultra-Slim Objective
for in-vivo Deep Optical Biopsy”. Optics Express, Mar. 1, 2010,
vol. 18, No. 5, pp. 4758-4775.
Levi, Michael A., “Comment on Hydrocarbon Emissions Characterization in the Colorado Front Range: A Pilot Study by Gabrielle
Pétron et al.”. Journal of Geophysical Research, 2012, vol. 117. No.
D21203, pp. 1-5.
Levi, Michael A., “Reply to “Reply to Comment on “Hydrocarbon
Emissions Characterization in the Colorado Front Range—A Pilot
Study’ by Michael A. Levi' by Gabrielle Pétron et al.'”, Journal ofGeophysical Research: Atmospheres, 2013, vol. 118, pp. 3044
3046.

Low et al., “Remote Sensing and Characterization of Stack Gases byInfrared Spectroscopy. An Approach by Using Multiple-Scan Inter
ferometry”. Environmental Science & Technology, Jan. 1967, vol.
1. No. 1, pp. 73-74.
Luo et al., “Fast Processing of Imaging Spectrometer Data Cube
Based on FPGA Design”, MIPPR 2007: Multispectral Image Pro
cessing, Proceedings of SPIE vol. 6787, pp. 7.
Manolakis et al., “Long-Wave Infrared Hyperspectral Remote Sens
ing of Chemical Clouds'. IEEE Signal Processing Magazine, Jul.
2014, vol. 31, No. 4, pp. 120-141.
Mathews, “Design and Fabrication of a Low-Cost, Multispectral
Imaging System.” Applied Optics, 2008, pp. F71-F76, vol. 47.
Naranjo et al., “IR Gas Imaging in an Industrial Setting.”
Thermosense XXXII, Published in SPIE Proceedings vol. 7661,
May 4, 2010, pp. 1-8.
Nguyen et al., “Snapshot 3D Optical Coherence Tomography Sys
tem using Image Mapping Spectrometer. Biomedical Optics and
3D Imaging OSA, 2012, pp. 3.
Niu et al., “New Approach to Remote Gas-Phase Chemical Quan
tification: Selected-Band Algorithm”. Optical Engineering, Feb.
2014, vol. 53, No. 2, pp. 021111-1-021111-10.
"Oil and Natural Gas Sector Leaks”. U.S. EPA Office of Air Quality
Planning and Standards (OAQPS), Review Panel, Apr. 2014, pp. 63.
Pétron et al., “Hydrocarbon Emissions Characterization in the
Colorado Front Range: A Pilot Study”. Journal of GeophysicalResearch, 2012, vol. 117, No. D04304, pp. 1.-19.
Pétron et al., “Reply to Comment on Hydrocarbon Emissions
Characterization in the Colorado Front Range—A Pilot Study by
Michael A. Levi'. Journal of Geophysical Research: Atmospheres,
2013, vol. 118, pp. 236-242.
Pisano et al., “Thermal Illuminators for Far-Infrared and Submil

limeter Astronomical Instruments.” Applied Optics, Jun. 1, 2005,
vol. 44, No. 16, pp. 3208-3217.
Polak et al., “Passive Fourier-Transform Infrared Spectroscopy ofChemical Plumes: An Algorithm for Quantitiative Interpretation
and Real-Time Background Removal”. Applied Optics, Aug. 20.
1995, vol. 34, No. 24, pp. 5406-5412.
Sandsten et al., “Volume Flow Calculations on Gas Leaks Imaged
with Infrared Gas-Correlation.” Optics Express, 2012, vol. 20, No.
18, pp. 20318-20329.

US 9,625,318 B2
Page 4(56)

References Cited
OTHER PUBLICATIONS

Shogenji et al., “Multispectral Imaging Using Compact Compound
Optics,” Optics Express, Apr. 19, 2004, vol. 12, No. 8, pp. 1643
1655.

Walter Jr., et al., “Detection of Atmospheric Pollutants: a Correlation Technique'. Applied Optics, Jun. 1975, vol. 14, No. 6, pp.
1423-1428.

Young et al., “An In-Scene Method for Atmospheric Compensationof Thermal Hyperspectral Data”, Journal of Geophysical Research,
2002, vol. 107, No. D24, pp. 14-1-14-20.
Zheng et al., “A Static Multiplex Fabry-Perot Spectrometer'. Sen
Sors, Cameras, and Systems for Industrial Scientific Applications X.
Proceedings of SPIE-IS&T Electronic Imaging, SPIE vol. 7249,
2009, pp. 8.
Zheng et al., “Analytic-Domain Lens Design with Proximate Ray
Tracing”, Journal of the Optical Society of America A, Aug. 2010,
vol. 27, No. 8, pp. 1791-1802.
Preliminary Amendment as filed in U.S. Appl. No. 14/538,827 dated
Jan. 28, 2015 in 6 pages.
Office Action received in U.S. Appl. No. 14/538,827 dated Jun. 30.
2015 in 8 pages.
Non-Final Office Action Response as filed in U.S. Appl. No.
14/538,827 dated Dec. 28, 2015 in 11 pages.
Notice of Allowance received in U.S. Appl. No. 14/538,827 dated
Feb. 1, 2016 in 18 pages.
Corrected Notice of Allowance received in U.S. Appl. No.
14/538,827 dated Feb. 10, 2016 in 4 pages.
Corrected Notice of Allowance received in U.S. Appl. No.
14/538,827 dated Feb. 22, 2016 in 4 pages.
Office Action received in U.S. Appl. No. 14/539,899 dated Mar. 26,
2015 in 6 pages.
Non-Final Office Action Response as filed in U.S. Appl. No.
14/539,899 dated Aug. 26, 2015 in 8 pages.
Final Office Action received in U.S. Appl. No. 14/539,899 dated
Dec. 11, 2015 in 9 pages.
Extended European Search Report received in European Application No. 141928.62.2 dated Mar. 30, 2015 in 10 pages.
International Search Report in PCT Application No. PCT/US2013/041278 dated Aug. 27, 2013 in 4 pages.
International Preliminary Report on Patentability in PCT Application No. PCT/US2013/041278 dated Nov. 27, 2014 in 10 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,700,791 dated
Jul. 13, 2015 in 8 pages.
Extended European Search Report received in European Application No. 15165877.0 dated Oct. 8, 2015 in 12 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,700,567 dated
Jul. 10, 2015 in 6 pages.
Extended European Search Report received in European Application No. EP 15165880.4 dated Nov. 24, 2015 in 8 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,792.477 dated
Dec. 21, 2015 in 7 pages.
Adams, et al., “Advances in Detectors: Hot IR sensors improve IR
camera size, weight, and power'. Laser Focus World, vol. 50, Issue
01, Jan. 17, 2014, 6 pages. Also available at http://www.ircameras.
com/articles/advances-detectors-hot-ir-sensors-improve-ir-camera
size-weight-power?.
Brochure provided by Lofty Designs to Rebellion Photonics on Oct.
31, 2012 as noted from the email. Subsequent to that date brochure
was used in connection with potential customers.

“Directed Inspection and Maintenance at Gas Processing Plants andBooster Stations.” United States Environmental Protection Agency
Air and Radiation (6202J), EPA430-B-03-018, Oct. 2003 available
at https://www3.epa.gov/gasstart documents/II dimgasproc.pdf.
Gálfalk et al., “Making Methane Visable', Nature Climate Change,
Apr. 2016, vol. 6, pp. 426-430.
Gálfalk et al., “Making Methane Visable'. Supplementary Informa
tion, Nature Climate Change, 2015, pp. 1-14.
Hadlington, Simon, “New Camera Makes Methane Visible'. Chem
istry World, http://web.archive.org/web/20160305234907/http://
www.rsc.org/chemistryworld/2015/12/methane-camera-infared
greenhouse-gas, Dec. 14, 2015, pp. 2.
Rebellion Photonics, “Gas Cloud Imaging Camera: A Breakthroughin Leak Monitoring for the Rig & Refinery Safety Market'. Pre
sentation at SPIE Defense Security and Sensing, 28 pages, Apr.
29-May 3, 2013.
TELOPS,“Hyper-Cam',
http://web.archive.org/web/20160608180941/http://www.telops.com/en/hyperspectral-cam
eras/hyper-cam as archived Jun. 8, 2016 in 2 pages.
TELOPS, “Innovative Infrared Imaging'. http://web.archive.org/
web/201606032 12729/http://www.telops.com/en/ as archived Jun.
3, 2016 in 2 pages.
Request for Continued Examination and Response to Correct Application Papers as filed in U.S. Appl. No. 14/538,827 dated Apr. 29.
2016 in 14 pages.
Notice of Allowance received in U.S. Appl. No. 14/538,827 dated
May 26, 2016 in 9 pages.
Notice of Allowance received in U.S. Appl. No. 14/538,827 dated
Sep. 19, 2016 in 9 pages.
Amendment as filed in U.S. Appl. No. 14/539,899 dated Jun. 9.
2016 in 6 pages.
Notice of Allowance received in U.S. Appl. No. 14/539,899 dated
Jun. 21, 2016 in 17 pages.
Notice of Allowance received in U.S. Appl. No. 14/539,899 dated
Oct. 31, 2016 in 10 pages.
Official Communication received in European Application No.
141928.62.2 dated Apr. 19, 2016 in 6 pages.
Notice of Allowance received in U.S. Appl. No. 14,700,791 dated
Jun. 9, 2016 in 11 pages.
Notice of Allowance received in U.S. Appl. No. 14,700,791 dated
Sep. 30, 2016 in 19 pages.
Publication Request as filed in U.S. Appl. No. 14,700,567 dated
Aug. 24, 2016 in 237 pages.
Preliminary Amendment as filed in U.S. Appl. No. 15/166,092 dated
Aug. 15, 2016 in 7 pages.
International Search Report in PCT Application No. PCT/US2016/
034455 dated Oct. 24, 2016 in 12 pages.
Wikipedia entry https://en.wikipedia.org/wiki/Mobile computing
in 6 pages, last modified Dec. 30, 2016; retrieved from the internet
on Feb. 2, 2017.

Amendment as filed in U.S. Appl. No. 14/538,827 dated Dec. 16.
2016 in 9 pages.
Amendment as filed in U.S. Appl. No. 14/539,899 dated Jan. 27.
2017 in 5 pages.
Official Communication received in European Application No.
15165877.0 dated Jan. 3, 2017 in 9 pages.
Issue Notification received in U.S. Appl. No. 14/539,899 dated Mar.
1, 2017 in 1 page.
Notice of Allowance received in U.S. Appl. No. 14,700,791 dated
Feb. 21, 2017 in 8 pages.

* cited by examiner

U.S. Patent

Apr. 18, 2017

Sheet 1 of 11()

US 9,625,318 B2

U.S. Patent

Apr. 18, 2017

Sheet 3 of 11

is 8:8 is a set is

y

m.

yxxxx

sssssssax&^

yawswasssssssssssssscwsy&SSSSSXXXXXXSWWWWWWXXXXXXXXXX-SWWWWWXxxx xxxxxx

ass
3 & 3 - 3 - 33-S&

ww.swsw8ww.

US 9,625,318 B2

U.S. Patent

Apr. 18, 2017

Sheet S of 11

US 9,625,318 B2

U.S. Patent

Apr. 18, 2017

asR

Sheet 6 of 11a. . . . . .

if a. s. as a se -sa, as a sa, a

a

5

'''

US 9,625,318 B2ra

as at atext

if e iisa gas a ge...,,,,,,, Tr. T, TT, TT, Trier arror

is a as a s

t

as

ex

&

a

is esses

s

l

ki
&
(SSESS.E(Ns

U.S. Patent

Apr. 18, 2017

. . . .

is. . . .

is a

Sheet 7 of 11

US 9,625,318 B2.

lack to ei

is story arr's".

...,,,,,,,,,,, T.T.T.T.T.T. r. be resert
s

x

s

s

x
A
A.

E.
X

a.

s
k

k

&

s
k

is

. ss.

se

seases isrearU.S. Patent

Apr. 18, 2017

Sheet 8 of 11

US 9,625,318 B2s" ' " " " . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ex.
"" is a set

an in a

a

.

. ..

. . .

.

a

&

cy S.
s

is is is a siggs an

is is

sax

SSSS &

is is assassiU.S. Patent&sa

Apr. 18, 2017

-

0
&

l

&s

Kit
&

(SSS$8.

Sheet 9 of 11

(,
E.

l
3.

US 9,625,318 B2

U.S. Patent

Apr. 18, 2017

Sheet 10 of 11

US 9,625,318 B2

US 9,625,318 B2
1.
DIVIDED-APERTURE INFRA-RED
SPECTRAL MAGING SYSTEM FOR
CHEMICAL DETECTION

2
sponding optical channel from the at least two spatially andspectrally different optical channels and includes at least oneof a longpass optical filter and a shortpass optical filter (withor without a combination with another filter such as a notch

filter, for example). The optical system may further include
at least two reimaging lenses, each reimaging lens disposedto transmit IR radiation (in one embodiment between
The present application claims benefit of and priority about 1 micron and about 20 microns), that has beenfrom the U.S. Provisional Applications No. 61/688,630 filed 10 transmitted through a corresponding optical filter towards
on May 18, 2012 and titled “Divided Aperture Infrared the optical FPA unit. In one embodiment, the optical FPA
Spectral Imager (DAISI) for Chemical Detection; and unit is positioned to receive IR radiation from the object
61/764,776 filed on Feb. 14, 2013 and titled “Divided
through the at least two reimaging lenses to form respec
Aperture Infrared Spectral Imager for Chemical Detection'. tively-corresponding two or more sets of imaging data
The disclosure of each of the above-mentioned provisional 15 representing the object and the processor is configured topatent applications is incorporated by reference herein in its acquire said optical data from the two or more sets ofentirety. The application also claims priority to International imaging data.
Application No. PCT/US2013/041278, filed May 16, 2013,
Embodiments of the present invention additionally pro
which is also incorporated by reference herein in its entirety. videa method for operating an infrared (IR) imaging system.
The method includes receiving IR radiation from an object
TECHNICAL FIELD
along at least two optical channels defined by components of
The present invention generally relates to a system and an optical system of the IR imaging system, which at least
method for gas cloud detection and, in particular, to a system two optical channels are spatially and spectrally different
and method of detection of spectral signatures of chemical from one another. The method further includes transmitting
compositions in a mid- and long-wave infrared spectral 25 the received IR radiation towards an optical focal plane
region with a use of systemic compensation for parallax array (FPA) unit that is not being cooled in the course of
normal operation; and removably positioning at least one ofinduced and motion-induced imaging artifacts.
at least two temperature-controlled shutters in front of the
BACKGROUND
optical system to block IR radiation incident onto the optical30
system from the object.
Most of the existing IR spectral imaging systems require
focal plane detector arrays (FPAs) that have to be highly
BRIEF DESCRIPTION OF THE DRAWINGS
sensitive and cooled in order to compensate, during theoptical detection, for the reduction of the photon flux caused
invention will be more fully understood by referringby spectrum-scanning operation. There remains a need, 35 to The
thefollowing Detailed Description in conjunction withtherefore, in a system enabling an optical data acquisition the Drawings,
of which:mode that does not require the cooling of the used
FIG.
1
shows
embodiment of the system of thedetector(s), which detectors can be less sensitive to photons invention utilizing anacommonfront objective lens that has ain the IR but yet well fit for continuous monitoring appli pupil divided spectrally and re-imaged
onto an infrared FPA.
cations. There also remains a need in an IR imaging system 40
FIG. 2 shows an embodiment with a divided front objec
the operation of which is substantially not susceptible tomotion artifacts (which is a common problem with spec tive lens and an array of infrared sensing FPAs.
FIG. 3A represents an embodiment employing an array of
trally-scanning systems causing errors in either the spectral
front objective lenses operably matched with the re-imaging
data, spatial data, or both).
45 lens array.
SUMMARY
FIG. 3B illustrates a two-dimensional array of opticalcomponents corresponding to the embodiment of FIG. 3A.
FIG. 4 is a diagram of the embodiment employing an
Embodiments of the present invention provide an infrared(IR) imaging system for determining a concentration of a array of field references and an array of respectively corre
target species in an object. The imaging system includes (i) 50 sponding relay lenses.
an optical system, having an optical focal plane array (FPA)
FIG. 5A is a diagram of a 4-by-3 pupil array of circular
unit that is devoid of a cooling means, which optical system optical filters (and IR blocking material among them) used
is configured to receive IR radiation from the object along at to spectrally divide an optical wavefront imaged with anleast two optical channels defined by components of the embodiment of the invention.
optical system, said at least two optical channels being 55 FIG. 5B is a diagram of a 4-by-3 pupil array of rectangularspatially and spectrally different from one another; (ii) first optical filters (and IR blocking material among them) usedand second temperature-controlled shutters removably posi to spectrally divide an optical wavefront imaged with antioned to block IR radiation incident onto the optical system embodiment of the invention.
from the object; and (iii) a processor configured to acquire
FIG. 6A depicts theoretical plots of transmission charac
multispectral optical data representing said target species 60 teristics of a combination of band-pass filters used with anfrom the received IR radiation in a single occurrence of data embodiment of the invention.
acquisition. The optical system may include an optical
FIG. 6B depicts theoretical plots of transmission charac
aperture (a boundary of which is defined to circumscribe, teristics of spectrally multiplexed notch-pass filter combi
encompass said at least two spatially distinct optical chan nation used in an embodiment of the invention.
nels) and at least two spectrally-multiplexed optical filters. 65 FIG. 6C shows theoretical plots of transmission charac
Each of these optical filters is positioned to transmit a teristics of spectrally multiplexed long-pass filter combina
portion of the IR radiation received in a respectively corre tion used in an embodiment of the invention.
CROSS-REFERENCE TO RELATEDAPPLICATIONSUS 9,625,318 B2
4
In the drawings like numbers are used to represent thesame or similar elements wherever possible. The depictedstructural elements are generally not to Scale, and certain
tion used in an embodiment of the invention.
are enlarged relative to the other components
FIG. 7 is a set of video-frames illustrating operability of components
5 for purposes of emphasis and understanding. It is to bean embodiment of the invention used for gas detection.
that no single drawing is intended to support a
FIGS. 8A, 8B are plots illustrating results of dynamic understoodcomplete
descriptionof all features of the invention. In othercalibration of an embodiment of the invention.
words, a given drawing is generally descriptive of only
Some, and generally not all, features of the invention. A
DETAILED DESCRIPTION
10 given drawing and an associated portion of the disclosure
containing a description referencing Such drawing do not,Embodiments of the present invention illustrate a divided generally, contain all elements of a particular view or all
aperture infrared spectral imaging (DAISI) system that is features that can be presented is this view, for purposes ofstructured and adapted to provide identification of target simplifying the given drawing and discussion, and to direct
chemical contents of the imaged scene based on spectrally 15 the discussion to particular elements that are featured in thismultiplexed operation and single-shot (also referred to as drawing. A skilled artisan will recognize that the invention
possibly be practiced without one or more of theSnapshot), that is devoid of spectral and spatial scanning may
specificfeatures, elements, components, structures, details,
acquisition of infrared (IR) spectral signatures of the target or characteristics,
with the use of other methods, com
chemical contents with an IR detector (Such as, for example, ponents, materials,orandso forth. Therefore, although ainfrared focal plane array or FPA) to form a spectral cube of particular detail of an embodiment of the invention may not
imaging data. In contradistinction to commonly used IR be necessarily shown in each and every drawing describing
imaging systems, the DAISI system does not require cool Such embodiment, the presence of this detail in the drawing
ing.
may be implied unless the context of the description requires
Implementations of the present invention provide several otherwise. In other instances, well known structures, details,
operational advantages over existing IR spectral imaging 25 materials, or operations may be not shown in a given
systems, most if not all of which require FPAs that have to drawing or described in detail to avoid obscuring aspects ofbe highly sensitive and cooled in order to compensate, an embodiment of the invention that are being discussed.
during the optical detection, for the reduction of the photon Furthermore, the described single features, structures, orflux caused by spectrum-scanning operation. The highly characteristics of the invention may be combined in any
sensitive and cooled FPA systems are expensive and require 30 suitable manner in one or more further embodiments.
Moreover, if the schematic flow chart diagram is included,
a great deal of maintenance. As an embodiment of theinvention is configured to operate in single-shot acquisition it is generally set forth as a logical flow-chart diagram. As
mode, the instrument receives photons from every point of such, the depicted order and labeled steps of the logical flow
the object Substantially simultaneously, during the single are indicative of one embodiment of the presented method.
reading. In comparison with a system of related art, this 35 Other steps and methods may be conceived that are equiva
feature enables an embodiment to collect a substantially lent in function, logic, or effect to one or more steps, orgreater amount of optical power from the imaged scene (for portions thereof, of the illustrated method. Additionally, theexample, an order of magnitude more photons) at any given format and symbols employed are provided to explain the
moment in time. Consequently, an embodiment is enabled to logical steps of the method and are understood not to limit
operate using uncooled detector(s) (for example, FPA Such 40 the scope of the method. Although various arrow types andas an array of microbolometers) that are less sensitive to line types may be employed in the flow-chart diagrams, they
photons in the IR but are well fit for continuous monitoring are understood not to limit the scope of the corresponding
applications since they are capable of operating in extreme method. Indeed, some arrows or other connectors may beweather conditions, require less power, can operate both day used to indicate only the logical flow of the method. For
and night, and are less expensive. On the other hand, 45 instance, an arrow may indicate a waiting or monitoring
embodiments of the invention are advantageous in that their period of unspecified duration between enumerated steps ofoperation is Substantially immune to motion artifacts (which the depicted method. Without loss of generality, the order in
is a common problem with spectrally-scanning systems which processing steps or particular methods occur may or
causing errors in either the spectral data, spatial data, or may not strictly adhere to the order of the correspondingboth). Moreover, present embodiments are structured to 50 steps shown.
acquire spectrally-multiplexed datacubes during a single
The invention as recited in claims appended to thisshot acquisition which, when combined with the detector disclosure is intended to be assessed in light of the disclosure
noise limited performance of the FPA's, result in increase of as a whole, including features disclosed in prior art to whichlevel of the detected signal by a factor of 2 to 10 times, as reference is made.
55
FIG. 1 provides a diagram schematically illustrating acompared with the systems of related art.
References throughout this specification to “one embodi spatial and spectral division of incoming light by an embodi
ment,” “an embodiment,” “a related embodiment,” or simi
ment 100 of the system of the invention (also referred to as
lar language mean that a particular feature, structure, or DAISI system) that is enabled to image an object 110
characteristic described in connection with the referred topossessing IR spectral signature(s). An aperture of the"embodiment' is included in at least one embodiment of the 60 system (associated with a front objective lens system 124) ispresent invention. Thus, appearances of the phrases "in one spatially and spectrally divided. The spatial and spectral
embodiment,” “in an embodiment,” and similar language division of the aperture into distinct aperture portions cor
throughout this specification may, but do not necessarily, all responding to separate channels 120 (in object space and/orrefer to the same embodiment. It is to be understood that noimage space) along which light propagates through theportion of disclosure, taken on its own and in possible 65 aperture is enabled with the use of an array 128 of reconnection with a figure, is intended to provide a complete imaging lenses 128a and an array of spectral filters 130,
which respectively correspond to the distinct channels 120.
description of all features of the invention.

3
FIG. 6D shows theoretical plots of transmission characteristics of spectrally multiplexed short-pass filter combina

US 9,625,318 B2
5
6
In one implementation, the distinct channels 120 may
The front objective lens element of the system can gen
include optical channels that are separated in angular space. erally include an array of front objective lenses configured
The array of spectral filters 130 may additionally include a across the IR wavefront emitted by the object being imaged
filter-holding aperture mask (containing, for example, IR with the DAISI system such as to divide such wavefront
spatially in a non-overlapping fashion. To this end, FIG. 2
light-blocking materials such as ceramic, metal, or plastic).
Light from the object 110 (such as a cloud of gas, for illustrates a related embodiment 200, in which a front opticalexample), the optical properties of which in the IR are portion contributing to the spatial division of the aperture ofdescribed by a unique absorption, reflection and/or emission the system is defined by a multiplicity of objective lenses
spectrum, is received by the aperture of the system through 10 224 configured as a two-dimensional (2D) array of lenses.
each of the channels 120 and is further imaged onto an FIG. 2 presents a general view of the system 200 and, in
optical detector component 136 (which may include at least figure insert, a portion 202 of it in greater detail, including
one FPA). Each of the re-imaging lenses 128a is spatially a field reference (aperture stop) 204. The configuration 200aligned with a respectively-corresponding region of the has an operational advantage over embodiment 100 of FIG.
divided aperture and, therefore, with respectively-corre 15 1 in that the overall size and/or weight and/or cost of
sponding spatial channel 120 to form, on the FPA compo manufacture of the embodiment 200 is critically reduced
nent 136 a single sub-image of the object 110. Generally, while the associated parallax (the change in the FOVs of
two or more Sub-images of the object can be characterized individual lenses 224 of the lens-array disposed across andby close or substantially equal spectral signatures. The FPA Substantially perpendicularly to a general optical axis 226 ofcomponent 136 is further operably connected with a pro the embodiment 200; marked as 228) is substantially small.
cessor 150 (not shown) specifically programmed to aggre As the distance between the portion 202 and the object 110
gate the data acquired with the system 100 into a spectral increases, the overlapping region 230 between the FOVs of
datacube representing, in spatial (x, y) and spectral (W) the individual lenses 224 increases while the amount of
coordinates an overall spectral image of the object 110 parallax 228 remains approximately the same, thereby
within the spectral region defined by the combination of the 25 reducing its effect on the system 200. When the ratio of thefilters 130. Additionally, the processor 150 may be option parallax-to-object-distance is Substantially equal to theally and specifically programmed to determine the unique pixel-size-to-system-focal-length ratio then the parallax
absorption characteristic of the object 110 and, alternatively effect may be considered to be negligible and, for practicalor in addition, map the overall image datacube into a cube purposes, no longer distinguishable. While the lenses 224
of data representing spatial distribution of concentrations c 30 are shown to be disposed Substantially in the same plane,of targeted chemical components within the field of view optionally the array of front objective lenses such as lenses
224 can be defined in more than one plane. For example,
associated with the object 110.
In order to facilitate the operational performance of the some of the individual lenses 224 can be displaced withembodiment 100, an optional moveable temperature-con respect to some other individual lenses 224 along the axis
trolled reference target 160 (including, for example, a shut 35 226 (not shown). It is noted that when multiple detectors 236
ter system containing two reference shutters maintained at are employed with the embodiment 200, the embodiment isdifferent temperatures) is removably and, in one implemen preferably complemented with field reference 204 to operatetation, periodically inserted into an optical path of light properly, as discussed below.
In one implementation, the front objective lens system
traversing the system 100 from the object 110 to the FPA
component 130 along at least one of the channels 120 to 40 Such as the array of lenses 224 is configured as an array ofblock such optical path and to provide a reference IR lenses integrated or molded in association with a monolithic
spectrum required to recalibrate the operation of the system Substrate, thereby reducing the costs and complexity other100 in real time. The configuration of the moveable refer wise accompanying the optical adjustment of individual
lenses within the system. An individual lens 224 can optionence(s) 160 is further discussed below.
In the embodiment 100, the front objective lens system 45 ally include a lens with varying magnification. As one
124 is shown to include a single front objective lens posi example, a pair of thin and large diameter Alvarez plates cantioned to establish a common field-of-view (FOV) for the be used to define at least a portion of the front objective lensreimaging lenses 128a and to define an aperture stop for the system.
whole system (which, in this specific case, Substantially
In further reference to FIG. 1, the FPA component con
spatially coincides with limiting apertures corresponding to 50 figured to receive the optical data representing spectral
different optical channels 120). As a result, the positions for signature(s) of the imaged object can be configured as aspectral encoding of the different optical channels 120 single FPA 136 adapted to acquire more than one Sub-image
coincide with the position of the aperture stop of the whole (formed along more than one optical channel 120) simulta
system, which is defined as a surface between the lens neously. Alternatively, the detector component may include
system 124 and the array 128 of the reimaging lenses 128a. 55 a set of optical FPAS at least one of which can be configured
Generally, however, the field apertures corresponding to to acquired more than one spectrally distinct Sub-image ofdifferent optical channels may be located in different planes. the imaged object (For example, as shown in the embodi
In one implementation the field apertures corresponding to ment 200 of FIG. 2, an array of optical FPAS can include
FPAS 236 the number of which may correspond to thedifferent optical channels are located in different planes,
which planes are optical conjugates of one another (as 60 number of the front objective lenses 224). In one implemen
defined by the whole optical system). Similarly, while all of tation of the system, an array of optical FPAS includes an
the spectral filters 130 of the embodiment 100 are shown to array of microbolometers. The use of multiple microbolom
lie in one plane, generally spectral filters corresponding to eters advantageously allows for an inexpensive way todifferent optical filters can be associated with different increase the total number of detection elements (i.e. pixels)
planes. In one implementation, different spectral filters 130 65 for recording of the datacube in one Snapshot. An array ofare situated in different planes are that are optically conju microbolometers more efficiently utilizes the detector pixels
gate to one another.
for each FPA as the number of unused pixels is minimized

US 9,625,318 B2
7
8
and/or eliminated between the Sub-images that may exist 460a, 460b, and then onto the detector module 456 (awhen using a single microbolometer.
microbolometer array or other IR FPA).
FIG. 3A illustrates schematically a related embodiment
The embodiment 400 commissions several operational
300 of the imaging system of the invention, in which the advantages. It is configured to provide a spectrally known
number of the front objective lenses 324a in the lens array object within every Sub-image and for every Snapshot acqui
324, the number of re-imaging lenses 128a in the lens array sition which can be calibrated against. (Such spectral cer
128, and the number of FPAS 336 are the same. So config tainty is expedient when using an array of IR FPAS likeured, each combination of respectively corresponding front microbolometers the detection characteristics of which canobjective lens 324, re-imaging lens 128a, and FPA 336 change from one imaging frame to the next due to, in part,defines an individual imaging channel associated with acqui 10 changes in the scene being imaged as well as the thermal
sition of the IR light transmitted from the object 110 through effects caused by neighboring FPAs.) In addition, the field
an individual optical filter component 130. A field reference reference array 438 of the embodiment 400 is preferably—
338 of the system 300 is configured to have a uniform but not necessarily—disposed within the Rayleigh range
temperature across its surface and be characterized by a (-the depth of focus) associated with the front objective
predetermined spectral curve of radiation emanating there 15 lenses 424, thereby removing unusable blurred pixels due tofrom. The filed reference 338 is used for dynamically having the field reference outside of this range. Moreover,adjusting the data output from each FPA 336 after acquisi the embodiment 400 is more compact then, for example, thetion of light from the object 110 to ensure that output of each configuration 300 of FIG. 3A (which requires the employedof the FPAs 336 represents correct acquired data, with field reference 338 to be separated from the lens array 324
respect to the other FPAs 336 for analysis, as discussed by a distance greater than several (for example, five) focal
below in more detail. In one implementation, when a 4x3 lengths to minimize blur contributed by the field reference toarray 340 of optical components (lenses 324a, 128a; detec an image formed at a detector plane.
tor elements 336), shown schematically in FIG. 3B, is used
In another related embodiment (not shown in FIGS. 1, 2,
behind the temperature controlled reference target 160, the 3A, and 4), the multi-optical FPA unit of the IR imaging
field reference 338 is adapted to obscure and/or block a 25 system of the invention additionally includes an FPA con
peripheral portion of the bundle of light propagating from figured to operate in a visible portion of the spectrum. In
the object 110 towards the detector(s) 336. As a result, the reference to FIG. 1, for example, an image of the scene offield reference 338 obscures and/or blocks the border orinterest formed by such visible-light FPA may be used as aperipheral portion(s) of the images of the object 110 formed background to form a composite image by overlappingon the FPA elements located along the perimeter 346 of the 30 (whether virtually, with the use of a processor and specifi
detector system. Generally, two detector elements will be cally-designed computer program product enabling Suchproducing substantially equal values of digital counts when data processing, or actually, by a viewer) an IR image (thatthey are used to observe the same portion of the scene in the is created based on the image data acquired by the individual
same spectral region using the same optical train. If any of FPAS 130) with the visible-light image. The so-formed
these input parameters (scene to be observed, spectral con 35 composite image facilitates the identification of the precise
tent of light from the scene, or optical elements delivering spatial location of the target species the spectral signatures
light from the scene to the two detector elements) differ, the of which the system of the invention is enabled to detect andcounts associated with the detectors will differ as well.
recognize.
Optical Filters.
Accordingly, and as an example, in a case when the two
FPAs 336 (such as those denoted as #6 and #7 in FIG. 3B) 40 It is appreciated that the optical filters, used with anremain substantially un-obscured by the field reference 338, embodiment of the system, that define spectrally-distinct IR
the outputs from these FPAS can be -dynamically adjusted to Sub-images of the object can employ absorption filters,
the output from one of the FPAs located along border (such interference filters, and Fabry-Perot etalon based filters, toas, for example, the FPA element #2) that processes spec name just a few. When interference filters are used, the45 image acquisition through an individual imaging channel
trally similar light.
FIG. 4 illustrates schematically a portion of another defined by an individual reimaging lens (such as a lens 128a
embodiment 400 that contains an array 424 of front objec of FIGS. 1, 2, 3, and 4) may be carried out in a single
tive lenses 424a adapted to receive light from the object 110 spectral bandwidth or multiple spectral bandwidths. Refer
that relay the received light to the array 128 of re-imaging ring again to the embodiments 100, 200, 300, 400 of FIGS.
lenses 128a through an array 438 of field references (field 50 1 through 4, and in further reference to FIG. 3B, examples
stops) 438a the spectral characteristics of which are known, of a 4-by-3 array of spectral filters 130 is shown in FIGS.
and through an array 440 of the relay lenses. The field 5A, 5B, where individual filters 1 through 12 are juxtaposed
references 438a are disposed at corresponding intermediate with a Supporting opto-mechanical element (not shown) toimage planes defined, with respect to the object 110, by define a filter-array plane that is oriented, in operation,
respectively corresponding front objective lenses 424a. 55 Substantially perpendicularly to the general optical axis 226(When refractive characteristics of all of the front objective of the embodiment.
The optical filtering configuration of one present embodi
lenses 424a are substantially the same, all of the fieldreferences 438a are disposed in the same plane). A field ment advantageously differs from a common approach usedreference 438a of the array 438 obscures (casts a shadow on) to measure spectra with an array of FPAS, where a bandpass
a peripheral region of a corresponding Sub-image formed at 60 filter defining a specified spectral band (such as, forthe detector plane 444 through a respectively corresponding example, any of the filters 0a through 4a the transmission
spatial imaging channel 450 of the system 400 prior to such curves of which are shown in FIG. 6A) is placed in front of
sub-image being spectrally processed by the processor 150. the optical FPA (generally, between the optical FPA and the
The array 440 of relay lenses then transmits light along each object). In particular, and in further reference to FIGS. 1, 2of the imaging channels 450 through different spectral filters 65 3, and 4 when optical detector(s) 136, 236, 336, 456 of an
454a of the filter array 454, past the two-point calibration embodiment include(s) microbolometers, the predominant
apparatus that includes two temperature controlled shutters contribution to noise associated with image acquisition isUS 9,625,318 B2
10
due detector noise. To compensate and/or reduce the noise, that in practice an embodiment of the invention can employ
an embodiment of the invention utilizes spectrally-multi any number of filters). For illustration, the case of M-7 isplexed filters. An example of spectral transmission charac considered. Analysis presented below relates to one spatial
teristics of spectrally-multiplexed filters 0b through 4b for location in each of Sub-images formed by differing imaging
use with an embodiment of the invention is depicted in FIG. channels defined by the system. As similar analysis can be
6B. (Filters of FIG. 6B are so-called long-wavelength pass, performed for each point at a Sub-image, the analysis can be
LP filters. An LP filter generally attenuates shorter wave appropriately extended as required.
lengths and transmits (passes) longer wavelengths over the
The unknown amount of light within each of the M
active range of the target IR portion of the spectrum. It is spectral channels (corresponding to these M filters) isappreciated that, in a related embodiment, short-wave 10 denoted with f. f. f. . . . . f. and readings fromlength-pass filters, SP, may also be used. An SP filter corresponding detector elements receiving light transmitted
generally attenuates longer wavelengths and transmits by each filter is denoted as g1 g2. g . . . g., while(passes) shorter wavelengths over the active range of the measurement errors are represented by n, n2 ns. . . . n.
target IR portion of the spectrum.)
Then, the readings at the seven FPA pixels each of which is
The related art appears to be silent with respect to an IR 15 optically filtered by a corresponding bandpass filter of FIG.
imaging system, adapted for detection of spectral signatures 6A can be represented by:of chemical species that combines the use of the spectrally
gift--n 1,
multiplexed filters with a Snap-shot image acquisition. The
lack of such teaching can probably be explained by the fact
g2 f+m2,that related imaging systems require the use of highly
sensitive and, for that reason, expensive cooled FPAs with
g3 f--n3.
reduced noise characteristics. Accordingly, the systems of
the related art are commonly employing bandpass filters
gaft-na.
instead, to take full advantage of spectral sensitivity of theused FPAs. Simply put, the use of spectrally multiplexed 25
gs fishns,
filters such as notched, LP, and SP filters would be coun

terproductive in a system of the related art, and would at
go fins,
least reduce an otherwise achievable SNR thereby degrading
the performance of the related art system for the intended
purpose. In contradistinction with the systems of the related 30 These readings (pixel measurements) g, arc estimates ofart, however, and at least in part due to the Snap-shot/non the spectral intensities f. The estimates g are not equal to thescanning mode of operation, an embodiment of the imaging corresponding f, values because of the measurement errors
system of the invention is enabled to use less sensitive
However, if the measurement noise distribution has zero
microbolometers without compromising the SNR. The use n.
mean, then the ensemble mean of each individual measureof microbolometers, as detector-noise-limited devices, in 35 ment
be considered to be equal to the true value, i.e.
turn not only benefits from the use of spectrally multiplexed (g)=f.canHere,angle brackets indicate the operation offilters, but also does not require cooling of the imaging calculating thethe
ensemblemean of a stochastic variable. The
system during normal operation.
of the measurement can, therefore, be represented
Referring again to FIGS. 6A, 6B, each of the filters variance(0b . . . 4b) transmits light in a substantially wider region of 40 aS
spectrum as compared to those of the filters (0a . . . 4a).
Accordingly, when the spectrally-multiplexed set of filters
In an alternative design utilizing spectrally-multiplexed(0b ... 0d) is used with an embodiment of the invention, the filters and in comparison with the design utilizing bandpass
overall amount of light received by the FPAs (for example, filters, the amount of radiant energy transmitted by each of
236, 336) is larger than would be received when using the 45
spectrally-multiplexed LP or SP filters towards a given
bandpass filters (0a . . . 4a). This “added transmission of theelement can exceed that transmitted through alight defined by the use of the spectrally-multiplexed LP (or detector
spectralbandof a bandpass filter. IN this case, the intensities
SP) filters facilitates increase of the signal on the FPAs of light corresponding
to the independent spectral bands canabove the level of the detector noise. Additionally, by using, bereconstructedbycomputational
means. (Such design isin an embodiment of the invention, filters the spectra of 50
to as a “multiplex design”).
which are wider than those of conventionally used band-pass referred
One matrix of such “multiplexed filter measurements
filters, the uncooled FPAs of the embodiment experience includes
matrix (requiring “negative' filters thatless heating due radiation incident thereon from the imaged may not abeHadamard
necessarily
for the optical embodi
scene and from radiation emanation form the FPA in ques ments disclosed herein)appropriateAn
S-matrix(which istion itself, due to a reduction in the back-reflected thermal 55 restricted to having a number of filtersapproach
equal
to
an
emission(s) coming from the FPA and reflecting off of the that is multiple of four minus one) or a row-doubledinteger
Had
filter from the non band-pass regions. As the transmission amard matrix (requiring a number of filters to be equal toanregion of the multiplexed LP (or SP) filters is wider, such integer multiple of eight) present alternative methodologies.
parasitic effects are reduced thereby improving the overall Here, possible numbers of filters using an S-matrix setup are
60
performance of the FPA unit.
3, 7, 11, etc and, if a row-doubled Hadamard matrix setup is
In one implementation, the LP and SP filters can be used,then 8, 16, 24, etc. For example, the goal of the
combined, in a spectrally-multiplexed fashion as described, measurement
may be to measure seven spectral band f
in order to maximize the spectral extent of the transmission intensities using
seven measurements gas follows:
region of the filter system of the embodiment.
The advantage of using spectrally multiplexed filters is 65
appreciated based on the following derivation, in which ag-Off OOff, In 2
system of M filters is examined (although it is understoodUS 9,625,318 B2
11

12
facilitates maintenance of proper calibration of each of the
FPAs individually and the entire FPA unit as a whole.
In particular, and in further reference to FIGS. 1, 2, 3, and4, the temperature-controlled unit generally employs a sys
tem having first and second temperature Zones maintained at
first and second different temperatures. For example, shutter
system of each of the embodiments 100, 200, 300 and 400
can employ not one but at least two temperature-controlled
shutters that are Substantially parallel to one another andOptical transmission characteristics of the filters 10 transverse to the general optical axis 226 of thedescribed above are depicted in FIG. 6B. Here, we no longer embodiment(s) 100, 200, 300, 400. Referring, for example,
have a direct estimate of the f, through a relationship similar to FIG. 4, in which such multi-shutter structure is alreadyto (g) f. Instead, if a “hat notation is used to denote an indicated, the use of multiple shutters enables the user toestimate of a given value, then a linear combination of the create a known reference temperature difference perceived,
measurements can be used Such as, for example,
15 by the FPA's 456 through the IR radiation emitted by the
shutter(s) 160a, 160b when these shutters are positioned toblock the radiation from the object 110. As a result, not only
the offset values corresponding to each of the individual
FPAS pixels can be adjusted but also the gain values of these
FPAs. In an alternative embodiment, the system having firstand second temperature Zones may include a single ormulti-portion piece (Such as a plate, for example) mechani
cally-movable across the optical axis with the use of appro
priate guides and having a first portion at a first temperature
25 and a second portion at a second temperature.
Indeed, the process of calibration of an embodiment of theinvention starts with estimating gain and offset (that varyfrom detector pixel to detector pixel) by performing mea
These fare unbiased estimates when the n, are zero mean 30 Surements of radiation emanating, independently, from atstochastic variables, so that (f-f)=0. The measurement least two temperature-controlled shutters of known anddifferent radiances. Specifically, first the response of thevariance corresponding to ith measurement is
detector unit 456 to radiation emanating from one shutter(for example, shutter 160a that is blocking the FOV of the
Therefore, by employing spectrally-multiplexed system the detectors 456 and the temperature T of which is measured
signal-to-noise ratio (SNR) of a measurement has been 35 directly and independently with thermistors) is carried out.
improved by a factor of V16/7=1.51.
Following such initial measurement, the shutter 160a is
For N channels, the SNR improvement achieved with a removed from the optical path of light traversing thespectrally-multiplexed system can be expressed as (N+1)/(2 embodiment and another shutter (for example, 160b) is
N). For example, in an embodiment employing 12 spectral inserted in its place across the optical axis 226 to prevent thechannels is characterized by SNR improvement, over a 40 propagation of light through the system. The temperature ofnon-spectrally-multiplexed system, by a factor of up to 1.88. the second shutter 160b is TzT is also independently
Two additional examples of related spectrally-multi measured with thermistors placed in contact with this shutplexed filter arrangements 0c through 4c, 0d through 4d ter, and the detector response to radiation emanating fromfrom the use of which an embodiment of the invention can
the shutter 160b is also recorded. Denoting operational
benefit when such embodiment includes an uncooled FPA 45 response of FPA pixels (expressed in digital numbers, or(such as a microbolometer) are shown in FIGS. 6C and 6D. “counts”) as g, to a source of radiance L, the readings
FIG. 6C illustrates a set of spectrally-multiplexed long corresponding to the measurements of the two shutters canwavelength pass (LP) filters is used in the system. An LP be expressed as:
filter generally attenuates shorter wavelengths and transmits(passes) longer wavelengths over the active range of the 50
target IR portion of the spectrum. A single spectral channel
having a transmission characteristic corresponding to thedifference between the spectral transmission curved of atHere, g, is the pixel offset value (in units of counts),
least two of these LP filters can be used to procure imaging and Y is the pixel gain value (in units of counts per radiancedata for the datacube with an embodiment of the invention. 55 unit). The solutions of these two equations with respect toAs alluded to above, an embodiment may optionally, and the two unknowns g, and Y can be obtained if the values
in addition to temperature-controlled reference unit (for of g and g and the radiance values L and L are available
example temperature controlled shutters such as shutters (either measured by a reference instrument or calculated
160; 160a, 160b), employ a field reference component (338 from the known temperatures T and T together with thein FIG. 3A), or an array of field reference components (438 60 known spectral response of the optical system and FPA). For
in FIG. 4), to enable dynamic calibration for spectral acqui any Subsequent measurement, one can then invert the equasition of every datacube, a spectrally-neutral camera-to tion(s) above in order to estimate the radiance value of thecamera combination to enable dynamic compensation of object from the detector measurement, and this can be done
parallax artifacts, and a visible and/or IR camera for com for each pixel in each FPA array within the system.
pensation of motion artifacts. The use of the temperature 65 As already discussed, and in reference to FIGS. 1 throughcontrolled reference unit (for example, temperature-con 4 the field-reference apertures may be disposed in an object
trolled shutter system 160) and field-reference component(s) space or image space of the optical system, and dimensioned

US 9,625,318 B2
14
13
to block a particular portion of the IR radiation received 324, 128a, in reference to FIG. 3). The calibration isfrom the object that, in absence of the field-reference aper necessitated by a need to form individual images in equiva
ture, would transmit through the optical system without lent units (so that, for example, the reading from each of thetraversing at least two spectrally-multiplexed optical filters.
FPA pixels can be re-calculated in units of temperature or
For example, the field-reference aperture the opening of 5 radiance units). Moreover, while it is often neglected inwhich is Substantially similar, in shape, to the boundary of practice, each of the FPAs should be spatially co-registered
the filter array (for example, and in reference to a filter array with one another so that a given pixel of a particular FPA canof FIGS. 3B, 5B rectangular), can be placed in front of the be confidently optically re-mapped through the optical sys
objective lens (124, 224, 324, 424) at a distance that is at tem of the embodiment to the same location at the object as
least several times (in one implementation—at least five 10
pixel at another FPA.
times) larger than the focal length of the lens in order to theTocorrespondingachieveat
least
some of these goals, a so-called
minimize the image blur that would occur in absence of Such spectral differencing method
may be employed, which
field-reference. In the embodiment 400 of FIG. 4 the field
employsformingadifference
image
from various combi
reference aperture can be placed within the depth of focus of
an image conjugate plane formed by the front objective lens 15 nations of the images registered by two or more different
424. The field reference, generally, effectuates and/or FPAs. If the optical filter 130 corresponding to a particularenables dynamic compensation in the system by providing a FPA 336 transmits light from the object including a cloud ofspectrally known and temporally-stable object within every gas, for example, with a certain spectrum that contains thescene to reference and stabilize the output from the different gas absorption peak or a gas emission peak while anotherfilter 130 corresponding to another FPA 336 does not trans
FPAs in the array.
Because each FPA's offset value is generally adjusted mit such spectrum, then the difference between the images
from each frame to the next frame by the hardware, com formed by the two FPAS at issue will highlight the presence
paring the outputs of one FPA with another can have an error of gas in the difference image.
that is not compensated for by the calibration parameters
A shortcoming of the spectral differencing method is thatg, and Y. In order to ensure that FPAS operate in radio 25 contributions of Some auxiliary features associated withmetric agreement, it is necessary for a portion of each imaging (not just the target species Such as gas itself) can
detector array to view a reference source (such as the field also be highlighted in and contribute to the difference image.
reference 338 in FIG. 3A, for example). Since the reference The so contributing effects include, to name just a few,
Source spectrum is known a priori (such as a blackbody parallax-induced imaging of edges of the object, influence ofSource at a known temperature), one can measure the 30 magnification differences between the two or more opticalresponse of each FPA to the reference source in order to channels, and differences in rotational positioning and ori
estimate changes to the pixel offset value. An example entation between the FPAs. While magnification-related
calculation of the dynamic offset proceeds as follows.
and FPA-rotation-caused errors can be compensatedAmong the FPA elements in an array of FPAS in a given errors
by increasing the accuracy of the instrument construction
embodiment of the invention, we select one FPA to be the 35 for“reference FPA'. We will attempt to make all of the FPAs as well as by post-processing of the acquired imaging,
agree with this one about the field reference temperature. parallax is scene-induced and is not so easily correctable. In
The image measured in each FPA contains a set of pixels addition, while it is not widely recognized, the spectral
obscured by the field reference 338. Using the previously differencing method is vulnerable to radiance calibration
obtained calibration parameters g, and Y (the pixel offset 40 errors. Specifically, if one FPA registers radiance of lightand gain), we estimate the effective blackbody temperature from a given feature of the object as that having a tempera
T of the field reference as measured by each FPA i. That is, ture of 40°C., for example, while the data from another FPA
represents the temperature of the same object feature as
being 39° C., then such feature of the object will beHere, the mean value is procured over all pixels that are 45 enhanced or highlighted in the difference image (formed at
obscured by the field reference, and Ag, is the difference in least in part based on the images provided by these twooffset value of the current frame from Age, obtained FPAs) due to such radiance-calibration error.
One solution to some of the problems introduced by theduring the calibration step. For the reference FPA, Ag, issimply set to Zero. Then, using the temperature differences spectral differencing is to normalize the resulting image data
50 by the data corresponding to a temporal reference image.
measured by each FPA, one obtains
This is referred to, for the purposes of this disclosure, as atemporal differencing algorithm or method. A temporalreference image may be formed, for example, by creating a
Once At, for each FPA is measured, its value can be difference image from the two or more images registered by
Subtracted from each image in order to force operational 55 the two or more FPAS at a single instance in time. It does notagreement between such FPA and the reference FPA. While matter whether corollary of the use of the algorithm of thethe calibration procedure has been discussed above in ref invention is that a prior knowledge of whether the object or
erence to calibration of temperature, a procedurally similar scene contains a target species (such as gas of interest) does
methodology of calibration with respect to radiance value not affect the results because the algorithm highlights
60 changes in the scene characteristics. Then, a spectral differ
can be implemented.
Examples of Methodology of Measurements.
ence image can be calculated as discussed above based on a
Prior to optical data acquisition with an embodiment of Snap-shot image acquisition at any later time and Subtracted
the IR imaging system of the invention, it is preferred to from the temporal reference image to form a normalized
calibrate all the FPAs of the system (such as FPAs 336 each difference image. The difference between the two highlightsof which forms an image of the object in light delivered in 65 the target species (gas) within the normalized differencea corresponding optical channel defined by the combination image, since this species was not present in the temporalof the corresponding front objective and re-imaging lenses reference frame. If necessary, more than two FPAS can beUS 9,625,318 B2
15
16
used both for registering the temporal reference image and using (instead of compensating or negating) the parallax—
a later-acquired difference image to obtain a better SNR induced effects described above. In this case, the measured
parallax between two or more similar spectral response
figure of merit.
While the temporal differencing method can be used to images can be used to estimate a distance between the
reduce or eliminate some of the shortcomings of the spectral imaging system and the gas cloud or between the imaging
differencing, it can introduce unwanted problems of its own.
system and an object in the field of view of the system. The
For example, temporal differencing of imaging data is less parallax-induced transverse image shift d between two
sensitive to calibration and parallax induced errors than the images is related to the distance Z between the cloud orspectral differencing of imaging data. However, any change object and the imaging system according to Z -Sz/d, wherein the imaged scene which is not related to the target species 10 S is the separation between two similar spectral responseof interest (such as particular gas, for example) is high images, and Z' is the distance to the image plane from thelighted in a temporally-differenced image, and thus may be back lens (Z' is typically approximately equal to the focal
erroneously perceived as a location of the target species length fof the lens of the imaging system). Once the distancetriggering, therefore, an error in detection of target species. Z between the cloud and the imaging system is calculated,
For example, if the temperature of the Scenic background 15 the size of the gas cloud can be determined based on theagainst which the gas is being detected changes (due to magnification equation, m f/Z, where each image pixel on
natural cooling down as the day progresses, or increases due the gas cloud, AX, corresponds to a physical size in object
to a person or animal or another object passing through the space AXAX"/m. To estimate the Volume of the gas cloud, a
FOV of the IR imaging system), then such temperature particular symmetry in the thickness of the cloud based onchange produces a signal difference as compared to the the physical size of the cloud can be assumed. For example,
measurement taken earlier in time. Accordingly, the cause of the cloud image can be rotated about a central axis running
the scenic temperature change (the cooling object, the per through the cloud image to create a three dimensionalson walking, etc.) may appear as the detected target species Volume estimate of the gas cloud size. It is worth noting that(such as gas). It follows, therefore, that an attempt to only a single imaging system of the invention is required forcompensate for operational differences among the individual 25 Such volume estimation, in contradistinction with carrying
FPAs of a multi-FPA IR imaging system with the use of out Such estimate with a spectral imaging system of related
methods that turn on spectral or temporal differencing cause art (in which case at least two imaging systems would be
additional problems leading to false detection of target necessary). Indeed, due to the fact that the information about
species. Among these problems are scene-motion-induced the angle at which the gas cloud is seen by the system is
detection errors and parallax-caused errors that are not 30 decoded in the parallax-effect, the image data includes thereadily correctable and/or compensatable as of to-date. information about the imaged scene viewed by the system in
Accordingly, there is an unfulfilled need to compensate for association with at least two angles.
When the temporal differencing algorithm is used forimage data acquisition and processing errors caused bymotion of elements within the scene being imaged. Embodi processing the acquired imaging data, a change in the scene
ments of data processing algorithms of the present invention 35 that is caused not by the target species is highlighted in theaddress and fulfill the need to compensate for the motion resulting image. According to an embodiment of the inven
induced and parallax-induced image detection errors.
tion, compensation of this error makes use of the temporal
In particular, to minimize parallax-induced differences differencing between two FPAs that are substantially equally
between the images produced with two or more predeter spectrally sensitive to the target species. In this case, themined FPAS, another difference image is used that is formed 40 temporal difference image will highlight those pixels thefrom the images of at least two different FPAS to estimate spectra of which have changed in time. Subtracting the data
parallax effects. For example, the spectral differencing of the corresponding to these pixels at both FPAS to form theimage data is being performed with the use of the difference resulting image, therefore, excludes the contribution of the
between the images collected by the outermost two cameras target species to the resulting image. The differentiationin the array (such as, for example, the FPAS corresponding 45 between (i) changes in the scene due to the presence of target
to filters 2 and 3 of the array of filters of FIG. 5A), forming species and (ii) changes in the scene caused by changes ina difference image referred to as a “difference image 2-3. In the background not associated with the target species is,
this case, the alternative “difference image 1-4” is addition therefore, enabled. It should be noted that, quite unexpect
ally formed from the image data acquired by, for example, edly, the data acquired with the visible-light FPA (whenthe alternative FPAs corresponding to filters 1 and 4 of FIG. 50 present as part of the otherwise IR imaging system) can also
5A. Assuming or ensuring that both of these two alternative be used to facilitate such differentiation and compensation of
FPAs have approximately the same spectral sensitivity to the the motion-caused imaging errors. Visible cameras generally
target species, the alternative “difference image 1-4 will have much lower noise figure than IR cameras (at least
highlight pixels corresponding to parallax-induced features during daytime). Consequently, the temporal differencein the image. Accordingly, based on positive determination 55 image obtained with the use of image data from the visiblethat the same pixels are highlighted in the spectral “differ light FPA can be quite accurate. The visible FPA can be usedence image 2-3 used for target species detection, the to compensate for motion in the system as well as many
conclusion is made that the image features corresponding to potential false-alarms in the scene due to motion caused bythese pixels are likely to be induced by parallax and not the people, vehicles, birds, and steam, for example, as long aspresence of target species in the imaged scene. It should be 60 the moving object can be observed in the visible region of
noted that compensation of parallax can also be performed the spectra. This has the added benefit of providing anusing images created by individual re-imaging lenses, 128a. additional level of false alarm Suppression without reducing
when using a single FPA or multiple FPA's as discussed the sensitivity of the system since many targets such as gas
above.
clouds cannot be observed in the visible spectral region.
Another capability of the embodiment of the invention is 65 Another method for detection of the gases is to use a
the ability to perform the volumetric estimation of a gas spectral unmixing approach. A spectral unmixing; approach
cloud volumetric estimation. This can be accomplished by assumes that the spectrum measured at a detector pixel isUS 9,625,318 B2
17
18
composed of a sum of component spectra, and attempts to tional performance as compared with the use of a bandpass
estimate the relative weights of these components needed to filter centered on the chose wavelength in the same spectral
derive the measurement spectrum. The component spectra channel.
are generally taken from a predetermined spectral library
It is worth noting that the use of optically-filtered FPAs
(for example, from data collection that has been empirically rather than a more conventional Fourier Transform spec
assembled), though sometimes one can use the scene to trometer (FTS) in an embodiment of the invention is partly
estimate these as well (often called “endmember determi explained by a larger number of total pixels available withnation'). For the gas cloud detection, the component spectra a single large format microbolometer FPA array. Moreinclude the absorption spectra of various gases of interest, importantly, however, the use of the FTS is well recognizedwhile the “measurement spectrum' is not the raw measure 10 to require tight mechanical tolerances, leading to sufficiently
ment of spectral intensity values but rather an “absorption more complex assembly of the imaging system employing
spectrum', which includes the spectrum of background light
FTS as compared to the assembly of the embodiment ofabsorbed on transmission through a cloud The spectral thetheinvention. Additionally, the FTS does not offer a highunmixing methodology can also benefit from temporal,
15 enough optical throughput for a Substantially low number ofparallax, and motion compensation techniques.
optical channels (for example, between 4 and 50 optical
Examples of Practical Embodiments and Operation.
The embodiment 300 of FIG. 3 was configured to employ channels) (in part because many of the sampled wavenum
12 optical channels and 12 corresponding microbolometer ber values in the reconstructed spectrum do not correspond
FPAS as 336 and used to capture a video sequence (repre to regions of the spectrum that the FTS instrument issenting images of a standard laboratory scene) Substantially sensitive to, and so Such sampled data is eventually dis
immediately after performing calibration measurements carded and not used for image formation and detection ofwith the use of a reference source including two shutters, as target species). The FTS is better suited to higher resolution
spectroscopy. The problem with working with highly-re
discussed above (one at room temperature and one 5° C.
above room temperature). The use of 12 FPAS offers a good Solved spectra, however, is that by sampling the samechance of simultaneous detection and estimation of the 25 amount of incident light with Smaller spectral bins means
concentrations of about 8 or 9 gases present at the scene, but that image data corresponding each bin is actually noisier.
the number of FPAS 336 can vary, depending on the balance Therefore, while improved spectral resolution accorded by
between the operational requirements and consideration of the FTS can allow the user to pick locations in the spectrum
COSt.
that are highly specific to the absorption/emission signature
Due to the specifics of operation in the IR range of the 30 of the target species, it also makes Such signature weaker
spectrum, the use of the so-called noise-equivalent tempera relative to the detection noise.
ture difference (or NETD) is preferred and is analogous to
A major advantage of the embodiments of the present
the SNR commonly used in visible spectrum instruments.
systemover instruments of the related art that are configured
The array of microbolometer FPAS 336 is characterized to for target
species detection (for example, gas cloud detec
perform at NETDs72 mK at an f-number of 1.2. Each 35 tion) is that,
according to the present invention, the entire
measurement was carried out by Summing four consecutive
spectrumisresolved
in a Snapshot mode (for example,
frames, and the reduction in the NETD value expected due
to Such Summation would be described by corresponding during one image frame acquisition by the FPA array). Thisfactor of V4–2. Under ideal measurement conditions, there enables the system of the invention to take advantage of the
40 compensation algorithms such as the parallax and motion
fore, the FPANETD should be about 36 mK.
The commercially off-the-shelf LP, SP, and/or BP filters compensation algorithms mentioned above. Indeed, as thewere used as the filters 130. Using the image data acquired imaging data required to implement these algorithms areat each FPA pixel, the processor of the system was used to collected simultaneously with the target-species related data,
calculate the mean value and the standard deviation of thethe compensation algorithms are carried out with respect toimage data across the video sequence, to form a data output 45 target-species related data and not with respect to data
representing a “mean image' and a 'standard deviation acquired at another time interval, thereby ensuring accuracy
image' of the scene.
of the data compensation process. In addition, the frame rate
Table 1 summarizes the “mean NETD' values obtainedof data acquisition is much higher (the present system
by averaging of the NETD values over all pixels of the operates at up to video rates; from about 5 Hz and higher, forstandard deviation image', as well as the median NETD 50 example), so that the user is enabled to recognize in thevalues obtained in a similar fashion, in degrees Celsius. The images the wisps and Swirls typical of gas mixing without
top section of Table 1 shows the results for the bandpass blurring out of these dynamic image features and otherfilters, the middle section of Table 1 shows the results for thecaused by the change of scene (whether spatial or
LP and SP filters, and the bottom section of Table 1 presents artifacts
spectral) during the lengthy measurements. In contradistincdata for differences between NETD values of two specified 55 tion
the imaging systems of the related art that requireLP filters. (Note that no filter is treated as equivalent to a imagewith
data acquisition over a period of time exceeding a
longpass filter here.)
single-snap-shot time and, therefore, blur the target gas
The results of Table 1 indicate that the difference between
the NETD values of the two chosen LP filters is substantially features in the image and inevitably reduce the otherwise
smaller than the NETD value corresponding to a single BP 60 achievable sensitivity of the detection, embodiments of thefilter, thereby providing an experimental proof that the use present invention make detecting the localized concentraof an embodiment of the invention as discussed above
tions of gas without it being Smeared out and/or averaged
provides an unexpected increase in a SNR-type figure of with the areas of thinner gas concentrations. In addition, themerit of the IR spectral imaging in comparison with a system higher frame rate also enables a much faster response rate toof related art. In other words, the use of two (or more) LP 65 a leak of gas (when detecting Such leak is the goal): an alarm
or SP filters to extract the imaging data results in a spec can trigger within fractions of a second rather than several
trally-narrowed imaging channel having a higher opera seconds.

US 9,625,318 B2
19

20

TABLE 1.

embodiment 300 of FIG. 3A. The results of detection of
NETD (deg C.

ilter

BP-8224
BP-9000
BP-948O
BP-974O
BP-10240
BP-107OO
BP-10962
BP-11660
BP-12227
BP-13289(8.

median

O.691
O.938
O.318
0.372
0.275
O.409
0.44
O.S.48
O.48
1.309

0.675
O.923
O.315
O.369
O.286
O4OS
O437
O.S.42
O.475
1.26

none

O.O38

O.O37

LP-811 O
LP-85OO
LP-811 O
LP-83OS
LP-85OO
LP-9000
LP-9650
LP-9800
LP-11000
LP-11450
SP-1OSOO

O.O63
O.O76
O.O68
O.O68
O.O8
O.O73
O.O99
O.109
O.156
O.2O7
O.O7

O.O63
0.075
OO67
OO67
O.08
O.O73
O.098
O.108
O.156
O.2O6
O.O7

none-LP-8110
LP-9650)-LP-11450)
LP-980O-LP-11000

O.O7
O.208
O.227

(i.e. methane)(i.e. propylene)(i.e. butane)

radiation carried out after each detector pixel has been
subjected to the dynamic calibration procedure describedabove are expressed with the curved labeled “obj corr' (or“B”). Now, the difference in estimated temperature of theobject among the detector elements is reduced to about 0.5°
C. (thereby improving the original reading at least by afactor of 5).
FIG. 8B represents the results of similar measurements
10 corresponding to a different location in the scene (a location
which is at a temperature about 9° C. above the estimatedtemperature of the aperture 338 of FIG. 3A). As shown, thecorrection algorithm discussed above is operable and effective and applicable to objects kept at different temperature.
15 Accordingly, the algorithm is Substantially temperature
independent.
At least some elements of a device of the invention can becontrolled—and at least some steps of a method of theinvention can be effectuated, in operation—with a program
mable processor governed by instructions stored in amemory. The memory may be random access memory(RAM), read-only memory (ROM), flash memory or any
other memory, or combination thereof. Suitable for storing
control software or other instructions and data. Those skilledin the art should also readily appreciate that instructions orprograms defining the functions of the present invention
may be delivered to a processor in many forms, including,
To demonstrate the operation and gas detection capability but
not limited to, information permanently stored on nonof an embodiment of the invention, a prototype was con Writablestorage media (e.g. read-only memory devices
structed in accordance with the embodiment 300 of FIG. 3A 30
within a computer, such as ROM, or devices readable by aand used to detect a hydrocarbon gas cloud of propylene at computer I/O attachment, such as CD-ROM or DVD disks),
a distance of approximately 10 feet. FIG. 7 illustrates video information alterably stored on writable storage media (e.g.
frames 1 through 12 representing gas-cloud-detection output floppy disks, removable flash memory and hard drives) or
710 (seen as a streak of light) in a sequence from t=1 to t—12. information conveyed to a computer through communica
The images 1 through 12 are selected frames taken from a 35 tion media, including wired or wireless computer networks.
Video-data sequence captured at a video-rate of 15 frames/ In addition, while the invention may be embodied in softSec. The detected propylene gas is shown as a streak of light ware, the functions necessary to implement the invention
710 (highlighted in red) near the center of each image. The may optionally or alternatively be embodied in part or in
first image is taken just prior to the gas emerging from the whole using firmware and/or hardware components, such as
noZZle of a gas-contained, while the last image represents 40 combinatorial logic, Application Specific integrated Circuits
the system output shortly after the nozzle has been turned (ASICs), Field-Programmable Gate Arrays (FPGAs) or
off.

25

other hardware or some combination of hardware, software

Using the same prototype of the system, the demonstra and/or firmware components.
tion of the dynamic calibration improvement described
While examples of embodiments of the system andabove by imaging the scene Surrounding the system (the 45 method of the invention have been discussed in reference tolaboratory) with known temperature differences. The result the gas-cloud detection, monitoring, and quantification (inof implementing the dynamic correction procedure is shown cluding but not limited to greenhouse gases such as Carbon
in FIGS. 8A, 8B, where the curves labeled “obj” (or “A”) Dioxide, Carbon Monoxide, Nitrogen Oxide as well as
represent temperature estimates of an identified region in the hydrocarbon gases such as Methane, Ethane, Propane, n-Bu
scene. The abscissa in each of the plots of FIGS. 8A, 8B 50 tane, iso-Butane, n-Pentane, iso-Pentane, neo-Pentane,
indicates the number of a detector element, while the ordi
Hydrogen Sulfide, Sulfur Hexafluoride, Ammonia, Benzene,nate corresponds to temperature (in degrees C.). Accord p- and m-Xylene, Vinyl chloride, Toluene, Propylene oxide,
ingly, it is expected that when all detector elements receive Propylene, Methanol, Hydrazine, Ethanol, 1,2-dichloroeth
radiant data that, when interpreted as the objects tempera ane, 1,1-dichloroethane, Dichlorobenzene, Chlorobenzene,ture, indicates that the objects temperature perceived by all 55 to name just a few), embodiments of the invention can bedetector elements is the same, any given curve would be a readily adapted for other chemical detection applications.
substantially flat line. Data corresponding to each of the For example, detection of liquid and Solid chemical spills,
multiple “obj' curves are taken from a stream of video biological weapons, tracking targets based on their chemicalframes separated from one another by about 0.5 seconds (for composition, identification of satellites and space debris,a total of 50 frames). The recorded “obj’ curves shown in 60 ophthalmological imaging, microscopy and cellular imag
FIG. 8A indicate that the detector elements disagree about ing, endoscopy, mold detection, fire and flame detection, andthe objects temperature, and that difference in objects pesticide detection are within the scope of the invention.
temperature perceived by different detector elements is as
What is claimed is:high as about 2.5° C. In addition, all of the temperature
estimates are steadily drifting in time, from frame to frame. 65 1. An infrared (IR) imaging system for determining a
The curves labelled “ref (or “C”) correspond to the detec concentration of a target species in an object, the imaging
tors' estimates of the temperature of the aperture 338 of the system comprising:US 9,625,318 B2
21

22

an optical system including an optical focal plane array
14. The method of claim 12, wherein at least some of the(FPA) unit, the optical system comprising a plurality of plurality of spatially and spectrally different channels com
optical channels defined by components of the optical prise spectral filters.
System that are arranged in a two-dimensional array,
15. The method of claim 14, wherein the spectral filters
said plurality of optical channels being spatially and 5 includes one of a longpass optical filter or a shortpass optical
spectrally different from one another and configured to filter.
direct IR radiation from the object towards the optical
16. The method of claim 12, further comprising acquiring
FPA unit:
multispectral optical data representing the object from thea first movable shutter configured to be removably posi received IR radiation in a single occurrence of data acquitioned to block IR radiation incident onto the optical 10 sition with a programmable processor.
system from the object, wherein a portion of the
17. The method of claim 12, configured to identify target
movable shutter is maintained at a first temperature and species
in a gas cloud.
another portion of the movable shutter is maintained at
18.
The
method of claim 12, further comprising calibrat
a second temperature; anding the FPA unit based on the first and the second tempera
a processor configured to acquire multispectral optical 15 ture.
data representing said target species from the received
19. An infrared (IR) imaging system for determining a
IR radiation in a single occurrence of data acquisition. concentrationof a target species in an object, the imaging
2. The system of claim 1, wherein the FPA unit does not System comprising:require a cooler to cool during normal operation.
an optical system including an optical focal plane array
3. The system of claim 1, wherein the IR radiation(FPA) unit, the optical focal plane array (FPA) unit
comprises wavelengths between about 1 micron and aboutcomprisinga plurality of focal plane arrays arranged in
20 microns.
a two-dimensional array to form a plurality of spatially
4. The system of claim 1, wherein the processor isdifferent channels arranged in a two-dimensional array
configured to map an overall image datacube representing athat are configured to receive IR radiation from thespatial distribution of concentrations c of the target species. 25
object, wherein the two-dimensional array comprises at
5. The system of claim 1, wherein the system is config
least two rows and two columns of focal plane arrays;
ured to identify target species in a gas cloud.
a
first
and a second movable shutter configured to be
6. The system of claim 1, further comprising a plurality ofremovably positioned to block IR radiation incident
spectral filters, each filter positioned in a respective one ofonto the optical system from the object, wherein the30
said plurality of optical channels.
first movable shutter is maintained at a first temperature
7. The system of claim 6, wherein the spectral filters
and the second movable shutter is maintained at acomprise at least one long pass (LP) filter.
second temperature for calibration; and8. The system of claim 6, wherein the spectral filtersaprocessor
configured to acquire multispectral opticalcomprise at least one short pass (SP) filter.
data representing said target species from the received
9. The system of claim 6, further comprising a plurality of 35
IR radiation.
reimaging lenses, each reimaging lens positioned to transmit
20. The system of claim 19, wherein the FPA unit does nota portion of the IR radiation transmitted through a respective requirea cooler to provide cooling during normal operation.
one of said spectral filters towards the optical FPA unit.
21. The system of claim 19, wherein the IR radiation
10. The system of claim 1, further comprising an optical comprises
aperture that circumscribes said plurality of optical chan 40 20 microns.wavelengths between about 1 micron and about
nels.
22. The system of claim 19, wherein the plurality of
11. The system of claim 1, wherein the processor isconfigured to calibrate the FPA unit based on the first and the channels are spectrally distinct from each other.
23. The system of claim 19, wherein the system issecond temperature.
to identify target species in a gas cloud.
12. A method for operating an infrared (IR) imaging 45 configured
24.
The
systemof claim 19, further comprising a plurality
System, comprising:of spectral filters, each filter positioned in a respective one
receiving IR radiation from an object along a plurality of ofsaid plurality of channels.
spatially and spectrally different channels formed by a
25. The system of claim 24, wherein the spectral filters
plurality of focal plane arrays of an optical focal plane compriseleast one long pass (LP) filter.
array (FPA) unit, the plurality of focal plane arrays 50 26. Theatsystemof claim 24, wherein the spectral filtersarranged in a two-dimensional array comprising at least comprise
at
least
one
short pass (SP) filter.
two rows and two columns; and27. The system of claim 19, wherein the processor isremovably positioning a first and a second movable configured
to acquire multispectral optical data in a single
shutter in front of the optical system to block IR
of data acquisition.
radiation incident onto the optical system from the 55 occurrence
28.
The
system of claim 19, wherein the system isobject, wherein the first movable shutter is maintained
to identify target species in a gas cloud.
at a first temperature and the second movable shutter is configured
29. The system of claim 19, wherein the processor ismaintained at a second temperature.
configured to calibrate the FPA unit based on the first and the
13. The method of claim 12, wherein the FPA unit does
not require a cooler to provide cooling during normal 60 second temperature.
operation.

ck

ck

ci:

ck

ck