USO095995.08B2(12) United States Patent(10) Patent No.:US 9,599,508 B2(45) Date of Patent:
Mar. 21, 2017

Kester et al.
(54)

(56)

DIVIDED-APERTURE INFRA-RED
SPECTRAL MAGING SYSTEM

U.S. PATENT DOCUMENTS(71) Applicant: REBELLION PHOTONICS, INC.,
Houston, TX (US)4,134,683 A
4,963,963 A(73) Assignee: Rebellion Photonics, Inc., Houston, TX(US)(*) Notice: Subject to any disclaimer, the term of thispatent is extended or adjusted under 35
U.S.C. 154(b) by 0 days.
(21) Appl. No.: 14/539,899
Prior Publication Data

US 2015/O136981 A1

FOREIGN PATENT DOCUMENTS
CACA

2 365 866
2870 419

9, 2000
5/2O15(Continued)
OTHER PUBLICATIONS

George M. Williams, “Dual-Band MWIR/LWIR Radiometer for
Absolute Temperature Measurements'. Proc. of SPIE vol. 6205
62050, pp. 1-13.*(Continued)
Primary Examiner — David Porta
Assistant Examiner — Faye Boosalis(74) Attorney, Agent, or Firm — Knobbe Martens Olson

Nov. 12, 2014(65)

1/1979 Goetz et al.
10, 1990 Dorman(Continued)(72) Inventors: Robert Timothy Kester, Pearland, TX(US); Nathan Adrian Hagen, Houston,
TX (US)(22) Filed:

References Cited

May 21, 2015

& Bear LLP(57)

Related U.S. Application Data

(63) Continuation-in-partofapplication

No.

PCT/US2013/041278, filed on May 6, 2013.
(Continued)(51)

Int. C.

GOI 3/28
GOI 3/02(2006.01)(2006.01)(Continued)(52)

U.S. C.

CPC ........... G0IJ 3/0232 (2013.01); G0IJ 3/0229(2013.01); G0IJ 3/28 (2013.01); G0IJ 3/2823(2013.01);(Continued)(58)

Field of Classification Search

CPC ........ G01J 3/36; G01J 3/02; G01J 3/28: G01J
3/0232; G01N 21/3504: G01N 2201/068

ABSTRACT

Various embodiments disclosed herein describe a divided

aperture infrared spectral imaging (DAISI) system that is
adapted to acquire multiple IR images of a scene with asingle-shot (also referred to as a Snapshot). The plurality ofacquired images having different wavelength compositions
that are obtained generally simultaneously. The system
includes at least two optical channels that are spatially andspectrally different from one another. Each of the at least two
optical channels are configured to transfer IR radiation
incident on the optical system towards an optical FPA unit
comprising at least two detector arrays disposed in the focal
plane of two corresponding focusing lenses. The systemfurther comprises at least one temperature reference Sourceor surface that is used to dynamically calibrate the two
detector arrays and compensate for a temperature difference
between the two detector arrays.

See application file for complete search history.

19 Claims, 15 Drawing Sheets

to objectthermat:ycontroe
sites
3ea

occK
980c.

US 9,599,508 B2
Page 2
Related U.S. Application Data

OTHER PUBLICATIONS(60) Provisional application No. 61/903,075, filed on Nov.
12, 2013, provisional application No. 61/764,776,
filed on Feb. 14, 2013, provisional application No.
61/688,630, filed on May 18, 2012.
(51) Int. Cl.
GOIN 2L/3504
GOI 3/36
GOIJ 5/OO(2014.01)(2006.01)(2006.01)(52) U.S. Cl.
CPC ............ G0IJ 3/36 (2013.01); G0IN 21/3504(2013.01); G0IJ 2005/0077 (2013.01)(56)

References Cited
U.S. PATENT DOCUMENTS
5,127,742 A

7, 1992 Fraden

5,877,500 A

3/1999 Braig et al.

5,920,066 A

7, 1999 DiRenzo et al.

5,926,283 A

7/1999 Hopkins

6,023,061 A

2/2000 Bodkin

6,268,883 B1*

7/2001 Zehnder ................... HO4N 5/33

6,456,261 B1

9/2002 Zhang

6,680,778 B2
7,242.478 B1
7,315,377 B2

1/2004 Hinnrichs et al.
7/2007 Dombrowski et al.
1/2008 Holland et al.

7,750,802 B1*

7/2010 Parish .................... GON 21.64

7,835,002 B2
7,888,624 B1

11/2010 Muhammed et al.
2/2011 Murguia et al.

8,159,568 B2
8,653,461 B1

4/2012 Ahdoot
2/2014 Benson et al.

8,654,328 B2
2002fOO15151 A1
2002/O121370 A1

2/2014 Tkaczyk et al.
2/2002 Gorin
9/2002 Kurkjian et al.

2002fO159101 A1

10, 2002 Alderson et al.

2003/0134426 A1

7/2003 Jiang et al.

250/338.1

2005, 0103989 A1

5, 2005 Watson et al.

3/2006 Hagene et al.

2008/0204744 A1
2008, 0231719 A1

8, 2008 Mir et al.
9, 2008 Benson et al.

2008/025 1724 A1

10/2008 Baliga et al.

2012/0273680 A1

1 1/2012 Furry

2010/0211333 A1
2011/0261321 A1

2013, O181836 A1
2013/0321806 A1
2013/0341509 A1
2015,0069.239 A1
2015,0136982 A1
2015,0316473 A1
2016/009.7713 A1
2016/0349228 A1
2017, 0026588 A1

8, 2010 Pruet et al.
10/2011 Ramella-Roman et al.

7, 2013
12/2013
12/20133/2015
5, 2015
11/2015
4/2016
12/2016
1/2017

Cardoso et al.
Kester et al.
Nelson et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.

FOREIGN PATENT DOCUMENTS
EP
EP
EP
EP
GB
WO
WO
WO
WO
WO
WO
WO
WO
WO

O 837 600
4f19982871 4.52
5/20152.942 615
11/2015
2.955 496
12/20151014769
12/1965
WO 2004/097389
11 2004
WO 2007/0O8826
1, 2007
WO 2008/109183
9, 2008
WO 2009094782 A1 * 8, 2009
WO 2010/053979
5, 2010
WO 2012/078417
6, 2012
WO 2012/082366
6, 2012
WO 2013, 173541
11 2013
WO 2016, 196224
12/2016

Alvarez et al., “Greater Focus Needed on Methane Leakage fromNatural Gas Infrastructure', PNAS, Apr. 24, 2012, vol. 109, No. 17.
pp. 12.
Bedard et al., “Image Mapping Spectrometry: Calibration andCharacterization”. Optical Engineering, Nov. 2012, vol. 51, No. 11,
pp. 111711-1-111711-13.
Ben-David et al., “Probability Theory for 3-Layer Remote Sensing
Radiative Transfer Model: Univariate Case,” Optics Express, Apr.
2012, vol. 20, No. 9, pp. 10004-10033.
Ben-David et al., “Probability Theory for 3-Layer Remote Sensing
Radiative Transfer Model: Errata,” Optics Express, May 20, 2013,vol. 21, No. 10, pp. 11852.
Brady et al., “Multiscale Lens Design”. Optics Express, Jun. 22.
2009, vol. 17, No. 13, pp. 10659-10674.
Caulton et al., “Toward a Better Understanding and Quantificationof Methane Emissions from Shale Gas Development', PNAS, Apr.
29, 2014, vol. 111, No. 17, pp. 7.
Chen et al., “Quantitative Sectioning and Noise Analysis for Structured Illumination Microscopy: Erratum'. Optics Express, Oct. 19.
2015, vol. 23, No. 21, pp. 27633-27634.
Chidley et al., “Flow-Induced Birefringence: The Hidden PSF
Killer in High Performance Injection-Molded Plastic Optics'.
Endoscopic Microscopy, Proceedings of SPIE vol. 6082, 2006, pp.
11.

250/370.1

2006, OO44562 A1

Allen et al., “Measurements of Methane Emissions at Natural Gas
Production Sites in the United States', PNAS, Oct. 29, 2013, vol.
110, No. 44, pp. 7.

................ GO1J 3.02

Chu et al., “The NIST Quantitiative Infrared Database'. Journal of
Research of the National Institute of Standards and Technology,
Jan.-Feb. 1999, vol. 104, No. 1, pp. 59-81.
DiPietro et al., “Hyperspectral Matched Filter with False-Alarm
Mitigation”. Optical Engineering, Jan. 2012, vol. 51, No. 1, pp.
O162O2-1-0162O2-7.

Eriksson et al., “Radiative Cooling Computed for Model Atmo
spheres'. Applied Optics, Dec. 1, 1982, vol. 21, No. 23, pp.
4381-4388.

Flanigan, “Detection of Organic Vapors with Active and Passive
Sensors: A Comparison.” Applied Optics, 1986, vol. 25, No. 23, pp.
4253-4260.

Gálfalk et al., “Making Methane Visable', Nature Climate Change,
Apr. 2016, vol. 6, pp. 426-430.
Gálfalk et al., “Making Methane Visable'. Supplementary Informa
tion, Nature Climate Change, 2015, pp. 1-14.
Gallagher et al., “Error Analysis for Estimation of Trace Vapor
Concentration Pathlength in Stack Plumes”. Applied Spectroscopy,
2003, vol. 57, No. 6, pp. 614-621.
Gallagher et al., “Estimation of Trace Vapor Concentration
Pathlength in Plumes for Remote Sensing Applications from
Hyperspectral Images'. Analytica Chimica Acta, 2003, vol. 490, pp.
139-152.

Gao et al., “Compact Image Slicing Spectrometer (ISS) for
Hyperspectral Fluorescence Microscopy”. Optics Express, Jul. 20.
2009, vol. 17, No. 15, pp. 12293-12308.
Gao et al., “Depth-Resolved Image Mapping Spectrometer (IMS)with Structured Illumination”. Optics Express, Aug. 29, 2011, vol.
19, No. 18, pp. 17439-17452.
Gao et al., “Optical Design of a Snapshot High-Sampling Image
Mapping Spectrometer (IMS) for Hyperspectral Microscopy”.
Three-Dimensional and Multidimensional Microscopy:Image
Acquisition and Processing XVII, Proceedings of SPIE vol. 7570,
2010, pp. 1-7.
Gao et al., “Quantitative Comparison Between Full-Spectrum andFilter-Based Imaging in Hyperspectral Fluorescence Microscopy'.
Journal of Microscopy, 2012, vol. 246, No. 2, pp. 113-123.
Gao et al., “Snapshot Image-Mapping Spectrometer for
Hyperspectral Fluorescence Microscopy”. Optics and Photonics
News, Nov. 2010, vol. 21, No. 12, p. 50.
Gao et al., “Snapshot Image Mapping Spectrometer (IMS) withHigh Sampling Density for Hyperspectral Microscopy”. Optics
Express, Jul. 5, 2010, vol. 18, No. 4, pp. 14330-14344.

US 9,599,508 B2
Page 3(56)

References Cited
OTHER PUBLICATIONSGerhart et al., “Detection and Tracking of Gas Plumes in LWIR
Hyperspectral Video Sequence Data, Algorithms and Technologies
for Multispectral, Hyperspectral, and Ultraspectral Imagery XIX.
2013, SPIE Proceedings vol. 8743, pp. 1-14.
Gittins, Christopher M., “Detection and Characterization of Chemi
cal Vapor Fugitive Emissions by Nonlinear Optimal Estimation:
Theory and Simulation'. Applied Optics, Aug. 10, 2009, vol. 48,
No. 23, pp. 4545-4561.
Goldberg et al., “Dual Band MWIR/LWIR Focal Plane Array Test
Results.” Army Research Lab, Adelphi, MD, Aug. 1999, pp. 18.
Golowich et al., “Performance Limits of LWIR Gaseous Plume

Quantification'. Algorithms and Technologies for Multispectral,Hyperspectral, and Ultraspectral Imagery XVII, 2011, Proceedingsof SPIE vol. 8048, pp. 1-12.

Griffin et al., “The Herschel—SPIRE 1-15 Instrument and its

In-Flight Performance.” Astronomy and Astrophysics, Jul. 1, 2010,
vol. 518, pp. 7.
Gross et al., “Remote Identification and Quantification of Industrial
Smokestack Effluents via Imaging Fourier-Transform Spectros
copy”. Environmental Science & Technology, 2010, vol. 44, No. 24.
pp. 9390-9397.
Gupta et al., “Miniature Snapshot Multispectral Imager,” OpticalEngineering, 2011, vol. 50, pp. 033203-1-033203-9.
Hadlington, Simon, “New Camera Makes Methane Visible'. Chem
istry World, http://web.archive.org/web/20160305234907/http://
www.rsc.org/chemistry world/2015/12/methane-camera-infraredgreenhouse-gas, Dec. 14, 2015, pp.
Hagen et al., “Analysis of Computed Tomographic Imaging Spec
trometers. I. Spatial and Spectral Resolution'. Applied Optics, Oct.
1, 2008, vol. 47, No. 28, pp. F85-F95.
Hagen et al., “Coded Aperture DUV Spectrometer for Standoff
Raman Spectoscopy'. Next-Generation Spectroscopic Technolo
gies II, Proceedings of SPIE vol. 7319, 2009, pp. 1-10.
Hagen et al., “Compound Prism Design Principles, I'. Applied
Optics, Sep. 1, 2011, vol. 50, No. 25, pp. 4998-5011.
Hagen et al., “Compound Prism Design Principles, II: Triplet andJanssen Prisms”. Applied Optics, Sep. 1, 2011, vol. 50, No. 25, pp.
5O12-5022.

Hagen et al., “Compound Prism Design Principles, III: Linear-in
Wavenumber and Optical Coherence Tomography Prisms”. Applied
Optics, Sep. 1, 2011, vol. 50, No. 25, pp. 5023-5030.
Hagen et al., “Fourier Methods of Improving Reconstruction Speed
for CTIS Imaging Spectrometers'. Imagin Spectrometry XII, Pro
ceedings of SPIE vol. 6661, 2007, pp. 11.
Hagen et al., “Foveated Endoscopic Lens”, Journal of Biomedical
Optics, Feb. 2012, vol. 17, No. 2, pp. 021104-1-021104-6.
Hagen et al., “Gaussian Profile Estimation in One Dimension'.
Applied Optics, Aug. 1, 2007, vol. 46, No. 22, pp. 5374-5383.
Hagen et al., “Gaussian Profile Estimation in Two Dimension'.
Applied Optics, Dec. 20, 2008, vol. 47, No. 36, pp. 6842-6851.
Hagen et al., “Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy”. Optics Express, Jan. 2, 2012,vol. 20, No. 1, pp. 403-413.
Hagen et al., “Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy: Errata'. Optics Express, Feb.
27, 2012, vol. 20, No. 5, pp. 5343.
Hagen et al., “Real-Time Quantitative Hydrocarbon Gas Imaging
with the Gas Cloud Imager (GCI)”. Proceedings of SPIE, vol. 8358,
Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE) Sensing XIII, May 1, 2012, pp. 7.
Hagen et al., “Review of Snapshot Spectral Imaging Technologies'.
Optical Engineering, Sep. 2013, vol. 52, No. 9, pp. 090901-1090901-23.

Hagen et al., “Snapshot Advantage: A Review of the Light Collec
tion Improvement for Parallel High-Dimensional Measurement
Systems.” Optical Engineering, Jun. 13, 2012, vol. 51, No. 11, p.
111702-1-111702-7.

Hagen et al., "Snapshot Mueller Matrix Spectropolarimeter' Optics
Letters, Aug. 1, 2007, vol. 32, No. 15, pp. 2100-2102.

Hagen et al., “Spectrally-Resolved Imaging of Dynamic TurbidMedia”. Multimodal Biomedical Imaging VI, Proceedings of SPIE
vol. 7892, 2011, pp. 1-7.
Hagen et al., “Video-Rate Spectral Imaging of Gas Leaks in the
Longwave Infrared,” Chemical, Biological, Radiological, Nuclear,and Explosives (CBRNE) Sensing XIV. May 29, 2013, SPIE
Proceedings vol. 8710, pp. 7.
Harley et al., “Remote Quantification of Smokestack Effluent Mass
Flow Rates Using Imaging Fourier Transform Spectrometry.”
Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE) Sensing XII, Apr. 25-29, 2011, SPIE Proceedings vol.
8018, pp. 1-13.
Hayden et al., “Determination of Trace-Gas Amounts in Plumes by
the Use of Orthogonal Digital Filtering of Thermal-Emission Spec
tra'. Applied Optics, Jun. 1, 1996, vol. 35. No. 16, pp. 2802-2809.
Hirsch et al., “Detection of Gaseous Plumes in IR Hyperspectral
Images Using Hierarchical Clustering”. Applied Optics, Sep. 1,
2007, vol. 46, No. 25, pp. 6368-6374.
Johnston et al., “A Real-Time FPGA Implementation of a BarrelDistortion Correction Aglorithm”. Projects, 2003, vol. 10, pp.
91-96.

Karion et al., “Methane Emissions Estimate from Airborne Mea
surements Over a Western United States Natural Gas Field', Geophysical Research Letters, 2013, vol. 40, pp. 4393-4397.
Keshava et al., “A Survey of Spectral Unmixing Algorithms'.
Lincoln Laboratory Journal, 2003, vol. 14, No. 1, pp. 55-78.
Kester et al., “A Real-Time Gas Cloud Imaging Camera for Fugitive
Emission Detection and Monitoring. Imaging and Applied Optics
Technical Digest, 2012, pp. 3.
Kester et al., “Development of Image Mappers for Hyperspectral
Biomedical Imaging Applications'. Applied Optics, Apr. 1, 2010,
vol. 49, No. 10, pp. 1886-1899.
Kester et al., “High Numerical Aperture Microendoscope Objective
for a Fiber Confocal Reflectance Microscope”. Optics Express, Mar.
5, 2007, vol. 15. No. 5, pp. 2409-2420.
Kester et al., “Low Cost, High Performance, Self-Aligning Minia
ture Optical Systems”. Applied Optics, Jun. 20, 2009, vol. 48, No.
18, pp. 3375-3384.
Kester et al., “Real-Time Snapshot Hyperspectral Imaging Endo
scope'. Journal of Biomedical Optics, May 2011, vol. 16, No. 5, pp.
O56005-1-05.6005-12.

Kudenov et al., “Fourier Transform Channeled Spectropolarimetry
in the MWIR”. Optics Express, Oct. 1, 2007, vol. 15, No. 20, pp.
12792-12805.

Kudenov et al., “Snapshot Imaging Mueller Matrix Polarimeter
Using Polarization Gratings'. Optics Letters, Apr. 15, 2012, vol. 37.
No. 8, pp. 1367-1369.
Landau et al., “Design and Evaluation of an Ultra-Slim Objective
for in-vivo Deep Optical Biopsy”. Optics Express, Mar. 1, 2010, vol.
18, No. 5, pp. 4758-4775.
Levi, Michael A., “Comment on Hydrocarbon Emissions Characterization in the Colorado Front Range: A Pilot Study by Gabrielle
Petron et al.”. Journal of Geophysical Research, 2012, vol. 117. No.
D21203, pp. 1-5.
Levi, Michael A., “Reply to “Reply to 'Comment on Hydrocarbon
Emissions Characterization in the Colorado Front Range—A Pilot
Study’ by Michael A. Levi' by Gabrielle Pétron et al., Journal ofGeophysical Research: Atmospheres, 2013, vol. 118, pp. 3044
3046.

Low et al., “Remote Sensing and Characterization of Stack Gases byInfrared Spectroscopy. An Approach by Using Multiple-Scan Inter
ferometry”. Environmental Science & Technology, Jan. 1967, vol.
1. No. 1, pp. 73-74.
Luo et al., “Fast Processing of Imaging Spectrometer Data Cube
Based on FPGA Design”, MIPPR 2007: Multispectral Image Pro
cessing, Proceedings of SPIE vol. 6787, pp. 7.
Manolakis et al., “Long-Wave Infrared Hyperspectral Remote Sens
ing of Chemical Clouds'. IEEE Signal Processing Magazine, Jul.
2014, vol. 31, No. 4, pp. 120-141.
Mathews, “Design and Fabrication of a Low-Cost, Multispectral
Imaging System.” Applied Optics, 2008, pp. F71-F76, vol. 47.

US 9,599,508 B2
Page 4(56)

References Cited
OTHER PUBLICATIONSNaranjo et al., “IR Gas Imaging in an Industrial Setting.”
Thermosense XXXII, Published in SPIE Proceedings vol. 7661,
May 4, 2010, pp. 1-8.
Nguyen et al., “Snapshot 3D Optical Coherence Tomography Sys
tem using Image Mapping Spectrometer'. Biomedical Optics and
3D Imaging OSA, 2012, pp. 3.
Niu et al., “New Approach to Remote Gas-Phase Chemical Quan
tification: Selected-Band Algorithm”. Optical Engineering, Feb.
2014, vol. 53, No. 2, pp. 021111-1-021111-10.
"Oil and Natural Gas Sector Leaks”. U.S. EPA Office of Air Quality
Planning and Standards (OAQPS), Review Panel, Apr. 2014, pp. 63.
Petron et al., “Hydrocarbon Emissions Characterization in the
Colorado Front Range: A Pilot Study”. Journal of GeophysicalResearch, 2012, vol. 117, No. D04304, pp. 1.-19.
Petron et al., “Reply to Comment on Hydrocarbon Emissions
Characterization in the Colorado Front Range—A Pilot Study by
Michael A. Levi'. Journal of Geophysical Research: Atmospheres,
2013, vol. 118, pp. 236-242.
Pisano et al., “Thermal Illuminators for Far-Infrared and Submil

limeter Astronomical Instruments.” Applied Optics, Jun. 1, 2005,
vol. 44, No. 16, pp. 3208-3217.
Polak et al., “Passive Fourier-Transform Infrared Spectroscopy ofChemical Plumes: An Algorithm for Quantitiative Interpretation
and Real-Time Background Removal”. Applied Optics, Aug. 20.
1995, vol. 34, No. 24, pp. 5406-5412.
Kester, Robert “Gas Cloud Imaging Camera: A Breakthrough inLeak Monitoring for the Rig & Refinery Safety Market'. Presen
tation at SPIE Defense, Security and Sensing Conference in 28
pages, May 6, 2013.
Sandsten et al., “Volume Flow Calculations on Gas Leaks Imaged
with Infrared Gas-Correlation.” Optics Express, 2012, vol. 20, No.
18, pp. 20318-20329.
Shogenji et al., “Multispectral Imaging Using Compact Compound
Optics,” Optics Express, Apr. 19, 2004, vol. 12, No. 8, pp. 1643
1655.

Telops,“Hyper-Cam',
http://web.archive.org/web/20160608180941/http://www.telops.com/en/hyperspectral-cam
eras/hyper-cam as archived Jun. 8, 2016 in 2 pages.
Telops, “Innovative Infrared Imaging'. http://web.archive.org/web/20160603212729/http://www.telops.com/en/ as archived Jun. 3.
2016 in 2 pages.
Walter Jr., et al., “Detection of Atmospheric Pollutants: a Correlation Technique'. Applied Optics, Jun. 1975, vol. 14, No. 6, pp.
1423-1428.

Young et al., “An In-Scene Method for Atmospheric Compensationof Thermal Hyperspectral Data”, Journal of Geophysical Research,
2002, vol. 107, No. D24, pp. 14-1-14-20.
Zheng et al., “A Static Multiplex Fabry-Perot Spectrometer'. Sen
Sors, Cameras, and Systems for Industrial Scientific Applications X.
Proceedings of SPIE-IS&T Electronic Imaging, SPIE vol. 7249,
2009, pp. 8.
Zheng et al., “Analytic-Domain Lens Design with Proximate Ray
Tracing”, Journal of the Optical Society of America A, Aug. 2010,
vol. 27, No. 8, pp. 1791-1802.
Preliminary Amendment as filed in U.S. Appl. No. 14/538,827 dated
Jan. 28, 2015 in 6 pages.
Office Action received in U.S. Appl. No. 14/538,827 dated Jun. 30.
2015 in 8 pages.
Non-Final Office Action Response as filed in U.S. Appl. No.
14/538,827 dated Dec. 28, 2015 in 11 pages.
Notice of Allowance received in U.S. Appl. No. 14/538,827 dated
Feb. 1, 2016 in 18 pages.
Corrected Notice of Allowance received in U.S. Appl. No.
14/538,827 dated Feb. 10, 2016 in 4 pages.

Corrected Notice of Allowance received in U.S. Appl. No.
14/538,827 dated Feb. 22, 2016 in 4 pages.
Request for Continued Examination and Response to Correct Application Papers as filed in U.S. Appl. No. 14/538,827 dated Apr. 29.
2016 in 14 pages.
Notice of Allowance received in U.S. Appl. No. 14/538,827 dated
May 26, 2016 in 9 pages.
Office Action received in U.S. Appl. No. 14/543,692 dated Nov. 3,2015 in 7 pages.
Interview Summary received in U.S. Appl. No. 14/543,692 dated
Feb. 17, 2016 in 5 pages.
Response to Office Action as filed in U.S. Appl. No. 14/543,692
dated May 2, 2016 in 9 pages.
Office Action received in U.S. Appl. No. 14/543,692 dated Jun. 1,
2016 in 18 pages.
Extended European Search Report received in European Application No. 141928.62.2 dated Mar. 30, 2015 in 10 pages.
International Search Report in PCT Application No. PCT/US2013/041278 dated Aug. 27, 2013 in 4 pages.
International Preliminary Report on Patentability in PCT Application No. PCT/US2013/041278 dated Nov. 27, 2014 in 10 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,700,791 dated
Jul. 13, 2015 in 8 pages.
Extended European Search Report received in European Application No. 15165877.0 dated Oct. 8, 2015 in 12 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,700,567 dated
Jul. 10, 2015 in 6 pages.
Extended European Search Report received in European Application No. EP 15165880.4 dated Nov. 24, 2015 in 8 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,792.477 dated
Dec. 21, 2015 in 7 pages.
Adams, et al., “Advances in Detectors: Hot IR sensors improve IR
camera size, weight, and power'. Laser Focus World, vol. 50, Issue
01, Jan. 17, 2014, 6 pages. Also available at http://www.ircameras.
com/articles/advances-detectors-hot-ir-sensors-improve-ir-camera
size-weight-power?.
Brochure provided by Lofty Designs to Rebellion Photonics on Oct.
31, 2012 as noted from the email. Subsequent to that date brochure
was used in connection with potential customers.
“Directed Inspection and Maintenance at Gas Processing Plants andBooster Stations.” United States Environmental Protection Agency
Air and Radiation (6202J), EPA430-B-03-018, Oct. 2003 available
at https://www3.epa.gov/gasstart documents/ll dimgasproc.pdf.
Official Communication received in European Application No.
141928.62.2 dated Apr. 19, 2016 in 6 pages.
Notice of Allowance received in U.S. Appl. No. 14,700,791 dated
Jun. 9, 2016 in 11 pages.
Publication Request as filed in U.S. Appl. No. 14,700,567 dated
Aug. 24, 2016 in 237 pages.
Preliminary Amendment as filed in U.S. Appl. No. 15/166,092 dated
Aug. 15, 2016 in 7 pages.
Notice of Allowance received in U.S. Appl. No. 14/538,827 dated
Jun. 19, 2016 in 9 pages.
Official Communication received in European Application No.
15165877.0 dated Jan. 3, 2017 in 9 pages.
Wikipedia entry https://en.wikipedia.org/wiki/Mobile computing
in 6 pages, last modified on Dec. 30, 2016; retrieved from theinternet on Feb. 2, 2017.

Notice of Allowance received in U.S. Appl. No. 14/543,692 dated
Dec. 9, 2016 in 12 pages.
International Search Report in PCT Application No. PCT/US2016/
034455 dated Oct. 24, 2016 in 12 pages.
Notice of Allowance received in U.S. Appl. No. 14,700,791 dated
Sep. 30, 2016 in 19 pages.
Response to Final Action as filed in U.S. Appl. No. 14/543,692
dated Nov. 30, 2016 in 12 pages.

* cited by examiner

U.S. PatentMar. 21, 2017

US 9,599,508 B2

Sheet 1 of 15?Spue/WO

OGIJOS 30od

U.S. PatentMar. 21, 2017

Sheet 2 of 15

ki: Xefe888. 204Y.

s

8-X Y.Y. 8x:s%&Sess'ss...:US 9,599,508 B2

:38,

a.
s

'XSSS

- Paraliax 228

processor 150

U.S. PatentMar. 21, 2017.

Field Reference:

338

Sheet 3 of 15

3.
8.

8:8: six{{ject, 110

FIG. 3B

8kx.

US 9,599,508 B2

U.S. Patent

US 9,599,508 B2

U.S. PatentMar. 21, 2017.

Sheet 5 of 15

US 9,599,508 B2

U.S. PatentMar. 21, 2017.

Sheet 6 of 15

US 9,599,508 B2

sex

es

'''

''

".

.

. .

.

. . . .

Exe

mm mm mm mm - - - - -

CD

t

y

g

ex

se

xt

C

O

e

U.S. PatentMar. 21, 2017

Sheet 7 Of 15-+ |issiusuela

US 9,599,508 B2£19"SDIU.S. PatentMar. 21, 2017.

Sheet 8 of 15

US 9,599,508 B2

r

e

c

ex

xt
iOSSS8.

r

U.S. PatentMar. 21, 2017.

Sheet 9 of 15

US 9,599,508 B2

r

x

s

grox

r

?a
aess

O

e5

9

s

x

o

OC

ge

x

t

st
Sed
E.

SS

r

LL

U.S. PatentMar. 21, 2017

Sheet 10 of 15

FIG. 7

US 9,599,508 B2

U.S. PatentMar. 21, 2017

Sheet 11 of 15

US 9,599,508 B2

4.

ob

sea 8

obj corr

3.

- ref

3.3.
.

:

2

:

s

8

30

2

c

;"

Wavelength (microns)

FIG. 8A

3

wer-rrrrrr.

obj corr

Wavelength (microns)

FIG. 8B

U.S. PatentMar. 21, 2017

Sheet 12 of 15

US 9,599,508 B2

900

to object
975a

960

y

928a

thermally
controllied
shutters
C8

book

FIG. 9A

U.S. PatentMar. 21, 2017

Sheet 13 of 15

US 9,599,508 B2905

to object

?

972a

960

y
928a

controlled K H.

H.

Shutters

M
x.

FIG. 9B

936

U.S. PatentMar. 21, 2017

Sheet 14 of 15

US 9,599,508 B2000

mirrorsreference
SOCCS

1000

F.G. 1 OB

U.S. PatentMar. 21, 2017

Sheet 15 Of 15

f

CD

E

?tU

f

SD

S.

c) C

St. U)

US 9,599,508 B2

US 9,599,508 B2
1.

2
least two or more optical channels defined by components of
the optical system, the at least two or more optical channels
being spatially and spectrally different from one another;and (ii) a processor or processing electronics configured to
CROSS-REFERENCE TO RELATED
acquire multispectral optical data representing said target
APPLICATIONS
chemical species from the received IR radiation in a single
The present application claims benefit under 35 U.S.C. occurrence of data acquisition (or Snapshot). The optical
FPA unit includes an array of photo-sensitive devices that
S119(e) of U.S. Provisional Application No. 61/903,075,
filed on Nov. 12, 2013 and titled “DIVIDED-APERTURE 10 are disposed at the focus of one or more lenses. In various
embodiments, the array of photo-sensitive devices can
INFRA-RED SPECTRAL IMAGING SYSTEM. The disclosure of the above identified provisional patent application include a two-dimensional imaging sensor that is sensitive toradiation having wavelengths between 1 um and 20 Lum (foris incorporated by reference herein in its entirety.
The present application is a continuation-in-part of Inter example, in mid infra-red wavelength range, long infra-red
national Application No. PCT/US/2013/041278, filed on wavelength range, visible wavelength range, etc.). In various
May 16, 2013 and titled “DIVIDED-APERTURE INFRA 15 embodiments, the array of photo-sensitive devices can
include CCD or CMOS sensors, bolometers or other detec
RED SPECTRAL IMAGING SYSTEM FOR CHEMICAL
tors that are sensitive to infra-red radiation. The optical
DETECTION, which claims benefit under 35 U.S.C. S 119(e) of U.S. Provisional Application No. 61/688,630, filed on system may include an optical aperture (a boundary of
May 18, 2012 and of U.S. Provisional Application No. which is defined to circumscribe or encompass the at least
two or more spatially distinct optical channels) and one or
61/764,776, filed on Feb. 14, 2013. The disclosure of eachof the above identified application is incorporated by refer more optical filters. In various implementations, the one ormore optical filters can comprise at least two spectrally
ence herein in its entirety.
multiplexed filters. Each of these optical filters can beassociated with one of the at least two or more optical
TECHNICAL FIELD
25 channels and configured to transmit a portion of the IR
The present invention generally relates to a system and radiation received in the associated optical channel. In
method for gas cloud detection and, in particular, to a system various embodiments, the one or more optical filters can be
and method of detecting spectral signatures of chemical spectrally multiplexed and may include, for example, at least
compositions in a mid- and long-wave infrared spectral one of a longpass optical filter and a shortpass optical filter,
30 or a band pass filter (with or without a combination with
region.
another filter such as a notch filter, for example). The optical
DESCRIPTION OF THE RELATED
system may further include at least two reimaging lenses.
TECHNOLOGY
The at least two reimaging lenses, for example each of thereimaging lens, may be disposed to transmit IR radiation (forSpectral imaging systems and methods have applications 35 example, between about 1 micron and about 20 microns),
in a variety of fields. Spectral imaging systems and methods that has been transmitted through a corresponding opticalobtain a spectral image of a scene in one or more regions of filter towards the optical FPA unit. In one embodiment, thethe electromagnetic spectrum to detect phenomena, identify optical FPA unit is positioned to receive IR radiation frommaterial compositions or characterize processes. The spec the object through the at least two reimaging lenses to form
tral image of the scene can be represented as a three 40 respectively-corresponding two or more sets of imaging data
dimensional data cube where two axes of the cube represent representing the object. The processor or processing elec
two spatial dimensions of the scene and a third axes of the tronics is configured to acquire this optical data from the twodata cube represents spectral information of the scene in or more sets of imaging data. In various embodiments of thedifferent wavelength regions. The data cube can be pro imaging systems, the FPA unit may be devoid of coolingcessed using mathematical methods to obtain information 45 systems. In various embodiments, two or more of the array
about the scene. Some of the existing spectral imaging of photo-sensitive devices may be uncooled. In somesystems generate the data cube by Scanning the scene in the embodiments, the system further comprises two or more
spatial domain (e.g., by moving a slit across the horizontal temperature-controlled shutters removably positioned todimensions of the scene) and/or spectral domain (e.g., by block IR radiation incident onto the optical system from thescanning a wavelength dispersive element to obtain images 50 object.
Also disclosed herein is an implementation of a method ofof the scene in different spectral regions). Such scanningapproaches acquire only a portion of the full data cube at a operating an infrared (IR) imaging system. The method
time. These portions of the full data cube are stored and then includes receiving IR radiation from an object through at
least two optical channels defined by components of anlater processed to generate a full data cube.
55 optical system of the IR imaging system, which at least two
SUMMARY
optical channels are spatially and spectrally different fromone another. The method further includes transmitting the
The systems, methods and devices of this disclosure each received IR radiation towards an optical focal plane arrayhave several innovative aspects, no single one of which is (FPA) unit that is not being cooled in the course of normal
solely responsible for the desirable attributes disclosed 60 operation. For example, in various embodiments of theherein.
imaging systems, the FPA unit may be devoid of cooling
Various embodiments of the systems described herein systems. In various embodiments, two or more of the arrayprovide an infrared (IR) imaging system for determining a of photo-sensitive devices may be uncooled. Some embodi
concentration of a target chemical species in an object (e.g., ments further comprise removably positioning at least one ofa gas plume). The imaging system includes (i) an optical 65 at least two temperature-controlled shutters in front of thesystem, having an optical focal plane array (FPA) unit optical system to block IR radiation incident onto the opticalconfigured to receive IR radiation from the object along at system from the object.
DIVIDED-APERTURE INFRA-RED
SPECTRAL MAGING SYSTEM

US 9,599,508 B2
4gate image plane of the plurality of cameras Such that theimage of the at least one reference source captured by theplurality of cameras is blurred.

3
Various innovative aspects of the subject matter describedin this disclosure can be implemented in the following
embodiments:
Embodiment 1

Embodiment 10

An infrared (IR) imaging system, the imaging system
The system of any of Embodiments 1-9, wherein the atcomprising:least one reference Source is positioned at a conjugate image
plane of the plurality of cameras.
a plurality of cameras;
at least one thermal reference source having a known 10
Embodiment 11
temperature placed in front of the plurality of cameras
and configured to be imaged by the plurality of cam
eras; andThe system of any of Embodiments 1-10, further com
a data-processing unit comprising a processor, the imag 15 prising a plurality of mirrors configured to image the at leasting system configured to:
one reference source onto the plurality of cameras.
acquire with the plurality of cameras a plurality of frames
Embodiment 12
having regions that correspond to the image of thereference Source; andThe system of any of Embodiments 1-11, wherein theplurality of mirrors are disposed outside a central field ofview of the plurality of cameras.

apply a dynamic calibration correction to the plurality ofcameras to allow every camera in the plurality ofcameras to be calibrated to agree with every othercamera in the plurality imaging the reference source.

Embodiment 13
Embodiment 2

25

The system of any of Embodiments 1-12, further com
prising a first and a second temperature-controlled shutter
removably positioned to block IR radiation incident on thesystem from reaching the plurality of cameras.

The system of Embodiment 1, wherein the plurality ofcameras comprises a FPA unit and a plurality of lenses.
Embodiment 3

The system of any of Embodiments 1-2, wherein the FPA
unit comprises one FPA or a plurality of FPAs.

30

Embodiment 14

Embodiment 4

The system of any of Embodiments 1-13, wherein thesystem includes at least two spatially and spectrally different
35 optical channels.

The system of any of Embodiments 1-3, wherein the atleast one thermal reference source has a known spectrum.

Embodiment 15

Embodiment 5

40

The system of any of Embodiments 1-4, further compris
ing an additional thermal reference source imaged by theplurality of cameras.
Embodiment 6

The system of any of Embodiments 1-14, wherein thesystem includes at least three optical channels.
Embodiment 16

45

The system of any of Embodiments 1-5, wherein the
additional reference source has a temperature and a spec
trum different from the known temperature and the known 50
spectrum of the at least one reference source.

The system of any of Embodiments 1-15, wherein thesystem includes at least four optical channels.
Embodiment 17

The system of any of Embodiments 1-16, wherein thesystem includes at least five optical channels.

Embodiment 7
Embodiment 18

The system of any of Embodiments 1-6, wherein thetemperature of the additional thermal reference source is less 55 The system of any of Embodiments 1-17, wherein thethan the known temperature.
system includes at least six optical channels.
Embodiment 8

The system of any of Embodiments 1-7, wherein thetemperature of the additional thermal reference source isgreater than the known temperature.

Embodiment 19
60

The system of any of Embodiments 1-18, wherein thesystem includes at least seven optical channels.
Embodiment 20

Embodiment 9
65

The system of any of Embodiments 1-8, wherein the atleast one reference source is displaced away from a conju

The system of any of Embodiments 1-19, wherein thesystem includes at least eight optical channels.

US 9,599,508 B2
5

6

Embodiment 21

Embodiment 30

The system of any of Embodiments 1-20, wherein thesystem includes at least nine optical channels.

The system of any of Embodiments 26-29, wherein thefirst temperature is greater than the second temperature.

Embodiment 22

Embodiment 31

system of any of Embodiments 26-30, wherein the
The system of any of Embodiments 1-21, wherein the firstThetemperatureis less than the second temperature.
system includes at least ten optical channels.
10
Embodiment 32

Embodiment 23

of any of Embodiments 26-31, wherein the
The system of any of Embodiments 1-22, wherein the firstTheandsystemthesecondreference sources are displaced awaysystem includes at least twelve optical channels.
15 from a conjugate image plane of the plurality of cameras
such that the image of the first and the second reference
Embodiment 24
sources captured by the plurality of cameras is blurred.
The system of any of Embodiments 1-23, further com
prising one or more sensors configured to measure a tem
perature of the at least one reference source.

Embodiment 33

The system of any of Embodiments 26-32, wherein thefirst and the second reference sources are positioned at a
conjugate image plane of the plurality of cameras.

Embodiment 25

The system of any of Embodiments 1-24, wherein theplurality of cameras is configured to image the same portion

25

Embodiment 34of the at least one reference source.

The system of any of Embodiments 26-33, further com
prising:
Embodiment 26
a first mirror configured to image the first reference onto30
the plurality of cameras; anda second mirror configured to image the second reference
An infrared (IR) imaging System, the imaging Systemcomprising:
Source onto the plurality of cameras.
a plurality of cameras;
Embodiment 35
a first temperature-controlled reference Source imaged by 35
the plurality of cameras;
The system of any of Embodiments 26-34, further com
a second temperature-controlled reference source imaged
prising a first and a second temperature-controlled shutter
by the plurality of cameras; andpositioned to block IR radiation incident on thea data-processing unit comprising a processor, said data 40 removably
systemfromreaching the plurality of cameras.
processing unit configured to:acquire with the plurality of cameras a plurality of frames
Embodiment 36
having regions that correspond to the image of thereference Source; andThe system of any of Embodiments 26-35, wherein the
dynamically calibrate the plurality of cameras so that 45 system
includes at least two spatially and spectrally different
various cameras imaging a scene are forced to agree on optical channels.
a temperature estimate of the first and second reference
SOUCS.

Embodiment 37
Embodiment 27

50

The imaging system of any of Embodiment 26, wherein
the data-processing unit is configured to calculate a dynamic
calibration correction and apply the correction to the plu
55
rality of cameras for each of the plurality of frames.
Embodiment 28

The system of any of Embodiments 26-36, wherein thesystem includes at least four optical channels.
Embodiment 38

The system of any of Embodiments 26-37, wherein thesystem includes at least six optical channels.
Embodiment 39

The system of any of Embodiments 26-27, wherein the 60
first reference source is maintained at a first temperature.

The system of any of Embodiments 26-38, wherein thesystem includes at least eight optical channels.

Embodiment 29
Embodiment 40

The system of any of Embodiments 26-28, wherein the 65
second reference source is maintained at a second tempera
ture.

The system of any of Embodiments 26-39, wherein thesystem includes at least ten optical channels.

US 9,599,508 B2
Embodiment 41

8removably positioned to block IR radiation incident on thesystem from reaching the plurality of cameras.

The system of any of Embodiments 26-40, wherein thesystem includes at least twelve optical channels.

Embodiment 49

7

Embodiment 42

The system of any of Embodiments 26-41, further com
prising one or more sensors configured to measure a tem 10perature of the first or the second reference source.

The system of any of Embodiments 44-48, wherein thesystem includes at least two spatially and spectrally different
optical channels.
Embodiment 50

The system of any of Embodiments 44-49, wherein thesystem includes at least three optical channels.

Embodiment 43

Embodiment 51
The system of any of Embodiments 26-42, wherein the 15
plurality of cameras is configured to image the same portion
The system of any of Embodiments 44-50, wherein theof the first reference source and wherein plurality of cameras
is configured to image the same portion of the second system includes at least four optical channels.
reference Source.

Embodiment 52
Embodiment 44

The system of any of Embodiments 44-51, wherein thesystem includes at least five optical channels.

An infrared (IR) imaging system, the imaging systemcomprising:
Embodiment 53
25
a plurality of cameras;
The system of any of Embodiments 44-52, wherein thea reference having an unknown temperature configured tosystem includes at least six optical channels.
be imaged by the plurality of cameras; anda data-processing unit comprising a processor, the imag
Embodiment 54
30
ing system configured to:acquire with the plurality of cameras a plurality of frames
The system of any of Embodiments 44-52, wherein thehaving regions that correspond to the image of the system includes at least seven optical channels.
reference;

Embodiment 55
calculate a dynamic calibration correction using a tem 35
perature measured by one of the cameras in the plu
The system of any of Embodiments 44-53, wherein the
rality of cameras as a reference temperature; andsystemincludes at least eight optical channels.
apply the calibration correction to the other cameras in theplurality of cameras to match the temperature estimate
Embodiment 56of the other cameras in the plurality of cameras with the 40
reference temperature.
The system of any of Embodiments 44-54, wherein thesystem includes at least nine optical channels.
Embodiment 45

Embodiment 57

The system of any of Embodiment 44, wherein the 45
The system of any of Embodiments 44-55, wherein thereference is displaced away from a conjugate image plane ofthe plurality of cameras such that the image of the reference system includes at least ten optical channels.
source captured by the plurality of cameras is blurred.
Embodiment 58

Embodiment 46

50

The system of any of Embodiments 44-45, wherein thereference is positioned at a conjugate image plane of the55
plurality of cameras.
Embodiment 47

The system of any of Embodiments 44-56, wherein thesystem includes at least twelve optical channels.
Embodiment 59

The system of any of Embodiments 44-57, wherein theplurality of cameras is configured to image the same portionof the reference.

The system of any of Embodiments 44-46, further com
prising a plurality of mirrors configured to image the refer 60
ence onto the plurality of cameras.

Embodiment 60

An infrared (IR) imaging system, the imaging systemcomprising:
Embodiment 48
an optical system including an optical focal plane array
65(FPA) unit, the optical system includes components
The system of any of Embodiments 44-47, further com
associated with at least two optical channels, said atleast two optical channels being spatially and spectrally
prising a first and a second temperature-controlled shutterUS 9,599,508 B2
10
Embodiment 68

different from one another, each of the at least two

optical channels positioned to transfer IR radiation
incident on the optical system towards the optical FPA
unit, the optical FPA unit comprising at least two
detector arrays disposed at a distance from two corre
sponding focusing lenses;
at least one thermal reference having a known tempera

The system of any of Embodiments 60-67, further com
prising at least two reflecting elements configured to direct
radiation from the at least one reference source toward theat least two detector arrays.
Embodiment 69

ture, wherein radiation emitted from the at least one

reference is directed towards the optical FPA unit and 10
The system of any of Embodiments 60-68, wherein the atimaged by the at least two detector arrays; anda data-processing unit, said data-processing unit config least two reflecting elements are disposed outside a central
ured to:field of view of the at least two detector arrays.
acquire a plurality of frames with the at least two detector
Embodiment 70
arrays having regions in the plurality of image frames 15
that correspond to the image of the reference; andThe system of any of Embodiments 60-69, further com
dynamically calibrate the at least two detector arrays toaddress a difference in the temperature estimate of the prising a third detector array disposed between the at least
two detector arrays.
reference between the two detector arrays.
Embodiment 61

Embodiment 71

The system of any of Embodiment 60, wherein the at least
one thermal reference has a known spectrum.

The system of any of Embodiments 60-70, wherein thedata-processing unit is configured to:acquire a plurality of frames using the third detector array;25
andEmbodiment 62

The system of any of Embodiments 60-61, further com
prising an additional thermal reference, wherein radiationfrom the additional reference is directed towards the optical 30
FPA unit and imaged by the at least two detector arrays.

dynamically calibrate the third detector array to match atemperature estimate of the third detector array with thetemperature estimates of the at least two detector
arrays.

Embodiment 72

Embodiment 63

The system of any of Embodiments 60-71, wherein radia
The system of any of Embodiments 60-62, wherein the 35 tion emitted from the at least one reference source is not
additional reference has a temperature and a spectrum imaged by the third detector array.
different from the known temperature and the known spec
Embodiment 73

trum of the at least one reference source.
Embodiment 64

40

The system of any of Embodiments 60-63, wherein thetemperature of the additional thermal reference is less than
the known temperature.
45

The system of any of Embodiments 60-72, wherein thethird detector array has a field of view greater than a field ofview of the at least two detector arrays.
Embodiment 74

Embodiment 65

The system of any of Embodiments 60-73, wherein the atleast one reference is imaged by the third detector array.

The system of any of Embodiments 60-64, wherein thetemperature of the additional thermal reference is greater 50than the known temperature.

Embodiment 75

Embodiment 66

The system of any of Embodiments 60-74, further com
prising a third reflecting element disposed outside the fieldof view of the at least two detector arrays and configured toimage the at least one reference onto the third detector array.

The system of any of Embodiments 60-65, wherein the at 55
least one reference is displaced away from a conjugate
Embodiment 76
image plane of the at least two detector arrays such that theimage of the at least one reference captured by the at least
The system of any of Embodiments 60-75, further com
two detector arrays is defocused.
prising a first and a second temperature-controlled element
60 removably positioned to block IR radiation incident on the
Embodiment 67
optical system from reaching the optical FPA unit.
Embodiment 77
The system of any of Embodiments 60-66, wherein the atleast one reference is positioned at a conjugate image planeof the at least two detector arrays such that the image of the 65 The system of any of Embodiments 60-76, wherein theat least one reference captured by the at least two detector optical system includes components associated with three
optical channels.
arrays is focused.

US 9,599,508 B2
11
Embodiment 78

The system of any of Embodiments 60-77 wherein the
optical system includes components associated with four5
optical channels.
Embodiment 79

The system of any of Embodiments 60-78, wherein the
optical system includes components associated with six 10
optical channels.
Embodiment 80

The system of any of Embodiments 60-79, wherein the 15
optical system includes components associated with eight
optical channels.
Embodiment 81

12
an optical system including an optical focal plane array(FPA) unit, the optical system including at least two
optical channels, said at least two optical channels
being spatially and spectrally different from one
another, each of the at least two optical channels
positioned to transfer IR radiation incident on the
optical system towards the optical FPA unit, the optical
FPA unit comprising at least two detector arrays dis
posed at a distance from two corresponding focusing
lenses;a first temperature-controlled reference imaged by the atleast two detector arrays;a second temperature-controlled reference imaged by theat least two detector arrays; anda data-processing unit configured to:acquire a plurality of frames with the at least two detector
arrays having regions in the plurality of image framesthat correspond to the image of the first and second
references; anddynamically calibrate the at least two detector arrays toaddress a difference in a temperature estimate of the

The system of any of Embodiments 60-80, wherein the
optical system includes components associated with ten
optical channels.
Embodiment 82

first and second references between the two detector
arrays.
25

The system of any of Embodiments 60-81, wherein the
optical system includes components associated with twelve
optical channels.
Embodiment 83

Embodiment 90

The system of Embodiment 89, wherein the first reference
is maintained at a first temperature.
30

Embodiment 91

The system of any of Embodiments 89-90, wherein the
The system of any of Embodiments 60-82, wherein the
optical system includes components associated with sixteen second reference is maintained at a second temperature.
optical channels.
35
Embodiment 92

Embodiment 84

The system of any of Embodiments 89-91, wherein the
The system of any of Embodiments 60-83, wherein the first temperature is greater than the second temperature.
optical system includes components associated with twenty 40
Embodiment 93
four optical channels.
Embodiment 85
The system of any of Embodiments 89-92, wherein thefirst temperature is less than the second temperature.
The system of any of Embodiments 60-84, wherein each
Embodiment 94
of the at least two detector arrays is configured to image the 45
same portion of the at least one reference so as to consis
The system of any of Embodiments 89-93, wherein thetently provide a common reference temperature.
first and the second references are displaced away from a
conjugate image plane of the at least two detector arrays
Embodiment 86
50such that the image of the first and the second references
The system of any of Embodiments 60-85, wherein the captured by the at least two detector arrays is defocused.
data-processing unit comprises processing electronics.
Embodiment 95

Embodiment 87
55

The system of any of Embodiments 60-86, wherein thedata-processing unit comprises a processor.
Embodiment 88

The system of any of Embodiments 89-94, wherein thefirst and the second references are positioned at a conjugate
image plane of the at least two detector arrays Such that theimage of the first and the second references captured by theat least two detector arrays is focused.

60

Embodiment 96

The system of any of Embodiments 60-87, wherein thedata-processing unit comprises one or more processors.
Embodiment 89

An infrared (IR) imaging system, the imaging systemcomprising:

The system of any of Embodiments 89-95, further com
prising:
65
a first reflecting element configured to direct radiationfrom the first reference toward the at least two detector

arrays; andUS 9,599,508 B2
14
removably positioned to block IR radiation incident on the
optical system from reaching the optical FPA unit.

13
a second reflecting element configured to direct radiationfrom the second reference toward the at least two

detector arrays.
Embodiment 106
Embodiment 97

The system of any of Embodiments 89-96, wherein thefirst reflecting element is disposed outside a field of view of
the at least two detector arrays.
10

The system of any of Embodiments 89-105, wherein the
optical system includes components associated with three
optical channels.
Embodiment 107

Embodiment 98

The system of any of Embodiments 89-106, wherein the
optical system includes components associated with four
The system of any of Embodiments 89-97, wherein the optical channels.
second reflecting element is disposed outside a field of view 15of the at least two detector arrays.
Embodiment 108
Embodiment 99

The system of any of Embodiments 89-107, wherein the
optical system includes components associated with five
optical channels.

The system of any of Embodiments 89-98, further com
prising a third detector array disposed between the at least
two detector arrays.
Embodiment 100

Embodiment 109
25

The system of any of Embodiments 89-99, wherein thedata-processing unit is configured to:acquire a plurality of frames using the third detector array:

The system of any of Embodiments 89-108, wherein the
optical system includes components associated with six
optical channels.
Embodiment 110

30

and
The system of any of Embodiments 89-109, wherein the
dynamically calibrate the third detector array to address a optical system includes components associated with sevendifference in the temperature estimate of the first and optical channels.
second references between the third detector array and
Embodiment 111
the two detector arrays.
35
Embodiment 101

The system of any of Embodiments 89-100, wherein thefirst and second references are not imaged by the third 40
detector array.
Embodiment 102

The system of any of Embodiments 89-101, wherein thethird detector array has a field of view greater than a field ofview of the at least two detector arrays.

The system of any of Embodiments 89-110, wherein the
optical system includes components associated with eight
optical channels.
Embodiment 112

The system of any of Embodiments 89-111, wherein the
optical system includes components associated with ten
optical channels.
45
Embodiment 113

Embodiment 103

The system of any of Embodiments 89-112, wherein the
optical system includes components associated with twelve
50 optical channels.

The system of any of Embodiments 89-102, wherein r firstand second references are imaged by the third detector array.

Embodiment 114

The system of any of Embodiments 89-113, wherein eachof the at least two detector arrays is configured to image thesame portion of the first reference source so as to consis
The system of any of Embodiments 89-103, further com tently provide a common first reference temperature andprising a third reflecting element disposed outside the field wherein each of the at least two detector arrays is configuredof view of the at least two detector arrays and configured to 60 to image the same portion of the second reference Source soimage the first and second references onto the third detector as to consistently provide a common second reference
Embodiment 104

55

temperature.

array.

Embodiment 115
Embodiment 105
65

The system of any of Embodiments 89-104, further com
prising a first and a second temperature-controlled element

The system of any of Embodiments 89-114, further com
prising a temperature controller configured to control thetemperature of the first or second reference.

US 9,599,508 B2
15

16
the calibration surface of the cameras in the plurality ofcameras agree with each other.

Embodiment 116

The system of any of Embodiments 89-115, further com
prising one or more sensors configured to measure a tem
perature of the first or the second reference.

Embodiment 125

Embodiment 117

The system of Embodiment 124, wherein the one or moreparameters is associated with a gain of the cameras in theplurality of cameras.

The system of any of Embodiments 89-116, wherein the 10
one or more sensors are configured to communicate themeasured temperature of the first or the second reference toa temperature controller.
Embodiment 118

Embodiment 126

The system of any of Embodiments 124-125, wherein theone or more parameters is associated with again offset of thecameras in the plurality of cameras.
15
Embodiment 127

The system of any of Embodiments 89-117, wherein theone or more sensors are configured to communicate themeasured temperature of the first or the second reference tothe data-processing unit.

The system of any of Embodiments 124-126, wherein thecalibration Surface is displaced away from a conjugate
image plane of the plurality of cameras Such that an imageof the surface is defocused.

Embodiment 119

Embodiment 128

The system of any of Embodiments 89-118, wherein the 25

first or the second reference is associated with a heater.

The system of any of Embodiments 124-127, wherein thecalibration Surface is positioned at a conjugate image planeof the plurality of cameras Such that an image of the Surface

Embodiment 120

is focused.

The system of any of Embodiments 89-119, wherein the 30
first or the second reference is associated with a cooler.

Embodiment 129

The system of any of Embodiments 124-128, further
comprising at least one reflecting element configured to35
The system of any of Embodiments 89-120, wherein the image the Surface onto the plurality of cameras.
data-processing unit comprises processing electronics.
Embodiment 130
Embodiment 121

Embodiment 122
40

The system of any of Embodiments 89-121, wherein thedata-processing unit comprises a processor.
Embodiment 123

The system of any of Embodiments 124-129, further
comprising a first and a second temperature-controlled ele
ment removably positioned to block IR radiation incident on
the optical system from reaching the plurality of cameras.
Embodiment 131

45

The system of any of Embodiments 124-130, wherein the
optical system includes components associated with three
optical channels.

The system of any of Embodiments 89-122, wherein thedata-processing unit comprises one or more processors.
Embodiment 124

50

Embodiment 132

An infrared (IR) imaging system, the imaging system
The system of any of Embodiments 124-131, wherein thecomprising:
opticalsystem includes components associated with fouran optical system including components associated with optical channels.
55
at least two optical channels, said at least two opticalchannels being spectrally different from one another,
Embodiment 133
each of the at least two optical channels positioned totransfer IR radiation incident on the optical system
The system of any of Embodiments 124-132, wherein the
towards a plurality of cameras;
60 optical system includes components associated with six
at least one calibration Surface with unknown temperature optical channels.
and imaged by each of the at least two detectors; anda data-processing unit configured to:
Embodiment 134
acquire a plurality of image frames with the plurality ofcameras including the imaged Surface; and65
The system of any of Embodiments 124-133, wherein theadjust one or more parameters of the cameras in the optical system includes components associated with eightplurality of cameras Such that a temperature estimate of optical channels.

US 9,599,508 B2
17

18

Embodiment 135

Embodiment 145

The system of any of Embodiments 141-144, wherein a
The system of any of Embodiments 124-134, wherein the
optical system includes components associated with ten number of optical filters is two.
5
optical channels.
Embodiment 146

Embodiment 136

The system of any of Embodiments 141-145, wherein a
The system of any of Embodiments 124-135, wherein the 10 number of optical filters is three.
optical system includes components associated with twelve
Embodiment 147
optical channels.
The system of any of Embodiments 141-146, wherein a
Embodiment 137
number of optical filters is four.
The system of any of Embodiments 124-136, wherein the 15
Embodiment 148
data-processing unit comprises processing electronics.
The system of any of Embodiments 141-147, wherein the
Embodiment 138
plurality of optical filters comprise at least one long pass(LP) filter.
The system of any of Embodiments 124-137, wherein thedata-processing unit comprises a processor.
Embodiment 149
Embodiment 139

The system of any of Embodiments 124-138, wherein thedata-processing unit comprises one or more processors.

25

The system of any of Embodiments 141-148, wherein theplurality of optical filters comprise multiple long pass (LP)filters.
Embodiment 150

Embodiment 140

The system of any of Embodiments 141-149, wherein the
The system of any of Embodiments 124-139, wherein the 30 plurality of optical filters comprise at least one short passcalibration Surface comprises a sidewall of a housing of the (SP) filter.
system.

Embodiment 151
Embodiment 141
35

The system of any of Embodiments 141-150, wherein the
An infrared (IR) imaging system, the imaging system plurality of optical filters comprise multiple short pass (SP)comprising:filters.
at least four spatially and spectrally different optical
Embodiment 152
channels configured to receive IR radiation from a 40
common object, each of the at least four spatially andspectrally different optical channels comprising at least
The system of any of Embodiments 141-151, wherein the
one imaging lens configured to image the object on a plurality of optical filters comprise at least one band pass
Focal Plane Array (FPA) unit; and(BP) filter.
processing electronics in communication with the FPA 45
Embodiment 151

unit,

wherein said infrared system is configured to:acquire multispectral optical data from the at least four
The system of any of Embodiments 141-152, wherein theplurality of optical filters comprise at one short pass (SP)different optical channels; andprocess the multispectral optical data to detect one or 50 filter and one long pass (LP) filter.
more target species present in the object.
Embodiment 154
Embodiment 142

The system of any of Embodiments 141-153, wherein the
The system of Embodiment 141, further comprising FPA unit comprises a plurality of FPAs.
55
twelve optical channels.
Embodiment 155

Embodiment 143

The system of any of Embodiments 141-154, further
The system of any of Embodiments 141-142, configured comprising first and second temperature-controlled elements
to simultaneously acquire multispectral optical data from the 60 removably positioned to block IR radiation incident on theimaging system from reaching the FPA unit.
at least four different optical channels.
Embodiment 144

Embodiment 156

The system of any of Embodiments 141-143, further 65 The system of any of Embodiments 141-155, further
comprising a plurality of optical filters associated with the comprising a field reference configured for dynamicallycalibrating a plurality of the FPAS in the FPA unit.
optical channels.

US 9,599,508 B2
19

20

Embodiment 157

Embodiment 167

The system of any of Embodiments 141-166, further
The system of any of Embodiments 141-156, wherein thefield reference is configured to obscure a peripheral region comprising five optical channels.
of an image generated by a plurality of the FPAS in the FPA 5
Embodiment 168

unit.
Embodiment 158

The system of any of Embodiments 141-167, further
comprising six optical channels.

The system of any of Embodiments 141-157, configured 10
Embodiment 169
to compare spectral data in at least one of the four opticalchannels acquired at a first instant of time with spectral datain the at least one of the four optical channels acquired at a
The system of any of Embodiments 141-168, further
second instant of time to generate a temporal difference 15 comprising seven optical channels.
image.
Embodiment 170
Embodiment 159

The system of any of Embodiments 141-169, further
comprising eight optical channels.

The system of any of Embodiments 141-158, configured
to use a difference between the multispectral optical dataacquired by the two optical channels to correct parallax
induced imaging errors.
Embodiment 160

Embodiment 171

25

The system of any of Embodiments 141-170, further
comprising nine optical channels.

The system of any of Embodiments 141-159, configured
Embodiment 172
to use a difference between the multispectral optical dataacquired by the two optical channels to estimate a distance
The system of any of Embodiments 141-171, further
between the system and the object.
30 comprising ten optical channels.
Embodiment 161

Embodiment 173

The system of any of Embodiments 141-160, configured
to estimate a size of the object based on the estimated 35 The system of any of Embodiments 141-172, further
distance and an optical magnification factor of the two comprising eleven optical channels.
optical channels.
Embodiment 174
Embodiment 162

The system of any of Embodiments 141-173, wherein a
The system of any of Embodiments 141-161, configured 40 number of optical filters is five.
to compare spectral data in one of the at least four opticalchannels with spectral data in another one of the at least four
Embodiment 175
optical channels to generate a spectral difference image.
The system of any of Embodiments 141-174, wherein a
45
Embodiment 163
number of optical filters is six.
The system of any of Embodiments 141-162, further
Embodiment 176
comprising a visible light imaging sensor.
50
Embodiment 164
The system of any of Embodiments 141-175, wherein anumber of optical filters is seven.
The system of any of Embodiments 141-163, configured
Embodiment 177
to use the visible light imaging sensor to compensate formotion-induced imaging errors.
55
The system of any of Embodiments 141-176, wherein a
Embodiment 165
number of optical filters is eight.
The system of any of Embodiments 141-164, configured
Embodiment 178
to process the multispectral optical data by cross-correlating
60
multispectral optical data from at least one of the optical
The system of any of Embodiments 141-177, wherein achannels with a reference spectrum.
number of optical filters is nine.
Embodiment 166
Embodiment 179

The system of any of Embodiments 141-165, configured 65
to process the multispectral optical data using spectral
The system of any of Embodiments 141-178, wherein aunmixing.
number of optical filters is ten.

US 9,599,508 B2
21

22
portion of the thermal reference so as to consistently provide
a common reference temperature.

Embodiment 180

The system of any of Embodiments 141-179, wherein anumber of optical filters is eleven.

Embodiment 193

Embodiment 181

The system of any of Embodiments 141-192, wherein atemperature of the same portion of the thermal reference is

The system of any of Embodiments 141-180, wherein anumber of optical filters is twelve.
Embodiment 182

unknown.
10

The system of any of Embodiments 141-193, configured
to acquire the multispectral optical data at a frame rate

The system of any of Embodiments 141-181, wherein theprocessing electronics comprises one or more processors.
Embodiment 183

Embodiment 194

between about 5 Hz to about 200 HZ.
15

Embodiment 195

The system of any of Embodiments 141-182, further
comprising a thermal reference configured to be imaged
onto the FPA unit such that a plurality of frames of theacquired multispectral optical data has an image of the

The system of any of Embodiments 141-194, wherein one
or more of the at least four optical channels is configured tocollect IR radiation to provide spectral data corresponding toa discrete spectral band located in the wavelength range
between about 7.9 um and about 8.4 um.

thermal reference source.
Embodiment 184

The system of any of Embodiments 141-183, wherein the 25
thermal reference has a known temperature.

Embodiment 196

The system of any of Embodiments 141-195, whereineach optical channel is configured to transfer spectrally
distinct, two-dimensional image data of the common object
The system of any of Embodiments 141-184, wherein the 30 to one or more imaging sensors.
thermal reference is a temperature-controlled reference
Embodiment 196
Embodiment 185

SOUCC

Embodiment 186

The system of any of Embodiments 141-185, wherein thetemperature-controlled reference source includes a heater.

35

Embodiment 197

Embodiment 187

The system of any of Embodiments 141-186, wherein thetemperature-controlled reference source includes a cooler.

40

Embodiment 188

The system of any of Embodiments 141-187, further 45
comprising a mirror configured to image the thermal reference onto the FPA unit.
Embodiment 189

The system of any of Embodiments 1-24, wherein the one
or more sensors are configured to communicate the mea
Sured temperature of the at least one reference source to atemperature controller.
The system of any of Embodiments 1-24, wherein the one
or more sensors are configured to communicate the mea
sured temperature of the at least one reference source to thedata-processing unit.
Embodiment 198

The system of any of Embodiments 26-42, wherein theone or more sensors are configured to communicate mea
sured temperature of the first or the second reference to a
50 temperature controller.

The system of any of Embodiments 141-188, wherein atemperature of the thermal reference is unknown.

Embodiment 199

Embodiment 190
55

The system of any of Embodiments 141-189, wherein thethermal reference is a Surface.

The system of any of Embodiments 26-42, wherein theone or more sensors are configured communicate the mea
sured temperature of the first or the second reference to thedata-processing unit.
Embodiment 200

Embodiment 191
60

An infrared (IR) imaging system for imaging a target
species in an object, the imaging system comprising:an optical system comprising an optical focal plane array
Embodiment 192(FPA) unit and defining a plurality of spatially and65
spectrally different optical channels to transfer IR
radiation towards the optical FPA unit, each optical
The system of any of Embodiments 141-191, whereinchannel positioned to transfer a portion of the IR
different optical channels receive IR radiation from the same
The system of any of Embodiments 141-190, wherein the
Surface comprises a wall of a housing of the system.

US 9,599,508 B2
23
radiation incident on the optical system from the object
towards the optical FPA unit; anda programmable processor configured to execute instructions stored in a tangible, non-transitory computerreadable storage medium, to acquire, in a single occur
rence of data acquisition, multispectral optical data
representing in spatial (x, y) and spectral (W) coordi
nates said object and said target species from the IRradiation received at the optical FPA unit.

24
reference source removably positioned to block IR radiation
incident onto the optical system from reaching the optical
FPA unit.
Embodiment 210

The system of any of Embodiments 200-209, wherein themultispectral optical data from the plurality of optical channels is captured Substantially simultaneously by the optical
10

FPA unit.

Embodiment 201
Embodiment 211

The system of Embodiment 200, wherein the multispec
tral data comprises a number of spectrally different images 15 The system of any of Embodiments 200-210, wherein theof the object obtained from IR image data transferred to the multispectral optical data from the plurality of optical channels is captured during one image frame acquisition by the
optical EPA unit by a corresponding optical channel.
optical FPA unit.
Embodiment 202

Embodiment 212

The system of any of Embodiments 200-201, further
comprising an optical filter corresponding to a particular
optical channel and configured to transmit the portion of IR
radiation towards the optical FPA unit.

The system of any of Embodiments 200-211, further
comprising first and second temperature-controlled moveable shutters removably positioned to block IR radiation
25 incident onto the optical system from reaching the optical
FPA unit.

Embodiment 203
Embodiment 213

The system of any of Embodiments 200-202, wherein the
optical filter includes one of a longpass optical filter and a 30 The system of any of Embodiments 200-212, wherein the
optical FPA unit is devoid of a cooling device.
shortpass optical filter.
Embodiment 214

Embodiment 204

The system of any of Embodiments 200-213, further
The system of any of Embodiments 200-203, further 35 comprising a filter array.
comprising one or more front objective lenses.
Embodiment 215
Embodiment 205

The system of any of Embodiments 200-204, wherein the
optical system comprises a plurality of lenses, each lenscorresponding to an optical channel.
Embodiment 206

40

45

The system of any of Embodiments 200-214, wherein theprocessor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium
to acquire said optical data from the two or more sets ofimaging data.
Embodiment 216

The system of any of Embodiments 200-215, wherein theprocessor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium
50 to process the acquired optical data to compensate for atleast one of (i) parallax-induced differences between the two
Embodiment 207
or more sets of imaging data and (ii) difference between thetwo or more sets of imaging data induced by changes in the
The system of any of Embodiments 200-206, wherein the object that are not associated with the target species.
plurality of lenses comprises a lens array.
The system of any of Embodiments 200-205, whereineach optical channel is defined at least in part by a corre
sponding filter and a corresponding lens.

55

Embodiment 217
Embodiment 208

The system of any of Embodiments 200-216, wherein the
The system of any of Embodiments 200-207, further 60 processor is configured to execute instructions stored in acomprising a plurality of relay lenses configured to relay the tangible, non-transitory computer-readable storage medium
to process the acquired optical data to generate a temporal
IR radiation along the optical channels.
reference image.
Embodiment 209

Embodiment 218
65

The system of any of Embodiments 200-208, further
comprising a plurality of moveable temperature-controlled

The system of any of Embodiments 200-217, wherein theprocessor is configured to execute instructions stored in a

US 9,599,508 B2
25
tangible, non-transitory computer-readable storage medium
to use the temporal reference image to generate a temporal
difference image.

26
tangible, non-transitory computer-readable storage medium
to process data received from the visible light imaging
sensor to compensate for motion-induced imaging errors.

Embodiment 219

Embodiment 229

The system of any of Embodiments 200-218, wherein the
The system of any of Embodiments 200-228, wherein theprocessor is configured to execute instructions stored in a processor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium tangible, non-transitory computer-readable storage medium
to process the acquired optical data to estimate a volume of 10 to construct, in the single occurrence of data acquisition, aa gas cloud.
multispectral data cube of the object, the multispectral data
cube comprising a number of spectrally different images ofEmbodiment 220
the object, each spectrally different image comprising IR
The system of any of Embodiments 200-219, wherein IR 15 image data transferred to the optical FPA unit by a corre
radiation measured at a pixel comprises a spectrum com sponding optical channel.
prising a sum of component spectra, and wherein the pro
Embodiment 230
cessor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium
The system of any of Embodiments 200-229, wherein theto unmix the spectrum.
portion of the IR radiation corresponds to a region of
wavelengths of the spectrum of wavelengths, the region of
Embodiment 221
wavelengths at least partially overlapping another region of
The system of any of Embodiments 200-220, wherein the wavelengths transferred by another optical channel.
optical HA unit includes a bolometer configured to operate 25
Embodiment 231
without being cooled.
An infrared (IR) imaging system, the imaging system
Embodiment 222
comprising:of spatially and spectrally different optical
The system of any of Embodiments 200-221, further 30 a plurality
channels,
at least some of the plurality of opticalcomprising a field reference for dynamically adjusting datachannelsconfigured
to receive IR radiation from a
output from the optical FPA unit.
common object, each of the plurality of spatially andspectrally different optical channels comprising at least
Embodiment 223
one imaging lens configured to image the object on a
35
Focal Plane Array (FPA) unit; andThe system of any of Embodiments 200-222, wherein theprocessing electronics in communication with the FPA
field reference comprises an array of field stops.
unit,
wherein said infrared system is configured to:
Embodiment 224
acquire multispectral optical data from the plurality of40
different optical channels; andThe system of any of Embodiments 200-223, wherein theprocess the multispectral optical data to detect one or
field reference comprises a uniform temperature across its
more target species present in the object, andSurface.
wherein the system is further configured to compare
spectral data in at least one of the plurality of optical
Embodiment 225
45
channels acquired at a first instant of time with spectral
data in at least one of the plurality of optical channels
The system of any of Embodiments 200-224, wherein theacquired at a second instant of time to generate afield reference is adapted to obscure or block a peripheral
temporal difference image.
portion of the IR radiation propagating from the object
towards the optical FPA unit.
50
Embodiment 232
Embodiment 226

An infrared (IR) imaging system, the imaging systemcomprising:a plurality of spatially and spectrally different optical
55
channels, at least some of the plurality of optical
Embodiment 227
channels configured to receive IR radiation from acommon object, each of the plurality of spatially andspectrally different optical channels comprising at least
The system of any of Embodiments 200-226, wherein the
one imaging lens configured to image the object on aprocessor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium 60
Focal Plane Array (FPA) unit; andprocessing electronics in communication with the FPA
to process data received from an imaging sensor to com
unit,
pensate for motion-induced imaging errors.
wherein said infrared system is configured to:
Embodiment 228
acquire multispectral optical data from the plurality of65
different optical channels; andprocess the multispectral optical data to detect one or
The system of any of Embodiments 200-227, wherein the
more target species present in the object, andprocessor is configured to execute instructions stored in a
The system of any of Embodiments 200-225, further
comprising a visible light imaging sensor.

US 9,599,508 B2
27
wherein the system is configured to use a difference
between the multispectral optical data acquired by two
optical channels to correct parallax-induced imaging
COS.

Embodiment 232

The system of Embodiment 231, wherein spectral characteristics of the two optical channels are identical.
10
Embodiment 233

28
optical channels comprising at least one imaging lensconfigured to image the object on a Focal Plane Array(FPA) unit; andprocessing electronics in communication with the FPA
unit,

wherein said infrared system is configured to:acquire multispectral optical data from the plurality ofdifferent optical channels; andprocess the multispectral optical data to detect one ormore target species present in the object, andwherein the system is configured to compensate formotion-induced imaging errors.

An infrared (IR) imaging system, the imaging systemcomprising:
Embodiment 237
a plurality of spatially and spectrally different optical 15
channels, at least some of the plurality of optical
An infrared (IR) imaging system, the imaging systemchannels configured to receive IR radiation from a comprising:common object, each of the plurality of spatially anda plurality of spatially and spectrally different, at least
spectrally different optical channels comprising at least
Some of the plurality of optical channels configured to
one imaging lens configured to image the object on areceive IR radiation from a common object, each of the
Focal Plane Array (FPA) unit; andplurality of spatially and spectrally different opticalprocessing electronics in communication with the FPA
channels comprising at least one imaging lens config
unit,
ured to image the object on a Focal Plane Array (FPA)
wherein said infrared system is configured to:25unit; andacquire multispectral optical data from the plurality ofprocessing electronics in communication with the FPA
different optical channels; andunit,
process the multispectral optical data to detect one orwherein said infrared system is configured to:more target species present in the object, andacquire multispectral optical data from the plurality ofwherein the system is configured to use a difference 30
different optical channels; andbetween the multispectral optical data acquired by two
process the multispectral optical data to detect one or
optical channels to estimate a distance between the
more target species present in the object by crosssystem and the object.
correlating multispectral optical data from at least oneof the optical channels with a reference spectrum.
35
Embodiment 234

Embodiment 238

The system of Embodiment 233, wherein spectral characteristics of the two optical channels are identical.

An infrared (IR) imaging system, the imaging systemcomprising:40
Embodiment 235
a plurality of spatially and spectrally different optical, atleast some of the plurality of optical channels config
An infrared (IR) imaging system, the imaging systemured to receive IR radiation from a common object,comprising:each of the plurality of spatially and spectrally different
a plurality of spatially and spectrally different optical, at
optical channels comprising at least one imaging lens
45
least some of the plurality of optical channels config
configured to image the object on a Focal Plane Array
ured to receive IR radiation from a common object,(FPA) unit; andeach of the plurality of spatially and spectrally different
processing electronics in communication with the FPA
optical channels comprising at least one imaging lensunit,
configured to image the object on a Focal Plane Array50
wherein said infrared system is configured to:(FPA) unit:
acquire multispectral optical data from the plurality ofa visible light imaging sensor; anddifferent optical channels; andprocessing electronics in communication with the FPA
process the multispectral optical data to detect one orunit,
more target species present in the object by using
wherein said infrared system is configured to:spectral unmixing.
acquire multispectral optical data from the plurality of 55
different optical channels; andEmbodiment 239
process the multispectral optical data to detect one ormore target species present in the object.
An infrared (IR) imaging system, the imaging system
Embodiment 236
60 comprising:an optical system including an optical focal plane array
An infrared (IR) imaging system, the imaging system(FPA) unit, the optical system includes components
comprising:associated with at least two optical channels, said at
a plurality of spatially and spectrally different optical, atleast two optical channels being spatially and spectrally
least some of the plurality of optical channels config 65
different from one another, each of the at least two
optical channels positioned to transfer IR radiation
ured to receive IR radiation from a common object,
each of the plurality of spatially and spectrally different
incident on the optical system towards the optical FPA

US 9,599,508 B2
29
unit, the optical FPA unit comprising at least two
detector arrays disposed at a distance from two corre
sponding focusing lenses;
at least one thermal reference having a known tempera
ture, wherein one of the at least two detector arrays is
configured to image the at least one reference; anda data-processing unit, said data-processing unit config

30
FIG. 6B depicts theoretical plots of transmission characteristics of a spectrally multiplexed notch-pass filter combi
nation used in an embodiment of the system.
FIG. 6C shows theoretical plots of transmission characteristics of spectrally multiplexed long-pass filter combina
tion used in an embodiment of the system.
FIG. 6D shows theoretical plots of transmission characteristics of spectrally multiplexed short-pass filter combina
ured to:acquire a plurality of frames with one of the at least two 10 tion used in an embodiment of the system.
FIG. 7 is a set of video-frames illustrating operability ofdetector arrays having regions in the plurality of imageanembodiment of the system used for gas detection.
frames that correspond to the image of the reference;
FIGS. 8A and 8B are plots (on axes of wavelength inandversus the object temperature in Celsius represent
dynamically calibrate another of the at least two detector microns
ing effective optical intensity of the object) illustratingarray to match a temperature estimate of another of the 15 results of dynamic calibration of an embodiment of theat least two detector array with the temperature esti system.
mate of one of the at least two detector array.
FIGS. 9A and 9B illustrate a cross-sectional view ofdifferent
embodiments of an imaging system comprising an
Embodiment 240
arrangement of reference Sources and mirrors that can beused for dynamic calibration.
The system of any of Embodiments 1-59, wherein the
FIGS. 10A-10C illustrate a plan view of different embodi
plurality of cameras are configured to acquire multispectral ments of an imaging System comprising an arrangement ofimage data from an object continuously for a duration of reference sources and mirrors that can be used for dynamic

time.

calibration.
25

Embodiment 241

Like reference numbers and designations in the various
drawings indicate like elements.

DETAILED DESCRIPTION
The system of any of Embodiments 1-59, comprising atleast two spectrally and spatially distinct optical channels
configured to transfer two-dimensional image data of an 30 The following description is directed to certain imple
mentations for the purposes of describing the innovative
object to the plurality of cameras.
Details of one or more implementations of the subject aspects of this disclosure. However, a person having ordi
nary skill in the art will readily recognize that the teachings
matter described in this disclosure are set forth in theaccompanying drawings and the description below. Other herein can be applied in a multitude of different ways. The
features, aspects, and advantages will become apparent from 35 described implementations may be implemented in any
the description, the drawings and the claims. Note that the device, apparatus, or system that can be configured torelative dimensions of the following figures may not be operate as an imaging System Such as in an infra-red imaging
drawn to scale.
system. The methods and systems described herein can be
included in or associated with a variety of devices such as,
BRIEF DESCRIPTION OF THE DRAWINGS
40 but not limited to devices used for visible and infrared
spectroscopy, multispectral and hyperspectral imaging
FIG. 1 shows an embodiment of an imaging system devices used in oil and gas exploration, refining, and transincluding a common front objective lens that has a pupil portation, agriculture, remote sensing, defense and home
divided spectrally and re-imaged with a plurality of tenses land security, Surveillance, astronomy, environmental moni
onto an infrared FPA.
45 toring, etc. The methods and systems described herein have
FIG. 2 shows an embodiment with a divided front objec applications in a variety of fields including but not limited toagriculture, biology, physics, chemistry, defense and hometive lens and an array of infrared sensing FPAs.
FIG. 3A represents an embodiment employing an array of land security, environment, oil and gas industry, etc. The
front objective lenses operably matched with the re-imaging teachings are not intended to be limited to the implementa
lens array. FIG. 3B illustrates a two-dimensional array of 50 tions depicted solely in the Figures, but instead have wide
optical components corresponding to the embodiment of applicability as will be readily apparent to one having
FIG 3A
ordinary skill in the art.
Various embodiments disclosed herein describe a divided
FIG. 4 is a diagram of the embodiment employing anarray of field references (e.g., field stops that can be used as aperture infrared spectral imaging (DAISI) system that isreferences for calibration) and an array of respectively 55 structured and adapted to provide identification of target
corresponding relay lenses.
chemical contents of the imaged scene. The system is based
FIG. 5A is a diagram of a 4-by-3 pupil array comprising on spectrally-resolved imaging and can provide Such iden
circular optical filters (and IR blocking material between the tification with a single-shot (also referred to as a Snapshot)
optical filters) used to spectrally divide an optical wavefront comprising a plurality of images having different wave
60 length compositions that are obtained generally simultane
imaged with an embodiment of the system.
FIG. 5B is a diagram of a 4-by-3 pupil array comprising ously. Without any loss of generality, Snapshot refers to arectangular optical filters (and IR blocking material between system in which most of the data elements that are collected
the optical filters) used to spectrally divide an optical are continuously viewing the light emitted from the scene. In
contrast in Scanning systems, at any given time only awavefront imaged with an embodiment of the system.
FIG. 6A depicts theoretical plots of transmission charac 65 minority of data elements are continuously viewing a scene,teristics of a combination of band-pass filters used with an followed by a different set of data elements, and so on, until
the full dataset is collected. Relatively fast operation can beembodiment of the system.

US 9,599,508 B2
31
32
Some
embodiments
described
herein can also be less sus
achieved in a Snapshot system because it does not need touse spectral or spatial scanning for the acquisition of infra ceptible to motion artifacts in comparison with spatiallyred (IR) spectral signatures of the target chemical contents. and/or spectrally scanning systems which can cause errors in
Instead, IR detectors (such as, for example, infrared focal either the spectral data, spatial data, or both.
FIG. 1 provides a diagram Schematically illustrating spa
plane arrays or FPAs) associated with a plurality of different
optical channels having different wavelength profiles can be tial and spectral division of incoming light by an embodi
used to form a spectral cube of imaging data. Although ment 100 of a divided aperture infrared spectral imager
spectral data can be obtained from a single Snapshot com (DAISI) system that can image an object 110 possessing IR
prising multiple simultaneously acquired images corre spectral signature(s). The system 100 includes a front objec
sponding to different wavelength ranges, in various embodi 10 tive lens 124, an array of optical filters 130, an array ofments, multiple Snap shots may be obtained. In various reimaging lenses 128 and a detector array 136. In various
embodiments, these multiple Snapshots can be averaged. embodiments, the detector array 136 can include a single
Similarly, in certain embodiments multiple Snap shots may FPA or an array of FPAs. Each detector in the detector arraybe obtained and a portion of these can be selected and 136 can be disposed at the focus of each of the lenses in thepossibly averaged. Also, in contrast to commonly used IR 15 array of reimaging lenses 128. In various embodiments, thespectral imaging systems, the DAISI system does not detector array 136 can include a plurality of photo-sensitive
require cooling. Accordingly, it can advantageously use devices. In some embodiments, the plurality of photo
uncooled infrared detectors. For example, in various imple sensitive devices may comprise a two-dimensional imaging
mentations, the imaging systems disclosed herein do not sensor array that is sensitive to radiation having wavelengths
include detectors configured to be cooled to a temperature between 1 Lim and 20 Lum (for example, in near infra-red
below 300 Kelvin. As another example, in various imple wavelength range, mid infra-red wavelength range, or longmentations, the imaging systems disclosed herein do not infra-red wavelength range). In various embodiments, theinclude detectors configured to be cooled to a temperature plurality of photo-sensitive devices can include CCD or
below 273 Kelvin. As yet another example, in various CMOS sensors, bolometers, microbolometers or other
implementations, the imaging systems disclosed herein do 25 detectors that are sensitive to infra-red radiation.
not include detectors configured to be cooled to a tempera
An aperture of the system 100 associated with the front
ture below 250 Kelvin. As another example, in various objective lens system 124 is spatially and spectrally divided
implementations, the imaging systems disclosed herein do by the combination of the array of optical filters 130 and the
not include detectors configured to be cooled to a tempera array of reimaging lenses 128. In various embodiments, theture below 200 Kelvin.
30 combination of the array of optical filters 130 and the array
Implementations disclosed herein provide several advan of reimaging lenses 128 can be considered to form atages over existing IR spectral imaging systems, most if not spectrally divided pupil that is disposed forward of theall of which may require FPAs that are highly sensitive and optical detector array 136. The spatial and spectral division
cooled in order to compensate, during the optical detection, of the aperture into distinct aperture portions forms a plu
for the reduction of the photon flux caused by spectrum 35 rality of optical channels 120 along which light propagates.
scanning operation. The highly sensitive and cooled FPA In various embodiments, the array 128 of re-imaging lenses
systems are expensive and require a great deal of mainte
128a and the array of spectral filters 130 which respectively
nance. Since various embodiments disclosed herein are
correspond to the distinct optical channels 120. The plurality
configured to operate in single-shot acquisition mode with of optical channels 120 can be spatially and/or spectrally
out spatial and/or spectral scanning, the instrument can 40 distinct. The plurality of optical channels 120 can be formed
receive photons from a plurality of points (e.g., every point) in the object space and/or image space. In one implemenof the object Substantially simultaneously, during the single tation, the distinct channels 120 may include optical chanreading. Accordingly, the embodiments of imaging system nels that are separated angularly in space. The array ofdescribed herein can collect a substantially greater amount spectral filters 130 may additionally include a filter-holdingof optical power from the imaged scene (for example, an 45 aperture mask (comprising, for example, IR light-blocking
order of magnitude more photons) at any given moment in materials such as ceramic, metal, or plastic). Light from thetime especially in comparison with spatial and/or spectral object 110 (for example a cloud of gas), the optical prop
scanning systems. Consequently, various embodiments of erties of which in the IR are described by a unique absorp
the imaging systems disclosed herein can be operated using tion, reflection and/or emission spectrum, is received by the
uncooled detectors (for example, FPA unit including an 50 aperture of the system 100. This light propagates througharray of microbolometers) that are less sensitive to photons each of the plurality of optical channels 120 and is further
in the IR but are well fit for continuous monitoring appli imaged onto the optical detector array 136. In various
cations. For example, in various implementations, the imag implementations, the detector array 136 can include at leasting systems disclosed herein do not include detectors con one FPA. In various embodiments, each of the re-imaging
figured to be cooled to a temperature below 300 Kelvin. As 55 lenses 128a can be spatially aligned with a respectively
another example, in various implementations, the imaging corresponding spectral region. In the illustrated implemensystems disclosed herein do not include detectors configured tation, each filter element from the array of spectral filters
to be cooled to a temperature below 273 Kelvin. As yet 130 corresponds to a different spectral region. Each reanother example, in various implementations, the imaging imaging lens 128a and the corresponding filter element ofsystems disclosed herein do not include detectors configured 60 the array of spectral filter 130 can coincide with (or form) a
to be cooled to a temperature below 250 Kelvin. As another portion of the divided aperture and therefore with respec
example, in various implementations, the imaging systems tively-corresponding spatial channel 120. Accordingly, indisclosed herein do not include detectors configured to be various embodiment an imaging lens 128a and a corre
cooled to a temperature below 200 Kelvin. Imaging systems sponding spectral filter can be disposed in the optical path ofincluding uncooled detectors can be capable of operating in 65 one of the plurality of optical channels 120. Radiation fromextreme weather conditions, require less power, are capable the object 110 propagating through each of the plurality ofof operation during day and night, and are less expensive. optical channels 120 travels along the optical path of eachUS 9,599,508 B2
33
34
re-imaging lens 128a and the corresponding filter element of pendicular to optical axis of the lens 124, thus the function
the array of spectral filter 130 and is incident on the detector ality of the system 100 is not necessarily compromised whenarray (e.g., FPA component) 136 to form a single image the objective lens 124 is not included. Generally, however,(e.g., sub-image) of the object 110. The image formed by the the field apertures corresponding to different optical chandetector array 136 generally includes a plurality of sub nels may be located in the same or different planes. These
images formed by each of the optical channels 120. Each of field apertures may be defined by the aperture of the reim
the plurality of Sub-images can provide different spatial and aging lens 128a and/or filters in the divided aperture 130 inspectral information of the object 110. The different spatial certain implementations. In one implementation, the field
information results from Some parallax because of the apertures corresponding to different optical channels can bedifferent spatial locations of the smaller apertures of the 10 located in different planes and the different planes can bedivided aperture. In various embodiments, adjacent Sub optical conjugates of one another. Similarly, while all of theimages can be characterized by close or Substantially equal filter elements in the array of spectral filters 130 of thespectral signatures. The detector array (e.g., FPA compo embodiment 100 are shown to lie in one plane, generallynent) 136 is further operably connected with a processor 150 different filter elements of the array of spectral filter 130 can(not shown). The processor 150 can be programmed to 15 be disposed in different planes. For example, different filter
aggregate the data acquired with the system 100 into a elements of the array of spectral filters 130 can be disposed
spectral data cube. The data cube represents, in spatial (x,y) in different planes that are optically conjugate to one
and spectral (w) coordinates, an overall spectral image of the another. However, in other embodiments, the different filter
object 110 within the spectral region defined by the combi elements can be disposed in non-conjugate planes.
nation of the filter elements in the array of spectral filters
In contrast to the embodiment 100, the front objective lens
130. Additionally, in various embodiments, the processor or 124 need not be a single optical element, but instead canprocessing electronics 150 may be programmed to deter include a plurality of lenses 224 as shown in an embodimentmine the unique absorption characteristic of the object 110. 200 of the DAISI imaging system in FIG. 2. These lenses
Also, the processor/processing electronics 150 can, alterna 224 are configured to divide an incoming optical wavefront
tively or in addition, map the overall image data cube into 25 from the object 110. For example, the array of front objec
a cube of data representing, for example, spatial distribution tive lenses 224 can be disposed so as to receive an IRof concentrations, c, of targeted chemical components wavefront emitted by the object that is directed toward theDAISI system. The plurality of front objective lenses 224
within the field of view associated with the object 110.
Various implementations of the embodiment 100 can divide the wavefront spatially into non-overlapping sections.
include an optional moveable temperature-controlled refer 30 FIG. 2 shows three objective lenses 224 in a front opticalence source 160 including, for example, a shutter system portion of the optical system contributing to the spatial
comprising one or more reference shutters maintained at division of the aperture of the system in this example. The
different temperatures. The reference source 160 can include plurality of objective lenses 224, however, can be configured
a heater, a cooler or a temperature-controlled element con as a two-dimensional (2D) array of lenses. FIG. 2 presents
figured to maintain the reference source 160 at a desired 35 a general view of the imaging system 200 and the resultant
temperature. For example, in various implementations, the field of view of the imaging system 200. An exploded view
embodiment 100 can include two reference shutters main
202 of the imaging system 200 is also depicted in greater
tained at different temperatures. The reference source 160 is detail in a figure inset of FIG. 2. As illustrated in the detailed
removably and, in one implementation, periodically inserted view 202, the embodiment of the imaging system 200
into an optical path of light traversing the system 100 from 40 includes a field reference 204 at the front end of the system.
the object 110 to the detector array (e.g., FPA component) The field reference 204 can be used to truncate the field of
136 along at least one of the channels 120. The removable view. The configuration illustrated in FIG. 2 has an opera
reference source 160 thus can block such optical path. tional advantage over embodiment 100 of FIG. 1 in that theMoreover, this reference source 160 can provide a reference overall size and/or weight and/or cost of manufacture of the
IR spectrum to recalibrate various components including the 45 embodiment 200 can be greatly reduced because the objec
detector array 136 of the system 100 in real time. The tive lens is smaller. Each pair of the lenses in the array 224
configuration of the moveable reference source 160 is fur and the array 128 is associated with a field of view (FOV).
ther discussed below.
Each pair of lenses in the array 224 and the array 128
In the embodiment 100, the front objective lens system receives light from the object from a different angle. Accord
124 is shown to include a single front objective lens posi 50 ingly, the FOV of the different pairs of lenses in the arraytioned to establish a common field-of-view (FOV) for the 224 and the array 128 do not completely overlap as a result
reimaging lenses 128a and to define an aperture stop for the of parallax. As the distance between the imaging system 200
whole system. In this specific case, the aperture stop Sub (portion 202) and the object 110 increases, the overlapping
stantially spatially coincides with and/or is about the same region 230 between the FOVs of the individual lenses 224
size or slightly larger, as the plurality of Smaller limiting 55 increases while the amount of parallax 228 remains approxi
apertures corresponding to different optical channels 120. As mately the same, thereby reducing its effect on the systema result, the positions for spectral filters of the different 200. When the ratio of the parallax-to-object-distance is
optical channels 120 coincide with the position of the Substantially equal to the pixel-size-to-system-focal-length
aperture stop of the whole system, which in this example is ratio then the parallax effect may be considered to beshown as a surface between the lens system 124 and the 60 negligible and, for practical purposes, no longer distinguisharray 128 of the reimaging lenses 128a. In various imple able. While the lenses 224 are shown to be disposed submentations, the lens system 124 can be an objective lens stantially in the same plane, optionally different objective
124. However, the objective lens 124 is optional and various lenses in the array of front objective lenses 224 can beembodiments of the system 100 need not include the objec disposed in more than one plane. For example, Some of thetive lens 124. In various embodiments, the objective lens 65 individual lenses 224 can be displaced with respect to some124 can slightly shifts the images obtained by the different other individual lenses 224 along the axis 226 (not shown)detectors in the array 136 spatially along a direction per and/or have different focal lengths as compared to someUS 9,599,508 B2
35

36
opticalcomponents(e.g.,
objectivelenses 324a, filter ele
other lenses 224. As discussed below, the field reference 204
ments of the array of spectral filters 130, re-imaging lenses
can be useful in calibrating the multiple detectors 236.
In one implementation, the front objective lens system
128a and FPA units 336) are arranged as a 4x3 array. In one
Such as the array of lenses 224 is configured as an array of implementation, the 4x3 array 340 of optical components
lenses integrated or molded in association with a monolithic 5 (lenses 324a, 128a; detector elements 336) is used behind
Substrate. Such an arrangement can reduce the costs and the temperature controlled reference target 160. The fieldcomplexity otherwise accompanying the optical adjustment reference aperture 338 can be adapted to obscure and/orof individual lenses within the system. An individual lens block a peripheral portion of the bundle of light propagating
224 can optionally include a lens with varying magnifica from the object 110 towards the FPA units 336. As a result,tion. As one example, a pair of thin and large diameter 10 the field reference 338 obscures and/or blocks the border or
Alvarez plates can be used in at least a portion of the front peripheral portion(s) of the images of the object 110 formed
objective lens system. Without any loss of generality, the on the FPA elements located along the perimeter 346 of theAlvarez plates can produce a change in focal length when detector system. Generally, two elements of the FPA unit
will produce Substantially equal values of digital counts
translated orthogonally with respect to the optical beam.
In further reference to FIG. 1, the detector array 136 (e.g., 15 when they are used to observe the same portion of the scene
FPA component) configured to receive the optical data in the same spectral region using the same optical train. If
representing spectral signature(s) of the imaged object 110 any of these input parameters (for example, Scene to be
can be configured as a single imaging array (e.g., FPA) 136. observed, spectral content of light from the scene, or optical
This single array may be adapted to acquire more than one elements delivering light from the scene to the two detector
image (formed by more than one optical channel 120) elements) differ, the counts associated with the elements ofsimultaneously. Alternatively, the detector array 136 may the FPA unit will differ as well. Accordingly, and as aninclude a FPA unit. In various implementations, the FPA unit example, in a case when the two FPAs of the FPA unit 336
can include a plurality of optical FPAs. At least one of these (such as those denoted as #6 and #7 in FIG. 3B) remain
plurality of FPAS can be configured to acquire more than one substantially un-obscured by the field reference 338, thespectrally distinct image of the imaged object. For example, 25 outputs from these FPAS can be dynamically adjusted to the
output from one of the FPAs located along perimeter 346
as shown in the embodiment 200 of FIG. 2, in various
embodiments, the number of FPAS included in the FPA unit
(such as, for example, the FPA element #2 or FPA element
may correspond to the number of the front objective lenses #11) that processes light having similar spectral character
224. In the embodiment 200 of FIG. 2, for example, three istics.
FPAS 236 are provided corresponding to the three objective 30 FIG. 4 illustrates schematically a portion of anotherlenses 224. In one implementation of the system, the FPA embodiment of an imaging system 400 that contains an array
unit can include an array of microbolometers. The use of 424 of front objective lenses 424a. The array 424 of lenses
multiple microbolometers advantageously allows for an 424a adapted to receive light from the object 110 and relay
inexpensive way to increase the total number of detection the received light to the array 128 of re-imaging lenses 128a
elements (i.e. pixels) for recording of the three-dimensional 35 through an array 438 of field references (or field stops) 438a,data cube in a single acquisition event (i.e. one Snapshot). In and through an array 440 of the relay lenses. The spectral
various embodiments, an array of microbolometers more characteristics of the field references/field stops 438a can be
efficiently utilizes the detector pixels of the array of FPAs known. The field references 438a are disposed at corre
sponding intermediate image planes defined, with respect to(e.g., each FPA) as the number of unused pixels is reduced,
minimized and/or eliminated between the images that may 40 the object 110, by respectively corresponding front objective
lenses 424a. When refractive characteristics of all of theexist when using a single microbolometer.
FIG. 3A illustrates schematically an embodiment 300 of front objective lenses 424a are substantially the same, all of
the imaging system in which the number of the front the field references 438a are disposed in the same plane. A
objective lenses 324a in the lens array 324, the number of field reference 438a of the array 438 obscures (or casts a
re-imaging lenses 128a in the lens array 128, and the number 45 shadow on) a peripheral region of a corresponding image
of FPAS 336 are the same. So configured, each combination (e.g., Sub-image) formed at the detector plane 444 throughof respectively corresponding front objective lens 324, re a respectively corresponding spatial imaging channel 450 ofimaging lens 128a, and FPAS 336 constitutes an individual the system 400 prior to such image being spectrally pro
imaging channel Such a channel is associated with acquisi cessed by the processor 150. The array 440 of relay lenses
tion of the IR light transmitted from the object 110 through 50 then transmits light along each of the imaging channels 450
an individual filter element of the array of optical filters 130. through different spectral filters 454a of the filter array 454,
A field reference 338 of the system 300 is configured to have past the calibration apparatus that includes two temperaturea uniform temperature across its Surface and be character controlled shutters 460a, 460b, and then onto the detector
ized by a predetermined spectral curve of radiation emanat
module 456. In various embodiments, the detector module
ing therefrom. In various implementations, the field refer 55 456 can include a microbolometer array or some other IR
ence 338 can be used as a calibration target to assist in FPA.
The embodiment 400 has several operational advantages.
calibrating or maintaining calibration of the FPA. Accord
ingly, in various implementations, the field reference 338 is It is configured to provide a spectrally known object within
used for dynamically adjusting the data output from each every image (e.g., Sub-image) and for every Snapshot acqui
FPA 336 after acquisition of light from the object 110. This 60 sition which can be calibrated against. Such spectral cer
dynamic calibration process helps provide that output of the tainty can be advantageous when using an array of IR FPAS
different (e.g., most, or each of the) FPA 336 represents like microbolometers, the detection characteristics of whichcorrect acquired data, with respect to the other FPAS 336 for can change from one imaging frame to the next due to, inpart, changes in the scene being imaged as well as theanalysis, as discussed below in more detail.
FIG. 3B illustrates the plan view perpendicular to the axis 65 thermal effects caused by neighboring FPAs. In various
226 of an embodiment 300 of the imaging system illustrated embodiments, the field reference array 438 of the embodi
ment 400—can be disposed within the Rayleigh rangein FIG. 3A. For the embodiment shown in FIG. 3B, theUS 9,599,508 B2
37
38(approximately corresponding to the depth of focus) asso ments of imaging systems disclosed herein is depicted in
ciated with the front objective lenses 424, thereby removing FIG. 6B. Filters of FIG. 6C can be referred to as long
unusable blurred pixels due to having the field reference wavelength pass, LP filters. An LP filter generally attenuates
outside of this range. Additionally, the embodiment 400 of shorter wavelengths and transmits (passes) longer wave
FIG. 4 can be more compact than, for example, the con 5 lengths (e.g., over the active range of the target IR portion
figuration 300 of FIG. 3A. In the system shown in FIG. 3A, of the spectrum). In various embodiments, short-wave
for example, the field reference 338 may be separated from length-pass filters, SP may also be used. An SP filter
the lens array 324 by a distance greater than several (for generally attenuates longer wavelengths and transmits
example, five) focal lengths to minimize/reduce blur con (passes) shorter wavelengths (e.g., over the active range oftributed by the field reference to an image formed at a 10 the target IR portion of the spectrum). At least in part due todetector plane.
In various embodiments, the multi-optical FPA unit of the the Snap-shot/non-Scanning mode of operation, embodi
IR imaging system can additionally include an FPA config ments of the imaging system described herein can use lessured to operate in a visible portion of the spectrum. In sensitive microbolometers without compromising the SNR.
reference to FIG. 1, for example, an image of the scene of 15 The use of microbolometers, as detector-noise-limited
interest formed by such visible-light FPA may be used as a devices, in turn not only benefits from the use of spectrally
background to form a composite image by overlapping an IR multiplexed filters, but also does not require cooling of theimage with the visible-light image. The IR image may be imaging system during normal operation.
overlapped virtually, with the use of a processor and spe
Referring again to FIGS. 6A, 6B, 6C, and 6D, each of thecifically-designed computer program product enabling Such filters (Ob . . . 3d) transmits light in a substantially wider
data processing, or actually, by a viewer. The IR image may region of the electromagnetic spectrum as compared to thosebe created based on the image data acquired by the indi of the filters (0a . . .3a). Accordingly, when the spectrally
vidual FPAS 136. The so-formed composite image facilitates multiplexed set of filters (Ob . . . 0d) is used with an
the identification of the precise spatial location of the target embodiment of the imaging system, the overall amount ofspecies, the spectral signatures of which the system is able 25 light received by the FPAs (for example, 236, 336) is larger
to detect/recognize.
than would be received when using the bandpass filters
Optical Filters.
The optical filters, used with an embodiment of the (Oa. . . 4a). This “added transmission of light defined by thesystem, that define spectrally-distinct IR image (e.g., Sub use of the spectrally-multiplexed LP (or SP) filters facilitates
image) of the object can employ absorption filters, interfer 30 an increase of the signal on the FPAS above the level of theence filters, and Fabry-Perot etalon based filters, to name detector noise. Additionally, by using, in an embodiment ofjust a few. When interference filters are used, the image the imaging system, filters having spectral bandwidths
acquisition through an individual imaging channel defined greater than those of band-pass filters, the uncooled FPAs ofby an individual re-imaging lens (such as a lens 128a of the embodiment of the imaging system experience less
FIGS. 1, 2, 3, and 4) may be carried out in a single spectral 35 heating from radiation incident thereon from the imaged
bandwidth or multiple spectral bandwidths. Referring again scene and from radiation emanating from the FPA in questo the embodiments 100, 200, 300, 400 of FIGS. 1 through tion itself. This reduced heating is due to a reduction in the4, and in further reference to FIG. 3B, examples of a 4-by-3 back-reflected thermal emission(s) coming from the FPA
array of spectral filters 130 is shown in FIGS. 5A and 5B. and reflecting off of the filter from the non-band-pass
Individual filters 1 through 12 are juxtaposed with a Sup 40 regions. As the transmission region of the multiplexed LP
porting opto-mechanical element (not shown) to define a (or SP) filters is wider, such parasitic effects are reduced
filter-array plane that is oriented, in operation, Substantially thereby improving the overall performance of the FPA unit.
perpendicularly to the general optical axis 226 of the imag
In one implementation, the LP and SP filters can be
ing system. In various implementations, the individual filters
1 through 12 need not be discrete optical components. 45 combined, in a spectrally-multiplexed fashion, in order toInstead, the individual filters 1 through 12 can comprise one increase or maximize the spectral extent of the transmission
or more coatings that are applied to one or more Surfaces of region of the filter system of the embodiment.
The advantage of using spectrally multiplexed filters is
the reimaging lenses (such as a lens 128a of FIGS. 1, 2, 3,
and 4) or the Surfaces of one or more detectors.
appreciated based on the following derivation, in which a
The optical filtering configuration of various embodi 50 system of M filters is examined (although it is understoodments disclosed herein may advantageously use a bandpass that in practice an embodiment of the invention can employ
filter defining a specified spectral band. Any of the filters 0a any number of filters). As an illustrative example, the casethrough 3a the transmission curves of which are shown in of M-7 is considered. Analysis presented below relates to
FIG. 6A may, for example, be used. The filters may be one spatial location in each of the images (e.g., Sub-images)
placed in front of the optical FPA (or generally, between the 55 formed by the differing imaging channels (e.g., different
optical FPA and the object). In particular, and in further optical channels 120) in the system. A similar analysis canreference to FIGS. 1, 23, and 4, when optical detector arrays be performed for each point at an image (e.g., Sub-image),
136, 236, 336, 456 include microbolometers, the predomi
thus the analysis can be appropriately extended asnant contribution to noise associated with image acquisition andrequired.
60
is due to detector noise. To compensate and/or reduce the
The unknown amount of light within each of the M
noise, various embodiments disclosed herein utilize spec
trally-multiplexed filters. In various implementations, the spectral channels (corresponding to these M filters) is
spectral-multiplexed filters can comprise a plurality of long denoted with f. f. f. . . . f. and readings from corre
pass filters, a plurality long pass filters, a plurality of band sponding detector elements receiving light transmitted bypass filters and any combinations thereof. An example of the 65 each filter is denoted as g. g. g . . . g., while measurement
spectral transmission characteristics of spectrally-multi errors are represented by n, n2 ns. . . . n. Then, theplexed filters Ob through 3d for use with various embodi readings at the seven FPA pixels each of which is opticallyUS 9,599,508 B2
39
filtered by a corresponding band-pass filter of FIG. 6A canbe represented by:40
denote an estimate of a given value, then a linear combina
tion of the measurements can be used such as, for example,

gift--n1,
g2 f+m2,

gs f{-n3.
gaft-na,10

gs fins,
go fins,
15

These readings (pixel measurements) g, are estimates ofthe spectral intensities f. The estimates g, are not equal to thecorresponding f, values because of the measurement errorsn. However, if the measurement noise distribution has zeromean, then the ensemble mean of each individual measure

These fare unbiased estimates when the n, are Zero mean
stochastic variables, so that (f-f)=0. The measurement

ment can be considered to be equal to the true value, i.e.

(g) f. Here, the angle brackets indicate the operation ofvariance corresponding to i' measurement is given by thecalculating the ensemble mean of a stochastic variable. The
variance of the measurement can, therefore, be representedaS

equation below:
25

In embodiments utilizing spectrally-multiplexed filters, incomparison with the embodiments utilizing band-pass fil
ters, the amount of radiant energy transmitted by each of the 30
From the above equation, it is observed that by employing
spectrally-multiplexed LP or SP filters towards a given
detector element can exceed that transmitted through a spectrally-multiplexed system the signal-to-noise ratio
spectral band of aband-pass filter. In this case, the intensities (SNR) of a measurement is improved by a factor ofof light corresponding to the independent spectral bands can V1677–151.
be reconstructed by computational means. Such embodi 35 For N channels, the SNR improvement achieved with aments can be referred to as a “multiplex design”.
spectrally-multiplexed system can be expressed as (N+1)/(2One matrix of such “multiplexed filter’ measurements VN). For example, an embodiment employing 12 spectral
includes a Hadamard matrix requiring "negative' filters that channels (N=12) is characterized by a SNR improvement,
may not be necessarily appropriate for the optical embodi over a non-spectrally-multiplexed system, comprising aments disclosed herein. An S-matrix approach (which is 40 factor of up to 1.88.
restricted to having a number of filters equal to an integer
Two additional examples of related spectrally-multithat is multiple of four minus one) or a row-doubled Had plexed filter arrangements Oc through 3c and 0d through 3d
amard matrix (requiring a number of filters to be equal to an that can be used in various embodiments of the imaging
integer multiple of eight) can be used in various embodi systems described herein are shown in FIGS. 6C and 6D,
ments. Here, possible numbers of filters using an S-matrix 45 respectively. The spectrally-multiplexed filters shown insetup are 3, 7, 11, etc and, if a row-doubled Hadamard FIGS. 6C and 6D can be used in embodiments of imaging
matrix setup is used, then the possible number of filters is 8,
16, 24, etc. For example, the goal of the measurement may systems employing uncooled FPAS (Such as microbolom
be to measure seven spectral band 1; intensities using seven eters) FIG. 6C illustrates a set of spectrally-multiplexed
long-wavelength pass (LP) filters used in the system. An LP
measurements g as follows:50
filter generally attenuates shorter wavelengths and transmits(passes) longer wavelengths (e.g., over the active range ofthe target IR portion of the spectrum). A single spectral
channel having a transmission characteristic corresponding
55 to the difference between the spectral transmission curves ofat least two of these LP filters can be used to procure
imaging data for the data cube using an embodiment of thesystem described herein. In various implementations, thespectral filters disposed with respect to the different FPAs
60 can have different spectral characteristics. In various imple
mentations, the spectral filters may be disposed in front of
only some of the FPAS while the remaining FPAS may be
configured to receive unfiltered light. For example, in some
Optical transmission characteristics of the filters implementations, only 9 of the 12 detectors in the 4x3 arraydescribed above are depicted in FIG. 6B. Here, a direct 65 of detectors described above may be associated with aestimate of the f, is no longer provided through a relationship spectral filter while the other 3 detectors may be configured
similar to (g)=f. Instead, if a “hat notation is used to to received unfiltered light. Such a system may be config

US 9,599,508 B2
41
ured to acquire spectral data in 10 different spectral channels
in a single data acquisition event.
The use of microbolometers, as detector-noise-limited

devices, in turn not only can benefit from the use ofspectrally multiplexed filters, but also does not requirecooling of the imaging system during normal operation. In
contrast to imaging systems that include highly sensitive42
ture T is measured directly and independently with therm
istors. Following Such initial measurement, the first shutter
160a is removed from the optical path of light traversing theembodiment and another second shutter (for example, 160b)
is inserted in its place across the optical axis 226 to preventthe propagation of light through the system. The temperatureof the second shutter 160b can be different than the first

shutter (TzT). The temperature of the second shutter 160b
ments of imaging systems described herein can employ less is also independently measured with thermistors placed insensitive microbolometers without compromising the SNR. 10 contact with this shutter, and the detector response to radia
This result is at least in part due to the Snap-shot/non tion emanating from the shutter 160b is also recorded.
Denoting operational response of FPA pixels (expressed inscanning mode of operation.
As discussed above, an embodiment may optionally, and digital numbers, or “counts”) as g, to a source of radiance L.
in addition to a temperature-controlled reference unit (for the readings corresponding to the measurements of the two
example temperature controlled shutters such as shutters 15 shutters can be expressed as:
160; 160a, 160b, 460a, 460b), employ a field referencecomponent (e.g., field reference aperture 338 in FIG. 3A), or
an array of field reference components (e.g., filed reference
apertures 438 in FIG. 4), to enable dynamic calibration.
Such dynamic calibration can be used for spectral acquisi
Here, g, is the pixel offset value (in units of counts),tion of one or more or every data cube. Such dynamic and Y is the pixel gain value (in units of counts per radiancecalibration can also be used for a spectrally-neutral camera unit). The solutions of these two equations with respect toto-camera combination to enable dynamic compensation of the two unknowns g, and Y can be obtained if the values
parallax artifacts. The use of the temperature-controlled of g and g and the radiance values L and L are available.
reference unit (for example, temperature-controlled shutter 25 These values can, for example, be either measured by asystem 160) and field-reference component(s) facilitates reference instrument or calculated from the known tempera
maintenance of proper calibration of each of the FPAS tures T and T together with the known spectral response of
the optical system and FPA. For any Subsequent measureindividually and the entire FPA unit as a whole.
In particular, and in further reference to FIGS. 1, 2, 3, and ment, one can then invert the equation(s) above in order to4, the temperature-controlled unit generally employs a sys 30 estimate the radiance value of the object from the detector
tem having first and second temperature Zones maintained at measurement, and this can be done for each pixel in eachfirst and second different temperatures. For example, shutter FPA within the system.
As already discussed, and in reference to FIGS. 1 throughsystem of each of the embodiments 100, 200, 300 and 400
can employ not one but at least two temperature-controlled 4, the field-reference apertures may be disposed in an object
shutters that are Substantially parallel to one another and 35 space or image space of the optical system, and dimensioned
transverse to the general optical axis 226 of the to block a particular portion of the IR radiation received
embodiment(s) 100, 200, 300, 400. Two shutters at two from the object. In various implementations, the field
different temperatures may be employed to provide more reference aperture, the opening of which can be substantiallyinformation for calibration; for example, the absolute value similar in shape to the boundary of the filter array (forof the difference between FPAS at one temperature as well as 40 example, and in reference to a filter array of FIGS. 3B.
the change in that difference with temperature change can be 5B e.g., rectangular). The field-reference aperture can berecorded. Referring, for example, to FIG. 4, in which such placed in front of the objective lens (124, 224, 324, 424) at
multi-shutter structure is shown, the use of multiple shutters a distance that is at least several times (in one implemen
enables the user to create a known reference temperature tation—at least five times) larger than the focal length of thedifference perceived by the FPAS 456. This reference tem 45 lens such that the field-reference aperture is placed closer toperature difference is provided by the IR radiation emitted the object. Placing the field-reference aperture closer to theby the shutter(s) 160a, 160b when these shutters are posi object can reduce the blurriness of the image. In the embodi
tioned to block the radiation from the object 110. As a result, ment 400 of FIG. 4, the field-reference aperture can be
not only the offset values corresponding to each of the placed within the depth of focus of an image conjugate plane
individual FPAS pixels can be adjusted but also the gain 50 formed by the front objective lens 424. The field reference,
generally, can facilitate, effectuates and/or enable dynamic
values of these FPAS. In an alternative embodiment, thesystem having first and second temperature Zones may compensation in the system by providing a spectrally known
include a single or multi-portion piece. This single or and temporally-stable object within every scene to reference
multi-portion piece may comprise for example a plate. This and stabilize the output from the different FPAS in the array.
piece may be mechanically-movable across the optical axis 55 Because each FPA's offset value is generally adjustedwith the use of appropriate guides and having a first portion from each frame to the next frame by the hardware, com
at a first temperature and a second portion at a second paring the outputs of one FPA with another can have an error
temperature.
that is not compensated for by the static calibration param
Indeed, the process of calibration of an embodiment of the eters g., and Y established, for example, by the movableimaging system starts with estimating gain and offset by 60 shutters 160. In order to ensure that FPAS operate in radio
performing measurements of radiation emanating, indepen metric agreement over time, it is advantageous for a portion
dently, from at least two temperature-controlled shutters of of each detector array to view a reference Source (such as the
known and different radiances. The gain and offset can vary field reference 338 in FIG. 3A, for example) over a plurality
from detector pixel to detector pixel. Specifically, first the of frames obtained over time. If the reference source spec
response of the detector unit 456 to radiation emanating 65 trum is known a priori (such as a blackbody source at afrom one shutter is carried out. For example, the first shutter known temperature), one can measure the response of each160a blocks the FOV of the detectors 456 and the tempera FPA to the reference source in order to estimate changes to
FPA units with reduced noise characteristics, the embodi

US 9,599,508 B2
44
the filter element of the array of spectral filters 130 corre
sponding to a particular FPA 336 transmits light from theobject 110 including a cloud of gas, for example, with acertain spectrum that contains the gas absorption peak or aoffset for the various detectors from the time the movablegas emission peak while another filter element of the array
shutters used for static calibration are removed. An example of spectral filters 130 corresponding to another FPA 336
calculation of the dynamic offset proceeds as follows.
does not transmit such spectrum, then the difference between
Among the FPA elements in an array of FPAS in an the images formed by the two FPAS at issue will highlightembodiment of the imaging system, one FPA can be selected the presence of gas in the difference image.
to be the “reference FPA'. The field reference temperature 10 A shortcoming of the spectral differencing method is thatmeasured by all the other FPAS can be adjusted to agree with contributions of Some auxiliary features associated withthe field reference temperature measured by the reference as imaging (not just the target species Such as gas itself) can
discussed below. The image obtained by each FPA includes also be highlighted in and contribute to the difference image.
a set of pixels obscured by the field reference 338. Using the Such contributing effects include, to name just a few, parpreviously obtained calibration parameters g., and Y (the 15 allax-induced imaging of edges of the object, influence of
pixel offset and gain), the effective blackbody temperature T, magnification differences between the two or more opticalof the field reference as measured by each FPA is estimated channels, and differences in rotational positioning and ori
using the equation below:
entation between the FPAs. While magnification-related
errors and FPA-rotation-caused errors can be compensatedfor by increasing the accuracy of the instrument construction
Using the equation above, the mean value over all pixels as well as by post-processing of the acquired imaging,that are obscured by the field reference is obtained. In the parallax is scene-induced and is not so easily correctable. In
above equation Ag, is the difference in offset value of the addition, the spectral differencing method is vulnerable tocurrent frame from Age, obtained during the calibration 25 radiance calibration errors. Specifically, if one FPA registers
step. For the reference FPA, Ag, can be simply set to zero. radiance of light from a given feature of the object as that
Then, using the temperature differences measured by each having a temperature of 40°C., for example, while the datafrom another FPA represents the temperature of the same
FPA, one obtains
object feature as being 39°C., then such feature of the object
will be enhanced or highlighted in the difference image
30 (formed at least in part based on the images provided byOnce AT, for each FPA is measured, its value can be these two FPAs) due to such radiance-calibration error.
One solution to some of such problems is to compare
subtracted from each image in order to force operational
agreement between such FPA and the reference FPA. While (e.g., Subtract) images from the same FPA obtained at
the calibration procedure has been discussed above in ref different instances in time. For example, images can beerence to calibration of temperature, a procedurally similar 35 compared to or Subtracted from a reference image obtained
methodology of calibration with respect to radiance value at another time. Such reference image, which is subtracted
can also be implemented.
from other later obtained images, may be referred to as a
Examples of Methodology of Measurements.
temporal reference image. This solution can be applied toPrior to optical data acquisition using an embodiment of spectral difference images as well. For example, the image
the IR imaging system as described herein, one or more, 40 data resulting from spectral difference images can be nor
most, or potentially all the FPAs of the system can be malized by the data corresponding to a temporal reference
calibrated. For example, greater than 50%, 60%, 70%, 80% image. For instance, the temporal reference images can beor 90% or the FPAS336 can be initially calibrated. As shown subtracted from the spectral difference image to obtain thein FIG. 3, these FPAS 336 may form separate images of the temporal difference image. This process is referred to, forobject using light delivered in a corresponding optical 45 the purposes of this disclosure, as a temporal differencing
channel that may include the combination of the correspond algorithm or method and the resultant image from Subtract
ing front objective and re-imaging lenses 324, 128. The ing the temporal reference image from another image (suchcalibration procedure can allow formation of individual as the spectral difference image) is referred to as the tem
images in equivalent units (so that, for example, the reading poral difference image. In some embodiments where specfrom the FPA pixels can be re-calculated in units of tem 50 tral differencing is employed, a temporal reference image
perature or radiance units, etc.). Moreover, the calibration may be formed, for example, by creating a spectral differ
process can also allow the FPAs (e.g., each of the FPAs) to ence image from the two or more images registered by the
be spatially co-registered with one another so that a given two or more FPAS at a single instance in time. This spectral
pixel of a particular FPA can be optically re-mapped through difference image is then used as a temporal reference image.
the optical system to the same location at the object as the 55 The temporal reference image can then be subtracted fromcorresponding pixel of another FPA.
other later obtained images to provide normalization that can
To achieve at least Some of these goals, a spectral differ be useful in Subtracting out or removing various errors or
encing method may be employed. The method involves deleterious effects. For example, the result of the algorithm
forming a difference image from various combinations of is not affected by a prior knowledge of whether the object orthe images from different channels. In particular, the images 60 scene contains a target species (such as gas of interest),
used to form difference images can be registered by two or because the algorithm can highlight changes in the scene
more different FPAS in spectrally distinct channels having characteristics. Thus, a spectral difference image can bedifferent spectral filters with different spectral characteris calculated from multiple spectral channels as discussed
tics. Images from different channels having different spectral above based on a Snap-shot image acquisition at any later
characteristics will provide different spectral information. 65 time and can be subtracted from the temporal reference
Comparing (e.g., Subtracting) these images, can therefore image to form a temporal difference image. This temporalyield valuable spectral based information. For example, if difference image is thus a normalized difference image. The

43
the pixel offset value. However, the temperature of thereference Source need not be known. In Such implementa
tions, dynamic calibration of the different detectors can be
performed by monitoring the change in the gain and theUS 9,599,508 B2
45
46
difference between the two images (the temporal difference array of filters of FIG. 5A), a difference image referred to as
image) can highlight the target species (gas) within the a “difference image 2-3” is formed. In this case, the alter
normalized difference image, since this species was not native “difference image 1-4” is additionally formed frompresent in the temporal reference frame. In various embodi the image data acquired by, for example, the alternativements, more than two FPAS can be used both for registering FPAs corresponding to filters 1 and 4 of FIG. 5A. Assuming
the temporal reference image and a later-acquired difference or ensuring that both of these two alternative FPAS haveimage to obtain a better SNR figure of merit. For example, approximately the same spectral sensitivity to the target
if two FPAS are associated with spectral filters having the species, the alternative “difference image 1-4 will highlightsame spectral characteristic, then the images obtained by the pixels corresponding to parallax-induced features in the
two FPAS can be combined after they have been registered 10 image. Accordingly, based on positive determination that thesame pixels are highlighted in the spectral "difference image
to get a better SNR figure.
While the temporal differencing method can be used to 2-3 used for target species detection, a conclusion can be
reduce or eliminate some of the shortcomings of the spectral made that the image features corresponding to these pixels
differencing, it can introduce unwanted problems of its own. are likely to be induced by parallax and not the presence ofFor example, temporal differencing of imaging data is less 15 target species in the imaged scene. It should be noted thatsensitive to calibration and parallax induced errors than the compensation of parallax can also be performed using
spectral differencing of imaging data. However, any change images created by individual re-imaging lenses, 128a, whenin the imaged scene that is not related to the target species using a single FPA or multiple FPA's as discussed above.
of interest (such as particular gas, for example) is high FPAS spaced apart from each other in different directions canlighted in a temporally-differenced image. Thus Such change also be useful. Greater than 2, for example, 3 or 4, or morein the imaged scene may be erroneously perceived as a FPAs can be used to establish parallax for parallax correc
location of the target species triggering, therefore, an error tion. In certain embodiments two central FPAS and one
in detection of target species. For example, if the tempera corner FPA are used for parallax correction. These FPA may,ture of the background against which the gas is being in certain embodiments, have Substantially similar or thedetected changes (due to natural cooling down as the day 25 same spectral characteristics, for example, have filters hav
progresses, or increases due to a person or animal or another ing similar or the same transmission spectrum or have no
object passing through the FOV of the IR imaging system), filter at all.
then such temperature change produces a signal difference
Another capability of the embodiments described herein
as compared to the measurement taken earlier in time. is the ability to perform the volumetric estimation of a gas
Accordingly, the cause of the Scenic temperature change (the 30 cloud. This can be accomplished by using (instead ofcooling object, the person walking, etc.) may appear as the compensating or negating) the parallax induced effects
detected target species (such as gas). It follows, therefore, described above. In this case, the measured parallax between
that an attempt to compensate for operational differences two or more similar spectral response images (e.g., two oramong the individual FPAs of a multi-FPA IR imaging more channels or FPAs) can be used to estimate a distancesystem with the use of methods that turn on spectral or 35 between the imaging system and the gas cloud or between
temporal differencing can cause additional problems leading the imaging system and an object in the field of view of theto false detection of target species. Among these problems system. The parallax induced transverse image shift, d.
are scene-motion-induced detection errors and parallax between two images is related to the distance, Z, between thecaused errors that are not readily correctable and/or com cloud or object 110 and the imaging system according to thepensatable. Accordingly, there is a need to compensate for 40 equation Z-SZ/d. Here, S, is the separation between two
image data acquisition and processing errors caused by similar spectral response images, and Z' is the distance to the
motion of elements within the scene being imaged. Various image plane from the back lens. The value for Z is typicallyembodiments of data processing algorithms described herein approximately equal to the focal length f of the lens of theaddress and fulfill the need to compensate for such motion imaging system. Once the distance Z between the cloud and45 the imaging system is calculated, the size of the gas cloud
induced and parallax-induced image detection errors.
In particular, to reduce or minimize parallax-induced can be determined based on the magnification, m f/Z, where
differences between the images produced with two or more each image pixel on the gas cloud, AX, corresponds to apredetermined FPAS, another difference image can be used physical size in object space AXAX'/m. To estimate thethat is formed, from the images of at least two different FPAS Volume of the gas cloud, a particular symmetry in theto estimate parallax effects. Parallax error can be determined 50 thickness of the cloud based on the physical size of the cloudby comparing the images from two different FPAs where the can be assumed. For example, the cloud image can beposition between the FPAS is known. The parallax can be rotated about a central axis running through the cloud image
calculated from the known relative position difference. Dif to create a three dimensional Volume estimate of the gas
ferences between the images from these two FPAS can be cloud size. It is worth noting that in the embodiments
attributed to parallax, especially, if the FPA have the same 55 described herein only a single imaging system is required forspectral characteristics, for example have the same spectral such volume estimation. Indeed, due to the fact that thefilter or both have no spectral filters. Parallax error correc information about the angle at which the gas cloud is seenby the system is decoded in the parallax effect, the image
tion, however, can still be obtained from two FPAS that havedifferent spectral characteristics or spectral filters, especially data includes the information about the imaged scene
if the different spectral characteristics, e.g., the transmission 60 viewed by the system in association with at least two angles.
When the temporal differencing algorithm is used forspectra of the respective filters are known and/or negligible.
Use of more than two FPAS or FPAs of different locations
processing the acquired imaging data, a change in the scene
such as FPAS spaced farther apart can be useful. For that is not caused by the target species can inadvertently be
example, when the spectral differencing of the image data is highlighted in the resulting image. In various embodiments,
performed with the use of the difference between the images 65 compensation for this error makes use of the temporalcollected by the outermost two cameras in the array (such as, differencing between two FPAs that are substantially equallyfor example, the FPAS corresponding to filters 2 and 3 of the spectrally sensitive to the target species. In this case, theUS 9,599,508 B2
47
48
temporal difference image will highlight those pixels the can be obtained by obtaining a spectral difference image
intensity of which have changed in time (and not in wave between a filtered optical channel and/or another filteredlength). Therefore, Subtracting the data corresponding to optical channel/unfiltered optical channel or any combina
these pixels on both FPAS, which are substantially equally tions thereof.
For example, an embodiment of the system configured tospectrally sensitive to the target species, to form the result
ing image, excludes the contribution of the target species to detect the presence of methane in a gas cloud comprises
the resulting image. The differentiation between (i) changes optical components such that one or more of the plurality ofin the scene due to the presence of target species and (ii) optical channels is configured to collect IR radiation tochanges in the scene caused by changes in the background provide spectral data corresponding to a discrete spectral
not associated with the target species is, therefore, possible. 10 band located in the wavelength range between about 7.9 um
In some embodiments, these two channels having the same and about 8.4 um corresponding to an absorption peak ofor Substantially similar spectral response so as be substan methane. The multispectral data obtained in the one or more
tially equally spectrally sensitive to the target species may optical channels can be correlated with a predetermined
comprise FPAs that operate using visible light. It should also absorption spectrum of methane in the wavelength range
be noted that, the data acquired with a visible light FPA 15 between about 7.9 um and 8.4 Lum. In various implementa(when present as part of the otherwise IR imaging system) tions, the predetermined absorption spectrum of methane
can also be used to facilitate such differentiation and com
can be saved in a database or a reference library accessible
pensation of the motion-caused imaging errors. Visible by the system. Based on an amount of correlation (e.g., acameras generally have much lower noise figure than IR value of correlation coefficient), the presence or absence of
cameras (at least during daytime). Consequently, the tem methane in the gas cloud can be detected.
poral difference image obtained with the use of image data Examples of Practical Embodiments and Operation
The embodiment 300 of FIG. 3 is configured to employ 12
from the visible light FPA can be quite accurate. The visible
FPA can be used to compensate for motion in the system as optical channels and 12 corresponding microbolometer
well as many potential false-alarms in the scene due to FPAs 336 to capture a video sequence substantially imme
motion caused by people, vehicles, birds, and steam, for 25 diately after performing calibration measurements. The
example, as long as the moving object can be observed in the Video sequence corresponds to images of a standard labo
visible region of the spectra. This has the added benefit of ratory scene and the calibration measurements are perproviding an additional level of false alarm Suppression formed with the use of a reference source including two
without reducing the sensitivity of the system since many shutters, as discussed above, one at room temperature andtargets Such as gas clouds cannot be observed in the visible 30 one 5° C. above room temperature. The use of 12 FPAs
spectral region. In various implementations, an IR camera allows increased chance of simultaneous detection and esti
can be used to compensate for motion artifacts.
mation of the concentrations of about 8 or 9 gases present at
Another method for detection of the gases is to use a the scene. In various embodiments, the number of FPAS 336
spectral unmixing approach. A spectral unmixing approach can vary, depending on the balance between the operational
assumes that the spectrum measured at a detector pixel is 35 requirements and consideration of cost.
composed of a sum of component spectra (e.g., methane andDue to the specifics of operation in the IR range of theother gases). This approach attempts to estimate the relative spectrum, the use of the so-called noise-equivalent tempera
weights of these components needed to derive the measure ture difference (or NETD) is preferred and is analogous toment spectrum. The component spectra are generally taken the SNR commonly used in visible spectrum instruments.
from a predetermined spectral library (for example, from 40 The array of microbolometer FPAS 336 is characterized todata collection that has been empirically assembled), though perform at NETDs72 mK at an f-number of 1.2. Each
Sometimes one can use the scene to estimate these as wellmeasurement was carried out by Summing four consecutive(often called “endmember determination'). In various frames, and the reduction in the NETD value expected due
embodiments, the image obtained by the detector pixel is a to Such Summation would be described by corresponding
radiance spectrum and provides information about the 45 factor of V4–2. Under ideal measurement conditions, therebrightness of the object. To identify the contents of a gas fore, the FPANETD should be about 36 mK.
cloud in the scene and/or to estimate the concentration of the
It is worth noting that the use of optically-filtered FPAS invarious gases in the gas cloud, an absorption/emission various embodiments of the system described herein can
spectrum of the various gases of interest can be obtained by provide a system with higher number of pixels. For example,
comparing the measured brightness with an estimate of the 50 embodiments including a single large format microbolom
expected brightness. The spectral unmixing methodology eter FPA array can provide a system with large number of
can also benefit from temporal, parallax, and motion com pixels. Various embodiments of the systems describedpensation techniques.
herein can also offer a high optical throughput for a Sub
In various embodiments, a method of identifying the stantially low number of optical channels. For example, thepresence of a target species in the object includes obtaining 55 systems described herein can provide a high optical throughthe radiance spectrum (or the absorption spectrum) from the put for a number of optical channels between 4 and 50. By
object in a spectral region indicative of the presence of the having a lower number of optical channels (e.g., between 4target species and calculating a correlation (e.g., a correla and 50 optical channels), the systems described herein havetion coefficient) by correlating the obtained radiance spec wider spectral bins which allows the signals acquired withintrum (or the absorption spectrum) with a reference spectrum 60 each spectral bin to have a greater integrated intensity.
for the target species. The presence or absence of the target
An advantage of the embodiments described herein over
species can be determined based on an amount of correlation various scanning based hyperspectral systems that are con(e.g., a value of correlation coefficient). For example, the figured for target species detection (for example, gas cloudpresence of the target species in the object can be confirmed detection) is that, the entire spectrum can be resolved in aif the amount of correlation or the value of correlation 65 Snapshot mode (for example, during one image frame acquicoefficient is greater than a threshold. In various implemen sition by the FPA array). This feature enables the embodi
tations, the radiance spectrum (or the absorption spectrum) ments of the imaging systems described herein to takeUS 9,599,508 B2
49
50
advantage of the compensation algorithms such as the as about 2.5°C. In addition, all of the temperature estimatesparallax and motion compensation algorithms mentioned are steadily drifting in time, from frame to frame. The curves
above. Indeed, as the imaging data required to implement labeled “ref (or “C”) correspond to the detectors' estimatesthese algorithms are collected simultaneously with the tar of the temperature of the aperture 338 of the embodimentget-species related data, the compensation algorithms are 300 of FIG. 3A. The results of detection of radiation carried
carried out with respect to target-species related data and not out after each detector pixel has been subjected to thewith respect to data acquired at another time interval. This dynamic calibration procedure described above are
rapid data collection thus improves the accuracy of the data expressed with the curved labeled “objcorr” (or “B”). Now,
compensation process. In addition, the frame rate of data the difference in estimated temperature of the object amongacquisition is much higher. For example, embodiments of 10 the detector elements is reduced to about 0.5° C. (thereby
the imaging system described herein can operate at video improving the original reading at least by a factor of 5).
rates from about 5 Hz and higher. For example, various
FIG. 8B represents the results of similar measurements
embodiments described herein can operate at frame rates corresponding to a different location in the scene (a locationfrom about 5 Hz to about 60 Hz or 200 Hz. Thus, the userwhich is at a temperature about 9° C. above the estimatedis able to recognize in the images the wisps and Swirls 15 temperature of the aperture 338 of FIG. 3A). As shown, thetypical of gas mixing without blurring out of these dynamic correction algorithm discussed above is operable and effec
image features and other artifacts caused by the change of tive and applicable to objects kept at different temperature.
scene (whether spatial or spectral) during the lengthy mea Accordingly, the algorithm is Substantially temperature
Surements. In contradistinction, Scanning based imaging independent.
systems involve image data acquisition over a period of time Dynamic Calibration Elements and References
exceeding a single-snap-shot time and can, therefore, blur
FIGS. 9A and 9B illustrates schematically different imple
the target gas features in the image and inevitably reduce the mentations 900 and 905 respectively of the imaging systemotherwise achievable sensitivity of the detection. This result that include a variety of temperature calibration elements tois in contrast to embodiments of the imaging system facilitate dynamic calibration of the FPAS. The temperaturedescribed herein that are capable of detecting the localized 25 calibration elements can include mirrors 975a, 975b (repconcentrations of gas without it being Smeared out with the resented as M, Mo, etc.) as well as reference sources
areas of thinner gas concentrations. In addition, the higher 972a and 972b. The implementation 900 can be similarly
frame rate also enables a much faster response rate to a leak configured as the embodiment 300 and include one or moreof gas (when detecting Such leak is the goal). For example, front objective tens, a divided aperture, one or more spectral
an alarm can trigger within fractions of a second rather than 30 filters, an array of imaging lenses 928a and an imaging
several seconds.
element 936. In various implementations, the imaging ele
To demonstrate the operation and gas detection capability ment 936 (e.g., camera block) can include an array ofof the imaging systems described herein, a prototype was cameras. In various implementations, the array of cameras
constructed in accordance with the embodiment 300 of FIG.
can comprise an optical FPA unit. The optical FPA unit can
3A and used to detect a hydrocarbon gas cloud of propylene 35 comprise a single FPA, an array of FPAs. In various imple
at a distance of approximately 10 feet. FIG. 7 illustrates mentations, the array of cameras can include one or moreVideo frames 1 through 12 representing gas-cloud-detection detector arrays represented as detector array 1, detector array
output 710 (seen as a streak of light) in a sequence from t=1
5, detector array 9 in FIGS. 9A and 9B. In various embodi
to t—12. The images 1 through 12 are selected frames taken ments, the FOV of each of the detector arrays 1, 5, 9 can befrom a video-data sequence captured at a video-rate of 15 40 divided into a central region and a peripheral region. With
frames/sec. The detected propylene gas is shown as a streak out any loss of generality, the central region of the FOV ofof light 710 (highlighted in red) near the center of each each of the detector arrays 1, 5, 9 can include the region
image. The first image is taken just prior to the gas emerging where the FOV of all the detector arrays 1, 5, 9 overlap. In
from the nozzle of a gas-contained, while the last image the embodiment illustrated in FIG. 9A, some temperature
represents the system output shortly after the nozzle has 45 calibration elements, namely, reference sources 972a andbeen turned off.
972b are placed at a distance from the detector arrays 1, 5,
The same prototype of the system can also demonstrate 9, for example, and mirrors to image them onto the detector
the dynamic calibration improvement described above by arrays are then placed at the location of the scene reference
imaging the scene Surrounding the system (the laboratory) aperture (e.g., 338 of FIG. 3A).
with known temperature differences. The result of imple 50 In FIG. 9A, the temperature-calibration elements 975a
menting the dynamic correction procedure is shown in and 975b comprise mirrors that reflect radiation from the
FIGS. 8A, 8B, where the curves labeled “obj” (or “A”) reference sources 972a and 972b (represented as ref A andrepresent temperature estimates of an identified region in the ref B). The mirrors 975a and 975b can be disposed away
scene. The abscissa in each of the plots of FIGS. 8A, 8B from the central FOV of the detector arrays 1, 5, 9 such thatindicates the number of a FPA, while the ordinate corre 55 the central FOV is not blocked or obscured by the image ofsponds to temperature (in degrees C.). Accordingly, it is the reference source 972a and 972b. In various implemen
expected that when all detector elements receive radiant data tations, the FOV of the detector array 5 could be greater thanthat, when interpreted as the objects temperature, indicates the FOV of the detector arrays 1 and 9. In such implementhat the object's temperature perceived by all detector ele tations, the mirrors 975a and 975b can be disposed away
ments is the same, any given curve would be a substantially 60 from the central FOV of the detector array 5 at a location
flat line. Data corresponding to each of the multiple “obj” such that the reference source 972a and 972b is imaged bycurves are taken from a stream of video frames separated the detector array 5. The mirrors 975a and 975b mayfrom one another by about 0.5 seconds (for a total of 50 comprise imaging optical elements having optical power
frames). The recorded “obj' curves shown in FIG. 8A that image the reference sources 972a and 972b onto theindicate that the detector elements disagree about the 65 detector arrays 1 and 9. In this example, the reference
objects temperature, and that difference in objects tem sources 972a and 972b can be disposed in the same plane asperature perceived by different detector elements is as high the re-imaging lenses 928a, however, the reference sourcesUS 9,599,508 B2
51
52
In
the
implementations
depictedin FIGS. 9A and 9B, the972a and 972b can be disposed in a different plane or indifferent locations. For example, the reference sources 972a detector arrays 1, 5 and 9 are configured to view (or image)and 972b can be disposed in a plane that is conjugate to the both the reference sources 972a and 972b. Accordingly,
plane in which the detector array 1, detector array 5, and multiple frames (e.g., every or Substantially every frame)
detector array 9 are disposed Such that a focused, image of 5 within a sequence of images contains one or more regions in
the reference sources 972a and 972b is formed by the the image in which the object image has known thermal anddetector arrays. In some implementations, the reference spectral properties. This allows multiple (e.g., most or each)sources 972a and 972b can be disposed in a plane that is cameras within the array of cameras to be calibrated to agree
spaced apart from the conjugate plane Such that a defocused with other (e.g., most or every other) camera imaging theimage of the reference sources 972a and 972b is formed by 10 same reference source(s) or Surface(s). For example, detec
the detector arrays. In various implementations, the refer tor arrays 1 and 9 can be calibrated to agree with each other.
ence sources 972a and 972b need not be disposed in the As another example, detector arrays 1, 5 and 9 can besame plane.
calibrated to agree with each other. In various embodiments,
the lenses 928a provide blurred (or defocused) images of the
As discussed above, in some embodiments, the reference
sources 972a and 972b are imaged onto the detector array 1 15 reference sources 972a, 972b on the detector arrays 1 and 9
and detector array 9, without much blur such that the because the location of the reference Sources are not exactly
in a conjugate planes of the detector arrays 1 and 9. Althoughreference sources 972a and 972b are focused. In contrast, inother embodiments, the image of the formed of reference the lenses 928a are described as providing blurred or defo
sources 972a and 972b on the detector array 1, and detector cused images, in various embodiments, reference sources orarray 9 are blurred such that the reference sources 972a and Surfaces are imaged on the detectors arrays 1, 5, 9 without
972b are defocused, and thereby provide some averaging,
Such blur and defocus and instead are focused images.
Smoothing, and/or low pass filtering. The reference sources Additionally optical elements may be used. Such as for
972a and 972b may comprise a surface of known tempera example, the mirrors shown in FIG. 9A to provide such
ture and may or may not include a heater or cooler attached focused images.
thereto or in thermal communication therewith. For 25
The temperature of the reference sources 972b, 972a can
example, the reference source 972a and 972b may comprises be different. For example, the reference source 972a can beheaters and coolers respectively or may comprise a surface at a temperature T, and the reference source 972b can be atwith a temperature sensor and a heater and sensor respec a temperature T. lower than the temperature T. A heater cantively in direct thermal communication therewith to control be provided under the temperature-controlled element 972a
the temperature of the reference surface. In various imple 30 to maintain it at a temperature T, and a cooler can beprovided underneath the temperature-controlled element
mentations, the reference sources 972a and 972b can include
a temperature controller configured to maintain the reference 972b to maintain it at a temperature T. In various imple
sources 972a and 972b at a known temperature. In some mentations, the embodiments illustrated in FIGS. 9A and 9B
implementations, the reference sources 972a and 972b can can be configured to image a single reference source 972
be associated with one or more sensors that measure the 35 instead of two references sources 972a and 972b maintained

temperature of the reference sources 972a and 972b and at different temperatures. It is understood that the single
communicate the measured temperature to the temperature reference source need not be thermally controlled. For
controller. In some implementations, the one or more sen example, in various implementations, a plurality of detectors
sors can communicate the measured temperature to the in the detector array can be configured to image a samedata-processing unit. In various implementations, the refer 40 surface of at least one calibration element whose thermal
ence sources 972a and 972b may comprise a surface of and spectral properties are unknown. In Such implementa
unknown temperature. For example, the reference sources tions, one of the plurality of detectors can be configured as
may comprise a wall of a housing comprising the imaging a reference detector and the temperature of the surface of thesystem. In some implementations, the reference sources at least one calibration element imaged by the plurality of
972a and 972b comprising a surface need not be associated 45 detectors can be estimated using the radiance spectrumwith sensors, temperature controllers. However, in other obtained by the reference detector. The remaining plurality
implementations, the reference sources 972a and 972b com of detectors can be calibrated such that their temperature
prising a Surface can be associated with sensors, temperature and/or spectral measurements agree with the reference
controllers.
detector. For example, detector arrays 1 and 9 can be
In FIG. 9B, the temperature-calibration elements com 50 calibrated to agree with each other. As another example,
prise temperature-controlled elements 972a and 972b (e.g., detector arrays 1, 5 and 9 can be calibrated to agree witha thermally controlled emitter, a heating strip, a heater or a each other.
The reference sources 972a and 972b can be coated withcooler) disposed a distance from the detector arrays 1, 5, 9.
In various embodiments, the temperature-controlled ele a material to make it behave substantially as a blackbody
ments 972a and 972b can be disposed away from the central 55 (for which the emission spectrum is known for any given
FOV of the detector arrays 1, 5, 9 such that the central FOV temperature). If a temperature sensor is used at the location
is not blocked or obscured by the image of the reference of each reference source, then the temperature can be
Source 972a and 972b. The radiation emitted from thetracked at these locations. As a result, the regions in thereference sources 972a and 972b is also imaged by the image of each camera (e.g., on the detector arrays 1 and 9)
detector array 936 along with the radiation incident from the 60 in which the object has such known temperature (and,object. Depending on the position of the reference sources therefore, spectrum) can be defined. A calibration procedure
972a and 972b the image obtained by the detector array of can thus be used so that most of the cameras (if not every
the reference sources can be blurred (or defocused) or sharp camera) so operated agrees, operationally, with most or(or focused). The images 980a,980b, 980c, 980d, 980e and every other camera, for objects at the temperatures repre
980fof the temperature-controlled elements 972a and 972b 65 sented by those two sources. Calibrating infrared cameras
can be used as a reference to dynamically calibrate the one using sources at two different temperatures is known as a
“two-point calibration, and assumes that the measured
or more cameras in the array of cameras.

US 9,599,508 B2
53
54
signal at a given pixel is linearly related to the incident implementations, the one or more mirrors Mo....M. andirradiance. Since this calibration can be performed during Mo, . . . M can be placed outside the central FOV of themultiple, more, or even every frame of a sequence, it is cameras 5 and 6 and is placed in a peripheral FOV of thereferred to as a “dynamic calibration'.
cameras outer ring of cameras 0, 1, 2, 3, 7, 11, 10, 9, 8 andAn example of the dynamic calibration procedure is as 5 4 which does not overlap with the central FOV of thefollows. If there is a temperature sensor on the reference cameras 5 and 6 such that the reference sources A and B are
Sources or reference Surface, then the temperature measure not imaged by the cameras 5 and 6. In various implemenments obtained by these temperature sensors can be used to tations, the FOV of the central cameras 5 and 6 can be
determine their expected emission spectra. These tempera greater than the FOV of the outer ring of cameras 0, 1, 2, 3,
ture measurements are labeled as TR, TIR, and TR 10 7, 11, 10, 9, 8 and 4. In such implementations, the one orfor the “reference temperatures” of sources/surfaces A, B, more mirrors Mo. . . . M. and Moz. . . . M. can be
and C. These temperature measurements can be used as placed in a peripheral FOV of the cameras 5 and 6 which
Scalar correction factors to apply to the entire image of a does overlap with the central FOV of the outer ring ofgiven camera, forcing it to agree with the reference tem cameras 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4 such that the reference
peratures. Correcting the temperature estimate of a given 15 Sources A and B are imaged by the cameras 5 and 6.
pixel from T to T' can use formulae analogous to thoseThis design is an enhancement to the systems 300 and 400
discussed below in reference to FIGS. 10A, 10B, 10C. If noshown in FIGS. 3A and 4A. In the system 1000 shown in
direct temperature sensor is used, then one of the cameras FIG. 10A, an array of two or more imaging elements (curved
can be used instead. This camera can be referred to as themirrors, for example) is installed at a distance from theFPAS, for example, in the plane of the reference aperture 160“reference camera'. In this case, the same formulae as those
provided in paragraph 0340 below can be used, but with shown in FIG. 3A. These elements (mirror or imaging
TRI and TR representing the temperatures of the ref elements) are used to image one or more temperature
erence sources/surfaces A and B as estimated by the refer controlled reference sources A and B onto the detector
ence camera. By applying the dynamic calibration correc elements of two or more of the cameras. The primarytion formulae, all of the other cameras are forced to match the 25 difference between embodiment 1000 and embodiment 300

temperature estimates of the reference camera.
or 400 is that now a plurality or most or all of the outer ring
In the configuration illustrated in FIG. 9B, the reference of cameras in the array can image both the reference sources
sources 972a and 972b are placed such that the images of the A and B instead of imaging one of the two reference source
Sources on the detector arrays are blurred. The configuration A and B. Accordingly, most or all of the outer ring of
illustrated in FIG. 9A is similar to the system 400 illustrated 30 cameras image an identical reference source or an identicalin FIG. 4 where the reference sources are placed at an set of reference sources (e.g., both the reference Sources Aintermediate image plane (e.g., a conjugate image plane). In and B) rather than using different reference sources forthis configuration, the array of reference apertures, similar to different cameras or imaging different portions of the refer
reference apertures 438a in FIG. 4, will have an accompa ence sources as shown in FIGS. 3A and 4A. Thus, thisnying array of reference Sources or reference Surfaces Such 35 approach improves the robustness of the calibration, as itthat the reference sources or Surfaces (e.g., each reference eliminates potential failures and errors due to the having
Source or Surface) are imaged onto a camera or a detector additional thermal sensors estimating each reference source.
The imaging elements in the system 1000 (shown asarray such as FPAS 1, 5,9. With this approach, the reference
Source or Surface images are at a conjugate image plane and mirrors in FIGS. 10A and 10B) image one or more conthus are not appreciably blurred, so that their images can be 40 trolled-temperature reference sources or a surface of amade to block a smaller portion of each camera's field of calibration element (shown as A and B in FIGS. 10A andview.
10B) into the blocked region of the cameras fields of view.
A “static' calibration (a procedure in which the scene is FIG. 10B shows an example in which mirror Mo images
largely blocked with a reference source such as the moving reference Source/surface A onto camera 0, and mirror Mo
shutters 960 in FIGS. 9A and 9B, so that imaging of a 45 images reference source/surface B onto camera 0, andunknown scene cannot be performed in parallel with cali likewise for cameras 1, 2, and 3. This way, each of thebration) allows a plurality of the cameras (for example, most mirrors is used to image a reference source/surface onto a
or each camera) to accurately estimate the temperature of a detector array of a camera, so that many, most, or every
plurality of elements (for example, most or each element in frame within a sequence of images contains one or more
the scene) immediately after the calibration is complete. It 50 regions in the image in which the object image has known
cannot, however, prevent the cameras estimates from drift thermal and spectral properties. This approach allows mosting away from one another during the process of imaging an of the camera, if not each camera, within the array of
unknown scene. The dynamic calibration can be used to cameras to be calibrated to agree with most or every otherreduce or prevent this drift, so that all cameras imaging a camera imaging the same reference source or sources. For
scene can be forced to agree on the temperature estimate of 55 example, cameras 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4 can be
the reference sources/surfaces, and adjust this correction calibrated to agree with each other. As another example,
during every frame.
cameras 0, 1, 2 and 3 can be calibrated to agree with each
FIG. 10A illustrates schematically a related embodiment other. As yet another example, cameras 0, 1, 2, 3, 7, 11, 10.
1000 of the imaging system, in which one or more mirrors 9, 8, 4, 5 and 6 can be calibrated to agree with each other.
Mo, ... M. and Mo, ... Mare placed within the fields 60 Accordingly, in various implementations, two, three, four,of view of one or more cameras 0,..., 11, partially blocking five, six, seven, eight, nine, ten, eleven or twelve cameras
the field of view. The cameras 0, ... 11 are arranged to form can be calibrated to agree with each other. In certain
an outer ring of cameras including cameras 0, 1, 2, 3, 7, 11, embodiments, however, not all the cameras are calibrated to10, 9, 8 and 4 surrounding the central cameras 5 and 6. In agree with each other. For example, one, two, or morevarious implementations, the FOV of the central cameras 5 65 cameras may not be calibrated to agree with each other while
and 6 can be less than or equal to the FOV of the outer ring others may be calibrated to agree with each other. In various
embodiments, these mirrors may be configured to image theof cameras 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In suchUS 9,599,508 B2
55
reference sources/surfaces A and B onto different respective
pixels a given FPA. Without any loss of generality, FIGS.
10A and 10B represent a top view of the embodiment shown

56
The dynamic calibration is used to obtain a corrected
temperature T from the initial temperature T estimated ateach pixel in a camera using the following formulae:

in FIG. 9A.

FIG. 10C illustrates schematically a related embodiment1050 of the imaging system, in which one or more reference where is TR is a dynamic offset correction factor, and,Sources Roe. . . . . R1 and Rog. . . . . R11 are disposed
around the array of detectors 0, . . . , 11. In various
Te(R) - TAR
implementations, the one or more reference sources
Glel =
Ti,
Ro4. . . . . R114 and Roz. . . . . R11 can be a single reference 10
Source that is images by the detectors 0, . . . , 11. In various
implementations, central detector arrays 5 and 6 can have a is a dynamic gain correction factor. The term c discussed
FOV that is equal to or lesser than the FOV of the outer ring above is a camera index that identifies the camera whose
data is being corrected.
of the detectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In such
implementations, the reference sources Rol, . . . . R11 can 15 References throughout this specification to “one embodi
be disposed away from the central FOV of the detector ment,” “an embodiment,” “a related embodiment,” or simi
arrays 5 and 6 such that the radiation from the reference lar language mean that a particular feature, structure, orSources R. . . . . R. is imaged only by the outer ring of characteristic described in connection with the referred todetectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In various"embodiment' is included in at least one embodiment of theimplementations, central detector arrays 5 and 6 can have a
FOV that is greater than the FOV of the outer ring of thedetectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In such imple
mentations, the reference sources R. . . . . R can bedisposed in the peripheral FOV of the detector arrays 5 andpresent invention. Thus, appearances of the phrases "in one
embodiment,” “in an embodiment,” and similar language
throughout this specification may, but do not necessarily, allrefer to the same embodiment. It is to be understood that no

portion of disclosure, taken on its own and in possible
6 such that the radiation from the reference sources 25 connection with a figure, is intended to provide a complete
Rot, ..., R is imaged only by the outer ring of detectors description of all features of the invention.
In the drawings like numbers are used to represent the0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. The radiation from thereference sources R. . . . . R. is therefore imaged by the same or similar elements wherever possible. The depicted
outer ring of detectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4 as well structural elements are generally not to Scale, and certain
as central cameras 5 and 6. Without any loss of generality, 30 components are enlarged relative to the other components
FIG. 10C represents a top view of the embodiment shown in for purposes of emphasis and understanding. It is to be
FIG. 9B.
understood that no single drawing is intended to support a
In various implementations, one can use a heater under complete description of all features of the invention. In otherneath, adjacent to, or in thermal communication with refer words, a given drawing is generally descriptive of only
ence source/surface A to give it a higher temperature T, and 35 Some, and generally not all, features of the invention. A
a cooler underneath, adjacent to, or in thermal communica given drawing and an associated portion of the disclosuretion with reference source B to give it a lower temperature containing a description referencing Such drawing do not,T. In various implementations, the embodiments illustrated generally, contain all elements of a particular view or all
in FIGS. 10A, 10B and 10C can be configured to image a features that can be presented is this view, for purposes ofsingle reference source A instead of two references sources 40 simplifying the given drawing and discussion, and to direct
A and B maintained at different temperatures. As discussed the discussion to particular elements that are featured in thisdrawing. A skilled artisan will recognize that the invention
above, the embodiments illustrated in FIGS. 10A, 10B and
10C can be configured to image a same Surface of a may possibly be practiced without one or more of thecalibration element. In Such implementations, the tempera specific features, elements, components, structures, details,
ture of the surface of the calibration element need not be 45 or characteristics, or with the use of other methods, com

known. Many, most or each reference source/surface can be ponents, materials, and so forth. Therefore, although a
coated with a material to make it behave approximately as particular detail of an embodiment of the invention may nota blackbody, for which the emission spectrum is known for be necessarily shown in each and every drawing describing
any given temperature. If many, most, or each camera in the Such embodiment, the presence of this detail in the drawing
array of cameras images both of references A and B, so that 50 may be implied unless the context of the description requires
there are known regions in the image of these cameras in otherwise. In other instances, well known structures, details,
which the object has a known temperature (and therefore materials, or operations may be not shown in a given
spectrum). Then one can perform a calibration procedure. drawing or described in detail to avoid obscuring aspects ofThis procedure can provide that many, most or every camera an embodiment of the invention that are being discussed.
so operated agrees with various, most, or every other cam 55 Furthermore, the described single features, structures, orera, for objects at the temperatures represented by those two characteristics of the invention may be combined in any
suitable manner in one or more further embodiments.
Sources. For example, two, three, four, five, six, seven, eight,
nine, ten, eleven or twelve cameras can be calibrated to
The features recited in claims appended to this disclosureagree with each other. In certain embodiments, however, not are intended to be assessed in light of the disclosure as a
all the cameras are calibrated to agree with each other. For 60 whole, including features disclosed in prior art to which
example, one, two, or more cameras may not be calibrated reference is made.
At least some elements of a device of the invention can beto agree with each other while others may be calibrated toagree with each other. As discussed above, calibration of controlled—and at least some steps of a method of theinfrared cameras using sources at two different temperatures invention can be effectuated, in operation—with a programis known as a “two-point calibration, and assumes that the 65 mable processor governed by instructions stored in ameasured signal at a given pixel is linearly related to the memory. The memory may be random access memory
incident irradiance.
(RAM), read-only memory (ROM), flash memory or any

US 9,599,508 B2
57
other memory, or combination thereof. Suitable for storing
control software or other instructions and data. Those skilled

58
puter-readable media. Additionally, the operations of amethod or algorithm may reside as one or any combinationin the art should also readily appreciate that instructions or or set of codes and instructions on a machine readable
programs defining the functions of the present invention medium and computer-readable medium, which may be
may be delivered to a processor in many forms, including, incorporated into a computer program product.
but not limited to, information permanently stored on non
Various modifications to the implementations described in
Writable storage media (e.g. read-only memory devices this disclosure may be readily apparent to those skilled in thewithin a computer, such as ROM, or devices readable by a art, and the generic principles defined herein may be appliedcomputer I/O attachment, such as CD-ROM or DVD disks), to other implementations without departing from the spirit orinformation alterably stored on Writable storage media (e.g. 10 Scope of this disclosure. Thus, the claims are not intended to
floppy disks, removable flash memory and hard drives) or be limited to the implementations shown herein, but are to
information conveyed to a computer through communica
tion media, including wired or wireless computer networks. be accorded the widest scope consistent with this disclosure,
In addition, while the invention may be embodied in soft the principles and the novel features disclosed herein.
ware, the functions necessary to implement the invention 15 Certain features that are described in this specification in
may optionally or alternatively be embodied in part or in the context of separate implementations also can be imple
whole using firmware and/or hardware components, such as mented in combination in a single implementation. Con
combinatorial logic, Application Specific Integrated Circuits versely, various features that are described in the context of(ASICs), Field-Programmable Gate Arrays (FPGAs) or a single implementation also can be implemented in multiple
implementations separately or in any suitable Subcombina
other hardware or some combination of hardware, softwaretion. Moreover, although features may be described above as
and/or firmware components.
While examples of embodiments of the system and acting in certain combinations and even initially claimed as
method of the invention have been discussed in reference toSuch, one or more features from a claimed combination can

the gas-cloud detection, monitoring, and quantification (in in Some cases be excised from the combination, and thecluding but not limited to greenhouse gases Such as Carbon 25 claimed combination may be directed to a Subcombination
Dioxide, Carbon Monoxide, Nitrogen Oxide as well as or variation of a Subcombination.
hydrocarbon gases such as Methane, Ethane, Propane, n-Butane, iso-Butane, n-Pentane, iso-Pentane, neo-Pentane,

What is claimed is:on or transmitted over as one or more instructions or code on

IR radiation,

1. An infrared (IR) imaging system for identifying target
Hydrogen Sulfide, Sulfur Hexafluoride, Ammonia, Benzene,
p- and m-Xylene, Vinyl chloride, Toluene, Propylene oxide, 30 species in an object, the imaging system comprising:an optical system including an optical focal plane array
Propylene, Methanol, Hydrazine, Ethanol, 1,2-dichloroeth
ane, 1,1-dichloroethane, Dichlorobenzene, Chlorobenzene,(FPA) unit, the FPA unit comprising a microbolometer,
the optical system configured to receive IR radiation
to name just a few), embodiments of the invention can bereadily adapted for other chemical detection applications.
from the object along at least two optical channels
defined by components of the optical system, said at
For example, detection of liquid and solid chemical spills, 35
biological weapons, tracking targets based on their chemical
least two optical channels being spatially and spectrallycomposition, identification of satellites and space debris,different from one another;
ophthalmological imaging, microscopy and cellular imag
a movable temperature reference at a known temperature,ing, endoscopy, mold detection, fire and flame detection, andthe movable reference configured to be inserted into an
optical path from the object to the FPA unit along at
pesticide detection are within the scope of the invention. 40
As used herein, a phrase referring to “at least one of a list
least one of the two optical channels to block IR
radiation and configured to be removed from the opticalof items refers to any combination of those items, including
path; andsingle members. As an example, “at least one of: a, b, or c'a processor configured to acquire multispectral opticalis intended to cover: a, b, c, a-b, a-c, b-c, and a-b-c.
data representing said target species from the received
If implemented in software, the functions may be stored 45
wherein the processor is configured to map an overall
a computer-readable medium. The steps of a method oralgorithm disclosed herein may be implemented in a pro
image datacube representing spatial distribution of con
centrations c of the target species.
cessor-executable Software module which may reside on acomputer-readable medium. Computer-readable media 50 2. The system of claim 1, wherein the FPA unit does notincludes both computer storage media and communication require cooling during normal operation.
media including any medium that can be enabled to transfer
3. The system of claim 1, wherein the IR radiation
a computer program from one place to another. A storage comprises wavelengths between about 1 micron and about
media may be any available media that may be accessed by 20 microns.
a computer. By way of example, and not limitation, Such 55 4. The system of claim 1, wherein the system is config
computer-readable media may include RAM, ROM, ured to identify target species in a gas cloud.
EEPROM, CD-ROM or other optical disk storage, magnetic
5. The system of claim 1, further comprising at least two
disk storage or other magnetic storage devices, or any other spectral filters, each filter positioned in a respective one of
medium that may be used to store desired program code in said optical channels.
the form of instructions or data structures and that may be 60 6. The system of claim 5, wherein the spectral filters
accessed by a computer. Also, any connection can be prop comprise at least one long pass (LP) filter.
erly termed a computer-readable medium. Disk and disc, as
7. The system of claim 5, wherein the spectral filters
used herein, includes compact disc (CD), laser disc, optical comprise at least one short pass (SP) filter.
disc, digital versatile disc (DVD), floppy disk, and blu-ray
8. The system of claim 5, further comprising at least twodisc where disks usually reproduce data magnetically, while 65 reimaging lenses, each reimaging lens positioned to transmit
discs reproduce data optically with lasers. Combinations of a portion of the IR radiation transmitted through a respective
the above also may be included within the scope of com one of said spectral filters towards the optical FPA unit.

US 9,599,508 B2
59
9. The system of claim 1, further comprising an opticalaperture that circumscribes said at least two spatially andspectrally distinct optical channels.
10. The system of claim 1, wherein the movable tempera
ture reference comprises a first shutter at a first temperature
and a second shutter at a second temperature.
11. The system of claim 1, wherein the movable tempera
ture reference comprises a system having a first portion at afirst temperature and a second portion at a second tempera 10

ture.

60
inserting a movable temperature reference into an optical
path from the object to the FPA unit along at least oneof the two optical channels to block IR radiation,
wherein the movable temperature reference source isconfigured to be removed, and wherein the movabletemperature reference source is at a known tempera
ture; and

under the control of an electronic processor, mapping an
overall image datacube representing spatial distributionof concentrations c of the target species.
15. The method of claim 14, wherein the FPA unit does

not require cooling during normal operation.
16. The method of claim 14, wherein transmitting thetemperature.
received IR radiation includes transmitting the received IR
13. The system of claim 1, wherein the processor is 15 radiation through at least two spectral filters, each spectral
configured to acquire multispectral optical data in a single filter positioned in a respective one of said optical channels.
17. The method of claim 16, wherein each spectral filter
occurrence of data acquisition.
one of a longpass optical filter or a shortpass optical
14. A method of operating an infrared (IR) imaging includes
filter.
System, comprising:
18. The method of claim 14, further comprising acquiring,receiving IR radiation from an object along at least two multispectral
optical data representing the object from the IR
optical channels defined by components of an opticalradiationtransmitted
towards the optical FPA unit in a single
system of the IR imaging system, said at least twoof data acquisition with a programmable proces
optical channels being spatially and spectrally different occurrenceSO.
from one another;method of claim 14, further comprising calibrat
transmitting the received IR radiation towards an optical 25 ing19.theThe
FPA
unit based on the known temperature.
focal plane array (FPA) unit, the FPA unit comprising
12. The system of claim 1, wherein the processor isconfigured to calibrate the FPA unit based on the known

a microbolometer;

k

k

k

k

k