USOO9562849B2(12) United States Patent(10) Patent No.:US 9,562,849 B2(45) Date of Patent:
Feb. 7, 2017

Kester et al.
(54)

(56)

DIVIDED-APERTURE INFRA-RED
SPECTRAL MAGING SYSTEM

U.S. PATENT DOCUMENTS(71) Applicant: REBELLION PHOTONICS, INC.,
Houston, TX (US)4,134,683 A(73) Assignee: Rebellion Photonics, Inc., Houston, TX(US)(*) Notice: Subject to any disclaimer, the term of thispatent is extended or adjusted under 35
U.S.C. 154(b) by 0 days.
(21) Appl. No.: 14/538,827(65)(Continued)
FOREIGN PATENT DOCUMENTS
CACA

2 365 866
2870 419(Continued)
OTHER PUBLICATIONSAllen et al., “Measurements of Methane Emissions at Natural Gas
Production Sites in the United States', PNAS, Oct. 29, 2013, vol.
110, No. 44, pp. 7.

Prior Publication Data

& Bear LLP

May 21, 2015

(57)

Related U.S. Application Data
(60) Provisional application No. 61/903,075, filed on Nov.
12, 2013.

(51)

Int. C.

HOIL 25/00
GOIN 2L/3504(2006.01)(2014.01)(Continued)(52)

U.S. C.

CPC ......... G0IN 21/3504 (2013.01); G0IJ 3/0208(2013.01); G0IJ 3/0232 (2013.01);(Continued)(58)

Field of Classification Search

CPC .......... H04N 5/33; H04N 3/09; H04N 5/3655;
HO4N 5/3651

ABSTRACT

Various embodiments disclosed herein describe a divided

aperture infrared spectral imaging (DAISI) system that is
adapted to acquire multiple IR images of a scene with asingle-shot (also referred to as a Snapshot). The plurality ofacquired images having different wavelength compositions
that are obtained generally simultaneously. The system
includes at least two optical channels that are spatially andspectrally different from one another. Each of the at least two
optical channels are configured to transfer IR radiation
incident on the optical system towards an optical FPA unit
comprising at least two detector arrays disposed in the focal
plane of two corresponding focusing lenses. The systemfurther comprises at least one temperature reference Sourceor surface that is used to dynamically calibrate the two
detector arrays and compensate for a temperature difference
between the two detector arrays.

See application file for complete search history.

66 Claims, 15 Drawing Sheets

to object975a-M960

9, 2000
5/2O15(Continued)
Primary Examiner — Kiho Kim(74) Attorney, Agent, or Firm — Knobbe Martens Olson

Nov. 12, 2014

US 2015/O136982 A1

1/1979 Goetz et al.

4,963,963 A * 10/1990 Dorman ........................ 348, 164(72) Inventors: Robert Timothy Kester, Houston, TX(US); Nathan Adrian Hagen, Houston,
TX (US)(22) Filed:

References Citedythermally.
controledsitte's
ca:

tock

928;

US 9,562,849 B2
Page 2(51) Int. Cl.

FOREIGN PATENT DOCUMENTS

H04N 5/33
H04N 3/09(2006.01)(2006.01)

EPEP

O 837 600
2871 452

4f1998
5/2O15: 2: 3.

1.58:

H04N 5/365(2011.01)

E.

GOI 3/02
GOI 3/36(2006.01)(2006.01)

GOL 5/52
GOI 3/28(2006.01)(2006.01)

GOL 5/00
GOL 5/08(2006.01)(2006.01)

t :

C

GB
WO
WO
WO
WO
WO
WO
WO
WO

GOIN 2 1/17

(2006.01)

OTHER PUBLICATIONS

CPC ........... G0IJ 3/0256 (2013.01); G0IJ 3/0297

Alvarez et al., “Greater Focus Needed on Methane Leakage fromNatural Gas Infrastructure', PNAS, Apr. 24, 2012, vol. 109, No. 17.
pp. 12.
Bedard et al., “Image Mapping Spectrometry: Calibration andCharacterization”. Optical Engineering, Nov. 2012, vol. 51, No. 11,
pp. 111711-1-111711-13.
Ben-David et al., “Probability Theory for 3-Layer Remote Sensing
Radiative Transfer Model: Univariate Case,” Optics Express, Apr.
2012, vol. 20, No. 9, pp. 10004-10033.
Ben-David et al., “Probability Theory for 3-Layer Remote Sensing
Radiative Transfer Model: Errata,” Optics Express, May 20, 2013,vol. 21, No. 10, pp. 11852.
Brady et al., “Multiscale Lens Design”. Optics Express, Jun. 22.
2009, vol. 17, No. 13, pp. 10659-10674.
Caulton et al., “Toward a Better Understanding and Quantificationof Methane Emissions from Shale Gas Development', PNAS, Apr.
29, 2014, vol. 111, No. 17, pp. 7.
Chen et al., “Quantitative Sectioning and Noise Analysis for Structured Illumination Microscopy: Erratum'. Optics Express, Oct. 19.
2015, vol. 23, No. 21, pp. 27633-27634.
Chidley et al., “Flow-Induced Birefringence: The Hidden PSF
Killer in High Performance Injection-Molded Plastic Optics'.
Endoscopic Microscopy, Proceedings of SPIE vol. 6082, 2006, pp.

(52) U.S. Cl.

(2013.01); G0IJ 3/26 (2013.01); G0IJ 3/28
(2013.01); G0IJ 3/2823 (2013.01); G0IJ 3/36
(2013.01); G0IJ 5/0014 (2013.01); G0IJ
5/0806 (2013.01); G0IJ 5/0834 (2013.01);
G0IJ 5/0862 (2013.01); G0IJ 5/20 (2013.01);
G0IJ 5/522 (2013.01); H04N 3/09 (2013.01);
H04N 5/33 (2013.01); H04N 5/365 (2013.01);
H04N 5/3651 (2013.01); H04N 5/3655(2013.01); G0IJ 2003/2826 (2013.01); G0IJ
2005/0077 (2013.01); G0IN 2021/1793(2013.01); G0IN 202 1/3531 (2013.01); G0IN
2201/068 (2013.01); G0IN 2201/06106(2013.01)(56)

References Cited
U.S. PATENT DOCUMENTS
5,127,742 A

7, 1992 Fraden

5,877,500 A

3/1999 Braig et al.

6,023,061 A
6,268,883 B1

2/2000 Bodkin
7/2001 Zehnder et al.

6,456,261 B1

9/2002 Zhang

6,680,778 B2
7,242.478 B1
7,315,377 B2
7,750,802 B1

1/20047/20071/2008
7, 20105,920,066 A *5,926,283 A

7/1999 DiRenzo et al. .......... 250/252.1
7/1999 Hopkins
Hinnrichs et al.
Dombrowski et al.
Holland et al.
Parish et al.

7,835,002 B2 11/2010 Muhammed et al.
7,888,624 B1
2/2011 Murguia et al.
8,159,568 B2 * 4/2012 Ahdoot ......................... 348,241
8,653,461 B1

2/2014 Benson et al.

8,654,328 B2
2002/O121370 A1

2/2014 Tkaczyk et al.
2/2002 Gorin
9/2002 Kurkjian et al.

2002fOO15151 A1

2002fO159101 A1* 10, 2002 Alderson et al. ............. 358/.504

2003/0134426 A1

7/2003 Jiang et al.

2005, 0103989 A1*

5, 2005 Watson et al. ............. 250/252.1

2006, OO44562 A1

3/2006 Hagene et al.

2008/0204744 A1
2008, 0231719 A1

8, 2008 Mir et al.
9, 2008 Benson et al.

2008/025 1724 A1

10/2008 Baliga et al.

2012/0273680 A1

1 1/2012 Furry

2010/0211333 A1
2011/0261321 A1

2013, O181836 A1
2013/0321806 A1
2013/0341509 A1
2015,0069.239 A1
2015,0136981 A1
2015,0316473 A1
2016/009.7713 A1
2016/0349228 A1

8, 2010 Pruet et al.
10/2011 Ramella-Roman et al.

7, 2013
12/2013
12/20133/2015
5, 2015
11/2015
4/2016
12/2016

Cardoso et al.
Kester et al.
Nelson et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.
Kester et al.

WO

1014769 A * 12, 1965 .............. GO1J 5,505
WO 2004/097389
11 2004
WO 2007/OO8826
1, 2007
WO 2008/109183
9, 2008
WO 2009/094782
8, 2009
WO 2010/053979
5, 2010
WO 2012/078417
6, 2012
WO 2012/082366
6, 2012
WO 2013, 173541
11 2013

WO 2016, 196224

12/201611.

Chu et al., “The NIST Quantitiative Infrared Database'. Journal of
Research of the National Institute of Standards and Technology,
Jan.-Feb. 1999, vol. 104, No. 1, pp. 59-81.
DiPietro et al., “Hyperspectral Matched Filter with False-Alarm
Mitigation”. Optical Engineering, Jan. 2012, vol. 51, No. 1, pp.
O162O2-1-0162O2-7.

Eriksson et al., “Radiative Cooling Computed for Model Atmo
spheres'. Applied Optics, Dec. 1, 1982, vol. 21, No. 23, pp.
4381-4388.

Flanigan, “Detection of Organic Vapors with Active and Passive
Sensors: A Comparison.” Applied Optics, 1986, vol. 25, No. 23, pp.
4253-4260.

Gallagher et al., “Error Analysis for Estimation of Trace Vapor
Concentration Pathlength in Stack Plumes”. Applied Spectroscopy,
2003, vol. 57, No. 6, pp. 614-621.
Gallagher et al., “Estimation of Trace Vapor Concentration
Pathlength in Plumes for Remote Sensing Applications from
Hyperspectral Images'. Analytica Chimica Acta, 2003, vol. 490, pp.
139-152.

Gao et al., “Compact Image Slicing Spectrometer (ISS) for
Hyperspectral Fluorescence Microscopy”. Optics Express, Jul. 20.
2009, vol. 17, No. 15, pp. 12293-12308.
Gao et al., “Depth-Resolved Image Mapping Spectrometer (IMS)with Structured Illumination”. Optics Express, Aug. 29, 2011, vol.
19, No. 18, pp. 17439-17452.
Gao et al., “Optical Design of a Snapshot High-Sampling Image
Mapping Spectrometer (IMS) for Hyperspectral Microscopy”.
Three-Dimensional and Multidimensional Microscopy:Image
Acquisition and Processing XVII, Proceedings of SPIE vol. 7570,
2010, pp. 1-7.

US 9,562,849 B2
Page 3(56)

References Cited
OTHER PUBLICATIONS

Gao et al., “Quantitative Comparison Between Full-Spectrum andFilter-Based Imaging in Hyperspectral Fluorescence Microscopy.
Journal of Microscopy, 2012, vol. 246, No. 2, pp. 113-123.
Gao et al., “Snapshot Image-Mapping Spectrometer for
Hyperspectral Fluorescence Microscopy”. Optics and Photonics
News, Nov. 2010, vol. 21, No. 12, p. 50.
Gao et al., “Snapshot Image Mapping Spectrometer (IMS) withHigh Sampling Density for Hyperspectral Microscopy”. Optics
Express, Jul. 5, 2010, vol. 18, No. 4, pp. 14330-14344.
Gerhart et al., “Detection and Tracking of Gas Plumes in LWIR
Hyperspectral Video Sequence Data, Algorithms and Technologies
for Multispectral, Hyperspectral, and Ultraspectral Imagery XIX.
2013, SPIE Proceedings vol. 8743, pp. 1-14.
Gittins, Christopher M., “Detection and Characterization of Chemi
cal Vapor Fugitive Emissions by Nonlinear Optimal Estimation:
Theory and Simulation'. Applied Optics, Aug. 10, 2009, vol. 48,
No. 23, pp. 4545-4561.
Goldberg et al., “Dual Band MWIR/LWIR Focal Plane Array Test
Results.” Army Research Lab, Adelphi, MD, Aug. 1999, pp. 18.
Golowich et al., “Performance Limits of LWIR Gaseous Plume

Quantification'. Algorithms and Technologies for Multispectral,Hyperspectral, and Ultraspectral Imagery XVII, 2011, Proceedingsof SPIE vol. 8048, pp. 1-12.
Griffin et al., “The Herschel—SPIRE Instrument and its In-Flight
Performance.” Astronomy and Astrophysics, Jul. 1, 2010, vol. 518.
7 pages.
Gross et al., “Remote Identification and Quantification of Industrial
Smokestack Effluents via Imaging Fourier-Transform Spectros
copy”. Environmental Science & Technology, 2010, vol. 44, No. 24.
pp. 9390-9397.
Gupta et al., “Miniature Snapshot Multispectral Imager,” OpticalEngineering, 2011, vol. 50, pp. 033203-1-033203-9.
Hagen et al., “Analysis of Computed Tomographic Imaging Spec
trometers. I. Spatial and Spectral Resolution'. Applied Optics, Oct.
1, 2008, vol. 47, No. 28, pp. F85-F95.
Hagen et al., “Coded Aperture DUV Spectrometer for Standoff
Raman Spectoscopy'. Next-Generation Spectroscopic Technolo
gies II, Proceedings of SPIE vol. 7319, 2009, pp. 1-10.
Hagen et al., “Compound Prism Design Principles, I'. Applied
Optics, Sep. 1, 2011, vol. 50, No. 25, pp. 4998-5011.
Hagen et al., “Compound Prism Design Principles, II: Triplet andJanssen Prisms”. Applied Optics, Sep. 1, 2011, vol. 50, No. 25, pp.

Hagen et al., “Snapshot Advantage: A Review of the Light Collec
tion Improvement for Parallel High-Dimensional Measurement
Systems.” Optical Engineering, Jun. 13, 2012, vol. 51, No. 11, p.
111702-1-111702-7.

Hagen et al., "Snapshot Mueller Matrix Spectropolarimeter' Optics
Letters, Aug. 1, 2007, vol. 32, No. 15, pp. 2100-2102.
Hagen et al., “Spectrally-Resolved Imaging of Dynamic TurbidMedia”. Multimodal Biomedical Imaging VI, Proceedings of SPIE
vol. 7892, 2011, pp. 1-7.
Hagen et al., “Video-Rate Spectral Imaging of Gas Leaks in the
Longwave Infrared,” Chemical, Biological, Radiological, Nuclear,and Explosives (CBRNE) Sensing XIV. May 29, 2013, SPIE
Proceedings vol. 8710, pp. 7.
Harley et al., “Remote Quantification of Smokestack Effluent Mass
Flow Rates Using Imaging Fourier Transform Spectrometry.”
Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE) Sensing XII, Apr. 25-29, 2011, SPIE Proceedings vol.
8018, pp. 1-13.
Hayden et al., “Determination of Trace-Gas Amounts in Plumes by
the Use of Orthogonal Digital Filtering of Thermal-Emission Spec
tra'. Applied Optics, Jun. 1, 1996, vol. 35. No. 16, pp. 2802-2809.
Hirsch et al., “Detection of Gaseous Plumes in IR Hyperspectral
Images Using Hierarchical Clustering”. Applied Optics, Sep. 1,
2007, vol. 46, No. 25, pp. 6368-6374.
Johnston et al., “A Real-Time FPGA Implementation of a BarrelDistortion Correction Aglorithm”. Projects, 2003, vol. 10, pp.
91-96.

Karion et al., “Methane Emissions Estimate from Airborne Mea
surements Over a Western United States Natural Gas Field', Geophysical Research Letters, 2013, vol. 40, pp. 4393-4397.
Keshava et al., “A Survey of Spectral Unmixing Algorithms'.
Lincoln Laboratory Journal, 2003, vol. 14, No. 1, pp. 55-78.
Kester et al., “A Real-Time Gas Cloud Imaging Camera for Fugitive
Emission Detection and Monitoring. Imaging and Applied Optics
Technical Digest, 2012, pp. 3.
Kester et al., “Development of Image Mappers for Hyperspectral
Biomedical Imaging Applications'. Applied Optics, Apr. 1, 2010,
vol. 49, No. 10, pp. 1886-1899.
Kester et al., “High Numerical Aperture Microendoscope Objective
for a Fiber Confocal Reflectance Microscope”. Optics Express, Mar.
5, 2007, vol. 15. No. 5, pp. 2409-2420.
Kester et al., “Low Cost, High Performance, Self-Aligning Minia
ture Optical Systems”. Applied Optics, Jun. 20, 2009, vol. 48, No.
18, pp. 3375-3384.
Kester et al., “Real-Time Snapshot Hyperspectral Imaging Endo
scope'. Journal of Biomedical Optics, May 2011, vol. 16, No. 5, pp.

5O12-5022.

O56005-1-05.6005-12.

Hagen et al., “Compound Prism Design Principles, III: Linear-in
Wavenumber and Optical Coherence Tomography Prisms”. Applied
Optics, Sep. 1, 2011, vol. 50, No. 25, pp. 5023-5030.
Hagen et al., “Fourier Methods of Improving Reconstruction Speed
for CTIS Imaging Spectrometers'. Imagin Spectrometry XII, Pro
ceedings of SPIE vol. 6661, 2007, pp. 11.
Hagen et al., “Foveated Endoscopic Lens”, Journal of Biomedical
Optics, Feb. 2012, vol. 17, No. 2, pp. 021104-1-021104-6.
Hagen et al., “Gaussian Profile Estimation in One Dimension'.
Applied Optics, Aug. 1, 2007, vol. 46, No. 22, pp. 5374-5383.
Hagen et al., “Gaussian Profile Estimation in Two Dimension'.
Applied Optics, Dec. 20, 2008, vol. 47, No. 36, pp. 6842-6851.
Hagen et al., “Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy”. Optics Express, Jan. 2, 2012,vol. 20, No. 1, pp. 403-413.
Hagen et al., “Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy: Errata'. Optics Express, Feb.
27, 2012, vol. 20, No. 5, pp. 5343.
Hagen et al., “Real-Time Quantitative Hydrocarbon Gas Imaging
with the Gas Cloud Imager (GCI)”. Proceedings of SPIE, vol. 8358,
Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE) Sensing XIII, May 1, 2012, 7 pages.
Hagen et al., “Review of Snapshot Spectral Imaging Technologies'.
Optical Engineering, Sep. 2013, vol. 52, No. 9, pp. 090901-1-

Kudenov et al., “Fourier Transform Channeled Spectropolarimetry
in the MWIR”. Optics Express, Oct. 1, 2007, vol. 15, No. 20, pp.

090901-23.

12792-12805.

Kudenov et al., “Snapshot Imaging Mueller Matrix Polarimeter
Using Polarization Gratings'. Optics Letters, Apr. 15, 2012, vol. 37.
No. 8, pp. 1367-1369.
Landau et al., “Design and Evaluation of an Ultra-Slim Objective
for in-vivo Deep Optical Biopsy”. Optics Express, Mar. 1, 2010,
vol. 18, No. 5, pp. 4758-4775.
Levi, Michael A., “Comment on Hydrocarbon Emissions Characterization in the Colorado Front Range: A Pilot Study by Gabrielle
Pétron et al.”. Journal of Geophysical Research, 2012, vol. 117. No.
D21203, pp. 1-5.
Levi, Michael A., “Reply to “Reply to Comment on “Hydrocarbon
Emissions Characterization in the Colorado Front Range—A Pilot
Study’ by Michael A. Levi' by Gabrielle Pétron et al., Journal ofGeophysical Research: Atmospheres, 2013, vol. 118, pp. 3044
3046.

Low et al., “Remote Sensing and Characterization of Stack Gases byInfrared Spectroscopy. An Approach by Using Multiple-Scan Inter
ferometry”. Environmental Science & Technology, Jan. 1967, vol.
1. No. 1, pp. 73-74.
Luo et al., “Fast Processing of Imaging Spectrometer Data Cube
Based on FPGA Design”, MIPPR 2007: Multispectral Image Pro
cessing, Proceedings of SPIE vol. 6787, pp. 7.

US 9,562,849 B2
Page 4(56)

References Cited
OTHER PUBLICATIONSManolakis et al., “Long-Wave Infrared Hyperspectral Remote Sens
ing of Chemical Clouds'. IEEE Signal Processing Magazine, Jul.
2014, vol. 31, No. 4, pp. 120-141.
Mathews, “Design and Fabrication of a Low-Cost, Multispectral
Imaging System.” Applied Optics, 2008, pp. F71-F76, vol. 47.
Naranjo et al., “IR Gas Imaging in an Industrial Setting.”
Thermosense XXXII, Published in SPIE Proceedings vol. 7661,
May 4, 2010, pp. 1-8.
Nguyen et al., “Snapshot 3D Optical Coherence Tomography Sys
tem using Image Mapping Spectrometer'. Biomedical Optics and
3D Imaging OSA, 2012, pp. 3.
Niu et al., “New Approach to Remote Gas-Phase Chemical Quan
tification: Selected-Band Algorithm”. Optical Engineering, Feb.
2014, vol. 53, No. 2, pp. 021111-1-021111-10.
"Oil and Natural Gas Sector Leaks”. U.S. EPA Office of Air Quality
Planning and Standards (OAQPS), Review Panel, Apr. 2014, pp. 63.
Petron et al., “Hydrocarbon Emissions Characterization in the
Colorado Front Range: A Pilot Study”. Journal of GeophysicalResearch, 2012, vol. 117, No. D04304, pp. 1.-19.
Petron et al., “Reply to Comment on Hydrocarbon Emissions
Characterization in the Colorado Front Range—A Pilot Study by
Michael A. Levi'. Journal of Geophysical Research: Atmospheres,
2013, vol. 118, pp. 236-242.
Pisano et al., “Thermal Illuminators for Far-Infrared and Submil

limeter Astronomical Instruments.” Applied Optics, Jun. 1, 2005,
vol. 44, No. 16, pp. 3208-3217.
Polak et al., “Passive Fourier-Transform Infrared Spectroscopy ofChemical Plumes: An Algorithm for Quantitiative Interpretation
and Real-Time Background Removal”. Applied Optics, Aug. 20.
1995, vol. 34, No. 24, pp. 5406-5412.
Sandsten et al., “Volume Flow Calculations on Gas Leaks Imaged
with Infrared Gas-Correlation.” Optics Express, 2012, vol. 20, No.
18, pp. 20318-20329.
Shogenji et al., “Multispectral Imaging Using Compact Compound
Optics,” Optics Express, Apr. 19, 2004, vol. 12, No. 8, pp. 1643
1655.

Walter Jr., et al., “Detection of Atmospheric Pollutants: a Correlation Technique'. Applied Optics, Jun. 1975, vol. 14, No. 6, pp.
1423-1428.

Williams et al., “Dual-Band MWIR/LWIR Radiometer for Absolute

Temperature Measurements.” SPIE Thermosense Conference
XXVIII—Manuscript 6205-23, Apr. 18, 2006, 13 pages.
Young et al., “An In-Scene Method for Atmospheric Compensationof Thermal Hyperspectral Data”, Journal of Geophysical Research,
2002, vol. 107, No. D24, pp. 14-1-14-20.
Zheng et al., “A Static Multiplex Fabry-Perot Spectrometer'. Sen
Sors, Cameras, and Systems for Industrial Scientific Applications X.
Proceedings of SPIE-IS&T Electronic Imaging, SPIE vol. 7249,
2009, pp. 8.
Zheng et al., “Analytic-Domain Lens Design with Proximate Ray
Tracing”, Journal of the Optical Society of America A, Aug. 2010,
vol. 27, No. 8, pp. 1791-1802.
Office Action received in U.S. Appl. No. 14/543,692 dated Nov. 3,2015 in 7 pages.
Office Action received in U.S. Appl. No. 14/539,899 dated Mar. 26,
2015 in 6 pages.
Office Action received in U.S. Appl. No. 14/539,899 dated Dec. 11,2015 in 9 pages.
Amendment and Office Action Response submitted in U.S. Appl.
No. 14/539,899 dated Aug. 26, 2015 in 8 pages.
Extended European Search Report received in European Application No. 141928.62.2 dated Mar. 30, 2015 in 10 pages.
International Search Report in PCT Application No. PCT/US2013/041278 dated Aug. 27, 2013 in 4 pages.
International Preliminary Report on Patentability in PCT Application No. PCT/US2013/041278 dated Nov. 27, 2014 in 10 pages.

Extended European Search Report received in European Application No. 15165877.0 dated Aug. 10, 2015 in 12 pages.
Interview Summary received in U.S. Appl. No. 14/543,692 dated
Feb. 17, 2016 in 5 pages.
Extended European Search Report received in European Application No. EP 15165880.4 dated Nov. 24, 2015 in 8 pages.
Preliminary Amendment as filed in U.S. Patent Application No.
14/700,567 dated Jul. 10, 2015 in 6 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,700,791 dated
Jul. 13, 2015 in 8 pages.
Preliminary Amendment as filed in U.S. Appl. No. 14,792.477 dated
Dec. 21, 2015 in 7 pages.
Gálfalk et al., “Making Methane Visable', Nature Climate Change,
Apr. 2016, vol. 6, pp. 426-430.
Gálfalk et al., “Making Methane Visable'. Supplementary Informa
tion, Nature Climate Change, 2015, pp. 1-14.
Hadlington, Simon, “New Camera Makes Methane Visible'. Chem
istry World, http://web.archive.org/web/20160305234907/http://
www.rsc.org/chemistryworld/2015/12/methane-camera-infraredgreenhouse-gas, Dec. 14, 2015, pp.
TELOPS,“Hyper-Cam',
http://web.archive.org/web/20160608180941/http://www.telops.com/en/hyperspectral-cam
eras/hyper-cam as archived Jun. 8, 2016 in 2 pages.
TELOPS, “Innovative Infrared Imaging'. http://web.archive.org/
web/201606032 12729/http://www.telops.com/en/ as archived Jun.
3, 2016 in 2 pages.
Response to Office Action as filed in U.S. Appl. No. 14/543,692
dated May 2, 2016 in 9 pages.
Office Action received in U.S. Appl. No. 14/543,692 dated Jun. 1,
2016 in 18 pages.
Miscellaneous Communication dated Aug. 24, 2016 filed in U.S.
Appl. No. 14,700,567.
Office Action received in E.P. Application No. 141928.62.2 dated
Apr. 19, 2016 in 6 pages.
Rebellion Photonics, “Gas Cloud Imaging Camera: A Breakthroughin Leak Monitoring for the Rig & Refinery Safety Market'. Pre
sentation at SPIE Defense Security and Sensing, 28 pages, Apr.
29-May 3, 2013.
Brochure provided by Lofty Designs to Rebellion Photonics on Oct.
31, 2012 as noted from the email. Subsequent to that date brochure
was used in connection with potential customers.
“Directed Inspection and Maintenance at Gas Processing Plants andBooster Stations.” United States Environmental Protection Agency
Air and Radiation (6202J), EPA430-B-03-018, Oct. 2003 available
at https://www3.epa.gov/gasstart documents/ll dimgasproc.pdf.
Adams, et al., “Advances in Detectors: Hot IR sensors improve IR
camera size, weight, and power'. Laser Focus World, vol. 50, Issue
01, Jan. 17, 2014, 6 pages. Also available at http://www.ircameras.
com/articles/advances-detectors-hot-ir-sensors-improve-ir-camera
size-weight-power.
Amendment as filed in U.S. Appl. No. 14/539,899 dated Jun. 9.
2016 in 6 pages.
Notice of Allowance received in U.S. Appl. No. 14/539,899 dated
Jun. 21, 2016 in 17 pages.
Notice of Allowance received in U.S. Appl. No. 14,700,791 dated
Jun. 9, 2016 in 11 pages.
Preliminary Amendment as filed in U.S. Appl. No. 15/166,092 dated
Aug. 15, 2016 in 7 pages.
Response to Final Action as filed in U.S. Appl. No. 14/543,692
dated Nov. 30, 2016 in 12 pages.
International Search Report in PCT Application No. PCT/U52016/
034455 dated Oct. 24, 2016 in 12 pages.
Notice of Allowance received in U.S. Appl. No. 14/539,899 dated
Oct. 31, 2016 in 10 pages.
Notice of Allowance received in U.S. Appl. No. 14/543,692 dated
Dec. 9, 2016 in 12 pages.
Notice of Allowance received in U.S. Appl. No. 14,700,791 dated
Sep. 30, 2016 in 19 pages.

* cited by examiner

U.S. Patent

US 9,562,849 B2

Feb. 7, 2017

Og|Jos ao id

U.S. PatentFeb. 7, 2017

Sheet 2 of 15

US 9,562,849 B2

Owardsprocessor 150

U.S. PatentFeb. 7, 2017

Sheet 3 of 15

Field Reference

:

338

\

324

US 9,562,849 B2x 38* 88ss,:::::::
38{{ect, 110

A 324a28a

FIG. 3B

FIG. 3A

U.S. Patent

US 9,562,849 B2

U.S. PatentFeb. 7, 2017

Sheet 5 Of 15

US 9,562,849 B2

U.S. PatentFeb. 7, 2017

Sheet 6 of 15

US 9,562,849 B2

s
"L"

a

e

e

Lil' in and m...T.T.T.T.T. P. T. r. nana--

(x

c
et
C
iOSSS

c^

U.S. PatentFeb. 7, 2017

e
ECSS is:Sheet 7 Of 15-+|{}^

US 9,562,849 B2

89"SDIU.S. PatentFeb. 7, 2017

Sheet 8 of 15

US 9,562,849 B2

N

c

r

ce

to
JSS

ki
Ste.

r

O

U.S. PatentFeb. 7, 2017

Sheet 9 Of 15

US 9,562,849 B2

r

e

c

ex

c
et
ESSE Si.

(-)

U.S. PatentFeb. 7, 2017

Sheet 10 of 15

FIG. 7

US 9,562,849 B2

U.S. PatentFeb. 7, 2017

Sheet 11 of 15

US 9,562,849 B2

ob
ob Corr

i.

is

8

i

Wavelength (microns)

FIG. 8A

ob coir2

4.

3.

Wavelength (microns)

FIG. 8B

i2

U.S. PatentFeb. 7, 2017

Sheet 12 of 15

US 9,562,849 B2900

to object960

N

thermally
Controlled
shutters
936

Casa

bock

FIG. 9A

U.S. PatentFeb. 7, 2017

Sheet 13 of 15

US 9,562,849 B2905

to object

?

972a

960

N

thermally

controlled

928a

H. H.

shutters

936

Care

block

FIG. 9B

U.S. PatentFeb. 7, 2017

Sheet 14 of 15

US 9,562,849 B2

1 OOO(8)(900) Ocameras
mirrors(4) (5) (6) (7)B

FIG. 10B

3.

reference

SOCES

U.S. PatentFeb. 7, 2017

f

t
C
E
?t
U

Sheet 15 Of 15

U

f

S

9c) O

f)

US 9,562,849 B2

US 9,562,849 B2
1.
DIVIDED-APERTURE INFRA-RED
SPECTRAL MAGING SYSTEMCROSS-REFERENCE TO RELATEDAPPLICATIONS

The present application claims benefit under 35 U.S.C.
S119(e) of U.S. Provisional Application No. 61/903,075,

2
include a two-dimensional imaging sensor that is sensitive toradiation having wavelengths between 1 um and 20 Lum (forexample, in mid infra-red wavelength range, long infra-red
wavelength range, visible wavelength range, etc.). In various
embodiments, the array of photo-sensitive devices can
include CCD or CMOS sensors, bolometers or other detec

tors that are sensitive to infra-red radiation. The optical
filed on Nov. 12, 2013 and titled “DIVIDED-APERTURE 10 system may include an optical aperture (a boundary of
INFRA-RED SPECTRAL IMAGING SYSTEM. The dis
which is defined to circumscribe or encompass the at least
closure of the above identified provisional patent application two or more spatially distinct optical channels) and one oris incorporated by reference herein in its entirety.
more optical filters. In various implementations, the one ormore optical filters can comprise at least two spectrally
TECHNICAL FIELD
15 multiplexed filters. Each of these optical filters can be
The present invention generally relates to a system and associated with one of the at least two or more opticalmethod for gas cloud detection and, in particular, to a system channels and configured to transmit a portion of the IR
and method of detecting spectral signatures of chemical radiation received in the associated optical channel. In
compositions in a mid- and long-wave infrared spectral various embodiments, the one or more optical filters can beregion.
spectrally multiplexed and may include, for example, at leastoneof a longpass optical filter and a shortpass optical filter,
DESCRIPTION OF THE RELATED
or a band pass filter (with or without a combination withTECHNOLOGY
another filter such as a notch filter, for example). The optical
Spectral imaging systems and methods have applications 25 system may further include at least two reimaging lenses.
in a variety of fields. Spectral imaging systems and methods The at least two reimaging lenses, for example each of theobtain a spectral image of a scene in one or more regions of
the electromagnetic spectrum to detect phenomena, identify reimaging lens, may be disposed to transmit IR radiation (formaterial compositions or characterize processes. The spec example, between about 1 micron and about 20 microns),
tral image of the scene can be represented as a three 30 that has been transmitted through a corresponding opticaldimensional data cube where two axes of the cube represent filter towards the optical FPA unit. In one embodiment, thetwo spatial dimensions of the scene and a third axes of the optical FPA unit is positioned to receive IR radiation fromdata cube represents spectral information of the scene in the object through the at least two reimaging lenses to form
different wavelength regions. The data cube can be pro respectively-corresponding two or more sets of imaging data
cessed using mathematical methods to obtain information 35 representing the object. The processor or processing elec
about the scene. Some of the existing spectral imaging tronics is configured to acquire this optical data from the twosystems generate the data cube by Scanning the scene in thespatial domain (e.g., by moving a slit across the horizontal or more sets of imaging data. In various embodiments of thedimensions of the scene) and/or spectral domain (e.g., by imaging systems, the FPA unit may be devoid of coolingscanning a wavelength dispersive element to obtain images 40 systems. In various embodiments, two or more of the arrayof the scene in different spectral regions). Such scanning of photo-sensitive devices may be uncooled. In someapproaches acquire only a portion of the full data cube at a embodiments, the system further comprises two or moretime. These portions of the full data cube are stored and then temperature-controlled shutters removably positioned tolater processed to generate a full data cube.
45 block IR radiation incident onto the optical system from theobject.
SUMMARY
Also disclosed herein is an implementation of a method of
The systems, methods and devices of this disclosure each operating an infrared (IR) imaging system. The method
have several innovative aspects, no single one of which is includes receiving IR radiation from an object through at
solely responsible for the desirable attributes disclosed 50 least two optical channels defined by components of anherein.
system of the IR imaging system, which at least two
Various embodiments of the systems described herein optical
opticalchannels are spatially and spectrally different fromprovide an infrared (IR) imaging system for determining aone
another.
The method further includes transmitting theconcentration of a target chemical species in an object (e.g.,a gas plume). The imaging system includes (i) an optical 55 received IR radiation towards an optical focal plane arraysystem, having an optical focal plane array (FPA) unit (FPA) unit that is not being cooled in the course of normal
configured to receive IR radiation from the object along at operation. For example, in various embodiments of theleast two or more optical channels defined by components of imaging systems, the FPA unit may be devoid of cooling
the optical system, the at least two or more optical channels systems. In various embodiments, two or more of the array
being spatially and spectrally different from one another, 60 of photo-sensitive devices may be uncooled. Some embodi
and (ii) a processor or processing electronics configured to ments further comprise removably positioning at least one ofacquire multispectral optical data representing said target at least two temperature-controlled shutters in front of thechemical species from the received IR radiation in a single optical system to block IR radiation incident onto the opticaloccurrence of data acquisition (or Snapshot). The optical system from the object.
FPA unit includes an array of photo-sensitive devices that 65 Various innovative aspects of the subject matter describedare disposed at the focus of one or more lenses. In various in this disclosure can be implemented in the following
embodiments, the array of photo-sensitive devices can embodiments:

US 9,562,849 B2
3

4

Embodiment 1

Embodiment 10

An infrared (IR) imaging system, the imaging systemcomprising:a plurality of cameras;
at least one thermal reference source having a known
temperature placed in front of the plurality of cameras
and configured to be imaged by the plurality of cam

The system of any of Embodiments 1-9, wherein the atleast one reference Source is positioned at a conjugate image
plane of the plurality of cameras.
Embodiment 11

The system of any of Embodiments 1-10, further com
eras; anda data-processing unit comprising a processor, the imag 10 prising a plurality of mirrors configured to image the at least
one reference source onto the plurality of cameras.
ing system configured to:acquire with the plurality of cameras a plurality of frames
Embodiment 12
having regions that correspond to the image of thereference Source; and15
The system of any of Embodiments 1-11, wherein theapply a dynamic calibration correction to the plurality of pluralityof mirrors are disposed outside a central field ofcameras to allow every camera in the plurality of view of theplurality of cameras.
cameras to be calibrated to agree with every othercamera in the plurality imaging the reference source.
Embodiment 13
Embodiment 2

The system of Embodiment 1, wherein the plurality ofcameras comprises a FPA unit and a plurality of lenses.

The system of any of Embodiments 1-12, further com
prising a first and a second temperature-controlled shutter
removably positioned to block IR radiation incident on thesystem from reaching the plurality of cameras.
25

Embodiment 3

The system of any of Embodiments 1-2, wherein the FPA
unit comprises one FPA or a plurality of FPAs.
Embodiment 4

Embodiment 14

The system of any of Embodiments 1-13, wherein thesystemincludes at least two spatially and spectrally different
30
optical channels.
Embodiment 15

The system of any of Embodiments 1-3, wherein the atleast one thermal reference source has a known spectrum.
35

Embodiment 5

The system of any of Embodiments 1-4, further compris
ing an additional thermal reference source imaged by the 40
plurality of cameras.

The system of any of Embodiments 1-14, wherein thesystem includes at least three optical channels.
Embodiment 16

The system of any of Embodiments 1-15, wherein thesystem includes at least four optical channels.

Embodiment 6
Embodiment 17

The system of any of Embodiments 1-5, wherein the
additional reference source has a temperature and a spec 45 The system of any of Embodiments 1-16, wherein thetrum different from the known temperature and the known system includes at least five optical channels.
spectrum of the at least one reference source.
Embodiment 18
Embodiment 7
50

The system of any of Embodiments 1-6, wherein thetemperature of the additional thermal reference source is lessthan the known temperature.
Embodiment 8

The system of any of Embodiments 1-17, wherein thesystem includes at least six optical channels.
Embodiment 19

55

The system of any of Embodiments 1-7, wherein thetemperature of the additional thermal reference source isgreater than the known temperature.

The system of any of Embodiments 1-18, wherein thesystem includes at least seven optical channels.
Embodiment 20

60

Embodiment 9

The system of any of Embodiments 1-19, wherein thesystem includes at least eight optical channels.

The system of any of Embodiments 1-8, wherein the at
Embodiment 21
least one reference source is displaced away from a conju
gate image plane of the plurality of cameras Such that the 65
image of the at least one reference source captured by the
The system of any of Embodiments 1-20, wherein thesystem includes at least nine optical channels.
plurality of cameras is blurred.

US 9,562,849 B2
5

6

Embodiment 22

Embodiment 31

The system of any of Embodiments 1-21, wherein thesystem includes at least ten optical channels.

The system of any of Embodiments 26-30, wherein thefirst temperature is less than the second temperature.

Embodiment 23

Embodiment 32

The system of any of Embodiments 26-31, wherein the
The system of any of Embodiments 1-22, wherein thefirst
and the second reference sources are displaced awaysystem includes at least twelve optical channels.
10 from a conjugate image plane of the plurality of cameras
such that the image of the first and the second reference
Embodiment 24
sources captured by the plurality of cameras is blurred.
The system of any of Embodiments 1-23, further com
Embodiment 33
prising one or more sensors configured to measure a tem 15
perature of the at least one reference source.
The system of any of Embodiments 26-32, wherein thefirst and the second reference sources are positioned at a
Embodiment 25
conjugate image plane of the plurality of cameras.
The system of any of Embodiments 1-24, wherein theplurality of cameras is configured to image the same portion

Embodiment 34of the at least one reference source.

The system of any of Embodiments 26-33, further com
prising:
Embodiment 26
a first mirror configured to image the first reference onto25
the plurality of cameras; andAn infrared (IR) imaging system, the imaging systema second mirror configured to image the second referencecomprising:
Source onto the plurality of cameras.
a plurality of cameras;
Embodiment 35
a first temperature-controlled reference Source imaged by 30
the plurality of cameras;
The system of any of Embodiments 26-34, further com
a second temperature-controlled reference source imaged
prising a first and a second temperature-controlled shutter
by the plurality of cameras; anda data-processing unit comprising a processor, said data removably positioned to block IR radiation incident on the
35 system from reaching the plurality of cameras.
processing unit configured to:acquire with the plurality of cameras a plurality of frames
Embodiment 36
having regions that correspond to the image of thereference Source; andThe system of any of Embodiments 26-35, wherein the
dynamically calibrate the plurality of cameras so that 40 system
at least two spatially and spectrally different
various cameras imaging a scene are forced to agree on optical includes
channels.
a temperature estimate of the first and second reference
SOUCS.

Embodiment 37
Embodiment 27

45

The imaging system of any of Embodiment 26, wherein
the data-processing unit is configured to calculate a dynamic
calibration correction and apply the correction to the plu
rality of cameras for each of the plurality of frames.
50
Embodiment 28

The system of any of Embodiments 26-27, wherein thefirst reference source is maintained at a first temperature. 55
Embodiment 29

The system of any of Embodiments 26-36, wherein thesystem includes at least four optical channels.
Embodiment 38

The system of any of Embodiments 26-37, wherein thesystem includes at least six optical channels.
Embodiment 39

The system of any of Embodiments 26-38, wherein thesystem includes at least eight optical channels.
Embodiment 40

The system of any of Embodiments 26-28, wherein the 60
second reference source is maintained at a second tempera
ture.

Embodiment 30

The system of any of Embodiments 26-39, wherein thesystem includes at least ten optical channels.
Embodiment 41

65

The system of any of Embodiments 26-29, wherein thefirst temperature is greater than the second temperature.

The system of any of Embodiments 26-40, wherein thesystem includes at least twelve optical channels.

US 9,562,849 B2
7

8

Embodiment 42

Embodiment 50

The system of any of Embodiments 26-41, further com
prising one or more sensors configured to measure a tem
perature of the first or the second reference source.

The system of any of Embodiments 44-49, wherein thesystem includes at least three optical channels.
Embodiment 51

Embodiment 43

The system of any of Embodiments 44-50, wherein the
The system of any of Embodiments 26-42, wherein the 10 system includes at least four optical channels.
plurality of cameras is configured to image the same portion
Embodiment 52of the first reference source and wherein plurality of cameras
is configured to image the same portion of the second
reference Source.
The system of any of Embodiments 44-51, wherein thesystemincludes at least five optical channels.
15
Embodiment 44

Embodiment 53

An infrared (IR) imaging system, the imaging systemcomprising:
The system of any of Embodiments 44-52, wherein thesystem includes at least six optical channels.
a plurality of cameras;a reference having an unknown temperature configured to
Embodiment 54
be imaged by the plurality of cameras; anda data-processing unit comprising a processor, the imag
The system of any of Embodiments 44-52, wherein theing system configured to:acquire with the plurality of cameras a plurality of frames 25 system includes at least seven optical channels.
having regions that correspond to the image of the
Embodiment 55
reference;calculate a dynamic calibration correction using a tem
The system of any of Embodiments 44-53, wherein theperature measured by one of the cameras in the plu system
includes at least eight optical channels.
30
rality of cameras as a reference temperature; andapply the calibration correction to the other cameras in the
Embodiment 56
plurality of cameras to match the temperature estimateof the other cameras in the plurality of cameras with the
The system of any of Embodiments 44-54, wherein thereference temperature.
systemincludes at least nine optical channels.
35
Embodiment 45

Embodiment 57

The system of any of Embodiments 44-55, wherein the
The system of any of Embodiment 44, wherein thereference is displaced away from a conjugate image plane of 40 system includes at least ten optical channels.
the plurality of cameras such that the image of the reference
Embodiment 58
source captured by the plurality of cameras is blurred.
The system of any of Embodiments 44-56, wherein the
Embodiment 46
system includes at least twelve optical channels.
45
The system of any of Embodiments 44-45, wherein the
Embodiment 59
reference is positioned at a conjugate image plane of theplurality of cameras.
The system of any of Embodiments 44-57, wherein thepluralityof cameras is configured to image the same portion
Embodiment 47
50

The system of any of Embodiments 44-46, further com
prising a plurality of mirrors configured to image the reference onto the plurality of cameras.

of the reference.

Embodiment 60
55

Embodiment 48

The system of any of Embodiments 44-47, further com
prising a first and a second temperature-controlled shutter
removably positioned to block IR radiation incident on the 60
system from reaching the plurality of cameras.
Embodiment 49

The system of any of Embodiments 44-48, wherein the 65
system includes at least two spatially and spectrally different
optical channels.

An infrared (IR) imaging system, the imaging systemcomprising:an optical system including an optical focal plane array(FPA) unit, the optical system includes components
associated with at least two optical channels, said atleast two optical channels being spatially and spectrally
different from one another, each of the at least two

optical channels positioned to transfer IR radiation
incident on the optical system towards the optical FPA
unit, the optical FPA unit comprising at least two
detector arrays disposed at a distance from two corre
sponding focusing lenses;
at least one thermal reference having a known tempera
ture, wherein radiation emitted from the at least one

US 9,562,849 B2
9reference is directed towards the optical FPA unit andimaged by the at least two detector arrays; anda data-processing unit, said data-processing unit config

10
Embodiment 69

The system of any of Embodiments 60-68, wherein the atleast two reflecting elements are disposed outside a central
acquire a plurality of frames with the at least two detector 5 field of view of the at least two detector arrays.
arrays having regions in the plurality of image frames
Embodiment 70
that correspond to the image of the reference; anddynamically calibrate the at least two detector arrays toaddress a difference in the temperature estimate of the 10 The system of any of Embodiments 60-69, further com
prising a third detector array disposed between the at leastreference between the two detector arrays.
two detector arrays.
ured to:Embodiment 61
Embodiment 71

The system of any of Embodiment 60, wherein the at least 15
one thermal reference has a known spectrum.
Embodiment 62

anddynamically calibrate the third detector array to match atemperature estimate of the third detector array with thetemperature estimates of the at least two detector

The system of any of Embodiments 60-61, further com
prising an additional thermal reference, wherein radiationfrom the additional reference is directed towards the optical
FPA unit and imaged by the at least two detector arrays.
Embodiment 63

The system of any of Embodiments 60-70, wherein thedata-processing unit is configured to:acquire a plurality of frames using the third detector array;

arrays.
25

Embodiment 72

The system of any of Embodiments 60-71, wherein radia
The system of any of Embodiments 60-62, wherein theemitted from the at least one reference source is not
additional reference has a temperature and a spectrum tion
imagedby the third detector array.
different from the known temperature and the known spec 30

trum of the at least one reference source.

Embodiment 73

Embodiment 64

The system of any of Embodiments 60-72, wherein thethirdarray has a field of view greater than a field of
The system of any of Embodiments 60-63, wherein the 35 view detector
of
the
at
least two detector arrays.
temperature of the additional thermal reference is less than
the known temperature.
Embodiment 74
Embodiment 65
40

The system of any of Embodiments 60-64, wherein thetemperature of the additional thermal reference is greaterthan the known temperature.
Embodiment 66

Embodiment 75
45

The system of any of Embodiments 60-65, wherein the atleast one reference is displaced away from a conjugate
image plane of the at least two detector arrays such that the 50
image of the at least one reference captured by the at least
two detector arrays is defocused.
Embodiment 67

The system of any of Embodiments 60-73, wherein the atleast one reference is imaged by the third detector array.

The system of any of Embodiments 60-74, further com
prising a third reflecting element disposed outside the fieldof view of the at least two detector arrays and configured toimage the at least one reference onto the third detector array.
Embodiment 76

The system of any of Embodiments 60-75, further com
prising a first and a second temperature-controlled element
removably positioned to block IR radiation incident on the
55 optical system from reaching the optical FPA unit.

The system of any of Embodiments 60-66, wherein the at
Embodiment 77
least one reference is positioned at a conjugate image planeof the at least two detector arrays such that the image of theat least one reference captured by the at least two detector
The system of any of Embodiments 60-76, wherein thearrays is focused.
60 optical system includes components associated with three
optical channels.
Embodiment 68
Embodiment 78

The system of any of Embodiments 60-67, further com
prising at least two reflecting elements configured to direct 65
radiation from the at least one reference source toward theat least two detector arrays.

The system of any of Embodiments 60-77 wherein the
optical system includes components associated with four
optical channels.

US 9,562,849 B2
11
Embodiment 79

The system of any of Embodiments 60-78, wherein the
optical system includes components associated with six
optical channels.
Embodiment 80

The system of any of Embodiments 60-79, wherein the
optical system includes components associated with eight 10
optical channels.
Embodiment 81

12optical FPA unit, the optical FPA unit comprising atleast two detector arrays disposed at a distance fromtwo corresponding focusing lenses;
a first temperature-controlled reference imaged by the atleast two detector arrays;a second temperature-controlled reference imaged by theat least two detector arrays; anda data-processing unit configured to:acquire a plurality of frames with the at least two detector
arrays having regions in the plurality of image framesthat correspond to the image of the first and second
references; andThe system of any of Embodiments 60-80, wherein the 15
optical system includes components associated with ten
optical channels.

dynamically calibrate the at least two detector arrays toaddress a difference in a temperature estimate of thefirst and second references between the two detector
arrays.

Embodiment 82
Embodiment 90

The system of any of Embodiments 60-81, wherein the
optical system includes components associated with twelve
optical channels.
Embodiment 83

The system of Embodiment 89, wherein the first reference
is maintained at a first temperature.
25

The system of any of Embodiments 60-82, wherein the
optical system includes components associated with sixteen
optical channels.
Embodiment 84

Embodiment 91

The system of any of Embodiments 89-90, wherein thesecond reference is maintained at a second temperature.
30

Embodiment 92

The system of any of Embodiments 60-83, wherein the
optical system includes components associated with twenty
The system of any of Embodiments 89-91, wherein thefirst temperature is greater than the second temperature.
four optical channels.
35
Embodiment 85

Embodiment 93

The system of any of Embodiments 60-84, wherein eachof the at least two detector arrays is configured to image the 40 The system of any of Embodiments 89-92, wherein thesame portion of the at least one reference so as to consis first temperature is less than the second temperature.
tently provide a common reference temperature.
Embodiment 86

Embodiment 94

The system of any of Embodiments 60-85, wherein the 45 The system of any of Embodiments 89-93, wherein thefirst and the second references are displaced away from adata-processing unit comprises processing electronics.
conjugate image plane of the at least two detector arrays
Embodiment 87
such that the image of the first and the second references
captured by the at least two detector arrays is defocused.
The system of any of Embodiments 60-86, wherein the 50
data-processing unit comprises a processor.
Embodiment 95
Embodiment 88

The system of any of Embodiments 89-94, wherein the
The system of any of Embodiments 60-87, wherein the 55 first and the second references are positioned at a conjugate
image plane of the at least two detector arrays Such that thedata-processing unit comprises one or more processors.
image of the first and the second references captured by the
Embodiment 89
at least two detector arrays is focused.
An infrared (IR) imaging system, the imaging system 60
Embodiment 96
comprising: an optical system including an optical focal
plane array (FPA) unit, theThe system of any of Embodiments 89-95, further com
optical system including at least two optical channels,
said at least two optical channels being spatially and prising:
spectrally different from one another, each of the at 65 a first reflecting element configured to direct radiationfrom the first reference toward the at least two detector
least two optical channels positioned to transfer IR
arrays; andradiation incident on the optical system towards theUS 9,562,849 B2
14removably positioned to block IR radiation incident on the
optical system from reaching the optical FPA unit.

13
a second reflecting element configured to direct radiationfrom the second reference toward the at least two

detector arrays.
Embodiment 106
Embodiment 97

The system of any of Embodiments 89-96, wherein thefirst reflecting element is disposed outside a field of view of
the at least two detector arrays.
10

The system of any of Embodiments 89-105, wherein the
optical system includes components associated with three
optical channels.
Embodiment 107

Embodiment 98

The system of any of Embodiments 89-106, wherein the
optical system includes components associated with four
The system of any of Embodiments 89-97, wherein the optical channels.
second reflecting element is disposed outside a field of view 15of the at least two detector arrays.
Embodiment 108
Embodiment 99

The system of any of Embodiments 89-107, wherein the
optical system includes components associated with five
optical channels.

The system of any of Embodiments 89-98, further com
prising a third detector array disposed between the at least
two detector arrays.
Embodiment 100

Embodiment 109
25

The system of any of Embodiments 89-99, wherein thedata-processing unit is configured to:acquire a plurality of frames using the third detector array:

The system of any of Embodiments 89-108, wherein the
optical system includes components associated with six
optical channels.
Embodiment 110

30

and
The system of any of Embodiments 89-109, wherein the
dynamically calibrate the third detector array to address a optical system includes components associated with sevendifference in the temperature estimate of the first and optical channels.
second references between the third detector array and
Embodiment 111
the two detector arrays.
35
Embodiment 101

The system of any of Embodiments 89-100, wherein thefirst and second references are not imaged by the third 40
detector array.
Embodiment 102

The system of any of Embodiments 89-101, wherein thethird detector array has a field of view greater than a field ofview of the at least two detector arrays.

The system of any of Embodiments 89-110, wherein the
optical system includes components associated with eight
optical channels.
Embodiment 112

The system of any of Embodiments 89-111, wherein the
optical system includes components associated with ten
optical channels.
45
Embodiment 113

Embodiment 103

The system of any of Embodiments 89-112, wherein the
optical system includes components associated with twelve
50 optical channels.

The system of any of Embodiments 89-102, wherein r firstand second references are imaged by the third detector array.

Embodiment 114

The system of any of Embodiments 89-113, wherein eachof the at least two detector arrays is configured to image thesame portion of the first reference source so as to consis
The system of any of Embodiments 89-103, further com tently provide a common first reference temperature andprising a third reflecting element disposed outside the field wherein each of the at least two detector arrays is configuredof view of the at least two detector arrays and configured to 60 to image the same portion of the second reference Source soimage the first and second references onto the third detector as to consistently provide a common second reference
Embodiment 104

55

temperature.

array.

Embodiment 115
Embodiment 105
65

The system of any of Embodiments 89-104, further com
prising a first and a second temperature-controlled element

The system of any of Embodiments 89-114, further com
prising a temperature controller configured to control thetemperature of the first or second reference.

US 9,562,849 B2
15

16
the calibration surface of the cameras in the plurality ofcameras agree with each other.

Embodiment 116

The system of any of Embodiments 89-115, further com
prising one or more sensors configured to measure a tem
perature of the first or the second reference.

Embodiment 125

Embodiment 117

The system of Embodiment 124, wherein the one or moreparameters is associated with a gain of the cameras in theplurality of cameras.

The system of any of Embodiments 89-116, wherein the 10
one or more sensors are configured to communicate themeasured temperature of the first or the second reference toa temperature controller.
Embodiment 118

Embodiment 126

The system of any of Embodiments 124-125, wherein theone or more parameters is associated with again offset of thecameras in the plurality of cameras.
15
Embodiment 127

The system of any of Embodiments 89-117, wherein theone or more sensors are configured to communicate themeasured temperature of the first or the second reference tothe data-processing unit.

The system of any of Embodiments 124-126, wherein thecalibration Surface is displaced away from a conjugate
image plane of the plurality of cameras Such that an imageof the surface is defocused.

Embodiment 119

Embodiment 128

The system of any of Embodiments 89-118, wherein the 25

first or the second reference is associated with a heater.

The system of any of Embodiments 124-127, wherein thecalibration Surface is positioned at a conjugate image planeof the plurality of cameras Such that an image of the Surface

Embodiment 120

is focused.

The system of any of Embodiments 89-119, wherein the 30
first or the second reference is associated with a cooler.

Embodiment 129

The system of any of Embodiments 124-128, further
comprising at least one reflecting element configured to35
The system of any of Embodiments 89-120, wherein the image the Surface onto the plurality of cameras.
data-processing unit comprises processing electronics.
Embodiment 130
Embodiment 121

Embodiment 122
40

The system of any of Embodiments 89-121, wherein thedata-processing unit comprises a processor.
Embodiment 123

The system of any of Embodiments 124-129, further
comprising a first and a second temperature-controlled ele
ment removably positioned to block IR radiation incident on
the optical system from reaching the plurality of cameras.
Embodiment 131

45

The system of any of Embodiments 124-130, wherein the
optical system includes components associated with three
optical channels.

The system of any of Embodiments 89-122, wherein thedata-processing unit comprises one or more processors.
Embodiment 124

50

Embodiment 132

An infrared (IR) imaging system, the imaging system
The system of any of Embodiments 124-131, wherein thecomprising:
opticalsystem includes components associated with fouran optical system including components associated with optical channels.
55
at least two optical channels, said at least two opticalchannels being spectrally different from one another,
Embodiment 133
each of the at least two optical channels positioned totransfer IR radiation incident on the optical system
The system of any of Embodiments 124-132, wherein the
towards a plurality of cameras;
60 optical system includes components associated with six
at least one calibration Surface with unknown temperature optical channels.
and imaged by each of the at least two detectors; anda data-processing unit configured to:
Embodiment 134
acquire a plurality of image frames with the plurality ofcameras including the imaged Surface; and65
The system of any of Embodiments 124-133, wherein theadjust one or more parameters of the cameras in the optical system includes components associated with eightplurality of cameras Such that a temperature estimate of optical channels.

US 9,562,849 B2
17

18

Embodiment 135

Embodiment 145

The system of any of Embodiments 141-144, wherein a
The system of any of Embodiments 124-134, wherein the
optical system includes components associated with ten number of optical filters is two.
5
optical channels.
Embodiment 146

Embodiment 136

The system of any of Embodiments 141-145, wherein a
The system of any of Embodiments 124-135, wherein the 10 number of optical filters is three.
optical system includes components associated with twelve
Embodiment 147
optical channels.
The system of any of Embodiments 141-146, wherein a
Embodiment 137
number of optical filters is four.
The system of any of Embodiments 124-136, wherein the 15
Embodiment 148
data-processing unit comprises processing electronics.
The system of any of Embodiments 141-147, wherein the
Embodiment 138
plurality of optical filters comprise at least one long pass(LP) filter.
The system of any of Embodiments 124-137, wherein thedata-processing unit comprises a processor.
Embodiment 149
Embodiment 139

The system of any of Embodiments 124-138, wherein thedata-processing unit comprises one or more processors.

25

The system of any of Embodiments 141-148, wherein theplurality of optical filters comprise multiple long pass (LP)filters.
Embodiment 150

Embodiment 140

The system of any of Embodiments 141-149, wherein the
The system of any of Embodiments 124-139, wherein the 30 plurality of optical filters comprise at least one short passcalibration Surface comprises a sidewall of a housing of the (SP) filter.
system.

Embodiment 151
Embodiment 141
35

The system of any of Embodiments 141-150, wherein the
An infrared (IR) imaging system, the imaging system plurality of optical filters comprise multiple short pass (SP)comprising:filters.
at least four spatially and spectrally different optical
Embodiment 152
channels configured to receive IR radiation from a 40
common object, each of the at least four spatially andspectrally different optical channels comprising at least
The system of any of Embodiments 141-151, wherein the
one imaging lens configured to image the object on a plurality of optical filters comprise at least one band pass
Focal Plane Array (FPA) unit; and(BP) filter.
processing electronics in communication with the FPA 45
Embodiment 153

unit,

wherein said infrared system is configured to:acquire multispectral optical data from the at least four
The system of any of Embodiments 141-152, wherein theplurality of optical filters comprise at one short pass (SP)different optical channels; andprocess the multispectral optical data to detect one or 50 filter and one long pass (LP) filter.
more target species present in the object.
Embodiment 154
Embodiment 142

The system of any of Embodiments 141-153, wherein the
The system of Embodiment 141, further comprising FPA unit comprises a plurality of FPAs.
55
twelve optical channels.
Embodiment 155

Embodiment 143

The system of any of Embodiments 141-154, further
The system of any of Embodiments 141-142, configured comprising first and second temperature-controlled elements
to simultaneously acquire multispectral optical data from the 60 removably positioned to block IR radiation incident on theimaging system from reaching the FPA unit.
at least four different optical channels.
Embodiment 144

Embodiment 156

The system of any of Embodiments 141-143, further 65 The system of any of Embodiments 141-155, further
comprising a plurality of optical filters associated with the comprising a field reference configured for dynamicallycalibrating a plurality of the FPAS in the FPA unit.
optical channels.

US 9,562,849 B2
19

20

Embodiment 157

Embodiment 167

The system of any of Embodiments 141-166, further
The system of any of Embodiments 141-156, wherein thefield reference is configured to obscure a peripheral region comprising five optical channels.
of an image generated by a plurality of the FPAS in the FPA 5
Embodiment 168

unit.
Embodiment 158

The system of any of Embodiments 141-167, further
comprising six optical channels.

The system of any of Embodiments 141-157, configured 10
Embodiment 169
to compare spectral data in at least one of the four opticalchannels acquired at a first instant of time with spectral datain the at least one of the four optical channels acquired at a
The system of any of Embodiments 141-168, further
second instant of time to generate a temporal difference 15 comprising seven optical channels.
image.
Embodiment 170
Embodiment 159

The system of any of Embodiments 141-169, further
comprising eight optical channels.

The system of any of Embodiments 141-158, configured
to use a difference between the multispectral optical dataacquired by the two optical channels to correct parallax
induced imaging errors.
Embodiment 160

Embodiment 171

25

The system of any of Embodiments 141-170, further
comprising nine optical channels.

The system of any of Embodiments 141-159, configured
Embodiment 172
to use a difference between the multispectral optical dataacquired by the two optical channels to estimate a distance
The system of any of Embodiments 141-171, further
between the system and the object.
30 comprising ten optical channels.
Embodiment 161

Embodiment 173

The system of any of Embodiments 141-160, configured
to estimate a size of the object based on the estimated 35 The system of any of Embodiments 141-172, further
distance and an optical magnification factor of the two comprising eleven optical channels.
optical channels.
Embodiment 174
Embodiment 162

The system of any of Embodiments 141-173, wherein a
The system of any of Embodiments 141-161, configured 40 number of optical filters is five.
to compare spectral data in one of the at least four opticalchannels with spectral data in another one of the at least four
Embodiment 175
optical channels to generate a spectral difference image.
The system of any of Embodiments 141-174, wherein a
45
Embodiment 163
number of optical filters is six.
The system of any of Embodiments 141-162, further
Embodiment 176
comprising a visible light imaging sensor.
50
Embodiment 164
The system of any of Embodiments 141-175, wherein anumber of optical filters is seven.
The system of any of Embodiments 141-163, configured
Embodiment 177
to use the visible light imaging sensor to compensate formotion-induced imaging errors.
55
The system of any of Embodiments 141-176, wherein a
Embodiment 165
number of optical filters is eight.
The system of any of Embodiments 141-164, configured
Embodiment 178
to process the multispectral optical data by cross-correlating
60
multispectral optical data from at least one of the optical
The system of any of Embodiments 141-177, wherein achannels with a reference spectrum.
number of optical filters is nine.
Embodiment 166
Embodiment 179

The system of any of Embodiments 141-165, configured 65
to process the multispectral optical data using spectral
The system of any of Embodiments 141-178, wherein aunmixing.
number of optical filters is ten.

US 9,562,849 B2
21

22
portion of the thermal reference so as to consistently provide
a common reference temperature.

Embodiment 180

The system of any of Embodiments 141-179, wherein anumber of optical filters is eleven.

Embodiment 193

Embodiment 181

The system of any of Embodiments 141-192, wherein atemperature of the same portion of the thermal reference is

The system of any of Embodiments 141-180, wherein anumber of optical filters is twelve.
Embodiment 182

unknown.
10

The system of any of Embodiments 141-193, configured
to acquire the multispectral optical data at a frame rate

The system of any of Embodiments 141-181, wherein theprocessing electronics comprises one or more processors.
Embodiment 183

Embodiment 194

between about 5 Hz to about 200 HZ.
15

Embodiment 195

The system of any of Embodiments 141-182, further
comprising a thermal reference configured to be imaged
onto the FPA unit such that a plurality of frames of theacquired multispectral optical data has an image of the

The system of any of Embodiments 141-194, wherein one
or more of the at least four optical channels is configured tocollect IR radiation to provide spectral data corresponding toa discrete spectral band located in the wavelength range
between about 7.9 um and about 8.4 um.

thermal reference source.
Embodiment 184

The system of any of Embodiments 141-183, wherein the 25
thermal reference has a known temperature.

Embodiment 196

The system of any of Embodiments 141-195, whereineach optical channel is configured to transfer spectrally
distinct, two-dimensional image data of the common object
The system of any of Embodiments 141-184, wherein the 30 to one or more imaging sensors.
thermal reference is a temperature-controlled reference
Embodiment 196
Embodiment 185

SOUCC

Embodiment 186

The system of any of Embodiments 141-185, wherein thetemperature-controlled reference source includes a heater.

35

Embodiment 197

Embodiment 187

The system of any of Embodiments 141-186, wherein thetemperature-controlled reference source includes a cooler.

40

Embodiment 188

The system of any of Embodiments 141-187, further 45
comprising a mirror configured to image the thermal reference onto the FPA unit.
Embodiment 189

The system of any of Embodiments 1-24, wherein the one
or more sensors are configured to communicate the mea
Sured temperature of the at least one reference source to atemperature controller.
The system of any of Embodiments 1-24, wherein the one
or more sensors are configured to communicate the mea
sured temperature of the at least one reference source to thedata-processing unit.
Embodiment 198

The system of any of Embodiments 26-42, wherein theone or more sensors are configured to communicate mea
sured temperature of the first or the second reference to a
50 temperature controller.

The system of any of Embodiments 141-188, wherein atemperature of the thermal reference is unknown.

Embodiment 199

Embodiment 190
55

The system of any of Embodiments 141-189, wherein thethermal reference is a Surface.

The system of any of Embodiments 26-42, wherein theone or more sensors are configured communicate the mea
sured temperature of the first or the second reference to thedata-processing unit.
Embodiment 200

Embodiment 191
60

An infrared (IR) imaging system for imaging a target
species in an object, the imaging system comprising:an optical system comprising an optical focal plane array
Embodiment 192(FPA) unit and defining a plurality of spatially and65
spectrally different optical channels to transfer IR
radiation towards the optical FPA unit, each optical
The system of any of Embodiments 141-191, whereinchannel positioned to transfer a portion of the IR
different optical channels receive IR radiation from the same
The system of any of Embodiments 141-190, wherein the
Surface comprises a wall of a housing of the system.

US 9,562,849 B2
23
radiation incident on the optical system from the object
towards the optical FPA unit; anda programmable processor configured to execute instructions stored in a tangible, non-transitory computerreadable storage medium, to acquire, in a single occur
rence of data acquisition, multispectral optical data
representing in spatial (x, y) and spectral (W) coordi
nates said object and said target species from the IRradiation received at the optical FPA unit.

24
reference source removably positioned to block IR radiation
incident onto the optical system from reaching the optical
FPA unit.
Embodiment 210

The system of any of Embodiments 200-209, wherein themultispectral optical data from the plurality of optical channels is captured Substantially simultaneously by the optical
10

FPA unit.

Embodiment 201
Embodiment 211

The system of Embodiment 200, wherein the multispec
tral data comprises a number of spectrally different images 15 The system of any of Embodiments 200-210, wherein theof the object obtained from IR image data transferred to the multispectral optical data from the plurality of optical channels is captured during one image frame acquisition by the
optical FPA unit by a corresponding optical channel.
optical FPA unit.
Embodiment 202

Embodiment 212

The system of any of Embodiments 200-201, further
comprising an optical filter corresponding to a particular
optical channel and configured to transmit the portion of IR
radiation towards the optical FPA unit.

The system of any of Embodiments 200-211, further
comprising first and second temperature-controlled moveable shutters removably positioned to block IR radiation
25 incident onto the optical system from reaching the optical
FPA unit.

Embodiment 203
Embodiment 213

The system of any of Embodiments 200-202, wherein the
optical filter includes one of a longpass optical filter and a 30 The system of any of Embodiments 200-212, wherein the
optical FPA unit is devoid of a cooling device.
shortpass optical filter.
Embodiment 214

Embodiment 204

The system of any of Embodiments 200-213, further
The system of any of Embodiments 200-203, further 35 comprising a filter array.
comprising one or more front objective lenses.
Embodiment 215
Embodiment 205

The system of any of Embodiments 200-204, wherein the
optical system comprises a plurality of lenses, each lenscorresponding to an optical channel.
Embodiment 206

40

45

The system of any of Embodiments 200-214, wherein theprocessor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium
to acquire said optical data from the two or more sets ofimaging data.
Embodiment 216

The system of any of Embodiments 200-215, wherein theprocessor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium
50 to process the acquired optical data to compensate for atleast one of (i) parallax-induced differences between the two
Embodiment 207
or more sets of imaging data and (ii) difference between thetwo or more sets of imaging data induced by changes in the
The system of any of Embodiments 200-206, wherein the object that are not associated with the target species.
plurality of lenses comprises a lens array.
The system of any of Embodiments 200-205, whereineach optical channel is defined at least in part by a corre
sponding filter and a corresponding lens.

55

Embodiment 217
Embodiment 208

The system of any of Embodiments 200-216, wherein the
The system of any of Embodiments 200-207, further 60 processor is configured to execute instructions stored in acomprising a plurality of relay lenses configured to relay the tangible, non-transitory computer-readable storage medium
to process the acquired optical data to generate a temporal
IR radiation along the optical channels.
reference image.
Embodiment 209

Embodiment 218
65

The system of any of Embodiments 200-208, further
comprising a plurality of moveable temperature-controlled

The system of any of Embodiments 200-217, wherein theprocessor is configured to execute instructions stored in a

US 9,562,849 B2
25
tangible, non-transitory computer-readable storage medium
to use the temporal reference image to generate a temporal
difference image.

26
tangible, non-transitory computer-readable storage medium
to process data received from the visible light imaging
sensor to compensate for motion-induced imaging errors.

Embodiment 219

Embodiment 229

The system of any of Embodiments 200-218, wherein the
The system of any of Embodiments 200-228, wherein theprocessor is configured to execute instructions stored in a processor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium tangible, non-transitory computer-readable storage medium
to process the acquired optical data to estimate a volume of 10 to construct, in the single occurrence of data acquisition, aa gas cloud.
multispectral data cube of the object, the multispectral data
cube comprising a number of spectrally different images ofEmbodiment 220
the object, each spectrally different image comprising IR
The system of any of Embodiments 200-219, wherein IR 15 image data transferred to the optical FPA unit by a corre
radiation measured at a pixel comprises a spectrum com sponding optical channel.
prising a sum of component spectra, and wherein the pro
Embodiment 230
cessor is configured to execute instructions stored in atangible, non-transitory computer-readable storage medium
The system of any of Embodiments 200-229, wherein theto unmix the spectrum.
portion of the IR radiation corresponds to a region of
wavelengths of the spectrum of wavelengths, the region of
Embodiment 221
wavelengths at least partially overlapping another region of
The system of any of Embodiments 200-220, wherein the wavelengths transferred by another optical channel.
optical FPA unit includes a bolometer configured to operate 25
Embodiment 231
without being cooled.
An infrared (IR) imaging system, the imaging system
Embodiment 222
comprising:of spatially and spectrally different optical
The system of any of Embodiments 200-221, further 30 a plurality
channels,
at least some of the plurality of opticalcomprising a field reference for dynamically adjusting datachannelsconfigured
to receive IR radiation from a
output from the optical FPA unit.
common object, each of the plurality of spatially andspectrally different optical channels comprising at least
Embodiment 223
one imaging lens configured to image the object on a
35
Focal Plane Array (FPA) unit; andThe system of any of Embodiments 200-222, wherein theprocessing electronics in communication with the FPA
field reference comprises an array of field stops.
unit,
wherein said infrared system is configured to:
Embodiment 224
acquire multispectral optical data from the plurality of40
different optical channels; andThe system of any of Embodiments 200-223, wherein theprocess the multispectral optical data to detect one or
field reference comprises a uniform temperature across its
more target species present in the object, andSurface.
wherein the system is further configured to compare
spectral data in at least one of the plurality of optical
Embodiment 225
45
channels acquired at a first instant of time with spectral
data in at least one of the plurality of optical channels
The system of any of Embodiments 200-224, wherein theacquired at a second instant of time to generate afield reference is adapted to obscure or block a peripheral
temporal difference image.
portion of the IR radiation propagating from the object
towards the optical FPA unit.
50
Embodiment 232
Embodiment 226

An infrared (IR) imaging system, the imaging systemcomprising:a plurality of spatially and spectrally different optical
55 channels, at least some of the plurality of optical channels
Embodiment 227
configured to receive IR radiation from a common object,
each of the plurality of spatially and spectrally different
The system of any of Embodiments 200-226, wherein the optical channels comprising at least one imaging lens con
processor is configured to execute instructions stored in a figured to image the object on a Focal Plane Array (FPA)
tangible, non-transitory computer-readable storage medium 60 unit; andprocessing electronics in communication with the FPA
to process data received from an imaging sensor to com
unit,
pensate for motion-induced imaging errors.
wherein said infrared system is configured to:
Embodiment 228
acquire multispectral optical data from the plurality of65
different optical channels; andprocess the multispectral optical data to detect one or
The system of any of Embodiments 200-227, wherein the
more target species present in the object, andprocessor is configured to execute instructions stored in a
The system of any of Embodiments 200-225, further
comprising a visible light imaging sensor.

US 9,562,849 B2
27
wherein the system is configured to use a difference
between the multispectral optical data acquired by two
optical channels to correct parallax-induced imaging
COS.

Embodiment 232

The system of Embodiment 231, wherein spectral characteristics of the two optical channels are identical.
10
Embodiment 233

28
optical channels comprising at least one imaging lensconfigured to image the object on a Focal Plane Array(FPA) unit; andprocessing electronics in communication with the FPA
unit,

wherein said infrared system is configured to:acquire multispectral optical data from the plurality ofdifferent optical channels; andprocess the multispectral optical data to detect one ormore target species present in the object, andwherein the system is configured to compensate formotion-induced imaging errors.

An infrared (IR) imaging system, the imaging systemcomprising:
Embodiment 237
a plurality of spatially and spectrally different optical 15
channels, at least Some of the plurality of optical channels
An infrared (IR) imaging system, the imaging system
configured to receive IR radiation from a common object, comprising:each of the plurality of spatially and spectrally different
a plurality of spatially and spectrally different, at least
optical channels comprising at least one imaging lens con
Some of the plurality of optical channels configured tofigured to image the object on a Focal Plane Array (FPA)
receive IR radiation from a common object, each of theunit; andplurality of spatially and spectrally different opticalprocessing electronics in communication with the FPA
channels comprising at least one imaging lens config
unit,
ured to image the object on a Focal Plane Array (FPA)
wherein said infrared system is configured to:25unit; andacquire multispectral optical data from the plurality ofprocessing electronics in communication with the FPA
different optical channels; andunit,
process the multispectral optical data to detect one orwherein said infrared system is configured to:more target species present in the object, andacquire multispectral optical data from the plurality ofwherein the system is configured to use a difference 30
different optical channels; andbetween the multispectral optical data acquired by two
process the multispectral optical data to detect one or
optical channels to estimate a distance between the
more target species present in the object by crosssystem and the object.
correlating multispectral optical data from at least oneof the optical channels with a reference spectrum.
35
Embodiment 234

Embodiment 238

The system of Embodiment 233, wherein spectral characteristics of the two optical channels are identical.

An infrared (IR) imaging system, the imaging systemcomprising:40
Embodiment 235
a plurality of spatially and spectrally different optical, atleast some of the plurality of optical channels config
An infrared (IR) imaging system, the imaging systemured to receive IR radiation from a common object,comprising:each of the plurality of spatially and spectrally different
a plurality of spatially and spectrally different optical, at
optical channels comprising at least one imaging lens
45
least some of the plurality of optical channels config
configured to image the object on a Focal Plane Array
ured to receive IR radiation from a common object,(FPA) unit; andeach of the plurality of spatially and spectrally different
processing electronics in communication with the FPA
optical channels comprising at least one imaging lensunit,
configured to image the object on a Focal Plane Array50
wherein said infrared system is configured to:(FPA) unit:
acquire multispectral optical data from the plurality ofa visible light imaging sensor; anddifferent optical channels; andprocessing electronics in communication with the FPA
process the multispectral optical data to detect one orunit,
more target species present in the object by using
wherein said infrared system is configured to:spectral unmixing.
acquire multispectral optical data from the plurality of 55
different optical channels; andEmbodiment 239
process the multispectral optical data to detect one ormore target species present in the object.
An infrared (IR) imaging system, the imaging system
Embodiment 236
60 comprising:an optical system including an optical focal plane array
An infrared (IR) imaging system, the imaging system(FPA) unit, the optical system includes components
comprising:associated with at least two optical channels, said at
a plurality of spatially and spectrally different optical, atleast two optical channels being spatially and spectrally
least some of the plurality of optical channels config 65
different from one another, each of the at least two
optical channels positioned to transfer IR radiation
ured to receive IR radiation from a common object,
each of the plurality of spatially and spectrally different
incident on the optical system towards the optical FPA

US 9,562,849 B2
29
unit, the optical FPA unit comprising at least two
detector arrays disposed at a distance from two corre
sponding focusing lenses;
at least one thermal reference having a known tempera
ture, wherein one of the at least two detector arrays is
configured to image the at least one reference; anda data-processing unit, said data-processing unit config

30
FIG. 6B depicts theoretical plots of transmission characteristics of a spectrally multiplexed notch-pass filter combi
nation used in an embodiment of the system.
FIG. 6C shows theoretical plots of transmission characteristics of spectrally multiplexed long-pass filter combina
tion used in an embodiment of the system.
FIG. 6D shows theoretical plots of transmission characteristics of spectrally multiplexed short-pass filter combina
ured to:acquire a plurality of frames with one of the at least two 10 tion used in an embodiment of the system.
FIG. 7 is a set of video-frames illustrating operability ofdetector arrays having regions in the plurality of imageanembodiment of the system used for gas detection.
frames that correspond to the image of the reference;
FIGS. 8A and 8B are plots (on axes of wavelength inandversus the object temperature in Celsius represent
dynamically calibrate another of the at least two detector microns
ing effective optical intensity of the object) illustratingarray to match a temperature estimate of another of the 15 results of dynamic calibration of an embodiment of theat least two detector array with the temperature esti system.
mate of one of the at least two detector array.
FIGS. 9A and 9B illustrate a cross-sectional view ofdifferent
embodiments of an imaging system comprising an
Embodiment 240
arrangement of reference Sources and mirrors that can beused for dynamic calibration.
The system of any of Embodiments 1-59, wherein the
FIGS. 10A-10C illustrate a plan view of different embodi
plurality of cameras are configured to acquire multispectral ments of an imaging System comprising an arrangement ofimage data from an object continuously for a duration of reference sources and mirrors that can be used for dynamic

time.

calibration.
25

Embodiment 241

Like reference numbers and designations in the various
drawings indicate like elements.

DETAILED DESCRIPTION
The system of any of Embodiments 1-59, comprising atleast two spectrally and spatially distinct optical channels
configured to transfer two-dimensional image data of an 30 The following description is directed to certain imple
mentations for the purposes of describing the innovative
object to the plurality of cameras.
Details of one or more implementations of the subject aspects of this disclosure. However, a person having ordi
nary skill in the art will readily recognize that the teachings
matter described in this disclosure are set forth in theaccompanying drawings and the description below. Other herein can be applied in a multitude of different ways. The
features, aspects, and advantages will become apparent from 35 described implementations may be implemented in any
the description, the drawings and the claims. Note that the device, apparatus, or system that can be configured torelative dimensions of the following figures may not be operate as an imaging System Such as in an infra-red imaging
drawn to scale.
system. The methods and systems described herein can be
included in or associated with a variety of devices such as,
BRIEF DESCRIPTION OF THE DRAWINGS
40 but not limited to devices used for visible and infrared
spectroscopy, multispectral and hyperspectral imaging
FIG. 1 shows an embodiment of an imaging system devices used in oil and gas exploration, refining, and transincluding a common front objective lens that has a pupil portation, agriculture, remote sensing, defense and home
divided spectrally and re-imaged with a plurality of lenses land security, Surveillance, astronomy, environmental moni
onto an infrared FPA.
45 toring, etc. The methods and systems described herein have
FIG. 2 shows an embodiment with a divided front objec applications in a variety of fields including but not limited toagriculture, biology, physics, chemistry, defense and hometive lens and an array of infrared sensing FPAs.
FIG. 3A represents an embodiment employing an array of land security, environment, oil and gas industry, etc. The
front objective lenses operably matched with the re-imaging teachings are not intended to be limited to the implementa
lens array. FIG. 3B illustrates a two-dimensional array of 50 tions depicted solely in the Figures, but instead have wide
optical components corresponding to the embodiment of applicability as will be readily apparent to one having
FIG 3A
ordinary skill in the art.
Various embodiments disclosed herein describe a divided
FIG. 4 is a diagram of the embodiment employing anarray of field references (e.g., field stops that can be used as aperture infrared spectral imaging (DAISI) system that isreferences for calibration) and an array of respectively 55 structured and adapted to provide identification of target
corresponding relay lenses.
chemical contents of the imaged scene. The system is based
FIG. 5A is a diagram of a 4-by-3 pupil array comprising on spectrally-resolved imaging and can provide Such iden
circular optical filters (and IR blocking material between the tification with a single-shot (also referred to as a Snapshot)
optical filters) used to spectrally divide an optical wavefront comprising a plurality of images having different wave
60 length compositions that are obtained generally simultane
imaged with an embodiment of the system.
FIG. 5B is a diagram of a 4-by-3 pupil array comprising ously. Without any loss of generality, Snapshot refers to arectangular optical filters (and IR blocking material between system in which most of the data elements that are collected
the optical filters) used to spectrally divide an optical are continuously viewing the light emitted from the scene. In
contrast in Scanning systems, at any given time only awavefront imaged with an embodiment of the system.
FIG. 6A depicts theoretical plots of transmission charac 65 minority of data elements are continuously viewing a scene,teristics of a combination of band-pass filters used with an followed by a different set of data elements, and so on, until
the full dataset is collected. Relatively fast operation can beembodiment of the system.

US 9,562,849 B2
31
32
Some
embodiments
described
herein can also be less sus
achieved in a Snapshot system because it does not need touse spectral or spatial scanning for the acquisition of infra ceptible to motion artifacts in comparison with spatiallyred (IR) spectral signatures of the target chemical contents. and/or spectrally scanning systems which can cause errors in
Instead, IR detectors (such as, for example, infrared focal either the spectral data, spatial data, or both.
FIG. 1 provides a diagram Schematically illustrating spa
plane arrays or FPAs) associated with a plurality of different
optical channels having different wavelength profiles can be tial and spectral division of incoming light by an embodi
used to form a spectral cube of imaging data. Although ment 100 of a divided aperture infrared spectral imager
spectral data can be obtained from a single Snapshot com (DAISI) system that can image an object 110 possessing IR
prising multiple simultaneously acquired images corre spectral signature(s). The system 100 includes a front objec
sponding to different wavelength ranges, in various embodi 10 tive lens 124, an array of optical filters 130, an array ofments, multiple Snap shots may be obtained. In various reimaging lenses 128 and a detector array 136. In various
embodiments, these multiple Snapshots can be averaged. embodiments, the detector array 136 can include a single
Similarly, in certain embodiments multiple Snap shots may FPA or an array of FPAs. Each detector in the detector arraybe obtained and a portion of these can be selected and 136 can be disposed at the focus of each of the lenses in thepossibly averaged. Also, in contrast to commonly used IR 15 array of reimaging lenses 128. In various embodiments, thespectral imaging systems, the DAISI system does not detector array 136 can include a plurality of photo-sensitive
require cooling. Accordingly, it can advantageously use devices. In some embodiments, the plurality of photo
uncooled infrared detectors. For example, in various imple sensitive devices may comprise a two-dimensional imaging
mentations, the imaging systems disclosed herein do not sensor array that is sensitive to radiation having wavelengths
include detectors configured to be cooled to a temperature between 1 Lim and 20 Lum (for example, in near infra-red
below 300 Kelvin. As another example, in various imple wavelength range, mid infra-red wavelength range, or longmentations, the imaging systems disclosed herein do not infra-red wavelength range). In various embodiments, theinclude detectors configured to be cooled to a temperature plurality of photo-sensitive devices can include CCD or
below 273 Kelvin. As yet another example, in various CMOS sensors, bolometers, microbolometers or other
implementations, the imaging systems disclosed herein do 25 detectors that are sensitive to infra-red radiation.
not include detectors configured to be cooled to a tempera
An aperture of the system 100 associated with the front
ture below 250 Kelvin. As another example, in various objective lens system 124 is spatially and spectrally divided
implementations, the imaging systems disclosed herein do by the combination of the array of optical filters 130 and the
not include detectors configured to be cooled to a tempera array of reimaging lenses 128. In various embodiments, theture below 200 Kelvin.
30 combination of the array of optical filters 130 and the array
Implementations disclosed herein provide several advan of reimaging lenses 128 can be considered to form atages over existing IR spectral imaging systems, most if not spectrally divided pupil that is disposed forward of theall of which may require FPAs that are highly sensitive and optical detector array 136. The spatial and spectral division
cooled in order to compensate, during the optical detection, of the aperture into distinct aperture portions forms a plu
for the reduction of the photon flux caused by spectrum 35 rality of optical channels 120 along which light propagates.
scanning operation. The highly sensitive and cooled FPA In various embodiments, the array 128 of re-imaging lenses
systems are expensive and require a great deal of mainte
128a and the array of spectral filters 130 which respectively
nance. Since various embodiments disclosed herein are
correspond to the distinct optical channels 120. The plurality
configured to operate in single-shot acquisition mode with of optical channels 120 can be spatially and/or spectrally
out spatial and/or spectral scanning, the instrument can 40 distinct. The plurality of optical channels 120 can be formed
receive photons from a plurality of points (e.g., every point) in the object space and/or image space. In one implemenof the object Substantially simultaneously, during the single tation, the distinct channels 120 may include optical chanreading. Accordingly, the embodiments of imaging system nels that are separated angularly in space. The array ofdescribed herein can collect a substantially greater amount spectral filters 130 may additionally include a filter-holdingof optical power from the imaged scene (for example, an 45 aperture mask (comprising, for example, IR light-blocking
order of magnitude more photons) at any given moment in materials such as ceramic, metal, or plastic). Light from thetime especially in comparison with spatial and/or spectral object 110 (for example a cloud of gas), the optical prop
scanning systems. Consequently, various embodiments of erties of which in the IR are described by a unique absorp
the imaging systems disclosed herein can be operated using tion, reflection and/or emission spectrum, is received by the
uncooled detectors (for example, FPA unit including an 50 aperture of the system 100. This light propagates througharray of microbolometers) that are less sensitive to photons each of the plurality of optical channels 120 and is further
in the IR but are well fit for continuous monitoring appli imaged onto the optical detector array 136. In various
cations. For example, in various implementations, the imag implementations, the detector array 136 can include at leasting systems disclosed herein do not include detectors con one FPA. In various embodiments, each of the re-imaging
figured to be cooled to a temperature below 300 Kelvin. As 55 lenses 128a can be spatially aligned with a respectively
another example, in various implementations, the imaging corresponding spectral region. In the illustrated implemensystems disclosed herein do not include detectors configured tation, each filter element from the array of spectral filters
to be cooled to a temperature below 273 Kelvin. As yet 130 corresponds to a different spectral region. Each reanother example, in various implementations, the imaging imaging lens 128a and the corresponding filter element ofsystems disclosed herein do not include detectors configured 60 the array of spectral filter 130 can coincide with (or form) a
to be cooled to a temperature below 250 Kelvin. As another portion of the divided aperture and therefore with respec
example, in various implementations, the imaging systems tively-corresponding spatial channel 120. Accordingly, indisclosed herein do not include detectors configured to be various embodiment an imaging lens 128a and a corre
cooled to a temperature below 200 Kelvin. Imaging systems sponding spectral filter can be disposed in the optical path ofincluding uncooled detectors can be capable of operating in 65 one of the plurality of optical channels 120. Radiation fromextreme weather conditions, require less power, are capable the object 110 propagating through each of the plurality ofof operation during day and night, and are less expensive. optical channels 120 travels along the optical path of eachUS 9,562,849 B2
33
34
re-imaging lens 128a and the corresponding filter element of pendicular to optical axis of the lens 124, thus the function
the array of spectral filter 130 and is incident on the detector ality of the system 100 is not necessarily compromised whenarray (e.g., FPA component) 136 to form a single image the objective lens 124 is not included. Generally, however,(e.g., sub-image) of the object 110. The image formed by the the field apertures corresponding to different optical chandetector array 136 generally includes a plurality of sub nels may be located in the same or different planes. These
images formed by each of the optical channels 120. Each of field apertures may be defined by the aperture of the reim
the plurality of Sub-images can provide different spatial and aging lens 128a and/or filters in the divided aperture 130 inspectral information of the object 110. The different spatial certain implementations. In one implementation, the field
information results from Some parallax because of the apertures corresponding to different optical channels can bedifferent spatial locations of the smaller apertures of the 10 located in different planes and the different planes can bedivided aperture. In various embodiments, adjacent Sub optical conjugates of one another. Similarly, while all of theimages can be characterized by close or Substantially equal filter elements in the array of spectral filters 130 of thespectral signatures. The detector array (e.g., FPA compo embodiment 100 are shown to lie in one plane, generallynent) 136 is further operably connected with a processor 150 different filter elements of the array of spectral filter 130 can(not shown). The processor 150 can be programmed to 15 be disposed in different planes. For example, different filter
aggregate the data acquired with the system 100 into a elements of the array of spectral filters 130 can be disposed
spectral data cube. The data cube represents, in spatial (x,y) in different planes that are optically conjugate to one
and spectral (w) coordinates, an overall spectral image of the another. However, in other embodiments, the different filter
object 110 within the spectral region defined by the combi elements can be disposed in non-conjugate planes.
nation of the filter elements in the array of spectral filters
In contrast to the embodiment 100, the front objective lens
130. Additionally, in various embodiments, the processor or 124 need not be a single optical element, but instead canprocessing electronics 150 may be programmed to deter include a plurality of lenses 224 as shown in an embodimentmine the unique absorption characteristic of the object 110. 200 of the DAISI imaging system in FIG. 2. These lenses
Also, the processor/processing electronics 150 can, alterna 224 are configured to divide an incoming optical wavefront
tively or in addition, map the overall image data cube into 25 from the object 110. For example, the array of front objec
a cube of data representing, for example, spatial distribution tive lenses 224 can be disposed so as to receive an IRof concentrations, c, of targeted chemical components wavefront emitted by the object that is directed toward theDAISI system. The plurality of front objective lenses 224
within the field of view associated with the object 110.
Various implementations of the embodiment 100 can divide the wavefront spatially into non-overlapping sections.
include an optional moveable temperature-controlled refer 30 FIG. 2 shows three objective lenses 224 in a front opticalence source 160 including, for example, a shutter system portion of the optical system contributing to the spatial
comprising one or more reference shutters maintained at division of the aperture of the system in this example. The
different temperatures. The reference source 160 can include plurality of objective lenses 224, however, can be configured
a heater, a cooler or a temperature-controlled element con as a two-dimensional (2D) array of lenses. FIG. 2 presents
figured to maintain the reference source 160 at a desired 35 a general view of the imaging system 200 and the resultant
temperature. For example, in various implementations, the field of view of the imaging system 200. An exploded view
embodiment 100 can include two reference shutters main
202 of the imaging system 200 is also depicted in greater
tained at different temperatures. The reference source 160 is detail in a figure inset of FIG. 2. As illustrated in the detailed
removably and, in one implementation, periodically inserted view 202, the embodiment of the imaging system 200
into an optical path of light traversing the system 100 from 40 includes a field reference 204 at the front end of the system.
the object 110 to the detector array (e.g., FPA component) The field reference 204 can be used to truncate the field of
136 along at least one of the channels 120. The removable view. The configuration illustrated in FIG. 2 has an opera
reference source 160 thus can block such optical path. tional advantage over embodiment 100 of FIG. 1 in that theMoreover, this reference source 160 can provide a reference overall size and/or weight and/or cost of manufacture of the
IR spectrum to recalibrate various components including the 45 embodiment 200 can be greatly reduced because the objec
detector array 136 of the system 100 in real time. The tive lens is smaller. Each pair of the lenses in the array 224
configuration of the moveable reference source 160 is fur and the array 128 is associated with a field of view (FOV).
ther discussed below.
Each pair of lenses in the array 224 and the array 128
In the embodiment 100, the front objective lens system receives light from the object from a different angle. Accord
124 is shown to include a single front objective lens posi 50 ingly, the FOV of the different pairs of lenses in the arraytioned to establish a common field-of-view (FOV) for the 224 and the array 128 do not completely overlap as a result
reimaging lenses 128a and to define an aperture stop for the of parallax. As the distance between the imaging system 200
whole system. In this specific case, the aperture stop Sub (portion 202) and the object 110 increases, the overlapping
stantially spatially coincides with and/or is about the same region 230 between the FOVs of the individual lenses 224
size or slightly larger, as the plurality of Smaller limiting 55 increases while the amount of parallax 228 remains approxi
apertures corresponding to different optical channels 120. As mately the same, thereby reducing its effect on the systema result, the positions for spectral filters of the different 200. When the ratio of the parallax-to-object-distance is
optical channels 120 coincide with the position of the Substantially equal to the pixel-size-to-system-focal-length
aperture stop of the whole system, which in this example is ratio then the parallax effect may be considered to beshown as a surface between the lens system 124 and the 60 negligible and, for practical purposes, no longer distinguisharray 128 of the reimaging lenses 128a. In various imple able. While the lenses 224 are shown to be disposed submentations, the lens system 124 can be an objective lens stantially in the same plane, optionally different objective
124. However, the objective lens 124 is optional and various lenses in the array of front objective lenses 224 can beembodiments of the system 100 need not include the objec disposed in more than one plane. For example, Some of thetive lens 124. In various embodiments, the objective lens 65 individual lenses 224 can be displaced with respect to some124 can slightly shifts the images obtained by the different other individual lenses 224 along the axis 226 (not shown)detectors in the array 136 spatially along a direction per and/or have different focal lengths as compared to someUS 9,562,849 B2
35

36
opticalcomponents(e.g.,
objectivelenses 324a, filter ele
other lenses 224. As discussed below, the field reference 204
ments of the array of spectral filters 130, re-imaging lenses
can be useful in calibrating the multiple detectors 236.
In one implementation, the front objective lens system
128a and FPA units 336) are arranged as a 4x3 array. In one
Such as the array of lenses 224 is configured as an array of implementation, the 4x3 array 340 of optical components
lenses integrated or molded in association with a monolithic 5 (lenses 324a, 128a; detector elements 336) is used behind
Substrate. Such an arrangement can reduce the costs and the temperature controlled reference target 160. The fieldcomplexity otherwise accompanying the optical adjustment reference aperture 338 can be adapted to obscure and/orof individual lenses within the system. An individual lens block a peripheral portion of the bundle of light propagating
224 can optionally include a lens with varying magnifica from the object 110 towards the FPA units 336. As a result,tion. As one example, a pair of thin and large diameter 10 the field reference 338 obscures and/or blocks the border or
Alvarez plates can be used in at least a portion of the front peripheral portion(s) of the images of the object 110 formed
objective lens system. Without any loss of generality, the on the FPA elements located along the perimeter 346 of theAlvarez plates can produce a change in focal length when detector system. Generally, two elements of the FPA unit
will produce Substantially equal values of digital counts
translated orthogonally with respect to the optical beam.
In further reference to FIG. 1, the detector array 136 (e.g., 15 when they are used to observe the same portion of the scene
FPA component) configured to receive the optical data in the same spectral region using the same optical train. If
representing spectral signature(s) of the imaged object 110 any of these input parameters (for example, Scene to be
can be configured as a single imaging array (e.g., FPA) 136. observed, spectral content of light from the scene, or optical
This single array may be adapted to acquire more than one elements delivering light from the scene to the two detector
image (formed by more than one optical channel 120) elements) differ, the counts associated with the elements ofsimultaneously. Alternatively, the detector array 136 may the FPA unit will differ as well. Accordingly, and as aninclude a FPA unit. In various implementations, the FPA unit example, in a case when the two FPAs of the FPA unit 336
can include a plurality of optical FPAs. At least one of these (such as those denoted as #6 and #7 in FIG. 3B) remain
plurality of FPAS can be configured to acquire more than one substantially un-obscured by the field reference 338, thespectrally distinct image of the imaged object. For example, 25 outputs from these FPAS can be dynamically adjusted to the
output from one of the FPAs located along perimeter 346
as shown in the embodiment 200 of FIG. 2, in various
embodiments, the number of FPAS included in the FPA unit
(such as, for example, the FPA element #2 or FPA element
may correspond to the number of the front objective lenses #11) that processes light having similar spectral character
224. In the embodiment 200 of FIG. 2, for example, three istics.
FPAS 236 are provided corresponding to the three objective 30 FIG. 4 illustrates schematically a portion of anotherlenses 224. In one implementation of the system, the FPA embodiment of an imaging system 400 that contains an array
unit can include an array of microbolometers. The use of 424 of front objective lenses 424a. The array 424 of lenses
multiple microbolometers advantageously allows for an 424a adapted to receive light from the object 110 and relay
inexpensive way to increase the total number of detection the received light to the array 128 of re-imaging lenses 128a
elements (i.e. pixels) for recording of the three-dimensional 35 through an array 438 of field references (or field stops) 438a,data cube in a single acquisition event (i.e. one Snapshot). In and through an array 440 of the relay lenses. The spectral
various embodiments, an array of microbolometers more characteristics of the field references/field stops 438a can be
efficiently utilizes the detector pixels of the array of FPAs known. The field references 438a are disposed at corre
sponding intermediate image planes defined, with respect to(e.g., each FPA) as the number of unused pixels is reduced,
minimized and/or eliminated between the images that may 40 the object 110, by respectively corresponding front objective
lenses 424a. When refractive characteristics of all of theexist when using a single microbolometer.
FIG. 3A illustrates schematically an embodiment 300 of front objective lenses 424a are substantially the same, all of
the imaging system in which the number of the front the field references 438a are disposed in the same plane. A
objective lenses 324a in the lens array 324, the number of field reference 438a of the array 438 obscures (or casts a
re-imaging lenses 128a in the lens array 128, and the number 45 shadow on) a peripheral region of a corresponding image
of FPAS 336 are the same. So configured, each combination (e.g., Sub-image) formed at the detector plane 444 throughof respectively corresponding front objective lens 324, re a respectively corresponding spatial imaging channel 450 ofimaging lens 128a, and FPAS 336 constitutes an individual the system 400 prior to such image being spectrally pro
imaging channel. Such a channel is associated with acqui cessed by the processor 150. The array 440 of relay lenses
sition of the IR light transmitted from the object 110 through 50 then transmits light along each of the imaging channels 450
an individual filter element of the array of optical filters 130. through different spectral filters 454a of the filter array 454,
A field reference 338 of the system 300 is configured to have past the calibration apparatus that includes two temperaturea uniform temperature across its Surface and be character controlled shutters 460a, 460b, and then onto the detector
ized by a predetermined spectral curve of radiation emanat
module 456. In various embodiments, the detector module
ing therefrom. In various implementations, the field refer 55 456 can include a microbolometer array or some other IR
ence 338 can be used as a calibration target to assist in FPA.
The embodiment 400 has several operational advantages.
calibrating or maintaining calibration of the FPA. Accord
ingly, in various implementations, the field reference 338 is It is configured to provide a spectrally known object within
used for dynamically adjusting the data output from each every image (e.g., Sub-image) and for every Snapshot acqui
FPA 336 after acquisition of light from the object 110. This 60 sition which can be calibrated against. Such spectral cer
dynamic calibration process helps provide that output of the tainty can be advantageous when using an array of IR FPAS
different (e.g., most, or each of the) FPA 336 represents like microbolometers, the detection characteristics of whichcorrect acquired data, with respect to the other FPAS 336 for can change from one imaging frame to the next due to, inpart, changes in the scene being imaged as well as theanalysis, as discussed below in more detail.
FIG. 3B illustrates the plan view perpendicular to the axis 65 thermal effects caused by neighboring FPAs. In various
226 of an embodiment 300 of the imaging system illustrated embodiments, the field reference array 438 of the embodi
ment 400—can be disposed within the Rayleigh rangein FIG. 3A. For the embodiment shown in FIG. 3B, theUS 9,562,849 B2
37
38(approximately corresponding to the depth of focus) asso various embodiments of imaging systems disclosed herein isciated with the front objective lenses 424, thereby removing depicted in FIG. 6B. Filters of FIG. 6C can be referred to as
unusable blurred pixels due to having the field reference long-wavelength pass, LP filters. An LP filter generally
outside of this range. Additionally, the embodiment 400 of attenuates shorter wavelengths and transmits (passes) longer
FIG. 4 can be more compact than, for example, the con 5 wavelengths (e.g., over the active range of the target IR
figuration 300 of FIG. 3A. In the system shown in FIG. 3A, portion of the spectrum). In various embodiments, shortfor example, the field reference 338 may be separated from wavelength-pass filters, SP, may also be used. An SP filter
the lens array 324 by a distance greater than several (for generally attenuates longer wavelengths and transmits
example, five) focal lengths to minimize/reduce blur con (passes) shorter wavelengths (e.g., over the active range oftributed by the field reference to an image formed at a 10 the target IR portion of the spectrum). At least in part due todetector plane.
In various embodiments, the multi-optical FPA unit of the the Snap-shot/non-Scanning mode of operation, embodi
IR imaging system can additionally include an FPA config ments of the imaging system described herein can use lessured to operate in a visible portion of the spectrum. In sensitive microbolometers without compromising the SNR.
reference to FIG. 1, for example, an image of the scene of 15 The use of microbolometers, as detector-noise-limited
interest formed by such visible-light FPA may be used as a devices, in turn not only benefits from the use of spectrally
background to form a composite image by overlapping an IR multiplexed filters, but also does not require cooling of theimage with the visible-light image. The IR image may be imaging system during normal operation.
Referring again to FIGS. 6A, 6B, 6C, and 6D, each of theoverlapped virtually, with the use of a processor and spe
cifically-designed computer program product enabling Such filters (Ob . . . 3d) transmits light in a substantially wider
data processing, or actually, by a viewer. The IR image may region of the electromagnetic spectrum as compared to thosebe created based on the image data acquired by the indi of the filters (0a . . .3a). Accordingly, when the spectrally
vidual FPAS 136. The so-formed composite image facilitates multiplexed set of filters (Ob . . . 0d) is used with an
the identification of the precise spatial location of the target embodiment of the imaging system, the overall amount ofspecies, the spectral signatures of which the system is able 25 light received by the FPAs (for example, 236, 336) is larger
to detect/recognize.
than would be received when using the bandpass filters
Optical Filters.
(Oa. . . 4a). This “added transmission of light defined by the
The optical filters, used with an embodiment of the use of the spectrally-multiplexed LP (or SP) filters facilitatessystem, that define spectrally-distinct IR image (e.g., Sub
increase of the signal on the FPAS above the level of theimage) of the object can employ absorption filters, interfer 30 an
detector noise. Additionally, by using, in an embodiment ofence filters, and Fabry-Perot etalon based filters, to nameimaging system, filters having spectral bandwidths
just a few. When interference filters are used, the image thegreaterthan those of band-pass filters, the uncooled FPAs of
acquisition through an individual imaging channel defined the embodimentof the imaging system experience lessby an individual re-imaging lens (such as a lens 128a of heating from radiation
incident thereon from the imaged
FIGS. 1, 2, 3, and 4) may be carried out in a single spectral 35
scene
and
from
radiation
emanating from the FPA in ques
bandwidth or multiple spectral bandwidths. Referring againtionitself.
This
reducedheating
is due to a reduction in theto the embodiments 100, 200, 300, 400 of FIGS. 1 through4, and in further reference to FIG. 3B, examples of a 4-by-3 back-reflected thermal emission(s) coming from the FPA
array of spectral filters 130 is shown in FIGS. 5A and 5B. and reflecting off of the filter from the non-band-pass
Individual filters 1 through 12 are juxtaposed with a Sup 40 regions. As the transmission region of the multiplexed LP
porting opto-mechanical element (not shown) to define a (or SP) filters is wider, such parasitic effects are reduced
filter-array plane that is oriented, in operation, Substantially thereby improving the overall performance of the FPA unit.
perpendicularly to the general optical axis 226 of the imag
In one implementation, the LP and SP filters can be
ing system. In various implementations, the individual filters combined, in a spectrally-multiplexed fashion, in order to
1 through 12 need not be discrete optical components. 45 increase or maximize the spectral extent of the transmission
Instead, the individual filters 1 through 12 can comprise one region of the filter system of the embodiment.
The advantage of using spectrally multiplexed filters isor more coatings that are applied to one or more Surfaces of
the reimaging lenses (such as a lens 128a of FIGS. 1, 2, 3, appreciated based on the following derivation, in which asystem of M filters is examined (although it is understoodand 4) or the Surfaces of one or more detectors.
The optical filtering configuration of various embodi 50 that in practice an embodiment of the invention can employ
ments disclosed herein may advantageously use a bandpass any number of filters). As an illustrative example, the casefilter defining a specified spectral band. Any of the filters 0a of M-7 is considered. Analysis presented below relates tothrough 3a the transmission curves of which are shown in one spatial location in each of the images (e.g., Sub-images)
FIG. 6A may, for example, be used. The filters may be formed by the differing imaging channels (e.g., different
placed in front of the optical FPA (or generally, between the 55 optical channels 120) in the system. A similar analysis can
optical FPA and the object). In particular, and in further be performed for each point at an image (e.g., Sub-image),reference to FIGS. 1, 2, 3, and 4, when optical detector and thus the analysis can be appropriately extended asarrays 136, 236, 336, 456 include microbolometers, the required.
predominant contribution to noise associated with image
The unknown amount of light within each of the M
acquisition is due to detector noise. To compensate and/or 60 spectral channels (corresponding to these M filters) is
reduce the noise, various embodiments disclosed herein
denoted with f. f. f. . . . f. and readings from correutilize spectrally-multiplexed filters. In various implemen sponding detector elements receiving light transmitted bytations, the spectral-multiplexed filters can comprise a plu each filter is denoted as g. g. g . . . g., while measurement
rality of long pass filters, a plurality long pass filters, a errors are represented by n, n2 ns. . . . n. Then, theplurality of band pass filters and any combinations thereof. 65 readings at the seven FPA pixels each of which is opticallyAn example of the spectral transmission characteristics of filtered by a corresponding band-pass filter of FIG. 6A canspectrally-multiplexed filters Ob through 3d for use with be represented by:US 9,562,849 B2
39

40

g-fi+n',g2 f+m2,
gs f+ns,
gaft-na.
gs fishns,
go fins,

10

These readings (pixel measurements) g, are estimates of
These fare unbiased estimates when the n, are Zero mean
the spectral intensities f. The estimates g, are not equal to the 15 stochastic variables, so that (f-f)=0. The measurement
corresponding f, values because of the measurement errors variance corresponding to i' measurement is given by then. However, if the measurement noise distribution has zero equation below:
mean, then the ensemble mean of each individual measure
ment can be considered to be equal to the true value, i.e.
(g) f. Here, the angle brackets indicate the operation of
From the above equation, it is observed that by employing
calculating the ensemble mean of a stochastic variable. The spectrally-multiplexed system the signal-to-noise ratio
variance of the measurement can, therefore, be represented (SNR) of a measurement is improved by a factor ofaS
W1677–151.
For N channels, the SNR improvement achieved with a
25 spectrally-multiplexed system can be expressed as (N+1)/(2In embodiments utilizing spectrally-multiplexed filters, in
N). For example, an embodiment employing 12 spectral
comparison with the embodiments utilizing band-pass fil channels(N=12) is characterized by a SNR improvement,
ters, the amount of radiant energy transmitted by each of the over a non-spectrally-multiplexed system, comprising aspectrally-multiplexed LP or SP filters towards a givenof up to 1.88.
detector element can exceed that transmitted through a 30 factor
Two additional examples of related spectrally-multi
spectral band of aband-pass filter. In this case, the intensities plexed filter arrangements Oc through 3c and 0d through 3dof light corresponding to the independent spectral bands can thatcan be used in various embodiments of the imaging
be reconstructed by computational means. Such embodi systems
described herein are shown in FIGS. 6C and 6D,
ments can be referred to as a “multiplex design”.
respectively. The spectrally-multiplexed filters shown in
One matrix of such “multiplexed filter’ measurements FIGS. 6C and 6D can be used in embodiments of imaging
includes a Hadamard matrix requiring "negative' filters that systems employing uncooled FPAS (Such as microbolom
may not be necessarily appropriate for the optical embodi eters). FIG. 6C illustrates a set of spectrally-multiplexed
ments disclosed herein. An S-matrix approach (which is long-wavelength pass (LP) filters used in the system. An LP
restricted to having a number of filters equal to an integer filter generally attenuates shorter wavelengths and transmits
that is multiple of four minus one) or a row-doubled Had 40 (passes) longer wavelengths (e.g., over the active range ofamard matrix (requiring a number of filters to be equal to an the target IR portion of the spectrum). A single spectral
integer multiple of eight) can be used in various embodi channel having a transmission characteristic corresponding
ments. Here, possible numbers of filters using an S-matrix to the difference between the spectral transmission curves ofsetup are 3, 7, 11, etc and, if a row-doubled Hadamard
least two of these LP filters can be used to procure
matrix setup is used, then the possible number of filters is 8, 45 at
imagingdata for the data cube using an embodiment of the16, 24, etc. For example, the goal of the measurement may system describedherein. In various implementations, thebe to measure seven spectral band f, intensities using seven spectral filters disposed with respect to the different FPAs
measurements g, as follows:
can have different spectral characteristics. In various imple
mentations, the spectral filters may be disposed in front of
50 only some of the FPAS while the remaining FPAS may be
configured to receive unfiltered light. For example, in some
implementations, only 9 of the 12 detectors in the 4x3 arrayof detectors described above may be associated with aspectral filter while the other 3 detectors may be configured
55 to received unfiltered light. Such a system may be config
ured to acquire spectral data in 10 different spectral channels
in a single data acquisition event.
The use of microbolometers, as detector-noise-limited

devices, in turn not only can benefit from the use of
60 spectrally multiplexed filters, but also does not requirecooling of the imaging system during normal operation. In
Optical transmission characteristics of the filters contrast to imaging systems that include highly sensitive
described above are depicted in FIG. 6B. Here, a direct FPA units with reduced noise characteristics, the embodi
estimate of the f, is no longer provided through a relationship ments of imaging systems described herein can employ lesssimilar to (g) f. Instead, if a “hat notation is used to 65 sensitive microbolometers without compromising the SNR.
denote an estimate of a given value, then a linear combina This result is at least in part due to the Snap-shot/nonscanning mode of operation.
tion of the measurements can be used such as, for example,

US 9,562,849 B2
41
42
As discussed above, an embodiment may optionally, and digital numbers, or “counts”) as g, to a source of radiance L.
in addition to a temperature-controlled reference unit (for the readings corresponding to the measurements of the two
example temperature controlled shutters such as shutters shutters can be expressed as:
160; 160a, 160b, 460a, 460b), employ a field referencecomponent (e.g., field reference aperture 338 in FIG. 3A), or 5
an array of field reference components (e.g., filed reference
apertures 438 in FIG. 4), to enable dynamic calibration.
Such dynamic calibration can be used for spectral acquisi
Here, g, is the pixel offset value (in units of counts),tion of one or more or every data cube. Such dynamic and Y is the pixel gain value (in units of counts per radiancecalibration can also be used for a spectrally-neutral camera 10 unit). The solutions of these two equations with respect toto-camera combination to enable dynamic compensation of the two unknowns g, and Y can be obtained if the values
parallax artifacts. The use of the temperature-controlled of g and g and the radiance values L and L are available.
reference unit (for example, temperature-controlled shutter These values can, for example, be either measured by asystem 160) and field-reference component(s) facilitates reference instrument or calculated from the known tempera
maintenance of proper calibration of each of the FPAS 15 tures T and T together with the known spectral response of
the optical system and FPA. For any Subsequent measureindividually and the entire FPA unit as a whole.
In particular, and in further reference to FIGS. 1, 2, 3, and ment, one can then invert the equation(s) above in order to4, the temperature-controlled unit generally employs a sys estimate the radiance value of the object from the detector
tem having first and second temperature Zones maintained at measurement, and this can be done for each pixel in eachfirst and second different temperatures. For example, shutter FPA within the system.
As already discussed, and in reference to FIGS. 1 throughsystem of each of the embodiments 100, 200, 300 and 400
can employ not one but at least two temperature-controlled 4, the field-reference apertures may be disposed in an object
shutters that are Substantially parallel to one another and space or image space of the optical system, and dimensioned
transverse to the general optical axis 226 of the to block a particular portion of the IR radiation received
embodiment(s) 100, 200, 300, 400. Two shutters at two 25 from the object. In various implementations, the field
different temperatures may be employed to provide more reference aperture, the opening of which can be substantiallyinformation for calibration; for example, the absolute value similar in shape to the boundary of the filter array (forof the difference between FPAS at one temperature as well as example, and in reference to a filter array of FIGS. 3B.
the change in that difference with temperature change can be 5B e.g., rectangular). The field-reference aperture can berecorded. Referring, for example, to FIG. 4, in which such 30 placed in front of the objective lens (124, 224, 324, 424) at
multi-shutter structure is shown, the use of multiple shutters a distance that is at least several times (in one implemen
enables the user to create a known reference temperature tation—at least five times) larger than the focal length of thedifference perceived by the FPAS 456. This reference tem lens such that the field-reference aperture is placed closer toperature difference is provided by the IR radiation emitted the object. Placing the field-reference aperture closer to theby the shutter(s) 160a, 160b when these shutters are posi 35 object can reduce the blurriness of the image. In the embodi
tioned to block the radiation from the object 110. As a result, ment 400 of FIG. 4, the field-reference aperture can be
not only the offset values corresponding to each of the placed within the depth of focus of an image conjugate plane
individual FPAS pixels can be adjusted but also the gain formed by the front objective lens 424. The field reference,
generally, can facilitate, effectuates and/or enable dynamic
values of these FPAS. In an alternative embodiment, thesystem having first and second temperature Zones may 40 compensation in the system by providing a spectrally known
include a single or multi-portion piece. This single or and temporally-stable object within every scene to reference
multi-portion piece may comprise for example a plate. This and stabilize the output from the different FPAS in the array.
piece may be mechanically-movable across the optical axis
Because each FPA's offset value is generally adjustedwith the use of appropriate guides and having a first portion from each frame to the next frame by the hardware, com
at a first temperature and a second portion at a second 45 paring the outputs of one FPA with another can have an error
temperature.
that is not compensated for by the static calibration param
Indeed, the process of calibration of an embodiment of the eters g, and Y established, for example, by the movableimaging system starts with estimating gain and offset by shutters 160. In order to ensure that FPAS operate in radio
performing measurements of radiation emanating, indepen metric agreement over time, it is advantageous for a portion
dently, from at least two temperature-controlled shutters of 50 of each detector array to view a reference Source (such as the
known and different radiances. The gain and offset can vary field reference 338 in FIG. 3A, for example) over a plurality
from detector pixel to detector pixel. Specifically, first the of frames obtained over time. If the reference source spec
response of the detector unit 456 to radiation emanating trum is known a priori (such as a blackbody source at afrom one shutter is carried out. For example, the first shutter known temperature), one can measure the response of each160a blocks the FOV of the detectors 456 and the tempera 55 FPA to the reference source in order to estimate changes toture T is measured directly and independently with therm the pixel offset value. However, the temperature of theistors. Following Such initial measurement, the first shutter reference Source need not be known. In Such implementa
160a is removed from the optical path of light traversing the tions, dynamic calibration of the different detectors can beembodiment and another second shutter (for example, 160b) performed by monitoring the change in the gain and theis inserted in its place across the optical axis 226 to prevent 60 offset for the various detectors from the time the movable
the propagation of light through the system. The temperature shutters used for static calibration are removed. An exampleof the second shutter 160b can be different than the first
calculation of the dynamic offset proceeds as follows.
Among the FPA elements in an array of FPAS in an
shutter (TzT). The temperature of the second shutter 160b
is also independently measured with thermistors placed in embodiment of the imaging system, one FPA can be selectedcontact with this shutter, and the detector response to radia 65 to be the “reference FPA'. The field reference temperaturetion emanating from the shutter 160b is also recorded. measured by all the other FPAS can be adjusted to agree with
Denoting operational response of FPA pixels (expressed in the field reference temperature measured by the reference asUS 9,562,849 B2
44also be highlighted in and contribute to the difference image.
Such contributing effects include, to name just a few, parpreviously obtained calibration parameters g., and Y (the allax-induced imaging of edges of the object, influence of
pixel offset and gain), the effective blackbody temperature T, magnification differences between the two or more opticalof the field reference as measured by each FPA is estimated 5 channels, and differences in rotational positioning and ori
using the equation below:
entation between the FPAs. While magnification-related
errors and FPA-rotation-caused errors can be compensatedfor by increasing the accuracy of the instrument construction
Using the equation above, the mean value over all pixels as well as by post-processing of the acquired imaging,that are obscured by the field reference is obtained. In the 10 parallax is scene-induced and is not so easily correctable. In
above equation Ag, is the difference in offset value of the addition, the spectral differencing method is vulnerable tocurrent frame from Age, obtained during the calibration radiance calibration errors. Specifically, if one FPA registers
step. For the reference FPA, Ag, can be simply set to zero. radiance of light from a given feature of the object as that
Then, using the temperature differences measured by each having
a temperature of 40°C., for example, while the data
15
FPA, one obtains
from another FPA represents the temperature of the same
object feature as being 39°C., then such feature of the object
will be enhanced or highlighted in the difference image
Once AT, for each FPA is measured, its value can be (formed at least in part based on the images provided by
Subtracted from each image in order to force operational these two FPAs) due to such radiance-calibration error.
One solution to Some of Such problems is to compare
agreement between such FPA and the reference FPA. Whilethe calibration procedure has been discussed above in ref (e.g., Subtract) images from the same FPA obtained at
erence to calibration of temperature, a procedurally similar different instances in time. For example, images can bemethodology of calibration with respect to radiance value compared to or Subtracted from a reference image obtained
25 at another time. Such reference image, which is subtracted
can also be implemented.
Examples of Methodology of Measurements.
from other later obtained images, may be referred to as a
Prior to optical data acquisition using an embodiment of temporal reference image. This solution can be applied tospectral difference images as well. For example, the image
the IR imaging system as described herein, one or more,most, or potentially all the FPAs of the system can be data resulting from spectral difference images can be nor
calibrated. For example, greater than 50%, 60%, 70%, 80% 30 malized by the data corresponding to a temporal referenceor 90% or the FPAS336 can be initially calibrated. As shown image. For instance, the temporal reference images can bein FIG. 3, these FPAS 336 may form separate images of the subtracted from the spectral difference image to obtain theobject using light delivered in a corresponding optical temporal difference image. This process is referred to, forchannel that may include the combination of the correspond the purposes of this disclosure, as a temporal differencing
ing front objective and re-imaging lenses 324, 128. The 35 algorithm or method and the resultant image from Subtract
calibration procedure can allow formation of individual ing the temporal reference image from another image (suchimages in equivalent units (so that, for example, the reading as the spectral difference image) is referred to as the temfrom the FPA pixels can be re-calculated in units of tem poral difference image. In some embodiments where spec
perature or radiance units, etc.). Moreover, the calibration tral differencing is employed, a temporal reference image
process can also allow the FPAs (e.g., each of the FPAs) to 40 may be formed, for example, by creating a spectral differ
be spatially co-registered with one another so that a given ence image from the two or more images registered by the
pixel of a particular FPA can be optically re-mapped through two or more FPAS at a single instance in time. This spectral
the optical system to the same location at the object as the difference image is then used as a temporal reference image.
corresponding pixel of another FPA.
The temporal reference image can then be subtracted fromTo achieve at least Some of these goals, a spectral differ 45 other later obtained images to provide normalization that can
encing method may be employed. The method involves be useful in Subtracting out or removing various errors or
forming a difference image from various combinations of deleterious effects. For example, the result of the algorithm
the images from different channels. In particular, the images is not affected by a prior knowledge of whether the object orused to form difference images can be registered by two or scene contains a target species (such as gas of interest),
more different FPAS in spectrally distinct channels having 50 because the algorithm can highlight changes in the scenedifferent spectral filters with different spectral characteris characteristics. Thus, a spectral difference image can betics. Images from different channels having different spectral calculated from multiple spectral channels as discussed
characteristics will provide different spectral information. above based on a Snap-shot image acquisition at any later
Comparing (e.g., Subtracting) these images, can therefore time and can be subtracted from the temporal reference
yield valuable spectral based information. For example, if 55 image to form a temporal difference image. This temporalthe filter element of the array of spectral filters 130 corre difference image is thus a normalized difference image. The
sponding to a particular FPA 336 transmits light from the difference between the two images (the temporal difference
object 110 including a cloud of gas, for example, with a image) can highlight the target species (gas) within thecertain spectrum that contains the gas absorption peak or a normalized difference image, since this species was not
gas emission peak while another filter element of the array 60 present in the temporal reference frame. In various embodi
of spectral filters 130 corresponding to another FPA 336 ments, more than two FPAS can be used both for registering
does not transmit such spectrum, then the difference between the temporal reference image and a later-acquired difference
the images formed by the two FPAS at issue will highlight image to obtain a better SNR figure of merit. For example,
the presence of gas in the difference image.
if two FPAS are associated with spectral filters having the
A shortcoming of the spectral differencing method is that 65 same spectral characteristic, then the images obtained by thecontributions of Some auxiliary features associated with two FPAS can be combined after they have been registeredimaging (not just the target species Such as gas itself) can to get a better SNR figure.

43
discussed below. The image obtained by each FPA includes
a set of pixels obscured by the field reference 338. Using theUS 9,562,849 B2
45
46
While the temporal differencing method can be used to 2-3 used for target species detection, a conclusion can be
reduce or eliminate some of the shortcomings of the spectral made that the image features corresponding to these pixels
differencing, it can introduce unwanted problems of its own. are likely to be induced by parallax and not the presence ofFor example, temporal differencing of imaging data is less target species in the imaged scene. It should be noted thatsensitive to calibration and parallax induced errors than the compensation of parallax can also be performed using
spectral differencing of imaging data. However, any change images created by individual re-imaging lenses, 128a, whenin the imaged scene that is not related to the target species using a single FPA or multiple FPA's as discussed above.
of interest (such as particular gas, for example) is high FPAS spaced apart from each other in different directions canlighted in a temporally-differenced image. Thus Such change also be useful. Greater than 2, for example, 3 or 4, or morein the imaged scene may be erroneously perceived as a 10 FPAs can be used to establish parallax for parallax correc
location of the target species triggering, therefore, an error tion. In certain embodiments two central FPAS and one
in detection of target species. For example, if the tempera corner FPA are used for parallax correction. These FPA may,ture of the background against which the gas is being in certain embodiments, have Substantially similar or thedetected changes (due to natural cooling down as the day same spectral characteristics, for example, have filters hav
progresses, or increases due to a person or animal or another 15 ing similar or the same transmission spectrum or have no
object passing through the FOV of the IR imaging system), filter at all.
then such temperature change produces a signal difference
Another capability of the embodiments described herein
as compared to the measurement taken earlier in time. is the ability to perform the volumetric estimation of a gas
Accordingly, the cause of the Scenic temperature change (the cloud. This can be accomplished by using (instead ofcooling object, the person walking, etc.) may appear as the compensating or negating) the parallax induced effects
detected target species (such as gas). It follows, therefore, described above. In this case, the measured parallax between
that an attempt to compensate for operational differences two or more similar spectral response images (e.g., two oramong the individual FPAs of a multi-FPA IR imaging more channels or FPAs) can be used to estimate a distancesystem with the use of methods that turn on spectral or between the imaging system and the gas cloud or between
temporal differencing can cause additional problems leading 25 the imaging system and an object in the field of view of theto false detection of target species. Among these problems system. The parallax induced transverse image shift, d.
are scene-motion-induced detection errors and parallax between two images is related to the distance, Z, between thecaused errors that are not readily correctable and/or com cloud or object 110 and the imaging system according to thepensatable. Accordingly, there is a need to compensate for equation Z-SZ/d. Here, S, is the separation between two
image data acquisition and processing errors caused by 30 similar spectral response images, and Z' is the distance to the
motion of elements within the scene being imaged. Various image plane from the back lens. The value for Z is typicallyembodiments of data processing algorithms described herein approximately equal to the focal length f of the lens of theaddress and fulfill the need to compensate for such motion imaging system. Once the distance Z between the cloud andinduced and parallax-induced image detection errors.
the imaging system is calculated, the size of the gas cloudIn particular, to reduce or minimize parallax-induced 35 can be determined based on the magnification, m f/Z, where
differences between the images produced with two or more each image pixel on the gas cloud, AX, corresponds to apredetermined FPAS, another difference image can be used physical size in object space AXAX'/m. To estimate thethat is formed from the images of at least two different FPAS Volume of the gas cloud, a particular symmetry in theto estimate parallax effects. Parallax error can be determined thickness of the cloud based on the physical size of the cloudby comparing the images from two different FPAs where the 40 can be assumed. For example, the cloud image can beposition between the FPAS is known. The parallax can be rotated about a central axis running through the cloud image
calculated from the known relative position difference. Dif to create a three dimensional Volume estimate of the gas
ferences between the images from these two FPAS can be cloud size. It is worth noting that in the embodiments
attributed to parallax, especially, if the FPA have the same described herein only a single imaging system is required forspectral characteristics, for example have the same spectral 45 such volume estimation. Indeed, due to the fact that thefilter or both have no spectral filters. Parallax error correc information about the angle at which the gas cloud is seenby the system is decoded in the parallax effect, the image
tion, however, can still be obtained from two FPAS that havedifferent spectral characteristics or spectral filters, especially data includes the information about the imaged scene
if the different spectral characteristics, e.g., the transmission viewed by the system in association with at least two angles.
spectra of the respective filters are known and/or negligible. 50 When the temporal differencing algorithm is used for
Use of more than two FPAS or FPAs of different locations
processing the acquired imaging data, a change in the scene
such as FPAS spaced farther apart can be useful. For that is not caused by the target species can inadvertently be
example, when the spectral differencing of the image data is highlighted in the resulting image. In various embodiments,
performed with the use of the difference between the images compensation for this error makes use of the temporalcollected by the outermost two cameras in the array (such as, 55 differencing between two FPAs that are substantially equallyfor example, the FPAS corresponding to filters 2 and 3 of the spectrally sensitive to the target species. In this case, thearray of filters of FIG. 5A), a difference image referred to as temporal difference image will highlight those pixels thea “difference image 2-3 is formed. In this case, the alter intensity of which have changed in time (and not in wave
native “difference image 1-4” is additionally formed from length). Therefore, Subtracting the data corresponding to
the image data acquired by, for example, the alternative 60 these pixels on both FPAS, which are substantially equally
FPAs corresponding to filters 1 and 4 of FIG. 5A. Assuming spectrally sensitive to the target species, to form the result
or ensuring that both of these two alternative FPAS have ing image, excludes the contribution of the target species toapproximately the same spectral sensitivity to the target the resulting image. The differentiation between (i) changes
species, the alternative “difference image 1-4 will highlight in the scene due to the presence of target species and (ii)pixels corresponding to parallax-induced features in the 65 changes in the scene caused by changes in the backgroundimage. Accordingly, based on positive determination that the not associated with the target species is, therefore, possible.
same pixels are highlighted in the spectral "difference image In some embodiments, these two channels having the sameUS 9,562,849 B2
47or Substantially similar spectral response so as be substan
tially equally spectrally sensitive to the target species may
comprise FPAs that operate using visible light. It should also
be noted that, the data acquired with a visible light FPA(when present as part of the otherwise IR imaging system)

48
methane. The multispectral data obtained in the one or more
optical channels can be correlated with a predeterminedabsorption spectrum of methane in the wavelength range
between about 7.9 um and 8.4 Lum. In various implementa
tions, the predetermined absorption spectrum of methane
can also be used to facilitate such differentiation and com
can be saved in a database or a reference library accessible
pensation of the motion-caused imaging errors. Visible by the system. Based on an amount of correlation (e.g., acameras generally have much lower noise figure than IR value of correlation coefficient), the presence or absence of
cameras (at least during daytime). Consequently, the tem methane in the gas cloud can be detected.
poral difference image obtained with the use of image data 10
Examples of Practical Embodiments and Operationfrom the visible light FPA can be quite accurate. The visible
FPA can be used to compensate for motion in the system as
The embodiment 300 of FIG. 3 is configured to employ 12
well as many potential false-alarms in the scene due tomotion caused by people, vehicles, birds, and steam, for optical channels and 12 corresponding microbolometer
example, as long as the moving object can be observed in the 15 FPAs 336 to capture a video sequence substantially imme
visible region of the spectra. This has the added benefit of diately after performing calibration measurements. The
providing an additional level of false alarm Suppression Video sequence corresponds to images of a standard labo
without reducing the sensitivity of the system since many ratory scene and the calibration measurements are pertargets Such as gas clouds cannot be observed in the visible formed with the use of a reference source including two
spectral region. In various implementations, an IR camera shutters, as discussed above, one at room temperature andcan be used to compensate for motion artifacts.
one 5° C. above room temperature. The use of 12 FPAs
Another method for detection of the gases is to use a allows increased chance of simultaneous detection and esti
spectral unmixing approach. A spectral unmixing approach mation of the concentrations of about 8 or 9 gases present atassumes that the spectrum measured at a detector pixel is the scene. In various embodiments, the number of FPAS 336
composed of a sum of component spectra (e.g., methane and 25 can vary, depending on the balance between the operational
other gases). This approach attempts to estimate the relative requirements and consideration of cost.
Due to the specifics of operation in the IR range of theweights of these components needed to derive the measure
ment spectrum. The component spectra are generally taken spectrum, the use of the so-called noise-equivalent tempera
from a predetermined spectral library (for example, from ture difference (or NETD) is preferred and is analogous todata collection that has been empirically assembled), though 30 the SNR commonly used in visible spectrum instruments.
Sometimes one can use the scene to estimate these as well
The array of microbolometer FPAS 336 is characterized to(often called “endmember determination”). In various perform at NETDs.72 mK at an f-number of 1.2. Eachembodiments, the image obtained by the detector pixel is a measurement was carried out by Summing four consecutive
radiance spectrum and provides information about the frames, and the reduction in the NETD value expected duebrightness of the object. To identify the contents of a gas 35 to Such Summation would be described by corresponding
cloud in the scene and/or to estimate the concentration of thefactor of V4–2. Under ideal measurement conditions, there
various gases in the gas cloud, an absorption/emission fore, the FPANETD should be about 36 mK.
It is worth noting that the use of optically-filtered FPAS inspectrum of the various gases of interest can be obtained bycomparing the measured brightness with an estimate of the various embodiments of the system described herein can
expected brightness. The spectral unmixing methodology 40 provide a system with higher number of pixels. For example,
can also benefit from temporal, parallax, and motion com embodiments including a single large format microbolom
pensation techniques.
eter FPA array can provide a system with large number ofIn various embodiments, a method of identifying the pixels. Various embodiments of the systems describedpresence of a target species in the object includes obtaining herein can also offer a high optical throughput for a Sub
the radiance spectrum (or the absorption spectrum) from the 45 stantially low number of optical channels. For example, theobject in a spectral region indicative of the presence of the systems described herein can provide a high optical throughtarget species and calculating a correlation (e.g., a correla put for a number of optical channels between 4 and 50. Bytion coefficient) by correlating the obtained radiance spec having a lower number of optical channels (e.g., between 4trum (or the absorption spectrum) with a reference spectrum and 50 optical channels), the systems described herein havefor the target species. The presence or absence of the target 50 wider spectral bins which allows the signals acquired within
species can be determined based on an amount of correlation each spectral bin to have a greater integrated intensity.
(e.g., a value of correlation coefficient). For example, theAn advantage of the embodiments described herein overpresence of the target species in the object can be confirmed various scanning based hyperspectral systems that are con
if the amount of correlation or the value of correlation
figured for target species detection (for example, gas cloud
coefficient is greater than a threshold. In various implemen 55 detection) is that, the entire spectrum can be resolved in atations, the radiance spectrum (or the absorption spectrum) Snapshot mode (for example, during one image frame acqui
can be obtained by obtaining a spectral difference image sition by the FPA array). This feature enables the embodi
between a filtered optical channel and/or another filtered ments of the imaging systems described herein to take
optical channel/unfiltered optical channel or any combina advantage of the compensation algorithms such as thetions thereof.
60 parallax and motion compensation algorithms mentioned
For example, an embodiment of the system configured to above. Indeed, as the imaging data required to implement
detect the presence of methane in a gas cloud comprises these algorithms are collected simultaneously with the tar
optical components such that one or more of the plurality of get-species related data, the compensation algorithms are
optical channels is configured to collect IR radiation to carried out with respect to target-species related data and notprovide spectral data corresponding to a discrete spectral 65 with respect to data acquired at another time interval. Thisband located in the wavelength range between about 7.9 um rapid data collection thus improves the accuracy of the data
and about 8.4 um corresponding to an absorption peak of compensation process. In addition, the frame rate of dataUS 9,562,849 B2
49acquisition is much higher. For example, embodiments of
the imaging system described herein can operate at video
rates from about 5 Hz and higher. For example, various
embodiments described herein can operate at frame rates

50
the detector elements is reduced to about 0.5° C. (thereby
improving the original reading at least by a factor of 5).
FIG. 8B represents the results of similar measurements
corresponding to a different location in the scene (a locationfrom about 5 Hz to about 60 Hz or 200 Hz. Thus, the user 5 which is at a temperature about 9° C. above the estimatedis able to recognize in the images the wisps and Swirls temperature of the aperture 338 of FIG. 3A). As shown, thetypical of gas mixing without blurring out of these dynamic correction algorithm discussed above is operable and effec
image features and other artifacts caused by the change of tive and applicable to objects kept at different temperature.
scene (whether spatial or spectral) during the lengthy mea Accordingly, the algorithm is Substantially temperature
Surements. In contradistinction, Scanning based imaging 10 independent.
systems involve image data acquisition over a period of time Dynamic Calibration Elements and References
exceeding a single-snap-shot time and can, therefore, blur
FIGS. 9A and 9B illustrates schematically different imple
the target gas features in the image and inevitably reduce the mentations 900 and 905 respectively of the imaging systemotherwise achievable sensitivity of the detection. This result that include a variety of temperature calibration elements tois in contrast to embodiments of the imaging system 15 facilitate dynamic calibration of the FPAS. The temperaturedescribed herein that are capable of detecting the localized calibration elements can include mirrors 975a, 975b (repconcentrations of gas without it being Smeared out with the resented as M, Mo, etc.) as well as reference sources
areas of thinner gas concentrations. In addition, the higher 972a and 972b. The implementation 900 can be similarly
frame rate also enables a much faster response rate to a leak configured as the embodiment 300 and include one or moreof gas (when detecting Such leak is the goal). For example, front objective lens, a divided aperture, one or more spectral
an alarm can trigger within fractions of a second rather than filters, an array of imaging lenses 928a and an imaging
several seconds.
element 936. In various implementations, the imaging ele
To demonstrate the operation and gas detection capability ment 936 (e.g., camera block) can include an array ofof the imaging systems described herein, a prototype was cameras. In various implementations, the array of cameras
constructed in accordance with the embodiment 300 of FIG. 25 can comprise an optical FPA unit. The optical FPA unit can
3A and used to detect a hydrocarbon gas cloud of propylene comprise a single FPA, an array of FPAs. In various imple
at a distance of approximately 10 feet. FIG. 7 illustrates mentations, the array of cameras can include one or moreVideo frames 1 through 12 representing gas-cloud-detection detector arrays represented as detector array 1, detector array
output 710 (seen as a streak of light) in a sequence from t=1
5, detector array 9 in FIGS. 9A and 9B. In various embodi
to t—12. The images 1 through 12 are selected frames taken 30 ments, the FOV of each of the detector arrays 1, 5, 9 can befrom a video-data sequence captured at a video-rate of 15 divided into a central region and a peripheral region. With
frames/sec. The detected propylene gas is shown as a streak out any loss of generality, the central region of the FOV ofof light 710 (highlighted in red) near the center of each each of the detector arrays 1, 5, 9 can include the region
image. The first image is taken just prior to the gas emerging where the FOV of all the detector arrays 1, 5, 9 overlap. In
from the nozzle of a gas-contained, while the last image 35 the embodiment illustrated in FIG. 9A, some temperature
represents the system output shortly after the nozzle has calibration elements, namely, reference sources 972a andbeen turned off.
972b are placed at a distance from the detector arrays 1, 5,
The same prototype of the system can also demonstrate 9, for example, and mirrors to image them onto the detector
the dynamic calibration improvement described above by arrays are then placed at the location of the scene reference
imaging the scene Surrounding the system (the laboratory) 40 aperture (e.g., 338 of FIG. 3A).
In FIG. 9A, the temperature-calibration elements 975awith known temperature differences. The result of imple
menting the dynamic correction procedure is shown in and 975b comprise mirrors that reflect radiation from the
FIGS. 8A, 8B, where the curves labeled “obj” (or “A”) reference sources 972a and 972b (represented as ref A andrepresent temperature estimates of an identified region in the ref B). The mirrors 975a and 975b can be disposed away
scene. The abscissa in each of the plots of FIGS. 8A, 8B 45 from the central FOV of the detector arrays 1, 5, 9 such thatindicates the number of a FPA, while the ordinate corre
the central FOV is not blocked or obscured by the image ofsponds to temperature (in degrees C.). Accordingly, it is the reference source 972a and 972b. In various implemen
expected that when all detector elements receive radiant data tations, the FOV of the detector array 5 could be greater thanthat, when interpreted as the objects temperature, indicates the FOV of the detector arrays 1 and 9. In such implementhat the object's temperature perceived by all detector ele 50 tations, the mirrors 975a and 975b can be disposed away
ments is the same, any given curve would be a substantially from the central FOV of the detector array 5 at a location
flat line. Data corresponding to each of the multiple “obj” such that the reference source 972a and 972b is imaged bycurves are taken from a stream of video frames separated the detector array 5. The mirrors 975a and 975b mayfrom one another by about 0.5 seconds (for a total of 50 comprise imaging optical elements having optical power
frames). The recorded “obj' curves shown in FIG. 8A 55 that image the reference sources 972a and 972b onto theindicate that the detector elements disagree about the detector arrays 1 and 9. In this example, the reference
objects temperature, and that difference in objects tem sources 972a and 972b can be disposed in the same plane asperature perceived by different detector elements is as high the re-imaging lenses 928a, however, the reference sources
as about 2.5°C. In addition, all of the temperature estimates 972a and 972b can be disposed in a different plane or inare steadily drifting in time, from frame to frame. The curves 60 different locations. For example, the reference sources 972alabeled “ref (or “C”) correspond to the detectors estimates and 972b can be disposed in a plane that is conjugate to theof the temperature of the aperture 338 of the embodiment plane in which the detector array 1, detector array 5, and300 of FIG. 3A. The results of detection of radiation carrieddetector array 9 are disposed such that a focused image ofout after each detector pixel has been subjected to the the reference sources 972a and 972b is formed by the
dynamic calibration procedure described above are 65 detector arrays. In some implementations, the reference
expressed with the curved labeled “objcorr” (or “B”). Now,sources 972a and 972b can be disposed in a plane that isthe difference in estimated temperature of the object among spaced apart from the conjugate plane Such that a defocusedUS 9,562,849 B2
51image of the reference sources 972a and 972b is formed by
the detector arrays. In various implementations, the reference sources 972a and 972b need not be disposed in thesame plane.

52
same reference source(s) or Surface(s). For example, detec
tor arrays 1 and 9 can be calibrated to agree with each other.
As another example, detector arrays 1, 5 and 9 can becalibrated to agree with each other. In various embodiments,
the lenses 928a provide blurred (or defocused) images of the
As discussed above, in some embodiments, the reference
sources 972a and 972b are imaged onto the detector array 1
reference sources 972a, 972b on the detector arrays 1 and 9
and detector array 9, without much blur such that the because the location of the reference Sources are not exactly
in a conjugate planes of the detector arrays 1 and 9. Althoughreference sources 972a and 972b are focused. In contrast, inother embodiments, the image of the formed of reference the lenses 928a are described as providing blurred or defo
sources 972a and 972b on the detector array 1, and detector 10 cused images, in various embodiments, reference sources orarray 9 are blurred such that the reference sources 972a and Surfaces are imaged on the detectors arrays 1, 5, 9 without
972b are defocused, and thereby provide some averaging,
Such blur and defocus and instead are focused images.
Smoothing, and/or low pass filtering. The reference sources Additionally optical elements may be used. Such as for
972a and 972b may comprise a surface of known tempera example, the mirrors shown in FIG. 9A to provide such
ture and may or may not include a heater or cooler attached 15 focused images.
thereto or in thermal communication therewith. For
The temperature of the reference sources 972b, 972a can
example, the reference source 972a and 972b may comprises be different. For example, the reference source 972a can beheaters and coolers respectively or may comprise a surface at a temperature T, and the reference source 972b can be atwith a temperature sensor and a heater and sensor respec a temperature T. lower than the temperature T. Aheater cantively in direct thermal communication therewith to control be provided under the temperature-controlled element 972a
the temperature of the reference surface. In various imple to maintain it at a temperature T, and a cooler can beprovided underneath the temperature-controlled element
mentations, the reference sources 972a and 972b can include
a temperature controller configured to maintain the reference 972b to maintain it at a temperature T. In various imple
sources 972a and 972b at a known temperature. In some mentations, the embodiments illustrated in FIGS. 9A and 9B
implementations, the reference sources 972a and 972b can 25 can be configured to image a single reference source 972
be associated with one or more sensors that measure theinstead of two references sources 972a and 972b maintained

temperature of the reference sources 972a and 972b and at different temperatures. It is understood that the single
communicate the measured temperature to the temperature reference source need not be thermally controlled. For
controller. In some implementations, the one or more sen example, in various implementations, a plurality of detectors
sors can communicate the measured temperature to the 30 in the detector array can be configured to image a samedata-processing unit. In various implementations, the refer surface of at least one calibration element whose thermal
ence sources 972a and 972b may comprise a surface of and spectral properties are unknown. In such implementa
unknown temperature. For example, the reference sources tions, one of the plurality of detectors can be configured as
may comprise a wall of a housing comprising the imaging a reference detector and the temperature of the surface of thesystem. In some implementations, the reference sources 35 at least one calibration element imaged by the plurality of
972a and 972b comprising a surface need not be associated detectors can be estimated using the radiance spectrumwith sensors, temperature controllers. However, in other obtained by the reference detector. The remaining plurality
implementations, the reference sources 972a and 972b com of detectors can be calibrated such that their temperature
prising a Surface can be associated with sensors, temperature and/or spectral measurements agree with the reference
controllers.
40 detector. For example, detector arrays 1 and 9 can be
In FIG. 9B, the temperature-calibration elements com calibrated to agree with each other. As another example,
prise temperature-controlled elements 972a and 972b (e.g., detector arrays 1, 5 and 9 can be calibrated to agree witha thermally controlled emitter, a heating strip, a heater or a each other.
The reference sources 972a and 972b can be coated withcooler) disposed a distance from the detector arrays 1, 5, 9.
In various embodiments, the temperature-controlled ele 45 a material to make it behave substantially as a blackbody
ments 972a and 972b can be disposed away from the central (for which the emission spectrum is known for any given
FOV of the detector arrays 1, 5, 9 such that the central FOV temperature). If a temperature sensor is used at the location
is not blocked or obscured by the image of the reference of each reference source, then the temperature can be
Source 972a and 972b. The radiation emitted from thetracked at these locations. As a result, the regions in thereference sources 972a and 972b is also imaged by the 50 image of each camera (e.g., on the detector arrays 1 and 9)
detector array 936 along with the radiation incident from the in which the object has such known temperature (and,object. Depending on the position of the reference sources therefore, spectrum) can be defined. A calibration procedure
972a and 972b the image obtained by the detector array of can thus be used so that most of the cameras (if not every
the reference sources can be blurred (or defocused) or sharp camera) so operated agrees, operationally, with most or(or focused). The images 980a,980b, 980c, 980d, 980e and 55 every other camera, for objects at the temperatures repre
980fof the temperature-controlled elements 972a and 972b sented by those two sources. Calibrating infrared cameras
can be used as a reference to dynamically calibrate the one using sources at two different temperatures is known as a
“two-point calibration, and assumes that the measured
or more cameras in the array of cameras.
In the implementations depicted in FIGS. 9A and 9b, the signal at a given pixel is linearly related to the incident
detector arrays 1, 5 and 9 are configured to view (or image) 60 irradiance. Since this calibration can be performed during
both the reference sources 972a and 972b. Accordingly, multiple, more, or even every frame of a sequence, it ismultiple frames (e.g., every or Substantially every frame) referred to as a “dynamic calibration'.
An example of the dynamic calibration procedure is aswithin a sequence of images contains one or more regions in
the image in which the object image has known thermal and follows. If there is a temperature sensor on the reference
spectral properties. This allows multiple (e.g., most or each) 65 Sources or reference Surface, then the temperature measurecameras within the array of cameras to be calibrated to agree ments obtained by these temperature sensors can be used towith other (e.g., most or every other) camera imaging the determine their expected emission spectra. These tempera

US 9,562,849 B2
53
547, 11, 10, 9, 8 and 4. In such implementations, the one orture measurements are labeled as TR, TIR, and TR
for the “reference temperatures” of sources/surfaces A, B, more mirrors Mo, M and Mo M can be placed in a
and C. These temperature measurements can be used as peripheral FOV of the cameras 5 and 6 which does overlap
Scalar correction factors to apply to the entire image of a with the central FOV of the outer ring of cameras 0, 1, 2, 3,
given camera, forcing it to agree with the reference tem 5 7, 11, 10, 9, 8 and 4 such that the reference sources A andperatures. Correcting the temperature estimate of a given B are imaged by the cameras 5 and 6.
pixel from T to T' can use formulae analogous to thoseThis design is an enhancement to the systems 300 and 400
discussed below in reference to FIGS. 10A, 10B, 10C. If noshown in FIGS. 3A and 4A. In the system 1000 shown in
direct temperature sensor is used, then one of the cameras FIG. 10A, an array of two or more imaging elements (curved
can be used instead. This camera can be referred to as the 10 mirrors, for example) is installed at a distance from theFPAS, for example, in the plane of the reference aperture 160“reference camera'. In this case, the same formulae as those
provided below can be used, but with TR and TR
shown in FIG. 3A. These elements (mirror or imaging
representing the temperatures of the reference sources/sur elements) are used to image one or more temperature
faces A and B as estimated by the reference camera. By controlled reference sources A and B onto the detector
applying the dynamic calibration correction formulae, all of 15 elements of two or more of the cameras. The primary
the other cameras are forced to match the temperature difference between embodiment 1000 and embodiment 300
estimates of the reference camera.
or 400 is that now a plurality or most or all of the outer ring
In the configuration illustrated in FIG. 9B, the reference of cameras in the array can image both the reference sources
sources 972a and 972b are placed such that the images of the A and B instead of imaging one of the two reference source
Sources on the detector arrays are blurred. The configuration A and B. Accordingly, most or all of the outer ring of
illustrated in FIG. 9A is similar to the system 400 illustrated cameras image an identical reference source or an identicalin FIG. 4 where the reference sources are placed at an set of reference sources (e.g., both the reference Sources Aintermediate image plane (e.g., a conjugate image plane). In and B) rather than using different reference sources forthis configuration, the array of reference apertures, similar to different cameras or imaging different portions of the refer
reference apertures 438a in FIG. 4, will have an accompa 25 ence sources as shown in FIGS. 3A and 4A. Thus, thisnying array of reference Sources or reference Surfaces Such approach improves the robustness of the calibration, as itthat the reference sources or Surfaces (e.g., each reference eliminates potential failures and errors due to the having
Source or Surface) are imaged onto a camera or a detector additional thermal sensors estimating each reference source.
The imaging elements in the system 1000 (shown asarray such as FPAS 1, 5,9. With this approach, the reference
Source or Surface images are at a conjugate image plane and 30 mirrors in FIGS. 10A and 10B) image one or more conthus are not appreciably blurred, so that their images can be trolled-temperature reference sources or a surface of amade to block a smaller portion of each camera's field of calibration element (shown as A and B in FIGS. 10A andview.
10B) into the blocked region of the cameras fields of view.
A “static' calibration (a procedure in which the scene is FIG. 10B shows an example in which mirror Mo images
largely blocked with a reference source such as the moving 35 reference source/surface A onto camera 0, and mirror Mo
shutters 960 in FIGS. 9A and 9B, so that imaging of a images reference source/surface B onto camera 0, andunknown scene cannot be performed in parallel with cali likewise for cameras 1, 2, and 3. This way, each of thebration) allows a plurality of the cameras (for example, most mirrors is used to image a reference source/surface onto a
or each camera) to accurately estimate the temperature of a detector array of a camera, so that many, most, or every
plurality of elements (for example, most or each element in 40 frame within a sequence of images contains one or more
the scene) immediately after the calibration is complete. It regions in the image in which the object image has known
cannot, however, prevent the cameras estimates from drift thermal and spectral properties. This approach allows mosting away from one another during the process of imaging an of the camera, if not each camera, within the array of
unknown scene. The dynamic calibration can be used to cameras to be calibrated to agree with most or every otherreduce or prevent this drift, so that all cameras imaging a 45 camera imaging the same reference source or sources. For
scene can be forced to agree on the temperature estimate of example, cameras 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4 can be
the reference sources/surfaces, and adjust this correction calibrated to agree with each other. As another example,
during every frame.
cameras 0, 1, 2 and 3 can be calibrated to agree with each
FIG. 10A illustrates schematically a related embodiment other. As yet another example, cameras 0, 1, 2, 3, 7, 11, 10.
1000 of the imaging system, in which one or more mirrors 50 9, 8, 4, 5 and 6 can be calibrated to agree with each other.
Mo, M and Mo, . . . M. are placed within the fields Accordingly, in various implementations, two, three, four,of view of one or more cameras 0,..., 11, partially blocking five, six, seven, eight, nine, ten, eleven or twelve cameras
the field of view. The cameras 0, ..., 11 are arranged to form can be calibrated to agree with each other. In certain
an outer ring of cameras including cameras 0, 1, 2, 3, 7, 11, embodiments, however, not all the cameras are calibrated to10, 9, 8 and 4 surrounding the central cameras 5 and 6. In 55 agree with each other. For example, one, two, or morevarious implementations, the FOV of the central cameras 5 cameras may not be calibrated to agree with each other while
and 6 can be less than or equal to the FOV of the outer ring others may be calibrated to agree with each other. In various
embodiments, these mirrors may be configured to image theof cameras 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In such
implementations, the one or more mirrors Mo, M and reference sources/surfaces A and B onto different respective
Mo, . . . M can be placed outside the central FOV of the 60 pixels a given FPA. Without any loss of generality, FIGS.
cameras 5 and 6 and is placed in a peripheral FOV of the 10A and 10B represent a top view of the embodiment shown
cameras outer ring of cameras 0, 1, 2, 3, 7, 11, 10, 9, 8 and in FIG. 9A.
FIG. 10C illustrates schematically a related embodiment
4 which does not overlap with the central FOV of thecameras 5 and 6 such that the reference sources A and B are
1050 of the imaging system, in which one or more reference
not imaged by the cameras 5 and 6. In various implemen 65 Sources Ro4. . . . . R114 and Rog. . . . . R11 are disposed
tations, the FOV of the central cameras 5 and 6 can bearound the array of detectors 0, . . . , 11. In various
greater than the FOV of the outer ring of cameras 0, 1, 2, 3, implementations, the one or more reference SourcesUS 9,562,849 B2
56
where is TR is a dynamic offset correction factor, and,55
Ro4. . . . . R114 and Roz. . . . . R11 can be a single reference
Source that is images by the detectors 0, . . . , 11. In various
implementations, central detector arrays 5 and 6 can have a
FOV that is equal to or lesser than the FOV of the outer ring

TeR-TAR

Glel = i,of the detectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In suchre.

implementations, the reference sources Rol, . . . . R11 canbe disposed away from the central FOV of the detector is a dynamic gain correction factor. The term c discussed
arrays 5 and 6 such that the radiation from the reference above is a camera index that identifies the camera whose
Sources R. . . . . R. is imaged only by the outer ring of 10 data is being corrected.
detectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In various
References throughout this specification to “one embodi
implementations, central detector arrays 5 and 6 can have a ment,” “an embodiment,” “a related embodiment,” or simi
FOV that is greater than the FOV of the outer ring of the lar language mean that a particular feature, structure, or
detectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. In such imple characteristic described in connection with the referred tomentations, the reference sources R. . . . . R can be 15 "embodiment' is included in at least one embodiment of thedisposed in the peripheral FOV of the detector arrays 5 and present invention. Thus, appearances of the phrases "in one
6 such that the radiation from the reference sources
embodiment,” “in an embodiment,” and similar language
Rol, ..., R is imaged only by the outer ring of detectors throughout this specification may, but do not necessarily, all0, 1, 2, 3, 7, 11, 10, 9, 8 and 4. The radiation from therefer to the same embodiment. It is to be understood that noreference sources R. . . . . R. is therefore imaged by theouter ring of detectors 0, 1, 2, 3, 7, 11, 10, 9, 8 and 4 as wellas central cameras 5 and 6. Without any loss of generality,
FIG. 10C represents a top view of the embodiment shown in

portion of disclosure, taken on its own and in possible
connection with a figure, is intended to provide a complete
description of all features of the invention.
In the drawings like numbers are used to represent the
FIG. 9B.
same or similar elements wherever possible. The depictedstructural elements are generally not to Scale, and certain
In various implementations, one can use a heater under 25 components
enlarged relative to the other components
neath, adjacent to, or in thermal communication with refer for purposesareofand understanding. It is to beence Source/surface A to give it a higher temperature T, and understood that noemphasis
single drawing is intended to support aa cooler underneath, adjacent to, or in thermal communica complete description of all features of the invention. In othertion with reference source B to give it a lower temperature 30 words, a given drawing is generally descriptive of only
T. In various implementations, the embodiments illustrated Some, and generally not all, features of the invention. A
in FIGS. 10A, 10B and 10C can be configured to image a given drawing and an associated portion of the disclosure
single reference source A instead of two references sources containing a description referencing Such drawing do not,A and B maintained at different temperatures. As discussed generally, contain all elements of a particular view or all
above, the embodiments illustrated in FIGS. 10A, 10B and 35 features that can be presented is this view, for purposes of
10C can be configured to image a same Surface of a simplifying the given drawing and discussion, and to direct
calibration element. In Such implementations, the tempera the discussion to particular elements that are featured in thisture of the surface of the calibration element need not be
drawing. A skilled artisan will recognize that the invention
known. Many, most or each reference source/surface can be may possibly be practiced without one or more of thecoated with a material to make it behave approximately as 40 specific features, elements, components, structures, details,
a blackbody, for which the emission spectrum is known for or characteristics, or with the use of other methods, com
any given temperature. If many, most, or each camera in the ponents, materials, and so forth. Therefore, although aarray of cameras images both of references A and B, so that particular detail of an embodiment of the invention may notthere are known regions in the image of these cameras in 45 be necessarily shown in each and every drawing describing
Such embodiment, the presence of this detail in the drawing
which the object has a known temperature (and therefore may
be implied unless the context of the description requires
spectrum). Then one can perform a calibration procedure. otherwise.
other instances, well known structures, details,
This procedure can provide that many, most or every camera materials, Inor
operations may be not shown in a given
so operated agrees with various, most, or every other cam drawing or described
detail to avoid obscuring aspects ofera, for objects at the temperatures represented by those two 50 an embodiment of thein invention
that are being discussed.
Sources. For example, two, three, four, five, six, seven, eight,
Furthermore, the described single features, structures, ornine, ten, eleven or twelve cameras can be calibrated tocharacteristics of the invention may be combined in any
agree with each other. In certain embodiments, however, not suitable manner in one or more further embodiments.
all the cameras are calibrated to agree with each other. For
The features recited in claims appended to this disclosure
example, one, two, or more cameras may not be calibrated 55 are intended to be assessed in light of the disclosure as ato agree with each other while others may be calibrated to whole, including features disclosed in prior art to which
agree with each other. As discussed above, calibration of reference is made.
At least some elements of a device of the invention can be
infrared cameras using sources at two different temperatures
is known as a “two-point calibration, and assumes that the controlled—and at least some steps of a method of themeasured signal at a given pixel is linearly related to the 60 invention can be effectuated, in operation—with a program
mable processor governed by instructions stored in a
incident irradiance.
The memory may be random access memory
The dynamic calibration is used to obtain a corrected memory.
(RAM),
read-only
memory (ROM), flash memory or any
temperature T from the initial temperature T estimated at other memory, or combination
thereof. Suitable for storingeach pixel in a camera using the following formulae:
65

control software or other instructions and data. Those skilledin the art should also readily appreciate that instructions orprograms defining the functions of the present invention

US 9,562,849 B2
57
58
may be delivered to a processor in many forms, including, medium and computer-readable medium, which may be
but not limited to, information permanently stored on non incorporated into a computer program product.
Various modifications to the implementations described in
Writable storage media (e.g. read-only memory devices
within a computer, such as ROM, or devices readable by a this disclosure may be readily apparent to those skilled in thecomputer I/O attachment, such as CD-ROM or DVD disks), art, and the generic principles defined herein may be applied
information alterably stored on Writable storage media (e.g. to other implementations without departing from the spirit or
floppy disks, removable flash memory and hard drives) or Scope of this disclosure. Thus, the claims are not intended to
information conveyed to a computer through communica be limited to the implementations shown herein, but are totion media, including wired or wireless computer networks. 10 be accorded the widest scope consistent with this disclosure,principles and the novel features disclosed herein.
In addition, while the invention may be embodied in soft theCertainthat are described in this specification inware, the functions necessary to implement the invention the contextfeaturesof separate implementations also can be imple
may optionally or alternatively be embodied in part or in mented in combination
in a single implementation. Con
whole using firmware and/or hardware components, such as versely, various features that are described in the context of
combinatorial logic, Application Specific Integrated Circuits 15 a single implementation also can be implemented in multiple(ASICs), Field-Programmable Gate Arrays (FPGAs) or implementations separately or in any suitable Subcombina
other hardware or some combination of hardware, softwaretion. Moreover, although features may be described above as
and/or firmware components.
acting in certain combinations and even initially claimed as
While examples of embodiments of the system and Such, one or more features from a claimed combination canmethod of the invention have been discussed in reference toin Some cases be excised from the combination, and thethe gas-cloud detection, monitoring, and quantification (including but not limited to greenhouse gases Such as Carbon
Dioxide, Carbon Monoxide, Nitrogen Oxide as well ashydrocarbon gases such as Methane, Ethane, Propane, n-Bu

claimed combination may be directed to a Subcombinationtane, iso-Butane, n-Pentane, iso-Pentane, neo-Pentane, 25

1. An infrared (IR) imaging system, the imaging systemcomprising:a plurality of cameras;
at least one thermal reference Source having a known

Hydrogen Sulfide, Sulfur Hexafluoride, Ammonia, Benzene,
p- and m-Xylene, Vinyl chloride, Toluene, Propylene oxide,
Propylene, Methanol, Hydrazine, Ethanol, 1,2-dichloroeth
ane, 1,1-dichloroethane, Dichlorobenzene, Chlorobenzene,

to name just a few), embodiments of the invention can be 30
readily adapted for other chemical detection applications.
For example, detection of liquid and solid chemical spills,
biological weapons, tracking targets based on their chemicalcomposition, identification of satellites and space debris,
ophthalmological imaging, microscopy and cellular imag 35
ing, endoscopy, mold detection, fire and flame detection, andpesticide detection are within the scope of the invention.
As used herein, a phrase referring to “at least one of a listof items refers to any combination of those items, including
single members. As an example, “at least one of: a, b, or c' 40

or variation of a Subcombination.
What is claimed is:temperature;a plurality of mirrors configured to reflect radiation fromthe at least one thermal reference source towards theplurality of cameras; anda data-processing unit comprising a processor, the imag
ing system configured to:acquire with the plurality of cameras one or more
image frames of an object, a plurality of the one ormore image frames having regions with the known
temperature that corresponds to the at least one
reference Source; andapply a dynamic calibration correction to the plurality
is intended to cover: a, b, c, a-b, a-c, b-c, and a-b-c.
of cameras based on the plurality of the one or more
image frames having regions with the known tem
If implemented in software, the functions may be stored
on or transmitted over as one or more instructions or code on
perature to allow every camera in the plurality of
a computer-readable medium. The steps of a method orcameras to be calibrated to agree with every otheralgorithm disclosed herein may be implemented in a pro 45
camera in the plurality of cameras.
2. The system of claim 1, wherein the plurality of cameras
cessor-executable Software module which may reside on acomputer-readable medium. Computer-readable media comprises an FPA unit and a plurality of lenses.
3. The system of claim 2, wherein the FPA unit comprises
includes both computer storage media and communication
media including any medium that can be enabled to transfer one FPA or a plurality of FPAs.
a computer program from one place to another. A storage 50 4. The system of claim 1, wherein the at least one thermal
media may be any available media that may be accessed by reference Source has a known spectrum.
a computer. By way of example, and not limitation, Such
5. The system of claim 4, further comprising an additional
computer-readable media may include RAM, ROM, thermal reference Source disposed such that radiation from
EEPROM, CD-ROM or other optical disk storage, magnetic the additional thermal reference source is reflected by thedisk storage or other magnetic storage devices, or any other 55 plurality of mirrors towards the plurality of cameras.
medium that may be used to store desired program code in
6. The system of claim 5, wherein the additional reference
the form of instructions or data structures and that may be Source has a temperature and a spectrum different from theaccessed by a computer. Also, any connection can be prop known temperature and the known spectrum of the at least
erly termed a computer-readable medium. Disk and disc, as one reference source.
used herein, includes compact disc (CD), laser disc, optical 60 7. The system of claim 6, wherein the temperature of thedisc, digital versatile disc (DVD), floppy disk, and blu-ray additional thermal reference source is less than the known
disc where disks usually reproduce data magnetically, while temperature.
8. The system of claim 6, wherein the temperature of thediscs reproduce data optically with lasers. Combinations of
the above also may be included within the scope of com additional thermal reference source is greater than theputer-readable media. Additionally, the operations of a 65 known temperature.
method or algorithm may reside as one or any combination
9. The system of claim 1, wherein the at least one
or set of codes and instructions on a machine readablereference source is displaced away from a conjugate image

US 9,562,849 B2
59plane of the plurality of cameras such that an image of theat least one reference source captured by the plurality of60
a data-processing unit comprising a processor, said dataprocessing unit configured to:cameras is blurred.

acquire with the plurality of cameras one or more

10. The system of claim 1, wherein the at least one
reference source is positioned at a conjugate image plane of 5
the plurality of cameras.
11. The system of claim 1, wherein the plurality of mirrors
is disposed outside a central field of view of the plurality of
CaCaS.
12. The system of claim 1, further comprising a first and 10

image frames of an object, a plurality of the one ormore image frames having regions that correspond toradiation from the first and the second temperature
controlled reference sources; anddynamically calibrate the plurality of cameras so thata second temperature-controlled shutter removably posi-tioned to block IR radiation incident on the system fromreaching the plurality of cameras.

Various cameras imaging a scene agree on a tem
perature estimate of the first and second reference
SOUCS.
30. The imaging system of claim 29, wherein the dataprocessing unit is configured to calculate a dynamic cali

13. The system of claim 1, wherein the system includes at is bration correction and apply the correction to the plurality ofleast two spatially and spectrally different optical channels. cameras for each of the plurality of frames.
14. The system of claim 13, wherein the system includes
31. The system of claim 29, wherein the first reference
at least three optical channels.
Source is maintained at a first temperature.
15. The system of claim 13, wherein the system includes
32. The system of claim 31, wherein the second reference
at least four optical channels.
20 source is maintained at a second temperature.
16. The system of claim 13, wherein the system includes
33. The system of claim 32, wherein the first temperature
at least five optical channels.
is greater than the second temperature.
17. The system of claim 13, wherein the system includes
34. The system of claim 32, wherein the first temperature
at least six optical channels.
is less than the second temperature.

18. The system of claim 13, wherein the System includes 25 35. The system of claim 29, wherein the first and theat least seven spatially and spectrally different optical channels.
19. The system of claim 13, wherein the system includes
at least eight spatially and spectrally different optical chansecond reference sources are displaced away from a conju
gate image plane of the plurality of cameras Such that theimage of the first and the second reference sources capturedbythe plurality of cameras is blurred.
nels.
30
36.
system of claim 29, wherein the first and the
20. The system of claim 13, wherein the system includes secondThereferencesources are positioned at a conjugate image
at least nine spatially and spectrally different optical chan plane of the pluralityof cameras.
nels.
37.
The
systemof
claim
29, further comprising:
21. The system of claim 13, wherein the system includes
at least ten spatially and spectrally different optical channels. 35 a first mirror configured to image the first reference source
onto the plurality of cameras; and22. The system of claim 13, wherein the system includes
a second mirror configured to image the second reference
at least twelve spatially and spectrally different opticalchannels.
Source onto the plurality of cameras.
23. The system of claim 1, further comprising one or more
38. The system of claim 29, wherein the system includes

sensors configured to measure a temperature of the at least 40 at least two spatially and Spectrally different optical chan

one reference source.

nels.

24. The system of claim 23, wherein the one or more
39. The system of claim 38, wherein the system includes
sensors are configured to communicate the measured tem- at least four spatially and spectrally different optical chanperature of the at least one reference source to a temperature nels.
controller.
45 40. The system of claim 39, wherein the system includes
25. The system of claim 23, wherein the one or more at least six spatially and spectrally different optical channels.
sensors are configured to communicate the measured tem41. The system of claim 39, wherein the system includes
perature of the at least one reference source to the data- at least eight spatially and spectrally different optical chanprocessing unit.

nels.

26. The system of claim 1, wherein the plurality of 50 42. The system of claim 39, wherein the system includes
cameras is configured to image a same portion of the at least at least ten spatially and spectrally different optical channels.
one reference source.

43. The system of claim 39, wherein the system includes

27. The system of claim 1, wherein the plurality of at least twelve spatially and spectrally different opticalcameras is configured to acquire multispectral image data channels.
from the object continuously for a duration of time.
55 44. The system of claim 29, further comprising one or
28. The system of claim 1, comprising at least two more sensors configured to measure a temperature of thespectrally and spatially distinct optical channels configured

first or the second reference source.

to capture two-dimensional image data of the object.
45. The system of claim 44, wherein the one or more
29. An infrared (IR) imaging system, the imaging system sensors are configured to communicate the measured temcomprising:
60 perature of the first or the second reference source to aa plurality of cameras;temperature controller.
a first temperature-controlled reference Source imaged by
46. The system of claim 44, wherein the one or more
the plurality of cameras;
sensors are configured to communicate the measured tem
a second temperature-controlled reference source:
perature of the first or the second reference source to thea first and a second temperature-controlled shutter remov- 65 data-processing unit.
ably positioned to block IR radiation incident on the
47. The system of claim 29, wherein the plurality ofsystem from reaching the plurality of cameras; andcameras is configured to image a same portion of the firstUS 9,562,849 B2
61
reference source and wherein the plurality of cameras isconfigured to image a same portion of the second reference
SOUCe.

62
tioned to block IR radiation incident on the system fromreaching the plurality of cameras.
54. The system of claim 50, wherein the system includes
at least two spatially and spectrally different optical chan

48. The system of claim 29, wherein the plurality ofcameras are configured to acquire multispectral image data 5 nels.
55. The system of claim 54, wherein the system includes
from the object continuously for a duration of time.
at least three spatially and spectrally different optical chan
49. The system of claim 29, comprising at least two nels.
spectrally and spatially distinct optical channels configured
56. The system of claim 54, wherein the system includes
to capture two-dimensional image data of the object.
50. An infrared (IR) imaging system, the imaging system 10 at least four spatially and spectrally different optical channels.
comprising:
57. The system of claim 54, wherein the system includes
a plurality of cameras;
at least five spatially and spectrally different optical chana reference having an unknown temperature;a plurality of mirrors configured to reflect radiation from nels.
The system of claim 54, wherein the system includes
the reference towards the plurality of cameras; and 15 at 58.
least six spatially and spectrally different optical channels.

a data-processing unit comprising a processor, the imag
ing system configured to:acquire with the plurality of cameras one or more
image frames of an object, a plurality of the one ormore image frames having regions with the unknown
temperature that corresponds to the reference;calculate a dynamic calibration correction using a tem

59. The system of claim 54, wherein the system includes
at least seven spatially and spectrally different optical channels.

60. The system of claim 54, wherein the system includes
at least eight spatially and spectrally different optical channels.

61. The system of claim 54, wherein the system includes
at least nine spatially and spectrally different optical chanplurality of cameras based on the plurality of the one nels.
or more image frames having regions with the 25 62. The system of claim 54, wherein the system includes
unknown temperature as a reference temperature; at least ten spatially and spectrally different optical channels.
andThe system of claim 54, wherein the system includes
apply the calibration correction to the other cameras in at 63.
least
the plurality of cameras to match the temperature channels.twelve spatially and spectrally different opticalestimate of the other cameras in the plurality of 30 64. The system of claim 50, wherein the plurality ofcameras with the reference temperature.
is configured to image a same portion of the
51. The system of claim 50, wherein the reference is cameras
reference.
displaced away from a conjugate image plane of the plural
65. The system of claim 50, wherein the plurality of
ity of cameras such that the image of the reference captured 35 cameras
are configured to acquire multispectral image databy the plurality of cameras is blurred.
fromtheobject continuously for a duration of time.
52. The system of claim 50, wherein the reference is66.
Thesystem of claim 50, comprising at least two
positioned at a conjugate image plane of the plurality of spectrally andspatially distinct optical channels configured
CaCaS.
tocapturetwo-dimensional
image data of the object.
53. The system of claim 50, further comprising a first and
ck
ck
ck
ck
ck
a second temperature-controlled shutter removably posiperature measured by one of the cameras in the