THAT INA MI CORA UNA MABALAT MAINITTUR
US009756263B2(12) United States Patent(10) Patent No.:(45) Date of Patent:

Kester et al.

(54 ) MOBILE GAS AND CHEMICAL IMAGING(56 )CAMERA(71) Applicant: REBELLION PHOTONICS, INC.,@

Sep . 5 , 2017

References Cited
U . S. PATENT DOCUMENTS
4 , 134 ,683 A

Houston , TX (US)US 9 ,756 , 263 B2

1/ 1979 Goetz et al.

4 ,963,963 A * 10 / 1990 Dorman ...

H04N 5 / 33
250 /332(72 ) Inventors : Robert Timothy Kester, Pearland , TX(US); Nathan Adrian Hagen , Houston ,(Continued )@FOREIGN PATENT DOCUMENTS
2 365 866
9 /2000

TX (US )( 73 )@

Assignee : REBELLION PHOTONICS, INC .,@( * ) Notice:

WO2009094782??

Houston, TX (US )* 8 /2009

GO1J 3 /28(Continued )Subject to any disclaimer, the term of thisU . S .C . 154 (b ) by 0 days .

OTHER PUBLICATIONSWilliams et al. (“ Dual-Band MWIR / LWIR Radiometer for Absolutepatent is extended or adjusted under 35

Temperature Measurements” , Thermosense XXVII Edited by Jona

(21 ) Appl. No.: 14 /700 ,791

than Miles et al., Proc. of SPIE , vol. 6205 , pp . 62050M - 1 to(22) Filed: Apr. 30, 2015

62050M - 13 (2006 ).*(65 )Primary Examiner — Casey Bryant
Assistant Examiner — Meenakshi Sahu

3 Filed :@(Continued )Prior Publication Data
US 2015 /0316473 A1 Nov . 5 , 2015(74 ) Attorney, Agent, or Firm — Knobbe ,Martens, Olson& Bear LLP

Related U . S . Application Data
(60 ) Provisional application No.61/986 , 885, filed on May
1, 2014 , provisional application No .62/012 ,078 , filed(Continued )ABSTRACT(57)
In one embodiment, an infrared (IR ) imaging system fordetermining a concentration of a target species in an object
is disclosed . The imaging system can include an opticalsystem including an optical focal plane array (FPA ) unit . The
optical system can have components defining at least two
optical channels thereof, said at least two optical channels
being spatially and spectrally different from one another.
Each of the at least two optical channels can be positioned
to transfer IR radiation incident on the optical system
towards the optical FPA. The system can include a process(51) Int . Cl.
GO1J 5 /02( 2006 .01 )H04N 5 /33( 2006 .01)

(Continued )(52 ) U .S . CI.

CPC ............. H04N 5 /33 ( 2013 .01) ; GOIJ 3 /0232(2013 .01) ; GOIJ 370256 ( 2013 .01) ;(Continued )(58 ) Field of Classification Search???

ing unit containing a processor that can be configured toacquire multispectral optical data representing said target

species from the IR radiation received at the optical FPA .
Said optical system and said processing unit can be con(Continued )GOIN 21/3504. . . . . . . . . . UVI(Continued )Frent objective long, 124Spectrally divided penil 130
30%VAKJ AV

120
120

Romasing lang aray , 128

V

Detector array 136

120 l
110

doud
Towards

processor 150

120

128a
130

Movable reference tour , 160

US 9, 756 ,263 B2
Page 2

tained together in a data acquisition and processing module
configured to be worn or carried by a person .

20 Claims, 23 Drawing Sheets
Related U .S . Application Data
on Jun . 13 , 2014 , provisional application No. 62 /054 ,
894 , filed on Sep . 24 , 2014 , provisional application
No. 62 /055,342 , filed on Sep . 25 , 2014 , provisional
application No. 62/055,549 , filed on Sep . 25 , 2014 ,provisional application No .62/082,613 , filed on Nov.
20 , 2014 , provisional application No. 61/ 986 ,886 ,filed on May 1 , 2014 , provisional application No .

62/082,594 , filed on Nov. 20 , 2014 , provisional appli
cation No. 62 /021, 636 , filed on Jul. 7 , 2014 , provi
sional application No. 62 /021,907, filed on Jul. 8 ,2014 , provisional application No. 62/083 , 131, filed
on Nov. 21 , 2014 .

(51) Int. Ci.
( 2006 .01)
( 2006 .01)

GO1J 3 / 36

GO1J 3 /02
GO1J 3 /28
GO1J 5 /00(2006 .01)

GO1J 5 / 08
GOIN 21/ 3504
GO2B 5 / 20
G06 9700
HO4N 5 /225
(52 ) U . S . Cl.

( 2006 .01)
( 2006 .01)

( 2014 .01)

( 2006 .01)

( 2006 .01 )( 2006 .01)

CPC ........ . ... GO1J 3 / 2803 (2013 .01); GOIJ 3 / 36
( 2013 .01); GOIJ 5 / 0014 ( 2013 .01); G01J
5 / 0834 (2013 .01); GOIN 21/3504 ( 2013.01) ;GO2B 5 /201 (2013 .01) ; GO2B 5 /208(2013 .01 ) ; G06K 9/ 00624 ( 2013.01) ; H04N
5 / 2258 (2013 .01) ; GOIJ 2003/ 2826 ( 2013 .01);
GOIJ 2005 /0077 ( 2013 .01); GOIN 2021 / 3531

(2013 .01); GOIN 2201/0221 (2013 .01 )(58 ) Field of Classification Search
USPC . . .. . .. .. . .. .. . .. .
............... 250/ 339. 02
See application file for complete search history .
References Cited(56 )U .S . PATENT DOCUMENTS
5 , 127, 742 A

5 , 157,258 A

7 / 1992 Fraden

10 / 1992 Gunning, III et al.

5 , 559 ,336 A
9 / 1996 Kosai et al.
5 ,877,500 A
3/ 1999 Braig et al.
5 ,920, 066 A
7 / 1999 DiRenzo et al .
7 / 1999 Hopkins
5 ,926 , 283 A
2 / 2000 Bodkin
6 ,023 ,061 A
6 , 268, 883 B1 * 7 / 2001 Zehnder
HO4N 5 /33
250 / 338. 1
6,456, 261 B1 . 9 /2002 Zhang
6 , 680, 778 B2
1/ 2004 Hinnrichs et al.
7 ,109 ,488 B2
9 /2006 Milton
7 , 242,478 B1 7 / 2007 Dombrowski et al.
B2
1/ 2008 Holland et al.
7 ,433 ,042 B1 10 / 2008 Cavanaugh et al.
7 ,750, 802 B1
7 / 2010 Parish et al.
7 , 835 ,002 B2 11 / 2010 Muhammed et al.
7 ,888,624 B12 / 2011 Murguia et al.
8 ,027, 041 B1
9 /2011 Mitchell et al.
8, 159, 568 B2
4 /2012 Ahdoot
8, 653 ,461 B1 . 2 / 2014 Benson et al.

8 , 654, 328 B2
9 , 395, 516 B2
9 , 562, 849 B2
9, 599 ,508 B2
9 , 625, 318 B2
9 ,644,562 B2
2002 / 0015151 A1
2002/0121370 A1
2002 /0159101 Al
2003 /0102435 Al
2003/0134426 A1

2 / 2014 Tkaczyk et al.
7 /2016 Katsunuma et al .
2 /2017 Kester et al.
3 /2017 Kester et al.
4 /2017 Kester et al.
5 / 2017 Fujita

2 / 2002 Gorin
9 /2002 Kurkjian et al.

10 /2002 Alderson et al.
6 /2003 Myers et al.
7 /2003 Jiang et al.

2004 /0111232 A1 6 / 2004 Butler et al.
2004/ 0252300 AL 12 / 2004 Slater2005 /0103989 A1* 5 /2005 Watson ....
2006 /0044562 A1 3 /2006 Hagene et al.
2006 /0279632 AL 12/2006 Anderson
2007/0018105 A1
1/ 2007 Grimberg
2007 /0075888 A1 4 / 2007 Kelly et al .
2008 /0170140 A1
2008/ 0204744 A1

HO4N 5 /33
250 /252 . 1

7 /2008 Silver et al .
8 / 2008 Mir et al.

2008/0231719 Al 9 /2008 Benson et al.
2008/0251724 Al 10 / 2008 Baliga et al.
2009 /0252650 A1 10 / 2009 Lakshmanan
2010 /0162206 A16 / 2010 Roth et al.
2010 /0211333 A1
8 /2010 Pruet et al.
2010 /0309467 A1 12 / 2010 Fox et al.
2011/0176577 AL

7 /2011 Bandara et al .

2011/0261321 A

10 /2011 Ramella -Roman et al.

2012 / 0273680 A1

11/2012 Furry

2013 /0181836 A1

7 /2013 Cardoso et al .

2013/0206990 A1

8 /2013 Hsu et al .

2013/0228887 A1 9 / 2013 Wehner et al .
2013/0235256 Al 9/ 2013 Kodama
2013 /0321806 Al 12/2013 Kester et al.
2013 /0341509 Al 12/2013 Nelson et al.
2013 /0342680 A112/2013 Zeng et al .

2014 /0139643 A15 / 2014 Högasten et al.

G01J 3 / 36
2015/0069239 A1* 3/2015 Kester
250 /332
2015/ 0136981 A1 * 5/2015 Kester ............... GOIN 21/3504
250 /330
2015 /0136982 Al * 5 /2015 Kester ................... GO1J 3 /2823
250 / 332
2015/ 0144770 A1 5/ 2015 Choi
2015 /0288894 Al 10 /2015 Geelen et al.
2016 /0037089 AL 2 /2016 Silny et al.
2016 /0097713 A1 4 /2016 Kester et al .
2016 /0245698 A1 8 / 2016 Pau et al.
2016 /0313181 Al 10 /2016 Golub et al.
2016 /0349228 A1 12/2016 Kester et al.
2016 /0380014 Al 12 / 2016 Ganapathi et al.

2017 /0026588 A1

1/2017 Kester et al.

FOREIGN PATENT DOCUMENTS
CA
CA
EP
EP
EP
GB
WO

WO

WO

WO
WO
WO

WO

WO

2 787 303

7 /20112 870 419

5 / 2015
4 / 1998

0 837 600

2 871 452
2 942 615
2 955 496
1014769 A
WO 2004 /097389

11/2015
12 / 2015

5 /2015

WO 2007 /008826

1 / 2007

WO 2008 / 109183
WO 2010 /053979

9 /2008

WO 2012 /078417
WO 2012/ 082366
WO 2013 / 173541
WO 2016 / 196224

6 / 2012

12 / 1965

11/2004
5 /20106 / 2012

11/2013

12 /2016

OTHER PUBLICATIONSAllen et al., “ Measurements of Methane Emissions at Natural Gas
Production Sites in the United States” , PNAS, Oct. 29 , 2013 , vol.
110 , No. 44 , pp. 7 .

US 9,756,263 B2
Page 3( 56 )

References Cited
OTHER PUBLICATIONS

Alvarez et al., “ Greater Focus Needed on Methane Leakage fromNaturalGas Infrastructure” , PNAS, Apr. 24 , 2012 , vol. 109 , No. 17 ,pp. 12 .
Bedard et al., “ Image Mapping Spectrometry : Calibration andCharacterization ” , Optical Engineering, Nov . 2012 , vol. 51 , No. 11 ,pp . 111711 - 1 - 111711 - 13 .
Ben -David et al., “ Probability Theory for 3 - Layer Remote Sensing
Radiative Transfer Model : Univariate Case,” Optics Express, Apr.
2012 , vol. 20 , No . 9 , pp . 10004 - 10033.
Ben - David et al., “ Probability Theory for 3 -Layer Remote Sensing
Radiative Transfer Model: Errata,” Optics Express,May 20 , 2013 ,vol. 21, No. 10 , pp . 11852 .
Brady et al., “ Multiscale Lens Design ” , Optics Express, Jun . 22 ,Gao et al., “Snapshot Image-Mapping Spectrometer for

Hyperspectral Fluorescence Microscopy ” , Optics and Photonics
News, Nov. 2010 , vol. 21, No. 12 , p . 50 .
Gao et al., “ Snapshot Image Mapping Spectrometer ( IMS) withHigh Sampling Density for Hyperspectral Microscopy”, Optics
Express , Jul. 5 , 2010 , vol. 18 , No. 4 , pp . 14330 - 14344 .

Gerhart et al., “ Detection and Tracking of Gas Plumes in LWIR
Hyperspectral Video Sequence Data,” Algorithmsand Technologies
for Multispectral, Hyperspectral, and Ultraspectral Imagery XIX ,2013 , SPIE Proceedings vol. 8743, pp . 1 - 14 .

Gittins, Christopher M ., “ Detection and Characterization of Chemi

cal Vapor Fugitive Emissions by Nonlinear Optimal Estimation :Theory and Simulation ” , Applied Optics, Aug . 10 , 2009 , vol. 48 ,No . 23 , pp . 4545 -4561.
Goldberg et al., “ Dual Band MWIR /LWIR Focal Plane Array Test
Results," Army Research Lab , Adelphi, MD , Aug. 1999 , pp . 18 .

2009, vol. 17 , No. 13 , pp . 10659 - 10674 .

Golowich et al., “ Performance Limits of LWIR Gaseous Plume
Quantification” , Algorithms and Technologies for Multispectral,Hyperspectral, and Ultraspectral Imagery XVII, 2011 , Proceedings

31, 2012 as noted from the email . Subsequent to that date brochureof SPIE vol. 8048, pp . 1 - 12 .
Griffin et al., “ The Herschel — SPIRE 1 - 15 Instrument and its

Brochure provided by Lofty Designs to Rebellion Photonics on Oct.
was used in connection with potential customers.
Caulton et al., " Toward a Better Understanding and Quantificationof Methane Emissions from Shale Gas Development” , PNAS, Apr.
29, 2014 , vol. 111, No. 17 , pp . 7 .
Chen et al., “ Quantitative Sectioning and Noise Analysis for Structured Illumination Microscopy : Erratum ”, Optics Express, Oct. 19 ,2015 , vol. 23 , No . 21 , pp . 27633 - 27634 .
Chidley et al., “ Flow -Induced Birefringence : The Hidden PSF

Killer in High Performance Injection -Molded Plastic Optics” ,Endoscopic Microscopy , Proceedings of SPIE vol. 6082 , 2006 , pp .
11 .

Chu et al., “ The NIST Quantitiative Infrared Database ” , Journal of
Research of the National Institute of Standards and Technology ,Jan .- Feb . 1999 , vol. 104 , No . 1, pp . 59 -81 .

In - Flight Performance,” Astronomy and Astrophysics, Jul. 1 , 2010 ,vol. 518 , pp. 7 .
Gross et al., “ Remote Identification and Quantification of Industrial
Smokestack Effluents via Imaging Fourier- Transform Spectros
copy ” , Environmental Science & Technology , 2010 , vol. 44 , No. 24 ,pp. 9390 -9397.
Gupta et al., “ Miniature Snapshot Multispectral Imager,” OpticalEngineering, 2011 , vol. 50 , pp . 033203 -1 -033203 -9 .
Hadlington , Simon , “ New Camera Makes Methane Visible” , Chem
istry World , http ://web .archive.org/web /20160305234907/http://
www .rsc .org /chemistryworld /2015 / 12 /methane -camera -infraredgreenhouse -gas , Dec . 14 , 2015 , pp . 2 .
Hagen et al., “ Analysis of Computed Tomographic Imaging Spec
trometers. I. Spatial and Spectral Resolution ” , Applied Optics, Oct.

DiPietro et al., “ Hyperspectral Matched Filter with False -Alarm
Mitigation ” , Optical Engineering , Jan . 2012 , vol. 51, No. 1 , pp .
016202- 1 -016202 - 7 .
“ Directed Inspection andMaintenance at Gas Processing Plants and

1 , 2008, vol. 47 , No. 28 , pp . F85 - F95.
Hagen et al., " Coded Aperture DUV Spectrometer for Standoff
Raman Spectoscopy” , Next -Generation Spectroscopic Technolo
gies II, Proceedings of SPIE vol. 7319 , 2009, pp . 1 - 10 .
Hagen et al., " Compound Prism Design Principles, I” , Applied
Optics, Sep . 1, 2011 , vol. 50 , No. 25 , pp . 4998 - 5011 .
Hagen et al., “ Compound Prism Design Principles, II: Triplet andEriksson et al., “ Radiative Cooling Computed for Model Atmo
spheres” , Applied Optics, Dec . 1, 1982 , vol. 21, No. 23, pp .

Janssen Prisms” , Applied Optics , Sep . 1, 2011, vol. 50 , No. 25 , pp .
5012 - 5022 .

4381- 4388 .
Flanigan , “ Detection of Organic Vapors with Active and Passive
Sensors : A Comparison ,” Applied Optics, 1986 , vol. 25 , No . 23, pp .
4253 -4260 .

Wavenumber and Optical Coherence Tomography Prisms” , Applied
Optics , Sep . 1, 2011 , vol. 50 , No. 25 , pp . 5023 - 5030 .

Booster Stations," United States Environmental Protection Agency
Air and Radiation (6202J) , EPA430 - B -03-018 , Oct. 2003 available
at https ://www3. epa .gov/ gasstar/documents / II__ dimgasproc.pdf.

Gålfalk et al., “Making Methane Visable ” , Nature Climate Change ,Apr. 2016 , vol. 6 , pp . 426 - 430 .

Gålfalk et al., “ Making Methane Visable ” , Supplementary Informa
tion , Nature Climate Change , 2015 , pp . 1 - 14 .
Gallagher et al., “ Error Analysis for Estimation of Trace Vapor
Concentration Pathlength in Stack Plumes ” , Applied Spectroscopy,
2003, vol. 57, No. 6 , pp . 614 -621 .
Gallagher et al., “ Estimation of Trace Vapor Concentration
Pathlength in Plumes for Remote Sensing Applications from
Hyperspectral Images” , Analytica Chimica Acta , 2003 , vol. 490 , pp .
139 - 152 .
Gao et al., “ Compact Image Slicing Spectrometer ( ISS ) for

Hyperspectral Fluorescence Microscopy ” , Optics Express, Jul. 20 ,2009, vol. 17 , No. 15 , pp . 12293 - 12308.
Gao et al., “ Depth -Resolved Image Mapping Spectrometer ( IMS)
with Structured Illumination ” , Optics Express, Aug . 29 , 2011, vol.
19, No. 18 , pp. 17439 - 17452.
Gao et al., “ Optical Design of a Snapshot High - Sampling Image
Mapping Spectrometer (IMS) for Hyperspectral Microscopy” ,Three -Dimensional and Multidimensional Microscopy :Image
Acquisition and Processing XVII, Proceedings of SPIE vol. 7570 ,2010 , pp . 1 -7 .
Gao et al., “ Quantitative Comparison Between Full -Spectrum andFilter-Based Imaging in Hyperspectral Fluorescence Microscopy” ,Journal of Microscopy , 2012 , vol. 246 , No. 2, pp. 113 - 123.

Hagen et al., “ Compound Prism Design Principles, III : Linear-in

Hagen et al., “ Fourier Methods of Improving Reconstruction Speed
for CTIS Imaging Spectrometers” , Imaging Spectrometry XII ,Proceedings of SPIE vol. 6661, 2007, pp . 11 .
Hagen et al., “ Foveated Endoscopic Lens” , Journal of Biomedical
Optics, Feb . 2012 , vol. 17 , No. 2 , pp . 021104 -1 -021104 -6 .
Hagen et al., “ Gaussian Profile Estimation in One Dimension ” ,Applied Optics, Aug. 1, 2007 , vol. 46 , No. 22 , pp . 5374 -5383 .
Hagen et al., “ Gaussian Profile Estimation in Two Dimension ” ,Applied Optics, Dec . 20 , 2008 , vol. 47 , No . 36 , pp . 6842 -6851 .
Hagen et al., " Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy ” , Optics Express, Jan . 2 , 2012,vol. 20 , No. 1, pp . 403 -413 .
Hagen et al., “ Quantitative Sectioning and Noise Analysis for

Structured Illumination Microscopy: Errata” , Optics Express, Feb .
27 , 2012 , vol. 20 , No . 5 , pp . 5343.

Hagen et al., “ Real- Time Quantitative Hydrocarbon Gas Imaging

with theGas Cloud Imager (GCI) ” , Proceedings of SPIE , vol. 8358 ,Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE) Sensing XIII, May 1, 2012 , pp . 7.
Optical Engineering , Sep . 2013, vol. 52 , No. 9 , pp . 090901- 1
090901- 23
Hagen et al., “ Snapshot Advantage : A Review of the Light Collec

Hagen et al., “ Review of Snapshot Spectral Imaging Technologies” ,tion Improvement for Parallel High -Dimensional Measurement
Systems," Optical Engineering, Jun . 13, 2012 , vol. 51 , No . 11, p .
111702 - 1 - 111702-7 .

US 9,756,263 B2
Page 4( 56 )

References Cited
OTHER PUBLICATIONS

Hagen et al., “ SnapshotMueller Matrix Spectropolarimeter” , Optics
Letters, Aug. 1, 2007, vol. 32, No . 15, pp . 2100 -2102 .

Hagen et al., “ Spectrally - Resolved Imaging of Dynamic TurbidMedia ” , Multimodal Biomedical Imaging VI, Proceedings of SPIE

vol. 7892, 2011, pp . 1-7 .

Hagen et al., “ Video -Rate Spectral Imaging of Gas Leaks in the
Longwave Infrared ,” Chemical, Biological, Radiological, Nuclear,and Explosives (CBRNE ) Sensing XIV , May 29 , 2013 , SPIE
Proceedings vol. 8710 , pp. 7 .
Harley et al., “ Remote Quantification of Smokestack Effluent Mass
Flow Rates Using Imaging Fourier Transform Spectrometry,"
Chemical, Biological, Radiological, Nuclear, and Explosives(CBRNE ) Sensing XII, Apr. 25 -29 , 2011 , SPIE Proceedings vol.
8018 , pp . 1 - 13 .
Hayden et al., “ Determination of Trace -Gas Amounts in Plumes by
the Use of Orthogonal Digital Filtering of Thermal-Emission Spectra ” , Applied Optics, Jun. 1, 1996 , vol. 35 , No . 16 , pp . 2802 - 2809 .

Hirsch et al., “ Detection of Gaseous Plumes in IR Hyperspectral
Images Using Hierarchical Clustering ” , Applied Optics, Sep . 1,
2007, vol. 46 , No. 25 , pp. 6368 -6374 .
Johnston et al., “ A Real- Time FPGA Implementation of a BarrelDistortion Correction Aglorithm ” , Projects , 2003 , vol. 10 , pp .

91- 96 .

Karion et al., “Methane Emissions Estimate from Airborne Mea
surements Over a Western United States Natural Gas Field ” , Geophysical Research Letters, 2013 , vol. 40 , pp . 4393 -4397 .

Keshava et al., " A Survey of Spectral Unmixing Algorithms” ,Lincoln Laboratory Journal, 2003 , vol. 14 , No. 1 , pp . 55 -78.
Kester et al., “ A Real- Time Gas Cloud Imaging Camera for Fugitive
Emission Detection and Monitoring” , Imaging and Applied Optics

Technical Digest, 2012 , pp . 3 .
Kester et al., “ Development of Image Mappers for Hyperspectral

Biomedical Imaging Applications” , Applied Optics, Apr. 1, 2010 ,vol. 49 , No. 10 , pp . 1886 - 1899.
Kester et al., “ High Numerical Aperture Microendoscope Objective

for a Fiber Confocal Reflectance Microscope” , Optics Express ,Mar.
5 , 2007, vol. 15 . No. 5 , pp . 2409 - 2420 .
Kester et al., “ Low Cost, High Performance , Self-Aligning Minia
ture Optical Systems” , Applied Optics, Jun. 20 , 2009, vol. 48 , No.

18, pp. 3375 -3384.

Kester et al., “ Real- Time Snapshot Hyperspectral Imaging Endo

scope” , Journal of Biomedical Optics ,May 2011, vol. 16 , No. 5 , pp .
056005 - 1 - 056005 - 12 .
Kudenov et al., “ Fourier Transform Channeled Spectropolarimetry
in the MWIR ” , Optics Express , Oct. 1, 2007, vol . 15 , No . 20 , pp .
12792- 12805.

Kudenov et al., “ Snapshot Imaging Mueller Matrix Polarimeter
Using Polarization Gratings” , Optics Letters , Apr. 15 , 2012, vol. 37,
No. 8 , pp . 1367- 1369.
Landau et al., “ Design and Evaluation of an Ultra - Slim Objective
for in -vivo Deep Optical Biopsy ” , Optics Express, Mar. 1, 2010 ,vol. 18 , No. 5 , pp . 4758 - 4775 .
Levi, Michael A ., " Comment on ’Hydrocarbon Emissions Characterization in the Colorado Front Range : A Pilot Study by Gabrielle
Petron et al." , Journal ofGeophysical Research , 2012 , vol. 117 , No .

D21203 , pp . 1 -5 .
Levi, Michael A ., “ Reply to “ Reply to 'Comment on ’Hydrocarbon

Emissions Characterization in the Colorado Front Range A Pilot

Study' by Michael A . Levi?" " by Gabrielle Pétron et al., Journal ofGeophysical Research : Atmospheres, 2013 , vol. 118, pp . 3044
3046 .

Low et al., “ Remote Sensing and Characterization of Stack Gases byInfrared Spectroscopy . An Approach by Using Multiple -Scan Inter
ferometry " , Environmental Science & Technology , Jan . 1967, vol.
1 , No . 1 , pp . 73 - 74 .
Luo et al., “ Fast Processing of Imaging Spectrometer Data Cube

Based on FPGA Design ” , MIPPR 2007 : Multispectral Image Pro

cessing, Proceedings of SPIE vol. 6787, pp . 7.

Manolakis et al., “ Long-Wave Infrared Hyperspectral Remote Sens
ing of Chemical Clouds” , IEEE Signal Processing Magazine , Jul.
2014 , vol. 31 , No. 4 , pp . 120 - 141.
Mathews, “ Design and Fabrication of a Low - Cost, Multispectral
Imaging System ,” Applied Optics , 2008, pp . F71 -F76 , vol. 47.
Naranjo et al., “ IR Gas Imaging in an Industrial Setting,"

Thermosense XXXII, Published in SPIE Proceedings vol. 7661,

May 4 , 2010 , pp . 1 -8 .
Nguyen et al., “ Snapshot 3D Optical Coherence Tomography Sys
tem using Image Mapping Spectrometer” , Biomedical Optics and
3D Imaging OSA , 2012 , pp . 3 .

Niu et al., “ New Approach to Remote Gas- Phase Chemical Quan
tification : Selected -Band Algorithm ” , Optical Engineering , Feb .
2014 , vol. 53 , No. 2 , pp . 021111- 1 - 021111- 10 .
“ Oil and Natural Gas Sector Leaks” , U .S . EPA Office of Air Quality
Planning and Standards (OAQPS ), Review Panel, Apr. 2014 , pp .63 .

Pétron et al., “ Hydrocarbon Emissions Characterization in the
Colorado Front Range: A Pilot Study ” , Journal of GeophysicalResearch , 2012 , vol . 117 , No. D04304 , pp . 1 - 19 .
Pétron et al., “ Reply to Comment on 'Hydrocarbon Emissions
Characterization in the Colorado Front Range — A Pilot Study ' byMichael A . Levi” , Journal of Geophysical Research : Atmospheres,2013 , vol. 118 , pp . 236 -242 .

Pisano et al., “ Thermal Illuminators for Far-Infrared and Submil
limeter Astronomical Instruments,” Applied Optics, Jun . 1, 2005,
vol. 44 , No. 16 , pp . 3208- 3217 .
Polak et al., “ Passive Fourier - Transform Infrared Spectroscopy ofChemical Plumes: An Algorithm for Quantitiative Interpretation
and Real- Time Background Removal” , Applied Optics , Aug. 20 ,1995, vol. 34 , No . 24 , pp . 5406 - 5412 .
Rebellion Photonics, “ Gas Cloud Imaging Camera : A Breakthroughin Leak Monitoring for the Rig & Refinery Safety Market” , Pre
sentation at SPIE Defense Security and Sensing, 28 pages , Apr.
29 -May 3 , 2013 .

Sandsten et al., “ Volume Flow Calculations on Gas Leaks Imaged

with Infrared Gas -Correlation ,” Optics Express , 2012, vol. 20 , No.
18, pp . 20318 -20329 .
Shogenji et al., “Multispectral Imaging Using Compact Compound
Optics," Optics Express, Apr. 19 , 2004 , vol. 12 , No. 8 , pp . 1643
1655.

Telops,“Hyper -Cam ” ,http ://web .archive .org/web /20160608180941/http ://www .telops .com / en /hyperspectral- cameras/ hyper -cam as archived Jun . 8 , 2016 in 2 pages.

Telops, “ Innovative Infrared Imaging ” , http ://web .archive.org /web /20160603212729 /http ://www.telops.com /en / as archived Jun. 3 ,2016 in 2 pages.

Walter Jr., et al., “ Detection of Atmospheric Pollutants : a Correlation Technique” , Applied Optics, Jun . 1975, vol. 14 , No. 6 , pp .
1423 - 1428 .
Young et al., “ An In -Scene Method for Atmospheric Compensationof Thermal Hyperspectral Data ” , Journal of Geophysical Research ,2002, vol. 107, No . D24 , pp . 14 - 1 - 14 - 20 .
Zheng et al., " A Static Multiplex Fabry -Perot Spectrometer” , Sensors , Cameras, and Systems for Industrial/Scientific Applications X ,Proceedings of SPIE - IS & T Electronic Imaging, SPIE vol. 7249,
2009, pp . 8 .

Zheng et al., " Analytic -Domain Lens Design with Proximate Ray

Tracing” , Journal of the Optical Society of America A , Aug . 2010 ,vol. 27 , No. 8 , pp . 1791- 1802.

Preliminary Amendment as filed in U .S .Appl.No. 14 /538,827 dated
Jan . 28 , 2015 in 6 pages .

Office Action received in U .S . Appl. No. 14 / 538 ,827 dated Jun . 30 ,2015 in 8 pages .
Non - Final Office Action Response as filed in U . S . Appl. No .

14 /538 , 827 dated Dec. 28 , 2015 in 11 pages .
Interview Summary received in U .S . Appl. No . 14 /543 ,692 dated
Feb . 17 , 2016 in 5 pages.

Office Action received in U .S . Appl. No. 14 /543,692 dated Nov. 3 ,Response to Office Action as filed in U .S . Appl. No. 14 /543,692
dated May 2 , 2016 in 9 pages.
Office Action received in U .S . Appl. No. 14 /543,692 dated Jun. 1,
2016 in 18 pages.
2015 in 7 pages.

US 9,756,263 B2
Page 5( 56 )

References Cited

OTHER PUBLICATIONS

Office Action received in U .S. Appl. No. 14 /539 ,899 dated Mar. 26 ,2015 in 6 pages .

Non - Final Office Action Response as filed in U .S . Appl. No .
14 / 539 ,899 dated Aug. 26 , 2015 in 8 pages.
Final Office Action received in U .S . Appl. No. 14 /539 ,899 dated
Dec . 11 , 2015 in 9 pages.
Official Communication received in European Application No .
14192862.2 dated Apr. 19 , 2016 in 6 pages.
Extended European Search Report received in European Application No . 14192862.2 dated Mar. 30 , 2015 in 10 pages .
International Search Report in PCT Application No . PCT /US2013 /
041278 dated Aug . 27 , 2013 in 4 pages.
International Preliminary Report on Patentability in PCT Application No . PCT/US2013 /041278 dated Nov . 27 , 2014 in 10 pages.
Extended European Search Report received in European Application No. 15165877 . 0 dated Oct. 8 , 2015 in 12 pages .
Preliminary Amendment as filed in U . S . Appl.No. 14 /700 ,567 dated
Jul. 10 , 2015 in 6 pages.
Miscellaneous Communication dated Aug . 24 , 2016 filed in U .S .

Adams, et al., “ Advances in Detectors: Hot IR sensors improve IR
camera size , weight, and power” , Laser Focus World , vol. 50 , Issue

1, Jan . 17 , 2014 , 6 pages. Also available at http ://www .ircameras.
com /articles /advances -detectors -hot -ir-sensors -improve -ir -camera

size -weight-power/.
Notice of Allowance received in U . S . Appl. No. 14 /538 ,827 dated
Sep . 19 , 2016 in 9 pages .
Response to Final Action as filed in U . S . Appl. No. 14 /543,692
dated Nov . 30 , 2016 in 12 pages .

Notice of Allowance received in U .S . Appl. No. 14 /543,692 dated
Dec. 9 , 2016 in 12 pages.
Amendment as filed in U . S . Appl. No . 14 /539 ,899 dated Jun . 9 ,2016 in 6 pages .
Notice of Allowance received in U . S . Appl. No. 14 /539 ,899 dated
Oct. 31, 2016 in 10 pages .
Preliminary Amendment as filed in U .S . Appl. No. 15 / 166 ,092 dated
Aug. 15 , 2016 in 7 pages .

International Search Report in PCT Application No. PCT/US2016 /
034455 dated Oct. 24 , 2016 in 12 pages.

Appl. No . 14 /700 , 567.

Wikipedia entry https :// en .wikipedia .org/wiki/Mobile _ computing
in 6 pages, last modified on Dec . 30 , 2016 ; retrieved from theinternet on Feb . 2 , 2017 .
Amendment as filed in U .S . Appl. No. 14 /538 ,827 dated Dec . 16 ,tion No . EP 15165880 .4 dated Nov. 24 , 2015 in 8 pages.

Amendment as filed in U .S. Appl. No . 14/539, 899 dated Jan. 27,

Extended European Search Report received in European Applica
Preliminary Amendment as filed in U . S . Appl. No. 14 /792, 477 dated

2016 in 9 pages.

2017 in 5 pages.

Dec . 21 , 2015 in 7 pages.
Corrected Notice of Allowance received in U .S . Appl. No.
14 / 538 ,827 dated Feb . 10 , 2016 in 4 pages.
Corrected Notice of Allowance received in U .S . Appl. No .

Official Communication received in European Application No.

Notice of Allowance received in U .S . Appl. No. 14 /538 , 827 dated
Feb . 1, 2016 in 18 pages.
Notice of Allowance received in U .S . Appl. No. 14 /538 ,827 dated

dated Mar. 3 , 2017 in 6 pages .

14 / 538, 827 dated Feb . 22, 2016 in 4 pages.

15165877 .0 dated Jan . 3 , 2017 in 9 pages .
Official Communication received in U . S . Appl. No . 14 /792 ,477

dated Jan . 27 , 2017 in 10 pages.
Amendment after Allowance as filed in U .S . Appl. No. 14 /543,692

May 26 , 2016 in 9 pages.
Notice of Allowance received in U .S . Appl. No. 14 /539 ,899 dated

Notice of Allowance received in U .S . Appl. No. 14 /543 ,692 dated
Mar. 17 , 2017 in 4 pages.
Office Action received in U .S . Appl. No. 14 /700 ,567 dated Jun . 14 ,2017 in 29 pages .
Response to Restriction Requirement submitted in U . S . Appl. No.
14 /792,477 dated May 8, 2017 in 6 pages .

2016 in 14 pages.

* cited by examiner

Jun. 21 , 2016 in 17 pages.
Request for Continued Examination and Response to Correct Application Papers as filed in U . S . Appl. No. 14 /538 ,827 dated Apr. 29 ,

U . S . Patent

Sep. 5 , 2017

128
,
Ora

COLD

136aDert catoyr

1Towards pro5ces0or

130pdSiuevcptirdaledy
Popiwiki

oblective
Front
124
,
lens

US 9 ,756 ,263 B2

Sheet 1 of 23** * * **** * * * ** ** * ** *** ** * * * ** * ** * * *** * ** * * * ***,Oc09L'eoquBdmjAoaOjsNa* * * * * * ** * * ** * * * *nilineA

inf it on WinWindows

128a 0??

Nii

nit a

?????????

YUXX

.
www

120

wimshwahitrmnit

w ww

l

N

120

Wow

!
!
>
*

Object

WISHA

ro+BHlIiMsAhiSfHtAitIiet itut itutilitiativi{thisfi

l

120
120

H

}tat??+full

IMISHA110 gasdoud

ithilif4aitt:

**

***
*
*

hu
!"

ins#!

ity

FIG
.
1

120

U . S . Patent

Sep. 5 , 2017

US 9 ,756 ,263 B2

Sheet 2 of 23m

Field Reference 204
w

aisiawiiss MA li dili
WSH

wwwwwwwwww

?????

:
www

202

W Ww
wWw w

Witamination

w

w

w

itcoix922

w S.ow

224 100

Verlapping FOV
overlapping
FOV Region
Region, 230

wyArtist ME:4W0omwW uset

Wet ertintasu

WWW:

W1642

Wivitisw :WWWw

128a***

1 - Parallax 228

:
:
:
:

www:

- 226

goud toWS timinu irtin 982

WarWanli
id

-

sist

:

4

W
:
www

wurs

xegered4

be some como conoscono con la sua casa
200

Towards

processor 150

FIG . 2

U . S . Patent

Sep. 5 , 2017

Field Reference

US 9,756 ,263 B2

Sheet 3 of 23

OEL

338

WorjiM l i 128

300
- 300

Fuck

324
itinu

??
226
* S

T

.w w Wiwndiowtsh w w???????????????????????????????????????????????????????????????? ?? ??? ? ?

Object 110

w

W

Processor , 160

90€

??

wanita324a 128a

ww

Ww si
www
.

/11747 IW

FIG . 3A

T473462.mmm

ww

w Towardsw

wri ir

SAW

336
98€

wmrinuw
ARM

ih

POS

FIG . 3B

w 12

U . S . Patent

Sep. 5 , 2017

amaging

4.
FIG,150PTrowcaesrdosr

456,Cmetatecos

128
,
hirdy
Lans

US 9,756 ,263 B2

Sheet 4 of 23

128a

454
,
Anay
Flor

Moveabl tempraue scohnutrlesd?

????????????????

i

ummm
m.
:ista

tisininihit i*le *

nini
?
init

werk

11:.w w1Heh

460a,460b????????????????????

sirsarsiz

insa l yis nde

44?? ? ? ?? ???????
????? ?
wwwwwwwwwwww

wys

Www
ws
**

*

* ** * *

S

W

201

ww
ww

diteri prin:hi

RLASelrasy 440

.

*
**
*

11
.
17
:

tilt

ismecto2

WW.16:20

FooRrayef rence 438 viearmnlz

WWW
W
.
i

450

lens,424aodfercoindte

W

222
????

? X*W
–
226

Towards
.
km
110
,
Obed

.

mitsir tsit

438a

11:111111

w WwM w W w W
pitit

:

**

WW

WW

,ri

:41aft**** *

'438a
454a

*

?????????

**

128a
!
444

n

:17persepsitet

.
1

????
???

400

|

13H
:
31
(

7

surtit/Y!W W-148762.wirvn MW

424
450

U . S . Patent

Sep. 5 , 2017

02134

8
|
7
6
5
5678

US 9,756 ,263 B2

Sheet 5 of 23

12
11
10
9

5B
.
FIG

?????????

12
11
10
9

5A
.
FIG

U . S . Patent

Sep. 5 , 2017

US 9,756 ,263 B2

Sheet 6 of 23

filteroa filterla filter2a filter3a?????

?? ????

I

-

II

.
|

TT

LI
-

-

IIIIIIIIIIIIIIIILULLIV
L ' IYYZ
-

:

11
???

(um)Wavel ngth???

IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIITUTIMITIUMTURNA

HILT

10

IT

0F
.
1

MITTIILILIITILIITTITULTITIIIIIIIIIIIIIIIIIIIUL

84
.
0

0
.
24

transmission

H00

6A
.
FIG

U . S . Patent

Sep. 5 , 2017

US 9,756 ,263 B2

Sheet 7 of 23

filterob filterlb filter2b filter3b
I

-

.

-

.
.

13

IIIIIIIIIIIII UI-

YL' ISY -

-

-

-

-

- ArronT.

-

-

-

-L

I

|

-

I

I

I

.
.
.
-

101047

2
.
0

8
.
0

10ISSIUSURI

12

A . R . LI

LLLLL
IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIITATTUTT

I

I

?
400

11

(um)wavel ngth
)

10

6B
.
FIG

U . S . Patent - Sep . 5, 2017

US 9 ,756 ,263 B2

Sheet 8 of 23

filteroc filteric filter2c filter3c
II

-

Prook

H|

112
-

- ? ? ? - - - - -- - - - - - -- ???????? ???????????

A)um(wavel ngth

LI
IIIIIII

-

-

-

-

-

-

-

- - - - - - ??? ? ? - -

DE
OE

I

I

IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII? ? ?? ?
transmission

60
.
FIG

U . S . Patent

Sep. 5 , 2017

US 9,756 ,263 B2

Sheet 9 of 23

filterod filterld filter2d filter3d
-

II

-

.

F

ET

TIITTIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII

11
pood

(um)wavel ngthLTU

- - - - - - - - - - - - -10

9

IIIIIIIIIIIIIIIIIIIIIIIIII IIIIIIIIIIIIIIIHITALK8+
2.
0

voisssue

.
0

6D
.
FIG

U . S . Patent

Sep. 5 ,2017

US 9 , 756,263 B2

Sheet 10 of 23

womances
station.

FIG . 7

U . S . Patent

Sep. 5 , 2017

US 9 ,756 ,263 B2

Sheet 11 of 23

24 . 5

WALA*??????????????????

24 .01 .

sritisha)(CTemplesraiturse

-- -

ti

-

prw

wy

?????????

ref

PP

2PRJYI
22 . 04

21..
5.
.

chig i ti n

21.. 00 kmkommen
21

VULYYLU

N??????????????

LUVLY

A

10

???????????????1 2

Wavelength (microns )FIG . 8AYYYYYYYr*

HETHER)(CTemplesraiturseW

obj corr
UUTIN
Y

ref

YYYYYYYYYYY

WWW

2016

wwwwwwwwwwww

Wavelength (microns )FIG . 8B

?????

8

10

Mimng

Prime

12

U . S . Patent

Sep. 5 , 2017

Sheet 12 of 23

US 9 ,756 ,263 B2

900

to object975b9752 -

M1B

MOA

W

960

928a

thermally

controlled

shutters

camera
block

936

972a

array
980d

980a
980b - MA

M188

*

M2B

M9A980e
Mga
MGA

980c

980f

FIG . 9A

U . S . Patent

Sep. 5 , 2017

Sheet 13 of 23

US 9 ,756 ,263 B2- 905

to object
9722

960 –

thermally

controlled
shutters
camera
block
980a

M

980b

X

972b

W

928a
936

- 980dMes- 980e

M10M???????????????????????????

9800

11

980f

FIG . 9B

U . S . Patent

Sep. 5 , 2017

US 9 ,756 ,263 B2

Sheet 14 of 23

1000

MMMM

09
.
10
11
@ 600

camerasmirrorsreferencesources0000
FIG . 10A

1000

mammu

8 9 10 11v**TAVA*

iri** **TUYE

O
FIG . 10B

U . S . Patent

Sep. 5 , 2017

US 9 ,756 ,263 B2

Sheet 15 of 23

cameras s renfrencesources
o

1050

11A

R7B

RTA

b

R
3B
***

M

***

3A
R

10C
.
FIG

**
28
R

10B Art but

DIDI
R2A

RIVA
Rob
RA
RE
RgA

W

9B

RIB

RIA

*

th

.

I

rinn

tity

****
****

MX
R

RBA

ROB

ROA

U . S . Patent

Sep. 5 , 2017

US 9 ,756 ,263 B2

Sheet 16 of 23

02

203

ZIZAN

FOV

O

1000-

/12047X
1200

1275

11A
.
FIG

Sep. 5, 2017

U . S. Patent

Sheet 17 of 23

US 9,756,263 B2

150
-

fo d

225

-

302

10 5

zFOV

11B
.
FIG

|

10 da

+

1303

db.

109

1300

S

750

??
fo d

U . S . Patent

Sep. 5 , 2017

71021
71020
1022

PROCESOR

US 9 ,756 ,263 B2

Sheet 18 of 23

1023

1024

71025

COMUNIATO

M
O
D
U
L
E
P
U
R
O
N
C
E
I
S
T
N
G
ME ORYSOPYTICEAMLSPUOWPELRY

1015

1026

1000

GPSMODULE
12.
FIG

MPADCROQUNIESTLDANOEGU . S . Patent

Sep. 5 ,2017

|

-

|

?

? 10 8|-|

US 9 . 756,263 B2

Sheet 19 of 23

?109

1070

/
000
- -

?

-

-

# ?? ?
-

-

r10 3

-

-

"
%
100

-

-

.
-

-

-

-

1004
)
?
1004

-

13A
.
FIG

-

C

10 5

-

?
-

-

-

11015075?

-||| - f _

|

A

10 6

?

OBJECT 1007

U . S . Patent

Sep. 5 , 2017

US 9 ,756 ,263 B2

Sheet 20 of 23

1450
Opp

OETI
1408

13B
.
FIG

71403

A

v

1401 ZOPI
POP

1
+
90

1015
-0001

OBJECT 6011

U . S . Patent

Sep. 5 , 2017

Sheet 21 of 23

US 9 ,756 ,263 B2

14A
.
FIG
m

120

100010 0

U .S . Patent

Sep . 5 , 2017

Sheet 22 of 23

US 9 ,756 , 263 B271592-

10 0

14B
.
FIG
1590

U

+
9091
>

U . S . Patent

US 9,756 ,263 B2

Sheet 23 of 23

Sep. 5 , 2017

1586

1588
-

#

X
-

508
.
1

150214B

1502

1584

.
FIG
14C

I

1590

1015
1506

1580
1000%

102

US 9 , 756 , 263 B2
MOBILE GAS AND CHEMICAL IMAGINGCAMERACROSS -REFERENCE TO RELATEDAPPLICATIONSThis application claimspriority to U .S . Provisional Patent
Application No. 61/ 986 ,885 , filed May 1, 2014 , entitledof the scene in different spectral regions). Such scanningapproaches acquire only a portion of the full data cube at atime. These portions of the full data cube are stored and thenlater processed to generate a full data cube .
SUMMARY

The systems,methods and devices of this disclosure each

“ MINIATURE GAS AND CHEMICAL IMAGING CAM - have several innovative aspects , no single one of which is
ERA ;" U . S . Provisional Patent Application No . 62 /012 , 078 , 10 solely responsible for the desirable attributes disclosed

filed Jun. 13 , 2014 , entitled “ MINIATURE GAS AND
CHEMICAL IMAGING CAMERA ;” U .S . Provisional Pat ent Application No. 62/054 , 894 , filed Sep . 24 , 2014 , entitled“ MOBILE GAS AND CHEMICAL IMAGING CAM -herein .
In one embodiment, an infrared (IR ) imaging system for

filed Sep . 25 , 2014 , entitled “MOBILE GAS AND CHEMI

optical system can have components defining at least two

cation No. 62/055 ,549 , filed Sep . 25 , 2014 , entitledbeing spatially and spectrally different from one another .

determining a concentration of a target species in an object
is disclosed . The imaging system can include an optical

ERA ;” U .S . Provisional Patent Application No. 62/055 , 342 , 15 system including an optical focal plane array (FPA ) unit. The

CAL IMAGING CAMERA ;" U .S . Provisional Patent Appli-" MOBILE GAS AND CHEMICAL IMAGING CAM

optical channels thereof, said at least two optical channels

Each of the at least two optical channels can be positioned

ERA :” and U . S . Provisional Patent Application No . 62/ 082, 20 to transfer IR radiation incident on the optical system

613 , filed Nov. 20, 2014 , entitled " MOBILE GAS AND

CHEMICAL IMAGING CAMERA,” the contents of eachtowards the optical FPA . The system can include a process

ing unit containing a processor that can be configured toof which is hereby incorporated by reference herein in theiracquire multispectral optical data representing said target

APERTURE INFRARED SPECTRAL IMAGER (DAISI)

configured to be worn or carried by a person .

entirety and for all purposes . This application also claims species from the IR radiation received at the optical FPA .
priority to U . S . Provisional Patent Application No . 61/ 986 , 25 Said optical system and said processing unit can be con
886 , filed May 1 , 2014 , entitled “ DUAL BAND DIVIDED
tained together in a data acquisition and processing module

FOR CHEMICAL DETECTION ;" U . S . Provisional Patent

In another embodiment, an infrared (IR ) imaging system

Application No. 62 /082 ,594 , filed Nov. 20 , 2014, entitledfor determining a concentration of a target species in an" DUAL -BAND DIVIDED - APERTURE INFRA -RED 30 object is disclosed . The imaging system can comprise an

SPECTRAL IMAGING SYSTEM ;" U .S . Provisional Patent

Application No . 62 /021,636 , filed Jul. 7 , 2014 , entitled"GAS LEAK EMISSION QUANTIFICATION WITH AGAS CLOUD IMAGER ;" U .S . Provisional Patent Applica -

optical system including an optical focal plane array (FPA )unit. The optical system can have components defining atleast two optical channels thereof, said at least two opticalchannels being spatially and spectrally different from onetion No. 62/021, 907 , filed Jul. 8 , 2014 , entitled “GAS LEAK 35 another. Each of the at least two optical channels can be
EMISSION QUANTIFICATION WITH A GAS CLOUD
positioned to transfer IR radiation incident on the optical

IMAGER;" and U .S. Provisional Patent Application No.

62 /083 , 131 , filed Nov. 21 , 2014 , entitled “ GAS LEAK

system towards the optical FPA . The system can include a

processing unit containing a processor that can be config

EMISSION QUANTIFICATION WITH A GAS CLOUD ured to acquire multispectral optical data representing said
IMAGER ,” the contents of each of which is hereby incor- 40 target species from the IR radiation received at the opticalporated by reference herein in their entirety and for all
purposes .

FPA . Said data acquisition and processing module can have

TECHNICAL FIELD

for determining a concentration of a target species in an

dimensions less than 8 inchesx6 inchesx6 inches .

In another embodiment, an infrared (IR ) imaging system

45 object is disclosed . The imaging system can include an
optical system including an optical focal plane array (FPA )The present invention generally relates to a system and unit. The optical system can have components defining at

Field of the Invention

method for gas cloud detection and , in particular , to a system

least two optical channels thereof, said at least two opticaland method of detecting spectral signatures of chemical channels being spatially and spectrally different from one
compositions in infrared spectral regions .
50 another . Each of the at least two optical channels can bepositioned to transfer IR radiation incident on the optical
Description of the Related Technology

Spectral imaging systems and methods have applications
obtain a spectral image of a scene in one or more regions ofsystem towards the optical FPA . The system can include aured to acquire multispectral optical data representing said

material compositions or characterize processes. The spec -FPA . Said data acquisition and processing module can havetral image of the scene can be represented as a three dimensional data cube where two axes of the cube represent
two spatial dimensions of the scene and a third axis of thea volume of less than 300 cubic inches.
In yet another embodiment, a method of identifying atarget species or quantifying or characterizing a parameter ofcessed using mathematical methods to obtain information

ing module . The data acquisition and processingmodule canscanning a wavelength dispersive element to obtain images

image data at the FPA unit from at least two optical channels

in a variety of fields. Spectral imaging systems and methods

processing unit containing a processor that can be config

the electromagnetic spectrum to detect phenomena , identify 55 target species from the IR radiation received at the optical

data cube represents spectral information of the scene in 60 the target species in an object is disclosed . The method candifferent wavelength regions . The data cube can be pro - include wearing or carrying a data acquisition and process

about the scene. Some of the existing spectral imaging
comprise an optical system and a processing unit in com
systems generate the data cube by scanning the scene in the munication with the optical system , the optical system
spatial domain ( e . g ., by moving a slit across the horizontal 65 including an optical focal plane array (FPA ) unit . The
dimensions of the scene) and /or spectral domain ( e. g ., by method can include capturing multispectral infrared (IR )US 9, 756 ,263 B2
that are spatially and spectrally different from one another.

FIG . 6B depicts theoretical plots of transmission charac

The method can include acquiring multispectral optical data

teristics of a spectrally multiplexed notch - pass filter combi

representing the target species from the IR radiation nation used in an embodiment of the system .
FIG . 6C shows theoretical plots of transmission characreceived at the FPA .
In another embodiment, a system for monitoring the 5 teristics of spectrally multiplexed long -pass filter combina
presence of one or more target gases at one or moretion used in an embodiment of the system .

installation sites is disclosed . The system can include a

FIG . 6D shows theoretical plots of transmission characplurality of infrared ( IR ) imaging systems, each imaging

teristics of spectrally multiplexed short - pass filter combina

system comprising a data acquisition and processing mod -tion used in an embodiment of the system .

configured to capture infrared images of the one or more

an embodiment of the system used for gas detection .

ule . The data acquisition and processing module can be 10

FIG . 7 is a set of video - frames illustrating operability oftarget gases in real-time. The data acquisition and processing

FIGS. 8A and 8B are plots (on axes of wavelength inmodule can be configured to associate each captured infra -microns versus the object temperature in Celsius represent

red image with a location at which the one or more target

ing effective optical intensity of the object ) illustrating

module can be configured to transmit image data associatedwith the one or more target gases and location data associsystem .
FIGS. 9A and 9B illustrate a cross -sectional view ofdifferent embodiments of an imaging system comprising angases are present. The data acquisition and processing 15 results of dynamic calibration of an embodiment of theated with the location of the one or more target gases to acentral server.

arrangement of reference sources and mirrors that can be
In yet another embodiment, a method for monitoring the 20 used for dynamic calibration .

presence of one or more target gases at one or moreinstallation sites is disclosed . The method can comprisereceiving image data from a plurality of IR imaging systems

FIGS . 10A - 10C illustrate a plan view of different embodi
ments of an imaging system comprising an arrangement ofreference sources and mirrors that can be used for dynamic

configured to capture infrared images of the one or moretarget gases in real -time and to associate each captured

infrared imaging system configured to be carried or worn by
a human user.

target gases are present. The method can include processing

tion site that can be monitored by multiple infrared imaging

located at a plurality of installation sites and configured to be calibration.
worn or carried by a person . Each IR imaging system can be 25 FIG . 11A is a schematic diagram illustrating a mobile
infrared image with a location at which the one or more

FIG . 11B is a schematic diagram illustrating an installa

the received image data to identify the installation sites at 30 systems

FIG . 12 is a schematic system block diagram showing awhich the one or more target gases is detected .
Details of one or more implementations of the subject

mobile infrared imaging system , according to one embodi

matter described in this disclosure are set forth in thement.

accompanying drawings and the description below . Other

FIG . 13A is a schematic system diagram of an opticalfeatures, aspects , and advantages will become apparent from 35 system configured to be used in the mobile infrared imaging

the description , the drawings and the claims. Note that thesystems disclosed herein , according to various embodi

relative dimensions of the following figures may not bements .
FIG . 13B is a schematic system diagram of an opticaldrawn to scale .

BRIEF DESCRIPTION OF THE DRAWINGS

system configured to be used in the mobile infrared imaging

40 systems disclosed herein , according to other embodiments.

FIG . 1 shows an embodiment of an imaging system

FIG . 14A is a schematic perspective view of a mobile
infrared imaging system mounted to a helmet, according toincluding a common front objective lens that has a pupil

various embodiments .

tive lens and an array of infrared sensing FPAs.

mobile infrared imaging system shown in FIGS. 14A - 14B .
Like reference numbers and designations in the various
drawings indicate like elements .

divided spectrally and re - imaged with a plurality of lenses
FIG . 14B is an enlarged schematic perspective view of theonto an infrared FPA .
45 mobile infrared imaging system shown in FIG . 14A .
FIG . 2 shows an embodiment with a divided front objec FIG . 14C is a perspective cross -sectional view of the

FIG . 3A represents an embodiment employing an array offront objective lenses operably matched with the re -imaging

lens array . FIG . 3B illustrates a two - dimensional array of 50

optical components corresponding to the embodiment of

DETAILED DESCRIPTION

FIG . 3A .

FIG . 4 is a diagram of the embodiment employing anarray of field references ( e.g ., field stops that can be used as

I. Overview of Various Embodiments

references for calibration ) and an array of respectively 55 The following description is directed to certain imple
corresponding relay lenses .
mentations for the purposes of describing the innovative
FIG . 5A is a diagram of a 4 -by -3 pupil array comprising aspects of this disclosure . However, a person having ordi

circular optical filters ( and IR blocking material between the nary skill in the art will readily recognize that the teachings
herein can be applied in a multitude of different ways . The
60 described implementations may be implemented in any
imaged with an embodiment of the system .
optical filters ) used to spectrally divide an optical wavefront
FIG . 5B is a diagram of a 4 -by -3 pupil array comprising
rectangular optical filters (and IR blocking material between

device, apparatus , or system that can be configured tooperate as an imaging system such as in an infra -red imaging

the optical filters ) used to spectrally divide an optical system . The methods and systems described herein can be
included in or associated with a variety of devices such as,
FIG . 6A depicts theoretical plots of transmission charac - 65 but not limited to devices used for visible and infrared
teristics of a combination of band - pass filters used with anspectroscopy, multispectral and hyperspectral imaging
embodiment of the system .
devices used in oil and gas exploration, refining, and transwavefront imaged with an embodiment of the system .

US 9 ,756 ,263 B2
5
portation , agriculture , remote sensing, defense and homeland security , surveillance , astronomy, environmental moni
toring, etc. The methods and systems described herein haveapplications in a variety of fields including but not limited to

implementations, the imaging systems disclosed herein do
not include detectors configured to be cooled to a tempera
ture below 200 Kelvin .

Implementations disclosed herein provide several advan
land security, environment, oil and gas industry, etc. The all of which may require FPAs that are highly sensitive andteachings are not intended to be limited to the implementa
cooled in order to compensate , during the optical detection ,for the reduction of the photon flux caused by spectrum
tions depicted solely in the Figures, but instead have wide scanning. The highly sensitive and cooled FPA
applicability as will be readily apparent to one having 10 systems areoperation
expensiveand require a great deal of mainte
ordinary skill in the art.
nance
.
Since
variousembodiments
disclosed herein are
The spectral image of the scene can be represented as a

agriculture, biology , physics , chemistry , defense and home- 5 tages over existing IR spectral imaging systems, most if notthree -dimensional data cube where two axes of the cube

configured to operate in single -shot acquisition mode without spatial and /or spectral scanning, the instrument canrepresent two spatial dimensions of the scene and a third
receive photons from a plurality of points (e .g ., every point)
axes of the data cube represents spectral information of the 15 of the object substantially simultaneously , during the single

scene in different wavelength regions. The data cube can beprocessed using mathematical methods to obtain information about the scene. Some of the existing spectral imaging
systems generate the data cube by scanning the scene in thereading . Accordingly, the embodiments of imaging systemdescribed herein can collect a substantially greater amountof optical power from the imaged scene ( for example , anorder of magnitude more photons ) at any given moment inspatial domain (e .g ., by moving a slit across the horizontal 20 time especially in comparison with spatial and /or spectral

and vertical dimensions of the scene) and /or spectral
domain . Such scanning approaches acquire only a portion of

scanning systems. Consequently , various embodiments of

the imaging systems disclosed herein can be operated using

the full data cube at a time. These portions of the full data

uncooled detectors (for example, FPA unit including anarray of microbolometers ) that are less sensitive to photons
data cube .
25 in the IR but are well fit for continuous monitoring appli
Various embodiments disclosed herein describe a divided
cations. For example, in various implementations, the imag
aperture infrared spectral imaging (DAISI) system that is ing systems disclosed herein do not include detectors con
structured and adapted to provide identification of target figured to be cooled to a temperature below 300 Kelvin . As
cube are stored and then later processed to generate a fullchemical contents of the imaged scene . The system is based

another example , in various implementations, the imaging

on spectrally - resolved imaging and can provide such iden - 30 systems disclosed herein do not include detectors configuredtification with a single - shot ( also referred to as a snapshot)
to be cooled to a temperature below 273 Kelvin . As yet

comprising a plurality of images having different wave -another example , in various implementations, the imaging

length compositions that are obtained generally simultane -systems disclosed herein do not include detectors configured

ously . Without any loss of generality, snapshot refers to a

to be cooled to a temperature below 250 Kelvin . As anotherare continuously viewing the light emitted from the scene . In

disclosed herein do not include detectors configured to becontrast in scanning systems, at any given time only aminority of data elements are continuously viewing a scene ,cooled to a temperature below 200 Kelvin . Imaging systems
including uncooled detectors can be capable of operating infollowed by a different set of data elements , and so on , until

extreme weather conditions, require less power, are capable

use spectral or spatial scanning for the acquisition of infra -ceptible to motion artifacts in comparison with spatiallyred ( IR ) spectral signatures of the target chemical contents.

and /or spectrally scanning systemswhich can cause errors in

Instead , IR detectors (such as , for example, infrared focal

either the spectral data , spatial data, or both .

optical channels having different wavelength profiles can besystem can be mobile . For example , the DAISI system cansystem in which most of the data elements that are collected 35 example , in various implementations, the imaging systems

the full dataset is collected . Relatively fast operation can be 40 of operation during day and night, and are less expensive .
achieved in a snapshot system because it does not need toSome embodiments described herein can also be less sus
plane arrays or FPAs) associated with a plurality of different 45

In various embodiments disclosed herein , the DAISI

used to form a spectral cube of imaging data . Although
spectral data can be obtained from a single snapshot com -be configured to be worn or carried by a person , e . g ., the
DAISI system can be miniaturized to fit in a relatively small

prising multiple simultaneously acquired images corre -housing or compartment. For example , the components ofsponding to different wavelength ranges, in various embodi- 50 the DAISI system can be sized and shaped to fit within small

ments, multiple snap shots may be obtained . In various
embodiments, these multiple snapshots can be averaged
Similarly, in certain embodiments multiple snap shots may

dimensions and can have a mass sufficiently small to enable
the human user to carry or wear the system without undueexertion . As explained herein , in some embodiments, thebe obtained and a portion of these can be selected and

DAISI system can be sized and shaped to fit within a volume

possibly averaged . Also , in contrast to commonly used IR 55 of less than about 300 cubic inches , or in some embodi

spectral imaging systems, the DAISI system does not

require cooling. Accordingly , it can advantageously use

uncooled infrared detectors. For example , in various imple

mentations , the imaging systems disclosed herein do notments , less than about 200 cubic inches . In still otherembodiments, the DAISI system can be sized and shaped to

fit within a volume less than about 100 cubic inches. For

example , in some arrangements , the DAISI system can be

include detectors configured to be cooled to a temperature 60 sized and shaped to fit within a volume in a range of about

below 300 Kelvin . As another example , in various imple -mentations, the imaging systems disclosed herein do not

50 cubic inches to about 300 cubic inches . In other arrange

ments, the DAISI system can be sized and shaped to fit

include detectors configured to be cooled to a temperature within a volume in a range of about 80 cubic inches to about
200 cubic inches .
implementations , the imaging systems disclosed herein do 65 Advantageously, such a portable and /or wearable DAISI

below 273 Kelvin . As yet another example , in various

not include detectors configured to be cooled to a tempera -ture below 250 Kelvin . As another example, in various

system can enable the user to monitor installations in remotelocations and to detect the presence of various gases (e.g.,

US 9 ,756 ,263 B2
poisonous gases) in real-time. Further, the portable DAISI
system can enable the user to travel to different installations
to monitor the presence of gases or chemicals in multiple
locations. For example , the user may travel to an oil drilling

aperture mask ( comprising , for example , IR light-blocking
materials such as ceramic ,metal, or plastic ). Light from theobject 110 ( for example a cloud of gas), the optical prop
erties of which in the IR are described by a unique absorp

installation in which oil is pumped from the ground . The 5 tion , reflection and /or emission spectrum , is received by theuser can carry or attach the portable DAISI system to his or

aperture of the system 100 . This light propagates throughher clothing or body ( e.g .,by way of a clip , hat, etc .) and caneach of the plurality of optical channels 120 and is further

activate the system while he or she is on -site . Opticalcomponents on board the portable DAISI system can capture

imaged onto the optical detector array 136 . In various
implementations, the detector array 136 can include at least

one or more snapshotmultispectral images of portions ofthe 10 one FPA . In various embodiments , each of the re - imaging

installation susceptible to gas or chemical leaks . Computing

lenses 128a can be spatially aligned with a respectively

captured multispectral image data to detect and / or classify
gases or chemicals present at the site . A communications

tation , each filter element from the array of spectral filters
130 corresponds to a different spectral region . Each reunits on board the portable DAISI system can process thecorresponding spectral region . In the illustrated implemen

module can notify the user of the detected gases. For 15 imaging lens 128a and the corresponding filter element of
example , in various embodiments, the communications the array of spectral filter 130 can coincide with (or form ) a
module can send a notification to a user interface ( such as a portion of the divided aperture and therefore with respec
set of computing eyeglasses, a mobile computing device
such as a mobile smartphone , a tablet computing device , atively - corresponding spatial channel 120 . Accordingly , invarious embodiment an imaging lens 128a and a corre

and the user interface can display information about thedetected gases to the user in real-time, e. g ., at the oil drilling
installation .

one of the plurality of optical channels 120. Radiation fromthe object 110 propagating through each of the plurality of

136 can be disposed at the focus of each of the lenses in thenent) 136 is further operably connected with a processor 150

laptop computing device , or any other suitable interface ), 20 sponding spectral filter can be disposed in the optical path of

optical channels 120 travels along the optical path of eachre - imaging lens 128a and the corresponding filter elementof
II. Examples of Divided Aperture Infrared Spectral 25 the array of spectral filter 130 and is incident on the detector
Imager Systems
array ( e. g ., FPA component) 136 to form a single image(e.g ., sub -image ) of the object 110 . The image formed by the
FIG . 1 provides a diagram schematically illustrating spa
detector array 136 generally includes a plurality of subtial and spectral division of incoming light by an embodiimages formed by each of the optical channels 120 . Each ofment 100 of a divided aperture infrared spectral imager 30 the plurality of sub - images can provide different spatial and(DAISI) system that can image an object 110 possessing IR
spectral information of the object 110 . The different spatial
spectral signature (s ). The system 100 includes a front objec - information results from some parallax because of thetive lens 124 , an array of optical filters 130, an array ofdifferent spatial locations of the smaller apertures of thereimaging lenses 128 and a detector array 136 . In various divided aperture . In various embodiments, adjacent sub
embodiments , the detector array 136 can include a single 35 images can be characterized by close or substantially equal
FPA or an array of FPAs. Each detector in the detector array spectral signatures. The detector array (e.g ., FPA compo
array of reimaging lenses 128 . In various embodiments , the(not shown ) . The processor 150 can be programmed todetector array 136 can include a plurality of photo - sensitive

aggregate the data acquired with the system 100 into a

devices . In some embodiments , the plurality of photo - 40 spectral data cube . The data cube represents , in spatial ( x , y )sensitive devices may comprise a two - dimensional imaging

and spectral (a ) coordinates , an overall spectral image of thesensor array that is sensitive to radiation having wavelengths
between 1 um and 20 um ( for example , in near infra - red

object 110 within the spectral region defined by the combi
nation of the filter elements in the array of spectral filters

wavelength range , mid infra -red wavelength range , or long

130 . Additionally , in various embodiments, the processor or

infra -red wavelength range ). In various embodiments , the 45 processing electronics 150 may be programmed to deter

plurality of photo -sensitive devices can include CCD or
CMOS sensors , bolometers, microbolometers or otherdetectors that are sensitive to infra -red radiation .
An aperture of the system 100 associated with the front

mine the unique absorption characteristic of the object 110 .
Also , the processor /processing electronics 150 can , alterna

tively or in addition , map the overall image data cube intoa cube of data representing, for example , spatial distribution

objective lens system 124 is spatially and spectrally divided 50 of concentrations, C , of targeted chemical components

by the combination of the array of optical filters 130 and thearray of reimaging lenses 128 . In various embodiments , thecombination of the array of optical filters 130 and the arrayof reimaging lenses 128 can be considered to form a

within the field of view associated with the object 110 .

Various implementations of the embodiment 100 canence source 160 including , for example, a shutter system

include an optional moveable temperature - controlled refer

spectrally divided pupil that is disposed forward of the 55 comprising one or more reference shutters maintained at

optical detector array 136 . The spatial and spectral divisiondifferent temperatures . The reference source 160 can includeof the aperture into distinct aperture portions forms a plu rality of optical channels 120 along which light propagates .
In various embodiments, the array 128 of re -imaging lenses

a heater, a cooler or a temperature -controlled element con
temperature . For example , in various implementations, thefigured to maintain the reference source 160 at a desired

128a and the array of spectral filters 130 which respectively 60 embodiment 100 can include two reference shutters maincorrespond to the distinct optical channels 120. The plurality tained at different temperatures. The reference source 160 isof optical channels 120 can be spatially and /or spectrally
removably and , in one implementation , periodically inserted
distinct. The plurality of optical channels 120 can be formed into an optical path of light traversing the system 100 fromin the object space and /or image space . In one implemen - the object 110 to the detector array ( e . g ., FPA component )tation , the distinct channels 120 may include optical chan - 65 136 along at least one of the channels 120 . The removablenels that are separated angularly in space. The array of reference source 160 thus can block such optical path .
spectral filters 130 may additionally include a filter-holding Moreover, this reference source 160 can provide a referenceUS 9 ,756 ,263 B2
10
IR spectrum to recalibrate various components including theembodiment 200 can be greatly reduced because the objec

detector array 136 of the system 100 in real time. The tive lens is smaller. Each pair of the lenses in the array 224
configuration of the moveable reference source 160 is furand the array 128 is associated with a field of view (FOV ) .
ther discussed below .
Each pair of lenses in the array 224 and the array 128
In the embodiment 100 , the front objective lens system 5 receives light from the object from a different angle . Accord
124 is shown to include a single front objective lens posiingly , the FOV of the different pairs of lenses in the arraytioned to establish a common field - of -view ( FOV ) for the

224 and the array 128 do not completely overlap as a result

apertures corresponding to different optical channels 120 . As

mately the same, thereby reducing its effect on the systema result , the positions for spectral filters of the different

200 . When the ratio of the parallax -to -object-distance is

optical channels 120 coincide with the position of thesubstantially equal to the pixel- size - to -system - focal- length

reimaging lenses 128a and to define an aperture stop for the of parallax . As the distance between the imaging system 200
whole system . In this specific case , the aperture stop sub (portion 202 ) and the object 110 increases , the overlapping
stantially spatially coincides with and/ or is about the same 10 region 230 between the FOVs of the individual lenses 224
size as or slightly larger than the plurality of smaller limiting increases while the amount of parallax 228 remains approxi

aperture stop of the whole system , which in this example is 15 ratio then the parallax effect may be considered to beshown as a surface between the lens system 124 and thenegligible and , for practical purposes, no longer distinguisharray 128 of the reimaging lenses 128a . In various imple mentations, the lens system 124 can be an objective lens
124 . However, the objective lens 124 is optional and various

able. While the lenses 224 are shown to be disposed sub
stantially in the same plane , optionally different objective

lenses in the array of front objective lenses 224 can beembodiments of the system 100 need not include the objec - 20 disposed in more than one plane . For example, some of thetive lens 124 . In various embodiments, the objective lens
124 can slightly shift the images obtained by the different

detectors in the array 136 spatially along a direction per pendicular to optical axis of the lens 124 , thus the function

individual lenses 224 can be displaced with respect to someother individual lenses 224 along the axis 226 ( not shown )and /or have different focal lengths as compared to someother lenses 224 . As discussed below , the field reference 204

ality of the system 100 is not necessarily compromised when 25 can be useful in calibrating the multiple detectors 236 .

the objective lens 124 is not included . Generally, however,the field apertures corresponding to different optical chan -In one implementation , the front objective lens systemsuch as the array of lenses 224 is configured as an array ofnels may be located in the same or different planes . These

lenses integrated or molded in association with a monolithic

field apertures may be defined by the aperture of the reim -substrate . Such an arrangement can reduce the costs andaging lens 128a and / or filters in the divided aperture 130 in 30 complexity otherwise accompanying the optical adjustment

certain implementations. In one implementation , the fieldof individual lenses within the system . An individual lens

apertures corresponding to different optical channels can belocated in different planes and the different planes can be
optical conjugates of one another. Similarly , while all of the224 can optionally include a lens with varying magnifica
tion . As one example , a pair of thin and large diameter
Alvarez plates can be used in at least a portion of the front

embodiment 100 are shown to lie in one plane , generally
different filter elements of the array of spectral filter 130 can

Alvarez plates can produce a change in focal length whentranslated orthogonally with respect to the optical beam .

filter elements in the array of spectral filters 130 of the 35 objective lens system . Without any loss of generality , thebe disposed in different planes. For example , different filter
elements of the array of spectral filters 130 can be disposed

In further reference to FIG . 1, the detector array 136 ( e.g .,FPA component) configured to receive the optical data

in different planes that are optically conjugate to one 40 representing spectral signature (s ) of the imaged object 110
another. However, in other embodiments , the different filter can be configured as a single imaging array ( e .g ., FPA ) 136 .
elements can be disposed in non - conjugate planes .

This single array may be adapted to acquire more than one

In contrast to the embodiment 100, the front objective lensimage ( formed by more than one optical channel 120 )124 need not be a single optical element, but instead cansimultaneously . Alternatively , the detector array 136 may
include a plurality of lenses 224 as shown in an embodiment 45 include a FPA unit. In various implementations, the FPA unit

200 of the DAISI imaging system in FIG . 2. These lenses

can include a plurality of optical FPAs. At least one of these

224 are configured to divide an incoming optical wavefront

plurality of FPAs can be configured to acquire more than one

tive lenses 224 can be disposed so as to receive an IR

as shown in the embodiment 200 of FIG . 2 , in various

from the object 110 . For example , the array of front objec -spectrally distinct image of the imaged object. For example ,wavefront emitted by the object that is directed toward the 50 embodiments , the number of FPAs included in the FPA unit
DAISI system . The plurality of front objective lenses 224 may correspond to the number of the front objective lenses

divide thewavefrontspatially into non -overlapping sections .
FIG . 2 shows three objective lenses 224 in a front optical

portion of the optical system contributing to the spatial

224 . In the embodiment 200 of FIG . 2 , for example , three
FPAS 236 are provided corresponding to the three objective

lenses 224 . In one implementation of the system , the FPA

division of the aperture of the system in this example . The 55 unit can include an array of microbolometers . The use ofplurality of objective lenses 224 , however, can be configured

multiple microbolometers advantageously allows for an

as a two -dimensional (2D ) array of lenses . FIG . 2 presents
inexpensive way to increase the total number of detection
a general view of the imaging system 200 and the resultant elements (i.e . pixels ) for recording of the three -dimensional
field of view of the imaging system 200 . An exploded view
data cube in a single acquisition event (i.e . one snapshot ). In
202 of the imaging system 200 is also depicted in greater 60 various embodiments, an array of microbolometers moredetail in a figure inset of FIG . 2 . As illustrated in the detailed

efficiently utilizes the detector pixels of the array of FPAS

view 202, the embodiment of the imaging system 200(e . g ., each FPA ) as the number of unused pixels is reduced ,includes a field reference 204 at the front end of the system .

minimized and /or eliminated between the images that may

tional advantage over embodiment 100 of FIG . 1 in that thethe imaging system in which the number of the front

overall size and /or weight and/ or cost of manufacture of theobjective lenses 324a in the lens array 324 , the number of

The field reference 204 can be used to truncate the field of exist when using a single microbolometer.
view . The configuration illustrated in FIG . 2 has an opera - 65 FIG . 3A illustrates schematically an embodiment 300 of

US 9 ,756 ,263 B2
re - imaging lenses 128a in the lens array 128 , and the number
of FPAs 336 are the same. So configured , each combinationof respectively corresponding front objective lens 324, re imaging lens 128a , and FPAs 336 constitutes an individual

shadow on ) a peripheral region of a corresponding image(e.g ., sub -image ) formed at the detector plane 444 througha respectively corresponding spatial imaging channel 450 of
the system 400 prior to such image being spectrally pro

an individual filter element of the array of optical filters 130 .

through different spectral filters 454a of the filter array 454 ,a uniform temperature across its surface and be character

controlled shutters 460a , 460b , and then onto the detector

imaging channel. Such a channel is associated with acqui- 5 cessed by the processor 150 . The array 440 of relay lenses
sition of the IR light transmitted from the object 110 throughthen transmits light along each of the imaging channels 450

A field reference 338 of the system 300 is configured to havepast the calibration apparatus that includes two temperatureized by a predetermined spectral curve of radiation emanat- 10 module 456 . In various embodiments , the detector module

ing therefrom . In various implementations, the field refer

456 can include a microbolometer array or some other IR

ence 338 can be used as a calibration target to assist in
calibrating or maintaining calibration of the FPA . Accord -FPA .
The embodiment 400 has several operational advantages.

ingly, in various implementations , the field reference 338 is
FPA 336 after acquisition of light from the object 110 . This

It is configured to provide a spectrally known object withinsition which can be calibrated against. Such spectral cer

optical components ( e . g ., objective lenses 324a, filter ele -(approximately corresponding to the depth of focus) assoused for dynamically adjusting the data output from each 15 every image ( e . g ., sub - image ) and for every snapshot acqui

dynamic calibration process helps provide that output of the tainty can be advantageous when using an array of IR FPAS
different (e.g., most , or each of the) FPA 336 represents like microbolometers, the detection characteristics ofwhichcorrect acquired data , with respect to the other FPAs 336 for can change from one imaging frame to the next due to , in
20 part, changes in the scene being imaged as well as theanalysis, as discussed below in more detail.
FIG . 3B illustrates the plan view perpendicular to the axis thermal effects caused by neighboring FPAs. In various
226 of an embodiment 300 of the imaging system illustrated
embodiments , the field reference array 438 of the embodi
in FIG . 3A . For the embodiment shown in FIG . 3B , the ment 400 can be disposed within the Rayleigh range
ments of the array of spectral filters 130 , re - imaging lenses 25 ciated with the front objective lenses 424 , thereby removing
128a and FPA units 336 ) are arranged as a 4x3 array. In one
unusable blurred pixels due to having the field reference

implementation , the 4x3 array 340 of optical components( lenses 324a , 128a ; detector elements 336 ) is used behind
the temperature controlled reference target 160 . The fieldoutside of this range . Additionally , the embodiment 400 of

FIG . 4 can be more compact than , for example , the con

figuration 300 of FIG . 3A . In the system shown in FIG . 3A ,reference aperture 338 can be adapted to obscure and/ or 30 for example , the field reference 338 may be separated fromblock a peripheral portion of the bundle of light propagating the lens array 324 by a distance greater than several (forfrom the object 110 towards the FPA units 336 . As a result, example , five ) focal lengths to minimize /reduce blur con

the field reference 338 obscures and /or blocks the border or tributed by the field reference to an image formed at aperipheral portion ( s) of the images of the object 110 formed detector plane .
on the FPA elements located along the perimeter 346 of the 35 In various embodiments , the multi -optical FPA unit of thedetector system . Generally , two elements of the FPA unit
IR imaging system can additionally include an FPA config
will produce substantially equal values of digital counts u red to operate in a visible portion of the spectrum . In
when they are used to observe the same portion of the scene reference to FIG . 1, for example , an image of the scene ofin the same spectral region using the same optical train . If interest formed by such visible -light FPA may be used as aany of these input parameters (for example , scene to be 40 background to form a composite image by overlapping an IR
observed , spectral content of light from the scene , or optical image with the visible -light image. The IR image may beelements delivering light from the scene to the two detector

elements ) differ, the counts associated with the elements ofoverlapped virtually, with the use of a processor and spe
cifically - designed computer program product enabling such

the FPA unit will differ as well. Accordingly , and as andata processing, or actually , by a viewer . The IR image may
example , in a case when the two FPAs of the FPA unit 336 45 be created based on the image data acquired by the indi

( such as those denoted as # 6 and # 7 in FIG . 3B ) remain

substantially un -obscured by the field reference 338 , thevidual FPAs 136 . The so - formed composite image facilitates

the identification of the precise spatial location of the target

outputs from these FPAs can be dynamically adjusted to the species , the spectral signatures of which the system is able
output from one of the FPAs located along perimeter 346
to detect/recognize .
( such as, for example , the FPA element # 2 or FPA element 50 Optical Filters .

# 11 ) that processes light having similar spectral character
istics .

The optical filters, used with an embodiment of thesystem , that define spectrally -distinct IR image ( e. g., sub

FIG . 4 illustrates schematically a portion of anotherimage) of the object can employ absorption filters , interfer

the received light to the array 128 of re- imaging lenses 128a
through an array 438 of field references (or field stops) 438a ,by an individual re - imaging lens (such as a lens 128a of
FIGS. 1, 2 , 3 , and 4 )may be carried out in a single spectral

embodiment of an imaging system 400 that contains an arraye nce filters, and Fabry -Perot etalon based filters, to name424 of front objective lenses 424a . The array 424 of lenses 55 just a few . When interference filters are used , the image
424a adapted to receive light from the object 110 and relay acquisition through an individual imaging channel defined
and through an array 440 of the relay lenses . The spectral bandwidth or multiple spectral bandwidths . Referring again
characteristics of the field references/ field stops 438a can be 60 to the embodiments 100 , 200 , 300, 400 of FIGS. 1 throughknown . The field references 438a are disposed at corre sponding intermediate image planes defined , with respect to

4 , and in further reference to FIG . 3B , examples of a 4 -by- 3
array of spectral filters 130 is shown in FIGS . 5A and 5B .
the object 110 ,by respectively corresponding front objective Individual filters 1 through 12 are juxtaposed with a suplenses 424a . When refractive characteristics of all of the
porting opto -mechanical element (not shown ) to define afront objective lenses 424a are substantially the same, all of 65 filter-array plane that is oriented , in operation , substantiallythe field references 438a are disposed in the same plane . A
perpendicularly to the general optical axis 226 of the imag
field reference 438a of the array 438 obscures (or casts a ing system . In various implementations, the individual filtersUS 9 ,756 ,263 B2
13

14

1 through 12 need not be discrete optical components.

increase or maximize the spectral extent of the transmission

Instead, the individual filters 1 through 12 can comprise one

region of the filter system of the embodiment.

or more coatings that are applied to one or more surfaces of

The advantage of using spectrally multiplexed filters is

the reimaging lenses (such as a lens 128a of FIGS. 1, 2 , 3 , appreciated based on the following derivation , in which a
5 system of M filters is examined (although it is understoodand 4 ) or the surfaces of one or more detectors .
The optical filtering configuration of various embodi that in practice an embodiment of the invention can employ
ments disclosed herein may advantageously use a bandpass any number of filters). As an illustrative example , the caseM = 7 is considered . Analysis presented below relates tofilter defining a specified spectralband . Any of the filters 0a of
one spatial location in each of the images ( e. g ., sub -images )through 3a , the transmission curves of which are shown informed by the differing imaging channels (e .g., different
FIG . 6A may, for example , be used . The filters may be 10 opticalchannels 120 ) in the system . A similar analysis canplaced in front of the optical FPA (or generally , between the be performed
for each point at an image (e .g ., sub- image ),optical FPA and the object). In particular, and in further
and
thus
the
analysis can be appropriately extended as
reference to FIGS . 1 , 23, and 4 , when optical detector arrays
required .
136 , 236 , 336 , 456 include microbolometers, the predomi- 15. The unknown amount of light within each of the M
nant contribution to noise associated with image acquisition
spectral channels (corresponding to these M filters ) is
is due to detector noise . To compensate and /or reduce the denoted with ffff
and readings from corre
noise , various embodiments disclosed herein utilize spec sisponding detector elements receiving light transmitted bytrally -multiplexed filters. In various implementations, the each filter is denoted as 81, 82, 83 . . . 8m , whilemeasurement
spectrally -multiplexed filters can comprise a plurality of 20 errors are represented by ni, ng, nz , . . . ny . Then , thelong pass filters, a plurality long pass filters, a plurality of readings at the seven FPA pixels each of which is opticallyband pass filters and any combinations thereof. An example filtered by a corresponding band-pass filter of FIG . 6A canof the spectral transmission characteristics of spectrally
be represented by :multiplexed filters Ob through 3d for use with various

embodiments of imaging systems disclosed herein is 25

81 = f1 + 1,

long-wavelength pass , LP filters . An LP filter generally

depicted in FIG . 6B . Filters of FIG . 6C can be referred to as

82 = 42 + n2,

attenuates shorter wavelengths and transmits ( passes ) longer

83 = f3+ N3,

wavelengths (e . g ., over the active range of the target IR
portion of the spectrum ) . In various embodiments , short - 30

wavelength -pass filters , SP, may also be used . An SP filter
generally attenuates longer wavelengths and transmits(passes ) shorter wavelengths ( e .g ., over the active range ofthe target IR portion of the spectrum ). At least in part due to85 = fs +N5,8646 +N62

the snap - shot/non - scanning mode of operation , embodi- 35

ments of the imaging system described herein can use lesssensitive microbolometers without compromising the SNR .

The use of microbolometers , as detector-noise - limited
devices, in turn not only benefits from the use of spectrally

84 = f4 +n4,

87 = f4 +17,

These readings (pixel measurements) g ; are estimates ofthe spectral intensities f;. The estimates g ; are not equal tothe corresponding f; values because of the measurement

multiplexed filters, but also does not require cooling of the 40 errors n ;.However, if the measurement noise distribution has
zero mean , then the ensemble mean of each individual
imaging system during normal operation .

Referring again to FIGS. 6A , 6B , 6C , and 6D , each of thefilters(0b . . . 3d ) transmits light in a substantially wider
region of the electromagnetic spectrum as compared to thosemeasurement can be considered to be equal to the true value,icalculating.e. ( g:) = f;. theHereensemble, the anglemeanbrackets
indicate the operation ofof a stochastic variable . Theof the filters ( 0a . . . 3a ). Accordingly , when the spectrally - 45 variance of the measurement can , therefore , be represented
multiplexed set of filters (0b . . . 0d ) is used with anas :embodiment of the imaging system , the overall amount oflight received by the FPAs ( for example , 236 , 336 ) islarger than would be received when using the bandpass(18 - 832) ={n })=02
In embodiments utilizing spectrally -multiplexed filters, infilters (0a . . . 4a ). This added ” transmission of light defined 50 comparison with the embodiments utilizing band - pass fil

by the use of the spectrally -multiplexed LP ( or SP ) filters

ters , the amount of radiant energy transmitted by each of the

facilitates an increase of the signal on the FPAs above thespectrally -multiplexed LP or SP filters towards a given

level of the detector noise. Additionally , by using, in an detector element can exceed that transmitted through aembodiment of the imaging system , filters having spectral spectral band of a band -pass filter . In this case, the intensities
bandwidths greater than those of band - pass filters , the 55 of light corresponding to the independent spectral bands canuncooled FPAs of the embodiment of the imaging system be reconstructed by computational means. Such embodi
experience less heating from radiation incident thereon fromments can be referred to as a “ multiplex design ” .
the imaged scene and from radiation emanating from the
One matrix of such “ multiplexed filter” measurements

FPA in question itself . This reduced heating is due to a
includes a Hadamard matrix requiring “ negative” filters that
reduction in the back - reflected thermal emission (s ) coming 60 may not be necessarily appropriate for the optical embodi

from the FPA and reflecting off of the filter from thenon -band - pass regions. As the transmission region of themultiplexed LP (or SP ) filters is wider , such parasitic effects
are reduced thereby improving the overall performance of
the FPA unit.

In one implementation , the LP and SP filters can be

combined , in a spectrally -multiplexed fashion , in order toments disclosed herein . An S -matrix approach (which is
restricted to having a number of filters equal to an integerthat is multiple of four minus one ) or a row -doubled Had
amard matrix (requiring a number of filters to be equal to an
65 integer multiple of eight) can be used in various embodi
ments. Here , possible numbers of filters using an S -matrix

setup are 3, 7 , 11 , etc and, if a row -doubled HadamardUS 9 ,756 , 263 B2
15
matrix setup is used , then the possible number of filters is 8 ,systems employing uncooled FPAs (such as microbolometa

be to measure seven spectral band f; intensities using sevenlong -wavelength pass (LP ) filters used in the system . An LP

16 , 24 , etc . For example , the goal of the measurement may

measurements g ; as follows :g1 = f1+ 0 +f3 + 0 + f3 + 0 + f7 +n?

82= 0+ f2+ f3 +0 +0 + f6 +f7+n2
83=fi +f2+ 0 +0 +f3+ 0 + f7+ n3
84 + 0 + 0 +0 +f4 +fs+ f+ +fg+ n4
85 = f1+ 0 + f3+ f4 +0 +f6+ 0 +ng

16
eters ). FIG . 6C illustrates a set of spectrally -multiplexedfilter generally attenuates shorter wavelengths and transmits(passes ) longer wavelengths ( e . g ., over the active range ofthe target IR portion of the spectrum ). A single spectral

channel having a transmission characteristic corresponding

to the difference between the spectral transmission curves ofat least two of these LP filters can be used to procure
10 imaging data for the data cube using an embodiment of thesystem described herein . In various implementations, thespectral filters disposed with respect to the different FPAS

can have different spectral characteristics . In various imple
mentations, the spectral filters may be disposed in front of

15 only some of the FPAS while the remaining FPAs may be
86= 0 + f2 + f3 + f4 + fs + 0 +0 +16
configured to receive unfiltered light . For example , in some
G =fi+ f2 + 0 + f4 + 0 + 0 + f7+n7
implementations, only 9 of the 12 detectors in the 4x3 array
Optical transmission characteristics of the filters of detectors described above may be associated with adescribed above are depicted in FIG . 6B . Here , a direct spectral filter while the other 3 detectors may be configured
estimate of the f; is no longer provided through a relationship 20 to received unfiltered light. Such a system may be config
similar to ( g :) = f;. Instead , if a “ hat ” notation is used to ured to acquire spectral data in 10 different spectral channels

denote an estimate of a given value, then a linear combina
tion of the measurements can be used such as, for example ,in a single data acquisition event.

The use of microbolometers, as detector-noise - limited

devices, in turn not only can benefit from the use of
25 spectrally multiplexed filters , but also does not requirecooling of the imaging system during normal operation. In

i = 7 ( + 81 – 82 + 83 - 84 + 85 – 86 + 87 ),

.

7

IL

1
+

+ 82 + 83 - 84 - 85 + 86 + 87 ),? ( + 81 + 82 - 83 - 84 + 95 + 86 - 87),=+

81 – 82 – 83 + 84 + 95 + 86 + 87 ),1 - 82 + 83 + 84 - 85 + 86 – 87 ) ,+=1 + 82 + 83 + 84 + 85 - 86 - 87 ) ,( + 81 + 82 – 83 + 84 - 85 - 86 + 87 ) ,These f; are unbiased estimates when the n ; are zero meancontrast to imaging systems that include highly sensitive
FPA units with reduced noise characteristics , the embodi
ments of imaging systems described herein can employ less
30 sensitive microbolometers without compromising the SNR .
This result is at least in part due to the snap - shot/nonscanning mode of operation .
As discussed above , an embodiment may optionally , andin addition to a temperature -controlled reference unit (for35 example temperature controlled shutters such as shutters160, 460a , 460b ), employ a field reference component ( e. g .,field reference aperture 338 in FIG . 3A ), or an array of fieldreference components ( e . g ., filed reference apertures 438 in

FIG . 4 ), to enable dynamic calibration . Such dynamic cali
40 bration can be used for spectral acquisition of one or moreor every data cube. Such dynamic calibration can also be
used for a spectrally -neutral camera - to -camera combination

to enable dynamic compensation of parallax artifacts . The

stochastic variables , so that ( f4 -f;) = 0. The measurement use of the temperature -controlled reference unit ( forvariance corresponding to ith measurement is given by the 45

variance corresponding to 1 measurement is given by the 45 example , temperature -controlled shutter system 160 ) andequation below :field -reference component(s ) facilitates maintenance ofproper calibration of each of the FPAs individually and the((9:- ))= IDO?

entire FPA unit as a whole .
In particular, and in further reference to FIGS. 1 , 2 , 3 , and
50 4 , the temperature - controlled unit generally employs asystem having first and second temperature zones main

From the above equation , it is observed that by employing

tained at first and second different temperatures. For( SNR ) of a measurement is improved by a factor of

200 , 300 and 400 can employ not one but at least two

spectrally -multiplexed system the signal -to -noise ratio

example , shutter system of each of the embodiments 100 ,55 temperature - controlled shutters that are substantially paral
V16 /7 = 1.51 .
For N channels, the SNR improvement achieved with alel to one another and transverse to the general optical axis
spectrally -multiplexed system can be expressed as (N + 1)/( 2 226 of the embodiment(s ) 100 , 200 , 300 , 400 . Two shutters
VN ). For example , an embodiment employing 12 spectral at two different temperatures may be employed to provide
channels (N = 12 ) is characterized by a SNR improvement, more information for calibration ; for example , the absolute

over a non -spectrally -multiplexed system , comprising a 60 value of the difference between FPAs at one temperature aswell as the change in that difference with temperaturefactor of up to 1.88 .
Two additional examples of related spectrally -multichange can be recorded . Referring, for example , to FIG . 4 ,plexed filter arrangements Oc through 3c and Od through 3d in which such multi-shutter structure is shown, the use ofmultiple shutters enables the user to create a known refersystems described herein are shown in FIGS . 6C and 6D , 65 ence temperature difference perceived by the FPAs 456 . Thisthat can be used in various embodiments of the imaging

respectively . The spectrally -multiplexed filters shown in
FIGS. 6C and 6D can be used in embodiments of imaging

reference temperature difference is provided by the IRradiation emitted by the shutter (s ) 460a , 460b when theseUS 9, 756 ,263 B2
17

18
shutters are positioned to block the radiation from the object m ent 400 of FIG . 4 , the field -reference aperture can be
110 . As a result, not only the offset values corresponding toplaced within the depth of focus of an image conjugate planeeach of the individual FPAs pixels can be adjusted but also formed by the front objective lens 424 . The field reference ,the gain values of these FPAs . In an alternative embodiment, generally, can facilitate , effectuates and/or enable dynamic
the system having first and second temperature zones may 5 compensation in the system by providing a spectrally known
include a single or multi -portion piece . This single ora nd temporally - stable object within every scene to reference

multi-portion piece may comprise for example a plate . This
piece may be mechanically -movable across the optical axis

and stabilize the output from the different FPAs in the array .
Because each FPA 's offset value is generally adjustedwith the use of appropriate guides and having a first portion

from each frame to the next frame by the hardware , com

at a first temperature and a second portion at a second 10 paring the outputs of one FPA with another can have an error

temperature .

that is not compensated for by the static calibration param

Indeed , the process of calibration of an embodiment of theimaging system starts with estimating gain and offset by
performing measurements of radiation emanating , indepen

eters goffset and y established , for example, by the movable
metric agreement over time, it is advantageous for a portion
shutters 160 . In order to ensure that FPAs operate in radio

dently , from at least two temperature - controlled shutters of 15 of each detector array to view a reference source ( such as the

known and different radiances. The gain and offset can varyfrom detector pixel to detector pixel. Specifically , first thefield reference 338 in FIG . 3A , for example ) over a pluralityof frames obtained over time. If the reference source spec

response of the detector unit 456 to radiation emanatingfrom one shutter is carried out. For example , the first shutter

trum is known a priori ( such as a blackbody source at aknown temperature ), one can measure the response of each460a blocks the FOV of the detectors 456 and the tempera - 20 FPA to the reference source in order to estimate changes toture T , is measured directly and independently with therm - the pixel offset value . However, the temperature of theistors. Following such initial measurement, the first shutter reference source need not be known. In such implementa
460a is removed from the optical path of light traversing the tions , dynamic calibration of the different detectors can beembodiment and another second shutter ( for example , 460b )performed by monitoring the change in the gain and theis inserted in its place across the optical axis 226 to prevent 25 offset for the various detectors from the time the movable
the propagation of light through the system . The temperature
shutters used for static calibration are removed . An exampleof the second shutter 460b can be different than the first
shutter ( T + T , ) . The temperature of the second shutter 460b
is also independently measured with thermistors placed in

calculation of the dynamic offset proceeds as follows.
Among the FPA elements in an array of FPAs in an
embodiment of the imaging system , one FPA can be selectedcontact with this shutter, and the detector response to radia - 30 to be the “ reference FPA ” . The field reference temperaturetion emanating from the shutter 460b is also recorded .
measured by all the other FPAs can be adjusted to agree with

Denoting operational response of FPA pixels (expressed in
the readings corresponding to the measurements of the twodigital numbers , or " counts ” ) as g ; to a source of radiance Lig

shutters can be expressed as:

81 = YL1(71)+Eofset

the field reference temperature measured by the reference as
discussed below . The image obtained by each FPA includes
a set of pixels obscured by the field reference 338 . Using the

35 previously obtained calibration parameters goffset and y (thepixel offset and gain ), the effective blackbody temperature T ;of the field reference as measured by each FPA is estimatedusing the equation below :82 =yL2 (T2) + goffset
T;=mean {(g + Agi+8offse/ Y}=mean {(8- 8offset)/Y}+AT;
Here , goffset is the pixel offset value ( in units of counts ), 40
and y is the pixel gain value ( in units of counts per radiance
Using the equation above , the mean value over all pixels
unit ). The solutions of these two equations with respect to that are obscured by the field reference is obtained . In thethe two unknowns goffer and y can be obtained if the values
above equation Ag; is the difference in offset value of theof g , and g2 and the radiance values L , and L2 are available . current frame from Agoffset obtained during the calibration

These values can , for example , be either measured by a 45 step . For the reference FPA , Ag; can be simply set to zero .
reference instrument or calculated from the known tempera Then , using the temperature differences measured by eachtures T , and T , together with the known spectral response of FPA , one obtains
the optical system and FPA . For any subsequent measure
T;- Tref=mean { (g + Agi + goffsery} + AT;-mean {(gment, one can then invert the equation ( s ) above in order toestimate the radiance value of the object from the detector 50

measurement, and this can be done for each pixel in each
FPA within the system .
As already discussed , and in reference to FIGS. 1 through4 , the field -reference apertures may be disposed in an object
to block a particular portion of the IR radiation received
from the object. In various implementations, the fieldreference aperture, the opening of which can be substantiallysimilar in shape to the boundary of the filter array ( forSoffset)/y} =AT;
Once AT ; for each FPA is measured , its value can be
subtracted from each image in order to force operational
agreement between such FPA and the reference FPA . Whilethe calibration procedure has been discussed above in refmethodology of calibration with respect to radiance value
can also be implemented .
Examples ofMethodology of Measurements .
Prior to optical data acquisition using an embodiment of

space or image space of the optical system , and dimensioned 55 erence to calibration of temperature , a procedurally similar

example , and in reference to a filter array of FIGS. 3B , 60 the IR imaging system as described herein , one or more ,5B - e .g ., rectangular ). The field - reference aperture can be most, or potentially all the FPAs of the system can beplaced in front of the objective lens ( 124 , 224 , 324 , 424 ) at
calibrated . For example , greater than 50 % , 60 % , 70 % , 80 %a distance that is at least several times (in one implemen

tation — at least five times ) larger than the focal length of theor 90 % of the FPAs 336 can be initially calibrated . As shownin FIG . 3A , these FPAS 336 may form separate images of thelens such that the field -reference aperture is placed closer to 65 object using light delivered in a corresponding opticalthe object. Placing the field -reference aperture closer to thechannel that may include the combination of the correspondobject can reduce the blurriness of the image . In the embodi-

ing front objective and re - imaging lenses 324 , 128 . The

US 9 ,756 ,263 B2
19

20

calibration procedure can allow formation of individual

as the spectral difference image ) is referred to as the tem

images in equivalent units so that, for example , the readingp oral difference image . In some embodiments where specfrom the FPA pixels can be re -calculated in units of tem -tral differencing is employed , a temporal reference image

perature or radiance units, etc .). Moreover, the calibration

may be formed , for example , by creating a spectral differ

process can also allow the FPAs (e . g ., each of the FPAs) to 5 ence image from the two or more images registered by the
be spatially co -registered with one another so that a given
two or more FPAs at a single instance in time. This spectral
pixel of a particular FPA can be optically re -mapped throughdifference image is then used as a temporal reference image.
the optical system to the same location at the object as the

The temporal reference image can then be subtracted fromcorresponding pixel of another FPA .

other later obtained images to provide normalization that can

To achieve at least some of these goals , a spectral differ - 10 be useful in subtracting out or removing various errors or
encing method may be employed . The method involves
deleterious effects . For example , the result of the algorithm
forming a difference image from various combinations of is not affected by a prior knowledge of whether the object orthe images from different channels . In particular, the images
scene contains a target species (such as gas of interest ),used to form difference images can be registered by two or because the algorithm can highlight changes in the scene
more different FPAs in spectrally distinct channels having 15 characteristics . Thus , a spectral difference image can bedifferent spectral filters with different spectral characteris calculated from multiple spectral channels as discussed

tics. Images from different channels having different spectral

above based on a snap -shot image acquisition at any later
characteristics will provide different spectral information. time and can be subtracted from the temporal reference
Comparing ( e . g ., subtracting ) these images , can therefore
image to form a temporal difference image . This temporalyield valuable spectral based information . For example , if 20 difference image is thus a normalized difference image . The
the filter element of the array of spectral filters 130 corre - difference between the two images (the temporal difference
sponding to a particular FPA 336 transmits light from the image ) can highlight the target species (gas ) within theobject 110 including a cloud of gas , for example, with a normalized difference image , since this species was notcertain spectrum that contains the gas absorption peak or apresent in the temporal reference frame. In various embodi

gas emission peak while another filter element of the array 25 ments , more than two FPAs can be used both for registeringof spectral filters 130 corresponding to another FPA 336
the temporal reference image and a later- acquired difference

does not transmit such spectrum , then the difference between

the images formed by the two FPAs at issue will highlightthe presence of gas in the difference image.

image to obtain a better SNR figure of merit . For example ,if two FPAs are associated with spectral filters having thesame spectral characteristic , then the images obtained by the

A shortcoming of the spectral differencing method is that 30 two FPAs can be combined after they have been registeredcontributions of some auxiliary features associated withto get a better SNR figure .

imaging (not just the target species such as gas itself) can

While the temporal differencing method can be used toalso be highlighted in and contribute to the difference image .

reduce or eliminate some of the shortcomings of the spectral

Such contributing effects include, to name just a few , par -differencing , it can introduce unwanted problems of its own .

allax - induced imaging of edges of the object , influence of 35 For example , temporal differencing of imaging data is lessmagnification differences between the two or more opticalsensitive to calibration and parallax induced errors than thechannels, and differences in rotational positioning and ori entation between the FPAs. While magnification - related

spectral differencing of imaging data . However , any change
in the imaged scene that is not related to the target species

errors and FPA -rotation -caused errors can be compensatedof interest (such as particular gas , for example ) is highparallax is scene- induced and is not so easily correctable. In

location of the target species triggering, therefore, an error

radiance calibration errors. Specifically, if one FPA registers

ture of the background against which the gas is being

for by increasing the accuracy of the instrument construction 40 lighted in a temporally -differenced image . Thus such change
as well as by post- processing of the acquired imaging,in the imaged scene may be erroneously perceived as aaddition , the spectral differencing method is vulnerable toin detection of target species. For example , if the tempera

radiance of light from a given feature of the object as having 45 detected changes (due to natural cooling down as the day

a temperature of 40° C ., for example, while the data fromanother FPA represents the temperature of the same object

progresses , or increases due to a person or animal or anotherobject passing through the FOV of the IR imaging system ),feature as being 39° C ., then such feature of the object willthen such temperature change produces a signal difference

be enhanced or highlighted in the difference image ( formed
as compared to the measurement taken earlier in time.
at least in part based on the images provided by these two 50 Accordingly , the cause of the scenic temperature change (thecooling object, the person walking, etc .) may appear as theFPAs) due to such radiance -calibration error.
One solution to some of such problems is to compare

detected target species ( such as gas ). It follows, therefore ,( e. g ., subtract ) images from the same FPA obtained at
different instances in time. For example , images can bethat an attempt to compensate for operational differencesamong the individual FPAs of a multi- FPA IR imaging

compared to or subtracted from a reference image obtained 55 system with the use of methods that turn on spectral orat another time. Such reference image , which is subtracted
temporal differencing can cause additional problems leadingfrom other later obtained images , may be referred to as a
to false detection of target species . Among these problems
temporal reference image . This solution can be applied toare scene-motion -induced detection errors and parallax

spectral difference images as well. For example , the image

caused errors that are not readily correctable and /or com

data resulting from spectral difference images can be nor- 60 pensatable . Accordingly , there is a need to compensate formalized by the data corresponding to a temporal reference
image data acquisition and processing errors caused byimage . For instance , the temporal reference images can bemotion of elements within the scene being imaged . Various

subtracted from the spectral difference image to obtain theembodiments of data processing algorithms described herein

temporal difference image . This process is referred to , foraddress and fulfill the need to compensate for such motion

the purposes of this disclosure , as a temporal differencing 65 induced and parallax - induced image detection errors .
algorithm or method and the resultant image from subtract -In particular, to reduce or minimize parallax - induced

ing the temporal reference image from another image (suchdifferences between the images produced with two or moreUS 9 ,756 ,263 B2
21

22

predetermined FPAs , another difference image can be usedthat is formed from the images of at least two different FPAS
to estimate parallax effects. Parallax error can be determinedby comparing the images from two different FPAs where thephysical size in object space Ax = Ax '/m . To estimate thevolume of the gas cloud , a particular symmetry in the
thickness of the cloud based on the physical size of the cloud
can be assumed . For example, the cloud image can becalculated from the known relative position difference . Differences between the images from these two FPAs can be
attributed to parallax , especially , if the FPA have the same
spectral characteristics, for example have the same spectral

to create a three dimensional volume estimate of the gas
cloud size. It is worth noting that in the embodiments
described herein only a single imaging system is required forsuch volume estimation . Indeed , due to the fact that theposition between the FPAs is known . The parallax can be 5 rotated about a central axis running through the cloud image

filter or both have no spectral filters . Parallax error correc - 10 information about the angle at which the gas cloud is seention, however, can still be obtained from two FPAs that havedifferent spectral characteristics or spectral filters, especiallyby the system is decoded in the parallax effect, the image
data includes the information about the imaged scene

if the different spectral characteristics , e . g ., the transmission

viewed by the system in association with at least two angles.

spectra of the respective filters are known and/or negligible.

When the temporal differencing algorithm is used for

Use of more than two FPAs or FPAs of different locations 15 processing the acquired imaging data , a change in the scene
such as FPAS spaced farther apart can be useful. For
that is not caused by the target species can inadvertently be

example , when the spectral differencing of the image data ishighlighted in the resulting image . In various embodiments ,performed with the use of the difference between the images

compensation for this error makes use of the temporalcollected by the outermost two cameras in the array (such as,

differencing between two FPAs that are substantially equally

the image data acquired by, for example , the alternativethese pixels on both FPAs, which are substantially equallyfor example, the FPAs corresponding to filters 2 and 3 of the 20 spectrally sensitive to the target species . In this case , thearray of filters of FIG . 5A ), a difference image referred to as temporal difference image will highlight those pixels thea “ difference image 2 - 3 ” is formed . In this case , the alter
intensity of which have changed in time (and not in wave
native “ difference image 1 -4 ” is additionally formed fromlength ). Therefore , subtracting the data corresponding to
FPAs corresponding to filters 1 and 4 of FIG . 5A . Assuming 25 spectrally sensitive to the target species, to form the result

or ensuring that both of these two alternative FPAs haveapproximately the same spectral sensitivity to the target
species, the alternative “ difference image 1 - 4 ” will highlightpixels corresponding to parallax - induced features in theing image , excludes the contribution of the target species toin the scene due to the presence of target species and ( ii )the resulting image . The differentiation between (i) changes

changes in the scene caused by changes in the backgroundimage . Accordingly , based on positive determination that the 30 not associated with the target species is , therefore , possible .

samepixels are highlighted in the spectral “ difference image

In some embodiments , these two channels having the same

2 -3 ” used for target species detection , a conclusion can beor substantially similar spectral response so as to be sub

made that the image features corresponding to these pixels

s tantially equally spectrally sensitive to the target species

are likely to be induced by parallax and not the presence of

may comprise FPAs that operate using visible light. It should

target species in the imaged scene. It should be noted that 35 also be noted that, the data acquired with a visible light FPA

compensation of parallax can also be performed using
images created by individual re -imaging lenses, 128a, when

(when present as part of the otherwise IR imaging system )can also be used to facilitate such differentiation and com

using a single FPA or multiple FPA ' s as discussed above.
FPAs spaced apart from each other in different directions canpensation of the motion -caused imaging errors. Visible
cameras generally have much lower noise figure than IRtion . In certain embodiments two central FPAs and one
in certain embodiments , have substantially similar or the

FPA can be used to compensate for motion in the system asalso be useful. Greater than 2 , for example , 3 or 4 , or more 40 cameras (at least during daytime). Consequently, the tem
FPAs can be used to establish parallax for parallax correc - poral difference image obtained with the use of image data
corner FPA are used for parallax correction . These FPA may,from the visible light FPA can be quite accurate. The visible
well as many potential false -alarms in the scene due tosame spectral characteristics , for example , have filters hav - 45 motion caused by people , vehicles , birds, and steam , foring similar or the same transmission spectrum or have nofilter at all.

Another capability of the embodiments described hereinis the ability to perform the volumetric estimation of a gas

example , as long as themoving object can be observed in thevisible region of the spectra . This has the added benefit ofproviding an additional level of false alarm suppression

without reducing the sensitivity of the system since many

cloud . This can be accomplished by using ( instead of 50 targets such as gas clouds cannot be observed in the visible

compensating or negating ) the parallax induced effects
described above . In this case , themeasured parallax between

spectral region . In various implementations , an IR camera
can be used to compensate for motion artifacts .

two or more similar spectral response images ( e . g ., two or

Another method for detection of the gases is to use amore channels or FPAs) can be used to estimate a distance spectral unmixing approach . A spectral unmixing approach
between the imaging system and the gas cloud or between 55 assumes that the spectrum measured at a detector pixel isthe imaging system and an object in the field of view of the composed of a sum of component spectra (e .g .,methane andsystem . The parallax induced transverse image shift, d ,other gases ). This approach attempts to estimate the relative

between two images is related to the distance, z , between theweights of these components needed to derive the measurecloud or object 110 and the imaging system according to thement spectrum . The component spectra are generally taken

equation z = - sZ '/ d . Here , s , is the separation between two 60 from a predetermined spectral library ( for example , fromsimilar spectral response images , and z ' is the distance to thedata collection that has been empirically assembled ) , thoughimage plane from the back lens. The value for z' is typicallyapproximately equal to the focal length f of the lens of thesometimes one can use the scene to estimate these as well(often called “ endmember determination ” ). In various

each image pixel on the gas cloud , Ax ', corresponds to a

cloud in the scene and /or to estimate the concentration of theimaging system . Once the distance z between the cloud andembodiments , the image obtained by the detector pixel is a
the imaging system is calculated , the size of the gas cloud 65 radiance spectrum and provides information about thecan be determined based on the magnification , m = f/z , where brightness of the object. To identify the contents of a gas

US 9 ,756 ,263 B2
23various gases in the gas cloud, an absorption /emission
spectrum of the various gases of interest can be obtained by24

It is worth noting that the use of optically - filtered FPAs invarious embodiments of the system described herein cancomparing the measured brightness with an estimate of the provide a system with higher number of pixels. For example ,expected brightness. The spectral unmixing methodology embodiments including a single large format microbolom
can also benefit from temporal, parallax , and motion com - 5 eter FPA array can provide a system with large number ofpixels . Various embodiments of the systems describedpensation techniques .
can also offer a high optical throughput for a sub
In various embodiments, a method of identifying the hereinstantially low number of optical channels. For example , thepresence of a target species in the object includes obtaining systemsdescribed herein can provide a high optical throughthe radiance spectrum (or the absorption spectrum ) from thefor a number of optical channels between 4 and 50 . By
object in a spectral region indicative of the presence offththe 10 put
havinga lower number of optical channels (e. g ., between 4target species and calculating a correlation ( e.g ., a correla and 50 opticalchannels ), the systems described herein havetion coefficient) by correlating the obtained radiance spec wider
spectralbinswhich allows the signals acquired withintrum (or the absorption spectrum ) with a reference spectrum
each spectral bin to have a greater integrated intensity .
for the target species. The presence or absence
absence ofthe targe
target 1515 An
A advantage of the embodiments described herein overoi me
species can be determined based on an amount of correlation( e. g., a value of correlation coefficient). For example, thevarious scanning based hyperspectral systems that are conpresence of the target species in the object can be confirmed
if the amount of correlation or the value of correlation

detection ) is that, the entire spectrum can be resolved in asnapshotmode ( for example, during one image frameacqui

figured for target species detection (for example, gas cloud

coefficient is greater than a threshold . In various implemen - 20 sition by the FPA array ). This feature enables the embodi
tations, the radiance spectrum (or the absorption spectrum ) ments of the imaging systems described herein to takecan be obtained by obtaining a spectral difference image
advantage of the compensation algorithms such as the
between a filtered optical channel and /or another filteredparallax and motion compensation algorithms mentioned
optical channel/unfiltered optical channel or any combina above . Indeed , as the imaging data required to implement
25 these algorithms are collected simultaneously with the tar
tions thereof.
For example , an embodiment of the system configured toget- species related data , the compensation algorithms aredetect the presence of methane in a gas cloud comprises carried out with respect to target -species related data and not
optical components such that one or more of the plurality ofwith respect to data acquired at another time interval. This

optical channels is configured to collect IR radiation to

rapid data collection thus improves the accuracy of the data

provide spectral data corresponding to a discrete spectral 30 compensation process . In addition , the frame rate of data

band located in the wavelength range between about 7 .9 um

acquisition is much higher. For example , embodiments ofmethane. The multispectral data obtained in the one or more

rates from about 5 Hz and higher . For example , various

optical channels can be correlated with a predetermined

embodiments described herein can operate at frame rates

and about 8 . 4 um corresponding to an absorption peak of

the imaging system described herein can operate at video

absorption spectrum of methane in the wavelength range 35 from about 5 Hz to about 60 Hz or 200 Hz. Thus , the user
between about 7 . 9 um and 8 . 4 um . In various implementa
is able to recognize in the images the wisps and swirls

tions, the predetermined absorption spectrum of methane

typical of gasmixing without blurring out of these dynamic

can be saved in a database or a reference library accessible

image features and other artifacts caused by the change ofby the system . Based on an amount of correlation (e. g., ascene (whether spatial or spectral) during the lengthy mea

value of correlation coefficient ), the presence or absence of 40 surements. In contradistinction , scanning based imaging

methane in the gas cloud can be detected .
Examples of Practical Embodiments and Operation
The embodiment 300 of FIG . 3 is configured to employ 12
optical channels and 12 corresponding microbolometersystems involve image data acquisition over a period of timeexceeding a single -snap - shot time and can , therefore, blur
the target gas features in the image and inevitably reduce theotherwise achievable sensitivity of the detection . This result

FPAS 336 to capture a video sequence substantially imme- 45 is in contrast to embodiments of the imaging system
diately after performing calibration measurements . The
described herein that are capable of detecting the localized

video sequence corresponds to images of a standard labo

concentrations of gas without it being smeared out with theratory scene and the calibration measurements are per

areas of thinner gas concentrations. In addition , the higher

formed with the use of a reference source including two

frame rate also enables a much faster response rate to a leak

shutters , as discussed above , one at room temperature and 50 of gas (when detecting such leak is the goal) . For example ,one 5º C . above room temperature. The use of 12 FPAS
allows increased chance of simultaneous detection and esti-an alarm can trigger within fractions of a second rather thanseveral seconds .

mation of the concentrations of about 8 or 9 gases present at

To demonstrate the operation and gas detection capability

the scene . In various embodiments , the number of FPAs 336

of the imaging systems described herein , a prototype was

Due to the specifics of operation in the IR range of theat a distance of approximately 10 feet. FIG . 7 illustrates

can vary, depending on the balance between the operational 55 constructed in accordance with the embodiment 300 of FIG .
3A and used to detect a hydrocarbon gas cloud of propylene
requirements and consideration of cost.
spectrum , the use of the so - called noise -equivalent tempera -video frames 1 through 12 representing gas - cloud -detection

ture difference (or NETD ) is preferred and is analogous to

output 710 (seen as a streak of light) in a sequence from t= 1

the SNR commonly used in visible spectrum instruments. 60 to t = 12 . The images 1 through 12 are selected frames taken
The array of microbolometer FPAs 336 is characterized tofrom a video -data sequence captured at a video - rate of 15

perform at NETD = 72 mK at an f-number of 1.2 . Eachmeasurement was carried out by summing four consecutive

frames/sec . The detected propylene gas is shown as a streakof light 710 (highlighted in red ) near the center of eachframes , and the reduction in the NETD value expected due

image. The first image is taken just prior to the gas emerging

to such summation would be described by corresponding 65 from the nozzle of a gas-contained , while the last image

factor of V4 = 2 . Under ideal measurement conditions, there

fore , the FPA NETD should be about 36 mK .

represents the system output shortly after the nozzle has
been turned off.

US 9 ,756 ,263 B2
25
The same prototype of the system can also demonstrate

26
can image them onto the detector arrays are then placed at

the dynamic calibration improvement described above byimaging the scene surrounding the system (the laboratory )the location of the scene reference aperture ( e. g., 338 of FIG .
3A ).

with known temperature differences. The result of imple

In FIG . 9A , the mirrors 975a and 975b are configured tomenting the dynamic correction procedure is shown in 5 reflect radiation from the reference sources 972a and 972b

FIGS. 8A , 8B , where the curves labeled " obj” (or “ A ” )(represented as ref A and ref B ). The mirrors 975a and 975b

represent temperature estimates of an identified region in thecan be disposed away from the central FOV of the detector

scene. The abscissa in each of the plots of FIGS. 8A , 8B

arrays 1 , 5 , 9 such that the central FOV is not blocked or

expected that when all detector elements receive radiant dataarray 5 could be greater than the FOV of the detector arraysthat the object's temperature perceived by all detector ele

can be disposed away from the central FOV of the detector

indicates the number of a FPA , while the ordinate corre obscured by the image of the reference source 972a andsponds to temperature ( in degrees C . ). Accordingly , it is 10 972b . In various implementations, the FOV of the detector

that, when interpreted as the object' s temperature , indicatesments is the same, any given curve would be a substantially

1 and 9 . In such implementations , themirrors 975a and 9756

array 5 at a location such that the reference source 972a andflat line. Data corresponding to each of the multiple “ obj” 15 972b is imaged by the detector array 5 . The mirrors 975a andcurves are taken from a stream of video frames separated

975b may comprise imaging optical elements having opticalfrom one another by about 0 . 5 seconds ( for a total of 50

power that image the reference sources 972a and 972b onto

frames ). The recorded " obj” curves shown in FIG . 8A the detector arrays 1 and 9 . In this example, the referenceindicate that the detector elements disagree about thesources 972a and 972b can be disposed in the same plane as
object’s temperature , and that difference in object’ s tem - 20 the re -imaging lenses 928a , however, the reference sources
perature perceived by different detector elements is as high972a and 972b can be disposed in a different plane or inas about 2 .5° C . In addition , all of the temperature estimates
are steadily drifting in time, from frame to frame. The curves

different locations. For example, the reference sources 972a
and 972b can be disposed in a plane that is conjugate to the

300 of FIG . 3A . The results of detection of radiation carried
dynamic calibration procedure described above areexpressed with the curved labeled “ obj corr” (or “ B ” ) . Now ,the reference sources 972a and 972b is formed by thesources 972a and 972b can be disposed in a plane that isspaced apart from the conjugate plane such that a defocused

improving the original reading at least by a factor of 5 ).
FIG . 8B represents the results of similar measurements
corresponding to a different location in the scene ( a locationence sources 972a and 972b need not be disposed in thesame plane .
As discussed above, in some embodiments, the reference

correction algorithm discussed above is operable and effec

reference sources 972a and 972b are focused . In contrast, inlabeled “ ref” (or “ C ” ) correspond to the detectors ' estimates plane in which the detector array 1 , detector array 5 , andof the temperature of the aperture 338 of the embodiment 25 detector array 9 are disposed such that a focused image ofout after each detector pixel has been subjected to thedetector arrays. In some implementations , the reference

the difference in estimated temperature of the object among 30 image of the reference sources 972a and 972b is formed by
the detector elements is reduced to about 0 .5° C . (thereby
the detector arrays. In various implementations , the refer
which is at a temperature about 9° C . above the estimated 35 sources 972a and 972b are imaged onto the detector array 1
temperature of the aperture 338 of FIG . 3A ) . As shown , theand detector array 9 , without much blur such that thetive and applicable to objects kept at different temperature .
Accordingly , the algorithm is substantially temperature
independent.

Dynamic Calibration Elements and References

FIGS. 9A and 9B illustrates schematically different imple -mentations 900 and 905 respectively of the imaging systemthat include a variety of temperature calibration elements toother embodiments , the image of reference sources 972a and972b formed on the detector array 1 , and detector array 9 are

40 blurred such that the reference sources 972a and 972b are

defocused , and thereby provide some averaging, smoothing,and /or low pass filtering . The reference sources 972a and972b may comprise a surface of known temperature andmay or may not include a heater or cooler attached thereto

facilitate dynamic calibration of the FPAs. The temperature 45 or in thermal communication therewith . For example , thecalibration elements can include mirrors 975a, 975b (rep resented as M , 4 , Moa, etc .) as well as reference sources

972a and 972b . The implementation 900 can be similarly
configured as the embodiment 300 and include one or morereference source 972a and 972b may comprises heaters andcoolers respectively or may comprise a surface with atemperature sensor and a heater and sensor respectively indirect thermal communication therewith to control the tem

front objective lens , a divided aperture , one or more spectral 50 perature of the reference surface . In various implementa

filters , an array of imaging lenses 928a and an imaging

tions , the reference sources 972a and 972b can include aelement 936 . In various implementations, the imaging ele -temperature controller configured to maintain the referencecomprise a single FPA , an array of FPAs. In various imple

temperature of the reference sources 972a and 972b andment 936 (e . g ., camera block ) can include an array of sources 972a and 972b at a known temperature . In somecameras . In various implementations, the array of cameras
implementations, the reference sources 972a and 972b can
can comprise an optical FPA unit. The optical FPA unit can 55 be associated with one or more sensors that measure thementations, the array of cameras can include one or more
detector arrays represented as detector array 1 , detector array
5 , detector array 9 in FIGS. 9A and 9B . In various embodi-

communicate the measured temperature to the temperature
controller. In some implementations, the one or more sensors can communicate the measured temperature to thements , the FOV of each of the detector arrays 1 , 5 , 9 can be 60 data -processing unit . In various implementations, the refer
divided into a central region and a peripheral region . With ence sources 972a and 972b may comprise a surface ofout any loss of generality, the central region of the FOV of

unknown temperature . For example, the reference sources

arrays 1, 5 , 9 , for example, and mirrors 975a and 975b thatin other implementations , the reference sources 972a andeach of the detector arrays 1, 5 , 9 can include the region
may comprise a wall of a housing comprising the imaging
where the FOV of all the detector arrays 1, 5 , 9 overlap . In
system . In some implementations , the reference sources
the embodiment illustrated in FIG . 9A , the reference sources 65 972a and 972b can comprise a surface that need not be
972a and 972b are placed at a distance from the detector associated with sensors , temperature controllers . However,US 9 ,756 ,263 B2
27

28

972b can comprise a surface that can be associated withand /or spectral measurements agree with the reference

sensors , temperature controllers .

detector. For example , detector arrays 1 and 9 can be

In FIG . 9B , the temperature - calibration elements com

calibrated to agree with each other . As another example ,prise temperature -controlled elements 972a and 972b ( e. g.,

detector arrays 1 , 5 and 9 can be calibrated to agree withreference sources 972a and 972b is also imaged by theimage of each camera (e.g., on the detector arrays 1 and 9)

a thermally controlled emitter, a heating strip , a heater or a 5 each other .
cooler ) disposed a distance from the detector arrays 1 , 5 , 9 .
The reference sources 972a and 972b can be coated with
In various embodiments , the temperature -controlled ele a material to make it behave substantially as a blackbody
ments 972a and 972b can be disposed away from the central(for which the emission spectrum is known for any given
FOV of the detector arrays 1 , 5 , 9 such that the central FOV
temperature ). If a temperature sensor is used at the location
is not blocked or obscured by the image of the reference 10 of each reference source , then the temperature can besource 972a and 972b . The radiation emitted from the tracked at these locations. As a result, the regions in thedetector array 936 along with the radiation incident from thein which the object has such known temperature ( and ,object. Depending on the position of the reference sources
therefore , spectrum ) can be defined . A calibration procedure
972a and 972b the image obtained by the detector array of 15 can thus be used so that most of the cameras (if not every
the reference sources can be blurred (or defocused ) or sharp
camera ) so operated agrees, operationally, with most or( or focused ). The images 980a , 9805 , 9800, 9800 , 980e andevery other camera , for objects at the temperatures repre

980f of the temperature - controlled elements 972a and 9726

sented by those two sources. Calibrating infrared cameras

can be used as a reference to dynamically calibrate the one
using sources at two different temperatures is known as aor more cameras in the array of cameras .
20 " two -point" calibration , and assumes that the measured
In the implementations depicted in FIGS. 9A and 9B , thesignal at a given pixel is linearly related to the incident

detector arrays 1 , 5 and 9 are configured to view (or image )irradiance . Since this calibration can be performed during

both the reference sources 972a and 972b . Accordingly,

multiple , more , or even every frame of a sequence, it ismultiple frames (e.g ., every or substantially every frame)

within a sequence of images contains one or more regions in 25

the image in which the object image has known thermal andspectral properties . This allows multiple ( e . g ., most or each )referred to as a " dynamic calibration ” .

An example of the dynamic calibration procedure is asfollows. If there is a temperature sensor on the reference
sources or reference surface , then the temperature measurecameras within the array of cameras to be calibrated to agree
m ents obtained by these temperature sensors can be used towith other ( e . g ., most or every other camera imaging thedetermine their expected emission spectra . These tempera
same reference source ( s ) or surface ( s ) . For example , detec - 30 ture measurements are labeled as TAR ), TRR ), and TCR

tor arrays 1 and 9 can be calibrated to agree with each other.
calibrated to agree with each other. In various embodiments,

for the “ reference temperatures” of sources/surfaces A , B ,scalar correction factors to apply to the entire image of a

As another example , detector arrays 1 , 5 and 9 can be

and C . These temperature measurements can be used as

the lenses 928a provide blurred (or defocused ) images of thegiven camera , forcing it to agree with the reference tem

because the location of the reference sources are not exactly
in a conjugate planes of the detector arrays 1 and 9 . Althoughpixel from T to T ' can use formulae analogous to those
discussed below in reference to FIGS . 10A , 10B , 10C . If nothe lenses 928a are described as providing blurred or defo -direct temperature sensor is used , then one of the cameras

cused images , in various embodiments , reference sources orcan be used instead . This camera can be referred to as the

such blur and defocus and instead are focused images.

provided in paragraph below can be used , but with T [ R ]example , the mirrors shown in FIG . 9A to provide such

sources / surfaces A and B as estimated by the reference

reference sources 972a , 972b on the detector arrays 1 and 9 35 peratures. Correcting the temperature estimate of a given

surfaces are imaged on the detectors arrays 1 , 5 , 9 without 40 “ reference camera ” . In this case , the same formulae as those

Additionally optical elements may be used , such as for

and TB [R ] representing the temperatures of the reference

focused images.
camera . By applying the dynamic calibration correction
The temperature of the reference sources 972b , 972a can 45 formulae , all of the other cameras are forced to match the
be different. For example, the reference source 972a can be temperature estimates of the reference camera .
at a temperature T2, and the reference source 972b can be at
In the configuration illustrated in FIG . 9B , the referencea temperature TR lower than the temperature T4. A heater cansources 972a and 972b are placed such that the images of thebe provided under the temperature - controlled element 972asources on the detector arrays are blurred . The configuration

to maintain it at a temperature T4, and a cooler can be 50 illustrated in FIG . 9A is similar to the system 400 illustrated

provided underneath the temperature -controlled elementin FIG . 4 where the reference sources are placed at an

mentations, the embodiments illustrated in FIGS. 9A and 9B
can be configured to image a single reference source 972

this configuration , the array of reference apertures, similar toreference apertures 438a in FIG . 4 , will have an accompa

972b to maintain it at a temperature TB . In various imple

intermediate image plane (e .g., a conjugate image plane). In

instead of two references sources 972a and 972b maintained 55 nying array of reference sources or reference surfaces suchat different temperatures . It is understood that the single
reference source need not be thermally controlled . For
example , in various implementations, a plurality of detectorsthat the reference sources or surfaces (e . g ., each reference
source or surface ) are imaged onto a camera or a detector
array such as FPAs 1 , 5 , 9 . With this approach , the referencein the detector array can be configured to image a same
source or surface images are at a conjugate image plane andsurface of at least one calibration element whose thermal 60 thus are not appreciably blurred , so that their images can beand spectral properties are unknown . In such implementa made to block a smaller portion of each camera ' s field oftions , one of the plurality of detectors can be configured asa reference detector and the temperature of the surface of theview .

A " static" calibration (a procedure in which the scene is

at least one calibration element imaged by the plurality oflargely blocked with a reference source such as the moving
detectors can be estimated using the radiance spectrum 65 shutters 960 in FIGS. 9A and 9B , so that imaging of an

obtained by the reference detector. The remaining pluralityof detectors can be calibrated such that their temperature

unknown scene cannot be performed in parallel with cali

bration ) allows a plurality of the cameras (for example,mostUS 9 ,756 ,263 B2
29

30

or each camera ) to accurately estimate the temperature of adetector array of a camera , so that many, most, or every

plurality of elements (for example , most or each element in
the scene ) immediately after the calibration is complete . It

frame within a sequence of images contains one or moreregions in the image in which the object image has known

cannot, however, prevent the cameras ' estimates from drift

thermal and spectral properties . This approach allowsmost

ing away from one another during the process of imaging an 5 of the camera , if not each camera , within the array of

unknown scene. The dynamic calibration can be used tocameras to be calibrated to agree with most or every otherreduce or prevent this drift, so that all cameras imaging acamera imaging the same reference source or sources. For

scene can be forced to agree on the temperature estimate of example , cameras 0 , 1, 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 can be
the reference sources/ surfaces , and adjust this correction
calibrated to agree with each other. As another example ,during every frame.
10 cameras 0 , 1 , 2 and 3 can be calibrated to agree with each
FIG . 10A illustrates schematically a related embodiment other. As yet another example , cameras 0 , 1 , 2 , 3 , 7 , 11 , 10 ,1000 of the imaging system , in which one or more mirrors
9 , 8 , 4 , 5 and 6 can be calibrated to agree with each other.
Mo4 , . . . My and More . . . My pare placed within the fields Accordingly , in various implementations , two , three , four,of view of one or more cameras 0 , . . . , 11 , partially blocking

five, six , seven , eight, nine, ten , eleven or twelve cameras

an outer ring of cameras including cameras 0 , 1 , 2 , 3 , 7 , 11,embodiments , however , not all the cameras are calibrated to

10 , 9 , 8 and 4 surrounding the central cameras 5 and 6 . In
various implementations , the FOV of the central cameras 5
and 6 can be less than or equal to the FOV of the outer ringagree with each other. For example, one, two , or morecameras may not be calibrated to agree with each other whileothers may be calibrated to agree with each other. In various

implementations, the one ormore mirrors Moa , . . .M14 andMob , . . . M11B can be placed outside the central FOV of thecameras 5 and 6 and is placed in a peripheral FOV of thereference sources/ surfaces A and B onto different respective
pixels a given FPA . Without any loss of generality , FIGS.
10A and 10B represent a top view of the embodiment shown

cameras outer ring of cameras 0 , 1, 2 , 3 , 7 , 11 , 10 , 9 , 8 andin FIG . 9A .

cameras 5 and 6 such that the reference sources A and B are

1050 of the imaging system , in which one or more

the field of view . The cameras 0 , . . . , 11 are arranged to form 15 can be calibrated to agree with each other. In certainof cameras 0 , 1, 2, 3, 7 , 11 , 10 , 9 , 8 and 4 . In such 20 embodiments , these mirrors may be configured to image the4 which does not overlap with the central FOV of the 25

not imaged by the cameras 5 and 6 . In various implemen tations, the FOV of the central cameras 5 and 6 can begreater than the FOV of the outer ring of cameras 0 , 1 , 2 , 3 ,FIG . 10C illustrates schematically a related embodimentreference sources R249 . . . , R114 and Rob . . , R11B

are disposed around the array of detectors 0 , . . . , 11 . In
various implementations , the one or more reference sources

7 , 11 , 10 , 9 , 8 and 4 . In such implementations, the one or 30 RA, . . . , R114 and Rob , . . . , Rus can be a single reference
source that is imaged by the detectors 0 , . . . , 11. In various
implementations, central detector arrays 5 and 6 can have amore mirrors MoA . . . M114 and Mob ; • . . M11B can beplaced in a peripheral FOV of the cameras 5 and 6 which
does overlap with the central FOV of the outer ring ofcameras 0 , 1 , 2, 3 , 7 , 11 , 10 , 9 , 8 and 4 such that the reference

sources A and B are imaged by the cameras 5 and 6 .

This design is an enhancement to the systems 300 and 400

shown in FIGS. 3A and 4A . In the system 1000 shown in
FIG . 10A , an array of two or more imaging elements ( curved
mirrors, for example ) is installed at a distance from the

FOV that is equal to or lesser than the FOV of the outer ring
35 implementations, the reference sources ROA , . . . , R114 canof the detectors 0 , 1, 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 . In suchbe disposed away from the central FOV of the detector

arrays 5 and 6 such that the radiation from the reference
is imaged only by the outer ring ofsources Roa, . . . R,
detectors 0 , 1 , 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 . In various

FPAs, for example , in the plane of the reference aperture 160 40 implementations, central detector arrays 5 and 6 can have ashown in FIG . 3A . These elements (mirror or imaging
FOV that is greater than the FOV of the outer ring of theelements ) are used to image one or more temperature controlled reference sources A and B onto the detector
elements of two or more of the cameras . The primary

detectors 0 , 1, 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 . In such
implementations, the reference sources Roa , . . . , R14 canbe disposed in the peripheral FOV of the detector arrays 5

difference between embodiment 1000 and embodiment 300 45 and 6 such that the radiation from the reference sources

or 400 is that now a plurality ormost or all of the outer ringof cameras in the array can image both the reference sources

A and B instead of imaging one of the two reference source
A and B . Accordingly , most or all of the outer ring ofRoa , . . . , RU14 is imaged only by the outer ring of detectors

0 , 1 , 2 , 3, 7 , 11, 10 , 9 , 8 and 4 . The radiation from thereference sources Ro4 , . . . , RU4 is therefore imaged by theouter ring of detectors 0 , 1 , 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 as wellcameras image an identical reference source or an identical 50 as central cameras 5 and 6 . Without any loss of generality ,set of reference sources (e. g., both the reference sources A

and B ) rather than using different reference sources fordifferent cameras or imaging different portions of the refer ence sources as shown in FIGS. 3A and 4A . Thus , this

FIG . 10C represents a top view of the embodiment shown in

FIG . 9B .
In various implementations, a heater can be provided
underneath , adjacent to , or in thermal communication withapproach improves the robustness of the calibration , as it 55 reference source / surface A to give it a higher temperatureeliminates potential failures and errors due to the having
T2, and a cooler can be provided underneath , adjacent to , oradditional thermal sensors estimating each reference source .
The imaging elements in the system 1000 (shown asin thermal communication with reference source B to give ita lower temperature TB . In various implementations, themirrors in FIGS. 10A and 10B ) image one or more con embodiments illustrated in FIGS. 10A , 10B and 10C can betrolled -temperature reference sources or a surface of a 60 configured to image a single reference source A instead ofcalibration element ( shown as A and B in FIGS. 10A and

two references sources A and B maintained at different

10B ) into the blocked region of the cameras ' fields of view .
FIG . 10B shows an example in which mirror MoA images

temperatures. As discussed above, the embodiments illus
trated in FIGS. 10A , 10B and 10C can be configured toreference source/surface A onto camera 0 , and mirror MB

image a same surface of a calibration element. In such

images reference source / surface Bonto camera 0 , and 65 implementations, the temperature of the surface of the

likewise for cameras 1, 2 , and 3 . This way, each of themirrors is used to image a reference source / surface onto acalibration element need not be known . Many,most or eachreference source/ surface can be coated with a material toUS 9 ,756 ,263 B2
31

32

make it behave approximately as a blackbody , for which thesupport the DAISI system . In addition , it can be challenging

emission spectrum is known for any given temperature . If

to move the DAISI system from site to site to monitor

many, most, or each camera in the array of cameras images

different locations. For example , installing and removing theboth of references A and B , so that there are known regions

DAISI system from a site for transport may involve subin the image of these cameras in which the object has a 5 stantial effort and time for the user when the system isknown temperature (and therefore spectrum ), then one canperform a calibration procedure . This procedure can providethat many, most or every camera so operated agrees withvarious , most, or every other camera , for objects at theconnected to infrastructure at the site to be monitored .

Accordingly , it can be desirable to provide a DAISI systemthat can be used independently of the facilities or infrastructure at the site to be monitored . Furthermore , it can betemperatures represented by those two sources. For 10 advantageous to implement the DAISI system in a form

example , two, three , four, five , six , seven , eight, nine , ten ,eleven or twelve cameras can be calibrated to agree withfactor and with a weight that can be carried or worn by auser. For example, a mobile DAISI system can enable theeach other. In certain embodiments , however, not all theuser to easily transport the system from site -to - site , whilecameras are calibrated to agree with each other. For

monitoring the presence of gases or chemicals in real-time.

example , one, two, or more cameras may not be calibrated 15
It should be appreciated that each of the systems disclosed
to agree with each other while others may be calibrated toherein can be used to monitor potential gas leaks in any

agree with each other. As discussed above, calibration of
infrared cameras using sources at two different temperatures

suitable installation site , including , without limitation , drill
ing rigs, refineries , pipelines, transportations systems, ships

The dynamic calibration is used to obtain a correctedillustrated herein such as above, e .g ., with respect to FIGS .
1 - 10C , can be used in combination with each of the embodi
ments disclosed and illustrated herein with respect to FIGS .

is known as a " two - point" calibration , and assumes that theor other vessels ( such as off- shore oil rigs, trains, tanker
measured signal at a given pixel is linearly related to the 20 trucks, petro -chemical plants, chemical plants , etc . In addi
incident irradiance .
tion , each of the embodiments and aspects disclosed andtemperature T' from the initial temperature T estimated ateach pixel in a camera using the following formulae :25 11A - 14C .
T"[x,y,c]= (7[x,y,c]- TA [R ])G [c]+ TA[R ]FIG . 11A is a schematic diagram illustrating a mobile
where is TA [R ] is a dynamic offset correction factor, and , infrared imaging system 1000 (e. g., a mobile or portable

DAISI system ) configured to be carried or worn by a human

TB [R ] – TA [ R ]G[c] = Tel - T .[c]is a dynamic gain correction factor. The term c discussed
above is a camera index that identifies the camera whoseuser 1275 . The user 1275 may wear a hat or helmet 1200

30 when he travels to a site to be monitored , such as an oil wellsite, a refinery , etc . The system 1000 shown in FIG . 11A is
attached to the helmet 1200 by way of a support 1204 that

securely mounts the system 1000 to the helmet 1200 . For
example, the support 1204 can comprise a fastener, a strap ,35 or any other suitable structure . Advantageously , mounting
data is being corrected .
the system 1000 to the helmet 1200 can enable the user 1275
III. Examples of a Mobile DAISI System
to capture images within the system 's field of view (FOV )by turning his head to face a particular location to be
The DAISI systems disclosed herein can be configured to monitored . For example, the user 1275 can walk through the

be installed at a suitable location on a long - term basis , 40 site and can capture video images of each portion of the site ,according to some embodiments . For example , the DAISI e .g ., various structures that may be susceptible to gas orsystems disclosed in Section II above can be affixed to achemical leaks, such as valves , fittings , etc . Thus , in the

fixture mounted to the ground at a location to continuously

embodiment shown in FIG . 11A , the user 1275 can image

orperiodically monitor the presence of gases or chemicals at
each portion of the site by facing the area to be imaged andthe location . In some embodiments , for example , the DAISI 45 ensuring that the system 1000 is activated . In addition , bysystems can be attached to a pole , post, or any suitable mounting the system 1000 to the user 's helmet 1200 , thefixture at the location to bemonitored . In such arrangements, user 1275 may use his hands for other tasks while the system
the DAISI system can continuously or periodically capture1000 images the site . Although the system 1000 of FIG . 11A
multispectral, multiplexed image data of the scene, and an
is shown as being mounted to the user 's helmet 1200 , iton -board or remote computing unit can process the captured 50 should be appreciated that the system 1000 can instead be
image data to identify or characterize gases or chemicals at worn on other parts of the user ' s clothing or can be carried
the location . A communications module can communicate by the user, e . g ., in a bag , case , or other suitable container.
data relating to the identified gases or chemicals to any

Furthermore , in some embodiments , a wind sensor can besuitable external system , such as a central computing server,

provided to the user, e .g ., on the user 's clothing and/or on or

etc . For such long -term installations of the DAISI system , 55 near the system 1000 . The wind sensor can be used tothe installation site may include a power source ( e . g .,electrical transmission lines connected to a junction box at

estimate wind conditions at the installation site , which canbe used to improve the detection of gas leaks. In otherthe site ) and network communications equipment ( e . g .,embodiments , the system 1000 can be coupled to or formed

network wiring , routers , etc .) to provide network commu-

with a housing that defines a " gun ” -like structure which cannication between the DAISI system and the external sys - 60 be aimed or pointed by the user in a particular direction .
As explained herein , a gas cloud 1202 emitted from atems.

It can be advantageous to provide a mobile DAISI system
structure at the site can be imaged by pointing the system
configured to be worn or carried by a user. For example , it1000 towards the gas cloud 1202 and capturing an image of
may be unsuitable or undesirable to install a DAISI system
the gas cloud 1202 when the cloud 1202 is within the FOV
at some locations on a long -term basis . As an example , some 65 of the system 1000 . Unlike other systems, the system 1000
oil well sites may not have sufficient infrastructure , such as can capture multispectral image data of a single scene overpower sources or network communication equipment, toa range of IR wavelengths with a single snapshot, asUS 9 ,756 ,263 B2
33

34

explained in further detail herein . The single snapshot can beor laptop computer 1203 to conduct additional analysis of

captured in a short timeframe, e. g ., less than about 3

the imaged and processed gas cloud 1202 . Furthermore ,seconds, less than about 2 seconds, or less than about 1 . 5

information about the gas cloud (including the processed

seconds ( for example , in about 1 second , in some embodi

data and /or the raw image data ) may also be transmitted toabout 5 milliseconds, greater than about 0 . 2 seconds, orgreater than about 0 .5 seconds. The captured image data cananalysis . In various arrangements , a global positioning sys
t em (GPS ) module can also be installed on board the systemments ) . The single snapshot can be captured in greater than 5 a central server for centralized collection , processing, andbe processed on board the system 1000 by a processing unit,

1000 and / or on the mobile computing device (such as a

as explained in further detail herein . For example , thetablet computing device , smartphone, etc .). The GPS modprocessing unit can process the image data from the different 10 ule can identify the coordinates of the user 1275 when aoptical channels and can compare the captured spectral
information with a database ofknown chemicals to identify

particular image is captured . The location data for thecaptured image data can be stored on the central server forand / or characterize the gases that are included in the gas

further analysis .

cloud 1202.

Thus , the system 1000 shown in FIG . 11A can enable the

A communications module on board the system 1000 can 15 user 1275 to image multiple locations of a particular site totransmit information relating to the identified gases or be monitored , such as an oil well site . Advantageously , thechemicals to any suitable external device . For example , the optical components, the processing components, and thecommunications module can wirelessly communicate ( e . g .,by Bluetooth , WiFi, etc .) the information to a suitablecommunications components of the system 1000 can be
integrated within a relatively small housing that can bemobile computing device , such as an electronic eyewear 20 carried or worn by the user 1275 . For example, in various
apparatus 1201, a tablet computing device 1212 , a mobile
smartphone, a laptop or notebook computer 1203 , or any

embodiments , the system 1000 does not include complex
mechanical components for movement, such as gimbals,

other suitable mobile computing device . In some embodi-

actuators , motors , etc . Without such components, the size ofments, if a gas cloud is detected , the system 1000 can warn

the system 1000 can be reduced relative to other systems.

the user by way of sending a signal to the mobile device 25

Unlike other systems, in which the system components( e . g ., tablet computing device 1212 or a mobile smartphone .
The mobile device can emit an audible ring and /or can

are bulky or are assembled over a large form factor, themobile system 1000 can be sized and shaped in such avibrate to notify the user of a potential gas leak . In themanner so as to be easily moved and manipulated when theembodiment of FIG . 11A , the electronic eyewear apparatus user 1275 moves about the site . Indeed , it can be very
1201 can include a user interface comprising a display that 30 challenging to integrate the various system components in a
the user 1275 can view in real-time as he visits the site . In
small form -factor. Advantageously, the systems 1000 can besome embodiments, the electronic eyewear apparatus 1201 worn or carried by a human user. For example, the compo
nents of the system 1000 can be contained together in a data
eyewear apparatus 1201 can be further configured to present acquisition and processingmodule 1020, which may include
images from this display to the wearer. The electronics 35 a housing to support the system components . The compo
eyewear apparatus 1201 may for example include projection nents of the system 1000 ( including the optical or imaging
optics that projects the image into the eye . The electronic components, the focal plane array, the on -board processing
eyewear apparatus 1201 may comprise heads up display
electronics , and the communications components ) may be
optics the presents the image on the lens portion ( s) of the packaged or assembled in the data acquisition and process
comprises eyewear that includes a display. The electronics

eyewear so that the wearer can view the image and also see 40 ing module 1020 and may occupy a volume less than about

through the eyewear and peer at objects in the distance .

300 cubic inches, less than about 200 cubic inches , or less

Other configurations are possible . In some arrangements, thethan about 100 cubic inches . In various embodiments , thedevice , sold by Google , Inc., of Mountain View , Calif.

imaging components, the focal plane array, the on -board

data such that the types of identified gases are displayed tothe user 1275 on the display of the eyewear apparatus 1201 .

nents ) may be packaged or assembled in the data acquisitionand processing module 1020 and may occupy a volume

eyewear apparatus 1201 can comprise a Google Glasscomponents of the system 1000 ( including the optical or

The processing unit can configure the processed image 45 processing electronics , and the communications compo

For example , in some embodiments , color- coded data may

greater than about 2 cubic inches , or greater than about 16

represent different types of gases or concentrations of a

cubic inches .

of the scene. For example , the color- coded data and imageof the gas cloud can be seen by the user on the electronic
eyewear apparatus 1201. In various embodiments, text data

the system components mounted therein or thereon ) may besized and shaped to fit within a box - shaped boundary having
dimensions XxYxZ . For example , the data acquisition andand statistics about the composition of the gas cloud 1202
can walk the site and can view the different types of gases

processing module 1020 , including the imaging optics, focal
included in a package that is sized and shaped to fit withingeously , such real-time display of the composition of the gas

package may also contain a power supply , such as a battery

particular gas , and may be overlaid on a visible light image 50

The data acquisition and processing module 1020 (withmay also be displayed to the user 1275 . Thus, the user 1275 55 plane array, and on board processing electronics , may bein the gas cloud 1202 substantially in real-time. Advanta

the box -shaped boundary having dimensions XxYxZ . This

cloud 1202 can enable the user 1275 to quickly report urgent and /or solar module . In some embodiments , the data acqui
events , such as the leakage of a toxic gas or chemical. In 60 sition and processing module 1020 (including the imaging
some embodiments, detection of a toxic leak can trigger an
optics, focal plane array, and on board processing electronics
alarm , which may cause emergency personnel to help evacu - may ) can be sized and shaped to fit within a box -shaped
ate the site and /or fix the leak .
boundary smaller than 8 inchesx6 inchesx6 inches . In some
In some embodiments , the processed image data can beembodiments , the data acquisition and processing module

transmitted from the system 1000 to the tablet computing 65 1020 ( including the imaging optics, focal plane array, and on
device 1212 , laptop computer 1203 , and /or smartphone. The

board processing electronics may ) can be sized and shaped

user 1275 can interact with the table computing device 1212

to fit within a box -shaped boundary smaller than 7 inchesx5

US 9 ,756 ,263 B2
35
inchesx5 inches, e. g., a box -shaped boundary small than 7
inchesx3 inchesx3 inches. In some embodiments , the dataacquisition and processing module 1020 (including theimaging optics , focal plane array, and on board processing

36

than 1 cubic inches . The data acquisition and processing
module 1020 ( including the imaging optics , focal planearray, and on board processing electronics may ) can have avolume more than 4 cubic inches. The data acquisition andelectronics may ) can be sized and shaped to fit within a 5 processing module 1020 (including the imaging optics , focalbox -shaped boundary smaller than 6 inchesx4 inchesx4 plane array , and on board processing electronics may) can
inches . In some embodiments, the data acquisition and have a volumemore 5 cubic inches. The data acquisition andprocessing module 1020 (including the imaging optics, focal

plane array , and on board processing electronics may ) can beprocessing module 1020 ( including the imaging optics, focal

plane array , and on board processing electronics may) cansized and shaped to fit within a box - shaped boundary 10 have a volume more 10 cubic inches. This package may also

smaller than 2 inchesx2 inchesx6 inches. In some embodiments, the data acquisition and processing module 1020
(including the imaging optics , focal plane array , and on

contain a power supply , including a battery and/ or solar
module , a communications module , or both and fit into theabove-referenced dimensions . It should be appreciated thatboard processing electronics may ) can be sized and shaped

the dimensions disclosed herein may not correspond to theto fit within a box -shaped boundary having dimensions 15 directions shown in FIG . 11A with respect to X , Y, and Z .
larger than 4 inchesx2 inchesx2 inches. In some embodi
Moreover, the system 1000 can have a mass and weightments , the data acquisition and processing module 1020 sufficiently small so as to enable the user 1275 to easily carry( including the imaging optics, focal plane array, and on or wear the data acquisition and processing module 1020 atboard processing electronics may ) can be sized and shaped the site . Thus , the embodiment shown in FIG . 11A can be
to fit within a box -shaped boundary having dimensions 20 sized and shaped and configured to have a mass that enables
larger than 3 inchesx3 inchesx7 inches. In some embodi- a human user to easily and effectively manipulate the systemments, the data acquisition and processing module 1020
1000 .
( including the imaging optics, focal plane array , and on
FIG . 11B is a schematic diagram illustrating an installa

board processing electronics may ) can be sized and shapedtion site ( e . g ., an oil well site , etc .) that can be monitored by
to fit within a box -shaped boundary having dimensions 25 multiple infrared imaging systems 1000 (e . g ., a DAISI

larger than 2 inchesx1 inchesx1 inches . The data acquisition

system ). For example , as shown in FIG . 11B , an imaging

the imaging optics, focal plane array, and on board process ing electronics may ) can have dimensions greater than 1

imaging system 1000D can be mounted on an aerial platform 1501, such as an unmanned aerial vehicle (UAV ) or a

ing module 1020 (including the imaging optics, focal planeprise an airplane, a helicopter (such as a quad helicopter ),and processing module 1020 (including the imaging optics,
system 1000A can be mounted to a pole 1309 or otherfocal plane array , and on board processing electronics may
stationary structure at the site . An imaging system 1000B
can have dimensions less than 2 inchesx2 inchesx6 inches.
can be worn or carried by multiple users 1275 , an imaging
The data acquisition and processing module 1020 (including 30 system 1000C can be mounted on a truck 1500, and/ or an
inchesx1 inchesx3 inches . The data acquisition and process -piloted airplane . In some arrangements , the UAV can com

array , and on board processing electronics may ) can have 35 etc . The embodiments disclosed herein can utilize the image

dimensions greater than 2 inchesx2 inchesx4 inches . said

data captured by any combination of the systems 1000A

data acquisition and processing module has dimensions lessthan 6 inchesx3 inchesx3 inches. The data acquisition and

1000D at the installation site to image the entire installationsite in an efficient manner . Indeed , each installation site canelectronics may ) can have dimensions less than 3 inchesx2

installation site may include less than about 100 systems

processing module 1020 (including the imaging optics, focal include any suitable number and type of system 1000A
plane array, and on board processing electronics may ) can 40 1000D . For example , each installation site can include
have dimensions less than 4 inchesx3 inchesx3 inches. The
greater than two systems 1000A - 1000D , greater than fivedata acquisition and processing module 1020 (including thesystems 1000A - 1000D , greater than ten systems 1000A
imaging optics , focal plane array , and on board processing
1000D , greater than twenty systems 1000A - 1000D . Each
inchesx2 inches. The data acquisition and processing mod - 45 1000A - 1000D .

ule 1020 (including the imaging optics, focal plane array,
For example , the central server can track the real-timeand on board processing electronics may ) can have dimen - locations of each imaging system 1000A - 1000D based onsions greater than 2 inchesx1 inchesxl inches. The data
the GPS coordinates of the particular system or on preacquisition and processing module 1020 (including thedetermined knowledge about the system 's stationary loca
imaging optics , focal plane array , and on board processing 50 tion . The distributed nature of the imaging systems 1000A
electronics may ) can have dimensions greater than 1 inchesx

1000D can provide rich information to the central server

0 .5 inchx0 .5 inch . The data acquisition and processing
module 1020 (including the imaging optics , focal planearray , and on board processing electronics may ) can have aabout the types and locations of gas leaks or other problems
throughout multiple installation sites . Although FIG . 11B
illustrates a stationary system 1000A mounted to a fixture , a

volume less than 30 cubic inches . The data acquisition and 55 portable system 1000B to be worn or carried by a human , a

processing module 1020 (including the imaging optics, focal

truck -based system 1000C , and an aerial-based systemhave a volume less than 20 cubic inches . The data acquisition and processing module 1020 (including the imaging

may be suitable. For example , in some embodiments , arobotic vehicle or a walking robot can be used as a platform

plane array, and on board processing electronics may ) can

1000D , it should be appreciated that other types of systems

optics , focalplane array, and on board processing electronics 60 for the systems 1000 disclosed herein . In various embodi
may ) can have a volume less than 15 cubic inches . The data
ments , a floating platform (such as a boat ) can be used as aacquisition and processing module 1020 (including the platform for the systems 1000 disclosed herein . It should
imaging optics, focal plane array, and on board processing also be appreciated that the systems disclosed herein canelectronics may ) can have a volume less than 10 cubic utilize any combination of the platforms ( e .g ., stationary
inches. The data acquisition and processing module 1020 65 fixtures such as a pole, human user(s ), truck (s ) or other( including the imaging optics, focal plane array , and on
vehicle , aerial platform ( s ), floating platform ( s ), robotic platboard processing electronics may ) can have a volumemore form (s ), etc .) to support the systems 1000 .

US 9 ,756 ,263 B2
37
The systems 1000 shown in FIG . 11B can comprise a

mobile DAISI system , similar to that illustrated in FIG . 11A .

38
ered by the truck ; in some embodiments, a generator can besystem 1000C can be battery powered and / or can be pow

In other embodiments , the systems 1000 can comprise aused to supply power to the system 1000C . A user can drive

larger DAISI system configured for use on a relatively

the truck 1500 throughout the installation site to image

long - term basis . For example , the stationary imaging system 5 various portions of the site to detect leaks. In addition , the1000A shown in FIG . 11B can be installed on a pole 1309
user can drive the truck 1500 to other installation sites toor other suitable structure for monitoring a storage tankdetect gas leaks. As explained herein , the location of the1301 . A solar panel 1300 can be provided at or near the truck 1500 can be communicated to the central server andsystem 1000 to help provide power to the system 1000 . An
the location of the truck 1500 can be associated with eachantenna 1303 can electrically couple to the system and can 10 captured image. The truck 1500 may include GPS electron
provide wireless communication between the system 1000 ics to assist in tracking the location of the truck 1500 and/ orand any other external entity, such as a central server, for system 1000C over time as the user drives from place tostoring and /or processing the data captured by the system place . Similarly, the aerial platform 1501 (such as an
1000 .
unmanned aerial vehicle , or UAV ) can support the imaging
The stationary infrared imaging system 1000A can be 15 system 1000D . The aerial platform 1501 can be piloted
programmed to continuously or periodically monitor the(either remotely or non -remotely ) to numerous installationsite . If a gas cloud 1302 escapes from the storage tank 1301 , sites to capture multispectral image data to detect gas clouds.

such as by leaking from a broken valve, then the systemThus, the systems 1000A - 1000D can provide extensive

1000A can capture a multispectral, snapshot image or series

data regarding the existence of leaks at numerous installaof images ( e . g ., a video stream of the gas cloud 1302 . As 20 tions across an organization .Monitoring numerous cameras

with the embodiment of FIG . 11A , the imaging systemsimultaneously or concurrently across an organization , site ,1000A can include imaging , processing , and communica -region , or the entire country can be enabled at least in parttions components on board the system 1000A to identify andby providing wireless (or wired ) communication between

characterize the types of gases in the cloud 1302 and to

the systems 1000A - 1000D and one or more central servers .

transmit the processed data to the central server, e . g ., by way 25 Advantageously , the collection of image data from multiple

sources and multiple platforms can enable the organizationof the antenna 1303.
The imaging systems 1000B worn or carried by themultiple users 1275 can advantageously capture and process

to create a real- time mapping of potential gas leaks, thetypes and amounts of gases being leaks , the locations of themultispectral image data of the portions of the installation

leaks, and the time the image data of the leak was captured .

site that each user 1275 visits. It should be appreciated that 30 In some arrangements, the aggregation of data about a site

the different users 1275 may work in or travel throughdifferent portions of the installation site (and also to acan improve the safety of installation sites . For example , ifa gas leak is detected at a particular installation , the embodi

number of installation sites ) over a period of time. Whenments disclosed herein can alert the appropriate personnel,

activated , the imaging systems 1000B worn or carried by thewho can begin safety and/or evacuation procedures. More

spectral image data of the different locations at the instal-an oil service company ) can provide site -wide, region -wide,

herein , the system 1000B can transmit the image data andexample , a given facility can monitor its total emissions overthe location at which the image was captured to the central

time and use the resulting data to help determine theusers 1275 can continuously or periodically capture multi- 35 over , the aggregation of data across an organization ( such as

lation site (s ) to which the user 1275 travels. As explained

and /or company -wide metrics for performance . For

server. If the system 1000B or the central server detects a 40 facility ' s overall performance . A given region ( such as aproblem (such as a gas leak ), then the central server canmetropolitan area , a state , etc .) can monitor trends in emis

associate that leak with a particular location and time.

sions over time, providing a value on which to base deci

Furthermore , because the central server can receive image

sions . Likewise , a company can look at the emissions

data and location data from multiple users at different

performance at all of its facilities and can make decisions

locations and viewing from different perspectives, the cen - 45 about whether some facilities should make new investments

tral server can create an organization -wide mapping of gas
to improve performance , and/ or whether the entire company
leaks that include, e . g ., the locations of gas leaks in any ofshould make various improvements . The mobile systems
multiple installation sites, the type and concentrations and1000 disclosed herein can thus provide a ubiquitous moni
expanse or extent of each gas leaked , the particular user
toring system for decision making . In addition , the systems
1275 that captured the image data , and the time at which the 50 1000 disclosed herein can be used in a feedback control
image was taken . Thus , each user 1275 that carries or wears process to improve variousmanufacturing procedures based

a portable imaging system 1000B can contribute information on the gases detected by the system (s ) 1000 . Accordingly , acontrolmodule may be provided to adjust themanufacturing
server, provides rich details on the status of any gas leaks at procedure and/or parameters according to the gases mea
any installation sites across the organization .
55 sured by the system 1000 .
The truck -mounted imaging system 1000C can be
The embodiments of the mobile infrared imaging system

to the central server that, when aggregated by the central

mounted to a truck or other type of vehicle ( such as a car ,1000 disclosed herein provide various advantages over othervan , all -terrain vehicle , etc . ) . As shown in FIG . 11B , thesystems. As explained above, aggregation of data about aimaging system 1000C can be connected to an end of ansite and its potential gas leaks can provide an organization

extendable pole or extension member mounted to the truck 60 or system -wide mapping of potential problems. Further

1500 . The system 1000C can be raised and lowered by acontrol system to enable the system 1000C to image a wide
area of the installation site . In some embodiments , actuators

can be provided to change the angular orientation of themore , automatic detection of gas leaks (and identification of
the gases in the gas cloud ) can simplify operation of thesystem 1000 and can reduce the risk of user errors inattempting to detect or identify gas clouds manually . Moresystem 1000C , e. g., its pitch and yaw . A vibration isolation 65 over, the small size of the systems 1000 disclosed herein are

or reduction mechanism can also be provided to reduce

vibrations, which may disturb the imaging process . The

more easily carried or worn by the user than other systems.

In addition , the systems 1000 disclosed herein can overlayUS 9 ,756 ,263 B2
39
the identified gas clouds on a visible image of the scene andcan color code the gas cloud according to , e .g., type of gas,
concentration , etc .

40
as explained herein , the processor 1023 can analyze cali
bration data provided by the optical system 1015 to improve
the accuracy of the measurements.

FIG . 12 is a schematic system block diagram showing a
Advantageously, the processor 1023 can comprise one ormobile infrared imaging system 1000 (e .g., a mobile DAISI 5 more field -programmable gate arrays (FPGA ) configured tosystem ), according to one embodiment. The imaging system
execute methods used in the analysis of the images captured
1000 can include a data acquisition and processing module by the optical system 1015 . For example , the FPGA can
1020 configured to be worn or carried by a person . The data
include logic gates and read access memory (RAM ) blocks
acquisition and processing module 1020 can include , conthat are designed to quickly implement the computations
tain , or house an optical system 1015 , a processing unit 10 used to detect the types of gases in a gas cloud . The small
1021, a power supply 1026 , a communication module 1025 ,size/weight, and high performance characteristics of the
FPGA can enable on board computation and analysis within
the data acquisition and detection unit 1020 worn or carried
as a post. The power unit 1026 can be provided on board the 15 by the user. The use of FPGA (or similar electronics ) on

and GPS module 1025 . In other embodiments, the dataacquisition and processing module 1020 can be configured
to be mounted to a structure at the site to be monitored , such

system 1000 . The power unit 1026 can be configured to board the system 1000 can reduce costs associated with
provide power to the various system components , such as using an off-site central server or larger computing device tothe optical system 1015 , the processing unit 1021 , the conduct the image analysis computations. In addition ,communication module 1024, and / or the GPS module 1025 . enabling computation with one or more FPGA devices on
In some embodiments , the power unit 1026 can comprise 20 board the wearable system can also prevent or reduce
one or more batteries (which may be rechargeable ) to power communication bottlenecks associated with wirelessly transthe system components . In some embodiments , the power mitting large amounts of raw data from the system 1000 tounit 1026 can include a solar power system including one ora remote server or computer, which can be used in somemore solar panels for powering the system by sunlight. In
embodiments.
some embodiments, the power unit 1026 can include various 25 The communication module 1024 can be configured to
power electronics circuits for converting AC power supplied
communicate with at least one device physically separate
by standard power transmission lines to DC power for from the data acquisition and processing module 1020. For
powering the system components. Other types of power evo
example , the communication module 1024 can include a
supply may be suitable for the power unit 1026 .
wireless communication module configured to wirelessly
The system 1000 can include an optical system 1015 30 communicate with the at least one separate device . The
configured to capture multispectral image data in a single
wireless communication module can be configured to pro

snapshot, as explained herein . The optical system 1015 can vide wireless communication over wireless networks ( e.g .,correspond to any suitable type of DAISI system , such as,
WiFi internet networks, Bluetooth networks, etc .) and / or
but not limited to , the optical systems and apparatus illus over telecommunications networks (e .g ., 3G networks, 4G
trated in FIGS. 1- 10C above and /or in the optical systems 35 networks, etc .).
1015 illustrated in FIGS . 13A - 13B below . For example , theIn some embodiments , for example, the wireless commu
optical system 1015 can include an optical focal plane array nication module can provide data communication between( FPA ) unit and components that define at least two optical the data acquisition and processing module 1020 and achannels that are spatially and spectrally different from one

mobile device such as an electronic eyewear apparatus, aanother. The two optical channels can be positioned to 40 tablet computing device, a mobile smartphone, a laptop ortransfer IR radiation incident on the optical system towards

the optical FPA . The multiple channels can be used to

notebook computer, or any other suitable mobile computing

device . As explained herein , the mobile device can include

multiplex different spectral images of the same scene and toa display on which the processed image data can be dis

image the different spectral images on the FPA unit.

played to the user. For example , the types (and/or concen

The processing unit 1021 can also be provided on board 45 trations ) of gases in a gas cloud can be illustrated on thethe data acquisition and processing module 1020 . The pro -cessing unit 1021 can include a processor 1023 and amemory 1022 . The processor 1023 can be in operable
cooperation with the memory 1022, which can contain adisplay, e .g ., by color coding or other suitable illustration

scheme. The processed data can overlie a visible image ofthe scene in some arrangements. In some embodiments, thewireless communication module can provide data commu

computer - readable code that, when loaded onto the proces- 50 nication between the system 1000 and an external device

sor 1023 , enables the processor 1023 to acquire multispec tral optical data representing a target species of gas or

remote from the system 1000 , such as a central server. For
example , the processed image data and/ or the raw image

chemical from IR radiation received at the optical FPA unit

data may be transmitted over a telecommunications networkof the optical system 1015 . The memory 1022 can be any

to the central server for storage and/or further analysis. In

suitable type ofmemory ( such as a non -transitory computer - 55 some embodiments , the processed or raw image data can bereadable medium ) that stores data captured by the optical
uploaded to the mobile device ( e . g ., notebook computer,system 1015 and /or processed by the processing unit 1021 .

smartphone , tablet computing device, etc.), which can in

The memory 1022 can also store the software that isturn communicate the image data to the central server .

executed on the processor 1023 . The processor 1023 can be
TheGPS module 1025 can be configured to determine theconfigured to execute software instructions that process the 60 location of the data acquisition and processing module 1020
multispectral image data captured by the optical system

at a particular time. The processing unit 1021 can store the1015 . For example , the processor 1023 can analyze thelocation data and can associate the location data with adifferent images detected by the FPA and can compare theparticular image captured by the optical system 1015 in

captured data with known signatures of various types ofsome arrangements . The location data associated with thegases or chemicals . Based on the analysis of the captured 65 captured images can be transmitted by the communication
image data , the processor can be programmed to determine module 1024 ( or by an external device ) to a central server inthe types and concentrations of gases in a gas cloud . Further,some arrangements .

US 9 ,756 ,263 B2
The optical system 1015 , the processing unit 1021, thepower supply 1026 , the communication module 1024 , andor the GPS module 1025 may be contained or housed in thedata acquisition and processing module 1020 , which can becloud , chemical spill , etc . The components of the opticalsystem 1015 shown in FIG . 13A may be similar to or thesame as the components of the optical systems and devices

explained herein with respect to FIGS. 1- 10C . The opticalcarried or worn by the user. The components of the system 5 system 1015 can include a focal plane array (FPA ) unit 1008

1000 (including the optical components , the processing

configured to record infrared image data captured by theadvantageously be uncooled , e.g ., devoid of a cooling sys
ing module 1020 and may occupy a volume less than about tem .
300 cubic inches, less than about 200 cubic inches, or less 10 The optical system 1015 can include a front window 1006
than about 100 cubic inches. In various embodiments , thethrough which light from the object 1007 passes. A firstcomponents of the system 1000 (including the optical com

components , and the communications components ) may be
packaged or assembled in the data acquisition and processsystem 1000 . As shown in FIG . 13A , the FPA unit 1008 may

ponents, the processing components, and the communica

moveable blackbody source 1003 and a second moveabletions components ) may be packaged or assembled in the blackbody source 1004 can be provided to enable calibration
may 1515 01of the optical system 1015. The moveable sources 1003 ,data acquisition and processing module 1020 andd may
occupy a volume greater than about 2 cubic inches, or 1004 can be moved in front of the field of view such that thegreater than about 16 cubic inches . A power supply , includ - optics image these sources for calibration . For example, theing a battery and/ or solar module may also be included
first and second blackbody sources 1003, 1004 can beamong the components packaged or assembled in the data
maintained at different known temperatures in a stableacquisition and processing module 1020 and fit into the 20 manner. For example , a heater and a temperature sensor canabove- referenced volumetric dimensions.

be attached to each blackbody source 1003 , 1004 to provide

The data acquisition and processing module 1020 (withthe system components mounted therein or thereon , includ -

feedback to create a stable and known temperature differ
ence ( e .g ., at least 50 mK in some arrangements ) between

ing the imaging optics, focal plane array , and on board

different spatial regions of the sources.

within a box - shaped boundary having dimensions XxYxZ .

dynamic calibration apparatus to dynamically calibrate theFor example, in some embodiments, the data acquisition andsystem 1000. As shown in FIG . 13A , one or more calibration

processing electronics may ) may be sized and shaped to fit 25
processingmodule 1020 ( including the imaging optics, focal
plane array , and on board processing electronics may ) can be

In addition , the optical system 1000 can include asources 1009 , 1010 can be provided . The calibration sources1009 , 1010 can comprise a thermal electrically controlled

sized and shaped to fit within a box - shaped boundary 30 ( TEC ) material with a temperature sensor attached thereto .

smaller than 8 inchesx6 inchesx6 inches. In some embodi-

The calibration sources 1009 , 1010 can be coated with aments , the data acquisition and processing module 1020 spectrally measured coating or paint. Light from the cali(including the imaging optics , focal plane array , and on
bration sources 1009, 1010 can be reflected from one orboard processing electronics may ) can be sized and shaped more mirrors 1005 and directed through the lens array 1002
to fit within a box -shaped boundary smaller than 7 inchesx5 35 (described below ) to be imaged on a portion of the FPA unit
inchesx5 inches . In some embodiments, the data acquisition
1008 to assist in dynamically calibrating the system 1000
and processing module 1020 ( including the imaging optics, (e .g., while imaging of the target gas cloud is simultaneously
focal plane array, and on board processing electronics may
being imaged ) .
can be sized and shaped to fit within a box -shaped boundary
The optical system 1000 can include a lens array 1002 tosmaller than 6 inchesx4 inchesx4 inches. In some embodi- 40 focus the incoming light onto the FPA unit 1008 . As shownments, the data acquisition and processing module 1020 in FIG . 13A , each lens of the lens array 1002 can at least( including the imaging optics, focal plane array, and on partially define or be included in an optical channel to beboard processing electronics may) can be sized and shaped
imaged by the FPA unit 1008 . To improve the mobility andto fit within a box -shaped boundary having dimensions portability ofthe mobile imaging system 1000 , the lens arraylarger than 4 inches by 2 inchesx2 inches . In some embodi- 45 1002 can comprise an integrated unit formed from orments, the data acquisition and processing module 1020
(including the imaging optics , focal plane array , and on

assembled into a single unitary body. Such an integrated lensarray 1002 can reduce the size of the imaging system 1015 ,board processing electronics may ) can be sized and shaped

and therefore, the size of the system 1000 , to at least

to fit within a box - shaped boundary having dimensions partially enable the system 1000 to be worn or carried by thelarger than 2 inches by 1 inchesx1 inches. A power supply , 50 user. The lens array 1002 can be monolithically formed inincluding a battery and/ or solar module, a communications
any suitable manner. For example , in various embodiments ,module, or both may be included in the data acquisition and the lens array 1002 can be formed by a diamond milling tool.
processing module 1020 and fit into the above -referenced
In some embodiments, the lens array 1002 can comprise adimensions. It should be appreciated that the dimensions monolithic piece of transparentmaterial which has separate
disclosed herein may not correspond to the directions shown 55 regions shaped into curved refractive surfaces for creatingin FIG . 11A with respect to X , Y, and Z . Moreover, the separate lenses. In some embodiments, the lenses can besystem 1000 can have a mass and weight sufficiently small

inserted into an array of openings formed in a plate orso as to enable the user 1275 to easily carry or wear the data

substrate to create the lens array 1002 .

acquisition and processing module 1020 at the site .

The optical system 1000 can also include an array of

FIG . 13A is a schematic system diagram of an optical 60 infrared ( IR ) filters 1001 configured to filter wavelengths ofsystem 1015 configured to be used in the mobile infrared infrared light in an appropriate manner. Examples of IR

imaging systems 1000 disclosed herein , according to various

filters and filtering techniques are disclosed herein , forshown in FIG . 13A can be provided in the data acquisition

13A , the IR filters 1001 can be disposed between the lensembodiments. As explained herein , the optical system 1015

example , with respect to FIGS. 5A -6D . As shown in FIG .

and processing module 1020 to be worn or carried by the 65 array 1002 and the FPA unit 1008 . The IR filters 1001 canuser. The optical system 1015 can be configured to capture
at least partially define the multiple optical channels to be
multispectral image data of an object 1007 , such as a gas imaged by the FPA unit 1008 . In some embodiments , the IR

US 9,756 ,263 B2
43

44

filters 1001 can be positioned between the lens array 1002
and the first and second moveable blackbody sources 1009 ,1010 .

the aperture of the system 1000 . In some embodiments , thewindow 1506 comprises germanium . A diamond like coat

FIG . 13B is a schematic system diagram of an opticalsystem 1015 configured to be used in the mobile infrared 5
imaging systems 1000 disclosed herein , according to various
embodiments . As explained herein , the optical system 1015
shown in FIG . 13B can be provided in the data acquisitionand processing module 1020 to be worn or carried by theing (DLC ) or other coating or layer can be disposed over thewindow 1506 to provide a durable surface .

As explained herein , the system 1000 can be configured
to be worn or carried by a human user. Accordingly , the dataacquisition and processing module 1020 can be suitably
dimensioned such that a user can easily wear or carry thesystem 1000 . For example , the data acquisition and processuser . The components of the optical system 1015 shown in 10 ing module 1020 can be defined at least in part by dimen

FIG . 13B may be similar to or the same as the components

sions XxYxZ , as shown in FIGS. 14A and 14B .

respect to FIGS . 1 - 10C and 13A .

are bulky or are assembled over a large form factor, themobile system 1000 can be sized and shaped in such aof the optical systems and devices explained herein with
The optical system 1015 of FIG . 13B can include an FPA

Unlike other systems, in which the system components

unit 1408 configured to image an object 1409 , such as a gas 15 manner so as to be easily moved and manipulated when thecloud or chemical leak . As with the embodiment illustrated usermoves about the site . Indeed , it can be very challenging
in FIG . 13A , the system 1015 of FIG . 13B can include ato integrate the various system components in a small
front window 1406 through which light from the object 1409 form - factor. Advantageously, the systems 1000 disclosed
passes, first and second moveable blackbody sources 1403, herein can be worn or carried by a human user. For example,
1404 , an IR filter array 1401 , and a lens array 1402 . Aswith 20 the components of the system 1000 can be contained
the embodiment of FIG . 13A , the lens array 1402 cantogether in the data acquisition and processingmodule 1020 ,comprise a unitary or monolithic body . In the embodiment

which may include the housing 1590 to support the system

of FIG . 13B , the lens array 1402 may be disposed between

components . The components of the system 1000 ( including

the filter array 1401 and the FPA unit 1408 . In otherthe optical or imaging components , the focal plane array , thearrangements , the filter array 1401 may be disposed between 25 on -board processing electronics , and the communications

the lens array 1402 and the FPA unit 1408 .

The optical system 1015 of FIG . 13B can include a
cooling unit 1430 configured to cool the FPA unit 1408 . The

components) may be packaged or assembled in the dataacquisition and processing module 1020 and may occupy a
volume less than about 300 cubic inches , less than about 200

cooling unit 1430 can comprise a cooling finger configured
cubic inches, or less than about 100 cubic inches. In various
to cryogenically cool the FPA array 1408 in various arrange - 30 embodiments , the components of the system 1000 (includ

ments. As shown in FIG . 13B , the filter array 1401, the lensarray 1402, and the FPA unit 1408 can be disposed in acooled region 1440. The blackbody sources 1403 , 1404 andfront window 1406 can be disposed in an uncooled region

ing the optical or imaging components, the focal plane array ,the on -board processing electronics , and the communica
tions components ) may be packaged or assembled in thedata acquisition and processing module 1020 and may

1402, and FPA unit 1408 at cooled temperatures can assist

thethe components of the systemments . FIG . 14B is an enlarged schematic perspective viewof the mobile infrared imaging system 1000 shown in FIG .

range of about 4 cubic inches to about 15 cubic inches . In
some embodiments , the components of the system 1000

14A . The helmet 1200 can comprise a portion of a user' s( including the optical or imaging components, the focal

be used as a platform for the imaging system 1000 . As
explained above , the helmet 1200 can be worn by a user ascommunications components ) may be packaged or
assembled in the data acquisition and processing module

1450. Disposing the blackbody sources 1403 , 1404 at 35 occupy a volume greater than about 2 cubic inches , or
uncooled temperatures and the filter array 1401 , lens arraygreater than about 16 cubic inches . In some embodiments ,1000 ( including the optical or

imaging components , the focal plane array, the on -board
in the periodic calibration of the system 1000 .
FIG . 14A is a schematic perspective view of a mobile processing electronics, and the communications compo
infrared imaging system 1000 (e . g ., mobile DAISI system ) 40 nents ) may be packaged or assembled in the data acquisition
mounted to a helmet 1200 , according to various embodi
and processing module 1020 and may occupy a volume in apersonal protective equipment and can also advantageously 45 plane array , the on -board processing electronics , and thethe user visits a particular installation site to be monitored ,1020 and may occupy a volume in a range of about 5 cubic
such as an oil well site , a refinery , etc . The system 1000 can
inches to about 12 cubic inches. In some embodiments , thebe activated to continuously monitor and analyze the sites 50 components of the system 1000 ( including the optical orthat the user visits. The system 1000 can thereby continu

imaging components , the focal plane array, the on -board

ously and actively search for gas leaks wherever the user

processing electronics, and the communications compo

visits and can initiate an alarm or other notification if a leakis detected .

nents ) may be packaged or assembled in the data acquisition

system 1000 can comprise a housing 1590 , within or towhich a data acquisition and processing module 1020 (see ,and processing module 1020 and may occupy a volume in aabout 5 .63 cubic inches in one embodiment. In someembodiments , the components of the system 1000 (includ

In the embodiment illustrated in FIG . 14B , the imaging 55 range of about 4 cubic inches to about 6 .5 cubic inches, e . g .,e . g ., FIG . 12 and associated description ) is mounted or

ing the optical or imaging components , the focal plane array,

coupled . A support 1592 can be coupled to or formed withthe on -board processing electronics, and the communica

helmet 1200 or to any other suitable platform . For example ,in some embodiments, the support 1592 can include one ordata acquisition and processing module 1020 and may
occupy a volume in a range of about 9 cubic inches to about

more mounting holes for attaching to the helmet 1200 by

13 cubic inches, e .g., about 11 .25 cubic inches in one

be transparent to IR radiation and can at least partially define

and the communications components ) may be packaged or

the housing 1590 and can be configured to attach to the 60 tions components ) may be packaged or assembled in theway of, e . g ., screws, bolts, or other fasteners . In addition , asembodiment. In some embodiments, the components of theshown in FIG . 14B , a front window 1506 can be provided at 65 system 1000 (including the optical or imaging components ,a front end of the housing 1590 . The front window 1506 canthe focal plane array , the on -board processing electronics ,US 9 ,756 ,263 B2
45

46

assembled in the data acquisition and processing module(including the imaging optics, focal plane array, and on

1020 and may occupy a volume in a range of about 6 cubic

board processing electronics may ) can have a volumemore

inches to about 10 cubic inches.
The data acquisition and processing module 1020 (with

5 cubic inches. The data acquisition and processing module1020 (including the imaging optics, focal plane array, and on

the system components mounted therein or thereon ) may be 5 board processing electronics may ) can have a volumemore

sized and shaped to fit within a box -shaped boundary having
dimensions XxYxZ . For example , the data acquisition and

10 cubic inches . This package may also contain a power
supply , including a battery and /or solar module , a commu

processing module 1020 , including the imaging optics , focal n ications module , or both and fit into the above - referenced

plane array, and on board processing electronics may bedimensions. It should be appreciated that the dimensions

included in a package that is sized and shaped to fit within 10 disclosed herein may not correspond to the directions shown

the box -shaped boundary having dimensions XxYxZ . Thisin FIG . 11A with respect to X , Y , and Z . This package may

and/ or solar module . In some embodiments, the data acqui-

module , a communications module , or both and fit into the

package may also contain a power supply, such as a battery

sition and processing module 1020 (including the imagingmay ) can be sized and shaped to fit within a box -shaped

also contain a power supply, including a battery and /or solar

above -referenced dimensions. It should be appreciated that
directions shown in FIG . 11A with respect to X , Y, and Z .

optics , focalplane array, and on board processing electronics 15 the dimensions disclosed herein may not correspond to theboundary smaller than 8 inchesx6 inchesx6 inches . In someembodiments, the data acquisition and processing module1020 ( including the imaging optics, focal plane array , and on

In some embodiments , the dimension X shown in FIG .

14B can be in a range of about 2 inches to about 7 inches ,or more particularly, in a range of about 2 inches to about 4

board processing electronics may ) can be sized and shaped 20 inches , e . g ., about 2 . 5 inches in one embodiment. In someto fit within a box - shaped boundary smaller than 7 inchesx5
embodiments , the dimension X shown in FIG . 14B can be in

inchesx5 inches. In some embodiments , the data acquisitionand processing module 1020 (including the imaging optics,
focal plane array , and on board processing electronics may

a range of about 4 inches to about 6 inches, e. g., about 5

inches in one embodiment. In some embodiments , thedimension Y shown in FIG . 14B can be in a range of about

can be sized and shaped to fit within a box -shaped boundary 25 1 inch to about 5 inches, or more particularly, in a range ofsmaller than 6 inchesx4 inchesx4 inches. In some embodi
about 1 inch to about 3 inches, e .g ., about 1. 5 inches in one

ments, the data acquisition and processing module 1020
(including the imaging optics , focal plane array , and on

embodiment. In some embodiments, the dimension Z shownin FIG . 14B can be in a range of about 1 inch to about 5

board processing electronics may ) can be sized and shaped
inches, or more particularly , in a range of about 1 inch toto fit within a box - shaped boundary smaller than 6 inchesx2 30 about 3 inches , e. g ., about 1.5 inches in one embodiment.
inchesx2 inches . In some embodiments, the data acquisition
Moreover, the system 1000 can have a mass and weightand processing module 1020 ( including the imaging optics , sufficiently small so as to enable the user 1275 to easily carry
focal plane array , and on board processing electronics may
or wear the data acquisition and processing module 1020 atcan be sized and shaped to fit within a box - shaped boundary the site . For example , the system 1000 can have a weight inhaving dimensions larger than 4 inchesx2 inchesx2 inches . 35 a range of about 0 . 5 pounds to 5 pounds, or more particu
In some embodiments, the data acquisition and processing larly, in a range of about 0 .5 pounds to 2 pounds , or more

module 1020 ( including the imaging optics , focal planeparticularly in a range of about 0 .25 pounds to about 2

array, and on board processing electronics may ) can be sized

pounds, or more particularly , in a range of about 0 .25 pounds

and shaped to fit within a box -shaped boundary having

to about 1. 5 pounds. In one embodiment, for example, thedimensions larger than 2 inchesx1 inchesx1 inches. The data 40 system 1000 can weight about 1 pound . In another embodi
acquisition and processing module 1020 (including the ment, for example , the system 1000 can weigh about 0 .5
imaging optics, focal plane array, and on board processing pounds . Thus, the embodiment shown in FIG . 11A can beelectronics may ) can have dimensions less than 3 inchesx2
sized and shaped and configured to have a mass that enables
inchesx2 inches . The data acquisition and processing mod - a human user to easily and effectively manipulate the systemule 1020 (including the imaging optics, focal plane array, 45 1000 .
and on board processing electronics may ) can have dimen FIG . 14C is a perspective cross -sectional view of thesions greater than 1 inchesx0 . 5 inchx0 .5 inch . The data mobile infrared imaging system 1000 shown in FIGS. 14A

acquisition and processing module 1020 ( including the 14B . The mobile infrared imaging system 1000 can include
imaging optics, focal plane array, and on board processing one or more movable shutters 1503 (e .g ., two shutters ) rearelectronics may ) can have a volume less than 30 cubic 50 of the window 1506 and a lens assembly 1502 rear of theinches . The data acquisition and processing module 1020
shutter (s ) 1503. A filter array 1501 can be disposed rear (or(including the imaging optics, focal plane array, and on
forward ) of the second lens array 1502B , and an opticalboard processing electronics may ) can have a volume lessfocal plane array ( FPA ) unit 1508 can be disposed rear of thethan 20 cubic inches . The data acquisition and processing
filter array 1501 . The optical FPA unit 1508 can bemechani
module 1020 ( including the imaging optics, focal plane 55 cally and electrically coupled with one or more substrates
volume less than 15 cubic inches. The data acquisition andarray, and on board processing electronics may ) can have aprocessing module 1020 (including the imaging optics, focal

1586 , which may comprise printed circuit board or PCB
comprises a single FPA or detector array. Additionally , asplane array, and on board processing electronics may ) can

explained herein , the lens assembly 1502 , filter array 1501,

substrates . In various embodiments , the FPA unit 1508

have a volume less than 10 cubic inches . The data acquisi- 60 and optical FPA unit can at least partially define one or moretion and processing module 1020 ( including the imaging

optical channels that are spatially and spectrally different. A

optics, focal plane array, and on board processing electronics

number of the optical channels can be at least 4 , at least 5 ,acquisition and processing module 1020 (including thesome embodiments , a number of the optical channels ismay ) can have a volumemore than 1 cubic inches . The data

at least 8 , at least 9 , at least 12 , at least 13 , or at least 20 . In

imaging optics , focal plane array , and on board processing 65 between 4 and 50 .
electronics may ) can have a volume more than 4 cubic
One or more batteries 1588 can supply power to theinches . The data acquisition and processing module 1020

system 1000 by way of the substrate (s ) 1586 . In addition, aUS 9 ,756 ,263 B2
47

48

visible light imaging sensor 1580 can be disposed in the
housing 1590 and can be configured to provide a visible lightimage of the scene being captured by the system 1000. The

or greater than 6 microns and possibly lower than 20
microns , or 15 microns, or 13 microns, or 12 microns or 10
microns , in wavelength , and may be cooled or uncooled . In

1eprocessed IR image data can be overlaid upon the visible

some embodiments the optical FPA unit 1508 comprises one

light image. In various embodiments the visible light imag - 5 or more microbolometer arrays, which may be uncooled . For

ing sensor 1580 can be used for reduction of scene-motion induced detection errors , for example , to detect a moving
object that enters the field of view (such as an animal or

person ) and would interfere with the data being collected .

example , an array of about 1000x1000 microbolometer
arrays may be used in the embodiments disclosed herein .
Microbolometer arrays such as those manufactured by DRS

Technologies of Arlington, Va ., and Sofradir EC , Inc., of

As explained herein , the movable shutter (s) 1503 can be 10 Fairfield , N .J., may be suitable for the embodiments disconfigured to provide spectral- radiometric calibration forthe system 1000 . The shutter( s ) 1503 can be configured toclosed herein . For example , the DRS U8000 FPA manufac
tured by DRS Technologies may be used in some embodi

move in and out of the field of view of the lens assembly

ments . In some arrangements , the microbolometer array may

1502 periodically, e. g., in a time period in a range of about

have a resolution of 1024x768 with a pixel pitch of 12

14C , it should be appreciated that two or more shutters may

pixel array with on the detector array (microbolometer

bration is performed , e .g ., no dynamic calibration is per
formed . In some embodiments, both static and dynamic

smaller than 12 and the detection are on the detector arrayfor a single channel may be larger or smaller than 250x250
pixels . For example , the detection region may comprise

1 minute to about 15 minutes , or more particularly, in a 15 microns. The array of lenses can form separate channels
range of about 3 minutes to about 7 minutes , e .g ., about 5
having image detection regions that form part of the array .
minutes. Although one shutter 1503 is illustrated in FIG . For example , 12 channels can be included in the 1024x768

be provided . The shutter( s ) 1503 can be used in static
array ) that are for example 250x250 pixels for each of the 12
calibration procedures to provide the system with absolute 20 channels. Detector arrays having more or less pixels may betemperature values . In some embodiments , only static caliemployed . Similarly the number of channels be larger orcalibration procedures are performed .
The lens assembly 1502 can include a first lens array 25 from between 100 - 200 pixelsx100 - 200 pixels per detection
1502A and a second lens array 1502B . In some embodi- region , For example , the detection region may comprisements, the lens assembly 1502 can comprise an array of
from between 100 - 200 pixelsx100 - 200 pixels per detection

two - part lenses denoted by the first and second arrays region , from between 200 - 300 pixelsx200 - 300 pixels per
1502A , 1502B . In some embodiments, the lens assembly detection region , or from between 300 -400 pixelsx300 -400
1502 can comprise an array of two separate lenses denoted 30 pixels or from between 400 - 500 pixelsx400 -500 pixels.

by the first and second arrays 1502A , 1502B . Each of thelens arrays 1502A , 1502B can comprise a 4x3 array of

Likewise the detection region for a channel may measure
100 - 200 pixels on a side , 200 -300 pixels on a side, 300 - 400

lenses, each of which may correspond to a particular detec -pixels on a side , 400 - 500 pixels on side or larger or smaller.

tor region in the FPA unit 1508 and can define an optical In some arrangements, the spectral band of the microbo
channel of the system 1000 . The lenses used in the first lens 35 lometer can be about 7 .5 microns to 14 microns . The
array 1502A may be different from the lenses used in the microbolometer array can operate at a frame rate of about 30
second lens array 1502B . The lenses can be any suitable type Hz and can operate at operating temperatures of about - 40°of lens, including, e . g ., spherical lenses , aspheric lenses , rod

C . to + 70° C . In various embodiments , the microbolometer

lenses , etc . or any combination thereof. For example , thearray is an uncooled microbolometer that does not include a

lenses used in the first lens array 1502A can comprise 40 cooler. The sensitivity of the microbolometer at F / 1 can beaspheric lenses , and the lenses used in the second lens array< about 40 mK . The systems 1000 disclosed herein can be

1502B can comprise rod lenses. Although the lens assemblyused to detect wavelengths in a range of about 1 micron to1502 shown in FIG . 15C includes two lens arrays , it should

about 20 microns. For example , the systems 1000 disclosed

be appreciated that additional lens arrays may be used , e.g .,herein can be used to detect wavelengths above about 6

three lens arrays , four lens arrays, five lens arrays, etc . In 45 microns, e . g ., in a range of about 6 microns to about 18

addition , to assist in enabling a small system size , thediameter of each lens in the assembly 1502 can be less thanabout 0 .5 " , e . g ., in a range of about 0 . 1 " to about 0 . 5 " . The
f-number of each lens can be less than about 2 , e .g., in a

microns, or more particularly , in a range of about 7 microns

to about 14 microns. In various embodiments , the individual
relatively close together to at least partially enable a small,detector elements ofthe microbolometer array can be spaced

range of about 0 .2 to 2 , or more particularly , in a range of 50 compact system . For example , adjacent detector elements of
the array can be spaced apart by a distance in a range ofabout 0 . 5 to 2 , or 1 .0 to 2 or 1 . 1 to 2 .

The first lens array 1502A and the second lens array

1502B can be coupled to one another by way of a mounting

about 7 microns to about 15 microns, or more particularly ina range of about 9 microns to about 13 microns , e . g ., aboutplate 1584 sized and shaped to support or receive each lens 11 microns. The individual lenses can be spaced apart by aarray 1502A , 1502B . For example, the first lens array 1502A 55 distance in a range of about 20 mm to about 35 mm , e. g . incan be mounted on one side of themounting plate 1584 , and a range of about 24 mm to about 30 mm , e .g ., about 27 . 5
the second lens array 1502B can be mounted on an opposite
mm . Likewise the spatially and spectrally spaced channels
side of the mounting plate 1584 . The mounting plate 1584 may be physically spaced apart by 20 to 35 mm , 24 mm tocan be machined to have diameter tolerances of about + / - 25 30 mm , etc . Although various embodiments of the system
microns . The lenses of the arrays 1502A , 1502B can be 60 are described as including an FPA comprising for example
secured to the mounting plate 1584 with a curable epoxy. a mircobolometer array , certain embodiments comprise a

For example, the lenses may fit into opposite sides of holes
formed in the mounting plate 1584.
The optical FPA unit 1508 can comprise any suitable type

plurality of FPAs. In some embodiments , a single optical

FPA is used . In some embodiments, detectors of the optical
FPA are configured to detect radiation in the same band ofof detector array that is configured to detect infrared radia - 65 IR wavelengths.

tion , for example , greater than 1 micron , or greater than 2

The on -board processing electronics of the data acquisi

microns , or greater than 3 microns or greater than 5 microns,tion and processing module 1020 can process the IR opticalUS 9 ,756 , 263 B2
49

50

data to detect and / or identify a target species from the IRradiation received at the optical FPA . For example , themodule 1020 can be configured to acquire multispectral
image data and analyze the acquired image data to identify

tions modules, etc . Advantageously , the mobile infrared
imaging systems 1000 disclosed herein can be configured tooperate by battery power for long periods of time without
recharging or replacing the batteries 1588 . In some arrange

the target species . For example , the mobile imaging systems 5 ments the one or more batteries 1588 can comprise lithium
1000 disclosed herein can be configured to image a 10 mx10
ion batteries , which have relatively high energy densities . Inm object area at a distance of about 17 m at a resolution of
about 0 .04 m . In this example , any gas leaks that generate aaddition , to help reduce power consumption within thesystem 1000 , the FPGAs of the data acquisition and pro

gas cloud of at least about 1 . 5 inches in size can be detectedcessing module 1020 can be advantageously programmed
and / or identified by the system 1000 . The detection and 10 such that power consumption is lower than that used for

identification methods can be performed substantially inreal- time such that the user can be alerted if any leaks are
identified .

As explained above, the infrared image data captured byother types of processing electronics .
The systems 1000 disclosed herein can advantageously

operate for between 8 hours and 36 hours withoutrecharging
or replacing the batteries, or more particularly between

the system 1000 can be processed on board the data acqui- 15 about 10 hours and 24 hours without recharging or replacing
sition and processing module 1020 of the imaging system
the batteries. In some embodiments , the system 1000 can

1000 . One way to provide a smaller system 1000 is tooperate for at least about 12 hours without recharging orprocess the image data using one or more field - program -replacing the batteries . The components of the data acquimable gate arrays (FPGA ) configured to execute methods
sition and processing module 1020 (including the imaging
used in the analysis of the images captured by the optical 20 optics , focal plane array , and on board processing electronics
system 1015 . In some embodiments , one or more Applica - may) can be configured to operate at relatively low electricaltion Specific Integrated Circuits (ASICs) may be used
power levels, e . g ., at power levels in a range of about 3 W

instead of, or in addition to , the FPGAs. For example, an

to about 10 W , or more particularly in a range of about 4 W

ASICs chip may include an FPGA . The FPGA ( S ) ( and /orto about 7 W , or in a range of about 4 W to about 6 W , e . g .,ASIC ( s )) can be mounted to and electrically coupled with 25 about 5 W in some embodiments. The components of thethe substrate (s ) 1586 shown in FIG . 14C and can be physi data acquisition and processing module 1020 ( including thecally located proximate the optical system . For example, theFPGA can include logic gates and read access memory

imaging optics , focal plane array , and on board processing
electronics may ) can also be configured to operate at rela

(RAM ) blocks that are designed to quickly implement the tively low overall energy levels for a single charge of thecomputations used to detect the types of gases in a gas cloud. 30 batteries 1588 , e. g ., at energy levels in a range of about 60
The small size /weight, and high performance characteristics

Watt -hours (Wh ) to about 100 Wh , or more particularly in aof the FPGA can enable on board computation and analysis

range of about 80 Wh to about 95 Wh , or in a range of about

within the data acquisition and detection unit 1020 worn or
carried by the user . The use of FPGA ( or similar electronics )85 Wh to about 90 Wh .
In addition , for each of the embodiments disclosed herein ,on board the system 1000 can reduce costs associated with 35 various motion detection and / or compensation techniques

using an off -site central server or larger computing device toconduct the image analysis computations. Advantageously ,the embodiments disclosed herein can enable on - board

computation even though it can be challenging to implement

can be implemented to account for relatively large -scale
motions that are induced by the user moving his or her head
during use . For example, when a user is visiting a well siteor other installation , the user may be continuously walking

complex methods on the limited computing platform that 40 and looking in different directions ( e . g ., by rotating his or

FPGAs provide.
In addition , enabling computation with one or more
FPGA devices on board the wearable system can also

her head ). Additionally, vibration can be introduced by theprevent or reduce communication bottlenecks associateduser 's natural unsteadiness . Such movement can continu
rate, which can affect the accuracy of the methods used tothe system 1000 to a remote server or computer. For
example , the infrared optical system 1015 disclosed herein

object. Accordingly , it can be desirable to provide improved
motion detection and/ or compensation techniques to reduce

ously change the system ' s field of view at a relatively rapid

with wirelessly transmitting large amounts of raw data from 45 determine the identity of species in a gas cloud or othermay generate up to about 380 Mbps of raw image data at 30
errors associated with the movements of the user .
frames per second , and the visible sensor 1580 may generate
Each of the embodiments disclosed herein can be used toabout 425 Mbps of raw image data at 30 frames per second . 50 estimate various characteristics of gases present in a gas leak

The resulting data rate of about 800 Mbps is faster than mostconventional wireless technologies. While data compression
and/ or pre - processing may reduce the raw data rates for thevisible and IR images , in some embodiments , the IR image

imaged by the infrared imaging systems disclosed herein .
References throughout this specification to " one embodi
ment," " an embodiment," " a related embodiment,” or simi
lar language mean that a particular feature , structure , ordata may only be compressed by a ratio of about 2 : 1 . The 55 characteristic described in connection with the referred toresulting overall data rate of about 192 Mbps may not betransmitted effectively by conventional wireless communi-“ embodiment” is included in at least one embodiment of thepresent invention . Thus, appearances of the phrases in one

cations devices . Accordingly , performing the image process -embodiment,” “ in an embodiment," and similar language

ing calculations on board the system 1000 ( e .g ., on the data

throughout this specification may, but do not necessarily , allacquisition and processing module 1020 ) can reduce the 60 refer to the same embodiment. It is to be understood that nooccurrence of or avoid bottlenecks generated by wirelessly
portion of disclosure , taken on its own and in possible
communicating the raw image data to an off -site centralconnection with a figure , is intended to provide a complete
server.
description of all features of the invention .

One challenge to implementing a mobile imaging systemin the drawings like numbers are used to represent theis the power requirements of each component of the system , 65 same or similar elements wherever possible . The depicted
including, e. g ., the IR optical system 1015 , the visible sensor
structural elements are generally not to scale , and certain

1580 , the processing electronics , the wireless communica

components are enlarged relative to the other components

US 9 ,756 , 263 B2

52

51
for purposes of emphasis and understanding. It is to be

information conveyed to a computer through communica

understood that no single drawing is intended to support acomplete description of all features of the invention . In otherwords, a given drawing is generally descriptive of onlytion media , including wired or wireless computer networks .
In addition , while the invention may be embodied in soft
ware , the functions necessary to implement the invention

some, and generally not all, features of the invention . A 5 may optionally or alternatively be embodied in part or ingiven drawing and an associated portion of the disclosure

whole using firmware and /or hardware components, such as

containing a description referencing such drawing do not

combinatorial logic , Application Specific Integrated Circuits

generally , contain all elements of a particular view or all
features that can be presented is this view , for purposes of(ASICs), Field -Programmable Gate Arrays (FPGAs) orother hardware or some combination of hardware , software

simplifying the given drawing and discussion , and to direct 10 and / or firmware components .

the discussion to particular elements that are featured in thisdrawing . A skilled artisan will recognize that the invention
may possibly be practiced without one or more of thespecific features, elements, components , structures, details,

While examples of embodiments of the system andmethod of the invention have been discussed in reference tothe gas - cloud detection , monitoring, and quantification (including but not limited to greenhouse gases such as Carbon

or characteristics, or with the use of other methods, com - 15 Dioxide, Carbon Monoxide, Nitrogen Oxide as well as

ponents , materials , and so forth . Therefore , although aparticular detail of an embodiment of the invention may nothydrocarbon gases such as Methane, Ethane, Propane, n -Bu
tane , iso -Butane, n -Pentane , iso - Pentane , neo -Pentane,

be necessarily shown in each and every drawing describing

Hydrogen Sulfide , Sulfur Hexafluoride , Ammonia , Benzene ,such embodiment, the presence of this detail in the drawing

p - and m - Xylene , Vinyl chloride , Toluene, Propylene oxide ,may be implied unless the context of the description requires 20 Propylene , Methanol, Hydrazine , Ethanol, 1,2 -dichloroeth
otherwise . In other instances , well known structures, details ,ane, 1, 1 -dichloroethane, Dichlorobenzene , Chlorobenzene,materials , or operations may be not shown in a given

to name just a few ) , embodiments of the invention can be

drawing or described in detail to avoid obscuring aspects ofreadily adapted for other chemical detection applications .

an embodiment of the invention that are being discussed .
For example , detection of liquid and solid chemical spills,
Furthermore , the described single features , structures, or 25 biological weapons, tracking targets based on their chemical

characteristics of the invention may be combined in any

suitable manner in one or more further embodiments .

Moreover, if the schematic flow chart diagram is included ,composition , identification of satellites and space debris ,ophthalmological imaging, microscopy and cellular imag

ing, endoscopy, mold detection , fire and flame detection , and

it is generally set forth as a logical flow -chart diagram . As
pesticide detection are within the scope of the invention .
such , the depicted order and labeled steps of the logical flow 30 As used herein , a phrase referring to " at least one of a list

are indicative of one embodiment of the presented method .

Other steps and methods may be conceived that are equiva lent in function , logic , or effect to one or more steps, or
portions thereof, of the illustrated method . Additionally , theof items refers to any combination of those items, including

single members. As an example, " at least one of: a , b , or c”
If implemented in software , the functions may be stored

is intended to cover : a , b , c , a - b , a - c , b - c , and a -b - c .

format and symbols employed are provided to explain the 35 on or transmitted over as one or more instructions or code on

logical steps of the method and are understood not to limit

the scope of the method . Although various arrow types andline types may be employed in the flow - chart diagrams, they
are understood not to limit the scope of the corresponding

a computer- readable medium . The steps of a method oralgorithm disclosed herein may be implemented in a pro
cessor -executable software module which may reside on acomputer -readable medium . Computer-readable media

method . Indeed , some arrows or other connectors may be 40 includes both computer storage media and communication

used to indicate only the logical flow of the method . For

media including any medium that can be enabled to transfer

instance , an arrow may indicate a waiting or monitoring
period of unspecified duration between enumerated steps of

a computer program from one place to another. A storage
media may be any available media that may be accessed by

the depicted method . Without loss of generality , the order in

a computer. By way of example, and not limitation , such

which processing steps or particular methods occurmay or 45 computer- readable media may include RAM , ROM ,may not strictly adhere to the order of the corresponding EEPROM , CD -ROM or other optical disk storage , magnetic
steps shown .
disk storage or othermagnetic storage devices, or any other
The features recited in claims appended to this disclosure medium that may be used to store desired program code inare intended to be assessed in light of the disclosure as a
the form of instructions or data structures and that may be
50 accessed by a computer. Also , any connection can be prop
whole .
At least some elements of a device of the invention can beerly termed a computer - readable medium . Disk and disc , ascontrolled — and at least some steps of a method of theinvention can be effectuated , in operation with a program -used herein , includes compact disc (CD ), laser disc , opticaldisc, digital versatile disc (DVD ), floppy disk , and blu -raymable processor governed by instructions stored in adisc where disks usually reproduce data magnetically , while

control software or other instructions and data . Those skilled

method or algorithm may reside as one or any combinationin the art should also readily appreciate that instructions oror set of codes and instructions on a machine readable

may be delivered to a processor in many forms, including,incorporated into a computer program product.

memory . The memory may be random access memory 55 discs reproduce data optically with lasers . Combinations of(RAM ), read - only memory (ROM ), flash memory or any the above also may be included within the scope of com
other memory, or combination thereof, suitable for storing puter -readable media . Additionally , the operations of aprograms defining the functions of the present invention 60 medium and computer - readable medium , which may be

but not limited to , information permanently stored on non
Variousmodifications to the implementations described inwritable storage media (e .g . read -only memory devices this disclosure may be readily apparent to those skilled in thewithin a computer, such as ROM , or devices readable by aart, and the generic principles defined herein may be appliedcomputer I/ O attachment, such as CD -ROM or DVD disks), 65 to other implementations without departing from the spirit orinformation alterably stored on writable storage media ( e. g .
floppy disks, removable flash memory and hard drives) or

scope of this disclosure . Thus, the claims are not intended tobe limited to the implementations shown herein , but are toUS 9 ,756 ,263 B2
5354

be accorded the widest scope consistent with this disclosure ,the principles and the novel features disclosed herein .
Certain features that are described in this specification in

6 . The system according to claim 5 , wherein said dataacquisition and processing module has a volume of less than
100 cubic inches.

7 . A method of identifying a target species or quantifying
the context of separate implementations also can be imple
mented in combination in a single implementation . Con - 5 or characterizing a parameter of the target species in anversely, various features that are described in the context of object, the method comprising:wearing or carrying a data acquisition and processing
module, the data acquisition and processing module
comprising an optical system and a processing unit incommunication with the optical system , the opticala single implementation also can be implemented in multiple

wcalmssuch , one or more features from a claimed combination canin some cases be excised from the combination , and theunit ;

implementations separately or in any suitable subcombina
tion . Moreover, although features may be described above as
acting in certain combinations and even initially claimed as 10

system including an optical focal plane array (FPA )an optical system including an optical focal plane array

capturing multispectral infrared (IR ) image data at the
FPA unit from at least two optical channels that are
spatially and spectrally different from one another; andacquiring multispectral optical data representing the target
species from the IR radiation received at the FPA .
8 . The method of claim 7 , further comprising analyzing

from one another, each of the at least two opticalphysically separate from said data acquisition and process

claimed combination may be directed to a subcombination
or variation of a subcombination .

What is claimed is:

1. An infrared (IR ) imaging system for imaging a target
species in an object, the imaging system comprising :15

15

the acquired multispectral optical data to identify the target
at least two optical channels thereof, said at least two 20 SP 9. The method of claim 8 , further comprising communi
optical channels being spatially and spectrally different(FPA ), the optical system having components defining

channels positioned to transfer IR radiation incident on

cating the identified target species to at least one device.
the optical system towards the optical FPA ; and De 25 ing10module. The system of claim 1, wherein said data acquisitiona processing unit containing a processor that can be
25 and processing
module has dimensions less than 2 inchesx2
configured to acquire multispectral optical data repre
inchesx6
inches.
senting said target species from the IR radiation
11 . The system of claim 1 , wherein said data acquisition
received at the optical FPA ,wherein said optical system and said processing unit are and processing module has dimensions greater than 1
contained together in a data acquisition and processing 30 inchesxl inchesx3 inches.

12 . The system of claim 1, further comprising a spectral
module configured to be worn or carried by a person .
shutter for calibration .
2 . The system according to claim 1 , wherein said data
13 . The system according to claim 5 , wherein said dataacquisition and processing module further comprise a com
munication module configured to communicate with at least acquisition and processing module has a volume of less thanrom said
said data
data acquisitionacquisition 35
35 20
20 cubic inches .
one device physically separate from
14 . The system according to claim 5 , wherein said dataand processing module .
acquisitionand processing module has a volume of more
3 . The system according to claim 1, wherein the data
than 5 cubic inches.
acquisition and processing module further comprises a
15 . The system according to claim 1 , wherein said data
power supply configured to be worn or carried by the person .

and processing module weighs between 0 .5 and
4 . The system according to claim 1, wherein said pro - 4040 acquisition
acq
5
lbs
.
cessing unit comprises one or more on board field program
16 . The system according to claim 1, wherein the data
mable gate arrays (FPGAs ) for data processing .
acquisitionand processing module is configured to operate
5 . An infrared (IR ) imaging system for imaging a targetat power levels in a range of about 3 W to about 10 W .
species in an object, the imaging system comprising:an optical system including an optical focal plane array 45 17 . The system according to claim 1 , wherein the data
(FPA ), the optical system having components defining acquisition and processing module is configured to operate
for a single charge of the batteries at energy levels in a range
at least two optical channels thereof, said at least two of
about 60 Watt- hours (Wh) to about 100 Wh .
optical channels being spatially and spectrally different
18. The system channels according to claim 1 ,wherein thefrom one another, each of the at least two opticalchannels are spaced apart by a distance in a range ofchannels positioned to transfer IR radiation incident on| 5050 optical
of
about
20
mm to about 35 mm .
the optical system towards the optical FPA ; and19 . The system of claim 1, wherein a number of thea processing unit containing a processor that can be
optical channels is between 4 and 50 .
configured to acquire multispectral optical data repre
20 . The system according to claim 1, wherein the pro
senting said target species from the IR radiation
received at the optical FPA ,wherein said data acquisition and processing module has
a volume of less than 300 cubic inches .

55 cessor is configured to analyze the acquired multispectral
optical data to identify said target species .
*

*

*

*

*