|HAO WATAT UT TIL EN ANTAANMANTE
USO10084975B2(12) United States Patent(10) Patent No.: US 10 ,084,975 B2(45) Date of Patent:

Kester et al.

(54 ) MOBILE GAS AND CHEMICAL IMAGING

References Cited
U . S . PATENT DOCUMENTS(56 )CAMERA(71) Applicant: REBELLION PHOTONICS, INC.,@

3, 841,763 A

Houston , TX (US)@(72 ) Inventors : Robert Timothy Kester, Pearland , TX(US); Nathan Adrian Hagen , Houston ,10 /1974 Lewis

4 , 134 ,683 A

1/ 1979 Goetz et al.

(Continued )FOREIGN PATENT DOCUMENTS

TX (US )2 365 866
2 787 303??

(73 ) Assignee: Rebellion Photonics, Inc ., Houston , TX@

camera size , weight, and power” , Laser Focus World , vol. 50 , Issue
01, Jan . 17 , 2014 , 6 pages. Also available at http ://www .ircameras.

(21 ) Appl. No.: 15 /623 ,942
3@(22 ) Filed : Jun . 15 , 2017
Prior Publication Data
(65 )US 2018 /0077363 A1 Mar. 15 , 2018com / articles/ advances -detectors -hot-ir -sensors -improve - ir - camera

size -weight-power/.

(Continued )Primary Examiner — David Porta
Assistant Examiner — Meenakshi Sahu
(74 ) Attorney, Agent, or Firm — Knobbe, Martens, Olson& Bear, LLP
ABSTRACT(57)
In one embodiment, an infrared (IR ) imaging system fordetermining a concentration of a target species in an object

Related U .S . Application Data
(63 ) Continuation of application No. 14 /700 ,791, filed on
Apr. 30 , 2015 , now Pat. No. 9,756 , 263 .
(Continued )is disclosed . The imaging system can include an opticalsystem including an optical focal plane array (FPA ) unit. The(51) Int . Cl.
H04N 5 /33

optical system can have components defining at least two( 2006 . 01)

optical channels thereof, said at least two optical channels( 2006 .01)

being spatially and spectrally different from one another.

(Continued )Each of the at least two optical channels can be positioned
towards the optical FPA. The system can include a process(52 ) U .S . CI.

to transfer IR radiation incident on the optical system

CPC ............. H04N 5 /33 ( 2013 .01) ; GOIJ 3 /0232(2013 .01) ; GOIJ 370256 ( 2013 .01) ;(Continued )(58 ) Field of Classification Search???

7 /2011

OTHER PUBLICATIONS
Adams, et al., “ Advances in Detectors: Hot IR sensors improve IR

Subject to any disclaimer, the term of thispatent is extended or adjusted under 35
U . S .C . 154 (b ) by 0 days .

GO1J 5 /02

9 / 2000(Continued )(US )@( * ) Notice:

Sep . 25 , 2018

. ... ..... . U

ing unit containing a processor that can be configured toacquire multispectral optical data representing said target

species from the IR radiation received at the optical FPA .
Said optical system and said processing unit can be con(Continued )GOIN
ITV 21/ 3504

(Continued )100

ObjectFront calcima na , 124:Spectady ked On
:

120
120

alews

Detector array 136

120 i

110

130

Roimaging long array , 128

4.Stresitew

fistini

WWW

Towards
processor 150

.

1280
120

130

Monsable rataresource , 160

US 10 ,Page
084,2975 B2
tained together in a data acquisition and processing module
configured to be worn or carried by a person .

20 Claims, 23 Drawing Sheets
Related U .S . Application Data
(60 ) Provisional application No .61/ 986 ,885 , filed on May
1 , 2014 , provisional application No.62/012 ,078, filedon Jun . 13 , 2014 , provisional application No .
62/ 054 , 894 , filed on Sep . 24 , 2014 , provisional
application No. 62 /055 ,342 , filed on Sep . 25 , 2014 ,provisional application No . 62 /055 ,549, filed on Sep .
25 , 2014 , provisional application No. 62 /082,613 ,filed on Nov. 20 , 2014 , provisional application No .
61/ 986 ,886 , filed on May 1 , 2014 , provisional
application No . 62 /082,594 , filed on Nov . 20 , 2014 ,provisional application No. 62 /021,636 , filed on Jul.
7 , 2014 , provisional application No. 62/021, 907, filedon Jul. 8 , 2014 , provisional application No .

6 ,456 , 261 B1 9 /2002 Zhang
6 ,465,785 B1 10 / 2002 McManus
6 , 556, 853 B1 . 4 /2003 Cabib et al.
6, 680,778 B2

1/2004 Hinnrichs et al.

6 , 700, 527 B1
7 ,109 ,488 B2
7 ,119,337 B1

3 /2004 Martin et al .
9 / 2006 Milton
10 / 2006 Johnson et al.

8 ,373,757 B1
8,653,461 B1

2/ 2013 Nguyen
2 /2014 Benson et al .

9 ,756 , 263 B2 *9 /2017 Kester

7 , 242, 478 B1 7/ 2007 Dombrowski et al.
7 , 315 ,377 B21 /2008 Holland et al.
7 , 433 ,042 B1 . 10 / 2008 Cavanaugh et al.
7 ,750, 802 B17 / 2010 Parish et al.
7 ,835 ,002 B2 11/2010 Muhammed et al.
7 ,888,624 B1 . 2 /2011 Murguia et al.
8 , 027, 041 B1
9 / 2011 Mitchell et al.
8 , 159, 568 B2 4 / 2012 Ahdoot
8 ,212 ,213 B2
7 /2012 Myrick et al.
8 , 654, 328 B2 2 / 2014 Tkaczyk et al.
9 , 395, 516 B2 7 /2016 Katsunuma et al.
9 ,562,849 B2
2 / 2017 Kester et al.
9 , 599 ,508 B2
3 /2017 Kester et al.
9, 625,318 B2
4 /2017 Kester et al .
9, 641 , 772 B2
5 /2017 Yujiri
9 ,644 , 562 B25 / 2017 Fujita

2001/0040216 AL 11/2001 Knauth et al.

(51) Int. Cl.
GO1J 3 / 02

(2006 .01)

G06K 9 /22( 2006 .01)
( 2006 .01)

2002/0015151 AL 2 /2002 Gorin
2002/0121370 A1 9 / 2002 Kurkjian et al.
2002/0159101 Al 10 /2002 Alderson et al.
2003/0102435 A1 6 / 2003 Myers et al.
2003/0134426 A1 7 /2003 Jiang et al.
2003/0183756 A
10 /2003 Huniu
2004 / 0093167 Al 5 /2004 Braig et al.

( 2006 .01)
( 2006 .01)
( 2006 .01 )2005/0103989 Al

62/083 , 131, filed on Nov. 21, 2014 .

GO1J 3 /36
GOIJ 5 /00

2004 /0111232 Al
2004 /0252300 A1

6 /2004 Butler et al.
12/2004 Slater(2006 .01)

2006 / 0044562 A1

3 / 2006 Hagene et al .

2006 /0183241 Al

8 /2006 Lehmann et al.

GO1J 3 /28( 2006 .01 )( 2006 .01 )( 2006 .01)

GOIN 21/ 3504

( 2014 .01)

2006 /0232675 Al 10 /2006 Chamberlain et al.
2006 /0279632 A1 12 / 2006 Anderson
2007 /0018105 Al 1/ 2007 Grimberg
2007 /0075888 Al 4 / 2007 Kelly et al.
2007/0108385 A1 5 /2007 Mantese et al .
2008/0170140 A1 7 / 2008 Silver et al.

GO1J 5 /08
GO6K 9 /00

H04N 5 /225
GO2B 5 /20

G06K 9 /20(52) U .S . CI.
CPC .

...... GO1J 3 /2803 (2013 .01) ; GO1J 3 / 36

(2013 .01) ; GOIJ 5 /0014 ( 2013 .01) ; GOIJ

5 / 0834 (2013 .01 ); GOIN 21 /3504 (2013. 01 );GO2B 5 /201 ( 2013 .01 ); GO2B 5 /208
( 2013 .01 ) ; G06K 9 /00624 ( 2013 .01) ; G06K
9 /00664 ( 2013.01); G06K 9 / 209 (2013 .01) ;GO6K 9 / 2018 ( 2013 .01) ; G06K 9 /22( 2013 .01) ; H04N 5 /2258 ( 2013 .01); GOIJ
2003/ 2826 (2013 .01) ; GOIJ 2005/0077(2013 .01); GOIN 2021/3531 (2013 .01) ; GOIN
2201 /0221 (2013.01 )(58 ) Field of Classification Search
USPC .................
............ 250 /339 .02
See application file for complete search history.

(56)

References Cited
U .S . PATENT DOCUMENTS
4 ,963,963 A

5 , 127 ,742 A
5 , 354 ,987 A
5 ,550, 373 A
5 ,559,336 A
5 ,877, 500 A
5 ,920, 066 A
5 ,926 ,283 A
6 ,023 ,061 A
6 ,184,529 B1
6 , 268,883 B1
5 , 157 ,258 A

10 / 1990 Dorman
7 / 1992 Fraden

10 / 1992 Gunning , III et al.
10 / 1994 MacPherson
8/ 1996 Cole et al.
9 / 1996 Kosai et al.
3 / 1999 Braig et al.
7 / 1999 DiRenzo et al.
7 / 1999 Hopkins

2 / 2000 Bodkin
2 / 2001 Contini

7 / 2001 Zehnder et al.

5/ 2005 Watson et al.

2008/ 0204744 Al

8 /2008 Mir et al.

2008/0231719 Al

9 /2008 Benson et al.

2008 /0251724 Al

10 /2008 Baliga et al.

2009 /0252650 Al
2010 /0162206 AL

10/2009 Lakshmanan
6 /2010 Roth et al.

2010 /0171866 A1
2010 / 0309467 A1

2010 /0211333 A1

2011 /0176577 A1
2011 /0185048 A1

G06K 9 /22

7 / 2010 Brady et al.
8 /2010 Pruet et al.
12 / 2010 Fox et al.

7/ 2011 Bandara et al.
7/2011 Yew et al.

2011/0261321 Al 10 /2011 Ramella -Roman et al.
2012 /0273680 AL

11/2012 Furry

2013 /0181836 A1
2013 /0206990 A1
2013/0228887 A1

7 /2013 Cardoso et al .
8 / 2013 Hsu et al .
9 / 2013 Wehner et al.

2013/0235256 Al

9 /2013 Kodama

2013/0307991 A1

11/ 2013 Olsen et al.

2013/0321806 Al

12 /2013 Kester et al.

2013 /0341509 Al 12/2013 Nelson et al.
2013 / 0342680 A112 / 2013 Zeng et al.
2014 /0139643 Al 5 /2014 Högasten et al.
2015 /0069239 A1

3 /2015 Kester et al.

2015 /0136981 A1
2015 /0136982 A1

5 / 2015 Kester et al.
5 / 2015 Kester et al.

2015 /0138534 A1

5 /2015 Tidhar

2015 /0144770 A1 5/2015 Choi
2015 /0288894 A1 10 / 2015 Geelen et al.
2016 / 0037089 Al 2 /2016 Silny et al.
2016 / 0041095 Al 2 /2016 Rothberg et al.
2016 / 0097713 Al 4 / 2016 Kester et al.
2016 / 0238454 A1 8 / 2016 Pillans
2016 /0245698 AL 8 /2016 Pau et al.

2016 /0313181 A1 10 /2016 Golub et al.

2016 /0349228 A1 12 / 2016 Kester et al.

US 10 ,Page
084,3975 B2
References Cited( 56 )

Eriksson et al., “ Radiative Cooling Computed for Model Atmo
spheres” , Applied Optics, Dec . 1, 1982, vol. 21, No. 23, pp .

U . S . PATENT DOCUMENTS
2016 /0380014 A1 12/ 2016 Ganapathi et al.

4381 -4388 .

2017 /0205290 AL 7 / 2017 Kester et al.
2017 /0350758 A1 12/2017 Kester et al.
2017 /0356802 Al 12 / 2017 Kester et al.

4253 - 4260
Gålfalk et al., “Making Methane Visable ” , Nature Climate Change,

2017/0026588 AL

1/ 2017 Kester et al .

FOREIGN PATENT DOCUMENTS
??

EP

EP
EP
EP

EP
GB
GB
WO
WO

WO
WO
WO

WO
WO
WO

WO
WO
WO

2 870 419
0 837 600
2 871 452
2 942 615

11 / 2015

2 955 496

12 /2015

3 040 706
10147692518224

WO 2004 /097389

WO 2007/008826
WO 2008/ 109183
WO 2009/094782
WO 2010 /053979
WO 2012 /078417

5 /2015
4 / 1998
5 /2015

7 /2016

12 / 1965
3 / 2015
11/ 2004
1 / 2007
9 /2008
8 /2009

5 / 2010

6 /20126 / 2012

Flanigan , “ Detection of Organic Vapors with Active and Passive

Sensors: A Comparison ,” Applied Optics, 1986 , vol. 25 , No. 23, pp.
Apr. 2016 , vol. 6 , pp . 426 -430 .
Gålfalk et al., “MakingMethane Visable” , Supplementary Informa
tion , Nature Climate Change , 2015 , pp . 1 - 14 .
Gallagher et al., “ Error Analysis for Estimation of Trace Vapor
Concentration Pathlength in Stack Plumes” , Applied Spectroscopy ,2003, vol. 57 , No . 6 , pp . 614 -621 .
Gallagher et al., " Estimation of Trace Vapor Concentration
Pathlength in Plumes for Remote Sensing Applications from
Hyperspectral Images” , Analytica Chimica Acta, 2003 ,vol. 490 ,pp.
139 - 152 .

Gao et al., “ Compact Image Slicing Spectrometer (ISS ) for Hyperspectral

Fluorescence Microscopy ” , Optics Express, Jul. 20 , 2009 , vol. 17 ,No . 15, pp . 12293- 12308 .
Gao et al., “ Depth -Resolved Image Mapping Spectrometer (IMS)with Structured Illumination” , Optics Express, Aug . 29 , 2011, vol.

19, No. 18 , pp. 17439 - 17452.

Gao et al., “ Optical Design of a Snapshot High -Sampling Image

WO 2012/082366
WO 2013 / 173541

11 /2013

Mapping Spectrometer (IMS) for HyperspectralMicroscopy ” , Three

WO 2016 / 196224

12 /2016

WO 2018 / 075957

4 /2018
4 /2018

Dimensional and Multidimensional Microscopy:Image Acquisition

WO 2018 /075964

OTHER PUBLICATIONSAllen et al., “ Measurements of Methane Emissions at Natural Gas
Production Sites in the United States ” , PNAS , Oct. 29 , 2013 , vol.

110 , No. 44 , pp . 7 .

Alvarez et al., “Greater Focus Needed on Methane Leakage fromNatural Gas Infrastructure ” , PNAS , Apr. 24 , 2012 , vol. 109 , No . 17 ,pp. 12 .
Bedard et al., “ Image Mapping Spectrometry : Calibration andCharacterization ” , Optical Engineering, Nov . 2012 , vol. 51 , No. 11,
pp . 111711 - 1 - 111711 - 13 .
Ben -David et al., “ Probability Theory for 3 - Layer Remote Sensing
Radiative Transfer Model: Univariate Case,” Optics Express, Apr.
2012 , vol. 20 , No . 9 , pp . 10004 - 10033 .
Ben - David et al., “ Probability Theory for 3 -Layer Remote Sensing
Radiative Transfer Model: Errata ,” Optics Express, May 20 , 2013 ,vol. 21, No. 10 , pp . 11852 .
Brady et al., “Multiscale Lens Design ” , Optics Express, Jun . 22 ,2009, vol. 17, No. 13 , pp. 10659 - 10674 .
Brochure provided by Lofty Designs to Rebellion Photonics on Oct.
31 , 2012 as noted from the email. Subsequent to that date brochure

was used in connection with potential customers.
Caulton et al., “ Toward a Better Understanding and Quantificationof Methane Emissions from Shale Gas Development” , PNAS, Apr.

29 , 2014 , vol. 111 , No. 17, pp . 7 .
Chen et al., “ Quantitative Sectioning and Noise Analysis for Structured Illumination Microscopy : Erratum ” , Optics Express, Oct. 19 ,2015 , vol. 23, No. 21 , pp. 27633 - 27634 .
Chidley et al., “ Flow -Induced Birefringence: The Hidden PSF
Killer in High Performance Injection -Molded Plastic Optics” , Endoscopic Microscopy , Proceedings of SPIE vol. 6082, 2006 , pp . 11.
Chu et al., “ The NIST Quantitative Infrared Database” , Journal of

Research of the National Institute of Standards and Technology ,Jan .- Feb . 1999, vol. 104 , No . 1 , pp . 59 -81 .

DiPietro et al., “ Hyperspectral Matched Filter with False - Alarm
Mitigation ” , Optical Engineering , Jan . 2012 , vol. 51, No. 1 , pp .
016202 - 1 -016202- 7 .
“ Directed Inspection and Maintenance at Gas Processing Plants andBooster Stations,” United States Environmental Protection Agency
Air and Radiation (6202J) , EPA430 - B -03-018 , Oct. 2003 available
at https://www3. epa. govigasstar/documents / II_ dimgasproc .pdf.

and Processing XVII , Proceedings of SPIE vol. 7570 , 2010 , pp . 1 - 7 .
Gao et al., “ Quantitative Comparison Between Full- Spectrum andFilter- Based Imaging in Hyperspectral Fluorescence Microscopy ” ,Journal of Microscopy, 2012 , vol. 246 , No. 2 , pp. 113 - 123.
Gao et al.,“ Snapshot Image -Mapping Spectrometer for Hyperspectral

FluorescenceMicroscopy ” , Optics and Photonics News,Nov . 2010 ,vol. 21 , No. 12 , p . 50 .
Gao et al., “ Snapshot Image Mapping Spectrometer ( IMS) withHigh Sampling Density for Hyperspectral Microscopy” , Optics
Express , Jul. 5 , 2010 , vol. 18 , No . 4 , pp . 14330 - 14344 .

Gerhart et al., “ Detection and Tracking of Gas Plumes in LWIR
Hyperspectral Video Sequence Data ,” Algorithms and Technologies
for Multispectral, Hyperspectral, and Ultraspectral Imagery XIX ,20133 , SPIE Proceedings vol. 8743 , pp . 1 - 14 .
Gittins, Christopher M ., “ Detection and Characterization of Chemi
cal Vapor Fugitive Emissions by Nonlinear Optimal Estimation :Theory and Simulation ” , Applied Optics, Aug. 10 , 2009, vol. 48,
No. 23, pp . 4545 -4561.
Goldberg et al., “ Dual Band MWIR /LWIR Focal Plane Array Test
Results,” Army Research Lab , Adelphi, MD , Aug . 1999 , pp . 18 .
Golowich et al., “ Performance Limits of LWIR Gaseous Plume
Quantification ” , Algorithms and Technologies for Multispectral,Hyperspectral, and Ultraspectral Imagery Xvii , 2011 , Proceedingsof SPIE vol. 8048, pp . 1 - 12 .
Griffin et al., “ The Herschel-SPIRE 1 - 15 Instrument and its In

Flight Performance ,” Astronomy and Astrophysics, Jul. 1, 2010 ,vol. 518 , pp . 7 .

Gross et al., “ Remote Identification and Quantification of Industrial
Smokestack Effluents via Imaging Fourier- Transform Spectros
copy ” , Environmental Science & Technology, 2010 , vol. 44 , No. 24 ,pp . 9390 -9397.
Gupta et al., “ Miniature Snapshot Multispectral Imager,” OpticalEngineering , 2011 , vol. 50 , pp . 033203 - 1 - 033203 - 9 .

Hadlington , Simon , “ New Camera Makes Methane Visible ” , Chem
istry World , http ://web . archive . org /web / 20160305234907/http : //www .rsc .org /chemistryworld / 2015 / 12 /methane -camera -infraredgreenhouse gas, Dec . 14 , 2015 , pp . 2 .
Hagen et al., “ Analysis of Computed Tomographic Imaging Spec
trometers. I. Spatial and Spectral Resolution ” , Applied Optics, Oct.
1, 2008 , vol. 47 , No. 28 , pp . F85 - F95 .
Hagen et al., “ Coded Aperture DUV Spectrometer for Standoff
Raman Spectoscopy” , Next-Generation Spectroscopic Technolo
gies II, Proceedings of SPIE vol. 7319 , 2009, pp . 1 - 10 .

Hagen et al., “ Compound Prism Design Principles, I” , Applied
Optics, Sep . 1 , 2011, vol. 50 , No. 25 , pp . 4998 - 5011 .

US 10 ,Page
084,4975 B2( 56 )

References Cited

OTHER PUBLICATIONS
Hagen et al., “ Compound Prism Design Principles, II: Triplet andJanssen Prisms” , Applied Optics, Sep . 1, 2011, vol. 50 , No. 25 , pp .
5012 -5022.
Hagen et al., “ Compound Prism Design Principles , III: Linear-in
Wavenumber and Optical Coherence Tomography Prisms” , Applied
Optics, Sep . 1, 2011 , vol. 50 , No . 25 , pp . 5023 -5030 .
Hagen et al., “ Fourier Methods of Improving Reconstruction Speed
for CTIS Imaging Spectrometers” , Imagin Spectrometry XII, Pro
ceedings of SPIE vol. 6661, 2007 , pp . 11.
Hagen et al., “ Foveated Endoscopic Lens” , Journal of Biomedical
Optics , Feb . 2012 , vol. 17 , No. 2 , pp . 021104 - 1 -021104 -6 .
Hagen et al., “Gaussian Profile Estimation in One Dimension ” ,Applied Optics, Aug. 1, 2007 , vol. 46 , No. 22 , pp . 5374 -5383.
Hagen et al., " Gaussian Profile Estimation in Two Dimension ” ,Applied Optics, Dec . 20 , 2008 , vol. 47 , No. 36 , pp. 6842 -6851 .

Hagen et al., “ Quantitative Sectioning and Noise Analysis for

Structured Illumination Microscopy ” , Optics Express, Jan . 2, 2012 ,vol. 20 , No. 1, pp . 403 -413 .

Hagen et al., “ Quantitative Sectioning and Noise Analysis for
Structured Illumination Microscopy: Errata ” , Optics Express, Feb .

Kester et al., “ Low Cost, High Performance, Self -Aligning Minia

ture Optical Systems” , Applied Optics, Jun . 20 , 2009, vol. 48, No .
18 , pp . 3375 -3384 .
Kester et al., “ Real- Time Snapshot Hyperspectral Imaging Endo

scope” , Journal of Biomedical Optics,May 2011 , vol. 16 , No . 5 , pp .
056005 - 1- 056005- 12 .

Kudenov et al., “ Fourier Transform Channeled Spectropolarimetry
in the MWIR ” , Optics Express, Oct. 1, 2007, vol. 15 , No. 20 , pp .
12792- 12805.
Kudenov et al., “ Snapshot Imaging Mueller Matrix Polarimeter
Using Polarization Gratings” , Optics Letters, Apr. 15 , 2012 , vol. 37 ,No. 8 , pp. 1367 - 1369 .
Landau et al., “ Design and Evaluation of an Ultra -Slim Objective

for in - vivo Deep Optical Biopsy ” , Optics Express, Mar. 1 , 2010 ,vol. 18 , No. 5 , pp . 4758 -4775 .

Levi, Michael A ., “ Comment on ‘Hydrocarbon Emissions Characterization in the Colorado Front Range : A Pilot Study by Gabrielle

Pétron et al.” , Journal of Geophysical Research , 2012 , vol. 117 , No .
D21203 , pp . 1 - 5 .
Levi, Michael A ., “ Reply to “ Reply to ' Comment on ‘Hydrocarbon

Emissions Characterization in the Colorado Front Range A Pilot
Study ' by Michael A . Levi" by Gabrielle Pétron et al.” , Journal ofGeophysical Research : Atmospheres, 2013 , vol. 118 , pp . 3044
3046 .

27 , 2012 , vol. 20 , No. 5 , pp . 5343.
Hagen et al., “ Real- Time Quantitative Hydrocarbon Gas Imaging

Low et al., “ Remote Sensing and Characterization of Stack Gases byInfrared Spectroscopy. An Approach by Using Multiple - Scan Inter

Chemical, Biological, Radiological,Nuclear, and Explosives (CBRNE )No. 1, pp . 73 -74 .
Luo et al., “ Fast Processing of Imaging Spectrometer Data Cube
Based on FPGA Design ” , MIPPR 2007: Multispectral Image Pro

with the Gas Cloud Imager (GCI)” , Proceedings of SPIE , vol. 8358 ,Sensing XIII , May 1 , 2012 , pp . 7.

Hagen et al., “ Review of Snapshot Spectral Imaging Technologies” ,Optical Engineering, Sep . 2013 , vol. 52 , No. 9 , pp . 090901-1
090901 - 23 .
Hagen et al., “ Snapshot Advantage: A Review of the Light Collec
tion Improvement for Parallel High -Dimensional Measurement

Systems,” Optical Engineering , Jun . 13 , 2012 , vol. 51 , No. 11 , p .
111702 - 1 -111702 - 7 .
Hagen et al., “ Snapshot Mueller Matrix Spectropolarimeter” Optics

Letters, Aug. 1, 2007, vol. 32 , No. 15, pp . 2100 -2102.

Hagen et al., “ Spectrally -Resolved Imaging of Dynamic TurbidMedia ” , Multimodal Biomedical Imaging VI, Proceedings of SPIE

vol. 7892 , 2011, pp . 1- 7 .
Hagen et al., “ Video - Rate Spectral Imaging of Gas Leaks in the

Longwave Infrared ,” Chemical, Biological, Radiological, Nuclear,and Explosives (CBRNE ) Sensing XIV , May 29, 2013 , SPIE

Proceedings vol. 8710 , pp . 7 .
Harley et al., “ Remote Quantification of Smokestack EffluentMass

ferometry " , Environmental Science & Technology , Jan . 1967, vol. 1 ,cessing, Proceedings of SPIE vol. 6787 , pp . 7 .
Manolakis et al., “ Long -Wave Infrared Hyperspectral Remote Sens
ing of Chemical Clouds” , IEEE Signal Processing Magazine , Jul.
2014 , vol. 31 , No. 4 , pp . 120 - 141 .
Mathews, “ Design and Fabrication of a Low -Cost, Multispectral
Imaging System ,” Applied Optics, 2008, pp. F71-F76 , vol. 47 .
Naranjo et al., “ Ir Gas Imaging in an IndustrialSetting ," Thermosense

XXXII, Published in SPIE Proceedings vol. 7661,May 4 , 2010 , pp .
1 -8 .
Nguyen et al., “ Snapshot 3D Optical Coherence Tomography Sys
tem using Image Mapping Spectrometer” , Biomedical Optics and
3D Imaging OSA , 2012 , pp . 3 .
Niu et al., " New Approach to Remote Gas - Phase Chemical Quan
tification : Selected -Band Algorithm ” , Optical Engineering, Feb .
2014 , vol. 53 , No . 2 , pp . 021111 - 1 - 021111 - 10 .

tra ” , Applied Optics, Jun . 1, 1996 , vol. 35 , No . 16 , pp . 2802- 2809 .

“ Oil and Natural Gas Sector Leaks” , U .S . EPA Office of Air Quality
Planning and Standards (OAQPS ), Review Panel, Apr. 2014 , pp .63.
Pétron et al., “ Hydrocarbon Emissions Characterization in the
Colorado Front Range : A Pilot Study ” , Journal of GeophysicalResearch , 2012 , vol. 117 , No. D04304 , pp . 1 - 19 .
Pétron et al., “ Reply to Comment on ‘Hydrocarbon Emissions
Characterization in the Colorado Front Range A Pilot Study ' byMichael A . Levi” , Journal pp . 236 - 242 .

2007, vol. 46 , No. 25 , pp . 6368 -6374 .

vol. 44, No. 16 , pp . 3208 - 3217 .

Flow RatesUsing Imaging Fourier Transform Spectrometry,” Chemi

cal, Biological, Radiological, Nuclear, and Explosives (CBRNE)

Sensing XII, Apr. 25 - 29 , 2011, SPIE Proceedings vol. 8018 , pp .
1 - 13
Hayden et al., “ Determination of Trace-Gas Amounts in Plumes by

the Use of Orthogonal Digital Filtering of Thermal-Emission Spec
Hirsch et al., “ Detection of Gaseous Plumes in IR Hyperspectral
Images Using Hierarchical Clustering” , Applied Optics, Sep . 1,

Johnston et al., “ A Real Time FPGA Implementation of a BarrelDistortion Correction Aglorithm ” , Projects, 2003 , vol. 10 , pp .
91 - 96 .
Karion et al., “ Methane Emissions Estimate from Airborne Mea
surements Over a Western United States Natural Gas Field ” , Geophysical Research Letters, 2013 , vol. 40 , pp . 4393-4397.

Keshava et al., “ A Survey of Spectral Unmixing Algorithms” ,Lincoln Laboratory Journal, 2003 , vol. 14 , No. 1, pp . 55 - 78 .
Kester et al., “ A Real- Time Gas Cloud Imaging Camera for Fugitive

Emission Detection and Monitoring ”, Imaging and Applied Optics

Technical Digest, 2012 , pp . 3 .
Kester et al., “ Development of Image Mappers for Hyperspectral
Biomedical Imaging Applications” , Applied Optics, Apr. 1, 2010 ,vol. 49 , No. 10 , pp . 1886 -1899 .
Kester et al., “ High Numerical Aperture Microendoscope Objective
for a Fiber Confocal Reflectance Microscope ” , Optics Express,Mar.
5 , 2007 , vol. 15. No. 5 , pp . 2409- 2420 .

Pisano et al., “ Thermal Illuminators for Far -Infrared and Submil

limeter Astronomical Instruments,” Applied Optics, Jun . 1, 2005 ,Polak et al., “ Passive Fourier- Transform Infrared Spectroscopy ofChemical Plumes: An Algorithm for Quantitiative Interpretation

and Real- Time Background Removal” , Applied Optics, Aug. 20 ,1995 , vol. 34 , No. 24 , pp . 5406 -5412 .

Rebellion Photonics , “ Gas Cloud Imaging Camera : A Breakthroughin Leak Monitoring for the Rig & Refinery Safety Market” , Pre

sentation at SPIE Defense Security and Sensing, 28 pages, Apr.
29 -May 3 , 2013 .
Sandsten et al., “ vol. Flow Calculations on Gas Leaks Imaged with

Infrared Gas -Correlation ," Optics Express , 2012 , vol. 20 , No. 18 ,pp. 20318 - 20329 .
Shogenji et al., “Multispectral Imaging Using Compact Compound

Optics,” Optics Express, Apr. 19 , 2004 , vol. 12 , No. 8 , pp . 1643
1655 .

Telops, “ Hyper- Cam ” , http ://web .archive .org/web /20160608180941/
http ://www .telops.com / en /hyperspectral-cameras/hyper-cam as archived
Jun. 8 , 2016 in 2 pages.

US 10 ,Page
084,5975 B2( 56 )

References Cited

OTHER PUBLICATIONS
Telops , “ Innovative Infrared Imaging ” , http ://web .archive .org/web /20160603212729 /http ://www .telops.com /en / as archived Jun . 3, 2016
in 2 pages.
Walter Jr., et al., “ Detection of Atmospheric Pollutants: a Correlation Technique ” , Applied Optics, Jun . 1975 , vol. 14 , No . 6 , pp .
1423 - 1428.

Notice of Allowance received in U . S . Appl. No. 14 /539 ,899 dated
Oct. 31, 2016 in 10 pages .

Amendment as filed in U .S . Appl. No. 14 /539, 899 dated Jan . 27 ,Extended European Search Report received in European Application No. 14192862.2 dated Mar. 30 , 2015 in 10 pages.
Official Communication received in European Application No.
14192862.2 dated Apr. 19 , 2016 in 6 pages.
International Search Report in PCT Application No. PCT/US2013/2017 in 5 pages.

Wikipedia entry https:// en . wikipedia.org/wiki/Mobile _ computing in

041278 dated Aug. 27 , 2013 in 4 pages.

6 pages, last modified on Dec . 30 , 2016 ; retrieved from the internet
on Feb . 2 , 2017 .

tion No. PCT/US2013 /041278 dated Nov . 27 , 2014 in 10 pages.
Extended European Search Report received in European Application No. 15165877 .0 dated Oct. 8 , 2015 in 12 pages.

Williams et al., “ Dual-Band MWIR /LWIR Radiometer for Absolute
TemperatureMeasurements ," SPIE Thermosense Conference XXVIII —Manuscript 6205-23, Apr. 18 , 2006 , pp . 13 .
Young et al., “ An In - SceneMethod for Atmospheric Compensationof Thermal Hyperspectral Data ” , Journal of GeophysicalResearch ,2002, vol. 107 , No . D24 , pp . 14 - 1 - 14 - 20 .

Zheng et al., “ A Static Multiplex Fabry -Perot Spectrometer”, Sen

International Preliminary Report on Patentability in PCT Applica

Official Communication received in European Application No .

15165877 .0 dated Jan . 3, 2017 in 9 pages .
Preliminary Amendment as filed in U .S . Appl. No. 14 /700 ,567 dated

Jul. 10 , 2015 in 6 pages .
Publication Request as filed in U .S . Appl. No . 14 / 700 , 567 dated

sors, Cameras, and Systems for Industrial/Scientific Applications X ,Aug. 24 , 2016 in 237 pages.
Official Communication received in U . S . Appl. No. 14 /700,567
dated Jun . 14 , 2017 in 29 pages.
Extended European Search Report received in European Application No. EP 15165880.4 dated Nov . 24 , 2015 in 8 pages .
Preliminary Amendment as filed in U .S . Appl. No. 14 /792 ,477 dated
Dec . 21 , 2015 in 7 pages.
Official Communication received in U . S . Appl. No . 14 / 792 ,477
dated Jan . 27 , 2017 in 10 pages.
Response to Restriction Requirement submitted in U .S . Appl. No.

14 / 538, 827 dated Dec. 28 , 2015 in 11 pages

Notice of Allowance received in U .S . Appl. No . 14 /538 ,827 dated

Preliminary Amendment as filed in U . S . Appl.No. 15 / 166 ,092 dated
Aug. 15 , 2016 in 7 pages .

Feb . 1 , 2016 in 18 pages.
Corrected Notice ofAllowance received in U .S . Appl. No . 14 /538 ,827

034455 dated Oct. 24 , 2016 in 12 pages .

Proceedings of SPIE — IS & T Electronic Imaging , SPIE vol. 7249 ,
2009, pp . 8 .
Zheng et al., “ Analytic -Domain Lens Design with Proximate Ray
Tracing ” , Journal of the Optical Society of America A , Aug. 2010 ,vol. 27 , No. 8 , pp. 1791- 1802.
Preliminary Amendment as filed in U . S . Appl. No. 14 /538 , 827 dated
Jan . 28 , 2015 in 6 pages.
Office Action received in U .S . Appl. No. 14 /538,827 dated Jun. 30 ,2015 in 8 pages.
Non -Final Office Action Response as filed in U . S . Appl. No .

dated Feb . 10 , 2016 in 4 pages.

Corrected Notice of Allowance received in U . S . Appl. No . 14 /538, 827
dated Feb . 22 , 2016 in 4 pages .

Request for Continued Examination and Response to Correct Application Papers as filed in U . S . Appl. No. 14 /538 ,827 dated Apr. 29 ,2016 in 14 pages.
Notice of Allowance received in U . S . Appl. No. 14 /538 , 827 dated
May 26 , 2016 in 9 pages.
Notice of Allowance received in U .S . Appl. No. 14 /538 , 827 dated
Sep . 19 , 2016 in 9 pages.

Amendment as filed in U .S . Appl. No. 14 /538 ,827 dated Dec . 16 ,2016 in 9 pages.
Office Action received in U . S . Appl. No. 14 /543,692 dated Nov . 3,2015 in 7 pages.
Interview Summary received in U .S . Appl. No. 14 /543 ,692 dated
Feb . 17 , 2016 in 5 pages.
Response to Office Action as filed in U .S . Appl. No. 14 /543 ,692
dated May 2 , 2016 in 9 pages.
Office Action received in U . S . Appl. No. 14 / 543,692 dated Jun . 1 ,2016 in 18 pages .
Response to Final Action as filed in U . S . Appl. No. 14 /543 ,692
dated Nov . 30 , 2016 in 12 pages .

Notice of Allowance received in U .S . Appl. No. 14 /543 ,692 dated
Dec . 9 , 2016 in 12 pages.
Amendment after Allowance as filed in U .S . Appl. No. 14 /543 ,692
dated Mar. 3 , 2017 in 6 pages.
Notice of Allowance received in U . S . Appl. No. 14 /543,692 dated
Mar. 17 , 2017 in 4 pages.

Office Action received in U . S. Appl.No. 14/539,899 dated Mar. 26 ,2015 in 6 pages .

Non -Final Office Action Response as filed in U .S . Appl. No .
14 / 539,899 dated Aug. 26 , 2015 in 8 pages .
Final Office Action received in U . S . Appl. No. 14 /539 ,899 dated

Dec . 11 , 2015 in 9 pages .

Amendment as filed in U . S . Appl. No. 14 /539,899 dated Jun . 9 ,2016 in 6 pages .

Notice of Allowance received in U .S . Appl. No. 14 /539 ,899 dated
Jun . 21, 2016 in 17 pages .

14 /792 ,477 dated May 8 , 2017 in 6 pages .

International Search Report in PCT Application No. PCT/US2016 /Preliminary Amendmentas filed in U . S . Appl. No. 14 /700 ,791 dated
Jul. 13 , 2015 in 8 pages .

Notice of Allowance received in U .S . Appl. No. 14 /700 ,791 dated
Jun. 9 , 2016 in 11 pages .

Notice of Allowance received in U .S . Appl. No. 14 /700 ,791 dated
Sep . 30 , 2016 in 19 pages .

Notice of Allowance received in U .S . Appl. No. 14 /700 ,791 dated

Feb . 21, 2017 in 20 pages .
Comments on Allowance received in U . S . Appl. No . 14 / 700 ,791
dated May 19 , 2017 in 2 pages .

Notice of Allowance received in U .S . Appl. No. 14 /700 ,791 dated
Jul. 10 , 2017 in 24 pages.

Amendment as filed in U .S. Appl. No. 14 /700 ,567 dated Dec. 13 ,2017 in 12 pages.

Amendment as filed in U . S . Appl. No. 14 / 792 ,477 dated Jan . 18 ,2018 in 10 pages .
Amendment as filed in U .S . Appl. No. 15/418,532 dated Nov . 22 ,2017 in 8 pages.

Amendment as filed in U .S . Appl. No. 15 /462 ,352 dated Feb . 28 ,2018 in 5 pages .

International Preliminary Report on Patentability in PCT Application No. PCT/US2016 /034455 dated Dec . 5 , 2017 in 8 pages.
International Search Report in PCT Application No. PCT/US2017/057712 dated Mar. 6 , 2018 in 12 pages.
International Search Report in PCT Application No. PCT/US2017 /
057725 dated Feb . 14 , 2018 in 14 pages .
Invitation to Pay Additional Fees in PCT Application No. PCT/US2017/057712 dated Jan . 10 , 2018 in 2 pages.
Invitation to Pay Additional Fees in PCT Application No . PCT/US2017 /057725 dated Dec . 14 , 2017 in 3 pages.
Notice of Allowance received in U . S . Appl. No. 14 /571 ,398 dated
Feb . 7 , 2018 in 20 pages.
Notice of Allowance received in U .S . Appl. No. 14 /571, 398 dated
Oct. 18 , 2017 in 8 pages .
Notice of Allowance received in U .S . Appl. No. 15 /623 ,942 dated
Jan . 24 , 2018 in 22 pages.
Official Communication received in U . S . Appl. No. 14 /700 , 567
dated Mar. 5 , 2018 in 38 pages.

US 10 ,Page
084,6975 B2
References Cited( 56 )OTHER PUBLICATIONS

Official Communication received in U .S . Appl. No. 14 /792,477
dated Jul. 19 , 2017 in 20 pages .
Official Communication received in U .S. Appl. No. 15 /418,532
dated Dec. 11, 2017 in 21 pages.
Official Communication received in U .S . Appl. No. 15 /418 ,532
dated Jun . 23, 2017 in 7 pages .
Official Communication received in U .S . Appl. No. 15/462,352
dated Sep . 28 , 2017 in 6 pages .
Preliminary Amendment as filed in U .S . Appl. No. 15 /471,398 dated
Oct. 6 , 2017 in 6 pages.
Preliminary Amendment as filed in U . S . Appl.No. 15/623, 942 dated
Dec . 7 , 2017 in 6 pages.
Preliminary Amendment as filed in U .S . Appl. No. 15 / 789 ,811 dated

Mar. 20 , 2018 in 6 pages .

Preliminary Amendment as filed in U .S . Appl.No. 15 / 789 ,829 dated

Mar. 20 , 2018 in 8 pages .

Sandsten et al., “ Development of Infrared Spectroscopy Techniques

for Environmental Monitoring ” , Doctoral Thesis, Aug . 2000 , pp .
123 .

Sandsten et al., “ Real- Time Gas-Correlation Imaging Employing

Thermal Background Radiation ” , Optics Express, Feb . 14 , 2000 ,vol. 6 , No . 4 , pp . 92 - 103 .

Cossel et al., “ Analysis of Trace Impurities in Semiconductor Gas
Via Cavity -Enhanced Direct Frequency Comb Spectroscopy” , Applied

Physics B , Sep . 2010 , vol. 100, No. 4 , pp . 917 - 924 .
Official Communication received in U .S . Appl. No. 15 / 166 ,092
dated May 15 , 2018 in 30 pages.

* cited by examinerSep. 25 , 2018

U . S . Patent,128lensaRorimagainyg array136,Detector

1Towards pro5ces0or

130pdSieuvctpirdaledy100

US 10 , 084 , 975 B2

Sheet 1 of 23** * ****

Mrs,160oevf uraebcnleo** ** * * **** * * **** *** ** ** * *** *** *** ** * * ** * *This128a 130

WW

124
,
lans
file

Hvit

WASWA

5

XXXXXXXXXX

Forb cnito

Vabimishtashi u * ?

KASAMA
ANCHE/101.www

W

120
120
120

i

wid NIH

ini itut /lotilului**ifts#trifftieri***h

utratt
! i

120

120
120

within

*!
1!
,

Harf
www
hi

s

torienX W16limiti!!

1
.
FIG
1
.
FIG

110

22hh
!

Sep. 25 , 2018

U . S . Patent

US 10 , 084 , 975 B2

Sheet 2 of 23

L

Field Reference 204

ihat

W

{ANNIN**WWWXWWWWWWWWWWWWWWWWWXXXANTAN

W

Wwwww

202

202

W

SAYA

WwVAHK:

HostiXXX

wWinks vw .19746
WA tituibusftritWAVE

ww.w w

WprilaAriIWta nimadiát

www

ww W w

tisti

***

236

heitir907

siesvan
KYAW KYAW. R

224 480

Vilke.34

When

NI N

Overlapping FOV Region 230

instigate A

it

M

A

Parallax 228:
W

XX ASAWARNAWANYAMA
- 977

:
7

POOD

44
:

.

2004

1

FIG . 2

Towards
processor 150

Sep. 25 , 2018

U . S . Patent

US 10,084,975 B2

Sheet 3 of 23

Field Reference

( ;

338

4

::i ??

/8 0412 ,8264J9w4,f83?????????????
y
%
95%N
l A4
an - s« » *8564844% A4 % &.462425262543554 / 434442454425:25: 4

NA??

?????? ? ?

226 ~~” ][ / l
]
10

??

/
4

336

8?f&? ?

? ? =? ?

_j.2382 ??

???/.w? ? ?

: ? ? ? as ; »* * * ** * *22122x45 = = = v = =.v -xvi = vv

: ? . S . v ????

???? = = == = = = = = = = = = =??????

3 .swwaaway????????????????????& iv= === = ==iv
= >*????llisiti??

4"" ?????? cscv ?? ???

#)48

csr @ rscpc

?2

"

?? {

S

25????:% A4% wxws .

e ses .Assis assess

.

324a

FIG BA28a? ? ???? ? ?

???????? ???? ????

2344(] 2 5 $ . {

????

"??

-spicerscb42c366ccc74
1
&
??
»
??
44
/
M
?????
Ma
????? ?

Asrc= pric &scip > < b4..

e?

????
X

?
FIG , 3Bs?

????? ??? ?)? ? ? ?

sy + + & +s4a8by + - S :2

Sep . 25, 2018

U . S . Patent

128
,
Assy
Lins

Moveabl tempraue scohnutrlesd

128a

ena1

444
w
w
10111
ODIOMITTITTUMTID

wakati

VVVV

lii

XW

460a,460b

4
.
FIG

300ld
$
o
Ook

456
D*
,
etector

454ForAloy Rpaienwing

US 10 ,084 ,975 B2

Sheet 4 of 23

:

hiso

109

*
*
*
*
013

8213*
*

SAYYYY

x

.

Kiwi
E

2

* * **
WWWw
WWW
Ewwwww Swast . wwwww

rit im trit W

Su
128a204
'
454a
438a

wwws

.

zv. wwwwwwwwwwwwww

PANYALAXY

.
AXA

i

xxx

ww

"

RLensIBMelay 440

Misfit

X

tritt

swag

W

4
/
"
NY

.
2

400

FRetewayield 438

WwxAX ? 14.20W
.
4
1

'

m

00fsewutagzmltp

xxx

sex

*** *

* **

11
.
1

*
*
*
*
***
*
*
*
*
*
*

.

! ret
74.

.
773

W.AW /Ww4

W/:

v.war ue W W

/

438a

h

a

tib

7

/
4

*
*
*
*
*
*
WW
7
.
1
W

W

? ? ? ? ? AN;

wwwwww
Wowwy/-NWAYWINYA/? ?

FN/NOyIriAUmWINK

424.m121a9n7in

/

.

w

"
:
wy
.

in

450

OLI40RONmenor

226

450

U . S . Patent

Sep. 25 , 2018

US 10 ,084 ,975 B2

Sheet 5 of 23

4
:

3

12
11
10
9
2

1

4
3

2

560780

.

5B
.
FIG
.
:

:

:

:: :

:: :::

12
33
5

NS :

11
23:53

10
:

.
3

17

1

9
: : : : ::

17th

::

5A
.
FIG

Sep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 6 of 23

filteroa filterla filterza filter3a
TIPS

$

&

gan
R

. IX

1

BINI

- .................. . 13
PIRICIX

PºIPf'ia1o9ia

12
SIKINI NIINI * . *

*

.

Korto

(un)ub ?Y APN* LineI1OYOYOUS $ 6 . . .

.OOONATITA.

?1MIRPE TI
9TORPNPAONIOXIKOKIMDIRIKUS

KIZUNLULULIT

8

04
.
2

8
.
0

90
UOSSILLISU617

24
.
0

6A
.
FIG

Sep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 7 of 23

filterob filter2b filter3b
ffe??

????

WW

1

ORKE

1

O

K

..... .

me

IXTB

1

PRSB1A7ZU9IE20N

1(um)Wavel ngth

6B
.
FIG
. . . . .. . .. . . .

REGIUNERIMA . . .

UUDULINIIINNOILTONOMIE

RAM
12.*RZINU AR STA&.7*16ZARIE USZ187

kahautikoa

IBIU
IO
.
I
:
IBI

0
.
24

transmission

0
.
04
L

10

Sep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 8 of 23

filteroc filteric filter2c filter3c
1 871

Ie

:
12
.
11

.

MILTE

12

ADKO
MOIY

# wi

. .

RI

Oxacia IP

9

HIU1OUT

11U(um)Wavel ngth

TI
R

07
.
1

E

KIKK FANSITETDI1ZATIRIKOKKUR 1807 1 & 1914

84.
0

§ 44.90transmission

24.
0

Reitit

01

.
0

6C
.
FIG

Sep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 9 of 23

filterod filterid filter2d filter3d
11

*
-

&

Trattam
12000rrara& & DIKOTIDIGITIZIZSPIRILIR11 ONI

BZI

wavelength
)
um
(

um
30

HP

100

o

m

10
w

MANIRRIOTT INDIO. EN

R4WH0T!1ULAH?????
transmission
transmission

6D
.
FIG

U . S . Patent

Sep. 25 , 2018

Sheet 10 of 23

Ww w

2

FIG . 7

US 10 ,084 ,975 B2

U . S . Patent

Sep. 25 , 2018

US 10 ,084 , 975 B2

Sheet 11 of 23

24 .5 mmwww women)(CTemplesraiturse

* **
**

** * *qomuis* ** * *obj com
rer

YYY

omen

*

JALUV

U19<

w

me

s

ne

more or3-

12
12Wavelength (microns )FIG . 8A
VALLUT)(CTemplesraiturse- NghtYANVULLY

wiY222YY.

vs

Est - ress .

VLLVCEVVALL - W

w<
A wwwwwwwwwwwwwwwwww
.

*

*

u

obj corr

-

1921
UNISK
.

??????????????

6

Wavelength (microns)

FIG . 8B

8

10

12

Sep. 25 , 2018

U . S . Patent

Sheet 12 of 23

US 10 ,084 ,975 B2

r 900
900

to object- 9756
975a

MOA

MB

960

w 928athermally

controlled
shutters
camera
block

936

9726
deletonSoraya980d

980a

980b _ MA
M18

RA

.
2

More

MA
9800

19980e
???

*

980f

FIG . 9A

U . S . Patent

Sep. 25 , 2018

Sheet 13 of 23

US 10 ,084 ,975 B2- 905

to object
972

960

thermally
controlled
shutters
camera

block
980an
M
980b

aratamento

M18

9800

W

www. 972b- 928a936

ter*** * *

Whendations????????????

Arment w

FIG . 9B

- 9800Mais
9801

U . S . Patent

Sep. 25 , 2018

US 10 ,084 ,975 B2

Sheet 14 of 23

1000

000000camerasmirrorsreferencesources

FIG . 10A

1000

M

0 . 10 11
80008
*
*
*

*
*
*
*
*

FIG . 10B

MODO

Sep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 15 of 23

cameras reofrencesources
1050
31A
R

. . . . ..

RA

R7B

R

::

:

Rule

R10A
Rice

2014.12.22

RBA

w

28
R

.

.

33

-

:
:
::

:
23

:

RjA
Rgp

RA
RIB

::::

!

RBA
*

A

R88

RELA

3:

: : ::

33 :: :: :: :
. . .

:

2

: :: :: : : :: : : .

4B

ROB

ROAI

10C
.
FIG

Sep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 16 of 23

207

1203
.

2121
A

10

2021

2014
mai
120

VEL I

www

120

1201

12751275

OO

11A
.
FIG

S ep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 17 of 23

1501
-

1301

10 0

90 01

FOV

11B
.
FIG

10 0

10 06

1303H 130

1309
7500
-

10 0

Sep. 25 , 2018

U . S . Patent

1021

PROCES R

1020

102

1023
1024

COMUNIATO

M
O
D
U
L
E
P
U
R
O
N
C
E
S
I
T
N
G
MEORY
SOPYTICEAMLUS 10 ,084 ,975 B2

Sheet 18 of 23

SPUOWPELRY

10 0

MGODPULSE12.
FIGMPADCROQUNIESTLDAOGN
.

.

1015

1026

.

.

.

.

.

U . S . Patent

Sep. 25 , 2018

80 1
como

-

10 0

- -10106001-

US 10 ,084 ,975 B2

Sheet 19 of 23-- - - --- -- -

-

--

1

-

my
~

-

-

710 3
~

Not

10011002

-

-

-

10 4

13A
.
FIG

-

-

-

-

-

-

-

-

-

~
-

-

-

-

.
L
.
-

10 62

1015

OBJECT 1007

10 5

S ep. 25 , 2018

U . S . Patent

US 10 ,084 ,975 B2

Sheet 20 of 23

1450
14 0
1430

1408
13B
.
FIG
1402
1401

1403
-

404

1406
7406

1015
1000
1000
-

OBJECT 1409

U . S . Patent

Sep. 25 , 2018

Sheet 21 of 23

US 10 ,084 ,975 B2

14A
.
FIG
-

-

-

*
*

*
*
*

120

*

10 0

to

nt

Sheet 22 of 23

Sep . 25 , 2018

-1592

US 10 ,084,975 B2

X

0 01
14B
.
FIG

1590
--

-

U .S . Patent

Sep. 25, 2018

Sheet 23 of 23

US 10 ,084 ,975 B2

1586
-

1588
X

1

-

508

1501
15021B
www

1502

1584

15024

FIG
14C
.

1

1590

0001-

1015
1580

1506

1020

US 10 ,084 ,975 B2
MOBILE GAS AND CHEMICAL IMAGINGCAMERACROSS -REFERENCE TO RELATEDAPPLICATIONSThis application is a continuation of U . S . patent applica

cessed using mathematical methods to obtain information
about the scene. Some of the existing spectral imaging
systems generate the data cube by scanning the scene in thespatial domain ( e.g ., by moving a slit across the horizontal
5 dimensions of the scene ) and /or spectral domain ( e. g ., byscanning a wavelength dispersive element to obtain imagesof the scene in different spectral regions). Such scanningtion Ser. No . 14 /700 ,791, filed Apr. 30 , 2015 , entitled approaches acquire only a portion of the full data cube at a
" MOBILE GAS AND CHEMICAL IMAGING CAMERA ,”time. These portions of the full data cube are stored and thenwhich claims priority to U .S . Provisional Patent Application 10 later processed to generate a full data cube.
No. 61/ 986 ,885 , filed May 1 , 2014 , entitled “MINIATURE
SUMMARYGAS AND CHEMICAL IMAGING CAMERA ;" U . S . Pro
visional Patent Application No. 62 /012,078 , filed Jun . 13 ,2014 , entitled “ MINIATURE GAS AND CHEMICALThe systems, methods and devices of this disclosure each
IMAGING CAMERA :" U . S . ProvisionalPatent Application 15 have several innovative aspects , no single one of which isNo. 62/054 ,894 , filed Sep . 24 , 2014 , entitled “ MOBILE solely responsible for the desirable attributes disclosed
GAS AND CHEMICAL IMAGING CAMERA ;" U .S . Pro herein .
visional Patent Application No . 62 /055, 342 , filed Sep . 25 ,In one embodiment, an infrared (IR ) imaging system for2014 , entitled “MOBILE GAS AND CHEMICAL IMAG
determining a concentration of a target species in an object
ING CAMERA :” U . S . Provisional Patent Application No. 20 is disclosed. The imaging system can include an optical
62/055 ,549 , filed Sep . 25, 2014 , entitled “ MOBILE GAS system including an optical focal plane array (FPA ) unit. The
AND CHEMICAL IMAGING CAMERA;” and U . S . Pro - optical system can have components defining at least two
visional Patent Application No. 62/ 082 ,613, filed Nov . 20 , optical channels thereof, said at least two optical channels
2014 , entitled " MOBILE GAS AND CHEMICAL IMAG - being spatially and spectrally different from one another .
ING CAMERA .” the contents of each of which is hereby 25 Each of the at least two optical channels can be positioned
rence herein in their entirety and for ail
incorporated by reference
all to transfer IR radiation incident on the optical system

purposes. U .S . patent application Ser . No. 14 /700 ,791 also

towards the optical FPA . The system can include a process

claims priority to U . S . Provisional Patent Application No.
ing unit containing a processor that can be configured to61/ 986 , 886 , filed May 1, 2014 , entitled “ DUAL BAND
acquire multispectral optical data representing said target
DIVIDED APERTURE INFRARED SPECTRAL IMAGER 30 species from the IR radiation received at the optical FPA .
(DAISI) FOR CHEMICAL DETECTION :” U . S . Provi- Said optical system and said processing unit can be con
sional Patent Application No. 62/082 ,594 , filed Nov. 20 , tained together in a data acquisition and processing module

2014 , entitled “ DUAL -BAND DIVIDED -APERTURE configured to be worn or carried by a person .
INFRA -RED SPECTRAL IMAGING SYSTEM ;" U . S . Pro
In another embodiment, an infrared (IR ) imaging system
visional Patent Application No. 62 /021 .636 , filed Jul. 7 , 35 for determining a concentration of a target species in an2014 , entitled “GAS LEAK EMISSION QUANTIFICA
object is disclosed . The imaging system can comprise an

TION WITH A GAS CLOUD IMAGER :” U . S . Provisional optical system including an optical focal plane array ( FPA )Patent Application No . 62 /021, 907, filed Jul. 8 , 2014, unit. The optical system can have components defining atentitled “GAS LEAK EMISSION QUANTIFICATION
least two optical channels thereof, said at least two optical
WITH A GAS CLOUD IMAGER :” and U . S . Provisional 40 channels being spatially and spectrally different from one
Patent Application No . 62/083, 131, filed Nov . 21 , 2014 , another. Each of the at least two optical channels can beentitled "GAS LEAK EMISSION QUANTIFICATION

positioned to transfer IR radiation incident on the opticalprocessing unit containing a processor that can be config
which is hereby incorporated by reference herein in their 45 uredto acquire multispectral optical data representing said
entirety and for all purposes.
target species from the IR radiation received at the optical
FPA . Said data acquisition and processing module can have
TECHNICAL FIELD
WITH A GAS CLOUD IMAGER .” the contents of each ofsystem towards the optical FPA . The system can include a

dimensions less than 8 inchesx6 inchesx6 inches .

In another embodiment, an infrared (IR ) imaging system
50 for determining a concentration of a target species in an
The present invention generally relates to a system and object is disclosed . The imaging system can include an

Field of the Invention

method for gas cloud detection and , in particular, to a system

and method of detecting spectral signatures of chemicalcompositions in infrared spectral regions.

Description of the Related Technology
ionsSpectral imaging systems and methods have applications
in a variety of fields. Spectral imaging systems and methods

optical system including an optical focal plane array (FPA )unit . The optical system can have components defining atleast two optical channels thereof, said at least two optical
55 channels being spatially and spectrally different from one

another. Each of the at least two optical channels can bepositioned to transfer IR radiation incident on the opticalsystem towards the optical FPA . The system can include a
processing unit containing a processor that can be config

obtain a spectral image of a scene in one or more regions of 60 ured to acquire multispectral optical data representing said

the electromagnetic spectrum to detect phenomena , identify

target species from the IR radiation received at the opticalmaterial compositions or characterize processes. The spec tral image of the scene can be represented as a three -FPA . Said data acquisition and processing module can havea volume of less than 300 cubic inches.

different wavelength regions. The data cube can be pro -include wearing or carrying a data acquisition and process

dimensional data cube where two axes of the cube represent
In yet another embodiment, a method of identifying a
two spatial dimensions of the scene and a third axis of the 65 target species or quantifying or characterizing a parameter ofdata cube represents spectral information of the scene in
the target species in an object is disclosed . The method canUS 10 ,084 ,975 B2
ing module. The data acquisition and processing module can
FIG . 5B is a diagram of a 4 -by -3 pupil array comprising
comprise an optical system and a processing unit in com
rectangular optical filters ( and IR blocking material between
munication with the optical system , the optical system
the optical filters ) used to spectrally divide an opticalincluding an optical focal plane array (FPA ) unit . The wavefront imaged with an embodiment of the system .
method can include capturing multispectral infrared (IR ) 5 FIG . 6A depicts theoretical plots of transmission charac
image data at the FPA unit from at least two optical channels teristics of a combination of band -pass filters used with anthat are spatially and spectrally different from one another. embodiment of the system .
FIG . 6B depicts theoretical plots of transmission charac
The method can include acquiring multispectral optical data
representing the target species from the IR radiation teristics of a spectrally multiplexed notch - pass filter combi
10 nation used in an embodiment of the system .
received at the FPA .
FIG . 6C shows theoretical plots of transmission charac
In another embodiment, a system for monitoring theteristics of spectrally multiplexed long- pass filter combina

presence of one or more target gases at one or morein an embodiment of the system .
installation sites is disclosed . The system can include a tionFIGused.
6D
shows theoretical plots of transmission characplurality of infrared (IR ) imaging systems, each imaging 1515 teristics
ter of spectrally
multiplexed short-pass filter combina

system comprising a data acquisition and processing modtion used in an embodiment of the system .
ule . The data acquisition and processing module can be
F IG . 7 is a set of video - frames illustrating operability ofconfigured to capture infrared images of the one ormore an embodiment of the system used for gas detection .
target gases in real- time. The data acquisition and processing
FIGS. 8A and 8B are plots (on axes of wavelength inmodule can be configured to associate each captured infra - 20 microns versus the object temperature in Celsius represent
red image with a location at which the one or more target ing effective optical intensity of the object) illustratinggases are present. The data acquisition and processing

results of dynamic calibration of an embodiment of themodule can be configured to transmit image data associatedwith the one or more target gases and location data associ-system .
FIGS. 9A and 9B illustrate a cross -sectional view of

ated with the location of the one or more target gases to a 25 different embodiments of an imaging system comprising an
arrangement of reference sources and mirrors that can becentral server .

In yet another embodiment, a method for monitoring theused for dynamic calibration .

presence of one or more target gases at one or more

FIGS. 10A - 10C illustrate a plan view of different embodi

installation sites is disclosed . The method can comprisements of an imaging system comprising an arrangement ofreceiving image data from a plurality of IR imaging systems 30 reference sources and mirrors that can be used for dynamic
located at a plurality of installation sites and configured to becalibration .

worn or carried by a person . Each IR imaging system can be

F IG . 11A is a schematic diagram illustrating a mobile

configured to capture infrared images of the one or moretarget gases in real-time and to associate each captured

infrared imaging system configured to be carried or worn by
a human user.

infrared image with a location at which the one or more 35 FIG . 11B is a schematic diagram illustrating an installa
target gases are present. The method can include processing tion site that can be monitored by multiple infrared imaging
the received image data to identify the installation sites at
FIG . 12 is a schematic system block diagram showing awhich the one or more target gases is detected .

Details of one or more implementations of the subject mobile infrared imaging system , according to one embodi
matter described in this disclosure are set forth in the 40 ment.
accompanying drawings and the description below . Other

FIG . 13A is a schematic system diagram of an opticalfeatures , aspects, and advantages will become apparent fromthe description , the drawings and the claims. Note that therelative dimensions of the following figures may not besystem configured to be used in the mobile infrared imaging
systems disclosed herein , according to various embodi
ments.

drawn to scale .
BRIEF DESCRIPTION OF THE DRAWINGS45

FIG . 1 shows an embodiment of an imaging system

FIG . 13B is a schematic system diagram of an opticalsystem configured to be used in the mobile infrared imaging

systems disclosed herein , according to other embodiments.
FIG . 14A is a schematic perspective view of a mobile

infrared imaging system mounted to a helmet, according toincluding a common front objective lens that has a pupil 50 various embodiments .
divided spectrally and re - imaged with a plurality of lenses
FIG . 14B is an enlarged schematic perspective view of theonto an infrared FPA .

FIG . 2 shows an embodiment with a divided front objec -tive lens and an array of infrared sensing FPAs.

mobile infrared imaging system shown in FIG . 14A .

FIG . 14C is a perspective cross -sectional view of themobile infrared imaging system shown in FIGS. 14A - 14B .

FIG . 3A represents an embodiment employing an array of 55 Like reference numbers and designations in the various
front objective lenses operably matched with the re -imaging drawings indicate like elements .
lens array . FIG . 3B illustrates a two - dimensional array of
optical components corresponding to the embodiment of
DETAILED DESCRIPTION

FIG . 3A .

FIG . 4 is a diagram of the embodiment employing an 60
references for calibration ) and an array of respectively

I. Overview of Various Embodiments
The following description is directed to certain imple
corresponding relay lenses .
mentations for the purposes of describing the innovative
FIG . 5A is a diagram of a 4 -by -3 pupil array comprising aspects of this disclosure . However, a person having ordi
circular optical filters (and IR blocking material between the 65 nary skill in the art will readily recognize that the teachings
optical filters ) used to spectrally divide an optical wavefront
herein can be applied in a multitude of different ways . The
described implementations may be implemented in any
imaged with an embodiment of the system .
array of field references ( e .g ., field stops that can be used as

US 10 ,084, 975 B2
device , apparatus, or system that can be configured tooperate as an imaging system such as in an infra - red imaging

mentations, the imaging systems disclosed herein do notinclude detectors configured to be cooled to a temperaturesystem . The methods and systems described herein can be below 273 Kelvin . As yet another example , in various
included in or associated with a variety of devices such as, implementations, the imaging systems disclosed herein do
but not limited to devices used for visible and infrared 5 not include detectors configured to be cooled to a tempera
spectroscopy, multispectral and hyperspectral imaging ture below 250 Kelvin . As another example , in various
devices used in oil and gas exploration , refining, and trans -implementations, the imaging systems disclosed herein do

portation , agriculture , remote sensing , defense and home-

not include detectors configured to be cooled to a tempera

land security, surveillance , astronomy, environmental moni-ture below 200 Kelvin .

applications in a variety of fields including but not limited totages over existing IR spectral imaging systems, most if notland security , environment, oil and gas industry, etc . The

cooled in order to compensate , during the optical detection ,toring, etc . The methods and systems described herein have 10

agriculture, biology , physics , chemistry , defense and home-

Implementations disclosed herein provide several advan

all of which may require FPAs that are highly sensitive andteachings are not intended to be limited to the implementa
for the reduction of the photon flux caused by spectrum
tions depicted solely in the Figures, but instead have wide 15 scanning operation . The highly sensitive and cooled FPA
applicability as will be readily apparent to one having

ordinary skill in the art.
The spectral image of the scene can be represented as athree -dimensional data cube where two axes of the cube

systems are expensive and require a great deal of mainte

nance. Since various embodiments disclosed herein are
configured to operate in single -shot acquisition mode without spatial and /or spectral scanning , the instrument canrepresent two spatial dimensions of the scene and a third 20 receive photons from a plurality of points ( e . g ., every point)

axes of the data cube represents spectral information of theof the object substantially simultaneously, during the single

scene in different wavelength regions. The data cube can bereading . Accordingly , the embodiments of imaging systemprocessed using mathematical methods to obtain informa-

described herein can collect a substantially greater amounttion about the scene . Some of the existing spectral imaging

of optical power from the imaged scene (for example , ansystems generate the data cube by scanning the scene in the 25 order of magnitude more photons ) at any given moment inspatial domain (e . g ., by moving a slit across the horizontal
and vertical dimensions of the scene) and/or spectral

time especially in comparison with spatial and /or spectral
scanning systems. Consequently , various embodiments ofdomain . Such scanning approaches acquire only a portion of
the imaging systems disclosed herein can be operated using
the full data cube at a time. These portions of the full data
uncooled detectors ( for example , FPA unit including an
cube are stored and then later processed to generate a full 30 array of microbolometers ) that are less sensitive to photons

data cube .

in the IR but are well fit for continuous monitoring appli

Various embodiments disclosed herein describe a divided
aperture infrared spectral imaging (DAISI) system that is
structured and adapted to provide identification of target

cations. For example, in various implementations, the imag
ing systems disclosed herein do not include detectors con
figured to be cooled to a temperature below 300 Kelvin . As

on spectrally -resolved imaging and can provide such iden -systems disclosed herein do not include detectors configuredtification with a single - shot (also referred to as a snapshot)

to be cooled to a temperature below 273 Kelvin . As yet

chemical contents of the imaged scene . The system is based 35 another example , in various implementations, the imaging

comprising a plurality of images having different wave -another example , in various implementations, the imaging

length compositions that are obtained generally simultane
systems disclosed herein do not include detectors configured
ously . Without any loss of generality, snapshot refers to a 40 to be cooled to a temperature below 250 Kelvin . As anothersystem in which most of the data elements that are collected example , in various implementations, the imaging systems
are continuously viewing the light emitted from the scene . In disclosed herein do not include detectors configured to becontrast in scanning systems, at any given time only acooled to a temperature below 200 Kelvin . Imaging systems
minority of data elements are continuously viewing a scene , including uncooled detectors can be capable of operating infollowed by a different set of data elements , and so on , until 45 extreme weather conditions , require less power , are capable

the full dataset is collected . Relatively fast operation can beof operation during day and night, and are less expensive .

achieved in a snapshot system because it does not need toSome embodiments described herein can also be less sus

use spectral or spatial scanning for the acquisition of infra -ceptible to motion artifacts in comparison with spatiallyred ( IR ) spectral signatures of the target chemical contents .

and /or spectrally scanning systemswhich can cause errors in

Instead , IR detectors (such as , for example , infrared focal 50 either the spectral data , spatial data , or both .

plane arrays or FPAs) associated with a plurality of different
optical channels having different wavelength profiles can beused to form a spectral cube of imaging data . Although

In various embodiments disclosed herein , the DAISI
system can be mobile . For example , the DAISI system canbe configured to be worn or carried by a person , e . g ., thespectral data can be obtained from a single snapshot com

DAISI system can be miniaturized to fit in a relatively small

embodiments , these multiple snapshots can be averaged

the human user to carry or wear the system without undue

Similarly, in certain embodiments multiple snap shots mayexertion . As explained herein , in some embodiments, theprising multiple simultaneously acquired images corre - 55 housing or compartment. For example , the components ofsponding to different wavelength ranges, in various embodi- the DAISI system can be sized and shaped to fit within small
ments, multiple snap shots may be obtained . In various dimensions and can have a mass sufficiently small to enable
be obtained and a portion of these can be selected and 60 DAISI system can be sized and shaped to fit within a volume
possibly averaged . Also , in contrast to commonly used IRof less than about 300 cubic inches , or in some embodi

spectral imaging systems, the DAISI system does notments , less than about 200 cubic inches. In still otherrequire cooling. Accordingly, it can advantageously use

embodiments , the DAISI system can be sized and shaped touncooled infrared detectors . For example , in various imple fit within a volume less than about 100 cubic inches . For
mentations, the imaging systems disclosed herein do not 65 example , in some arrangements , the DAISI system can be
include detectors configured to be cooled to a temperature
sized and shaped to fit within a volume in a range of about

below 300 Kelvin . As another example, in various imple -50 cubic inches to about 300 cubic inches . In other arrange

US 10 ,084,975 B2
ments , the DAISI system can be sized and shaped to fitof optical channels 120 can be spatially and /or spectrally

200 cubic inches .

in the object space and / or image space . In one implemen

within a volume in a range of about 80 cubic inches to about

distinct. The plurality of optical channels 120 can be formed

Advantageously , such a portable and /or wearable DAISI tation, the distinct channels 120 may include optical chansystem can enable the user to monitor installations in remote 5 nels that are separated angularly in space . The array oflocations and to detect the presence of various gases ( e . g ., spectral filters 130 may additionally include a filter- holding
poisonous gases) in real-time. Further, the portable DAISI aperture mask (comprising, for example, IR light-blocking

system can enable the user to travel to different installations materials such as ceramic , metal, or plastic ). Light from theto monitor the presence of gases or chemicals in multiple
object 110 (for example a cloud of gas ), the optical prop
locations . For example , the user may travel to an oil drilling 10 erties of which in the IR are described by a unique absorp
installation in which oil is pumped from the ground. The tion , reflection and/ or emission spectrum , is received by theuser can carry or attach the portable DAISI system to his or

her clothing or body ( e .g ., by way of a clip , hat, etc .) and canactivate the system while he or she is on -site . Opticalaperture of the system 100. This light propagates througheach of the plurality of optical channels 120 and is further
imaged onto the optical detector array 136 . In various

components on board the portable DAISI system can capture 15 implementations, the detector array 136 can include at least
one or more snapshotmultispectral images of portions of theone FPA . In various embodiments , each of the re - imaging

installation susceptible to gas or chemical leaks. Computing

lenses 128a can be spatially aligned with a respectively

units on board the portable DAISI system can process thecaptured multispectral image data to detect and /or classify

corresponding spectral region . In the illustrated implemen
tation , each filter element from the array of spectral filters

gases or chemicals present at the site . A communications 20 130 corresponds to a different spectral region . Each remodule can notify the user of the detected gases . For

imaging lens 128a and the corresponding filter element of

example, in various embodiments, the communications
module can send a notification to a user interface (such as aset of computing eyeglasses, a mobile computing device

the array of spectral filter 130 can coincide with (or form ) a
portion of the divided aperture and therefore with respec
tively - corresponding spatial channel 120. Accordingly , insuch as a mobile smartphone, a tablet computing device , a 25 various embodiment an imaging lens 128a and a corre
laptop computing device , or any other suitable interface ), sponding spectral filter can be disposed in the optical path ofand the user interface can display information about thedetected gases to the user in real- time, e. g., at the oil drilling
installation .

one of the plurality of optical channels 120 . Radiation fromthe object 110 propagating through each of the plurality of
optical channels 120 travels along the optical path of each

30 re - imaging lens 128a and the corresponding filter element of

II. Examples of Divided Aperture Infrared Spectral

Imager Systems

the array of spectral filter 130 and is incident on the detector

array ( e.g., FPA component) 136 to form a single image( e .g ., sub -image ) of the object 110 . The image formed by the
FIG . 1 provides a diagram schematically illustrating spa
detector array 136 generally includes a plurality of subtial and spectral division of incoming light by an embodi- 35 images formed by each of the optical channels 120 . Each ofment 100 of a divided aperture infrared spectral imager the plurality of sub -images can provide different spatial and(DAISI) system that can image an object 110 possessing IR
spectral information of the object 110 . The different spatial
spectral signature( s ). The system 100 includes a front objec -information results from some parallax because of thetive lens 124 , an array of optical filters 130 , an array ofdifferent spatial locations of the smaller apertures of thereimaging lenses 128 and a detector array 136 . In various 40 divided aperture . In various embodiments , adjacent sub

embodiments, the detector array 136 can include a single

images can be characterized by close or substantially equal

FPA or an array of FPAs. Each detector in the detector array
136 can be disposed at the focus of each of the lenses in thearray of reimaging lenses 128 . In various embodiments, thespectral signatures. The detector array (e. g., FPA component) 136 is further operably connected with a processor 150(not shown ). The processor 150 can be programmed todevices. In some embodiments, the plurality of photo sensitive devices may comprise a two - dimensional imaging

spectral data cube . The data cube represents , in spatial (x , y )and spectral (a ) coordinates , an overall spectral image of thedetector array 136 can include a plurality of photo - sensitive 45 aggregate the data acquired with the system 100 into asensor array that is sensitive to radiation having wavelengths object 110 within the spectral region defined by the combi
nation of the filter elements in the array of spectral filters
wavelength range , mid infra -red wavelength range , or long 50 130 . Additionally , in various embodiments, the processor or

between 1 um and 20 um ( for example, in near infra -red

infra - red wavelength range ,) . In various embodiments , theplurality of photo -sensitive devices can include CCD or
CMOS sensors , bolometers, microbolometers or otherprocessing electronics 150 may be programmed to deter
mine the unique absorption characteristic of the object 110.
Also , the processor /processing electronics 150 can , alterna
tively or in addition , map the overall image data cube into
detectors that are sensitive to infra-red radiation .
An aperture of the system 100 associated with the front 55 a cube of data representing, for example , spatial distribution
objective lens system 124 is spatially and spectrally divided

of concentrations, c , of targeted chemical components

by the combination of the array of optical filters 130 and thewithin the field of view associated with the object 110 .

array of reimaging lenses 128 . In various embodiments , theVarious implementations of the embodiment 100 can
combination of the array of optical filters 130 and the array include an optional moveable temperature -controlled referof reimaging lenses 128 can be considered to form a 60 ence source 160 including, for example , a shutter system
spectrally divided pupil that is disposed forward of the comprising one or more reference shutters maintained at
optical detector array 136 . The spatial and spectral divisiondifferent temperatures . The reference source 160 can includeof the aperture into distinct aperture portions forms a plu -a heater , a cooler or a temperature - controlled element con

rality of optical channels 120 along which light propagates .

figured to maintain the reference source 160 at a desired

In various embodiments , the array 128 of re - imaging lenses 65 temperature . For example , in various implementations, the128a and the array of spectral filters 130 which respectively
correspond to the distinct optical channels 120 . The plurality

embodiment 100 can include two reference shutters maintained at different temperatures. The reference source 160 isUS 10 ,084 ,975 B2
10removably and , in one implementation , periodically inserted
into an optical path of light traversing the system 100 fromthe object 110 to the detector array (e. g., FPA component)

136 along at least one of the channels 120 . The removablereference source 160 thus can block such optical path . 5
Moreover, this reference source 160 can provide a reference
IR spectrum to recalibrate various components including thedetector array 136 of the system 100 in real time. The
configuration of the moveable reference source 160 is fur -ther discussed below .

view 202 , the embodiment of the imaging system 200
The field reference 204 can be used to truncate the field of

includes a field reference 204 at the front end of the system .

view . The configuration illustrated in FIG . 2 has an opera
tional advantage over embodiment 100 of FIG . 1 in that theoverall size and/ or weight and/ or cost of manufacture of theembodiment 200 can be greatly reduced because the objec

tive lens is smaller. Each pair of the lenses in the array 224
10 Each pair of lenses in the array 224 and the array 128

and the array 128 is associated with a field of view ( FOV ) .

In the embodiment 100, the front objective lens system
receives light from the object from a different angle . Accord
124 is shown to include a single front objective lens posi- ingly, the FOV of the different pairs of lenses in the arraytioned to establish a common field - of -view ( FOV ) for the 224 and the array 128 do not completely overlap as a result
reimaging lenses 128a and to define an aperture stop for the of parallax . As the distance between the imaging system 200
whole system . In this specific case , the aperture stop sub - 15 ( portion 202 ) and the object 110 increases , the overlapping
stantially spatially coincides with and/ or is about the same region 230 between the FOVs of the individual lenses 224
size as or slightly larger than the plurality of smaller limiting increases while the amount of parallax 228 remains approxi
apertures corresponding to different optical channels 120 . As mately the same, thereby reducing its effect on the systema result , the positions for spectral filters of the different

200 . When the ratio of the parallax -to - object -distance is

optical channels 120 coincide with the position of the 20 substantially equal to the pixel- size-to -system - focal- length

aperture stop of the whole system , which in this example isshown as a surface between the lens system 124 and theratio then the parallax effect may be considered to be
negligible and, for practical purposes, no longer distinguisharray 128 of the reimaging lenses 128a . In various imple

able . While the lenses 224 are shown to be disposed submentations, the lens system 124 can be an objective lensstantially in the same plane, optionally different objective

124 . However, the objective lens 124 is optional and various 25 lenses in the array of front objective lenses 224 can beembodiments of the system 100 need not include the objec tive lens 124 . In various embodiments , the objective lensdisposed in more than one plane . For example, some of theindividual lenses 224 can be displaced with respect to some124 can slightly shift the images obtained by the different other individual lenses 224 along the axis 226 (not shown)detectors in the array 136 spatially along a direction per and / or have different focal lengths as compared to somependicular to optical axis of the lens 124 , thus the function - 30 other lenses 224 . As discussed below , the field reference 204
ality of the system 100 is not necessarily compromised whencan be useful in calibrating the multiple detectors 236 .
the objective lens 124 is not included . Generally , however,In one implementation , the front objective lens system
the field apertures corresponding to different optical chan such as the array of lenses 224 is configured as an array ofnels may be located in the same or different planes . These lenses integrated or molded in association with a monolithic
field apertures may be defined by the aperture of the reim - 35 substrate . Such an arrangement can reduce the costs andaging lens 128a and /or filters in the divided aperture 130 incertain implementations. In one implementation , the fieldcomplexity otherwise accompanying the optical adjustment
of individual lenses within the system . An individual lens

apertures corresponding to different optical channels can be

224 can optionally include a lens with varying magnifica

located in different planes and the different planes can betion . As one example , a pair of thin and large diameter

optical conjugates of one another. Similarly, while all of the 40 Alvarez plates can be used in at least a portion of the front
filter elements in the array of spectral filters 130 of theobjective lens system . Without any loss of generality , theembodiment 100 are shown to lie in one plane, generally

Alvarez plates can produce a change in focal length whendifferent filter elements of the array of spectral filter 130 cantranslated orthogonally with respect to the optical beam .

be disposed in different planes . For example, different filter

In further reference to FIG . 1 , the detector array 136 ( e. g.,elements of the array of spectral filters 130 can be disposed 45 FPA component) configured to receive the optical data

in different planes that are optically conjugate to one
another. However, in other embodiments, the different filter

representing spectral signature ( s ) of the imaged object 110
can be configured as a single imaging array ( e. g ., FPA ) 136 .
This single array may be adapted to acquire more than one
elements can be disposed in non -conjugate planes .
In contrast to the embodiment 100 , the front objective lensimage ( formed by more than one optical channel 120 )124 need not be a single optical element, but instead can 50 simultaneously . Alternatively, the detector array 136 may
include a plurality of lenses 224 as shown in an embodiment include a FPA unit. In various implementations, the FPA unit
200 of the DAISI imaging system in FIG . 2 . These lenses
can include a plurality of optical FPAs. At least one of these
224 are configured to divide an incoming optical wavefront plurality of FPAs can be configured to acquire more than one

from the object 110 . For example, the array of front objec
spectrally distinct image of the imaged object. For example,
tive lenses 224 can be disposed so as to receive an IR 55 as shown in the embodiment 200 of FIG . 2 , in various
wavefront emitted by the object that is directed toward the embodiments , the number of FPAs included in the FPA unit

DAISI system . The plurality of front objective lenses 224
divide the wavefront spatially into non - overlapping sections .
FIG . 2 shows three objective lenses 224 in a front opticalmay correspond to the number of the front objective lenses
224 . In the embodiment 200 of FIG . 2 , for example , three
FPAS 236 are provided corresponding to the three objective

portion of the optical system contributing to the spatial 60 lenses 224 . In one implementation of the system , the FPA
division of the aperture of the system in this example . The
unit can include an array of microbolometers . The use ofplurality ofobjective lenses 224 , however, can be configured

multiple microbolometers advantageously allows for an

as a two -dimensional (2D ) array of lenses. FIG . 2 presents

inexpensive way to increase the total number of detection

a general view of the imaging system 200 and the resultant

elements ( i.e . pixels ) for recording of the three -dimensional

field of view of the imaging system 200. An exploded view 65 data cube in a single acquisition event (i.e . one snapshot). In

202 of the imaging system 200 is also depicted in greater
detail in a figure inset of FIG . 2 . As illustrated in the detailed

various embodiments, an array of microbolometers more
efficiently utilizes the detector pixels of the array of FPAS

US 10 ,084 ,975 B2
12( e.g ., each FPA ) as the number of unused pixels is reduced ,minimized and/ or eliminated between the images that may
exist when using a single microbolometer.

FIG . 3A illustrates schematically an embodiment 300 of

sponding intermediate image planes defined , with respect tothe object 110 , by respectively corresponding front objective
lenses 424a . When refractive characteristics of all of thefront objective lenses 424a are substantially the same, all of

the imaging system in which the number of the front 5 the field references 438a are disposed in the same plane . A
objective lenses 324a in the lens array 324 , the number offield reference 438a of the array 438 obscures ( or casts are - imaging lenses 128a in the lens array 128 , and the number shadow on ) a peripheral region of a corresponding image
of FPAs 336 are the same. So configured , each combination(e. g ., sub - image ) formed at the detector plane 444 throughof respectively corresponding front objective lens 324 , re a respectively corresponding spatial imaging channel 450 ofimaging lens 128a , and FPAs 336 constitutes an individual 10 the system 400 prior to such image being spectrally pro
imaging channel. Such a channel is associated with acquicessed by the processor 150 . The array 440 of relay lenses
sition of the IR light transmitted from the object 110 throughthen transmits light along each of the imaging channels 450
an individual filter element of the array of optical filters 130 . through different spectral filters 454a of the filter array 454 ,A field reference 338 of the system 300 is configured to havepast the calibration apparatus that includes two temperaturea uniform temperature across its surface and be character - 15 controlled shutters 460a , 460b , and then onto the detector
ized by a predetermined spectral curve of radiation emanatmodule 456 . In various embodiments , the detector module

ing therefrom . In various implementations, the field refer - 456 can include a microbolometer array or some other IR
ence 338 can be used as a calibration target to assist in
FPA .
calibrating or maintaining calibration of the FPA . Accord The embodiment 400 has several operational advantages.
ingly , in various implementations, the field reference 338 is 20 It is configured to provide a spectrally known object within
used for dynamically adjusting the data output from eachevery image ( e .g ., sub - image ) and for every snapshot acqui
FPA 336 after acquisition of light from the object 110. This sition which can be calibrated against. Such spectral cer
dynamic calibration process helps provide that output of the tainty can be advantageous when using an array of IR FPAS
different ( e . g ., most, or each of the FPA 336 represents
like microbolometers , the detection characteristics of whichcorrect acquired data , with respect to the other FPAs 336 for 25 can change from one imaging frame to the next due to , inpart, changes in the scene being imaged as well as theanalysis, as discussed below in more detail.
FIG . 3B illustrates the plan view perpendicular to the axis
226 of an embodiment 300 of the imaging system illustrated

thermal effects caused by neighboring FPAs. In various
embodiments , the field reference array 438 of the embodi

in FIG . 3A . For the embodiment shown in FIG . 3B , thements of the array of spectral filters 130 , re - imaging lenses

ment 400 — can be disposed within the Rayleigh range
ciated with the front objective lenses 424 , thereby removing

optical components ( e . g ., objective lenses 324a , filter ele - 30 (approximately corresponding to the depth of focus) asso

128a and FPA units 336 ) are arranged as a 4x3 array. In one
implementation , the 4x3 array 340 of optical components

unusable blurred pixels due to having the field reference
outside of this range . Additionally, the embodiment 400 of(lenses 324a , 128a ; detector elements 336 ) is used behind FIG . 4 can be more compact than , for example , the con
the temperature controlled reference target 160. The field 35 figuration 300 of FIG . 3A . In the system shown in FIG . 3A ,reference aperture 338 can be adapted to obscure and /orblock a peripheral portion of the bundle of light propagatingfrom the object 110 towards the FPA units 336 . As a result,for example , the field reference 338 may be separated fromthe lens array 324 by a distance greater than several ( forexample , five ) focal lengths to minimize /reduce blur con

the field reference 338 obscures and /or blocks the border or

tributed by the field reference to an image formed at aon the FPA elements located along the perimeter 346 of thedetector system . Generally , two elements of the FPA unit

In various embodiments, the multi- optical FPA unit of the
IR imaging system can additionally include an FPA config

will produce substantially equal values of digital counts

u red to operate in a visible portion of the spectrum . In

when they are used to observe the same portion of the scene

reference to FIG . 1 , for example, an image of the scene ofobserved , spectral content of light from the scene , or opticalimage with the visible - light image . The IR image may be

peripheral portion(s ) of the images ofthe object 110 formed 40 detector plane .

in the same spectral region using the same optical train . If 45 interest formed by such visible -light FPA may be used as aany of these input parameters ( for example , scene to be background to form a composite image by overlapping an IR

elements delivering light from the scene to the two detector
elements ) differ, the counts associated with the elements ofoverlapped virtually, with the use of a processor and spe
cifically - designed computer program product enabling such

the FPA unit will differ as well. Accordingly , and as an 50 data processing, or actually , by a viewer . The IR image may

example , in a case when the two FPAs of the FPA unit 336

be created based on the image data acquired by the indi

( such as those denoted as # 6 and # 7 in FIG . 3B ) remain
substantially un -obscured by the field reference 338 , theoutputs from these FPAs can be dynamically adjusted to thevidual FPAs 136 . The so - formed composite image facilitates
species, the spectral signatures of which the system is able

the identification of the precise spatial location of the target

output from one of the FPAs located along perimeter 346 55 to detect/recognize .
( such as, for example , the FPA element # 2 or FPA element Optical Filters .

# 11) that processes light having similar spectral character

istics .
FIG . 4 illustrates schematically a portion of another

The optical filters, used with an embodiment of thesystem , that define spectrally - distinct IR image (e .g ., subimage ) of the object can employ absorption filters , interfer

embodiment of an imaging system 400 that contains an array 60 ence filters , and Fabry -Perot etalon based filters , to name424 of front objective lenses 424a . The array 424 of lenses

424a adapted to receive light from the object 110 and relay
the received light to the array 128 of re- imaging lenses 128a

just a few . When interference filters are used , the image

acquisition through an individual imaging channel definedby an individual re - imaging lens (such as a lens 128a ofthrough an array 438 of field references (or field stops ) 438a ,FIGS . 1 , 2 , 3 , and 4 ) may be carried out in a single spectral
and through an array 440 of the relay lenses . The spectral 65 bandwidth or multiple spectral bandwidths . Referring again

characteristics of the field references/ field stops 438a can be

known. The field references 438a are disposed at corre -to the embodiments 100 , 200 , 300, 400 of FIGS . 1 through4 , and in further reference to FIG . 3B , examples of a 4 -by -3

US 10 ,084 ,975 B2
13array of spectral filters 130 is shown in FIGS. 5A and 5B .
Individual filters 1 through 12 are juxtaposed with a supmultiplexed LP (or SP ) filters is wider, such parasitic effects
are reduced thereby improving the overall performance of

porting opto -mechanical element (not shown ) to define a

the FPA unit .

filter -array plane that is oriented , in operation , substantially14
In one implementation , the LP and SP filters can beperpendicularly to the general optical axis 226 of the imag - 5 combined , in a spectrally -multiplexed fashion , in order toing system . In various implementations, the individual filters

increase or maximize the spectral extent of the transmission

1 through 12 need not be discrete optical components .

region of the filter system of the embodiment.

Instead , the individual filters 1 through 12 can comprise one

The advantage of using spectrally multiplexed filters isor more coatings that are applied to one or more surfaces of appreciated based on the following derivation , in which athe reimaging lenses (such as a lens 128a of FIGS. 1 , 2 , 3 , 10 system of M filters is examined ( although it is understoodthat in practice an embodiment of the invention can employ
and 4 ) or the surfaces of one or more detectors .

The optical filtering configuration of various embodi

ments disclosed herein may advantageously use a bandpass
filter defining a specified spectral band . Any of the filters la

any number of filters ). As an illustrative example , the caseof M = 7 is considered . Analysis presented below relates to
one spatial location in each of the images ( e . g ., sub - images )through 3a , the transmission curves of which are shown in 15 formed by the differing imaging channels ( e . g ., different

FIG . 6A may , for example , be used . The filters may be

optical channels 120 ) in the system . A similar analysis canreference to FIGS. 1, 23 , and 4 , when optical detector arrays

required .

placed in front of the optical FPA (or generally, between the
optical FPA and the object). In particular, and in further
136 , 236 , 336 , 456 include microbolometers , the predomi- 20

be performed for each point at an image (e.g ., sub - image ),and thus the analysis can be appropriately extended as
The unknown amount of light within each of the M

nant contribution to noise associated with image acquisitionis due to detector noise. To compensate and/ or reduce thenoise , various embodiments disclosed herein utilize spec -spectral channels (corresponding to these M filters ) isdenoted with f , f , fz, . . . fy , and readings from corre
sponding detector elements receiving light transmitted bytrally -multiplexed filters. In various implementations, theeach filter is denoted as 81, 82, 83 . . . SM , while measurement

spectrally -multiplexed filters can comprise a plurality of 25 errors are represented by ni, n2, nz, . . . ny . Then , thelong pass filters , a plurality long pass filters , a plurality of readings at the seven FPA pixels each of which is opticallyband pass filters and any combinations thereof. An example

filtered by a corresponding band -pass filter of FIG . 6A canof the spectral transmission characteristics of spectrally

be represented by :multiplexed filters Ob through 3d for use with various
embodiments of imaging systems disclosed herein is 30

81 = fi+ n13
82 =42 +n2,

depicted in FIG . 6B . Filters of FIG . 6C can be referred to aslong -wavelength pass , LP filters . An LP filter generally

83= f3 + n3,
84 =f4+ 1 4,
85 = fs +ns,

attenuates shorter wavelengths and transmits ( passes ) longer

wavelengths (e .g ., over the active range of the target IR
portion of the spectrum ). In various embodiments , short - 35
wavelength -pass filters , SP, may also be used . An SP filter

generally attenuates longer wavelengths and transmits
the target IR portion of the spectrum ). At least in part due to(passes ) shorter wavelengths ( e .g ., over the active range of

86= fotno
the snap -shot/non -scanning mode of operation , embodi- 40
87= f7+N7,
ments of the imaging system described herein can use lessThese readings (pixel measurements ) g ; are estimates ofsensitive microbolometers without compromising the SNR . the spectral intensities f.. The estimates g , are not equal to the
The use of microbolometers , as detector-noise - limited
corresponding f , values because of the measurement errors
devices , in turn not only benefits from the use of spectrallyn .. However , if the measurement noise distribution has zeromultiplexed filters , but also does not require cooling of the 45 mean , then the ensemble mean of each individual measure

imaging system during normal operation .

ment can be considered to be equal to the true value, i. e .

region of the electromagnetic spectrum as compared to those(g :) = f;. Here , the angle brackets indicate the operation of
calculating the ensemble mean of a stochastic variable . The
variance of the measurement can , therefore , be represented

Referring again to FIGS. 6A , 6B , 6C , and 6D , each of thefilters (0b . . . 3d ) transmits light in a substantially widerof the filters (0a . . . 3a ). Accordingly , when the spectrally - 50 as:

multiplexed set of filters ( 0b . . . 0d ) is used with anembodiment of the imaging system , the overall amount of((9:-1;)2)=(n,2) =g ?
light received by the FPAs ( for example, 236 , 336 ) is larger
In embodiments utilizing spectrally -multiplexed filters, inthan would be received when using the bandpass filters comparison with the embodiments utilizing band -pass fil(0a . . . 4a ). This " added ” transmission of light defined by 55 ters , the amount of radiant energy transmitted by each of thethe use of the spectrally -multiplexed LP (or SP ) filters spectrally -multiplexed LP or SP filters towards a given
facilitates an increase of the signal on the FPAs above thelevel of the detector noise . Additionally , by using, in an
embodiment of the imaging system , filters having spectral

detector element can exceed that transmitted through aspectral band of a band -pass filter. In this case , the intensities
of light corresponding to the independent spectral bands canbandwidths greater than those of band - pass filters , the 60 be reconstructed by computational means. Such embodi

uncooled FPAs of the embodiment of the imaging systemments can be referred to as a “multiplex design ” .
experience less heating from radiation incident thereon from
One matrix of such “multiplexed filter ” measurements
the imaged scene and from radiation emanating from the includes a Hadamard matrix requiring " negative” filters thatFPA in question itself . This reduced heating is due to a may not be necessarily appropriate for the optical embodi
reduction in the back -reflected thermal emission (s ) coming 65 ments disclosed herein . An S -matrix approach (which isfrom the FPA and reflecting off of the filter from therestricted to having a number of filters equal to an integer

non -band - pass regions. As the transmission region of thethat is multiple of four minus one) or a row -doubled Had

US 10 ,084,975 B2
15

16

amard matrix ( requiring a number of filters to be equal to an
integer multiple of eight) can be used in various embodi-

channel having a transmission characteristic corresponding
to the difference between the spectral transmission curves ofments . Here , possible numbers of filters using an S -matrix

at least two of these LP filters can be used to procure

setup are 3 , 7 , 11, etc and , if a row -doubled Hadamard imaging data for the data cube using an embodiment of thematrix setup is used , then the possible number of filters is 8 , 5 system described herein . In various implementations, the16 , 24 , etc . For example , the goal of the measurement may spectral filters disposed with respect to the different FPAS
be to measure seven spectral band f; intensities using sevenmeasurements gi as follows:& i =fi+ O + f3+ 0 + f3+ 0 +f7+n1,

82= 0 + 62 + f3 +0 +0 + f6 + f7 +N2,
83 = fi+ f2 + 0 + 0 + f3 + 0 + f7+nz,
84 + 0 +0 + 0 + = f4 +fs+ f7 + f8 +94,

g5 + + 3 + 4+ 0 + ft + 0 + ns,
86= 0 + 12 +f3 +f4 +fs+ 0 + 0 + 16,

can have different spectral characteristics . In various imple
mentations, the spectral filters may be disposed in front of

only some of the FPAs while the remaining FPAs may be
10 configured to receive unfiltered light . For example , in some
implementations, only 9 of the 12 detectors in the 4x3 arrayof detectors described above may be associated with aspectral filter while the other 3 detectors may be configured

to received unfiltered light. Such a system may be config
in a single data acquisition event.
The use of microbolometers, as detector- noise -limited
devices, in turn not only can benefit from the use ofspectrally multiplexed filters , but also does not require
20 cooling of the imaging system during normal operation . In

15 ured to acquire spectral data in 10 different spectral channels

87= f1+ f3 +0 + f4 + 0 +0 + f7 +17
contrast to imaging systems that include highly sensitive
Optical transmission characteristics of the filters FPA
with reduced noise characteristics , the embodi
described above are depicted in FIG . 6B . Here, a direct mentsunitsofimagingsystems described herein can employ lessestimate of the f, is no longer provided through a relationshipsensitive
microbolometers
without compromising the SNR .
similar to ( 8 :) = f;. Instead , if a “hat” notation is used to0 25 This result is at least in part due to the snap -shot /nondenote an estimate of a given value, then a linear combina
scanning mode of operation .
tion of the measurements can be used such as, for example ,As discussed above , an embodiment may optionally , andfi =14 (+81- 82 +83- 84 + 85 - 86 +87),
f2 = 14 (- 81+ g2 + 83 - 84- 85 + g6 + 87),?z= 14(+81+ 82-83-84+85-86-87),

in addition to a temperature - controlled reference unit ( forexample temperature controlled shutters such as shutters

30 160 , 460a , 460b ), employ a field reference component (e.g.,field reference aperture 338 in FIG . 3A ), or an array of fieldreference components (e.g., filed reference apertures 438 in
FIG . 4 ), to enable dynamic calibration . Such dynamic cali

14 = 1/4 (818283|g4 |g5 86 87),

bration can be used for spectral acquisition of one or more

Ís=14(+81-82+83+84-85+86- 87),

or every data cube . Such dynamic calibration can also be
used for a spectrally -neutral camera -to - camera combination
to enable dynamic compensation of parallax artifacts . The
use of the temperature - controlled reference unit ( forfo=14 (-8 ,+82+83- 84+ 85 -86-87),

f =14(+81+82-83+84-85-86+87),

example , temperature -controlled shutter system 160) and

These f, are unbiased estimates when the n , are zero mean 40 field - reference component( s ) facilitates maintenance ofstochastic variables, so that (f:- f, = 0 . The measurement proper calibration of each of the FPAs individually and thevariance corresponding to ith measurement is given by the entire FPA unit as a whole .

equation below :CO - ;) )= 71602

In particular, and in further reference to FIGS. 1, 2, 3 , and
4 , the temperature -controlled unit generally employs a sys

45 tem having first and second temperature zones maintained at

From the above equation , it is observed that by employing
spectrally -multiplexed system the signal-to - noise ratio( SNR ) of a measurement is improved by a factor offirst and second different temperatures. For example, shutter
system of each of the embodiments 100 , 200 , 300 and 400
can employ not one but at least two temperature - controlled
shutters that are substantially parallel to one another andV16 /7 = 1.51 .
For N channels, the SNR improvement achieved with a 50 transverse to the general optical axis 226 of thespectrally -multiplexed system can be expressed as (N + 1 )/ (2embodiment(s ) 100 , 200 , 300 , 400 . Two shutters at two
VN ). For example , an embodiment employing 12 spectral different temperatures may be employed to provide morechannels ( N = 12 ) is characterized by a SNR improvement, information for calibration ; for example , the absolute value

over a non -spectrally -multiplexed system , comprising a of the difference between FPAs atone temperature as well as
55 the change in that difference with temperature change can befactor of up to 1.88 .
Two additional examples of related spectrally -multi- recorded . Referring, for example, to FIG . 4 , in which suchplexed filter arrangements Oc through 3c and Od through 3d

multi - shutter structure is shown , the use ofmultiple shuttersthat can be used in various embodiments of the imaging
systems described herein are shown in FIGS. 6C and 6D ,enables the user to create a known reference temperature
difference perceived by the FPAs 456 . This reference tem

respectively . The spectrally -multiplexed filters shown in 60 perature difference is provided by the IR radiation emitted

FIGS. 6C and 6D can be used in embodiments of imaging
systems employing uncooled FPAs (such as microbolom -by the shutter (s ) 460a , 460b when these shutters are positioned to block the radiation from the object 110 . As a result,long -wavelength pass (LP ) filters used in the system . An LP

individual FPAs pixels can be adjusted but also the gain

eters). FIG . 6C illustrates a set of spectrally -multiplexed

not only the offset values corresponding to each of thefilter generally attenuates shorter wavelengths and transmits 65 values of these FPAs. In an alternative embodiment, the(passes ) longer wavelengths (e . g ., over the active range ofthe target IR portion of the spectrum ). A single spectral

system having first and second temperature zones may

include a single or multi-portion piece. This single orUS 10 ,084 , 975 B2
17multi -portion piece may comprise for example a plate . This
piece may be mechanically -movable across the optical axis
with the use of appropriate guides and having a first portion18
and temporally - stable object within every scene to reference
and stabilize the output from the different FPAs in the array .
Because each FPA 's offset value is generally adjusted

response of the detector unit 456 to radiation emanatingof frames obtained over time. If the reference source spec

istors. Following such initial measurement, the first shutter

the pixel offset value . However, the temperature of theis inserted in its place across the optical axis 226 to prevent

performed by monitoring the change in the gain and theat a first temperature and a second portion at a second
from each frame to the next frame by the hardware, com
5 paring the outputs of one FPA with another can have an error
temperature .
Indeed , the process of calibration of an embodiment of the that is not compensated for by the static calibration param
imaging system starts with estimating gain and offset byeters Soffset and y established, for example , by the movable
performing measurements of radiation emanating, indepen - shutters 160 . In order to ensure that FPAs operate in radio
dently , from at least two temperature - controlled shutters of metric agreement over time, it is advantageous for a portion
known and different radiances. The gain and offset can vary 10 of each detector array to view a reference source (such as thefrom detector pixel to detector pixel. Specifically , first thefield reference 338 in FIG . 3A , for example ) over a plurality
from one shutter is carried out. For example , the first shutter trum is known a priori ( such as a blackbody source at a
460a blocks the FOV of the detectors 456 and the tempera - known temperature ), one can measure the response of eachture T , is measured directly and independently with therm - 15 FPA to the reference source in order to estimate changes to
460a is removed from the optical path of light traversing theembodiment and another second shutter ( for example, 460b )reference source need not be known . In such implementa
tions , dynamic calibration of the different detectors can be

the propagation of light through the system . The temperature 20 offset for the various detectors from the time the movableof the second shutter 460b can be different than the first
shutters used for static calibration are removed. An example
shutter ( T , + T ,) . The temperature of the second shutter 460b
calculation of the dynamic offset proceeds as follows.
Among the FPA elements in an array of FPAs in an
is also independently measured with thermistors placed incontact with this shutter, and the detector response to radia embodiment of the imaging system , one FPA can be selectedtion emanating from the shutter 460b is also recorded . 25 to be the “ reference FPA ” . The field reference temperature
Denoting operational response of FPA pixels ( expressed in measured by all the other FPAs can be adjusted to agree withdigitalnumbers, or “ counts” ) as g; to a source of radiance Lj,

the field reference temperature measured by the reference as

discussed below . The image obtained by each FPA includes
a set of pixels obscured by the field reference 338 . Using thethe readings corresponding to the measurements of the two
shutters can be expressed as:

30 previously obtained calibration parameters goffer and y (the81 = YL1 (71)+ goffset

82 =YL2 (T2 )+ 8offset

pixel offset and gain ), the effective blackbody temperature T ;of the field reference as measured by each FPA is estimatedusing the equation below :Here, goffer is the pixel offset value ( in units of counts ), 35
T; mean {(g+ Agi+& offse /Y} =mean {(8 -8offset)/Y}+ AT;
and y is the pixel gain value (in units of counts per radiance
Using the equation above , the mean value over all pixels
unit). The solutions of these two equations with respect to that are obscured by the field reference is obtained . In thethe two unknowns goffset and y can be obtained if the values
above equation Ag; is the difference in offset value of theof g , and g2 and the radiance values L , and L2 are available . current frame from Agoffset obtained during the calibration

These values can , for example , be either measured by a 40 step . For the reference FPA , Ag; can be simply set to zero .
reference instrument or calculated from the known tempera - Then , using the temperature differences measured by eachtures T , and T , together with the known spectral response of

the optical system and FPA . For any subsequent measure

ment, one can then invert the equation (s ) above in order toestimate the radiance value of the object from the detector 45

measurement, and this can be done for each pixel in each
FPA within the system .
As already discussed , and in reference to FIGS. 1 through4 , the field -reference apertures may be disposed in an object

FPA , one obtains
Ti- Tref=mean {(g + Agi+ goffse/ Y }+ AT;-mean{ 8 - 8offset)/Y } =AT;

Once AT; for each FPA is measured , its value can be
subtracted from each image in order to force operational
agreement between such FPA and the reference FPA . Whilethe calibration procedure has been discussed above in refspace or image space of the optical system , and dimensioned 50 erence to calibration of temperature , a procedurally similar

to block a particular portion of the IR radiation received
from the object. In various implementations, the field

methodology of calibration with respect to radiance value
can also be implemented .

reference aperture , the opening of which can be substantiallysimilar in shape to the boundary of the filter array (forExamples of Methodology of Measurements .

tation — at least five times ) larger than the focal length of thein FIG . 3A , these FPAs 336 may form separate images ofthecompensation in the system by providing a spectrally known

process can also allow the FPAs (e .g ., each of the FPAs) to

Prior to optical data acquisition using an embodiment of

example , and in reference to a filter array of FIGS. 3B , 55 the IR imaging system as described herein , one or more ,5B - e . g ., rectangular ). The field -reference aperture can be most, or potentially all the FPAs of the system can beplaced in front of the objective lens ( 124 , 224 , 324 , 424 ) at calibrated . For example , greater than 50 % , 60 % , 70 % , 80 %a distance that is at least several times in one implemen or 90 % of the FPAs 336 can be initially calibrated . As shownlens such that the field -reference aperture is placed closer to 60 object using light delivered in a corresponding opticalthe object. Placing the field - reference aperture closer to thechannel thatmay include the combination of the correspondobject can reduce the blurriness of the image . In the embodi- ing front objective and re -imaging lenses 324 , 128 . The
ment 400 of FIG . 4 , the field -reference aperture can becalibration procedure can allow formation of individual
placed within the depth of focus of an image conjugate plane images in equivalent units ( so that, for example , the reading
formed by the front objective lens 424 . The field reference , 65 from the FPA pixels can be re - calculated in units of tem
generally, can facilitate , effectuates and/ or enable dynamic perature or radiance units , etc .). Moreover, the calibration

US 10 ,084 , 975 B2
20
be spatially co -registered with one another so that a given

pixel of a particular FPA can be optically re -mapped throughtwo or more FPAs at a single instance in time. This spectral

difference image is then used as a temporal reference image .

the optical system to the same location at the object as the

The temporal reference image can then be subtracted fromcorresponding pixel of another FPA .

other later obtained images to provide normalization that can

encing method may be employed . The method involves

deleterious effects . For example , the result of the algorithm

To achieve at least some of these goals, a spectral differ - 5 be useful in subtracting out or removing various errors or

forming a difference image from various combinations of is not affected by a prior knowledge of whether the object orthe images from different channels . In particular, the images
scene contains a target species (such as gas of interest ),used to form difference images can be registered by two or because the algorithm can highlight changes in the scene
more different FPAs in spectrally distinct channels having 10 characteristics . Thus, a spectral difference image can bedifferent spectral filters with different spectral characteris calculated from multiple spectral channels as discussed
tics . Images from different channels having different spectral
above based on a snap - shot image acquisition at any later
characteristics will provide different spectral information
time and can be subtracted from the temporal reference

Comparing ( e.g., subtracting ) these images, can therefore

image to form a temporal difference image. This temporalthe filter element of the array of spectral filters 130 corre
sponding to a particular FPA 336 transmits light from theobject 110 including a cloud of gas, for example , with adifference between the two images (the temporal difference
normalized difference image , since this species was not

the images formed by the two FPAs at issue will highlightthe presence of gas in the difference image .

if two FPAs are associated with spectral filters having thesame spectral characteristic , then the images obtained by thecontributions of some auxiliary features associated withto get a better SNR figure .

yield valuable spectral based information . For example , if 15 difference image is thus a normalized difference image . The

image ) can highlight the target species (gas ) within thecertain spectrum that contains the gas absorption peak or apresent in the temporal reference frame. In various embodi
gas emission peak while another filter element of the array 20 ments, more than two FPAs can be used both for registeringof spectral filters 130 corresponding to another FPA 336
the temporal reference image and a later -acquired difference
does not transmit such spectrum , then the difference between
image to obtain a better SNR figure of merit . For example ,A shortcoming of the spectral differencing method is that 25 two FPAS can be combined after they have been registeredimaging (not just the target species such as gas itself) can
also be highlighted in and contribute to the difference image.

Such contributing effects include, to name just a few , par

While the temporal differencing method can be used toreduce or eliminate some of the shortcomings of the spectral
differencing, it can introduce unwanted problems of its own .

allax - induced imaging of edges of the object, influence of 30 For example , temporal differencing of imaging data is lessmagnification differences between the two or more opticalchannels, and differences in rotational positioning and ori -entation between the FPAs. While magnification - related

errors and FPA -rotation -caused errors can be compensatedsensitive to calibration and parallax induced errors than the

spectral differencing of imaging data . However , any change
in the imaged scene that is not related to the target species

of interest (such as particular gas, for example ) is highfor by increasing the accuracy of the instrument construction 35 lighted in a temporally -differenced image . Thus such change

as well as by post-processing of the acquired imaging
in the imaged scene may be erroneously perceived as aparallax is scene - induced and is not so easily correctable . In
location of the target species triggering, therefore , an error
addition , the spectral differencing method is vulnerable to in detection of target species. For example, if the tempera
radiance calibration errors. Specifically, if one FPA registers ture of the background against which the gas is being
radiance of light from a given feature of the object as having 40 detected changes (due to natural cooling down as the day
a temperature of 40° C ., for example, while the data fromprogresses , or increases due to a person or animal or another
another FPA represents the temperature of the same object object passing through the FOV of the IR imaging system ),feature as being 39° C ., then such feature of the object will then such temperature change produces a signal difference
be enhanced or highlighted in the difference image ( formed as compared to the measurement taken earlier in time.
at least in part based on the images provided by these two 45 Accordingly , the cause of the scenic temperature change ( thecooling object, the person walking , etc . ) may appear as theFPAs) due to such radiance -calibration error.
One solution to some of such problems is to compare

detected target species ( such as gas ). It follows, therefore ,( e . g ., subtract ) images from the same FPA obtained atthat an attempt to compensate for operational differences

different instances in time. For example , images can beamong the individual FPAs of a multi- FPA IR imaging

compared to or subtracted from a reference image obtained 50 system with the use of methods that turn on spectral orat another time. Such reference image , which is subtracted

temporal differencing can cause additional problems leading

temporal reference image . This solution can be applied toare scene-motion -induced detection errors and parallax

from other later obtained images, may be referred to as aspectral difference images as well. For example , the image

to false detection of target species. Among these problems
caused errors that are not readily correctable and/ or com

data resulting from spectral difference images can be nor- 55 pensatable . Accordingly , there is a need to compensate formalized by the data corresponding to a temporal reference
image . For instance , the temporal reference images can beimage data acquisition and processing errors caused bymotion of elements within the scene being imaged . Various

subtracted from the spectral difference image to obtain theembodiments of data processing algorithms described herein

temporal difference image . This process is referred to , foraddress and fulfill the need to compensate for such motion

the purposes of this disclosure , as a temporal differencing 60 induced and parallax - induced image detection errors .
algorithm or method and the resultant image from subtractIn particular , to reduce or minimize parallax -induced

ing the temporal reference image from another image ( suchdifferences between the images produced with two ormore

as the spectral difference image ) is referred to as the tem -predetermined FPAs, another difference image can be usedporal difference image . In some embodiments where spec - that is formed from the images of at least two different FPAS
tral differencing is employed , a temporal reference image 65 to estimate parallax effects. Parallax error can be determined
may be formed , for example , by creating a spectral differ - by comparing the images from two different FPAs where theence image from the two or more images registered by the

position between the FPAs is known. The parallax can beUS 10,084 ,975 B2
21
calculated from the known relative position difference . Dif
ferences between the images from these two FPAs can be
attributed to parallax , especially, if the FPA have the same22
to create a three dimensional volume estimate of the gas
cloud size . It is worth noting that in the embodiments
described herein only a single imaging system is required forspectral characteristics, for example have the same spectral such volume estimation . Indeed , due to the fact that thefilter or both have no spectral filters . Parallax error correc - 5 information about the angle at which the gas cloud is seention , however, can still be obtained from two FPAs that haveby the system is decoded in the parallax effect, the image
data includes the information about the imaged scene
if the different spectral characteristics , e . g ., the transmission
viewedby the system in association with at least two angles.
spectra of the respective filters are known and/ or negligible .
When
the temporal differencing algorithm is used for
Use of more than two FPAs or FPAs of different locations 10 processing the acquired imaging data , a change in the scene
such as FPAs spaced farther apart can be useful. For that is not caused by the target species can inadvertently be
example , when the spectral differencing of the image data ishighlighted in the resulting image . In various embodiments,
performed with the use of the difference between the images
compensation for this error makes use of the temporalcollected by the outermost two cameras in the array (such as,
differencing
betweenthat are substantially equallyfor example , the FPAs corresponding to filters 2 and 3 of the 15 spectrally
allierencingsensitive
be totwotheFPAs
target species . In this case , thearray of filters of FIG . 5A ), a difference image referred to as

different spectral characteristics or spectral filters, especiallya “ difference image 2 - 3 ” is formed . In this case , the alter native “ difference image 1 -4 ” is additionally formed from

temporal difference image will highlight those pixels theintensity of which have changed in time (and not in wave

the image data acquired by, for example , the alternative length ). Therefore , subtracting the data corresponding to
FPAs corresponding to filters 1 and 4 of FIG . 5A . Assuming 20 these pixels on both FPAs, which are substantially equallyor ensuring that both of these two alternative FPAs have spectrally sensitive to the target species, to form the result
approximately the same spectral sensitivity to the target ing image, excludes the contribution of the target species tospecies, the alternative " difference image 1 - 4 ” will highlight the resulting image . The differentiation between (i) changes

pixels corresponding to parallax -induced features in the in the scene due to the presence of target species and ( ii )image. Accordingly , based on positive determination that the 25 changes in the scene caused by changes in the backgroundsamepixels are highlighted in the spectral “ difference image not associated with the target species is, therefore , possible .

2 - 3” used for target species detection , a conclusion can bemade that the image features corresponding to these pixels

are likely to be induced by parallax and not the presence of

In some embodiments , these two channels having the sameor substantially similar spectral response so as to be sub
stantially equally spectrally sensitive to the target species

target species in the imaged scene. It should be noted that 30 may comprise FPAs that operate using visible light. It should
be noted that, the data acquired with a visible light FPA
images created by individual re-imaging lenses, 128a , when also(when
as part of the otherwise IR imaging system )using a single FPA or multiple FPA 's as discussed above . can alsopresentbeusedfacilitate such differentiation and com
FPAsspaced apart from each other in different directions can 5 pensation of thetomotion-caused imaging errors. Visible
also be useful. Greater than 2 , for example, 3 or 4 , or more 35 pen
cameras generally have much lower noise figure than IR
compensation of parallax can also be performed using

FPAs can be used to establish parallax for parallax correc
tion . In certain embodiments two central FPAS and one

corner FPA are used for parallax correction . These FPA may,cameras (at least during daytime). Consequently , the tem
poral difference image obtained with the use of image datain certain embodiments, have substantially similar or thefrom the visible light FPA can be quite accurate . The visible
same spectral characteristics , for example , have filters hav - 40 FPA can be used to compensate for motion in the system as
ing similar or the same transmission spectrum or have nowell as many potential false - alarms in the scene due tomotion caused by people , vehicles, birds, and steam , forfilter at all.

Another capability of the embodiments described hereinis the ability to perform the volumetric estimation of a gas

example, as long as the moving object can be observed in thevisible region of the spectra . This has the added benefit ofcloud . This can be accomplished by using (instead of 45 providing an additional level of false alarm suppression
compensating or negating ) the parallax induced effects
without reducing the sensitivity of the system since many
described above . In this case , themeasured parallax between
targets such as gas clouds cannot be observed in the visible
two or more similar spectral response images ( e. g ., two or
spectral region . In various implementations, an IR camera
more channels or FPAs) can be used to estimate a distancecan be used to compensate for motion artifacts .
between the imaging system and the gas cloud or between 50

Another method for detection of the gases is to use athe imaging system and an object in the field of view of thespectral unmixing approach . A spectral unmixing approachsystem . The parallax induced transverse image shift, d ,between two images is related to the distance , z , between theassumes that the spectrum measured at a detector pixel iscomposed of a sum of component spectra ( e . g .,methane andcloud or object 110 and the imaging system according to theother gases ). This approach attempts to estimate the relativeequation z = - sZ '/ d . Here , s , is the separation between two 55 weights of these components needed to derive the measure

similar spectral response images, and z' is the distance to thement spectrum . The component spectra are generally taken

volume of the gas cloud , a particular symmetry in thecloud in the scene and / or to estimate the concentration of theimage plane from the back lens . The value for z ' is typicallyapproximately equal to the focal length f of the lens of theimaging system . Once the distance z between the cloud andfrom a predetermined spectral library ( for example, fromdata collection that has been empirically assembled ), thoughsometimes one can use the scene to estimate these as well
the imaging system is calculated , the size of the gas cloud 60 (often called " endmember determination " ) . In various
can be determined based on the magnification, m = f/z , where embodiments, the image obtained by the detector pixel is aeach image pixel on the gas cloud, Ax ', corresponds to aradiance spectrum and provides information about thephysical size in object space Ax = Ax '/m . To estimate the brightness of the object. To identify the contents of a gas

thickness of the cloud based on the physical size of the cloud 65 various gases in the gas cloud , an absorption / emission
can be assumed . For example , the cloud image can bespectrum of the various gases of interest can be obtained byrotated about a central axis running through the cloud image comparing the measured brightness with an estimate of theUS 10 ,084 , 975 B2
23

expected brightness. The spectral unmixing methodology

can also benefit from temporal, parallax , and motion com

pensation techniques .

In various embodiments , a method of identifying the24eter FPA array can provide a system with large number ofpixels . Various embodiments of the systems describedherein can also offer a high optical throughput for a sub
stantially low number of optical channels. For example , thepresence of a target species in the object includes obtaining 5 systems described herein can provide a high optical throughthe radiance spectrum (or the absorption spectrum ) from theput for a number of optical channels between 4 and 50 . By

object in a spectral region indicative of the presence of thehaving a lower number of optical channels ( e . g ., between 4target species and calculating a correlation (e . g ., a correla tion coefficient ) by correlating the obtained radiance spec -and 50 optical channels ), the systems described herein havewider spectral bins which allows the signals acquired withintrum ( or the absorption spectrum ) with a reference spectrum 10 each spectral bin to have a greater integrated intensity .

for the target species . The presence or absence of the target
species can be determined based on an amount of correlation

An advantage of the embodiments described herein overvarious scanning based hyperspectral systems that are con( e . g ., a value of correlation coefficient). For example , thefigured for target species detection ( for example , gas cloudpresence of the target species in the object can be confirmed

detection ) is that, the entire spectrum can be resolved in aif the amount of correlation or the value of correlation 15 snapshotmode ( for example , during one image frame acquicoefficient is greater than a threshold . In various implemen sition by the FPA array ). This feature enables the embodi
tations, the radiance spectrum ( or the absorption spectrum )ments of the imaging systems described herein to takecan be obtained by obtaining a spectral difference image
between a filtered optical channel and/ or another filteredadvantage of the compensation algorithms such as theparallax and motion compensation algorithms mentioned

optical channel/ unfiltered optical channel or any combina - 20 above. Indeed , as the imaging data required to implement

tions thereof.
For example , an embodiment of the system configured todetect the presence of methane in a gas cloud comprises
optical components such that one or more of the plurality ofthese algorithms are collected simultaneously with the tar

get -species related data , the compensation algorithms arecarried out with respect to target -species related data and notwith respect to data acquired at another time interval. This

optical channels is configured to collect IR radiation to 25 rapid data collection thus improves the accuracy of the data

provide spectral data corresponding to a discrete spectral
band located in the wavelength range between about 7 . 9 um

compensation process. In addition , the frame rate of dataacquisition is much higher. For example , embodiments ofand about 8 . 4 um corresponding to an absorption peak of

the imaging system described herein can operate at video

methane . The multispectral data obtained in the one or more

rates from about 5 Hz and higher. For example , various

optical channels can be correlated with a predetermined 30 embodiments described herein can operate at frame rates

absorption spectrum of methane in the wavelength range

between about 7. 9 um and 8 .4 um . In various implementa

from about 5 Hz to about 60 Hz or 200 Hz. Thus, the user

is able to recognize in the images the wisps and swirls

tions, the predetermined absorption spectrum of methane

typical of gas mixing without blurring outof these dynamic

can be saved in a database or a reference library accessible

image features and other artifacts caused by the change ofvalue of correlation coefficient ), the presence or absence of
methane in the gas cloud can be detected .

surements . In contradistinction , scanning based imaging

FPAs 336 to capture a video sequence substantially immediately after performing calibration measurements . The
video sequence corresponds to images of a standard labo -is in contrast to embodiments of the imaging systemdescribed herein that are capable of detecting the localized
concentrations of gas without it being smeared out with theratory scene and the calibration measurements are per

areas of thinner gas concentrations. In addition , the higher

by the system . Based on an amount of correlation ( e . g ., a 35 scene (whether spatial or spectral) during the lengthy mea

systems involve image data acquisition over a period of timeexceeding a single -snap - shot time and can , therefore , blur
Examples of Practical Embodiments and Operation
The embodiment 300 of FIG . 3 is configured to employ 12 the target gas features in the image and inevitably reduce theoptical channels and 12 corresponding microbolometer 40 otherwise achievable sensitivity of the detection . This result

formed with the use of a reference source including two 45 frame rate also enables a much faster response rate to a leak
shutters , as discussed above, one at room temperature andof gas (when detecting such leak is the goal). For example ,one 5° C . above room temperature. The use of 12 FPAs
an alarm can trigger within fractions of a second rather thanallows increased chance of simultaneous detection and estiseveral seconds .
mation of the concentrations of about 8 or 9 gases present at
To demonstrate the operation and gas detection capability
the scene . In various embodiments , the number of FPAS 336 50 of the imaging systems described herein , a prototype wascan vary, depending on the balance between the operational

constructed in accordance with the embodiment 300 of FIG .

3A and used to detect a hydrocarbon gas cloud of propylene
requirements and consideration of cost.
Due to the specifics of operation in the IR range of the at a distance of approximately 10 feet. FIG . 7 illustrates
spectrum , the use of the so -called noise -equivalent tempera - video frames 1 through 12 representing gas -cloud -detection
ture difference (or NETD ) is preferred and is analogous to 55 output 710 ( seen as a streak of light) in a sequence from t= 1
the SNR commonly used in visible spectrum instruments . to t= 12 . The images 1 through 12 are selected frames taken
The array of microbolometer FPAS 336 is characterized tofrom a video - data sequence captured at a video - rate of 15
perform at NETD = 72 mK at an f-number of 1 .2 . Each frames/sec . The detected propylene gas is shown as a streakmeasurement was carried out by summing four consecutiveof light 710 (highlighted in red ) near the center of eachframes , and the reduction in the NETD value expected due 60 image . The first image is taken just prior to the gas emerging

to such summation would be described by corresponding

factor of V4 = 2. Under idealmeasurement conditions, there fore , the FPA NETD should be about 36 mK .

from the nozzle of a gas - contained , while the last image

represents the system output shortly after the nozzle has

been turned off.

It is worth noting that the use of optically - filtered FPAs in
The same prototype of the system can also demonstratevarious embodiments of the system described herein can 65 the dynamic calibration improvement described above by
provide a system with higher number of pixels . For example ,imaging the scene surrounding the system (the laboratory )embodiments including a single large format microbolom -with known temperature differences. The result of imple

US 10 ,084 ,975 B2
25

26

menting the dynamic correction procedure is shown in
FIGS. 8A , 8B , where the curves labeled " obj” (or “ A ” )represent temperature estimates of an identified region in the

In FIG . 9A , the mirrors 975a and 975b are configured toreflect radiation from the reference sources 972a and 972(represented as ref A and ref B ). The mirrors 975a and 975b

scene . The abscissa in each of the plots of FIGS. 8A , 8B
can be disposed away from the central FOV of the detector
indicates the number of a FPA , while the ordinate corre - 5 arrays 1, 5 , 9 such that the central FOV is not blocked or
sponds to temperature ( in degrees C .). Accordingly, it is obscured by the image of the reference source 972a andexpected that when all detector elements receive radiant data

972b . In various implementations, the FOV of the detector

that, when interpreted as the object's temperature , indicatesarray 5 could be greater than the FOV of the detector arraysthat the object's temperature perceived by all detector ele

1 and 9 . In such implementations, the mirrors 975a and 975b

ments is the same, any given curve would be a substantially " can be disposed away from the central FOV of the detector

flat line . Data corresponding to each of the multiple “ obj”

array 5 at a location such that the reference source 972a andcurves are taken from a stream of video frames separated
972b is imaged by the detector array 5 . Themirrors 975a andfrom one another by about 0 . 5 seconds ( for a total of 50
975b may comprise imaging optical elements having opticalframes ). The recorded " obj” curves shown in FIG . 8A 16 power that image the reference sources 972a and 972b onto

indicate that the detector elements disagree about theobject’ s temperature , and that difference in object' s tem
perature perceived by different detector elements is as highas about 2 .5° C . In addition , all of the temperature estimates

the detector arrays 1 and 9 . In this example, the reference
sources 972a and 972b can be disposed in the same plane as
the re -imaging lenses 928a , however, the reference sources
972a and 972b can be disposed in a different plane or inlabeled “ ref” (or “ C ” ) correspond to the detectors ' estimatesof the temperature of the aperture 338 of the embodimentand 972b can be disposed in a plane that is conjugate to theplane in which the detector array 1 , detector array 5 , andare steadily drifting in time, from frame to frame. The curves 20 different locations. For example, the reference sources 972a
300 of FIG . 3A . The results of detection of radiation carriedout after each detector pixel has been subjected to thedetector array 9 are disposed such that a focused image ofthe reference sources 972a and 972b is formed by the

dynamic calibration procedure described above are 25 detector arrays . In some implementations, the reference

expressed with the curved labeled “ obj con ” (or “ B ” ). Now ,the difference in estimated temperature of the object among
the detector elements is reduced to about 0 .5° C . (thereby

spaced apart from the conjugate plane such that a defocusedsources 972a and 972b can be disposed in a plane that isimage of the reference sources 972a and 972b is formed bycorresponding to a different location in the scene ( a location

same plane.

improving the original reading at least by a factor of 5 ).
the detector arrays. In various implementations, the refer
FIG . 8B represents the results of similar measurements 30 ence sources 972a and 972b need not be disposed in thewhich is at a temperature about 9° C . above the estimatedAs discussed above , in some embodiments, the reference

temperature of the aperture 338 of FIG . 3A ). As shown , the sources 972a and 972b are imaged onto the detector array 1
correction algorithm discussed above is operable and effec
and detector array 9, without much blur such that thetive and applicable to objects kept at different temperature . 35 reference sources 972a and 972b are focused . In contrast, in
Accordingly , the algorithm is substantially temperature other embodiments, the image of reference sources 972a andindependent.

972b formed on the detector array 1, and detector array 9 are

Dynamic Calibration Elements and References
FIGS. 9A and 9B illustrates schematically different imple -blurred such that the reference sources 972a and 972b are

defocused , and thereby provide some averaging , smoothing ,mentations 900 and 905 respectively of the imaging system 40 and / or low pass filtering. The reference sources 972a andthat include a variety of temperature calibration elements tofacilitate dynamic calibration of the FPAs. The temperature972b may comprise a surface of known temperature andmay or may not include a heater or cooler attached thereto

calibration elements can include mirrors 975a , 975b ( rep -or in thermal communication therewith . For example , theresented as M , 4 , Moa, etc .) as well as reference sourcesreference source 972a and 972b may comprises heaters and972a and 972b . The implementation 900 can be similarly 45 coolers respectively or may comprise a surface with aconfigured as the embodiment 300 and include one or more
front objective lens , a divided aperture , one or more spectral

filters, an array of imaging lenses 928a and an imaging

element 936 . In various implementations, the imaging ele -temperature sensor and a heater and sensor respectively indirect thermal communication therewith to control the tem

perature of the reference surface. In various implementa
tions, the reference sources 972a and 972b can include ament 936 ( e . g ., camera block ) can include an array of 50 temperature controller configured to maintain the reference

cameras . In various implementations, the array of cameras

sources 972a and 972b at a known temperature . In somecan comprise an optical FPA unit. The optical FPA unit can

implementations, the reference sources 972a and 972b cancomprise a single FPA , an array of FPAs. In various imple -be associated with one or more sensors that measure thementations, the array of cameras can include one or moretemperature of the reference sources 972a and 972b and

5 , detector array 9 in FIGS. 9A and 9B . In various embodiments, the FOV of each of the detector arrays 1 , 5 , 9 can bedivided into a central region and a peripheral region . With -controller. In some implementations, the one or more sensors can communicate the measured temperature to thedata -processing unit. In various implementations, the referout any loss of generality , the central region of the FOV ofence sources 972a and 972b may comprise a surface ofdetector arrays represented as detector array 1 , detector array 55 communicate the measured temperature to the temperatureeach of the detector arrays 1 , 5 , 9 can include the region 60 unknown temperature . For example , the reference sources

where the FOV of all the detector arrays 1, 5 , 9 overlap . In

the embodiment illustrated in FIG . 9A , the reference sources

may comprise a wall of a housing comprising the imaging

system . In some implementations, the reference sources

972a and 972b are placed at a distance from the detector

972a and 972b can comprise a surface that need not bearrays 1 , 5 , 9 , for example , and mirrors 975a and 975b thatassociated with sensors , temperature controllers . However,can image them onto the detector arrays are then placed at 65 in other implementations, the reference sources 972a andthe location of the scene reference aperture (e . g ., 338 of FIG .
972b can comprise a surface that can be associated withsensors, temperature controllers .
3A ).

US 10 ,084 ,975 B2
27

28

In FIG . 9B , the temperature -calibration elements com

calibrated to agree with each other. As another example ,prise temperature - controlled elements 972a and 972b ( e . g .,a thermally controlled emitter , a heating strip , a heater or acooler ) disposed a distance from the detector arrays 1 , 5 , 9 .

detector arrays 1 , 5 and 9 can be calibrated to agree witheach other .

The reference sources 972a and 972b can be coated with

In various embodiments , the temperature - controlled ele - 5 a material to make it behave substantially as a blackbody

ments 972a and 972b can be disposed away from the central(for which the emission spectrum is known for any given

FOV of the detector arrays 1, 5 , 9 such that the central FOV

temperature ). If a temperature sensor is used at the location

is not blocked or obscured by the image of the reference
source 972a and 972b . The radiation emitted from theof each reference source , then the temperature can betracked at these locations. As a result, the regions in thereference sources 972a and 972b is also imaged by the 10 image of each camera ( e . g ., on the detector arrays 1 and 9 )detector array 936 along with the radiation incident from thein which the object has such known temperature (and ,object . Depending on the position of the reference sources

therefore , spectrum ) can be defined . A calibration procedure

972a and 972b the image obtained by the detector array of

can thus be used so that most of the cameras ( if not every

or more cameras in the array of cameras .

" two -point” calibration , and assumes that the measured

both the reference sources 972a and 972b . Accordingly,

multiple, more, or even every frame of a sequence , it is

the reference sources can be blurred (or defocused ) or sharp
camera ) so operated agrees, operationally, with most or( or focused ). The images 980a , 980b , 9800 , 9800 , 980e and 15 every other camera , for objects at the temperatures repre
980f of the temperature - controlled elements 972a and 972b
sented by those two sources. Calibrating infrared cameras
can be used as a reference to dynamically calibrate the one using sources at two different temperatures is known as a

In the implementations depicted in FIGS. 9A and 9B , the signal at a given pixel is linearly related to the incident
detector arrays 1 , 5 and 9 are configured to view ( or image ) 20 irradiance . Since this calibration can be performed duringmultiple frames (e . g ., every or substantially every frame)
within a sequence of images contains one ormore regions in
the image in which the object image has known thermal andreferred to as a “ dynamic calibration " .
An example of the dynamic calibration procedure is asfollows. If there is a temperature sensor on the reference

spectral properties. This allows multiple ( e . g ., most or each ) 25 sources or reference surface, then the temperature measurecameras within the array of cameras to be calibrated to agreewith other (e . g ., most or every other) camera imaging them ents obtained by these temperature sensors can be used todetermine their expected emission spectra . These tempera

same reference source (s ) or surface ( s). For example, detec -ture measurements are labeled as T _ [R ], T [ R ], and T [ R ]tor arrays 1 and 9 can be calibrated to agree with each other.

for the “ reference temperatures ” of sources /surfaces A , B ,As another example , detector arrays 1 , 5 and 9 can be 30 and C . These temperature measurements can be used as

calibrated to agree with each other. In various embodiments,

the lenses 928a provide blurred (or defocused ) images of thereference sources 972a , 972b on the detector arrays 1 and 9
because the location of the reference sources are not exactly

scalar correction factors to apply to the entire image of agiven camera, forcing it to agree with the reference tem

peratures. Correcting the temperature estimate of a given
pixel from T to T ' can use formulae analogous to thosein a conjugate planes of the detector arrays 1 and 9 . Although 35 discussed below in reference to FIGS . 10A , 10B , 10C . If nothe lenses 928a are described as providing blurred or defo -direct temperature sensor is used, then one of the cameras

cused images, in various embodiments , reference sources orcan be used instead . This camera can be referred to as thesurfaces are imaged on the detectors arrays 1 , 5 , 9 without
such blur and defocus and instead are focused images.

" reference camera ” . In this case , the same formulae as those
provided in paragraph below can be used , but with TA [ R ]Additionally optical elements may be used , such as for 40 and TB [R ] representing the temperatures of the reference
sources/ surfaces A and B as estimated by the reference
camera . By applying the dynamic calibration correction
focused images.
The temperature of the reference sources 972b , 972a canformulae , all of the other cameras are forced to match the
be different. For example, the reference source 972a can be temperature estimates of the reference camera .
example , the mirrors shown in FIG . 9A to provide suchat a temperature Tx , and the reference source 972b can be at 45 In the configuration illustrated in FIG . 9B , the referencea temperature TR lower than the temperature T 4 . A heater cansources 972a and 972b are placed such that the images of thebe provided under the temperature - controlled element 972asources on the detector arrays are blurred . The configuration
to maintain it at a temperature Tx , and a cooler can be illustrated in FIG . 9A is similar to the system 400 illustrated
provided underneath the temperature -controlled element in FIG . 4 where the reference sources are placed at an
972b to maintain it at a temperature To . In various imple - 50 intermediate image plane ( e .g ., a conjugate image plane ). Inm
mentations, the embodiments illustrated in FIGS. 9A and 9B

this configuration , the array of reference apertures, similar to

can be configured to image a single reference source 972
reference apertures 438a in FIG . 4 , will have an accompa
instead of two references sources 972a and 972b maintained
nying array of reference sources or reference surfaces suchat different temperatures . It is understood that the single
that the reference sources or surfaces (e . g ., each reference
reference source need not be thermally controlled . For 55 source or surface ) are imaged onto a camera or a detector
example, in various implementations, a plurality of detectors
array such as FPAs 1 , 5 , 9 . With this approach , the referencein the detector array can be configured to image a same

source or surface images are at a conjugate image plane andsurface of at least one calibration element whose thermalthus are not appreciably blurred , so that their images can beand spectral properties are unknown . In such implementa -made to block a smaller portion of each camera 's field oftions, one of the plurality of detectors can be configured as 60 view .

a reference detector and the temperature of the surface of theat least one calibration element imaged by the plurality ofdetectors can be estimated using the radiance spectrum

largely blocked with a reference source such as the moving" static" calibration (a procedure in which the scene isshutters 960 in FIGS . 9A and 9B , so that imaging of andetector. For example , detector arrays 1 and 9 can beplurality of elements (for example , most or each element in

obtained by the reference detector. The remaining plurality
unknown scene cannot be performed in parallel with caliof detectors can be calibrated such that their temperature 65 bration ) allows a plurality of the cameras ( for example ,mostand/ or spectral measurements agree with the referenceor each camera ) to accurately estimate the temperature of aUS 10 ,084,975 B2

30

29
the scene ) immediately after the calibration is complete. It

regions in the image in which the object image has known

ing away from one another during the process of imaging an
unknown scene. The dynamic calibration can be used toreduce or prevent this drift, so that all cameras imaging a 5
scene can be forced to agree on the temperature estimate of
the reference sources /surfaces , and adjust this correctionof the camera , if not each camera , within the array ofcameras to be calibrated to agree with most or every othercamera imaging the same reference source or sources . For
example , cameras 0 , 1 , 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 can becalibrated to agree with each other. As another example ,cameras 0 , 1 , 2 and 3 can be calibrated to agree with eachcannot , however, prevent the cameras' estimates from drift-

thermal and spectral properties . This approach allows mostduring every frame.
FIG . 10A illustrates schematically a related embodiment

other . As yet another example , cameras 0 , 1, 2 , 3 , 7 , 11, 10 ,the field of view . The cameras 0, . .. , 11 are arranged to form

can be calibrated to agree with each other. In certain

9 , 8 , 4 , 5 and 6 can be calibrated to agree with each other.
1000 of the imaging system , in which one or more mirrors 10 Accordingly, in various implementations , two , three , four,
Moa . ..M114 and Mob .. . M118 are placed within the fields five, six , seven, eight, nine, ten , eleven or twelve camerasof view ofone or more cameras 0 , . . . , 11 , partially blocking

an outer ring of cameras including cameras 0 , 1 , 2, 3 , 7 , 11,embodiments , however , not all the cameras are calibrated toagree with each other. For example, one, two, or more

10, 9 , 8 and 4 surrounding the central cameras 5 and 6 . In 15 camerasmay notbe calibrated to agree with each other whileothers may be calibrated to agree with each other. In various
and 6 can be less than or equal to the FOV of the outer ring embodiments , these mirrors may be configured to image theof cameras 0 , 1 , 2 , 3 , 7 , 11 , 10, 9 , 8 and 4 . In suchreference sources/ surfaces A and B onto different respective
various implementations , the FOV of the central cameras 5

implementations, the one or moremirrorsMA, . . .M114 and pixels a given FPA . Without any loss of generality , FIGS .
can be placed outside the central FOV of the 20 10A and 10B represent a top view of the embodiment shown
cameras 5 and 6 and is placed in a peripheral FOV of the in FIG . 9A .

More . . .M

cameras outer ring of cameras 0 , 1, 2 , 3 , 7 , 11 , 10 , 9 , 8 and

FIG . 10C illustrates schematically a related embodimenttations , the FOV of the central cameras 5 and 6 can begreater than the FOV of the outer ring of cameras 0 , 1 , 2 , 3 ,7 , 11 , 10 , 9 , 8 and 4 . In such implementations, the one or

Ro4 , . . . , R14 and Rob , . . . , Rus can be a single reference

does overlap with the central FOV of the outer ring ofof the detectors 0 , 1 , 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 . In such

sources A and B are imaged by the cameras 5 and 6 .
This design is an enhancement to the systems 300 and 400

be disposed away from the central FOV of the detector
arrays 5 and 6 such that the radiation from the reference

4 which does not overlap with the central FOV of the 1050 of the imaging system , in which one or more ' reference
cameras 5 and 6 such that the reference sources A and B are
sources Ro4 , . . . , R14 and Rob , . . . ,R B are disposed
not imaged by the cameras 5 and 6 . In various implemen - 25 around the array of detectors 0, . . . , 11 . In various

implementations , the one or more reference sources
source that is imaged by the detectors 0 , . . . , 11 . In various

more mirrors Mo4 , . . .M 4 and More . . .M e can be implementations, central detector arrays 5 and 6 can have aplaced in a peripheral FOV of the cameras 5 and 6 which 30 FOV that is equal to or lesser than the FOV of the outer ringcameras 0 , 1, 2, 3 , 7 , 11, 10 , 9, 8 and 4 such that the reference

implementations, the reference sources Roa, . . . , R114 canshown in FIGS . 3A and 4A . In the system 1000 shown in 35 sources R4, . . . , R14 is imaged only by the outer ring of

FIG . 10A , an array of two ormore imaging elements (curved

detectors 0 , 1, 2 , 3, 7, 11 , 10 , 9, 8 and 4 . In various

FPAs , for example, in the plane of the reference aperture 160
shown in FIG . 3A . These elements (mirror or imaging

detectors 0 , 1, 2 , 3 , 7 , 11, 10 , 9 , 8 and 4 . In such imple

mirrors , for example ) is installed at a distance from theimplementations, central detector arrays 5 and 6 can have a

FOV that is greater than the FOV of the outer ring of theelements ) are used to image one or more temperature - 40 mentations, the reference sources Roa , . . . , Rug can becontrolled reference sources A and B onto the detector disposed in the peripheral FOV of the detector arrays 5 andelements of two or more of the cameras . The primary
6 such that the radiation from the reference sources

difference between embodiment 1000 and embodiment 300

R2, . . . , R 14 is imaged only by the outer ring of detectors

or 400 is that now a plurality or most or all of the outer ring

0 , 1, 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 . The radiation from theof cameras in the array can image both the reference sources 45 reference sources Rp4, . . . , R 14 is therefore imaged by the

A and B instead of imaging one of the two reference source
A and B . Accordingly , most or all of the outer ring ofcameras image an identical reference source or an identicalouter ring of detectors 0 , 1 , 2 , 3 , 7 , 11 , 10 , 9 , 8 and 4 as wellas central cameras 5 and 6 . Without any loss of generality,
FIG . 10C represents a top view of the embodiment shown in

set of reference sources ( e . g ., both the reference sources A .

FIG . 9B .

different cameras or imaging different portions of the refer -underneath , adjacent to , or in thermal communication withence sources as shown in FIGS . 3A and 4A . Thus, thisreference source /surface A to give it a higher temperatureapproach improves the robustness of the calibration , as it
eliminates potential failures and errors due to the having

T4, and a cooler can be provided underneath , adjacent to , orin thermal communication with reference source B to give it

and B ) rather than using different reference sources for 50

In various implementations , a heater can be provided

additional thermal sensors estimating each reference source . 55 a lower temperature Tk. In various implementations, theThe imaging elements in the system 1000 ( shown as
embodiments illustrated in FIGS . 10A , 10B and 10C can be

mirrors in FIGS. 10A and 10B ) image one or more con configured to image a single reference source A instead oftrolled -temperature reference sources or a surface of a
two references sources A and B maintained at different
calibration element (shown as A and B in FIGS. 10A andtemperatures . As discussed above , the embodiments illus
10B ) into the blocked region of the cameras ' fields of view . 60 trated in FIGS. 10A , 10B and 10C can be configured to
FIG . 10B shows an example in which mirror Mo, images
image a same surface of a calibration element. In suchreference source /surface A onto camera 0 , and mirror MOB
images reference source / surface Bonto camera 0 , andimplementations, the temperature of the surface of thecalibration element need not be known. Many, most or each

likewise for cameras 1 , 2 , and 3 . This way, each of thereference source/surface can be coated with a material tomirrors is used to image a reference source / surface onto a 65 make it behave approximately as a blackbody, for which thedetector array of a camera , so that many , most, or every

frame within a sequence of images contains one or more

emission spectrum is known for any given temperature . If

many, most, or each camera in the array of cameras imagesUS 10 ,084 , 975 B2
31
both of references A and B , so that there are known regionsin the image of these cameras in which the object has a

32
DAISI system from a site for transport may involve sub
stantial effort and time for the user when the system isconnected to infrastructure at the site to be monitored .

known temperature (and therefore spectrum ), then one canperform a calibration procedure . This procedure can provide Accordingly, it can be desirable to provide a DAISI systemthat many, most or every camera so operated agrees with 5 that can be used independently of the facilities or infrastrucvarious, most, or every other camera , for objects at the ture at the site to be monitored . Furthermore , it can betemperatures represented by those two sources. For advantageous to implement the DAISI system in a form
example , two, three, four, five , six , seven , eight, nine , ten ,factor and with a weight that can be carried or worn by aeleven or twelve cameras can be calibrated to agree withuser. For example, a mobile DAISI system can enable theeach other. In certain embodiments , however, not all the 10 user to easily transport the system from site -to - site , whilecameras are calibrated to agree with each other. For

monitoring the presence of gases or chemicals in real-time.

to agree with each other while others may be calibrated toherein can be used to monitor potential gas leaks in any

agree with each other. As discussed above , calibration ofsuitable installation site , including, without limitation , drill

example , one, two, or more cameras may not be calibrated

It should be appreciated that each of the systems disclosed

infrared cameras using sources at two different temperatures 15 ing rigs , refineries, pipelines, transportations systems, ships
is known as a " two -point" calibration , and assumes that theor other vessels ( such as off - shore oil rigs, trains, tanker

measured signal at a given pixel is linearly related to theincident irradiance .

The dynamic calibration is used to obtain a correctedtrucks, petro -chemical plants, chemical plants , etc. In addi
tion , each of the embodiments and aspects disclosed andillustrated herein such as above, e . g., with respect to FIGS.

temperature T' from the initial temperature T estimated at 20 1- 10C , can be used in combination with each of the embodi

each pixel in a camera using the following formulae :T'[x,y,c]=(7[x,y,c]- TA [R ])G [c]+ TA [R ]where is TA[R ] is a dynamic offset correction factor, and ,[R ]G [c] = TBTB[[Rc]] –– TA
Ta[c] '

ments disclosed and illustrated herein with respect to FIGS .
11A - 14C .
FIG . 11A is a schematic diagram illustrating a mobile
infrared imaging system 1000 (e .g., a mobile or portable

DAISI system ) configured to be carried or worn by a humanuser 1275 . The user 1275 may wear a hat or helmet 1200
when he travels to a site to be monitored , such as an oil wellsite , a refinery , etc . The system 1000 shown in FIG . 11A is
is a dynamic gain correction factor. The term c discussed 30 securely mounts the system 1000 to the helmet 1200 . For
above is a camera index that identifies the camera whose
example , the support 1204 can comprise a fastener, a strap ,or any other suitable structure . Advantageously , mounting
data is being corrected .
the system 1000 to the helmet 1200 can enable the user 1275
attached to the helmet 1200 by way of a support 1204 thatto capture images within the system 's field of view ( FOV )III. Examples of a Mobile DAISI System
35 by turning his head to face a particular location to be
The DAISI systems disclosed herein can be configured to monitored . For example , the user 1275 can walk through the
be installed at a suitable location on a long -term basis , site and can capture video images of each portion of the site ,according to some embodiments . For example , the DAISI

e . g ., various structures that may be susceptible to gas orsystems disclosed in Section II above can be affixed to achemical leaks, such as valves , fittings , etc . Thus , in the

fixture mounted to the ground at a location to continuously 40 embodiment shown in FIG . 11A , the user 1275 can image

or periodically monitor the presence of gases or chemicals at
the location . In some embodiments, for example, the DAISI
systems can be attached to a pole , post, or any suitableeach portion of the site by facing the area to be imaged andensuring that the system 1000 is activated . In addition , bymounting the system 1000 to the user 's helmet 1200 , thefixture at the location to be monitored . In such arrangements ,user 1275 may use his hands for other tasks while the system

the DAISI system can continuously or periodically capture 45 1000 images the site . Although the system 1000 of FIG . 11A
multispectral, multiplexed image data of the scene , and an
is shown as being mounted to the user 's helmet 1200 , iton - board or remote computing unit can process the captured
should be appreciated that the system 1000 can instead be
image data to identify or characterize gases or chemicals atworn on other parts of the user ' s clothing or can be carried

the location . A communications module can communicate

by the user , e . g ., in a bag , case , or other suitable container.

data relating to the identified gases or chemicals to any 50 Furthermore , in some embodiments , a wind sensor can besuitable external system , such as a central computing server,

provided to the user, e. g ., on the user 's clothing and /or on or

etc . For such long -term installations of the DAISI system ,near the system 1000 . The wind sensor can be used tothe installation site may include a power source ( e . g .,electrical transmission lines connected to a junction box at

estimate wind conditions at the installation site , which canbe used to improve the detection of gas leaks. In otherthe site ) and network communications equipment ( e . g ., 55 embodiments , the system 1000 can be coupled to or formed

network wiring , routers , etc . ) to provide network communication between the DAISI system and the external sys tems.

It can be advantageous to provide a mobile DAISI systemwith a housing that defines a " gun ” -like structure which canbe aimed or pointed by the user in a particular direction .
As explained herein , a gas cloud 1202 emitted from astructure at the site can be imaged by pointing the system

configured to be worn or carried by a user . For example , it 60 1000 towards the gas cloud 1202 and capturing an image of

may be unsuitable or undesirable to install a DAISI systemat some locations on a long -term basis . As an example , somethe gas cloud 1202 when the cloud 1202 is within the FOVof the system 1000 . Unlike other systems, the system 1000

oil well sites may not have sufficient infrastructure , such as

can capture multispectral image data of a single scene overpower sources or network communication equipment, toa range of IR wavelengths with a single snapshot, as

support the DAISI system . In addition , it can be challenging 65 explained in further detail herein . The single snapshot can be
to move the DAISI system from site to site to monitor captured in a short timeframe, e . g ., less than about 3
different locations . For example , installing and removing the seconds, less than about 2 seconds, or less than about 1.5

US 10 ,084,975 B2
33

34

seconds ( for example , in about 1 second , in some embodi -data and /or the raw image data ) may also be transmitted toments). The single snapshot can be captured in greater thanabout 5 milliseconds, greater than about 0 .2 seconds , orgreater than about 0 .5 seconds. The captured image data can

a central server for centralized collection , processing, andanalysis . In various arrangements, a global positioning sys
t em (GPS ) module can also be installed on board the systembe processed on board the system 1000 by a processing unit, 5 1000 and / or on the mobile computing device (such as a
as explained in further detail herein . For example , thetablet computing device , smartphone, etc .) . The GPS modprocessing unit can process the image data from the different
optical channels and can compare the captured spectral
information with a database ofknown chemicals to identify

ule can identify the coordinates of the user 1275 when aparticular image is captured . The location data for thecaptured image data can be stored on the central server forand / or characterize the gases that are included in the gas 10 further analysis,
Thus, the system 1000 shown in FIG . 11 A can enable thecloud 1202.

A communicationsmodule on board the system 1000 canuser 1275 to image multiple locations of a particular site totransmit information relating to the identified gases or

be monitored , such as an oil well site . Advantageously , thechemicals to any suitable external device . For example , theoptical components , the processing components , and thecommunications module can wirelessly communicate ( e . g ., 15 communications components of the system 1000 can be

by Bluetooth , WiFi, etc .) the information to a suitable

integrated within a relatively small housing that can bemobile computing device , such as an electronic eyewear
apparatus 1201 , a tablet computing device 1212 , a mobile

carried or worn by the user 1275 . For example , in various
embodiments , the system 1000 does not include complex

smartphone , a laptop or notebook computer 1203 , or any mechanical components for movement, such as gimbals ,other suitable mobile computing device . In some embodi- 20 actuators , motors, etc . Without such components, the size ofments , if a gas cloud is detected , the system 1000 can warn

the system 1000 can be reduced relative to other systems.

the user by way of sending a signal to the mobile device

Unlike other systems, in which the system components
are bulky or are assembled over a large form factor, themobile system 1000 can be sized and shaped in such a( e.g., tablet computing device 1212 or a mobile smartphone.
The mobile device can emit an audible ring and /or canvibrate to notify the user of a potential gas leak . In the 25 manner so as to be easily moved and manipulated when theembodiment of FIG . 11A , the electronic eyewear apparatus
user 1275 moves about the site . Indeed , it can be very

1201 can include a user interface comprising a display thatthe user 1275 can view in real-time as he visits the site . In

challenging to integrate the various system components in asmall form - factor. Advantageously , the systems 1000 can besome embodiments , the electronic eyewear apparatus 1201

worn or carried by a human user. For example , the compo

eyewear apparatus 1201 can be further configured to present

acquisition and processing module 1020 , which may include

comprises eyewear that includes a display. The electronics 30 nents of the system 1000 can be contained together in a data
images from this display to the wearer. The electronics

a housing to support the system components. The compo

eyewear apparatus 1201may for example include projection

n ents of the system 1000 ( including the optical or imaging

optics that projects the image into the eye. The electronic

components , the focal plane array, the on -board processing

optics the presents the image on the lens portion (s ) of theeyewear so that the wearer can view the image and also see

packaged or assembled in the data acquisition and process
ing module 1020 and may occupy a volume less than about

eyewear apparatus 1201 may comprise heads up display 35 electronics , and the communications components ) may bethrough the eyewear and peer at objects in the distance .

300 cubic inches , less than about 200 cubic inches, or less

Other configurations are possible . In some arrangements, thethan about 100 cubic inches. In various embodiments, thedevice , sold by Google , Inc ., of Mountain View , Calif.

imaging components, the focal plane array, the on -board

eyewear apparatus 1201 can comprise a Google Glass 40 components of the system 1000 ( including the optical or
The processing unit can configure the processed image

processing electronics , and the communications compo

data such that the types of identified gases are displayed tonents) may be packaged or assembled in the data acquisition

the user 1275 on the display of the eyewear apparatus 1201.

and processing module 1020 and may occupy a volume

For example, in some embodiments , color- coded data may 45 greater than about 2 cubic inches , or greater than about 16

represent different types of gases or concentrations of a

cubic inches .

of the scene . For example , the color -coded data and imageof the gas cloud can be seen by the user on the electronic

the system components mounted therein or thereon ) may besized and shaped to fit within a box - shaped boundary having

particular gas, and may be overlaid on a visible light image

The data acquisition and processing module 1020 (witheyewear apparatus 1201. In various embodiments, text data 50 dimensions XxYx Z . For example, the data acquisition andand statistics about the composition of the gas cloud 1202
may also be displayed to the user 1275 . Thus, the user 1275
can walk the site and can view the different types of gasesin the gas cloud 1202 substantially in real-time. Advanta

processing module 1020 , including the imaging optics , focal
plane array, and on board processing electronics , may be
included in a package that is sized and shaped to fit within

the box - shaped boundary having dimensions Xx Yx Z . This

geously , such real-time display of the composition of the gas 55 package may also contain a power supply , such as a battery
cloud 1202 can enable the user 1275 to quickly report urgent and /or solar module . In some embodiments , the data acqui
events , such as the leakage of a toxic gas or chemical. In
sition and processing module 1020 (including the imaging
some embodiments , detection of a toxic leak can trigger an

optics, focal plane array, and on board processing electronics

alarm , which may cause emergency personnel to help evacu -may) can be sized and shaped to fit within a box -shaped

ate the site and /or fix the leak .

In some embodiments , the processed image data can betransmitted from the system 1000 to the tablet computing

60 boundary smaller than 8 inchesx6 inchesx6 inches . In someembodiments , the data acquisition and processing module1020 (including the imaging optics, focal plane array , and on

device 1212 , laptop computer 1203 , and /or smartphone . The board processing electronics may ) can be sized and shaped
to fit within a box - shaped boundary smaller than 7 inchesx5or laptop computer 1203 to conduct additional analysis of 65 inchesx5 inches, e. g., a box -shaped boundary small than 7
the imaged and processed gas cloud 1202 . Furthermore ,inchesx3 inchesx3 inches . In some embodiments , the data
information about the gas cloud (including the processed acquisition and processing module 1020 (including theuser 1275 can interact with the table computing device 1212

US 10 ,084 , 975 B2

35

imaging optics , focal plane array, and on board processing
electronics may ) can be sized and shaped to fit within abox -shaped boundary smaller than 6 inchesx4 inchesx4
inches. In some embodiments , the data acquisition and

36
volumemore than 4 cubic inches. The data acquisition andprocessing module 1020 (including the imaging optics, focal
plane array, and on board processing electronics may) canhave a volumemore 5 cubic inches. The data acquisition andprocessing module 1020 ( including the imaging optics, focal 5 processing module 1020 (including the imaging optics , focal
plane array , and on board processing electronics may ) can beplane array , and on board processing electronics may ) cansized and shaped to fit within a box - shaped boundary

have a volumemore 10 cubic inches. This package may also

ments, the data acquisition and processing module 1020

module , a communications module , or both and fit into thesmaller than 2 inchesx2 inchesx6 inches. In some embodi

contain a power supply , including a battery and/ or solar( including the imaging optics, focal plane array, and on 10 above -referenced dimensions. It should be appreciated thatboard processing electronics may ) can be sized and shaped
to fit within a box -shaped boundary having dimensions
larger than 4 inchesx2 inchesx2 inches . In some embodi-

ments, the data acquisition and processing module 1020

the dimensions disclosed herein may not correspond to thedirections shown in FIG . 11A with respect to X , Y , and Z .
Moreover, the system 1000 can have a mass and weightsufficiently small so as to enable the user 1275 to easily carry(including the imaging optics , focal plane array, and on 15 or wear the data acquisition and processing module 1020 atboard processing electronics may ) can be sized and shaped
the site . Thus, the embodiment shown in FIG . 11A can be
to fit within a box -shaped boundary having dimensions sized and shaped and configured to have a mass that enables

larger than 3 inchesx3 inchesx7 inches. In some embodi- a human user to easily and effectively manipulate the system
1000 .
(including the imaging optics, focal plane array, and on 20 FIG . 11B is a schematic diagram illustrating an installa

ments, the data acquisition and processing module 1020

board processing electronics may ) can be sized and shaped
to fit within a box -shaped boundary having dimensions

larger than 2 inchesx1 inchesx1 inches . The data acquisitionand processing module 1020 ( including the imaging optics,

tion site ( e. g . an oil well site , etc. ) that can be monitored bymultiple infrared imaging systems 1000 ( e .g ., a DAISI

system ). For example, as shown in FIG . 11B , an imaging
system 1000A can be mounted to a pole 1309 or otherfocal plane array , and on board processing electronics may ) 25 stationary structure at the site . An imaging system 1000B

can have dimensions less than 2 inchesx2 inchesx6 inches

can be worn or carried by multiple users 1275 , an imaging

The data acquisition and processing module 1020 (including
the imaging optics, focal plane array, and on board process -system 1000C can be mounted on a truck 1500, and/ or an
imaging system 1000D can be mounted on an aerial plating electronics may ) can have dimensions greater than 1

form 1501, such as an unmanned aerial vehicle (UAV ) or aarray, and on board processing electronics may ) can havedimensions greater than 2 inchesx2 inchesx4 inches . said

etc . The embodiments disclosed herein can utilize the image
data captured by any combination of the systems 1000A

data acquisition and processing module has dimensions less

1000D at the installation site to image the entire installation

inchesx1 inchesx3 inches . The data acquisition and process - 30 piloted airplane. In some arrangements, the UAV can com
ing module 1020 ( including the imaging optics, focal plane prise an airplane, a helicopter (such as a quad helicopter),than 6 inchesx3 inchesx3 inches. The data acquisition and 35 site in an efficientmanner . Indeed , each installation site canprocessing module 1020 (including the imaging optics, focal

include any suitable number and type of system 1000A

have dimensions less than 4 inchesx3 inchesx3 inches . The

greater than two systems 1000A - 1000D , greater than fiveplane array, and on board processing electronics may ) can

1000D . For example , each installation site can include

data acquisition and processing module 1020 (including thesystems 1000A - 1000D , greater than ten systems 1000A

and on board processing electronics may ) can have dimen -locations of each imaging system 1000A - 1000D based on

imaging optics , focal plane array , and on board processing

tion . The distributed nature of the imaging systems 1000A

imaging optics , focal plane array , and on board processing 40 1000D , greater than twenty systems 1000A - 1000D . Eachelectronics may ) can have dimensions less than 3 inchesx2 installation site may include less than about 100 systems
inchesx2 inches . The data acquisition and processing mod 1000A - 1000D .
ule 1020 (including the imaging optics, focal plane array,
For example , the central server can track the real- timesions greater than 2 inchesx1 inchesx1 inches. The data 45 the GPS coordinates of the particular system or on preacquisition and processing module 1020 ( including thedetermined knowledge about the system ' s stationary loca
electronics may ) can have dimensions greater than 1 inchesx

1000D can provide rich information to the central server

0 .5 inchx0 .5 inch . The data acquisition and processing

about the types and locations of gas leaks or other problems

plane array , and on board processing electronics may ) can

1000D , it should be appreciated that other types of systems

module 1020 ( including the imaging optics, focal plane 50 throughout multiple installation sites . Although FIG . 11B
array , and on board processing electronics may ) can have aillustrates a stationary system 1000A mounted to a fixture , a
volume less than 30 cubic inches . The data acquisition and portable system 1000B to be worn or carried by a human , a
processing module 1020 ( including the imaging optics , focal truck -based system 1000C , and an aerial-based systemhave a volume less than 20 cubic inches. The data acquisi- 55 may be suitable . For example , in some embodiments , ation and processing module 1020 ( including the imaging
robotic vehicle or a walking robot can be used as a platform

optics, focal plane array, and on board processing electronics

for the systems 1000 disclosed herein . In various embodi

may ) can have a volume less than 15 cubic inches . The data

ments, a floating platform ( such as a boat ) can be used as aelectronics may ) can have a volume less than 10 cubic

utilize any combination of the platforms ( e . g ., stationary

inches . The data acquisition and processing module 1020

fixtures such as a pole , human user( s ), truck ( s ) or otheracquisition and processing module 1020 (including the platform for the systems 1000 disclosed herein . It should
imaging optics , focal plane array, and on board processing 60 also be appreciated that the systems disclosed herein can( including the imaging optics, focal plane array, and on vehicle, aerial platform (s), floating platform ( s), robotic platboard processing electronics may ) can have a volumemore form (s ), etc .) to support the systems 1000 .
than 1 cubic inches . The data acquisition and processing 65 The systems 1000 shown in FIG . 11B can comprise amodule 1020 (including the imaging optics , focal plane mobile DAISI system , similar to that illustrated in FIG . 11A .
array, and on board processing electronics may ) can have a In other embodiments , the systems 1000 can comprise aUS 10 ,084 ,975 B2
37

38

larger DAISI system configured for use on a relatively

the truck 1500 throughout the installation site to image

long - term basis . For example , the stationary imaging systemvarious portions of the site to detect leaks. In addition , the1000A shown in FIG . 11B can be installed on a pole 1309
or other suitable structure for monitoring a storage tankdetect gas leaks . As explained herein , the location of theuser can drive the truck 1500 to other installation sites to1301. A solar panel 1300 can be provided at or near the 5 truck 1500 can be communicated to the central server andsystem 1000 to help provide power to the system 1000 . An
antenna 1303 can electrically couple to the system and can

the location of the truck 1500 can be associated with each
captured image . The truck 1500 may include GPS electron

provide wireless communication between the system 1000

i cs to assist in tracking the location of the truck 1500 and / orand any other external entity , such as a central server, for system 1000C over time as the user drives from place tostoring and/ or processing the data captured by the system 10 place . Similarly , the aerial platform 1501 (such as an
1000 .
unmanned aerial vehicle , or UAV ) can support the imaging
The stationary infrared imaging system 1000A can be system 1000D . The aerial platform 1501 can be piloted
programmed to continuously or periodically monitor the( either remotely or non -remotely ) to numerous installationsite . If a gas cloud 1302 escapes from the storage tank 1301,

sites to capture multispectral image data to detect gas clouds.

such as by leaking from a broken valve, then the system 15 Thus, the systems 1000A - 1000D can provide extensive
1000A can capture a multispectral, snapshot image or series data regarding the existence of leaks at numerous installaof images ( e .g ., a video stream ) of the gas cloud 1302 . Aswith the embodiment of FIG . 11A , the imaging systemtions across an organization . Monitoring numerous cameras
simultaneously or concurrently across an organization , site ,1000A can include imaging , processing, and communica -region , or the entire country can be enabled at least in parttions components on board the system 1000A to identify and 20 by providing wireless ( or wired ) communication between
characterize the types of gases in the cloud 1302 and to
the systems 1000A - 1000D and one or more central servers .

transmit the processed data to the central server, e .g., by wayof the antenna 1303.

The imaging systems 1000B worn or carried by the

Advantageously , the collection of image data from multiple
sources and multiple platforms can enable the organization

to create a real-time mapping of potential gas leaks, themultiple users 1275 can advantageously capture and process 25 types and amounts of gases being leaks, the locations of themultispectral image data of the portions of the installation
leaks, and the time the image data of the leak was captured .

site that each user 1275 visits . It should be appreciated that

In some arrangements , the aggregation of data about a site

different portions of the installation site ( and also to aa gas leak is detected at a particular installation , the embodi

the different users 1275 may work in or travel throughcan improve the safety of installation sites . For example , ifnumber of installation sites ) over a period of time. When 30 ments disclosed herein can alert the appropriate personnel,
activated , the imaging systems 1000B worn or carried by thewho can begin safety and /or evacuation procedures . Moreusers 1275 can continuously or periodically capture multi-

over , the aggregation of data across an organization ( such as

spectral image data of the different locations at the instal-an oil service company ) can provide site -wide , region -wide ,lation site (s) to which the user 1275 travels. As explained

and /or company -wide metrics for performance . For

herein , the system 1000B can transmit the image data and 35 example , a given facility can monitor its total emissions overthe location at which the image was captured to the central

time and use the resulting data to help determine theserver. If the system 1000B or the central server detects a

facility ' s overall performance . A given region ( such as aproblem ( such as a gas leak ), then the central server canmetropolitan area , a state , etc .) can monitor trends in emis

associate that leak with a particular location and time.

sions over time, providing a value on which to base deci

locations and viewing from different perspectives, the cen tral server can create an organization -wide mapping of gas
leaks that include, e .g ., the locations of gas leaks in any of

to improve performance , and /or whether the entire company

expanse or extent of each gas leaked , the particular user
1275 that captured the image data , and the time at which thetoring system for decision making. In addition , the systems
1000 disclosed herein can be used in a feedback control

a portable imaging system 1000B can contribute informationon the gases detected by the system ( s ) 1000 . Accordingly, aFurthermore ,because the central server can receive image 40 sions . Likewise , a company can look at the emissions
data and location data from multiple users at different performance at all of its facilities and can make decisions

about whether some facilities should make new investments
should make various improvements. The mobile systems

multiple installation sites , the type and concentrations and 45 1000 disclosed herein can thus provide a ubiquitous moni

image was taken . Thus, each user 1275 that carries or wears

process to improve various manufacturing procedures based

to the central server that, when aggregated by the central 50 controlmodule may be provided to adjust the manufacturing

server,provides rich details on the status of any gas leaks at
The truck -mounted imaging system 1000C can beprocedure and / or parameters according to the gases mea
The embodiments of the mobile infrared imaging system

any installation sites across the organization .

sured by the system 1000.

mounted to a truck or other type of vehicle ( such as a car,1000 disclosed herein provide various advantages over othervan , all -terrain vehicle , etc .) . As shown in FIG . 11B , the 55 systems. As explained above, aggregation of data about aimaging system 1000C can be connected to an end of an
extendable pole or extension member mounted to the truck
1500 . The system 1000C can be raised and lowered by acontrol system to enable the system 1000C to image a widesite and its potential gas leaks can provide an organizationor system -wide mapping of potential problems. Further
more , automatic detection of gas leaks (and identification of
the gases in the gas cloud ) can simplify operation of thearea of the installation site . In some embodiments , actuators 60 system 1000 and can reduce the risk of user errors incan be provided to change the angular orientation of theattempting to detect or identify gas clouds manually . Moresystem 1000C , e. g., its pitch and yaw . A vibration isolation over, the small size of the systems 1000 disclosed herein are
or reduction mechanism can also be provided to reduce more easily carried or worn by the user than other systems.
vibrations , which may disturb the imaging process . The
In addition , the systems 1000 disclosed herein can overlay
system 1000C can be battery powered and/or can be pow - 65 the identified gas clouds on a visible image of the scene andered by the truck ; in some embodiments, a generator can beused to supply power to the system 1000C . A user can drive

can color code the gas cloud according to , e . g ., type of gas ,concentration , etc .

US 10 ,084 , 975 B2
39

FIG . 12 is a schematic system block diagram showing amobile infrared imaging system 1000 ( e .g ., a mobile DAISI40
Advantageously , the processor 1023 can comprise one ormore field -programmable gate arrays ( FPGA ) configured tosystem ), according to one embodiment . The imaging system

execute methods used in the analysis of the images captured

1000 can include a data acquisition and processing moduleby the optical system 1015 . For example , the FPGA can

1020 configured to be worn or carried by a person . The data 5 include logic gates and read access memory (RAM ) blocks
acquisition and processing module 1020 can include, con - that are designed to quickly implement the computations
tain , or house an optical system 1015 , a processing unit
used to detect the types of gases in a gas cloud . The small
1021, a power supply 1026 , a communication module 1025 ,size/weight, and high performance characteristics of theand GPS module 1025 . In other embodiments , the data

FPGA can enable on board computation and analysis withinacquisition and processing module 1020 can be configured 10 the data acquisition and detection unit 1020 worn or carried

to be mounted to a structure at the site to be monitored , suchby the user. The use of FPGA ( or similar electronics ) on

as a post. The power unit 1026 can be provided on board thesystem 1000 . The power unit 1026 can be configured toboard the system 1000 can reduce costs associated withusing an off - site central server or larger computing device toprovide power to the various system components, such as

conduct the image analysis computations. In addition ,the optical system 1015 , the processing unit 1021 , the 15 enabling computation with one or more FPGA devices oncommunication module 1024 , and / or the GPS module 1025 .
board the wearable system can also prevent or reduce

In some embodiments , the power unit 1026 can comprise

one or more batteries (which may be rechargeable) to power
the system components. In some embodiments , the power

communication bottlenecks associated with wirelessly transmitting large amounts of raw data from the system 1000 toa remote server or computer, which can be used in someunit 1026 can include a solar power system including one or 20 embodiments .

more solar panels for powering the system by sunlight. In

The communication module 1024 can be configured tosome embodiments , the power unit 1026 can include various

communicate with at least one device physically separate

by standard power transmission lines to DC power for

example , the communication module 1024 can include a

optical system 1015 can include an optical focal plane arraythe data acquisition and processing module 1020 and aanother. The two optical channels can be positioned totransfer IR radiation incident on the optical system towards

notebook computer, or any other suitable mobile computing
device . As explained herein , the mobile device can include

power electronics circuits for converting AC power supplied

from the data acquisition and processing module 1020 . For

powering the system components . Other types of power 25 wireless communication module configured to wirelessly
supply may be suitable for the power unit 1026 .
communicate with the at least one separate device . The
The system 1000 can include an optical system 1015
wireless communication module can be configured to pro
configured to capture multispectral image data in a single
vide wireless communication over wireless networks ( e. g .,snapshot, as explained herein . The optical system 1015 can
WiFi internet networks, Bluetooth networks, etc .) and /orcorrespond to any suitable type of DAISI system , such as, 30 over telecommunications networks (e . g ., 3G networks, 4G
but not limited to , the optical systems and apparatus illus networks, etc .).
trated in FIGS. 1 - 10C above and /or in the optical systems
In some embodiments, for example, the wireless commu
1015 illustrated in FIGS. 13A - 13B below . For example , the nication module can provide data communication between( FPA ) unit and components that define at least two optical 35 mobile device such as an electronic eyewear apparatus, achannels that are spatially and spectrally different from one tablet computing device , a mobile smartphone, a laptop or

the optical FPA . The multiple channels can be used toa display on which the processed image data can be dis

multiplex different spectral images of the same scene and to 40 played to the user. For example , the types ( and/ or concen
image the different spectral images on the FPA unit.
trations) of gases in a gas cloud can be illustrated on the

The processing unit 1021 can also be provided on board

display, e. g ., by color coding or other suitable illustration

the data acquisition and processing module 1020 . The pro -scheme. The processed data can overlie a visible image ofcessing unit 1021 can include a processor 1023 and acooperation with the memory 1022 , which can contain acomputer - readable code that, when loaded onto the processor 1023 , enables the processor 1023 to acquire multispec -the scene in some arrangements . In some embodiments, thenication between the system 1000 and an external device
remote from the system 1000 , such as a central server. For
example , the processed image data and /or the raw image

memory 1022 . The processor 1023 can be in operable 45 wireless communication module can provide data commu

tral optical data representing a target species of gas ordata may be transmitted over a telecommunications network

chemical from IR radiation received at the optical FPA unit 50 to the central server for storage and / or further analysis . Inof the optical system 1015 . The memory 1022 can be any

suitable type ofmemory ( such as a non -transitory computer readable medium ) that stores data captured by the opticalsome embodiments, the processed or raw image data can be

uploaded to the mobile device ( e . g ., notebook computer,smartphone , tablet computing device , etc . ), which can insystem 1015 and /or processed by the processing unit 1021 . turn communicate the image data to the central server.
The memory 1022 can also store the software that is 55 The GPS module 1025 can be configured to determine theexecuted on the processor 1023 . The processor 1023 can be location of the data acquisition and processing module 1020
configured to execute software instructions that process theat a particular time. The processing unit 1021 can store themultispectral image data captured by the optical system

location data and can associate the location data with a1015 . For example , the processor 1023 can analyze theparticular image captured by the optical system 1015 indifferent images detected by the FPA and can compare the 60 some arrangements . The location data associated with thecaptured data with known signatures of various types of

captured images can be transmitted by the communication

gases or chemicals . Based on the analysis of the captured

module 1024 ( or by an external device ) to a central server inimage data , the processor can be programmed to determinesome arrangements .

the types and concentrations of gases in a gas cloud . Further,
The optical system 1015 , the processing unit 1021 , theas explained herein , the processor 1023 can analyze cali- 65 power supply 1026 , the communication module 1024 , and/bration data provided by the optical system 1015 to improveor the GPS module 1025 may be contained or housed in thedata acquisition and processing module 1020 , which can be
the accuracy of the measurements .

US 10 ,084 , 975 B2
42

carried or worn by the user. The components of the system

1000 (including the optical components, the processing
components , and the communications components ) may besystem 1015 can include a focal plane array (FPA ) unit 1008

configured to record infrared image data captured by thesystem 1000 . As shown in FIG . 13A , the FPA unit 1008 may

packaged or assembled in the data acquisition and process - advantageously be uncooled , e .g., devoid of a cooling sys
ing module 1020 and may occupy a volume less than about 5 tem .
300 cubic inches, less than about 200 cubic inches, or less
The optical system 1015 can include a frontwindow 1006
than about 100 cubic inches . In various embodiments , the through which light from the object 1007 passes. A firstcomponents of the system 1000 ( including the optical com - moveable blackbody source 1003 and a second moveable
ponents , the processing components, and the communica - blackbody source 1004 can be provided to enable calibration
tions components ) may be packaged or assembled in the 10 of the optical system 1015 . The moveable sources 1003 ,data acquisition and processing module 1020 and may
1004 can bemoved in front of the field of view such that theoccupy a volume greater than about 2 cubic inches , or

optics image these sources for calibration . For example , theing a battery and /or solar module may also be included

maintained at different known temperatures in a stableacquisition and processing module 1020 and fit into the

be attached to each blackbody source 1003 , 1004 to provide

above- referenced volumetric dimensions .

feedback to create a stable and known temperature differ

within a box -shaped boundary having dimensions XxYxZ .
For example, in some embodiments, the data acquisition andsystem 1000 . As shown in FIG . 13A , one or more calibration
sources 1009 , 1010 can be provided . The calibration sources

greater than about 16 cubic inches . A power supply , includ -first and second blackbody sources 1003, 1004 can beamong the components packaged or assembled in the data 15 manner. For example , a heater and a temperature sensor can

The data acquisition and processing module 1020 (withence ( e .g ., at least 50 mK in some arrangements ) between
the system components mounted therein or thereon , includ different spatial regions of the sources .
ing the imaging optics , focal plane array , and on board 20 In addition , the optical system 1000 can include a
processing electronics may ) may be sized and shaped to fit
dynamic calibration apparatus to dynamically calibrate theprocessing module 1020 (including the imaging optics , focal
1009 , 1010 can comprise a thermal electrically controlled
plane array, and on board processing electronics may ) can be 25 ( TEC ) material with a temperature sensor attached thereto .
sized and shaped to fit within a box - shaped boundary
The calibration sources 1009 , 1010 can be coated with asmaller than 8 inchesx6 inchesx6 inches. In some embodi- spectrally measured coating or paint. Light from the caliments, the data acquisition and processing module 1020 bration sources 1009 , 1010 can be reflected from one or(including the imaging optics , focal plane array , and on more mirrors 1005 and directed through the lens array 1002
board processing electronics may ) can be sized and shaped 30 (described below ) to be imaged on a portion of the FPA unit
to fit within a box - shaped boundary smaller than 7 inchesx5

inchesx5 inches . In some embodiments , the data acquisitionand processing module 1020 ( including the imaging optics,

focal plane array, and on board processing electronics may )1 008 to assist in dynamically calibrating the system 1000

(e.g.,while imaging of the target gas cloud is simultaneously

being imaged ).

The optical system 1000 can include a lens array 1002 to

can be sized and shaped to fit within a box - shaped boundary 35 focus the incoming light onto the FPA unit 1008 . As shownsmaller than 6 inchesx4 inchesx4 inches. In some embodiin FIG . 13A , each lens of the lens array 1002 can at least

ments, the data acquisition and processing module 1020

partially define or be included in an optical channel to be(including the imaging optics , focal plane array , and on

imaged by the FPA unit 1008 . To improve the mobility andboard processing electronics may ) can be sized and shaped

portability of the mobile imaging system 1000 , the lens arrayto fit within a box - shaped boundary having dimensions 40 1002 can comprise an integrated unit formed from orlarger than 4 inches by 2 inchesx2 inches . In some embodi-

assembled into a single unitary body. Such an integrated lensments, the data acquisition and processing module 1020

array 1002 can reduce the size of the imaging system 1015 ,( including the imaging optics, focal plane array, and on

and therefore , the size of the system 1000 , to at leastboard processing electronics may ) can be sized and shaped

partially enable the system 1000 to be worn or carried by thelarger than 2 inches by 1 inchesx1 inches . A power supply,
including a battery and/ or solar module, a communications

any suitable manner. For example , in various embodiments ,the lens array 1002 can be formed by a diamond milling tool.

to fit within a box - shaped boundary having dimensions 45 user. The lens array 1002 can be monolithically formed inmodule, or both may be included in the data acquisition andIn some embodiments, the lens array 1002 can comprise a

processing module 1020 and fit into the above -referenced monolithic piece of transparent material which has separate
dimensions . It should be appreciated that the dimensions 50 regions shaped into curved refractive surfaces for creating
disclosed herein may not correspond to the directions shown
separate lenses . In some embodiments , the lenses can bein FIG . 11A with respect to X , Y , and Z . Moreover, theinserted into an array of openings formed in a plate orsystem 1000 can have a mass and weight sufficiently small

substrate to create the lens array 1002 .

so as to enable the user 1275 to easily carry or wear the dataacquisition and processing module 1020 at the site .

FIG . 13A is a schematic system diagram of an optical

The optical system 1000 can also include an array of

55 infrared ( IR ) filters 1001 configured to filter wavelengths ofinfrared light in an appropriate manner. Examples of IR

system 1015 configured to be used in the mobile infrared

filters and filtering techniques are disclosed herein , forimaging systems 1000 disclosed herein , according to various

example , with respect to FIGS . 5A -6D . As shown in FIG .

embodiments . As explained herein , the optical system 1015
13A , the IR filters 1001 can be disposed between the lens
shown in FIG . 13A can be provided in the data acquisition 60 array 1002 and the FPA unit 1008 . The IR filters 1001 canand processing module 1020 to be worn or carried by theat least partially define the multiple optical channels to be

multispectral image data of an object 1007 , such as a gas

filters 1001 can be positioned between the lens array 1002

explained herein with respect to FIGS. 1 - 10C . The opticalsystem 1015 configured to be used in the mobile infrareduser . The optical system 1015 can be configured to capture

imaged by the FPA unit 1008 . In some embodiments , the IR

cloud , chemical spill, etc . The components of the opticaland the first and second moveable blackbody sources 1009 ,system 1015 shown in FIG . 13A may be similar to or the 65 1010 .
same as the components of the optical systems and devices
FIG . 13B is a schematic system diagram of an opticalUS 10 ,084 , 975 B2
43
44
imaging systems 1000 disclosed herein , according to various
As explained herein , the system 1000 can be configured
embodiments . As explained herein , the optical system 1015
to be worn or carried by a human user. Accordingly , the data
shown in FIG . 13B can be provided in the data acquisition acquisition and processing module 1020 can be suitably
and processing module 1020 to be worn or carried by the dimensioned such that a user can easily wear or carry theuser. The components of the optical system 1015 shown in 5 system 1000 . For example, the data acquisition and process

FIG . 13B may be similar to or the same as the componentsof the optical systems and devices explained herein withrespect to FIGS. 1 - 10C and 13A .
The optical system 1015 of FIG . 13B can include an FPA

ing module 1020 can be defined at least in part by dimensions XxYxZ , as shown in FIGS. 14A and 14B .
Unlike other systems, in which the system components
are bulky or are assembled over a large form factor, thein FIG . 13A , the system 1015 of FIG . 13B can include afront window 1406 through which light from the object 1409

usermoves about the site. Indeed, it can be very challenging
to integrate the various system components in a small

unit 1408 configured to image an object 1409, such as a gas " mobile system 1000 can be sized and shaped in such a
cloud or chemical leak . As with the embodiment illustrated manner so as to be easily moved and manipulated when thepasses , first and second moveable blackbody sources 1403 , 16 form - factor. Advantageously , the systems 1000 disclosed

1404 , an IR filter array 1401, and a lens array 1402 . As with herein can be worn or carried by a human user. For example ,the embodiment of FIG . 13A , the lens array 1402 can the components of the system 1000 can be contained
comprise a unitary or monolithic body . In the embodiment together in the data acquisition and processingmodule 1020,
of FIG . 13B , the lens array 1402 may be disposed between which may include the housing 1590 to support the system
the filter array 1401 and the FPA unit 1408 . In other 20 components . The components of the system 1000 ( including
arrangements , the filter array 1401may be disposed between the optical or imaging components, the focal plane array , thethe lens array 1402 and the FPA unit 1408 .
on -board processing electronics, and the communications
The optical system 1015 of FIG . 13B can include a
components ) may be packaged or assembled in the datacooling unit 1430 configured to cool the FPA unit 1408 . The
acquisition and processing module 1020 and may occupy a
cooling unit 1430 can comprise a cooling finger configured 25 volume less than about 300 cubic inches , less than about 200

to cryogenically cool the FPA array 1408 in various arrange ments . As shown in FIG . 13B , the filter array 1401, the lenscubic inches , or less than about 100 cubic inches . In various
embodiments , the components of the system 1000 ( includ

1450 . Disposing the blackbody sources 1403 , 1404 at

data acquisition and processing module 1020 and mayarray 1402, and the FPA unit 1408 can be disposed in a
ing the optical or imaging components, the focal plane array ,cooled region 1440. The blackbody sources 1403 , 1404 andthe on -board processing electronics, and the communica
front window 1406 can be disposed in an uncooled region 30 tions components ) may be packaged or assembled in the

uncooled temperatures and the filter array 1401 , lens array
occupy a volume greater than about 2 cubic inches , or1402, and FPA unit 1408 at cooled temperatures can assist greater than about 16 cubic inches . In some embodiments ,the components of the system 1000 ( including the optical orin the periodic calibration of the system 1000 .
FIG . 14A is a schematic perspective view of a mobile 35 imaging components , the focal plane array , the on -board
infrared imaging system 1000 ( e. g ., mobile DAISI system ) processing electronics, and the communications compo
mounted to a helmet 1200 , according to various embodi
nents )may be packaged or assembled in the data acquisition

ments . FIG . 14B is an enlarged schematic perspective viewof themobile infrared imaging system 1000 shown in FIG .

and processing module 1020 and may occupy a volume in arange of about 4 cubic inches to about 15 cubic inches . In

personal protective equipment and can also advantageously
be used as a platform for the imaging system 1000 . As
explained above , the helmet 1200 can be worn by a user as

including the optical or imaging components , the focal
plane array , the on -board processing electronics, and thecommunications components ) may be packaged or

the user visits a particular installation site to be monitored ,assembled in the data acquisition and processing module

be activated to continuously monitor and analyze the sitesthat the user visits. The system 1000 can thereby continu -inches to about 12 cubic inches. In some embodiments , thecomponents of the system 1000 ( including the optical or

ously and actively search for gas leaks wherever the user

imaging components , the focal plane array, the on -board

14A . The helmet 1200 can comprise a portion of a user ' s 40 some embodiments , the components of the system 1000

such as an oil well site , a refinery, etc . The system 1000 can 45 1020 and may occupy a volume in a range of about 5 cubic

visits and can initiate an alarm or other notification if a leakprocessing electronics, and the communications compo
50 nents ) may be packaged or assembled in the data acquisition
is detected .
In the embodiment illustrated in FIG . 14B , the imaging
and processing module 1020 and may occupy a volume in asystem 1000 can comprise a housing 1590 , within or torange of about 4 cubic inches to about 6 .5 cubic inches , e. g .,which a data acquisition and processing module 1020 ( see , about 5 .63 cubic inches in one embodiment. In somee . g ., FIG . 12 and associated description ) is mounted or embodiments, the components of the system 1000 (includ
coupled . A support 1592 can be coupled to or formed with 55 ing the optical or imaging components, the focal plane array ,the housing 1590 and can be configured to attach to the the on -board processing electronics, and the communica

helmet 1200 or to any other suitable platform . For example,
in some embodiments , the support 1592 can include one ormore mounting holes for attaching to the helmet 1200 bytions components ) may be packaged or assembled in thedata acquisition and processing module 1020 and may
occupy a volume in a range of about 9 cubic inches to about

way of, e . g ., screws, bolts , or other fasteners . In addition , as 60 13 cubic inches , e . g ., about 11. 25 cubic inches in one

shown in FIG . 14B , a front window 1506 can be provided ata front end of the housing 1590 . The front window 1506 canbe transparent to IR radiation and can at least partially define
the aperture of the system 1000. In some embodiments, theembodiment. In some embodiments, the components of thesystem 1000 (including the optical or imaging components ,the focal plane array, the on -board processing electronics ,and the communications components ) may be packaged orwindow 1506 comprises germanium . A diamond like coat- 65 assembled in the data acquisition and processing moduleing (DLC ) or other coating or layer can be disposed over thewindow 1506 to provide a durable surface .

1020 and may occupy a volume in a range of about 6 cubic
inches to about 10 cubic inches .

US 10 ,084 ,975 B2
45
The data acquisition and processing module 1020 ( withthe system components mounted therein or thereon ) may besized and shaped to fit within a box-shaped boundary having

dimensions XxYxZ . For example, the data acquisition and

461020 ( including the imaging optics , focal plane array, and on

board processing electronics may ) can have a volumemore

10 cubic inches . This package may also contain a power

supply, including a battery and /or solar module, a commu

processingmodule 1020 , including the imaging optics , focal 5 nications module , or both and fit into the above -referenced
plane array , and on board processing electronics may bedimensions. It should be appreciated that the dimensions
included in a package that is sized and shaped to fit within
disclosed herein may not correspond to the directions shown

the box - shaped boundary having dimensions XxYxZ . This

package may also contain a power supply , such as a battery

in FIG . 11A with respect to X , Y , and Z . This package may

also contain a power supply , including a battery and/or solar

and / or solar module . In some embodiments , the data acqui- 10 module , a communications module , or both and fit into thesition and processing module 1020 ( including the imaging

above - referenced dimensions . It should be appreciated that

optics , focal plane array, and on board processing electronics

the dimensions disclosed herein may not correspond to themay ) can be sized and shaped to fit within a box -shaped

directions shown in FIG . 11A with respect to X , Y , and Z .

boundary smaller than 8 inchesx6 inchesx6 inches. In some

In some embodiments , the dimension X shown in FIG .

embodiments , the data acquisition and processing module 15 14B can be in a range of about 2 inches to about 7 inches,

1020 (including the imaging optics, focal plane array, and on
board processing electronics may ) can be sized and shaped

or more particularly , in a range of about 2 inches to about 4
inches , e. g ., about 2 .5 inches in one embodiment. In someto fit within a box -shaped boundary smaller than 7 inchesx5

embodiments, the dimension X shown in FIG . 14B can be in

inchesx5 inches . In some embodiments , the data acquisitiona range of about 4 inches to about 6 inches, e. g ., about 5
and processing module 1020 ( including the imaging optics , 20 inches in one embodiment. In some embodiments , thefocal plane array , and on board processing electronics may
dimension Y shown in FIG . 14B can be in a range of about

can be sized and shaped to fit within a box -shaped boundary

smaller than 6 inchesx4 inchesx4 inches. In some embodi-sn

1 inch to about 5 inches, or more particularly, in a range ofabout 1 inch to about 3 inches, e.g ., about 1. 5 inches in one

ments, the data acquisition and processing module 1020
embodiment. In some embodiments, the dimension Z shown(including the imaging optics , focal plane array , and on 25 in FIG . 14B can be in a range of about 1 inch to about 5
board processing electronics may ) can be sized and shaped
inches , or more particularly , in a range of about 1 inch toto fit within a box -shaped boundary smaller than 6 inchesx2
about 3 inches , e. g ., about 1. 5 inches in one embodiment.
inchesx2 inches . In some embodiments , the data acquisition
Moreover, the system 1000 can have a mass and weightand processing module 1020 ( including the imaging optics ,sufficiently small so as to enable the user 1275 to easily carry
focal plane array, and on board processing electronics may ) 30 or wear the data acquisition and processing module 1020 atcan be sized and shaped to fit within a box -shaped boundary

the site . For example, the system 1000 can have a weight inhaving dimensions larger than 4 inchesx2 inchesx2 inches .
In some embodiments , the data acquisition and processing
module 1020 (including the imaging optics, focal planea range of about 0 .5 pounds to 5 pounds, or more particu
particularly in a range of about 0 .25 pounds to about 2
larly , in a range of about 0 . 5 pounds to 2 pounds, or morearray , and on board processing electronics may ) can be sized 35 pounds, or more particularly , in a range of about 0 .25 pounds

and shaped to fit within a box - shaped boundary having

to about 1. 5 pounds . In one embodiment, for example , thedimensions larger than 2 inchesx1 inchesx1 inches. The dataacquisition and processing module 1020 ( including theimaging optics , focal plane array, and on board processing

system 1000 can weight about 1 pound. In another embodi
ment, for example , the system 1000 can weigh about 0 .5
pounds. Thus, the embodiment shown in FIG . 11A can beule 1020 (including the imaging optics , focal plane array,

1000 .

and on board processing electronics may ) can have dimen -FIG . 14C is a perspective cross- sectional view of theelectronics may ) can have dimensions less than 3 inchesx2 40 sized and shaped and configured to have a mass that enables
inchesx2 inches . The data acquisition and processing mod - a human user to easily and effectively manipulate the systemsions greater than 1 inchesx0 .5 inchx0 .5 inch . The data mobile infrared imaging system 1000 shown in FIGS. 14A
acquisition and processing module 1020 ( including the 45 14B . The mobile infrared imaging system 1000 can include
imaging optics , focal plane array , and on board processing
one or more movable shutters 1503 ( e .g ., two shutters ) rearelectronics may ) can have a volume less than 30 cubicof the window 1506 and a lens assembly 1502 rear of theinches. The data acquisition and processing module 1020
(including the imaging optics , focal plane array , and on

shutter (s) 1503 . A filter array 1501 can be disposed rear (orforward ) of the second lens array 1502B , and an opticalboard processing electronics may ) can have a volume less 50 focal plane array (FPA ) unit 1508 can be disposed rear of thethan 20 cubic inches . The data acquisition and processing
filter array 1501 . The optical FPA unit 1508 can be mechani

module 1020 ( including the imaging optics, focal planearray, and on board processing electronics may ) can have acally and electrically coupled with one or more substrates
1586 , which may comprise printed circuit board or PCB

volume less than 15 cubic inches . The data acquisition and substrates. In various embodiments, the FPA unit 1508
processing module 1020 ( including the imaging optics, focal 55 comprises a single FPA or detector array . Additionally, as
plane array , and on board processing electronics may) can
explained herein , the lens assembly 1502 , filter array 1501 ,have a volume less than 10 cubic inches . The data acquisition and processing module 1020 (including the imaging

and optical FPA unit can at least partially define one or more
optical channels that are spatially and spectrally different. A

optics , focalplane array, and on board processing electronics

number of the optical channels can be at least 4 , at least 5 ,may) can have a volumemore than 1 cubic inches. The data 60 at least 8 , at least 9 , at least 12 , at least 13 , or at least 20 . In
acquisition and processing module 1020 (including theimaging optics , focal plane array, and on board processing

electronics may ) can have a volume more than 4 cubic

inches . The data acquisition and processing module 1020

some embodiments , a number of the optical channels is
between 4 and 50 .

One or more batteries 1588 can supply power to thesystem 1000 by way of the substrate (s ) 1586 . In addition , a( including the imaging optics, focal plane array, and on 65 visible light imaging sensor 1580 can be disposed in theboard processing electronics may ) can have a volumemore

5 cubic inches. The data acquisition and processing module

housing 1590 and can be configured to provide a visible lightimage of the scene being captured by the system 1000. The

US 10 ,084 , 975 B2
48
processed IR image data can be overlaid upon the visible

some embodiments the optical FPA unit 1508 comprises one

light image . In various embodiments the visible light imag
ing sensor 1580 can be used for reduction of scene -motion

or moremicrobolometer arrays , which may be uncooled . For
example , an array of about 1000x1000 microbolometer

induced detection errors, for example, to detect a moving arrays may be used in the embodiments disclosed herein .
object that enters the field of view (such as an animal or 5 Microbolometer arrays such as those manufactured by DRS
person ) and would interfere with the data being collected .
Technologies of Arlington , Va ., and Sofradir EC , Inc ., of

As explained herein , the movable shutter( s) 1503 can be

Fairfield , N . J., may be suitable for the embodiments disconfigured to provide spectral- radiometric calibration for

closed herein . For example , the DRS U8000 FPA manufac

the system 1000 . The shutter( s ) 1503 can be configured to

tured by DRS Technologies may be used in some embodi

move in and out of the field of view of the lens assembly 10 ments . In some arrangements , themicrobolometer array may

1502 periodically , e .g ., in a time period in a range of about
1 minute to about 15 minutes , or more particularly, in ahave a resolution of 1024x768 with a pixel pitch of 12
microns. The array of lenses can form separate channels

range of about 3 minutes to about 7 minutes , e . g ., about 5

having image detection regions that form part of the array.

minutes . Although one shutter 1503 is illustrated in FIG . For example , 12 channels can be included in the 1024x768
14C , it should be appreciated that two ormore shutters may 15 pixel array with on the detector array (microbolometerbe provided . The shutter ( s ) 1503 can be used in static
calibration procedures to provide the system with absolutetemperature values . In some embodiments , only static calibration is performed , e .g ., no dynamic calibration is per -array) that are for example 250x250 pixels for each of the 12channels. Detector arrays having more or less pixels may be
employed . Similarly the number of channels be larger orsmaller than 12 and the detection are on the detector array

The lens assembly 1502 can include a first lens array
1502A and a second lens array 1502B . In some embodi

from between 100 -200 pixelsx100 -200 pixels per detection
region, For example , the detection region may compriseby the first and second arrays 1502A , 1502B . Each of the

Likewise the detection region for a channel may measure

formed . In some embodiments, both static and dynamic 20 for a single channel may be larger or smaller than 250x250
calibration procedures are performed .
pixels . For example, the detection region may comprisements, the lens assembly 1502 can comprise an array of
from between 100 - 200 pixelsx100 - 200 pixels per detection
two -part lenses denoted by the first and second arrays 25 region , from between 200 - 300 pixelsx200 - 300 pixels per
1502A , 1502B . In some embodiments , the lens assembly
detection region , or from between 300 - 400 pixelsx300 -400
1502 can comprise an array of two separate lenses denoted
pixels or from between 400 - 500 pixelsx400 -500 pixels .
lens arrays 1502A , 1502B can comprise a 4x3 array of

100 - 200 pixels on a side, 200 - 300 pixels on a side, 300 -400

lenses , each of which may correspond to a particular detec - 30 pixels on a side, 400 - 500 pixels on side or larger or smaller.
tor region in the FPA unit 1508 and can define an optical

In some arrangements , the spectral band of the microbo

channel of the system 1000 . The lenses used in the first lenslometer can be about 7 .5 microns to 14 microns . The

array 1502 A may be different from the lenses used in the

microbolometer array can operate at a frame rate of about 30

second lens array 1502B . The lenses can be any suitable typeHz and can operate at operating temperatures of about - 40°of lens, including , e . g ., spherical lenses, aspheric lenses , rod 35 C . to + 70° C . In various embodiments, the microbolometer

lenses , etc . or any combination thereof. For example , thearray is an uncooled microbolometer that does not include a
lenses used in the first lens array 1502A can comprisecooler. The sensitivity of the microbolometer at F / 1 can beaspheric lenses, and the lenses used in the second lens array<about 40 mK . The systems 1000 disclosed herein can be
1502B can comprise rod lenses. Although the lens assemblyused to detect wavelengths in a range of about 1 micron to1502 shown in FIG . 15C includes two lens arrays , it should 40 about 20 microns . For example , the systems 1000 disclosed
be appreciated that additional lens arrays may be used , e . g ., herein can be used to detect wavelengths above about 6
three lens arrays , four lens arrays , five lens arrays , etc . In
addition , to assist in enabling a small system size , themicrons, e . g ., in a range of about 6 microns to about 18
microns, or more particularly , in a range of about 7 microns

diameter of each lens in the assembly 1502 can be less thanto about 14 microns. In various embodiments, the individual

about 0 . 5 " , e . g ., in a range of about 0 . 1 " to about 0 .5 " . The 45 detector elements of the microbolometer array can be spaced

f- number of each lens can be less than about 2 , e. g ., in a

relatively close together to at least partially enable a small ,range of about 0 . 2 to 2 , or more particularly , in a range ofcompact system . For example , adjacent detector elements ofcan be mounted on one side of the mounting plate 1584 , anda range of about 24 mm to about 30 mm , e . g ., about 27 . 5

can be machined to have diameter tolerances of about + / - 25

30 mm , etc . Although various embodiments of the system

the array can be spaced apart by a distance in a range ofabout 0 . 5 to 2 , or 1. 0 to 2 or 1 . 1 to 2 .
The first lens array 1502A and the second lens array
about 7 microns to about 15 microns, or more particularly in
1502B can be coupled to one another by way of a mounting 50 a range of about 9 microns to about 13 microns, e. g ., aboutplate 1584 sized and shaped to support or receive each lens
11 microns . The individual lenses can be spaced apart by aarray 1502A , 1502B . For example , the first lens array 1502A
distance in a range of about 20 mm to about 35 mm , e . g . inthe second lens array 1502B can be mounted on an opposite mm . Likewise the spatially and spectrally spaced channels
side of the mounting plate 1584 . The mounting plate 1584 55 may be physically spaced apart by 20 to 35 mm , 24 mm tomicrons . The lenses of the arrays 1502A , 1502B can be

are described as including an FPA comprising for example

secured to the mounting plate 1584 with a curable epoxy .

a mircobolometer array, certain embodiments comprise a

For example , the lenses may fit into opposite sides of holes

plurality of FPAs. In some embodiments, a single optical
60 FPA is used . In some embodiments , detectors of the opticalformed in the mounting plate 1584 .

The optical FPA unit 1508 can comprise any suitable typeof detector array that is configured to detect infrared radia -tion , for example , greater than 1 micron , or greater than 2
microns , or greater than 3 microns or greater than 5 microns,

FPA are configured to detect radiation in the same band of
IR wavelengths .

The on -board processing electronics of the data acquisition and processing module 1020 can process the IR opticalor greater than 6 microns and possibly lower than 20 65 data to detect and /or identify a target species from the IR

microns, or 15 microns , or 13 microns , or 12 microns or 10

microns , in wavelength , and may be cooled or uncooled . In

radiation received at the optical FPA . For example , themodule 1020 can be configured to acquire multispectralUS 10 ,084 ,975 B2
49

50

image data and analyze the acquired image data to identify
the target species . For example , the mobile imaging systems
1000 disclosed herein can be configured to image a 10 mx10
m object area at a distance of about 17 m at a resolution ofrecharging or replacing the batteries 1588 . In some arrange
ments the one or more batteries 1588 can comprise lithium
ion batteries, which have relatively high energy densities . In
addition , to help reduce power consumption within theabout 0 .04 m . In this example , any gas leaks that generate a 5 system 1000 , the FPGAs of the data acquisition and pro

gas cloud of at least about 1 .5 inches in size can be detectedcessing module 1020 can be advantageously programmed

and /or identified by the system 1000 . The detection andsuch that power consumption is lower than that used forsition and processing module 1020 of the imaging system

the batteries. In some embodiments , the system 1000 can

1000 . One way to provide a smaller system 1000 is tooperate for at least about 12 hours without recharging or

identification methods can be performed substantially inother types of processing electronics.
real-time such that the user can be alerted if any leaks are
The systems 1000 disclosed herein can advantageously
10 operate for between 8 hours and 36 hours without recharging
identified .
As explained above , the infrared image data captured byor replacing the batteries , or more particularly between
the system 1000 can be processed on board the data acquiabout 10 hours and 24 hours without recharging or replacing

process the image data using one or more field - program - 15 replacing the batteries . The components of the data acquimable gate arrays (FPGA ) configured to execute methods

sition and processing module 1020 (including the imaging

used in the analysis of the images captured by the optical

optics, focal plane array , and on board processing electronics

the substrate (s ) 1586 shown in FIG . 14C and can be physi
cally located proximate the optical system . For example, thedata acquisition and processing module 1020 ( including theimaging optics , focal plane array , and on board processing

system 1015 . In some embodiments , one or more Applica - may ) can be configured to operate at relatively low electricaltion Specific Integrated Circuits (ASICs ) may be used
power levels, e .g ., at power levels in a range of about 3 W
instead of, or in addition to the FPGAs. For example , an 20 to about 10 W , or more particularly in a range of about 4 W
ASICs chip may include an FPGA . The FPGA ( s ) ( and/or
to about 7 W , or in a range of about 4 W to about 6 W , e . g .,ASIC ( s )) can be mounted to and electrically coupled with
about 5 W in some embodiments . The components of the
FPGA can include logic gates and read access memory 25 electronics may ) can also be configured to operate at rela
(RAM ) blocks that are designed to quickly implement the
tively low overall energy levels for a single charge of thecomputations used to detect the types of gases in a gas cloud . batteries 1588 , e .g ., at energy levels in a range of about 60
The small size /weight, and high performance characteristics
Watt -hours (Wh ) to about 100 Wh, or more particularly in aof the FPGA can enable on board computation and analysis
range of about 80 Wh to about 95 Wh, or in a range of about
within the data acquisition and detection unit 1020 worn or 30 85 Wh to about 90 Wh.
carried by the user . The use of FPGA (or similar electronics )In addition , for each of the embodiments disclosed herein ,on board the system 1000 can reduce costs associated withvarious motion detection and /or compensation techniques
using an off - site central server or larger computing device to

can be implemented to account for relatively large - scaleconduct the image analysis computations. Advantageously ,motions that are induced by the user moving his or her headthe embodiments disclosed herein can enable on -board 35 during use . For example, when a user is visiting a well site

computation even though it can be challenging to implement
complex methods on the limited computing platform that
FPGAs provide.

or other installation , the user may be continuously walking
and looking in different directions ( e .g ., by rotating his or

prevent or reduce communication bottlenecks associatedwith wirelessly transmitting large amounts of raw data fromrate , which can affect the accuracy of the methods used todetermine the identity of species in a gas cloud or otherher head ). Additionally , vibration can be introduced by the
In addition , enabling computation with one or more user 's natural unsteadiness. Such movement can continu
FPGA devices on board the wearable system can also 40 ously change the system 's field of view at a relatively rapid
the system 1000 to a remote server or computer. For

object . Accordingly , it can be desirable to provide improved

example, the infrared optical system 1015 disclosed hereinmotion detection and/or compensation techniques to reduce

may generate up to about 380 Mbps of raw image data at 30 45 errors associated with the movements of the user.
frames per second , and the visible sensor 1580 may generate
Each of the embodiments disclosed herein can be used toabout 425 Mbps of raw image data at 30 frames per second

estimate various characteristics of gases present in a gas leakconventional wireless technologies. While data compression

References throughout this specification to " one embodi

The resulting data rate of about 800 Mbps is faster than mostimaged by the infrared imaging systems disclosed herein .

and/ or pre -processing may reduce the raw data rates for the 50 ment," " an embodiment," " a related embodiment,” or simi
visible and IR images , in some embodiments , the IR image
lar language mean that a particular feature , structure , ordata may only be compressed by a ratio of about 2 : 1 . The
characteristic described in connection with the referred toresulting overall data rate of about 192 Mbps may not be" embodiment” is included in at least one embodiment of thetransmitted effectively by conventional wireless communi-present invention . Thus , appearances of the phrases “ in one

cations devices. Accordingly , performing the image process - 55 embodiment," " in an embodiment," and similar language

ing calculations on board the system 1000 ( e .g ., on the dataacquisition and processing module 1020 ) can reduce thethroughout this specification may, but do not necessarily , allrefer to the same embodiment. It is to be understood that nooccurrence of or avoid bottlenecks generated by wirelessly

portion of disclosure , taken on its own and in possible

communicating the raw image data to an off -site centralconnection with a figure , is intended to provide a complete

60 description of all features of the invention .
server.
One challenge to implementing a mobile imaging system
In the drawings like numbers are used to represent theis the power requirements of each component of the system ,same or similar elements wherever possible . The depicted

including, e. g., the IR optical system 1015 , the visible sensor

structural elements are generally not to scale, and certain

1580 , the processing electronics , the wireless communica -components are enlarged relative to the other components

imaging systems 1000 disclosed herein can be configured tooperate by battery power for long periods of time without

understood that no single drawing is intended to support acomplete description of all features of the invention . In othertions modules , etc . Advantageously, the mobile infrared 65 for purposes of emphasis and understanding. It is to beUS 10 ,084,975 B2
51

words, a given drawing is generally descriptive of only

52

ware , the functions necessary to implement the invention

some, and generally not all, features of the invention . A
may optionally or alternatively be embodied in part or ingiven drawing and an associated portion of the disclosure whole using firmware and/ or hardware components , such as
containing a description referencing such drawing do not
combinatorial logic , Application Specific Integrated Circuits
generally, contain all elements of a particular view or all 5 (ASICs), Field -Programmable Gate Arrays (FPGAs ) orfeatures that can be presented is this view , for purposes ofother hardware or some combination of hardware, software
the discussion to particular elements that are featured in thisWhile examples of embodiments of the system anddrawing . A skilled artisan will recognize that the invention method
the invention have been discussed in reference to
may possibly be practiced without one or more of the 10 the gas- ofclouddetection , monitoring, and quantification (inspecific features, elements , components , structures , details ,cludingbut
notlimited to greenhouse gases such as Carbon
or characteristics , or with the use of other methods, com
simplifying the given drawing and discussion , and to direct

and /or firmware components .

Dioxide, Carbon Monoxide , Nitrogen Oxide as well as
particular detail of an embodiment of the invention may not hydrocarbon gases such as Methane, Ethane , Propane, n - Bu
be necessarily shown in each and every drawing describing 15 tane , iso - Butane, n -Pentane , iso -Pentane , neo -Pentane ,such embodiment, the presence of this detail in the drawing Hydrogen Sulfide, Sulfur Hexafluoride , Ammonia , Benzene ,ponents , materials , and so forth . Therefore , although a

may be implied unless the context of the description requires p - and m -Xylene, Vinyl chloride, Toluene , Propylene oxide,
otherwise. In other instances, well known structures, details , Propylene , Methanol, Hydrazine, Ethanol, 1,2 -dichloroethmaterials, or operations may be not shown in a givenane, 1 , 1 - dichloroethane , Dichlorobenzene , Chlorobenzene,
drawing or described in detail to avoid obscuring aspects of 20 to name just a few ), embodiments of the invention can be
an embodiment of the invention that are being discussed .

Furthermore , the described single features , structures, orcharacteristics of the invention may be combined in any

readily adapted for other chemical detection applications .

For example , detection of liquid and solid chemical spills ,biological weapons, tracking targets based on their chemicalcomposition , identification of satellites and space debris ,suitable manner in one or more further embodiments .
Moreover, if the schematic flow chart diagram is included , 25 ophthalmological imaging, microscopy and cellular imag
it is generally set forth as a logical flow -chart diagram . As ing, endoscopy , mold detection , fire and flame detection , andsuch , the depicted order and labeled steps of the logical flow
are indicative of one embodiment of the presented method .

pesticide detection are within the scope of the invention .
As used herein , a phrase referring to at least one of a list

Other steps and methods may be conceived that are equiva of items refers to any combination of those items, including
lent in function , logic , or effect to one or more steps, or 30 single members . As an example , " at least one of: a , b , or c "portions thereof, of the illustrated method . Additionally , theis intended to cover: a , b , c , a -b , a -c , b - c , and a - b - c .

format and symbols employed are provided to explain theIf implemented in software , the functions may be stored

logical steps of the method and are understood not to limit

on or transmitted over as one or more instructions or code on

the scope of the method. Although various arrow types anda computer-readable medium . The steps of a method or

are understood not to limit the scope of the corresponding
method . Indeed , some arrows or other connectors may be

cessor- executable software module which may reside on acomputer -readable medium . Computer -readable media

line types may be employed in the flow - chart diagrams, they 35 algorithm disclosed herein may be implemented in a pro
used to indicate only the logical flow of the method . For

includes both computer storage media and communication

instance, an arrow may indicate a waiting or monitoring

media including any medium that can be enabled to transfer

whole .
At least some elements of a device of the invention can becontrolled and at least some steps of a method of thethe form of instructions or data structures and that may beaccessed by a computer . Also , any connection can be prop
erly termed a computer - readable medium . Disk and disc , asperiod of unspecified duration between enumerated steps of 40 a computer program from one place to another. A storage
the depicted method . Without loss of generality , the order in media may be any available media that may be accessed bywhich processing steps or particular methods occur may or
a computer . By way of example , and not limitation , suchmay not strictly adhere to the order of the corresponding
computer -readable media may include RAM , ROM ,EEPROM , CD -ROM or other optical disk storage , magnetic
steps shown .
The features recited in claims appended to this disclosure 45 disk storage or other magnetic storage devices, or any otherare intended to be assessed in light of the disclosure as a medium that may be used to store desired program code ininvention can be effectuated , in operation with a program - 50 used herein , includes compact disc (CD ), laser disc , opticalmable processor governed by instructions stored in adisc , digital versatile disc (DVD ), floppy disk , and blu -ray
memory. The memory may be random access memory
disc where disks usually reproduce data magnetically , while(RAM ), read -only memory (ROM ), flash memory or any

discs reproduce data optically with lasers . Combinations ofother memory , or combination thereof, suitable for storing

the above also may be included within the scope of com

control software or other instructions and data . Those skilled 55 puter- readable media . Additionally , the operations of ain the art should also readily appreciate that instructions or method or algorithm may reside as one or any combination

programs defining the functions of the present invention

or set of codes and instructions on a machine readable

may be delivered to a processor in many forms, including,medium and computer - readable medium , which may be

but not limited to , information permanently stored on non -incorporated into a computer program product.

writable storage media ( e . g . read -only memory devices 60

Various modifications to the implementations described inwithin a computer, such as ROM , or devices readable by acomputer I/O attachment, such as CD -ROM or DVD disks ),information alterably stored on writable storage media ( e.g .

this disclosure may be readily apparent to those skilled in theart, and the generic principles defined herein may be applied
to other implementations without departing from the spirit or

floppy disks , removable flash memory and hard drives ) or scope of this disclosure . Thus, the claims are not intended to
information conveyed to a computer through communica - 65 be limited to the implementations shown herein , but are totion media , including wired or wireless computer networks. be accorded the widest scope consistent with this disclosure ,In addition , while the invention may be embodied in soft- the principles and the novel features disclosed herein .

US 10 ,084 ,975 B2
53

54

Certain features that are described in this specification in
the context of separate implementations also can be imple
mented in combination in a single implementation . Conversely, various features that are described in the context ofwherein receiving the image data comprises receiving multi
spectral image data from the plurality of spectrally andspatially distinct optical channels of each IR imaging systemof the plurality of IR imaging systems.

a single implementation also can be implemented in multiple 5
11 . The method of claim 10 , wherein processing theimplementations separately or in any suitable subcombina
received image data comprises processing the multi - spectraltion .Moreover, although features may be described above as image data to identify the location at which the one or moreacting in certain combinations and even initially claimed as target gases is detected .
1such , one or more features from a claimed combinationcan
HUI Vall
12 . A non -transitory computer-readable medium having a

in some cases be excised from the combination , and the 10 program stored thereon , the program comprising instrucclaimed combination may be directed to a subcombination
or variation of a subcombination .

What is claimed is:tions for implementing the method according to claim 1 ,when these instructions are executed by a process .

13 . A server for monitoring the presence of one or more
1. A method for monitoring the presence of one or more 15 target
gases at one or more installation sites, the systemtarget gases, the method comprising :receiving image data from a plurality of IR imaging

comprising :systems, each IR imaging system configured to capture

a communications module configured to receive image
data from a plurality of IR imaging systems, each IRwith a location atwhich the one or more target gases are 20
present; andprocessing the received image data to identify the location
at which the one or more target gases is detected .
2 . The method of claim 1 , further comprising mapping,of the one or more target gases in real-time and toassociate each captured infrared image with a location
at which the one or more target gases are present; andprocessing circuitry including a processor configured toprocess the received image data to identify the location
at which one or more target gases is detected .

infrared images of the one or more target gases inreal-time and to associate each captured infrared image

imaging system configured to capture infrared images

within the one or more installation sites , the location at 25
which the one or more target gases is detected .
14 . The server of claim 13 , wherein the processor is
3 . The method of claim 1 , further comprising mapping,further configured to map the location , within the one orwithin the one or more installation sites, the location at more installation sites, at which the one or more target gases
which the or more target gases is detected , a type of target is detected .
gas detected and a concentration of the type of target gas 30 15 . The server of claim 13 . wherein the processor is
detected .
further configured to identify a type of target gas detected

4 . The method of claim 1 , further comprising detecting theon the received image data .
presence of the one or more target gases based on the image based
16.
of claim 13, wherein the communications
data received from the plurality of IR imaging systems. 35 moduleTheis server
further configured to receive data identifying a

5 . The method of claim 1 , further comprising receiving 35 m

pre - processed image data from the plurality of IR imaging
systems, the pre -processed image data comprising dataassociated with the presence of the one or more target gases.

type of target gas detected by one or more of the IR imaging

systems.
17 . The server of claim 13 , wherein the processor is

6 . The method of claim 1, wherein the plurality of IR
further configured to identify a concentration of target gas
imaging systems are worn or carried by a plurality of 40 based on the received image data .
persons.
18 . The server of claim 13 , wherein the communications
7 . The method of claim 1, further comprising receiving IR

module is further configured to receive data identifying a

image data from a plurality of truck -based IR imaging

concentration of target gas detected by one ormore of the IR

systems, each truck -based IR imaging system configured toimaging systems.
be mounted to a truck and configured to capture infrared 45 19 . The server of claim 13 , wherein each IR imaging
images of the one or more target gases in real-time.
system of the plurality of IR imaging systems comprises a
8 . Themethod of claim 1 , further comprising receiving IR
plurality of spectrally and spatially distinct optical channels ,image data from a plurality of aerial-based IR imaging and wherein the communications module is configured tosystems, each aerial-based IR imaging system configured toreceive multi-spectral image data from the plurality of
be mounted to an aerial platform and configured to capture 50 spectrally and spatially distinct optical channels of each IR
infrared images of the one or more target gases in real- time.

imaging system of the plurality of IR imaging systems.
20 . The server of claim 19 , wherein the processor isimage data from a plurality of imaging systems mounted toconfigured to process the multi - spectral image data to iden
stationary structures at the one or more installation sites .
tify the location at which one or more target gases is
10 . The method of claim 1, wherein each IR imaging 55 detected.
system of the plurality of IR imaging systems comprises a* * * * *plurality of spectrally and spatially distinct optical channels,

9 . The method of claim 1 , further comprises receiving IR