System and Method for Drug Detection Using SWIR

A method for detecting unknown materials, such as drugs. A first location is surveyed using a video capture device to identify a second location comprising an unknown material. The second location is interrogated using SWIR spectroscopic and/or imaging methods to generate a SWIR hyperspectral image. The SWIR hyperspectral image is analyzed to associate the unknown material with a known drug material. A system for detecting unknown materials, such as drugs comprising a first collection lens for collecting interacted photons from a first location and a visible imaging device for generating a visible image. A second collection lens may collect a plurality of interacted photons from a second location and a tunable filter may filter the interacted photons. A spectroscopic imaging device may detect the interacted photons and generate a SWIR hyperspectral image. A processor may analyze the SWIR hypespectral image to associate an unknown material with a known material.

DETAILED DESCRIPTION

The present disclosure provides for a system and method for detecting drug materials using SWIR hyperspectral imaging. The systems and methods of the present disclosure may incorporate or comprise SWIR CONDOR™ and CONDOR-ST technology available from Chemlmage Corporation, Pittsburgh, Pa. and any developments and improvements thereto relating to standoff SWIR technology.

FIG. 1is representative of a method of the present disclosure. The method100may comprise surveying a first location (which may be referred to herein as a region of interest) to thereby identify a second location (which may be referred to herein as a target area) wherein the second location comprises at least one unknown material in step110. In one embodiment, the first location may be surveyed using a visible imaging device. This visible image device may output a dynamic image of a region of interest in real time and may comprise a video capture device. In another embodiment, the visible imaging device may comprise a RGB camera.

In one embodiment, the second location may be identified based on morphological features. These features may include but are not limited to: size, shape, and color of the second location or of at least one object in the second location.

The present disclosure also contemplates the first location may be surveyed using a SWIR spectroscopic imaging device. In such an embodiment, SWIR hyperspectral imaging may be used to both survey the first location (region of interest) and to locate a second location (a target area) within that first location. The SWIR spectroscopic imaging device may also be used to interrogate the second location to detect and/or identify the unknown material as a drug material.

In step120the second location is illuminated to thereby generate a plurality of interacted photons. In one embodiment, the plurality of interacted photons may comprise at least one of: photons reflected by the second location, photons absorbed by the second location, photons scattered by the second location, and photons emitted by the second location. In one embodiment, the interacted photons may be generated by using at least one of: active illumination and passive illumination.

In step130the plurality of interacted photons are passed through a tunable filter to filter the interacted photons into a plurality of wavelength bands. The plurality of interacted photons may be detected using a spectroscopic imaging device to thereby generate a SWIR hyperspectral image in step140. In one embodiment, the SWIR hyperspectral image may comprise a digital image and a spatially resolved SWIR spectrum for each pixel in said image. In one embodiment, the SWIR hyperspectral image may comprise a dynamic chemical image.

The method may further comprise analyzing the SWIR hyperspectral image to thereby associate the unknown material with at least one known drug material in step150. The unknown material may comprise at least one drug material. When used herein, “drug” or “drug material” may refer to at least one of: an illicit drug material and a non-illicit drug material. Other embodiments may be envisioned that detect other materials of interest including chemicals, biological materials, hazardous materials, and explosives.

In one embodiment, analyzing a SWIR hyperspectral image may comprise comparing at least one of a SWIR hyperspectral image and/or one or more SWIR spectra associated with said SWIR hyperspectral image with a reference data base wherein the reference data base comprises at least one reference SWIR data set associated with a known material, such as a known drug material. In one embodiment, the reference data base may also comprise at least one reference visible data set associated with a known material or object. This reference data base may be consulted during surveying of a first location.

Comparing the SWIR hyperspectral image (or a visible image) to a reference data set may be accomplished using one or more algorithmic techniques. These techniques may comprise at least one chemometric and/or ratiometric techniques (such as wavelength division). Chemometric techniques may include, but are not limited to: principle components analysis (PCA), PLSDA, cosine correlation analysis, Euclidian distance analysis, k-means clustering, multivariate curve resolution, band t. entropy method, MD, adaptive subspace detector, spectral mixture resolution, Bayesian fusion, and combinations thereof.

In one embodiment, the method may further provide for data fusion in which data generated by two or more different spectroscopic imaging modalities may be fused. This fusion may be accomplished by applying at least one fusion algorithm known in the art. The present disclosure contemplates a variety of different fusion combinations including at least two of the following: a visible image, a SWIR hyperspectral image, a MWIR hyperspectral image and a LWIR hyperspectral image may be generated.

The present disclosure also provides for a system for detecting and/or identifying drugs and/or other materials.FIG. 2is a schematic representation of a system of the present disclosure. The system200may comprise an illumination light source201configured to illuminate an unknown sample202to thereby generate a plurality of interacted photons. In one embodiment, the illumination light source201may comprise at least one of: a laser illumination source, a broadband light source, and an ambient light source. In one embodiment, the system200may be configured for passive illumination and/or active illumination.

In one embodiment, at least one illumination source will incorporate IR long pass filters to eliminate any visible light emitted from the source(s) and allow for only IR light to illuminate the scene. The IR light is eye safe and invisible to visible sensors. For daytime operation, one embodiment provides for the use of the sun as an illumination source. In an embodiment for nighttime operation using active illumination, a set of tungsten white light illumination sources may be used. Tungsten white light alone is eye safe but is not invisible to visible sensors. By coupling the tungsten white light sources with IR long pass filters all visible light will be blocked and only IR light will illuminate the scene. In one embodiment, four (4) spotlights with 5900 lumens each, with 6° angular divergence may produce an average intensity of about 1100 and about 5 m illumination diameter at a 50 m standoff distance. Additional lighting may be used to carry out measurements at standoff distances of 200-1000 m.

Interacted photons generated by illuminating the second location may be collected by one or more optics203. In one embodiment, telescope optics may be configured for at least one of: locating and focusing on a second location and/or collecting a plurality of interacted photons. In one embodiment, a telescope optics may be implemented to enable magnification and thereby SWIR hyperspectral imaging sensitivity.

The interacted photons may be passed through a tunable filter204. The tunable filter inFIG. 2is illustrated as a multi-conjugate liquid crystal tunable filter (MCF)204. In one embodiment, MCF technology available from ChemImage Corporation, Pittsburgh, Pa. may be used. A MCF, a type of LCTF, consists of a series of stages composed of polarizers, retarders and liquid crystals. The MCF is capable of providing diffraction limited spatial resolution, and a spectral resolution consistent with a single stage dispersive monochromator. A MCF may be computer controlled with no moving parts. It may be tuned to any wavelength in the given filter range. This results in an essentially infinite number of spectral bands available. A MCF provides high optical throughput, superior out-of-band rejection and faster tuning speeds. While images associated with spectral bands of interest must be collected individually, material-specific chemical images revealing target detections may be acquired, processed and displayed in numerous times each second. Combining MCF technology with software targeting algorithms holds great potential for detecting drug materials using SWIR hyperspectral imaging, including potential for OTM detection.

This technology is more fully described in the following U.S. patents and patent applications: U.S. Pat. No. 6,992,809, filed on Jan. 31, 2006, entitled “Multi-Conjugate Liquid Crystal Tunable Filter,” U.S. Pat. No. 7,362,489, filed on Apr. 22, 2008, entitled “Multi-Conjugate Liquid Crystal Tunable Filter,” Ser. No. 13/066,428, filed on Apr. 14, 2011, entitled “Short wave infrared multi-conjugate liquid crystal tunable filter.” These patents and patent applications are hereby incorporated by reference in their entireties.

The MCF may be used to filter light to the spectroscopic imaging device205and is capable of tuning to an infinite number of spectral bands. Therefore, for nighttime operation using active broadband IR illumination, decreasing spectral resolution may not be necessary. Nighttime operation of the system may cover the same spectral range and is capable of the same number of spectral bands as daytime operation. Transition from daytime to nighttime operations should be as simple as switching on a lamp.

The present disclosure is not limited to the use of a MCF and contemplates that the tunable filter204may comprise at least one of: a SWIR multi-conjugate liquid crystal tunable filter, a SWIR liquid crystal tunable filter, a Fabry Perot angle tuned filter, an acousto-optic tunable filter, a liquid crystal tunable filter, a Lyot filter, an Evans split element liquid crystal tunable filter, a Solc liquid crystal tunable filter, a fixed wavelength Fabry Perot tunable filter, an air-tuned Fabry Perot tunable filter, a mechanically-tuned Fabry Perot tunable filter, and a liquid crystal Fabry Perot tunable filter.

The plurality of interacted photons may detected using a spectroscopic imaging device205. The spectroscopic imaging device may be configured to generate a SWIR hyperspectral image representative of the second location interrogated (which comprises the unknown material). In another embodiment, the spectroscopic imaging device205may be configured so as to generate at least one of: a plurality of spatially resolved SWIR images, a plurality of spatially resolved SWIR spectra, a SWIR chemical image, and combinations thereof.

The system200may further comprise a reference database206comprising at least one SWIR reference data set. A processor may be configured to access this SWIR database206to analyze a SWIR hyperspectral image.

FIG. 3is a more detailed schematic of a system of the present disclosure. The system300may comprise one or more windows301,302, and303, which may also be referred to as collection lenses, or lenses, herein. The system may comprise a one or more zoom optics304,305. In one embodiment, a SWIR zoom optic304may be operatively coupled to a tunable filter307. InFIG. 3, the tunable filter is illustrated as a SWIR LCTF307. However, the tunable filter307may comprise any filter contemplated herein. The SWIR LCTF307may be configured to effectively separate a plurality of interacted photons into a plurality of wavelength bands. The plurality of interacted photons may be detected by a SWIR detector, illustrated inFIG. 3as a SWIR InGaAs Camera309. However, other embodiments may comprise other detectors known in the art including but not limited to a mercury cadmium telluride (MCT) detector, a CCD detector, an intensified charged coupled device (ICCD), a indium antimonide (InSb) detector, and an InGaAs detector. In one embodiment a SWIR detector309may be operatively coupled to a frame grabber310which may operate to capture image frames generated by the detector309.

The system300may further comprise a visible zoom optic, illustrated inFIG. 3as a

RGB zoom optic305. This RGB zoom optic305may be operatively coupled to visible detector, illustrated as an RGB camera308. However, this visible detector may also comprise a video capture device.

The system300may further comprise a number of controls and additional features to enable navigation, selection of a location, and overall operation and management of the system300. The system300may comprise a range finder306which may be configured to measure distance to a specific location or object. In one embodiment, at least one of a frame grabber310, a RGB camera308, a range finder306, and an inertial navigation system312may be operatively coupled to an acquisition computer311. This acquisition computer311may be coupled to at least one of: a local control315, a processing computer317, and a PTU319. In one embodiment, a local control315may comprise a computer and further comprise at least one of: a keyboard316a,a mouse316b, and a monitor316c. In one embodiment, a processing computer317may comprise at least one of: an Ethernet configuration317a,and a second processing computer317b. The processing computer317may be operatively coupled to a user control interface318. The user control interface318may comprise at least one of: a mouse318a, keyboard318b, and monitor318c. The system may further comprise a power management system320which may be operatively coupled to the system300.

In one embodiment, the system of the present disclosure may incorporate a high pixel resolution, high frame rate color video camera system to assist in locating targets of interest. This may be represented inFIG. 3as a RGB camera308. The SWIR HSI portion of the system may consist of an InGaAs focal plane camera coupled to a wavelength-agile MCF in combination with a zoom optic capable of viewing a large area, or imaging a localized area at high magnification. In one embodiment of operation, an area would first be screened using the wide field setting on the zoom lens. Once the area is screened and potential targets are identified, confirmation of the area may be accomplished as necessary by using the narrow field setting on the zoom lens.

FIGS. 4A-4Dare illustrative of exemplary embodiments of packaging of the systems of the present disclosure. In one embodiment, a20x magnification increase may be used to increase SWIR HSI detection sensitivity. In one example, sensitivity may be increased by integration of an8″ diameter telescope.

FIGS. 5A-5Eillustrate the detection capabilities of the present disclosure for unknown materials. WhileFIGS. 5A-5Eillustrate the detection of ammonium nitrate (AN), they are provided to show how the system and method of the present disclosure may be applied to detecting and identification of any unknown material. Therefore, a similar analysis may be used to detect drug materials.FIG. 5Adepicts an exemplary optical image,FIG. 5Bdepicts a NIR chemical image, a Raman image is illustrated inFIG. 5C, and Bicubic Expansion is illustrated inFIG. 5D. While the present disclosure focuses on the use of SWIR hyperspectral image and spectroscopy, these figures illustrate the potential of also applying other techniques. Absorption spectra and Raman spectra are depicted inFIGS. 5E and 5F, respectively.FIGS. 6A-6Efurther illustrate the potential of a system and method of the present disclosure for detecting unknown materials. These Figures illustrate AN detection, but a similar analysis may be applied to detecting drug materials.FIG. 6Adepicts an exemplary digital photograph,FIG. 6Billustrates a NIR Image and a NIR image is also presented inFIG. 6C. Absorption spectra are depicted inFIGS. 6D and 6E, respectively.

FIGS. 7-12provide further support of the detection capabilities of the present disclosure. Included in these Figures is evidence of the sensitivity enhancement capabilities of the system and method of the present disclosure. This data demonstrates AN, but the analysis can also be applied to embodiments detecting drug materials on various surfaces.FIG. 7is illustrative of the detection of AN on fingerprints on a slate surface obtained using a Gen3 sensor at 50 m standoff distance.FIG. 8is illustrative of a comparison between Gen2 and Gen3 sensors. The comparison is illustrative of the detection of AN on fingerprints on a slate surface at 50 m standoff distance.

FIG. 9is also illustrative of CONDOR-ST sensitivity enhancements. By increasing magnification of the image gathering optics, sensitivity of the CONDOR-ST SWIR HSI system can be increased. The sample inFIG. 9comprises AN on substrates (aluminum sheet metal, slate tile, dust/dirt covered slate tile, shoe) by fingerprint transfer. The sensor used to obtain the results was a CONDOR-ST (Gen3) sensor with an 8″ diameter telescope.FIG. 10Ais illustrative of the detection of AN fingerprints at 50 m standoff range on a shoe. This illustrates the potential of SWIR hyperspectral imaging for detection of unknown materials on a variety of surfaces. Such application is more fully described in U.S. patent application Ser. No. 12/754,229, filed on Apr. 5, 2010, entitled “Chemical Imaging Explosives (CHIMED) Optical Sensor using SWIR”, which is hereby incorporated by reference in its entirety.

FIG. 10Bis illustrative of detection of AN fingerprint residue transferred by touching a car trunk surface. This data was obtained at 20 m standoff range in real-time. A similar scenario may be applied to the detection of drug materials. For example, when a vehicle is suspected of carrying drug materials, various locations of interest on the vehicle may be selected and interrogated using SWIR hyperspectral imaging. These locations of interest may include a door handle or the trunk/storage area of the vehicle.

FIG. 11is illustrative of the ability of the system and method of the present disclosure to detect between aged and fresh concrete.FIG. 12is illustrative of the ability of the system and method of the present disclosure to detect disturbed earth at a 200 m standoff range. These figures are included to further support the detection capabilities of SWIR hyperspectral imaging for unknown materials on a variety of different surfaces and locations where drug material may be found.

In one embodiment, the systems and methods of the present disclosure may be configured to operate in at least one of the following configurations: proximal detection, standoff detection, stationary detection, and on-the-move detection. Standoff detection of explosives is more fully described in the following U.S. patents and patent applications, which are hereby incorporated by reference in their entireties: U.S. Pat. No. 7,692,775, filed on Jun. 9, 2006, entitled “Time and Space Resolved Standoff Hyperspectral IED Explosives LIDAR Detection”, Ser. No. 12/199,145, filed on Aug. 27, 2008, entitled “Time and Space Resolved Standoff Hyperspectral IED Explosives LIDAR Detection”, Ser. No. 12/802,994, filed on Jun. 17, 2010, entitled “SWIR Targeted Agile Raman (STAR) System for Detection of Emplace Explosives.”

In one embodiment, the system of the present disclosure may be used for stationary and OTM drug detection, explosive detection, disturbed earth detection and camouflage concealment and detection. In one embodiment, OTM detection may be enabled by using dynamic imaging in one or more modalities including visible and SWIR.FIGS. 13A and 13Bare provided to further explain OTM detection according to one embodiment of the present disclosure. The present disclosure also provides for a system and method of dynamic chemical imaging in which more than one object of interest passes continuously through the FOV. Such continuous stream of objects, results in the average amount of time required to collect all frames for a given object being equivalent to the amount of time to capture one frame as the total number of frames under collection approaches infinity (frame collection rate reaches steady state). In other words, the system is continually collecting the frames of data for multiple objects simultaneously and with every new frame, the set of frames for any single object is completed. In one embodiment, the objects of interest are of a size substantially smaller than the FOV to allow more than one object to be in the FOV at any given time. Referring toFIG. 13A, OTM detection may be enabled by collecting each frame at a different wavelength. One or more objects may be present in slightly translated positions in each image frame acquired. Tracking of objects across all n frames allows the spectrum to be generated for each pixel in the object. The same process may be followed for all objects in the frames. A continual stream of objects will be imaged with defined wavelengths at defined time intervals. This methodology may also utilize the benefits of signal averaging.FIG. 13Bis provided to illustrate approximate integration times associated with the configuration ofFIG. 13A.

FIG. 14is illustrative of OTM detection simulated by panning of PTU across a road.FIG. 15is illustrative of OTM of AN residue deposited on the ground at a standoff range of >50 m. The data was collected while moving at 3-5 mph. A similar approach may be applied to detecting drug materials.

Another example wherein different materials detected in a scene can be assigned different pseudo colors for easy discrimination between materials is illustrated byFIG. 16. Here disturbed earth, command wire, and foam are all detected and assigned different pseudo colors. Drug materials may also be detected and discriminated from other materials in a scene. Pixels containing the materials of interest may be pseudo colored to indicate positive detection. The use of pseudo color enhancement is more fully described in U.S. patent Ser. No. 12/799,779, filed on Apr. 30, 2010, entitled “System and Method for Component Discrimination Enhancement based on Multispectral Addition Imaging,” hereby incorporated by reference in its entirety.

The present disclosure contemplates the system and method disclosed herein may be configured so as to enable integration with LWIR, MM Wave, and/or GPR sensors via industry standard fusion software. In one embodiment, this fusion software may comprise Chemlmage's FIST (“Forensic Integrated Search”) technology, available from Chemlmage Corporation, Pittsburgh, Pa. This technology is more fully described in pending U.S. patent application Ser. Nos. 11/450,138, filed on Jun. 9, 2006, entitled “Forensic Integrated Search Technology”; Ser. No. 12/017,445, filed on Jan. 22, 2008, entitled “Forensic Integrated Search Technology with Instrument Weight Factor Determination”; Ser. No. 12/196,921, filed on Aug. 22, 2008, entitled “Adaptive Method for Outlier Detection and Spectral Library. Augmentation”; and Ser. No. 12/339,805, filed on Dec. 19, 2008, entitled “Detection of Pathogenic Microorganisms Using Fused Sensor Data”. Each of these applications is hereby incorporated by reference in their entireties.

The present disclosure also contemplates the incorporation of real-time anomaly detection and classification algorithms in a software package associated with the sensor. In such an embodiment, the system will have the ability to perform autonomous detection of a wide variety of targets. Such an embodiment provides for a single sensor system to support automated counter mine algorithms, aided target cuing, Aided Target Recognition (AiTR) of difficult targets, and anomaly detection and identification in complex/urban areas.

In another embodiment, the present disclosure provides for ChemFusion Improvements. Such improvements include the use of grid search methodology to establish improved weighting parameters for individual sensor modality classifiers under JFIST Bayesian architecture. Improvements in Pd and Pfa can be realized by full execution of combinatorial decision making applied to multiple detections afforded by hyperspectral imaging. In another embodiment, image weighted Bayesian fusion may be used.

In one embodiment, the system and method of the present disclosure may relate specifically to the use of SWIR technology for drug detection. Examples of the detection capabilities of the present disclosure are provided inFIGS. 17-31and illustrate the detection capabilities of SWIR technology. This data was generated using SWIR CONDOR™ technology, available from Chemlmage Corporation, Pittsburgh, Pa. and illustrates the ability of SWIR to detect various drug materials.

Various samples were deposited for analysis as shown inFIG. 17. Table 1 below illustrates the various drug samples and their corresponding locations inFIG. 17.

FIGS. 18-29illustrate video images, SWIR images, and spectra associated with each material deposited inFIG. 17. The data was analyzed using a method of the present disclosure by applying PLSDA and a scatter plot showing the results of this analysis is illustrated inFIG. 30. As can be seen fromFIG. 30, the drug materials can be discriminated from one another using this approach.FIG. 31is illustrative of the application of another embodiment of a method of the present disclosure applying a MD algorithm to the data. This metric displays a similarity of an unknown sample to a known sample (such as a known drug material). The dendogram illustrates the ability to differentiate between the drug materials. Unknown materials may be associated with known drug materials based on this similarity. These results illustrate the potential for SWIR hyperspectral imaging and/or spectroscopy for detecting and/or identifying drug materials.

The present disclosure may be embodied in other specific forms without departing from the spirit or essential attributes of the disclosure. Accordingly, reference should be made to the appended claims, rather than the foregoing specification, as indicating the scope of the disclosure. Although the foregoing description is directed to the embodiments of the disclosure, it is noted that other variations and modification will be apparent to those skilled in the art, and may be made without departing from the spirit or scope of the disclosure.