Patent Description:
The present invention relates to the field of hyperspectral scanners.

Hyperspectral imaging, like other spectral imaging, collects and processes information from across the electromagnetic spectrum. The goal of hyperspectral imaging is to obtain the spectrum for each pixel in the image of a scene, with the purpose of finding objects, identifying materials, or detecting processes.

Hyperspectral imaging combines spatial and detailed spectral information of a scene to construct images where the full spectrum of light at each pixel is known. Commercial hyperspectral imaging technology is used, for example, in the food industry, agriculture, astronomy, molecular biology, biomedical imaging, geosciences, physics, and surveillance.

To date, hyperspectral scanners are composed of scanning mirrors, however, scanning mirrors suffer from wear especially in conditions of measuring above a field while subjected to wind and/or dust.

For example, US Publication No. <CIT>, and entitled "Scanner System and Method for Registering Surfaces", discloses a method for registering surfaces, using a scanner system comprising a radiation source for emitting electromagnetic radiation (ES), a scanning device for guiding the radiation over the surface in order to scan the latter and a receiver for receiving the radiation (RS) that is reflected by the surface.

As another example, a technical paper by <NPL>. The paper presents a highly customizable cued beamsteering system consisting of a visible or infrared cueing imager co-aligned with an optical beam steering system's pointing-field-of-regard.

As yet another example, Patent Publication <CIT>, teaches a spatially and spectrally resolving hyperspectral camera with a beam splitter which splits the incoming light onto a high-resolution imaging module which is designed to capture an object on an areally resolved image plane and can deliver high-resolution image information, and furthermore on a spectrally resolving module which is designed to image the object to be recorded on a spectrally resolving image plane and can supply spectral data.

The present invention provides a system for identifying the condition of an object by analyzing the spectrum of a light reflected from the object, which may include emitted light generated by internal light conversion such as fluorogenic process, and generating a hyperspectral cube of the object. The scanning system comprises the features defined in independent claim <NUM>.

Optionally, the system additionally comprises a sensor for measuring the distance from the object.

Optionally, the sensor is a light detection and ranging (LIDAR) sensor.

Optionally, the light detection and ranging (LIDAR) sensor provides a range map.

Optionally, the camera is an RGB camera.

Optionally, the spectrometer is a point spectrometer.

Optionally, the first prism and the second prism are each connected to a rotating mechanism to allow the individual rotation of each wedge prism around its axis.

Optionally, the rotating mechanism is operated manually.

Optionally, the rotating mechanism is operated by a computerized controller.

Optionally, the system additionally comprises a filter configured for limiting illumination.

Optionally, the system additionally comprises an integral radiation meter configured for monitoring radiation from external sources.

Optionally, the system is mounted on one of a tripod, a post, and a tower.

Optionally, the system is mounted on an unmanned aerial vehicle (UAV).

Optionally, the system is stabilized by a gimbal set to prevent hovering errors.

Optionally, the camera and the spectrometer are aligned so as to allows accurate identification of the exact location of the spectrometer's measurements in the generated image.

This document references terms that are used consistently or interchangeably herein. These terms, including variations thereof, are as follows.

A "computer" includes machines, computers and computing or computer systems (for example, physically separate locations or devices), servers, computer and computerized devices, processors, processing systems, computing cores (for example, shared devices), and similar systems, workstations, modules and combinations of the aforementioned. The aforementioned "computer" may be in various types, such as a personal computer (e.g., laptop, desktop, tablet computer), or any type of computing device, including mobile devices that can be readily transported from one location to another location (e.g., smartphone, personal digital assistant (PDA), mobile telephone or cellular telephone).

"Linked" as used herein, includes both wired and/or wireless links, such that the computers, servers, components, devices and the like, are in electronic and/or data communications with each other, directly or indirectly.

Unless otherwise defined herein, all technical and/or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein may be used in the practice or testing of embodiments of the invention, exemplary methods and/or materials are described below.

Some embodiments of the present invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention.

Attention is now directed to the drawings, where like reference numerals or characters indicate corresponding or like components. In the drawings:.

The present invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description. The invention is capable of other embodiments, or of being practiced or carried out in various ways.

The present invention provides a hyperspectral scanning system which replaces the use of scanning mirrors with a mechanism that is based on two or more wedge prisms. The present invention is used to identify the condition of an object by analyzing the spectrum of a light reflected from the object, which may include emitted light generated by internal light conversion such as fluorogenic process, and generating a hyperspectral cube of the object.

<FIG> is a side perspective view of the system <NUM>. The system <NUM> includes a chamber <NUM> enclosing, for example, two prisms 104a-104b, e.g. wedge prisms, a beam splitter <NUM>, a camera <NUM>, a condensing lens <NUM> and a spectrometer <NUM>, for example, a point spectrometer. The chamber <NUM> further includes an aperture <NUM> through which a reflected light field is entered into the system <NUM>. The system <NUM> is linked to a computer (not shown).

A reflected beam of light from an object <NUM>, for example, a plant in a field is entered into the chamber <NUM> through the aperture <NUM>. The reflected beam reaches the wedge prisms 104a-104b which in turn refract the beam to direct it into the system <NUM>. Each wedge prism 104a-104b is separately connected, for example, to a rotating mechanism allowing the individual rotation of each wedge prism 104a-104b around its axis. The rotating mechanism can be operated manually or controlled by a computerized controller (not shown), over wired and/or wireless networks, or combinations thereof. The individual rotation of each wedge prism provides the system <NUM> with two degrees of freedom. The angles combination of the wedge prisms 104a-104b allows the steering of the system's line-of-sight (LOS) to attain a desired line-of-sight (LOS).

Following the prisms 104a-104b, the refracted beam reaches the beam splitter <NUM>. The beam splitter <NUM> splits the light intensity of the refracted beam into, for example, two separate beams, a first beam directed to the camera <NUM> and a second beam directed to the spectrometer <NUM>. The second beam directed to the spectrometer <NUM> passes through the condensing lens <NUM> which condense the beam before it reaches the spectrometer <NUM>. The spectrometer <NUM> analyzes the condensed beam and provides a spectrum of the reflected beam, while the camera <NUM> which is, for example, an RGB (red, green, and blue) camera generates images of the object <NUM>. Both the spectrum of the reflected beam and the generated images of the object <NUM> are transferred to the computer (not shown) connected to the system <NUM>. The computer provides a spectrum analysis and hyperspectral cube of the object <NUM>.

The RGB camera identifies features of sub-resolution of the hyperspectral resolution which in turn allow the determination of the mixture level of the hyperspectral pixel whenever the spatial resolution is coarser than the region of interest (ROI) and the pixel contains a mixed spectrum of the ROI and its background (mixed pixels). Additionally, since the camera <NUM> and the spectrometer <NUM> are aligned, the camera <NUM> allows accurate identification of the exact location of the spectrometer's measurements in the generated image.

Throughout its operation, the system <NUM> is calibrated for transmission losses such that each point of measurement is corrected according to the system's transmission at the specific point. The calibration is performed using the formula: <MAT> Î(Θ, Ω, Z) is the estimator for the point of radiation in a specific space location, Imeasure(Θ, Ω, Z) is the system's measurement in that point, and T(Θ, Ω, Z) is the system's transmission in that point.

The calibration may further include the subtracting of ghost imaging, internal reflections etc. and corrections of optical aberrations using, for example, lenses, prisms, splitters, and the like.

The system <NUM> may further feature a filter (not shown) positioned at the entry of the chamber <NUM> so as to limit the illumination effect while conducting outdoor measurements, and an integral radiation meter or sensor to monitor the radiation from external sources, such as global radiation from the sun.

The system <NUM> can be operated on the ground mounted, for example, on a tripod, a post, a tower and the like or it can be assembled on an airplane or unmanned aerial vehicle (UAV). When the system <NUM> is assembled on an unmanned aerial vehicle or airplane, the system <NUM> may be stabilized by a gimbal set, for example, a three-axis gimbal set (roll, pitch, and yaw) in which each axis offers a degree of freedom to prevent hovering errors.

In another embodiment, the beam splitting process may be operated by, for example, a fiber optic splitter and the like.

<FIG> is a flow chart of the system <NUM>. A reflected light field from the object <NUM> is directed by the prisms 104a-104b and transferred through the chamber <NUM>. In the chamber <NUM>, the light is split into, for example, two separate beams, a first beam directed to the camera <NUM> and a second beam directed to the spectrometer <NUM>. The motion of the chamber <NUM> to compensate external motion such as unmanned aerial vehicle (UAV) vibration is controlled by a controller <NUM> connected to an Inertial Measurement Unit (IMU) or other feedback orientation sensors. The chamber motion may also be estimated from changes in the instantaneous field of view of the camera <NUM>.

The controller <NUM> execute the movement of the prisms 104a-104b according to the scanning plan. Prisms' motion feedback may be controlled through motion indicator such as encoders and may also be estimated from changes in the instantaneous field of view of the camera <NUM>.

The controller <NUM>, which is processor-based, is controlled by a central processing unit (CPU). Programs for running the CPU, as well as programs for causing motion, in response to a motion command, are stored in a storage/memory unit. The CPU is in electronic and/or data communication with a motion module. The motion is carried out according to a scanning plan on a computer <NUM>.

The returned information of the scanning plan, the controller feedback/ IMU signal and the recording of the camera images are processed on the computer <NUM> to calculate the prisms 104a-104b orientation and exact measuring location. Using predetermined information on prism transfer and radiation measurments, the signal is calibrated with the measuring location. The data is then incorporated by the computer <NUM> to gain a hyperspectral cube <NUM>.

<FIG> is a side perspective view of the system <NUM>. The system <NUM> is similar in construction and operation to system <NUM>, as detailed above, except where indicated.

The system <NUM> includes a chamber <NUM> enclosing, for example, two prisms 304a-304b, e.g. wedge prisms, two beam splitters 306a-306b positioned sequentially to one another, a camera <NUM>, a condensing lens <NUM>, a spectrometer <NUM>, for example, a point spectrometer and a sensor <NUM>, for example, a light detection and ranging (LIDAR) sensor. The chamber <NUM> further includes an aperture <NUM> through which a reflected light field is entered into the system <NUM>. The system <NUM> is linked to a computer (not shown).

A reflected beam of light from an object <NUM>, for example, a plant in a field is entered into the chamber <NUM> through the aperture <NUM>. The reflected beam reaches the wedge prisms 304a-304b which in turn refract the reflected beam to direct it into the system <NUM>. Following the prisms 304a-304b, the refracted beam reaches the beam splitter 306a. The beam splitter 306a splits the light intensity of the refracted beam into, for example, two separate beams, a first beam directed to the camera <NUM> and a second beam directed to the beam splitter 306b. The second beam is split yet again by the beam splitter 306b into, for example, two separate beams, a first beam directed to the condensing lens <NUM> and a second beam directed to the sensor <NUM>. The LIDAR sensor <NUM> measures the distance to the object <NUM> by illuminating it with laser light and measuring the reflected light. The sensor <NUM> works sequentially in short proximity to the readings of the spectrometer <NUM> or parallel to the spectrometer <NUM>.

The first beam, split by the beam splitter 306b, passes through the condensing lens <NUM> which condense the beam before it reaches the spectrometer <NUM>. The spectrometer <NUM> analyzes the condensed beam's radiation and provides a spectrum of the reflected beam, while the camera <NUM> generates images of the object <NUM>. The computer utilizes the data from the spectrometer <NUM> and the camera <NUM> to provide a hyperspectral cube of the object <NUM>, while the data from sensor <NUM> is utilized to create a range map.

Upon different embodiments of the system <NUM>, the position of the camera <NUM> and the sensor <NUM> is interchangable.

The system <NUM> includes a chamber <NUM> enclosing, for example, two prisms 404a-404b, e.g. wedge prisms, two beam splitters 406a-406b positioned one on top of the other, a camera <NUM>, a condensing lens <NUM>, a spectrometer <NUM>, for example, a point spectrometer and a sensor <NUM>, for example, a light detection and ranging (LIDAR) sensor. The chamber <NUM> further includes an aperture <NUM> through which a reflected light field is entered into the system <NUM>. The system <NUM> is linked to a computer (not shown). Upon different embodiments of the system <NUM>, the position of the camera <NUM> and the sensor <NUM> may interchange
A reflected beam of light from an object (not shown), for example, a plant in a field is entered into the chamber <NUM> through the aperture <NUM>. The reflected beam reaches the wedge prisms 404a-404b which in turn refract the reflected beam to direct it into the system <NUM>. Following the prisms 404a-404b, the refracted beam reaches the beam splitter 406a. The beam splitter 406a splits the light intensity of the refracted beam into, for example, two separate beams, a first beam directed to the condensing lens <NUM> and a second beam directed to the beam splitter 406b. The second beam directed to the beam splitter 406b is split yet again into, for example, two separate beams, a first beam directed to the camera <NUM> and a second beam directed to sensor <NUM>, for example, light detection and ranging (LIDAR) sensor.

The first beam, split by the beam splitter 406a, passes through the condensing lens <NUM> which condense the beam before it reaches the spectrometer <NUM>. The spectrometer <NUM> analyzes the condensed beam and provides a spectrum of the reflected beam, while the camera <NUM> generates images of the object. The computer utilizes the data from the spectrometer <NUM> and the camera <NUM> to provide a hyperspectral cube of the object, while the data from sensor <NUM> is utilized to create a range map.

<FIG> is a flow chart of the systems <NUM> and <NUM>. The flow chart refers in numbers to system <NUM> but is also applicable for system <NUM> which is similar in construction and operation to system <NUM>.

A reflected light field from the object <NUM> is directed by the prisms system 404a-404b and transferred through the chamber <NUM>. In the chamber <NUM>, the light is split into, for example, two separate beams, a first beam directed to the camera <NUM> and a second beam directed to the spectrometer <NUM> and the LIDAR sensor <NUM>. The motion of the system <NUM> to compensate external motion such as UAV vibration is controlled by a controller <NUM> connected to an Inertial Measurement Unit (IMU) or other feedback orientation sensors. The chamber motion may also be estimated from changes in the instantaneous field of view of the camera <NUM>.

The controller <NUM> execute the movement of the prisms 304a-304b according to the scanning plan. Prisms' motion feedback may be controlled through motion indicator such as encoders and may also be estimated from changes in the instantaneous field of view of the camera <NUM>.

The returned information of the camera <NUM>, the spectrometer <NUM> and the LIDAR sensor <NUM> are processed on the computer <NUM> to gain a hyperspectral cube <NUM> and a range map <NUM>.

The following examples are not meant to limit the scope of the claims in any way. The following examples are put forth so as to provide those of ordinary skill in the art with a complete disclosure and description of how to make and use the described invention, and are not intended to limit the scope of the invention, nor are they intended to represent that the experiments below are all or the only experiments performed.

Experimental results of a lab prototype are presented in <FIG>. Fig. 6A presents indoor measurements of assembly of <NUM> white light-emitting diodes (LED) with filters, red (R), green (G) and white without a filter. The RGB images of the review camera are presented on the left while the measured spectrum is presented below. The field of view of the review camera is wider than that of the spectrometer. While measuring a specific led, other LEDs may be seen in the review camera without effecting the spectrometer readings. Fig. 6B presents outdoor measurements of a rose flower. The image of the rose flower through the review camera and the rose spectrum are presented side by side. Integration time of the spectrometer was <NUM>. This integration time allows coverage of <NUM> square meters when the system is mounted on a <NUM> hydraulic post and tilted in <NUM> degrees, with a GSD (Ground Sampling Distance) of <NUM>, or coverage of <NUM> square meters with a GSD of <NUM> when the system stares downwards from <NUM> meters.

<FIG> presents an implementation of a system for hyperspectral sampling of crop assembled on a hydraulic post. The system consists of the components presented in Table <NUM> below.

<FIG> present hyper spectral results using the lab prototype of Example <NUM> while using only a point spectrometer. <FIG> presents a standard image of an ordinary building taken by an ordinary RGB camera. <FIG> is a pseudo RGB image composed of <NUM> layers of an hyper spectral cube out of more than <NUM> layers. The mesurments are done approximetrly <NUM> from the building.

Claim 1:
A scanning system (<NUM>) comprising a chamber (<NUM>) configured for receiving reflected light from an object (<NUM>), said chamber (<NUM>) comprising:
a first prism (104a) and a second prism (104b) configured for refracting said reflected light;
at least one beam splitter (<NUM>) configured for splitting said reflected light into a first beam and a second beam;
a camera (<NUM>) configured for receiving said first beam so as to provide images of said object (<NUM>);
a condenser lens (<NUM>) configured for condensing said second beam;
a spectrometer (<NUM>) configured for receiving the condensed second beam and providing a spectrum analysis of said second beam;
a controller (<NUM>); and
a particular computer (<NUM>);
wherein the controller (<NUM>) executes the movement of the prisms in accordance with a scanning plan stored on the computer (<NUM>),
characterized in that
said system (<NUM>) is calibrated to correct for transmission losses such that each point of measurement is corrected according to the system's transmission at the specific point using the formula: Î(θ, Ω, Z) = F(Imeasure(θ, Ω, Z), T(θ, Ω, Z)),
wherein Î(θ, Ω, Z) is the estimator for the point of radiation in a specific space location, Imeasure(θ, Ω, Z) is the system's measurement in that point, and T(θ, Ω, Z) is the system's transmission in that point,
and in that the information of the scanning plan, the prism's motion feedback from the controller, and the recording of the camera images are processed on the computer (<NUM>) to calculate the prisms orientation and exact measurement location to calibrate the measured signal with the measurement location; and then used by the computer to combine said images of said object (<NUM>) produced by said camera (<NUM>) and said spectrum analysis produced by said spectrometer (<NUM>) to provide a hyperspectral image cube (<NUM>).