Patent Description:
Spectroscopy is the study of the interaction between matter and electromagnetic radiation as a function of the wavelength or frequency of the radiation. In simpler terms, spectroscopy is the precise study of colour as generalized from visible light to all bands of the electromagnetic spectrum; indeed, historically, spectroscopy originated as the study of the wavelength dependence of the absorption by gas phase matter of visible light dispersed by a prism.

Spectroscopy, primarily in the electromagnetic spectrum, is a fundamental exploratory tool in the fields of physics, chemistry, and astronomy, allowing the composition, physical structure and electronic structure of matter to be investigated at the atomic, molecular and macro scale, and over astronomical distances. Important applications arise from biomedical spectroscopy in the areas of tissue analysis and medical imaging.

Spectral imaging (SI) is imaging that uses multiple bands across the electromagnetic spectrum. While an ordinary camera captures light across only three wavelength bands in the visible spectrum, red, green, and blue (RGB), spectral imaging encompasses a wide variety of techniques that go beyond RGB.

Spectral imaging may use the infrared, the visible spectrum, the ultraviolet, x-rays, or some combination of the above. It may include the acquisition of image data in visible and non-visible bands simultaneously, illumination from outside the visible range, or the use of optical filters to capture a specific spectral range. It is also possible to capture hundreds of wavelength bands for each pixel in an image, so called Hyper Spectral Imaging (HSI).

HSI is an imaging method that integrates spectroscopy and imaging to obtain both spatial and spectral information from a given field of view. New technologies with increasing spatial resolution have allowed the application of HSI in a wide range of fields, which exploit the chemical information that can be retrieved from the spectrum on each pixel of the sample. These applications include food quality control, medical applications, material sorting, among others.

Analogous to RGB images where each pixel consists of three colour channels, spectral cubes' pixels can consist of several wavelength channels, for example ranging from VIR to NIR spectral regions. As a result, SI data is stored in a three-dimensional cube, where the third dimension belongs to the spectral information and each channel image belongs to the spectral response for a particular wavelength.

The final goal of SI is to be able to detect materials, contamination, or substance concentration in a sample, as well as aid in the sorting process, among other applications. Classification and anomaly detection algorithms are applied for this purpose. The processing of this data is challenging due to its high-dimensionality and the presence of intensity inhomogeneities, which derive from the fact that the imaging array possesses different physical properties than other imaging methods.

The measurement process of imaging is affected by various conditions. Technologically, imaging in a reflection geometry relies on the illumination of a scene and the detection of the backscattered waves. In the case of spectral imaging, there is the requirement, as in all spectroscopic applications, to prevent the detector from directly reflected waves as the scattered portion of the wave is typically weak compared to it.

Hence, there is the intrinsic need for two rather independent pathways. This is typically realized by a certain geometrical arrangement for the illumination and detection scheme (significant angle and distance between source and detector). Due to this geometric arrangement, sample structure and topography, can become obstacles to the measurement procedure. Furthermore, if parts of the sample are unfavourably positioned, there may be waves scattered off a first sample surface and reflected by a second sample surface hitting the detector and being falsely attributed to this second sample (detector pointed at surface two is receiving information from surface one).

For automation, the main purpose of machine vision is the segmentation of images. The more reliable and consistent the segmentation is, the more robust (and hence faster) the automation can be operated. To achieve a stable and robust image segmentation based on classification from spectral information, topographic effects must be minimized. This is achieved by correcting the recorded spectral image data (= spectral cubes) using appropriate pre-processing to compensate the effects of topography, shadowing, illumination, etc. on the intensity profiles.

Hence, the problems tackled in the application are:.

In the prior art various solutions for coping with these problems are described. The simplest and most-used approach is to apply pre-processing and normalization techniques to diminish scattering and shadowing effects. An example is disclosed in <NPL>. As further examples, <CIT> explains the reduction of disturbing effects, especially caused by sun-glint, in a multispectral image, while <CIT> deals with compensation of atmospheric effects and sensor calibration problems in multispectral imaging.

The objective of the present invention is to provide a solution for improving spectral imaging by overcoming above mentioned problems.

To accomplish the objective, the present invention provides a solution according to the independent claims. Advantageous embodiments are provided in the dependent claims.

According to a first aspect of the invention the proposed pre-processing method is based on an illumination mask, which compensates for topographically induced intensity variations which cannot be attributed to by the spectral response of the material on the entire SI cube of data (SI cube = 2x spatial + 1x spectral dimension) before entering data modelling and classification algorithms.

The illumination mask is extracted from the channel image of a spectral data cube (= one slice of the cube = one 2D-spatial representation at any selected wavelength channel) with the maximum (mean) reflectance (= minimum absorption).

A low-pass filter, e.g. Butterworth, Gaussian, or raised cosine is applied in the Fourier domain to the selected maximum reflectance channel image. Other low-pass filters can be used such as a 'step-function' or 'ideal filter'. The alternatives are typically subject to ringing artifacts.

The intensity correction challenge for SI stems from the fact that correcting only in the spatial domain may modify the spectral features of the corrected pixels. An independent correction for each channel would certainly lead to the loss of spectral information. For this reason, the illumination mask for the SI cube is extracted from only one channel image.

The proposed method consists of following steps:.

Smooth intensity changes in images are translated as small frequencies in the Fourier domain, while high contrast and small features are associated to large frequencies. Low pass filtering retrieves small frequencies from the Fourier domain, which allows the extraction of intensity inhomogeneities (low contrast variation associated with low frequencies).

Preferably, a Butterworth filter is chosen as a low-pass filter since it presents no ring artifacts and is adjustable with two parameters: the cut-off frequency and the order of the filter. The cut-off frequency changes the frequency limit to be filtered, and the filter order denotes the variation between a Gaussian approximation and an ideal step filter.

The invention claims a computer implemented method for modifying spectral imaging for gaining minimum spectral cross-contamination, wherein spectral channel images of a spectral cube of a scene are modified by an illumination mask, wherein the illumination mask is generated by convolutional low-pass filtering of a first spectral channel image of the spectral cube.

By selecting an appropriate first channel image the method yields to data which are rescaled and/or adjusted to an as if samples in the scene were flat within reasonable limits.

In a further embodiment the method comprising the following steps:.

In a further embodiment the first rule is defined by the fact that the spectral image contains the least significant sample information of the scene.

In a further embodiment the least significant sample information means maximum reflectance.

The spectral channel image with the maximum reflectance is selected since a low reflectance in this channel will be caused by the absence of incident light instead of the high absorption of the samples.

In a further embodiment average reflectance spectrum values are calculated for the spectral channel images and the first spectral channel image is the spectral channel image with the highest value.

In a further embodiment the second rule comprises order and cut-off frequency of the low-pass filter, wherein the order is chosen to approximate a smooth frequency response, and the cut-off frequency is based on a smallest resolvable 2D spatial dimension in the scene.

In a further embodiment the low-pass filter is a Butterworth filter, preferably of the order <NUM> or <NUM>.

In a further embodiment the third rule is: dividing the intensity value of every pixel of the spectral cube by the intensity value of the corresponding pixel of the illumination mask.

In a further embodiment the spectral imaging is hyper spectral imaging.

Furthermore, the invention claims a spectroscopy system comprising a computational device designed to perform a method according to the inventive method.

Furthermore, the invention claims a computer program product comprising instructions which, when the program is executed by a computational device, cause the computational device to carry out the steps of the method according to the invention.

Finally, the invention claims a computer-readable storage medium comprising instructions which, when executed by a computational device, cause the computational device to carry out the steps of the method of claim according to the invention.

The invention is described by an embodiment using Hyper Spectral Imaging (HIS). The intensity correction challenge for HSI stems from the fact that correcting only in the spatial domain may modify the spectral features of the corrected pixels. An independent correction for each channel would certainly lead to the loss of spectral information. For this reason, the illumination mask for the HSI cube is extracted from only one channel image.

The introduced correction method obtains the illumination mask from the highest reflectance channel image f (x, y, λmax) (= first rule). This spectral channel image is selected since a low reflectance in this channel will be caused by the absence of incident light instead of the high absorption of the scene/sample/object. The average reflectance spectrum is calculated for the entire HSI cube and the highest value is selected as observed as illustrated in <FIG> by the dotted line "Max. reflectance channel".

Once the highest reflectance channel image (= first spectral channel image) is selected, the low-pass filter (selected according to a second rule) is applied, hence extracting the illumination mask. This follows the standard steps for filtering in the Fourier domain and extracting the illumination mask.

The illumination mask is estimated with the highest reflectance wavelength and the same mask corrects the entire cube, hence maintaining the spectral integrity and coherency of every pixel.

The low-pass filter, preferably a Butterworth filter according to <FIG>, is defined by order (a measure for steepness) and cut-off frequency. Subsequently, an example for a scene with fruits and vegetables, comprising olives and tomatoes, is described.

As for the choice of the order it can be said:.

As for the choice of the cut-off frequency it can be said:.

<FIG> illustrates the effect of low-pass filtering according to the invention of HSI of a scene with vegetables and fruits as samples.

The illustrated pre-processing method is applied before further Machine Learning. Hence it is compensating for topography cp. to simple scaling procedures without local context (neighbouring positions on objects).

The latter is describing global correction approaches. These are typically causing spectral cross-contamination between different regions in the dataset. This means that the correction algorithm is adapting to one region of the dataset and then applying its "correction" to all other regions. This may work for scenes where only one type of material is present, but even there it will mix up the features between the different regions of the sample.

Example: given an HSI-cube of a sample region of sand castles (= topography) with inhomogeneous distributed local humidity (= wet at one site, dry at all other regions let the global correction be adapting to the wet region. The global correction will infer "humidity" to all the dry regions simultaneously when correcting for topographic features.

Claim 1:
Computer implemented method for modifying spectral imaging for gaining minimum spectral cross-contamination, characterized in that
spectral channel images of a spectral cube of a scene are modified by an illumination mask,
wherein the illumination mask is generated by convolutional low-pass filtering of a first spectral channel image of the spectral cube.