MULTISPECTRAL IMAGING AND CHARACTERIZATION OF INTERNAL BIOLOGICAL TISSUE

Abstract: A light image is conveyed from internal biological tissue through a flexible optical system (13) to an image receiver (14), where it is converted to a form which may be entered into a computer (15). The computer segments the image by generating a segmentation mask defining the boundary of a region of interest in at least one spectral band, estimates at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations of the digital images in at least one spectral band, characterizes the condition of the tissue based on the estimated values and outputs the characterization of the condition of the tissue.

Description: MU TISPECTRAL IMAGING AND CHARACTERIZATION OF INTERNAL BIOLOGICAL TISSUE FIELD OF THE INVENTION The field of the invention is automatic machine vision and classification, and, in particular, the classification of biological tissue based on images ofthe biological tissue. BACKGROUND OF THE INVENTION U.S. Patent No. 6208749 Bl, issued March 27, 2001, discloses systems and methods for the multispectral imaging of skin tissue enabling the automatic characterization of the condition of a region of interest of skin based on direct digital imaging of the region of interest or the digitization of color photographic slides of the region of interest, illuminated by appropriately filtered light. According to that patent, a digital image is automatically segmented in at least one spectral band. A digital processor segments the other images based on the segmentation mask. Parameters related to the texture, asymmetry, blotchiness and border irregularities are also automatically estimated. The region of interest is automatically characterized by the processor, based on the parameters. Skin lesions may be characterized as malignant or benign. U.S. Patent No. 6,081,612, issued June 27, 2000, discloses systems and methods for characterizing the condition of a region of interest of the skin, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function of the condition of the skin. The method comprises illuminating the region of interest of the skin by light in at least three spectral bands and digitally imaging the region of interest at the at least three spectral bands with the light re-emitted by the skin to generate digital images comprising digital signals whose values are a function ofthe condition ofthe skin. The digital images are provided to a processor which segments the digital images by generating a segmentation mask from a digital image in any one of the at least three spectral bands. The processor estimates at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations of the digital images in each spectral band, which are functions of the texture of the region of interest determined by the segmentation mask. The processor characterizes the condition of the skin based on the estimated values, and outputs the characterization of the condition of the skin. Preferably, the segmenting, estimating and characterizing steps are conducted without the intervention of an operator. Additional 5 parameters include measures of he texture, asymmetry, blotchiness and border irregularity of the portion of the region of interest. The digital images may be obtained by directly imaging the region of interest with a digital camera, or digitally imaging color slides ofthe region of interest, through appropriately filtered light. Not included in the above references are references to conducting light from place 0 to place so that the image is conserved. OBJECTS OF THE INVENTION It is an object ofthe invention to illuminate and image biological tissue, principally in vivo, and to convey the image in the form of light to an image receiver for conversion to 5 electrical signals and for automatic recognition of biological features. SUMMARY OF THE INVENTION A light image is conveyed from internal biological tissue through flexible optical system to an image receiver, where it is converted to a form that may be entered into a o computer. The computer segments the image by generating a segmentation mask defining the boundary of a region of interest in at least one spectral band, estimates at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations ofthe digital images in at least one spectral band, characterizes the condition of the tissue based on the estimated values, and outputs 5 the characterization of the condition of the tissue. Internal biological tissue encompasses tissue internal to a human or animal body, such as the stomach, the esophagus, the colon or the nasal cavities, for example. In particular, in accordance with the present invention, a method of characterizing the condition of a region of interest of internal biological tissue, wherein the absorption o and scattering of light in different spectral bands by the region of interest is a function of the condition of he tissue, is disclosed. The method comprises inserting a flexible optical system into a body, proximate the region of interest. A portion of the tissue is illuminated including the region of interest by light in at least one spectral band. The image is conveyed through the flexible optical system to an image receiver. A portion of the tissue 5 including the region of interest is digitally imaged at the at least one spectral band with the light re-emitted by the portion of the tissue. Digital images are generated comprising digital signals whose values are a function of the condition of the region of interest of the tissue. The digital images are provided to a processor. The processor segments the digital images by generating a segmentation mask defining the boundary of the region of interest 0 from a digital image in any one of the at least one spectral bands, preferably without the intervention of an operator. The processor also estimates at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations of the digital images in each spectral band and characterizes the condition of the tissue based on the estimated values, preferably without 5 the intervention of an operator. The processor then outputs the characterization of the condition ofthe tissue. The at least one statistical measure may be calculated separately within a border region or an interior region of the digital image. The border region is such that it encompasses the envelope of circles of fixed radius centered on the boundary of the o segmentation mask. The inside region comprises all points ofthe image that are within the segmentation mask boundary but not included in the border region. The computing step may comprise estimating at an individual level at least one value which is a statistical measure of texture ofthe portion ofthe region of interest within the border region and interior region. The statistical measure is chosen from the group 5 consisting of: the number of wavelet maxima per unit area; the ratio of the mean coefficient magnitude to the absolute deviation of the coefficient magnitudes from the mean value; the ratio of the mean coefficient magnitude to the standard deviation of the o coefficient magnitude; and the skewness of the coefficient magnitude, normalized to the cube of the standard deviation. The estimating step may comprise estimating either the degree of change of a statistic of the wavelet coefficient distribution with increment of wavelet level, or the 5 degree of deviation of such change from linearity. The estimating step may also comprise estimating the average rate of change, with respect to level, ofthe number of wavelet maxima per unit area. The estimated texture values may be compared to a threshold derived from statistical analysis of a multiscale wavelet transformation of the digital image, to 0 characterize the condition ofthe tissue. In accordance with another embodiment of the invention, a system for characterizing the condition of a region of interest of internal biological tissue is disclosed, comprising a source of light for illuminating the region of interest in at least one spectral band. A flexible optical system is provided for conveying the image of the region of 5 interest to an image receiver. An image receiver is provided for acquiring digital images of the region of interest based on the light re-emitted from the illuminated region of interest at each of the spectral bands. The digital image comprises digital signals whose values are a function of the condition of the region of interest. Memory is provided for storing the digital images provided by the image receiver. A digital processor is provided, o programmed to perform the steps of: segmenting the digital images stored in memory by generating a segmentation mask from a digital image in any one ofthe at least one spectral band; estimating at least one rotationally and translationally invariant statistical measure of coefficient distributions for the multiscale wavelet maxima representations ofthe digital 5 images in each spectral band, which are functions of the texture of the region of interest determined by the segmentation mask; characterizing the condition ofthe tissue based on the estimated values; and outputting the characterization ofthe region of interest. The flexible optical system may be an endoscope, a fiber optic bundle or an o articulated arm. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a sketch ofthe system ofthe invention; and Fig. 2 is a sketch of an articulated arm of a system ofthe invention. DETAILED DESCRIPTION OF THE INVENTION Fig 1 shows a sketch of the system of the invention. A patient 10 has a flexible endoscope 11 inserted into a natural or surgically provided orifice of the body. As is very well known in the art of endoscopy, light is provided for illuminating the tissue inside the body, and the light reflected from the tissue and scattered from under the surface of the tissue falls on the end 12 of the endoscope. The first end of the endoscope 12 generally has a lens (not shown) which images the tissue on to the end of a coherent fiber bundle (not shown). The light imaged on to the fiber bundle is carried to the other end of the endoscope 13, where it may be projected on to a film or an electronic image receiver 14 such as are found in video or digital cameras. The light for illuminating the tissue may be provided through the endoscope. The output ofthe image receiver is carried to a processor or computer 15 where the computer segments the image by generating a segmentation mask defining the boundary of a region of interest in at least one spectral band, estimates at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations of the digital images in at least one spectral band, characterizes the condition of the tissue based on the estimated values, and outputs the characterization of the condition of the tissue. The computer processes are described in great detail in U.S. Patent No. 6,208,749, Bl, issued March 27, 2001, U.S. Patent No. 6,081,612, issued June 27, 2000, International Publication No. WO99/44010, published 2 September 1999; and International Publication No. W098/37811, published 3 September 1998, which are incorporated by reference herein, in their entireties. Fig. 2 shows a sketch of an articulated arm 20 for conveying the images. The articulated arm is especially useful when the illuminating light is of a wavelength which is not transmitted well by the glass used in typical endoscopes. A lens 21 is placed so that the tissue to be imaged 22 is removed from the lens 21 by the focal distance / of the lens 22. Light 23 diverging from the tissue 22 is converted to a parallel beam 24 which propagates through a tube 25. The parallel light beam 24 may be reflected from mirrors 25 and 25' mechanically set to direct the beam 24 through other tubes 24' flexibly connected to tube 24. Tubes 24, 24', and 24" rotate on their axis to change the plane of propagation ofthe light beam 24. When the light beam 24 exits the articulated arm, it may be focused on an image receiver (not shown) to capture an image ofthe tissue 22. Obviously, many modifications and variations ofthe present invention are possible in light of the above teachings. It is therefore to be understood that, within the scope of the appended claims, the invention may be practiced otherwise than as specifically described.

Claims: We claim: 1. A method of characterizing the condition of a region of interest of internal 5 biological tissue, wherein the absorption and scattering of light in different spectral bands by the region of interest is a function ofthe condition ofthe tissue, the method comprising: inserting a flexible optical system into a body, proximate the region of interest; illuminating a portion of the tissue including the region of interest by light in at least one spectral band; 0 conveying an image ofthe region of interest through the flexible optical system to an image receiver; digitally imaging a portion of the tissue including the region of interest at the at least one spectral band with the light re-emitted by the portion of the tissue to generate digital images comprising digital signals whose values are a function of the condition of 5 the region of interest of the tissue; and providing the digital images to a processor, wherein the processor: segments the digital images by generating a segmentation mask defining the boundary of the region of interest from a digital image in any one of the at least one spectral band; o estimates at least one rotationally and translationally invariant statistical measure of coefficient distributions of the multiscale wavelet maxima representations ofthe digital images in each spectral band; characterizes the condition ofthe tissue based on the estimated values; and outputs the characterization ofthe condition ofthe tissue. 5 2. The method of claim 1, wherein the at least one statistical measure is calculated separately within either of a border region and an interior region of the digital image, wherein: the border region encompasses the envelope of circles of fixed radius centered on o the boundary of the segmentation mask; and the inside region comprises all points ofthe image that are within the segmentation mask boundary but not included in the border region. 3. The method of claim 2, wherein the computing step comprises estimating at an individual level at least one value which is a statistical measure of texture ofthe portion of the region of interest within the border region and interior region, chosen from the group consisting of: the number of wavelet maxima per unit area; the ratio of the mean coefficient magnitude to the absolute deviation of the coefficient magnitudes from the mean value; the ratio of the mean coefficient magnitude to the standard deviation of the coefficient magnitude; and the skewness of the coefficient magnitude, normalized to the cube of the standard deviation. 4. The method of claim 1, further comprising estimating either of the degree of change of a statistic ofthe wavelet coefficient distribution with increment of wavelet level, and the degree of deviation of such change from linearity. 5. The method of claim 2, further comprising estimating the average rate of change, with respect to level, ofthe number of wavelet maxima per unit area. 6. The method of claim 1, further comprising comparing the estimated texture values to the threshold derived from statistical analysis of a multiscale wavelet transformation of the digital image. 7. The method of claim 1, wherein the estimating and characterizing steps are conducted without the intervention of an operator. 8. The method of claim 1, wherein the segmenting step is conducted without the intervention of an operator. 9. The method of claim 1, comprising conveying the image through an endoscope. 10. The method of claim 1, comprising conveying the image through a fiber optic bundle. 11. The method of claim 1, comprising conveying the image through an articulated 5 arm. 12. A system for characterizing the condition of a region of interest of internal biological tissue, comprising: a source of light for illuminating the region of interest in at least one spectral band; 0 a flexible optical system for insertion into a body and for conveying the image of the region of interest to an image receiver; an image receiver for acquiring digital images ofthe region of interest based on the light re-emitted from the illuminated region of interest at each of the spectral bands, the digital image comprising digital signals whose values are a function ofthe condition ofthe 5 region of interest; memory for storing the digital images provided by the image receiver; a digital processor programmed to perform the steps of: segmenting the digital images stored in memory by generating a segmentation mask from a digital image in any one of the at least one o spectral band; estimating at least one rotationally and translationally invariant statistical measure of coefficient distributions for the multiscale wavelet maxima representations ofthe digital images in each spectral band, which are functions of the texture of the region of interest determined by the segmentation mask; 5 characterizing the condition ofthe tissue based on the estimated values; and outputting the characterization ofthe region of interest. 13. The system of claim 12, where the flexible optical system is an endoscope. o 14. The system of claim 12, where the flexible optical system is a fiber optic bundle. 5. The system of claim 12, where the flexible optical system is an articulated arm.

Inventors: BOGDAN ALEXANDRU, ELBAUM MAREK, GREENEBAUM MICHAEL, GUTKOWICZ-KRUSIN DINA, JACOBS ADAM, BOGDAN, ALEXANDRU, ELBAUM, MAREK, GREENEBAUM, MICHAEL, GUTKOWICZ-KRUSIN, DINA, JACOBS, ADAM
Applicants: ELECTRO OPTICAL SCIENCES INC, ELECTRO-OPTICAL SCIENCES, INC.
Application year: 2001
Main classification: G01B
IPC codes: A61B5