Electronic endoscope system and method for controlling the same

A light source device of an electronic endoscope system has first and second semiconductor lasers. In a vascular observation mode, one of the semiconductor lasers is used in a full light state (100% rated output) while the other is used in a reduced light state (for example, 10% rated output). First and second images of an internal body portion are captured with a color imaging device under illumination of two patterns, respectively. Correlation operation of pixel values of three colors is performed between the two images. Noise components, caused by the first or second semiconductor lasers in the light reduced state, are removed from the first and second images. An oxygen saturation level of blood in a blood vessel is calculated using the first and second images with the noise components removed.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an electronic endoscope system for imaging an internal body portion using two or more types of illumination light and a method for controlling the same.

2. Description Related to the Prior Art

In the medical field, electronic endoscopes are widely used in medical examinations. The electronic endoscope is provided with an insert section to be inserted into a patient's body. Illumination light is applied to an internal body portion of the patient from a distal end of the insert section. An imaging device disposed in the distal end captures an image of the internal body portion being illuminated.

The electronic endoscope is connected to a lighting device. The illumination light from a light source of the lighting device is supplied to the electronic endoscope. Conventionally, a white light source (e.g. a xenon lamp or a metal halide lamp) has been used. Recently, a technique using narrowband light as the illumination light has attracted attention (see Japanese Patent No. 3583731 corresponding to U.S. Patent Application Publication No. 2008/0281154). The narrowband light is applied to the internal body portion and an image of the reflection light is captured. The use of the narrowband light facilitates finding a lesion.

A method for obtaining vascular information (e.g. an oxygen saturation level of hemoglobin in blood) based on image signals of images has been researched (see Japanese Patent Laid-Open Publication No. 06-315477). In this method, the images are captured under the illumination of the respective different types of narrowband light applied alternately. The method utilizes the narrowband light of wavelength sets in the respective wavelength bands of 300 to 400 nm, around 400 nm, 400 to 500 nm, 500 to 600 nm, and 450 to 850 nm.

A semiconductor light source (e.g. a semiconductor laser diode) has been used for a narrowband light source. Frequent turning on and off of the semiconductor light source causes overshoot of its output mainly due to temperature factors. In other words, a light quantity of the illumination light increases instantaneously after the turning on, making the light quantity unstable and uneven. Thereby, accurate vascular information cannot be obtained. The U.S. Patent Application Publication No. 2008/0281154 and Japanese Patent Laid-Open Publication No. 06-315477 do not disclose solution to the problem.

SUMMARY OF THE INVENTION

An object of the present invention is to provide an electronic endoscope system for preventing unevenness in light quantity due to overshoot of a semiconductor light source and a method for controlling the same.

To achieve the above and other objects, an electronic endoscope system of the present invention includes at least first and second light source system, a color imaging device, a controller, a noise removal section, an image production section, and a display section. The first and second light source systems illuminate an internal body portion including a blood vessel. The first light source system generates first illumination light including first narrowband light. The second light source system generates second illumination light including second narrowband light. The first light source system has a first semiconductor light source. The color imaging device images the internal body portion illuminated with the first or second illumination light. The color imaging device has pixels of two or more colors. Electric charge accumulated in each pixel is read out periodically as a pixel value. The controller controls the first and second light source systems such that the first and second illumination light is applied alternately to the internal body portion in a vascular observation mode. The controller puts the first semiconductor light source of the first light source system into a reduced light state without turning off the first semiconductor light source during the application with the second illumination light. The noise removal section removes a noise component from the pixel value, used for imaging of vascular information of the internal body portion, with the use of the correlation operation of the pixel values of the two or more colors, to calculate a corrected pixel value. The noise component is caused by illumination of the first semiconductor light source in the reduced light state. The image production section produces a vascular information image based on the corrected pixel value. The display section displays the vascular information image.

It is preferable that one of the first and second light source systems is alternately put into the reduced light state for a charge accumulation period of the pixel.

It is preferable that the second light source system has a second semiconductor light source. The controller puts the second semiconductor light source of the second light source system into the reduced light state without turning off the second semiconductor light source during the application of the first illumination light. The noise removal section further removes a noise component, caused by the illumination of the second semiconductor light source in the reduced light state, to calculate the corrected pixel value.

It is preferable that illumination of a first emission pattern and illumination of a second emission pattern is applied alternately in the vascular observation mode. The first emission pattern is a mixture of the first illumination light and the second illumination light in the reduced light state. The second emission pattern is a mixture of the first illumination light in the reduced light state and the second illumination light.

It is preferable that the first light source system has the first semiconductor laser for generating the first narrowband light, and a wavelength converter for generating fluorescence in a wavelength range from green to red upon excitation with the first and second narrowband light. It is preferable that the second light source system has the second semiconductor laser for generating the second narrowband light and the wavelength converter shared with the first light source system.

It is preferable that the first narrowband light has an emission peak in a blue wavelength range. It is preferable that the second narrowband light has an emission peak in a wavelength range from blue and green. It is preferable that the fluorescence is broadband light in a wavelength range from green to red. The first illumination light is a mixture of the fluorescence from the wavelength converter excited by the first narrowband light, and the first narrowband light passed through the wavelength converter. The second illumination light is a mixture of the fluorescence from the wavelength converter excited by the second narrowband light, and the second narrowband light passed through the wavelength converter.

It is preferable that the two or more colors are red, green, and blue. It is preferable that the red and green pixel values obtained under the illumination of the first emission pattern and a blue pixel value obtained under the illumination of the second emission pattern are used for producing the vascular information image.

It is preferable that the noise removal section removes the noise component, caused by the second illumination light in the reduced light state, from the green pixel value obtained under the illumination of the first emission pattern, and removes a noise component, caused by the first illumination light in the reduced state, from the blue pixel value obtained under the illumination of the second emission pattern.

It is preferable that the two or more colors are cyan, magenta, and yellow. It is preferable that magenta and yellow pixel values obtained under the illumination of the first emission pattern and a cyan pixel value obtained under the illumination of the second emission pattern are used for producing the vascular information image.

It is preferable that the noise removal section removes the noise component, caused by the second illumination light in the reduced light state, from the magenta and yellow pixel values obtained under the illumination of the first emission pattern, and removes a noise component, caused by the first illumination light in the reduced state, from the cyan pixel value obtained under the illumination of the second emission pattern.

It is preferable that the vascular information is an oxygen saturation level of hemoglobin in blood in the blood vessel, and the blood vessels in the vascular information image are color-coded in accordance with the oxygen saturation level.

It is preferable that the electronic endoscope system further includes a normal observation mode. In the normal observation mode, the internal body portion is illuminated with the first illumination light.

A method for controlling an electronic endoscope system includes an applying step, an imaging step, a producing step, and a displaying step. In the applying step, first and second illumination light is applied alternately to an internal body portion including a blood vessel. The first semiconductor light source of the first light source system is kept in a reduced light state without being turned off during the application of the second illumination light. In the imaging step, the internal body portion is imaged with a color imaging device. The color imaging device has pixels of two or more colors. Electric charge accumulated in each pixel is read out periodically as a pixel value. In the producing step, a vascular information image is produced based on the pixel value of the each color. In the displaying step, the vascular information image is displayed on a display section.

It is preferable that the second light source system has a second semiconductor light source, and the second semiconductor light source is kept in a reduced light state without being turned off during the application of the first illumination light.

It is preferable that the method further including a noise removing step. In the noise removing step, a noise component is removed from the pixel value, used for imaging of vascular information of the internal body portion, with the use of correlation operation of the pixel values of the two or more colors, to calculate a corrected pixel value. The noise component is caused by the illumination of the first semiconductor light source or the second semiconductor light source in the reduced light state. The vascular information image is produced based on the corrected pixel value.

According to the present invention, the semiconductor light source is kept turned on with its light quantity reduced even when it is supposed to be turned off. Accordingly, the overshoot, caused by the turning on of the semiconductor light source, is prevented. The noise components caused by the light from the constantly turned on semiconductor light sources are removed from the pixel values of multiple colors. Thereby, accurate vascular information image is produced.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

InFIG. 1, an electronic endoscope system2is provided with an electronic endoscope10, a processor device11, a light source device12, and the like. The electronic endoscope10has a flexible insert section13to be inserted into a subject (patient's body), a handling section14joined to a basal portion of the insert section13, a connector15connected to each of the processor device11and the light source device12, and a universal cord16connecting the handling section14to the connector15.

The handling section14is provided with operation members, for example, an angle knob for bending a distal portion17of the insert section13in horizontal and vertical directions, an air/water button for ejecting air and/or water from an air/water nozzle, and a release button for capturing a still observation image (endoscopic image).

A forceps inlet is provided on a distal side of the handling section14. A medical instrument such as an electric scalpel is inserted into the forceps inlet. The forceps inlet is connected to a forceps outlet provided on the distal portion17through a forceps channel in the insert section13.

The processor device11is connected electrically to the light source device12and controls operation of the whole electronic endoscope system2. The processor device11supplies power to the electronic endoscope10through a transmission cable routed through the universal cord16and the insert section13. The processor device11controls operation of a color CCD (seeFIG. 2, hereinafter simply referred to as the CCD)33in the distal portion17. The processor device11receives an image signal outputted from the CCD33through the transmission cable. The processor device11performs various image processing steps to the image signal to produce image data. The image data is sent to a monitor18, cable-connected to the processor device11, and displayed as an observation image on a screen of the monitor18.

The electronic endoscope system2is provided with a normal observation mode and a vascular observation mode (narrowband light mode). In the normal observation mode, an internal body portion of the subject is observed under illumination with white light. In the vascular observation mode, the white light including specific narrowband light is applied to the internal body portion to calculate vascular information of a blood vessel included in the internal body portion. The vascular information is, for example, an oxygen saturation level of hemoglobin in the blood vessel. A mode switch19on the handling section14is used for switching between the modes. When turned on, the electronic endoscope system is automatically set to the normal observation mode by a command from the processor device11.

InFIG. 2, an imaging window30, a lighting window31, and the like are provided on a distal end surface of the distal portion17. Behind the imaging window30, an objective optical system32composed of a lens group and a prism is disposed. A CCD33is disposed behind the objective optical system32. A lighting lens35is attached to the lighting window31. The lighting lens35applies the illumination light to the internal body portion. The illumination light from the light source device12is transmitted to the lighting lens35through a light guide34routed through the universal cord16and the insert section13.

Reflection light from the internal body portion is incident on the CCD33through the imaging window30and the objective optical system32. The CCD33converts the reflection light photoelectrically into the image signal, and outputs the image signal. Red (R) pixels, green (G) pixels, and blue (B) pixels are arranged in a matrix on an imaging surface of the CCD33. Each pixel is composed of a color filter segment and a photodiode. In this embodiment, a three primary color filter36of a Bayer arrangement is used (seeFIG. 3).FIG. 4shows spectral sensitivity characteristics of each of the R, G, and B pixels of the CCD33, determined by spectral transmittance of the three primary color filter36and the spectral sensitivity of pixels themselves. The R pixel has a sensitivity peak at around 600 nm. The G pixel has a sensitivity peak at around 530 nm. The B pixel has a sensitivity peak at around 460 nm. Wavelength bands of the R, G, and B pixels overlap with each other. For example, both the B and G pixels are sensitive in a wavelength band of 450 nm to 530 nm.

An analog front end (AFE)37is composed of a correlated double sampling circuit (CDS), an automatic gain controller (AGC), and an analog/digital converter (A/D), as is well known. The CDS performs correlated double sampling to the image signal outputted from the CCD33, to remove reset noise and amplification noise occurred in the CCD33. Then the AGC amplifies the image signal with a gain specified by the processor device11. Thereafter, the A/D converts the image signal into a digital image signal of a predetermined bit number. The digital image signal is inputted to an image processor49of the processor device11through a transmission cable.

A CCD driver (timing generator)38generates drive pulses (vertical/horizontal scan pulses, electronic shutter pulse, read-out pulse, reset pulse, and the like) for the CCD33and a synchronization pulse for the AFE37. In response to the drive pulse from the CCD driver38, the CCD33carries out imaging operations to output the image signal. Each section of the AFE37operates in response to the synchronization pulse from the CCD driver38.

After the electronic endoscope10is connected to the processor device11, a CPU39actuates the CCD driver38in response to an operation start command from a CPU45of the processor device11. The CPU39adjusts the gain of the AGC in the AFE37.

The CPU45controls the operation of the whole processor device11. The CPU45is connected to each section through a data bus, an address bus, and control lines (all not shown). A ROM46stores various programs (OS, application programs, and the like) for controlling the operation of the processor device11, and data (graphic data, and the like). The CPU45reads out the necessary programs and the data from the ROM46and loads them into a RAM47being a working memory, and runs the programs in sequence. The CPU45obtains information, such as text data including examination date and time, a patient's name, and a doctor's name, on an examination-by-examination basis from an operation panel of the processor device11or through a network, for example, LAN (local Area Network), and writes the information to the RAM47.

An operation unit48is a well-known input device such as the operation panel provided on a housing of the processor device11, a mouse, or a keyboard. The CPU45operates each section in response to an operation signal from the operation unit48or from a release button or the mode switch19provided on the handling section14of the electronic endoscope10.

The image processor49performs various image processing steps such as color interpolation, white balance adjustment, gamma correction, image enhancement, image noise reduction, and color conversion to the image signal inputted from the electronic endoscope10. The image processor49calculates the vascular information (oxygen saturation level) which will be described later.

A display controller50receives the graphic data from the ROM46and the RAM47through the CPU45. The graphic data includes a display mask, text data, and a graphical user interface (GUI). The display mask covers an ineffective pixel area of the observation image to display only an effective pixel area. The text data includes the examination date and time, the patient's name, the doctor's name, and the current mode selected. The display controller50performs various display control processing steps to the image sent from the image processor49. The display control processing steps include superimposition of the display mask, the text data, and the GUI on the image, and a drawing process for displaying the image on the screen of the monitor18.

The display controller50has a frame memory (not shown) for temporarily storing the image from the image processor49. The display controller50reads out the image from the frame memory and then converts the image into a video signal (component signal, composite signal, or the like) conforming to a display format of the monitor18. Thereby, an observation image is displayed on the screen of the monitor18.

In addition, the processor device11is provided with a compression circuit, a media I/F, a network I/F, and the like (all not shown). The compression circuit compresses the image with a predetermined compression format (for example, a JPEG format). The media I/F writes the compressed image to a removable medium such as a CF card, a magneto-optical disk (MO), or a CD-R. The network I/F controls transmission of various types of data to and from the network such as the LAN. The compression circuit, the media I/F, the network I/F, and the like are connected to the CPU45through the data bus and the like.

The light source device12has a first semiconductor laser55and a second semiconductor laser56. Each of the first and second semiconductor lasers55and56is a semiconductor laser diode, for example, a broad area type InGaN laser diode, an InGaNAs laser diode, or a GaNAs laser diode. As shown inFIG. 5, the first semiconductor laser55emits blue first excitation light L1with a center wavelength of, for example, 445 nm. The first excitation light L1causes (excites) a wavelength converter64to emit fluorescence. A part of the first excitation light L1, being the narrowband light, passes through the wavelength converter64. The second semiconductor laser56emits second excitation light L2with a center wavelength in a wavelength range from blue to green, for example, 473 nm. The second excitation light L2causes (excites) the wavelength converter64to emit fluorescence, though less efficiently than the first excitation light L1. A part of the second excitation light L2, being the narrowband light, passes through the wavelength converter64, and is used for calculating the oxygen saturation level of hemoglobin in the blood vessel. A first light source system is composed of the first semiconductor laser55and the wavelength converter64. A second light source system is composed of the second semiconductor laser56and the wavelength converter64.

The first and second semiconductor lasers55and56are driven by light source drivers57and58, respectively. Condenser lenses59and60gather light from the first and second semiconductor lasers55and56to allow the light to be incident on light guides34aand34b, respectively. The light guides34aand34bare disposed on exit end sides of the first and second semiconductor lasers55and56, respectively. The light guides34aand34bare connected to a single light guide34through a coupler61. A variable aperture stop62is disposed between the condenser lens59and the light guide34a. A variable aperture stop63is disposed between the condenser lens60and the light guide34b. The variable aperture stops62and63control light quantities of the light incident on the light guides34aand34b, respectively. Instead of the coupler61, the first and second semiconductor lasers55and56may be provided with their respective light guides to transmit the light separately to the lighting window31.

The wavelength converter64is a glass plate on which several kinds of phosphor are applied or dispersed. The phosphor absorb a part of the blue first excitation light L1from the first semiconductor laser55and a part of the cyan second excitation light L2from the second semiconductor laser56, to emit fluorescence in a wavelength range from green to red. Note that the wavelength converter64emits a small quantity of blue light. However, the color of the fluorescence emitted from the phosphor is referred to as “green to red”. When the first semiconductor laser55is turned on, the fluorescence (“L3” inFIG. 5) in the wavelength range from green to red and the blue first excitation light L1left unabsorbed is mixed to generate the white light. The white light is applied to the internal body portion through the lighting window31. Examples of the phosphor include YAG fluorescent substances or BAM (BaMgAl10O17) fluorescent substances. The phosphor sold under the product name Micro White (or MW) (registered trademark) can be used.

As shown inFIG. 4, only the B pixel is sensitive to the reflection light of the first excitation light L1with the center wavelength of 445 nm. The B and G pixels are sensitive to the reflection light of the second excitation light L2with the center wavelength of 473 nm. Because the fluorescence L3is the light in a broad range of approximately 450 nm to 700 nm, all of the R, G, and B pixels are sensitive to the fluorescence L3. Note that the output of the B pixel is small.

A CPU66of the light source device12communicates with the CPU45of the processor device11. The CPU66separately controls ON/OFF of the first semiconductor laser55through the light source driver57and that of the second semiconductor laser55through the light source driver58. The CPU66separately controls the light quantity of the first semiconductor laser55through the variable aperture stop62and that of the second semiconductor laser56through the variable aperture stop63.

When the normal observation mode is selected, the CPU45controls the light source driver57through the CPU66to turn on only the first semiconductor laser55. Namely, the illumination light applied to the internal body portion is the white light, being a mixture of the fluorescence L3, emitted from the wavelength converter64excited by the first excitation light L1with the center wavelength of 445 nm from the first semiconductor laser55, and the first excitation light L1passed through the wavelength converter64.

When the vascular observation mode is selected, the CPU45controls the light source drivers57and58through the CPU66so as to allow one of the lasers55and56to emit light in a full light state, at 100% rated output (light quantity), and the other to emit light in a reduced light state, for example, at 10% rated output, as shown inFIG. 6. Then, each of the lasers55and56switches between the full and reduced light states on a unit by unit basis of accumulation and readout periods. In other words, in the vascular observation mode, both the lasers55and56are kept turned on without being turned off. This prevents overshoot because the overshoot occurs due to turning on the light source after turning it off. Note that each or one of the lasers55and56may emit light at 100% rated output only during the accumulation periods. During the readout periods between the accumulation periods, each or one of the lasers55and56may emit light at 10% rated output.

As shown inFIGS. 7A and 7B, during the above-described emission control of the lasers55and56, the first and second emission patterns are repeated alternately. The first and second emission patterns are different from each other in emission intensity. As shown inFIG. 7A, in the first emission pattern, the white light, being the mixture of the first excitation light L1at 100% rated output, the second excitation light “ 1/10(L2)” at 10% rated output, and the fluorescence L3, is applied to the internal body portion. The fluorescence L3refers to the light emitted from the wavelength converter64excited by the first excitation light L1and the second excitation light 1/10(L2). As shown inFIG. 7B, in the second emission pattern, the substantially white light, being the mixture of the first excitation light “ 1/10(L1)” at 10% rated output, the second excitation light L2at 100% rated output, and the fluorescence “ 9/10(L3)”, is applied to the internal body portion. The fluorescence “ 9/10(L3)” refers to the light emitted from the wavelength converter64excited by the first excitation light 1/10(L1) and the second excitation light L2. The wavelength converter64absorbs the first excitation light L1at higher absorptivity than the second excitation light L2. Accordingly, “ 9/10(L3)” represents that the emission intensity of the fluorescence L3in the second emission pattern decreases by 10% compared to that in the first emission pattern.

InFIG. 8, the image processor49is provided with a vascular area determination section71, a vascular information calculation section70, and a vascular information image production section73. The vascular area determination section71analyzes an image inputted from the AFE37. For example, the vascular area determination section71obtains or refers to a difference in luminance value between a vascular area and a non-vascular area to determine (extract) the vascular area in (from) the image. The vascular area determination section71outputs information of the vascular area extracted, together with an image, to the vascular information calculation section70. The vascular information calculation section70calculates the vascular information, for example, the oxygen saturation level of hemoglobin in a blood vessel. The vascular information calculation section70calculates the oxygen saturation level based on two frames of images P1and P2(seeFIG. 6) of the internal body portion captured successively in the vascular observation mode. The image P1is captured under illumination light of the first emission pattern. The image P2is captured under illumination light of the second emission pattern. The illumination light of the first emission pattern and that of the second emission pattern is applied to the internal body portion alternately. Additionally, a blood flow rate in the blood vessel can be measured. Furthermore, a position (depth) of the blood vessel can be located or determined, and information on the blood vessel located can be obtained.

As shown inFIG. 9, an absorption coefficient pa of the hemoglobin varies with a wavelength of the illumination light. The absorption coefficient μa represents absorbance or magnitude of light absorbed by the hemoglobin. The absorption coefficient is a coefficient of an expression I0exp(−μa×x) representing attenuation of the light applied to the hemoglobin. Note that “I0” denotes intensity of the illumination light, and “x” (unit: cm) denotes the depth of the blood vessel from the surface of the internal body portion.

Deoxyhemoglobin Hb not combined with oxygen differs from oxyhemoglobin HbO combined with the oxygen in light absorption properties. An absorption coefficient pa of the deoxyhemoglobin is different from that of the oxyhemoglobin except at isosbestic points. The isosbestic point is a point of intersection of the absorption coefficients pa of the deoxyhemoglobin and oxyhemoglobin, at which the absorption coefficients pa of the deoxyhemoglobin and the oxyhemoglobin have the same value.

When there is a difference in absorption coefficient pa between the deoxyhemoglobin and the oxyhemoglobin, intensities of the reflection light from the blood vessel vary even if light of a constant wavelength and constant intensity is applied to the blood vessel. When light of different wavelengths and constant intensity is applied to the blood vessel, the intensities of the reflection light still vary because the absorption coefficient pa varies with the wavelength. Accordingly, a ratio between the oxyhemoglobin and the deoxyhemoglobin in the blood vessel, that is, the information of oxygen saturation level is obtained or determined by analyzing images captured under the illumination including two or more wavelength bands of the narrowband light.

The vascular information calculation section70has a frame memory (not shown) for temporarily storing the image P1captured under the illumination light of the first emission pattern and the image P2captured under the illumination of the second emission pattern in the vascular observation mode. The vascular information calculation section70reads out each of the images P1and P2from the frame memory. The vascular information calculation section70uses the pixel values of the vascular area, determined by the vascular area determination section71, of each of the images P1and P2to carry out various calculations. For example, the vascular information calculation section70calculates a ratio or a difference in pixel values between the images P1and P2to obtain an image parameter.

By way of example, calculation of the oxygen saturation level using the first image P1captured under the illumination light of the first emission pattern and the second image P2captured under the illumination light of the second emission pattern is described.

The R, G, and B pixel values “r1”, “g1”, and “b1” of the first image P1are obtained from the reflection light of the illumination light L1to L3. Referring to the spectral transmittance of each of the R, G, and B pixels of the CCD33shown inFIG. 4and the spectral intensity characteristic of each of the illumination light L1to L3shown inFIGS. 7A and 7B, each of the R pixel value r1, the G pixel value g1, and the B pixel value b1of the first image P1is expressed using at least one of the illumination light L1to L3, as shown in expressions (1) to (3) below.
r1=L3  (1)
g1= 1/10(L2)+L3  (2)
b1=L1+ 1/10(L2)+L3  (3)
The second excitation light L2(center wavelength: 473 nm) of the first emission pattern is at 10% rated output, so the L2is multiplied by 1/10. Similarly, the R pixel value r2, the G pixel value g2, and the B pixel value b2of the second image P2are expressed using at least one of the illumination light L1to L3, as shown in expressions (4) to (6) below.
r2= 9/10(L3)  (4)
g2=L2+ 9/10(L3)  (5)
b2= 1/10(L1)+L2+ 9/10(L3)  (6)

The vascular information calculation section70calculates a ratio “b2/g1” (a ratio between the B pixel value b2of the second image P2and the G pixel value g1of the first image P1), relative to the corresponding pixels in the first and second images P1and P2, and a ratio “r1/g1” (a ratio between the R pixel value r1and the G pixel value g1of the first image P1) as the image parameters.

Conventionally, to calculate the oxygen saturation level of the hemoglobin in the blood vessel, two frames of images are obtained successively with one of the lasers55and56turned on and the other turned off alternately. Namely, one of the lasers55and56is used in each unit of the accumulation and readout periods. In this embodiment, to prevent the overshoot due to turning on and off the first and second semiconductor lasers55and56, the first and second images P1and P2are obtained without turning off the lasers55and56. In other words, a component, of the pixel value of each of the R, G, and B pixels in each of the images P1and P2, corresponding to the light of 10% rated output is a noise component. The vascular information calculation section70performs correlation operation of the pixel values of the corresponding pixels in the images P1and P2to remove the noise component. Then, the vascular information calculation section70calculates an image parameter.

As described above, the B pixel value b2of the second image P2includes 1/10(L1) being the noise component. In calculating the oxygen saturation level, the component 9/10(L3) of the fluorescence L3is also regarded as the noise component. First, to remove the noise component 1/10(L1), the G pixel value g1of the first image P1is subtracted from the B pixel value b1of the first image P1, and then the difference is multiplied by 1/10. An expression (7) is obtained from the expressions (2) and (3).
1/10(b1−g1)= 1/10[L1+ 1/10(L2)+L3−{ 1/10(L2)+L3}]= 1/10(L1)  (7)
To remove the noise component 9/10(L3), the R pixel value r2of the second image P2is used. From the expressions (4), (6), and (7), a corrected B pixel value b2′, with the noise component removed, of the second image P2is obtained with an expression (8). Thus, a component corresponding to the second excitation light L2with the center wavelength of 473 nm is extracted.
b2′=b2− 1/10(b1−g1)−r2= 1/10(L1)+L2+ 9/10(L3)− 1/10(L1)− 9/10(L3)=L2  (8)

The G pixel value g1of the first image P1includes 1/10(L2) being a noise component. To remove the noise component 1/10 (L2), the R pixel value r2of the second image P2is subtracted from the G pixel value g2of the second image P2, and then the difference is multiplied by 1/10. An expression (9) is obtained from the expressions (4) and (5).
1/10(g2−r2)= 1/10{L2+ 9/10(L3)− 9/10(L3)}= 1/10(L2)  (9)

From the expressions (2) and (9), a corrected G pixel value g1′, with the noise component removed, of the first image P1is obtained with an expression (10).
g1′=g1− 1/10(g2−r2)= 1/10(L2)+L3− 1/10(L2)=L3  (10)
Thus, a component corresponding to the fluorescence L3is extracted using the expression (10). The R pixel value r1of the first image P1does not include a noise component, so the correlation operation is not performed. The vascular information calculation section70uses the pixel value r1and the corrected pixel values b2′ and g1′ to calculate the image parameters (b2′/g1′ and r1/g1′).

Note that before calculating the difference between the R, G, and B pixel values as shown in the expressions (7) to (10), each of the R, G, and B pixel values is multiplied by a predetermined correlation coefficient. The correlation coefficient is determined in advance based on the spectral transmittance of each of the R, G, and B pixels of the CCD33inFIG. 4, and the spectral intensity characteristics of each of the illumination light L1to L3inFIG. 5. The correlation coefficient is set such that the noise component is removed by calculating the difference between the R, G, and B pixel values. For example, when a spectral sensitivity ratio between the G pixel value and the R pixel value is 1.25 (G/R=1.25) relative to the fluorescence L3, the R pixel value r2is multiplied by 1.25.

As shown inFIG. 10, reference data72shows relation between the image parameter and the oxygen saturation level in a form of a function or a data table. The relation between the image parameter and the oxygen saturation level is determined in advance by an experiment or the like. A signal ratio b2′/g1′ increases as a signal ratio r1/g1′ increases, namely, a contour line “oxygen saturation level=0% limit” extends or slides up diagonally. This is because there is a correlation between the signal ratio r1/g1′ and blood volume. The blood volume increases as the signal ratio r1/g1′ increases. Of the signals b2′, g1′, and r1, the increase in the blood volume most decreases the signal value of the green signal g1′. An amount of the signal value of the blue signal b2′ decreased by the increase in the blood volume is second to that of the green signal g1′.

This is because the absorption coefficient of a wavelength component of 540 nm to 580 nm included in the G signal g1′ is higher than that of a wavelength component of approximately 470 nm included in the B signal b2′ (seeFIG. 9). Accordingly, in the signal ratio b2′/g1′, the decrease in the signal value g1′ (denominator) is greater than the decrease in the signal value b2′ (nominator), as the blood volume increases. In other words, the signal ratio b2′/g1′ increases as the blood volume increases.

The vascular information calculation section70substitutes the image parameter into the function to calculate the oxygen saturation level corresponding to the image parameter, or retrieves the oxygen saturation level corresponding to the image parameter from the data table. The oxygen saturation level calculated or obtained (referred to as the calculation result) is outputted to the vascular information image production section73.

Based on a color map for displaying the calculation result in pseudo color, the vascular information image production section73produces a vascular information image (in this embodiment, an oxygen saturation image) reflecting or representing the calculation result of the vascular information calculation section70. The oxygen saturation image is an image of the oxygen saturation level obtained by the vascular information calculation section70with the use of the reference data72. To produce a pseudo color image, the color map assigns, for example, cyan to an area with a relatively low oxygen saturation level, magenta to an area with a medium oxygen saturation level, and yellow to an area with a high oxygen saturation level in a vascular image.

Next, an operation of the electronic endoscope system2of the above-described configuration is described. Patient information is inputted and the start of the examination is commanded using the operation unit48. Then the insert section13of the electronic endoscope10is inserted into the subject (patient's body). While being illuminated with the illumination light from the light source device12, an observation image of the internal body portion is captured with the CCD33. The observation image is displayed on the monitor18, and observed.

To be more specific, the image signal outputted from the CCD33is subjected to various processing steps in each section of the AFE37. Then, the image signal is inputted to the image processor49. The image processor49performs various image processing steps to the image signal to produce the image of the internal body portion. The image is inputted to the display controller50. The display controller50performs various display control processing steps in accordance with the graphic data. Thereby, the observation image is displayed on the monitor18.

When the insert section13of the electronic endoscope10is inserted into the subject, a normal observation mode is selected to illuminate the internal body portion with the white light. Thereby, a wide view is ensured while the insert section13is inserted. When a lesion requiring careful observation is found and it is necessary to obtain its oxygen saturation level, the vascular observation mode is selected. When necessary, a still image of the lesion is captured with the operation of a release button provided on the electronic endoscope10. When a treatment is needed, a medical instrument is inserted into the forceps channel of the electric endoscope10to remove the lesion or to give medicine to the lesion.

In the normal observation mode, in response to the command of the CPU45, the CPU66turns on only the first semiconductor laser55to apply the white light, being the mixture of the first excitation light L1and the fluorescence L3, to the internal body portion through the lighting window31.

On the other hand, as shown in S10ofFIG. 11, when the vascular observation mode is selected using the mode switch19, the CPU66puts one of the lasers55and56into 100% rated output (full light state), and the other into 10% rated output (reduced light state), for example. Each of the lasers55and56switches between the full and reduced light states alternately and repeatedly on a unit by unit basis of the accumulation and readout periods. Namely, one of the lasers55and56is put into the reduced light state while the other is put into the full light state for the unit of the accumulation and readout periods. Thereby, the illumination of the first emission pattern and that of the second emission pattern is applied alternately to the internal body portion (seeFIGS. 7A and 7B). The CCD33outputs the first image P1captured under the illumination of the first emission pattern, and the second image P2captured under the illumination of the second emission pattern (S11).

In the image processor49, first, the vascular area determination section71determines the vascular area. Then, the vascular information calculation section70performs the correlation operation to remove the noise components from the respective R, G, B pixel values in each of the vascular areas in the first and second images P1and P2. Thereby, reflection light component(s) of the specific illumination light is extracted (S12). Thereafter, the image parameter is calculated using the corrected pixel values with the noise component removed by the correlation operation (S13). Based on the reference data72, the oxygen saturation level of the hemoglobin in the blood vessel is calculated (S14). Based on the calculation result of the oxygen saturation level, the vascular information image production section73produces the oxygen saturation image in which the blood vessels are color-coded according to their oxygen saturation levels, and the oxygen saturation image is displayed on the monitor18(S15). The above-described steps are repeated until the normal observation mode is selected using the mode switch19(YES in S16).

As described above, in the present invention, when the two types of illumination light each including the specific narrowband light are applied alternately from the respective first and second semiconductor lasers55and56, each of the lasers55and56is switched alternately between the full and reduced light states without being turned off. The two images P1and P2are obtained under the illumination of both the semiconductor lasers55and56. To extract a necessary component, the noise component is removed using the correlation operation of the R, G, and B pixel values of each of the images P1and P2. Thereby, the overshoot of each of the outputs of the lasers55and56due to turning on and off is prevented. Problems resulting from the overshoot are also prevented.

In particular, when the light quantity of the illumination light is unstable due to the overshoot, accuracy of the image parameter calculated using the pixel values of the images P1and P2cannot be maintained. This reduces reliability of the calculation result of the oxygen saturation level. The present invention, on the other hand, prevents the overshoot and makes the light quantity stable, which improves the reliability of the calculation result of the oxygen saturation level.

The B and G pixels of the CCD33are sensitive to the reflection light of the second excitation light L2with the center wavelength of 473 nm. All of the R, G, and B pixels are sensitive to the reflection light of the fluorescence L3. Namely, each of the B and G pixel values includes the component corresponding to the second excitation light L2. Each of the R, G, and B pixel values includes the component corresponding to the fluorescence L3. This enables removal of the noise components with the use of the correlation operation and extraction of only the specific components. The correlation operation can be performed when a wavelength band of one of the illumination light overlaps with the sensitive areas of two of the R, G, and B pixels, because the spectral sensitivity characteristics of each pixel is well-known.

In the above embodiment, by way of example, the corrected G pixel value g1′ of the first image P1and the corrected B pixel value b2′ of the second image P2are used for the correlation operation. Alternatively, as shown in an expression (11) below, the R pixel value r2of the second image is subtracted from the G pixel value g2of the second image P2to extract the component corresponding to the second excitation light L2with the center wavelength of 473 nm.
g2′=g2−r2=L2+ 9/10(L3)− 9/10(L3)=L2  (11)

Alternatively, as shown in an expression (12), the G pixel value g1of the first image P1is subtracted from the B pixel value b1of the first image P1to extract the component corresponding to the first excitation light L1with the center wavelength of 445 nm.
b1′=b1−g1=L1+ 1/10(L2)+L3−{ 1/10(L2)+L3}=L1  (12)

The vascular information is not limited to the oxygen saturation level of the above embodiment. The vascular information may be the blood volume in a blood vessel. The blood volume can be expressed based on a ratio r1/g1′. The vascular information may be vascular images (visible images of blood flow in blood vessels) of the superficial and middle mucosal layers. The vascular images are produced using the b2′, the b1′ obtained with the expression (12), and the g2′ obtained with the expression (11) and displayed.

In the above embodiment, the CCD33with the three primary color filter (R, G, and B)36of an additive color system is used. Instead, a complementary color CCD with a three primary color filter (C, M, and Y)100of a subtractive color system shown inFIG. 12may be used.

As shown inFIG. 13, a C (cyan) color filter segment has high transmittance in the blue and green wavelength bands, while having extremely low transmittance in the red wavelength band. An M (magenta) color filter segment has high transmittance in the blue and red wavelength bands, while having extremely low transmittance in the green wavelength band. A Y (yellow) color filter segment has high transmittance in a wavelength band of 450 nm or more, while having extremely low transmittance in a wavelength band less than 450 nm.

Referring to the first and second emission patterns shown inFIGS. 7A and 7Bin the vascular observation mode, and the transmission characteristics of the C, M, and Y color filter segments shown inFIG. 13, a C pixel provided with the C color filter segment is sensitive to the first excitation light L1, the second excitation light L2, and a green component of the fluorescence L3. An M pixel provided with the M color filter segment is sensitive to the first excitation light L1, the second excitation light L2, and a red component of the fluorescence L3. A Y pixel provided with the Y color filter segment is sensitive to the second excitation light L2and the green and red components of the fluorescence L3.

Accordingly, the pixel values C1, M1, and Y1of the first image P1and the pixel values C2, M2, and Y2of the second image P2, outputted from the complementary color CCD, are expressed as follows.
C1=L1+ 1/10(L2)+α(L3)  (X1)
M1=L1+ 1/10(L2)+β(L3)  (X2)
Y1= 1/10(L2)+L3  (X3)
C2= 1/10(L1)+L2+α× 9/10(L3)  (X4)
M2= 1/10(L1)+L2+β× 9/10(L3)  (X5)
Y2=L2+ 9/10(L3)  (X6)
In the expressions, the α(<1) represents a percentage of a component (mostly the green component), of the fluorescence L3, passing the C color filter segment. The β(<1) represents a percentage of a component (mostly the red component), of the fluorescence L3, passing the M color filter segment.

When the complementary color CCD is used, the pixel values M1and Y1of the first image P1and the pixel value C2of the second image P2are converted into R, G, and B values. The oxygen saturation level is calculated based on the converted R, G, and B values. Note that, before the conversion into the R, G, and B values, a noise component, being the component of the light of the 10% rated output for preventing the overshoot, is removed from the pixel values M1, Y1, and C2.

The pixel value C2of the second image P2includes the noise component 1/10(L1). In the calculation of the oxygen saturation level, “α× 9/10(L3)”, being the component of the fluorescence L3, is also regarded as the noise component. As shown below, by removing the noise components 1/10(L1) and α× 9/10(L3) from the pixel value C2, a corrected pixel value C2′ having only the component corresponding to the second excitation light L2is obtained.
C2′=C2− 1/10(L1)−α× 9/10(L3)=L2

Note that, for the L3, a value obtained from the expressions (X3) and (X6) is used as shown below.
L3=90/91(Y1)  (X7)
For the L1, a value obtained from the expressions (X1), (X4), and (X7) is used as shown below.
L1=100/99{(C1− 1/10(C2))− 9/10(Y1)}

On the other hand, the pixel value M1of the first image P1includes the noise component 1/10(L2), so the noise component 1/10(L2) is removed from the pixel value M1. Thereby, a corrected pixel value M1′, having the component corresponding to the first excitation light L1and the component corresponding to the fluorescence β(L3), is obtained.
M1′=M1− 1/10(L2)=L1+β(L3)
The pixel value Y1of the first image P1includes the noise component 1/10(L2), so the noise component 1/10(L2) is removed from the pixel value Y1. Thereby, a corrected pixel value Y1′, having only the component corresponding to the fluorescence L3, is obtained.
Y1′=Y1− 1/10(L2)=L3

Note that, for the L2, a value obtained from the expressions (X1), (X4), and (X7) is used as shown below.
L2=100/99{(C2− 1/10(C1))− 72/91(Y1)}

The corrected pixel values C2′, M1′, and Y1′ are converted into B, G, and R values of the additive color system. A well-known method is used for converting the CMY of the subtractive color system into the R, G, and B values of the additive color system. The converted B value corresponds to the corrected pixel value b2′ of the above embodiment. The converted G value corresponds to the corrected pixel value g1′ of the above embodiment. The converted R value corresponds to the pixel value r1of the above embodiment. The oxygen saturation level is calculated based on the R, G, and B values in a manner similar to the above embodiment. Note that it is preferable to check whether the relation between the emission intensity and the converted pixel values is appropriate.

In the above embodiment, the pixel values C2, M1, and Y1are used for calculating the oxygen saturation level. Alternatively, the oxygen saturation level may be calculated using other pixel values, for example, C1, M2, and Y2. In this case, the pixel value Y2does not have a noise component. Accordingly, a noise component is removed from the pixel values C1and M2to obtain the corrected pixel values C1′ and M2′. The corrected pixel values C1′ and M2′ and the pixel value Y2are converted into the R, G, and B values. The oxygen saturation level is calculated based on the converted R, G, and B values. Note that the noise component in the pixel value C1is 1/10(L2). The noise component in the pixel value M2is 1/10(L1) and β× 9/10(L3).

In the above embodiments, the two semiconductor light sources are used by way of example. At least one semiconductor light source is used. In this case, in the vascular observation mode, light source(s) other than the semiconductor light source is turned on and off repeatedly, while the semiconductor light source is switched alternately between the full and reduced light states. The light may not be reduced for the unit of the accumulation and readout periods of the CCD33. For example, the light may be in full or reduced light state in two consecutive frames. Furthermore, the light may be reduced at irregular intervals. For example, one of the light is reduced in one frame and then the other light may be reduced in the consecutive two frames. In a word, the semiconductor light source is kept turned on without being turned off. When the noise component corresponding to the reduced illumination light is negligible, the reduction of the noise component may be omitted.

Two or more types of narrowband light, for example, four types of narrowband light with the center wavelengths of, for example, 405 nm, 450 nm, 550 nm, and 780 nm may be applied to the internal body portion to obtain the vascular images of the mucosal superficial, middle, and deep layers. A fluorescent substance may be ejected into living tissue and excitation light may be applied to the living tissue to observe fluorescence from the living tissue, or intrinsic fluorescence of the living tissue may be observed. Alternatively, the wavelength converter64may be disposed in front of the first semiconductor laser55. The wavelength converter64is inserted into a light path of the first semiconductor laser55in the normal observation mode.

In the above embodiments, the electronic endoscope is described by way of example. Instead, an endoscope of a different type, for example, an ultrasonic endoscope incorporating an imaging device and an ultrasonic transducer at its tip may be used.

In the present invention, an image of the oxygen saturation level is produced. Alternatively or in addition, an image of information on, for example, oxyhemoglobin index or deoxyhemoglobin index may be produced. The oxyhemoglobin index is obtained from “blood volume (sum of oxyhemoglobin and deoxyhemoglobin)×oxygen saturation level (%)”. The deoxyhemoglobin index is obtained from “blood volume×(100-oxygen saturation level) (%)”.

Various changes and modifications are possible in the present invention and may be understood to be within the present invention.