Image processing device, image processing method, and computer readable storage device

An image processing device includes: a candidate abnormal region determining unit that determines a candidate abnormal region from an image using a first determination criterion; a bubble region determining unit that determines a bubble region from the image; a bubble inside determining unit that determines whether the candidate abnormal region is present inside the bubble region based on a determination result of the bubble region; and an abnormal region determining unit that determines whether the candidate abnormal region is an abnormal region using a second determination criterion different from the first determination criterion when the candidate abnormal region is determined to be present inside the bubble region.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority from Japanese Patent Application No. 2011-229229, filed on Oct. 18, 2011, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing device, an image processing method, and a computer readable storage device for determining an abnormal region from an image obtained by imaging a lumen of a living body.

2. Description of the Related Art

In the related art, as image processing on images (hereinafter referred to as intra-luminal images or simply as images) obtained by imaging the lumen of a living body using a medical observation device such as an endoscope or a capsule endoscope, a technique of detecting an abnormal region based on hue information of the image is disclosed in Japanese Laid-open Patent Publication No. 2005-192880. More specifically, in Japanese Laid-open Patent Publication No. 2005-192880, the pixel value of each pixel or a mean pixel value thereof is mapped onto a feature space based on the color feature data, a normal mucosal cluster and an abnormal cluster are specified after performing clustering in the feature space, and a pixel region belonging to the abnormal cluster is detected as an abnormal region.

SUMMARY OF THE INVENTION

An image processing device according to an aspect of the invention includes: a candidate abnormal region determining unit that determines a candidate abnormal region from an image using a first determination criterion; a bubble region determining unit that determines a bubble region from the image; a bubble inside determining unit that determines whether the candidate abnormal region is present inside the bubble region based on a determination result of the bubble region; and an abnormal region determining unit that determines whether the candidate abnormal region is an abnormal region using a second determination criterion different from the first determination criterion when the candidate abnormal region is determined to be present inside the bubble region.

An image processing method according to another aspect of the invention includes: determining a candidate abnormal region from an image using a first determination criterion; determining a bubble region from the image; determining whether the candidate abnormal region is present inside the bubble region based on the determination result of the bubble region; and determining whether the candidate abnormal region is an abnormal region using a second determination criterion different from the first determination criterion when the candidate abnormal region is determined to be present inside the bubble region.

A computer readable storage device according to still another aspect of the invention has an executable program stored thereon, wherein the program instructs a processor to perform: determining a candidate abnormal region from an image using a first determination criterion; determining a bubble region from the image; determining whether the candidate abnormal region is present inside the bubble region based on the determination result of the bubble region; and determining whether the candidate abnormal region is an abnormal region using a second determination criterion different from the first determination criterion when the candidate abnormal region is determined to be present inside the bubble region.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter an image processing device, an image processing method, and an image processing program according to embodiments of the invention will be explained with reference to the drawings. The invention is not limited to these embodiments. In the respective drawings, the same portions are denoted by the same reference numerals.

In the following embodiments, as an example, a process on a series of intra-luminal images (hereinafter simply referred to as images) obtained by imaging the lumen of a subject in a time-sequential order using a medical observation device such as an endoscope or a capsule endoscope will be explained. In the following description, the images subjected to image processing are color images which have 256 pixel levels (pixel values) for each of the respective color components of R (red), G (green), and B (blue) at the respective pixel positions, for example. The invention is not limited to intra-luminal images but can be widely applied to a case of extracting a specific region from images acquired using other general image acquisition devices.

First Embodiment

FIG. 1is a block diagram illustrating the configuration of an image processing device according to a first embodiment of the invention. An image processing device1illustrated inFIG. 1includes a control unit10that controls the operation of the entire image processing device1, an image acquiring unit20that acquires image data corresponding to an image captured by a medical observation device, an input unit30that receives input signals input from an external device, a display unit40that displays various screens, a storage unit50that stores the image data acquired by the image acquiring unit20and various programs, and a arithmetic unit100that executes predetermined image processing on the image data.

The control unit10is implemented by hardware such as a CPU. The control unit10reads various programs stored in the storage unit50to thereby transmit instructions and data to the respective units of the image processing device1in accordance with the image data input from the image acquiring unit20or an operation signal input from the input unit30and control the operation of the entire image processing device1in an integrated manner.

The image acquiring unit20is appropriately configured according to an aspect of a system that includes the medical observation device. For example, when the medical observation device is a capsule endoscope, and a portable recording medium is used in exchanging of image data with the medical observation device, the image acquiring unit20is configured as a reader device that detachably attaches the recording medium and reads image data of the intra-luminal images stored in the recording medium. Moreover, when a server that stores image data of the intra-luminal images captured by the medical observation device is provided, the image acquiring unit20is configured as a communication device that is connected to the server and performs data communication with the server to acquire the image data of the intra-luminal images. Alternatively, the image acquiring unit20may be configured as an interface device or the like that receives image signals from the medical observation device via a cable.

The input unit30is implemented as an input device such as, for example, a keyboard, a mouse, a touch panel, or various switches, and outputs received input signals to the control unit10.

The display unit40is implemented as a display device such as an LCD or an EL display, and displays various screens including intra-luminal images under control of the control unit10.

The storage unit50is implemented as various IC memories called a ROM or a RAM such as a rewritable flash memory, an internal hard disk or a hard disk connected to a data communication terminal, or an information storage medium such as a CD-ROM, and a reading device thereof. The storage unit50stores a program for causing the image processing device1to operate and causing the image processing device1to execute various functions, and data or the like used during the execution of the program in addition to the image data of the intra-luminal images acquired by the image acquiring unit20. Specifically, the storage unit50stores an image processing program51for executing a process of determining a candidate abnormal region from an image using a first predetermined criterion, determining a bubble region, and determining whether the candidate abnormal region is an abnormal region using a second criterion when it is determined that the candidate abnormal region is present within the bubble region. The storage unit50also stores various determination criteria used during the execution of the image processing program51.

The arithmetic unit100is implemented by hardware such as a CPU. The arithmetic unit100performs image processing on the image data corresponding to the intra-luminal image by reading the image processing program51and performs various arithmetic processes for determining an abnormal region from an intra-luminal image.

Next, a detailed configuration of the arithmetic unit100will be explained.

As illustrated inFIG. 1, the arithmetic unit100includes a candidate abnormal region determining unit110that determines a candidate abnormal region from an image using a first determination criterion, a bubble region determining unit120that determines a bubble region from the image, a bubble inside determining unit130that determines whether the candidate abnormal region is present inside the bubble region based on the determination result of the bubble region, and an abnormal region determining unit140that determines whether the candidate abnormal region is an abnormal region using a second determination criterion different from the first determination criterion when the candidate abnormal region is determined to be present inside the bubble region.

The bubble region determining unit120includes a peripheral region determining unit121that determines a region having the features of the peripheral region of the bubble region and an inner region determining unit122that determines a region having the features of the inner region of the bubble region. The peripheral region determining unit121includes an arc-shaped region determining unit121athat determines an arc-shaped region which is the feature of the peripheral region of the bubble region. Moreover, the inner region determining unit122includes a halation region determining unit122athat determines a halation region which is the feature of the inner region of the bubble region.

The bubble inside determining unit130includes a surrounding region determining unit131that determines a surrounding region of the candidate abnormal region and a surrounding region feature data calculating unit132that calculates the feature data of the surrounding region based on the determination result of the bubble region in the surrounding region and a region in the vicinity of the surrounding region. The surrounding region feature data calculating unit132includes an area calculating unit132athat calculates the area of the bubble region in the surrounding region and the region in the vicinity of the surrounding region. More specifically, the area calculating unit132aincludes a contour extracting unit132a′ that extracts contour pixels of the surrounding region.

The abnormal region determining unit140includes a determination criterion switching unit141that switches the determination Criterion based on whether the surrounding region of the candidate abnormal region is present inside the bubble region.

Next, the operation of the image processing device1will be explained.FIG. 2is a flowchart illustrating the operation of the image processing device1.

First, in step S01, the image acquiring unit20acquires a series of intra-luminal images obtained by imaging the lumen of a subject and stores the intra-luminal images in the storage unit50. The arithmetic unit100sequentially reads image data corresponding to a processing target image from the storage unit50.FIG. 3is a schematic view illustrating a part of the processing target image read by the arithmetic unit100as an example.

Subsequently, in step S02, the candidate abnormal region determining unit110determines a candidate abnormal region based on the color feature data of the image. More specifically, the candidate abnormal region determining unit110calculates a G/R value from the pixel values of the respective pixels that constitute an image and determines a region in which the G/R value is smaller than a predetermined determination criterion value as a candidate abnormal region A10. That is, a region in which red colors are relatively strong (a region in which bleeding or reddening is suspicious) is extracted from the image as the candidate abnormal region A10.

In step S03, the peripheral region determining unit121(the arc-shaped region determining unit121a) determines an arc-shaped region in the image. Although various methods can be used as a method of determining the arc-shaped region, in the first embodiment, a method disclosed in Japanese Laid-open Patent Publication No. 2007-313119 is used.

FIG. 4is a flowchart illustrating a detailed process of step S03. First, in step S101, the arc-shaped region determining unit121acalculates a gradient intensity (G value) in the image. Subsequently, in step S102, a correlation value between the gradient intensity (G value) and an arc-shaped model created in advance is calculated. Further, in step S103, a region in which the correlation value between the gradient intensity and the arc-shaped model is equal to or greater than a predetermined threshold value is determined as an arc-shaped region A11(seeFIG. 3). After that, the processing returns to a main routine.

In step S04, the inner region determining unit122(the halation region determining unit122a) determines a halation region in the image. Specifically, a luminance value Y is calculated from the pixel values (RGB values) of the respective pixels using the following equation (1) (see Digital Image Processing, CG-ARTS Society, p. 299). Moreover, a region in which the luminance value Y is equal to or greater than a predetermined threshold value is determined as a halation region A12(seeFIG. 3).
Y=0.3×R+0.59×G+0.11×B(1)

In step S05, the bubble region determining unit120sets the arc-shaped region A11determined by the peripheral region determining unit121and the halation region A12determined by the inner region determining unit122as the bubble region. In many cases, the bubble region includes both the arc-shaped region A11and the halation region A12.

In step S06, the surrounding region determining unit131determines the surrounding region of the candidate abnormal region A10. More specifically, first, the surrounding region determining unit131calculates the gravity center position G of the candidate abnormal region A10in the image as illustrated inFIG. 5. Moreover, a region included in a circle of which the origin is located at the gravity center position G and which has a radius of r1is extracted as a surrounding region A13. The value of the radius r1may be a predetermined value or may be determined adaptively based on the area of the candidate abnormal region A10.

In step S07, the contour extracting unit132a′ executes a contour tracking process (see Digital Image Processing, CG-ARTS Society, p. 178) to extract a contour (outline) region A14of the surrounding region A13.

In step S08, the area calculating unit132acalculates the area of the bubble region in the surrounding region A13and a region in the vicinity of the surrounding region A13. For example, in the case ofFIG. 3, the bubble region (the halation region) A12is present in the surrounding region A13, and the bubble region (the arc-shaped region) A11is present so as to overlap the contour region A14of the surrounding region A13. The area calculating unit132acalculates the total areas of the bubble regions A11and A12.

FIG. 6is a flowchart illustrating a detailed process of step S08. In step S111, first, the area calculating unit132acalculates the total area S1of the bubble regions A11and A12in the surrounding region A13and the region in the vicinity of the surrounding region A13. Subsequently, in step S112, the area S2of a contour region (outline) A14of the surrounding region A13is calculated. Further, in step S113, the area S1of the bubble region is normalized by the area S2of the contour region A14of the surrounding region using the following equation (2) to calculate a normalized bubble region area S.
S=S1/S2(2)

In the above equation (2), the area S1may be divided by the area of the surrounding region A13instead of the area S2of the contour region A14, or the area S1may be divided by the sum of the areas of the surrounding region A13and the contour region A14.

After that, the processing returns to the main routine.

In step S09, the surrounding region feature data calculating unit132uses the normalized bubble region area S as the feature data of the surrounding region A13. Incidentally, the value itself of the area S1of the bubble regions A11and A12or the determination result (1 for present and 0 for absent) on the presence of a bubble region in the surrounding region A13and a region in the vicinity of the surrounding region A13may be used as the feature data of the surrounding region A13.

In step S10, the bubble inside determining unit130determines whether the normalized bubble region area S in the surrounding region A13of the candidate abnormal region A10is equal to or greater than a predetermined threshold value (predetermined value). When the normalized bubble region area S is equal to or greater than the predetermined value (Yes in step S10), the bubble inside determining unit130determines that the candidate abnormal region A10is inside the bubble region (step S11). This is because the greater the area of the region having the features of the bubble region in the surrounding region A13and a region in the vicinity of the surrounding region A13is, the higher the possibility of the candidate abnormal region A10being an inner region of the bubble region is. When the area S1of the bubble regions A11and A12or the presence of the bubble region in the surrounding region A13and the region in the vicinity of the surrounding region A13is used as the feature data of the surrounding region A13, the candidate abnormal region A10may be determined to be present inside the bubble region when the area S1is equal to or greater than a predetermined value or the bubble region is determined to be present.

Subsequently, in step S12, the determination criterion switching unit141reads a determination criterion value created in advance from the storage unit50. The criterion value read at this time is smaller than the criterion value used by the candidate abnormal region determining unit110in step S02. That is, the region (the abnormal region) which is determined to fall within the range defined by the criterion value read in step S12is fewer than the region (the candidate abnormal region) which is determined to fall within the range defined by the criterion value in step S02. In other words, the criterion value read in step S12is stricter than the criterion value used in step S02.

In steps S13to S15, the abnormal region determining unit140determines whether the candidate abnormal region A10is an abnormal region based on the determination criterion read in step S12. Specifically, first, in step S13, the abnormal region, determining unit140calculates a mean value (G/R mean value) of the G/R value of the candidate abnormal region A10. Subsequently, in step S14, the abnormal region determining unit140determines whether the calculated G/R mean value is smaller than a determination criterion. When the G/R mean value is determined to be smaller than the determination criterion (Yes in step S14), the abnormal region determining unit140determines that the candidate abnormal region A10is an abnormal region such as a bleeding region or a reddening region (step S15). In this case, a region in which the red colors are stronger than the determination result in step S02is determined as an abnormal region.

On the other hand, when the G/R mean value is equal to or greater than the determination criterion (No in step314), the abnormal region determining unit140determines that the candidate abnormal region A10is not an abnormal region (step S16).

Moreover, in step S10, when the normalized bubble region area S is determined to be smaller than a predetermined value (No in step S10), the candidate abnormal region A10is determined not to be present inside the bubble region (step S17). In this case, the candidate abnormal region A10is determined as an abnormal region according to the determination in step S02(step S15).

As described above, according to the first embodiment, when a candidate abnormal region, which is determined based on a predetermined criterion value from an image, is determined to be present inside a bubble region, it is determined whether the candidate abnormal region is an abnormal region in accordance with a different criterion value such that the corresponding regions are fewer than that obtained when the predetermined criterion value is used. Therefore, it is possible to suppress a mucosal region inside bubbles from being erroneously detected as an abnormal region.

First Modification Example

In the first embodiment described above, the G/R value and the luminance value Y have been calculated for each pixel, and the candidate abnormal region and the halation region have been determined. However, an image may be divided into small regions, and the candidate abnormal region and the halation region may be determined for each small region. In this case, a mean value in each small region of the G/R value and the luminance value Y of the respective pixels is used for the determination process.

A process of dividing an image based on edge intensity is performed as follows. First, the edge intensity of each of the respective pixels included in a processing target image is calculated. In calculating the edge intensity, a known method such as a differential filtering process using the Sobel filter may be used. Subsequently, the image is divided into multiple edge regions using the ridges of the edge intensities as boundaries. More specifically, an edge intensity image which uses the respective pixels edge intensities as pixel values is created, and a gradient direction of the edge intensities in the pixels of the edge intensity image is acquired. In this case, the gradient direction is set in a direction where the value of the edge intensity decreases. Moreover, pixels having the minimum value which the respective pixels reach when moving along the gradient direction are searched, and the image is divided so that the pixels at the starting point which have reached the adjacent pixels having the minimum value are included in the same region (see WO 2006/080239 A).

As another image division method, an existing method such as a watershed algorithm can be also used (see Luc Vincent and Pierre Soille, “Watersheds in digital spaces: An efficient algorithm based on immersion simulations”, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 13, No. 6, pp. 583-598, June 1991).

According to the first modification example, since the candidate abnormal region and the halation region are determined based on the feature data of each of the small regions including multiple pixels, it is possible to perform the determination process by reflecting the features of each small region and to improve the computation speed.

Second Modification Example

In the first embodiment, it is determined whether the candidate abnormal region is inside the bubble region using the value obtained by normalizing the total area of the bubble region (the halation region) present in the surrounding region of the candidate abnormal region and the bubble region (the arc-shaped region) near the contour region of the surrounding region by the area of the contour region as the feature data. However, it is not always necessary to extract the contour region of the surrounding region. For example, the determination may be performed using the total area of the halation region and the arc-shaped region present in the surrounding region and the region in the vicinity of the surrounding region or a value obtained by normalizing the total area by the area of the surrounding region as the feature data. Alternatively, the determination may be performed based on whether a predetermined amount of the arc-shaped region is present in the surrounding region and the region in the vicinity of the surrounding region (for example, the total area of the arc-shaped region or the sum of the lengths of the arcs is equal to or greater than a predetermined threshold value).

Second Embodiment

Next, a second embodiment of the invention will be explained.

FIG. 7is a block diagram illustrating the configuration of an image processing device according to the second embodiment. As illustrated inFIG. 7, an image processing device2according to the second embodiment includes an arithmetic unit200which includes a candidate abnormal region determining unit210that determines a candidate abnormal region from an image, a bubble region determining unit220that determines a bubble region from the image, a bubble inside determining unit230that determines whether the candidate abnormal region is present inside the bubble region based on the determination result of the bubble region, and an abnormal region determining unit240that determines whether the candidate abnormal region is an abnormal region. The configuration of the image processing device2other than the arithmetic unit200is the same as that illustrated inFIG. 1.

The bubble region determining unit220includes a peripheral region determining unit221that determines a region having the features of the peripheral region of the bubble region and a determination range determining unit222that determines a determination range where the bubble region in the image is determined based on the determination result of the candidate abnormal region. The peripheral region determining unit221includes an arc-shaped region determining unit221athat determines an arc-shaped region.

The bubble inside determining unit230includes a surrounding region determining unit231that determines the surrounding region of the candidate abnormal region and a surrounding region feature determining unit232that determines the features of the bubble region in the surrounding region based on the determination result of the bubble region in the surrounding region and the region in the vicinity of the surrounding region.

More specifically, the surrounding region feature determining unit232includes an arc-shaped bubble region determining unit232athat determines whether the bubble region in the surrounding region and the region in the vicinity of the surrounding region includes an arc-shaped region and an arc-shaped inner region determining unit232bthat determines whether the surrounding region is an inner region of the arc-shaped region.

Next, the operation of the image processing device according to the second embodiment will be explained.FIG. 8is a flowchart illustrating the operation of the image processing device according to the second embodiment.

First, in step S21, the arithmetic unit200acquires image data corresponding to a processing target image. The detailed process of step S21is the same as that of step S01of the first embodiment.

Subsequently, in step S22, the candidate abnormal region determining unit210determines a candidate abnormal region based on a color feature data.

FIG. 9is a flowchart illustrating a detailed process of determining the candidate abnormal region. In step S201, the candidate abnormal region determining unit210calculates the G/R value from the pixel values of the respective pixels that constitute the image. In this case, similarly to the first modification example, the mean value of the G/R values may be calculated for each of the small regions that are obtained by dividing the image.

In step S202, the candidate abnormal region determining unit210determines whether each of the calculated G/R values is smaller than a first criterion value set in advance. When a region in which the G/R value is smaller than the first criterion value is detected (Yes in step S202), the region is determined as an abnormal region such as a bleeding region or a reddening region (step S203).

On the other hand, with respect to the region in which the G/R value is equal to or greater than the first criterion value (No in step S202), the candidate abnormal region determining unit210determines whether the G/R value in the region is smaller than a second criterion value set in advance (second criterion value>first criterion value) (step S204). When the region in which the G/R value is smaller than the second criterion value is detected (Yes in step S204), the region is determined as a candidate abnormal region which is likely to be an abnormal region but with low confidence (step S205). The region in which the G/R value is equal to or greater than the second criterion value (No in step S204) is determined neither as an abnormal region nor a candidate abnormal region. After that, the processing returns to the main routine.

In step S23, the determination range determining unit222determines a determination range where the bubble region in the image is determined. Specifically, as illustrated inFIG. 10, the gravity center position G of a candidate abnormal region A20determined from the image is calculated, and a region included in a circle of which the origin is located at the gravity center position G and which has a radius of r2is determined as a determination range A21. The value of the radius r2may be a predetermined value set in advance and may be determined adaptively based on the area of the candidate abnormal region A20.

In step S24, the peripheral region determining unit221(the arc-shaped region determining unit221a) determines an arc-shaped region in the determination range A21. A detailed process of a method of determining the arc-shaped region is the same as that described with reference toFIG. 4in step S03of the first embodiment.FIG. 11illustrates an example of arc-shaped regions A22to A26determined in this way.

In step S25, the bubble region determining unit220sets the arc-shaped regions A22to A26determined in step S24as candidate bubble regions.

In step S26, the surrounding region determining unit231determines the surrounding region of the candidate abnormal region A20. A detailed process of step S26is the same as that described in step S06of the first embodiment.FIG. 12illustrates a surrounding region A27determined in step S26.

In step S27, the arc-shaped bubble region determining unit232aextracts an arc-shaped candidate bubble region from the candidate bubble regions (arc-shaped regions) A22to A26in the surrounding region A27of the candidate abnormal region A20and a region in the vicinity of the surrounding region A27and extracts an arc-shaped inner region.

FIG. 13is a flowchart illustrating a detailed process of extracting an arc-shaped candidate bubble region and an arc-shaped inner region.

First, in step S211, the arc-shaped bubble region determining unit232aextracts a candidate bubble region in the surrounding region A27of the candidate abnormal region A20and a region in the vicinity of the surrounding region A27. In the case ofFIG. 12, the candidate bubble regions A22to A24and A26are extracted from the candidate bubble regions A22to A26set in step S25. Subsequently, in step S212, the arc-shaped bubble region determining unit232acalculates a correlation value between the extracted candidate bubble regions A22to A24and A26and an arc-shaped model created in advance. In step S213, a candidate bubble region in which the correlation value with the arc-shaped model is equal to or greater than a predetermined threshold value is determined as an arc-shaped candidate bubble region. For example, in the case ofFIG. 12, the candidate bubble regions A22to A24are determined as an arc-shaped candidate bubble region. Further, in step S214, the arc-shaped bubble region determining unit232aextracts an inner region A28of the candidate bubble regions A22to A24determined as an arc-shaped region. After that, the processing returns to the main routine.

In step S28, the arc-shaped inner region determining unit232bdetermines whether the arc-shaped inner region A28extracted in step S27or the surrounding region A27of the candidate abnormal region A20is an inner region of an arc-shaped region.

When the surrounding region A27of the candidate abnormal region A20is an inner region of an arc-shaped region (Yes in step S28), the bubble inside determining unit230determines that the surrounding region A27is inside a bubble region (step S29). In this case, in step S30, the abnormal region determining unit240determines that the candidate abnormal region A20in the surrounding region A27is not an abnormal region.

On the other hand, when the surrounding region A27of the candidate abnormal region A20is not an inner region of an arc-shaped region (No in step S28), the bubble inside determining unit230determines that the surrounding region A27is not inside the bubble region (step S31). In this case, in step S32, the abnormal region determining unit240determines that the candidate abnormal region A20in the surrounding region A27is an abnormal region.

As described above, according to the second embodiment, first, a region in which the color feature data is smaller than a first criterion value is determined as an abnormal region. With respect to only a region in which the color feature data is between the first criterion value and the second criterion value and which is likely to be an abnormal region with low confidence, it is determined whether the region is an abnormal region using a bubble region. Therefore, it is possible to improve the efficiency of the computation process.

Third Embodiment

Next, a third embodiment of the invention will be explained.

FIG. 14is a block diagram illustrating the configuration of an image processing device according to the third embodiment. As illustrated inFIG. 14, an image processing device3according to the third embodiment includes an arithmetic unit300which includes the candidate abnormal region determining unit110, the bubble region determining unit120, a bubble inside determining unit310that determines whether the candidate abnormal region is present inside the bubble region based on the determination result of the bubble region, and an abnormal region determining unit320that determines whether the candidate abnormal region is an abnormal region based on the determination result of the bubble inside determining unit310. The configuration and the operation of the candidate abnormal region determining unit110and the bubble region determining unit120are the same as those described in the first embodiment. Moreover, the configuration of the image processing device3other than the arithmetic unit300is the same as that illustrated inFIG. 1.

The bubble inside determining unit310includes a surrounding region determining unit311that determines the surrounding region of the candidate abnormal region and a surrounding region feature data calculating unit312that calculates a feature data based on the surrounding region. The surrounding region feature data calculating unit312includes a positional relation-based feature data calculating unit312athat calculates a feature data based on a positional relation of the bubble region in the surrounding region and the region in the vicinity of the surrounding region. The positional relation-based feature data calculating unit312aincludes a distance calculating unit312a-1that calculates the distance between bubble regions and a bubble region positional relation-based feature data calculating unit312a-2that calculates the feature data based on a positional relation between the arc-shaped region and the halation region.

The abnormal region determining unit320includes a determination criterion creating unit321that adaptively creates a determination criterion based on the information on the surrounding region of the candidate abnormal region.

Next, the operation of the image processing device according to the third embodiment will be explained.FIG. 15is a flowchart illustrating the operation of the image processing device according to the third embodiment.

InFIG. 15, the operations of steps S41to S46correspond to the operations of steps S01to S06illustrated inFIG. 2.FIG. 16illustrates a candidate abnormal region A30, an arc-shaped region A31, a halation region A32, and a surrounding region A33which are determined from the image in steps S41to S46.

In step S47, the bubble region positional relation-based feature data calculating unit312a-2determines whether both the arc-shaped region A31and the halation region A32which constitute the bubble region are mixedly present in the surrounding region A33and the region in the vicinity of the surrounding region A33. Here, in the bubble region, a halation is generally observed in the inner side of an arc-shaped region that constitutes a portion having an approximately circular shape (including a shape similar to a circle such as an ellipse). Thus, a region where both a portion determined as the arc-shaped region A31and a portion determined as the halation region A32are mixedly present can be determined not to be the bubble region.

When both the arc-shaped region A31and the halation region A32are not mixedly present in the surrounding region A33and the region in the vicinity of the surrounding region A33(No in step S47), the positional relation-based feature data calculating unit312acalculates the feature data based on the positional relation in each portion of the bubble region (step S48).

FIG. 17is a flowchart illustrating a detailed process of calculating the feature data based on the positional relation in each portion of the bubble region. First, in step S401, the bubble region positional relation-based feature data calculating unit (hereinafter, simply referred to as a feature data calculating unit)312a-2calculates the gravity center position G of the candidate abnormal region A30in the image. Subsequently, in step S402, the feature data calculating unit312a-2calculates the mean distance C from the gravity center position G of the candidate abnormal region A30to the respective arc-shaped regions A31present in the surrounding region A33and the region in the vicinity of the surrounding region A33. Moreover, in step S403, the feature data calculating unit312a-2calculates a mean distance H from the gravity center position G of the candidate abnormal region A30to the respective halation regions A32present in the surrounding region A33and the region in the vicinity of the surrounding region A33. Further, the feature data calculating unit312a-2calculates a difference D between the mean distance C to the arc-shaped region A31and the mean distance H to the halation region A32using the following equation (3) (step S404).
D=C−H(3)

In step S405, the positional relation-based feature data calculating unit312asets the difference D calculated in this way as a feature data based on the positional relation in each portion of the bubble region in the surrounding region A33and the region in the vicinity of the surrounding region A33.

In step S49, the surrounding region feature data calculating unit312sets the feature data (the difference D) based on the positional relation in each portion of the bubble region as a feature data in the surrounding region A33.

In step S50, the bubble inside determining unit310determines whether the feature data (namely, the difference D) in the surrounding region A33is equal to or greater than zero.

When the feature data in the surrounding region A33is equal to or greater than zero (Yes in step S50), the bubble inside determining unit310determines that the candidate abnormal region730is inside the bubble region (step S51). This is because the fact that the difference D is equal to or greater than zero means that the halation region A32can be determined to be present inside the arc-shaped region A31.

Subsequently, in step S52, the determination criterion creating unit321calculates the mean value (G/R mean value) of the G/R values in portions of the surrounding region A33other than the candidate abnormal region A30and the halation region A32.

In steps S53to S55, the abnormal region determining unit320determines whether the candidate abnormal region A30is an abnormal region using the G/R mean value calculated in step S52as a determination criterion. Specifically, first, in step S53, the G/R mean value in the respective candidate abnormal regions A30is calculated. Moreover, in step S54, it is determined whether a difference DABbetween the G/R mean value in the surrounding region A33and the G/R mean value in the candidate abnormal region A30is equal to or greater than a predetermined threshold value (predetermined value). When the difference DABis equal to or greater than the predetermined value (Yes in step S54), the abnormal region determining unit320determines that the candidate abnormal region A30is an abnormal region (step S55). On the other hand, when the difference DABis smaller than the predetermined value (No in step S54), the abnormal region determining unit320determines that the candidate abnormal region A30is not an abnormal region (step S56).

Moreover, when it is determined in step S47that both the arc-shaped region and the halation region are mixedly present in the surrounding region A33and the region in the vicinity of the surrounding region A33(Yes in step S47), the bubble inside determining unit310determines that the candidate abnormal region is not inside the bubble region (step S57). In this case, in step S55, the abnormal region determining unit320determines that the candidate abnormal region A30is an abnormal region.

Moreover, even when the feature data D is smaller than zero (No in step S50), the bubble inside determining unit310determines that the candidate abnormal region is not inside the bubble region (step S57).

As described above, according to the third embodiment, since it is determined whether the surrounding region of the candidate abnormal region is present inside the bubble region based on the positional relation of the portions that constitute the bubble region, it is possible to improve the detection accuracy of the bubble region.

Third Modification Example

In the first to third embodiments described above, although the G/R value has been used as the color feature data which is used in determining the candidate abnormal region, various color feature dates such as the respective RGR values, relative values (B/G values or the like) of the respective RGB values, brightness and color difference calculated by YCbCr conversion, or hue, saturation, and lightness calculated by HSI conversion can be used. For example, when the B/G value is used as the color feature data, it is easy to determine a lesion of a region covered by the bile that is yellow. Here, B and G components of an illumination light that illuminates the lumen of a subject are absorbed in a red region by approximately the same amount. Thus, there is not a great difference between the B and G values in many regions of the lumen and a bleeding region. However, since the amount of absorption of the B component increases in the bile, the B value decreases in the bile region, and it becomes easy to detect a change in the G value. Thus, a region in the image in which the B/G value is equal to or greater than a predetermined value can be determined as a lesion, and a region in which the B/G value is smaller than the predetermined value can be determined as a region other than the lesion.

According to the first to third embodiments and the modification examples thereof, when the candidate abnormal region determined based on the first determination criterion is determined to be present inside the bubble region, it is determined whether the candidate abnormal region is an abnormal region using the second determination criterion different from the first determination criterion. Thus, it is possible to suppress a mucosal region inside bubbles from being erroneously detected as an abnormal region.

The image processing device according to the first to third embodiments and the modification examples thereof can be implemented by an image processing program recorded on a recording medium being executed by a computer system such as a personal computer or a workstation. Moreover, the computer system may be used in a state of being connected to an apparatus such as another computer system or a server via a local area network, a wide area network (LAN/WAN) or a public line such as the Internet. In this case, the image processing device according to the first to third embodiments may acquire image data of the intra-luminal images via these networks, output image processing results to various output apparatuses (a viewer, a printer, and the like) connected via these networks, and store the image processing results in a storage device (a recording medium, a reading device thereof, and the like) connected via these networks.

The invention is not limited to the first to third embodiments and the modification examples thereof, but various inventions can be formed by appropriately combining multiple constituent components disclosed in the respective embodiments and modification examples. For example, some constituent components may be removed from all constituent components illustrated in the respective embodiments and modification examples, and constituent components illustrated in other embodiments and modification examples may be appropriately combined.