Automated method and system for the segmentation of medical images

Abstract: A method for the automated segmentation of medical images, including generating image data from radiographic images of the breast. The method is applicable to breast mammograms including the extraction of the skinline as well as correction for non-uniform exposure conditions, hand radiographs, and chest radiographs. Techniques for the segmentation include noise filtering, local gray value range determination, modified global histogram analysis, region growing and determination of object contour. The method also is applicable to skin detection and analysis of skin thickening in medical images, where image segmentation, local optimization of external skinline, creation of a gradient image, identification of the internal skinline and then skin thickness determination are carried out.

Description: Technical Field The invention relates generally to a method and system for the computerized automatic segmentation of medical images. Specific applications are given for breast mammograms including the extraction of the skin line as well as correction for non-uniform exposure conditions, for hand radiographs, and for chest radiographs. Techniques include novel developments and implementations including noise filtering, local gray value range determination, modified global histogram analysis, region growing and determination of object contour. Background Art Although mammography is currently the best method for the detection of breast cancer, between 10-30% of women who have breast cancer and undergo mammography have negative mammograms. In approximately two-thirds of these false-negative mammograms, the radiologist failed to detect the cancer that was evident retrospectively. The missed detections may be due to the subtle nature of the radiographic findings (i.e., low conspicuity of the lesion), poor image quality, eye fatigue or oversight by the radiologists. In addition, it has been suggested that double reading (by two radiologists) may increase sensitivity. It is apparent that the efficiency and effectiveness of screening procedures could be increased by using a computer system, as a "second opinion or second reading", to aid the radiologist by indicating locations of suspicious abnormalities in mammograms. In addition, mammography is becoming a high volume x-ray procedure routinely interpreted by radiologists. If a suspicious region is detected by a radiologist, he or she must then visually extract various radiographic characteristics. Using these features, the radiologist then decides if the abnormality is likely to be malignant or benign, and what course of action should be recommended (i.e., return to screening, return for follow-up or return for biopsy). Many patients are referred for surgical biopsy on the basis of a radiographically detected mass lesion or cluster of microcalcifications. Although general rules for the differentiation between benign and malignant breast lesions exist, considerable misclassification of lesions occurs with current radiographic techniques. On average, only 10-20% of masses referred for surgical breast biopsy are actually malignant. It is apparent that the efficiency and effectiveness of screening procedures could be increased by using a computer system, which is capable of segmenting the mammogram into breast and non breast regions, detecting the skin line and perform enhancement that allows viewing of the complete dynamic range without loss in contrast. The enhancement techniques could improve detection of various lesions by increasing their conspicuity. Also, accurate segmentation allows for determination of the skin line. Although breast skin thickening may occur in a variety of benign disorders like edema, inflammation or scarring, it can also indicate underlying malignant disease and may be the only mammographic sign of an inflammatory carcinoma. Also, such capabilities would be useful in other regions of the body, such as those examined by bone radiography and chest radiography. EP-A-0 427 912 describes a medical image segmentation system based on region growing. Growing criteria include local and global density changes. Disclosure of the Invention Accordingly, an object of this invention is to provide an automated method and system for segmenting medical images. Another object of this invention is to provide an automated method and system for the determination of skin line in medical images. These and other objects are achieved by a method and system according to the invention. The invention is defined by independent claims 1 and 28. Further embodiments are defined by the dependent claims. In particular, the invention is directed to an automated system and method for segmentation of an image of an anatomic feature of a subject, said method comprising: determining a local gray value range for each pixel of said image of said anatomic feature; obtaining a global histogram using said image; modifying said global histogram based on said local gray value ranges to obtain a modified global histogram; and segmenting said image using said modified global histogram. A further embodiment of the invention is directed to a method wherein said step of segmenting said image produces a segmented image; and said method further comprises: determining an external skinline of said feature using said image segmented image; determining an internal skinline of said feature using said segmented image; and measuring a skin thickness of said feature using said internal and external skinlines. A further method comprises also obtaining a gradient image of said segmented image; and determining said internal and external skinlines using said gradient image. In a further method, determining said internal skinline comprises: obtaining a gray value profile perpendicular to a border of said feature; and determining a local gradient minimum in said gray value profile as said internal skinline. In a further method, determining said external skinline comprises: determining an initial external skinline of said feature; calculating a second derivative of a dark-side Laplacian; and determining said external skinline as a ridge in said second derivative. In a further method, determining said external skinline as said ridge comprises: determining a gray scale skeleton of pixels in said gradient image; and determining a maxima ridge in said skeleton as said external skinline. A further method comprises determining skin thickening using said internal and external skinlines. A further method comprises superimposing said internal and external skinlines on said image. In a further method said step of segmenting said image produces a segmented image; and said method further comprises: determining an external skinline using said segmented image; calculating a distance of pixels of said image being potential feature pixels to said external skinline; determining an average gray value of said potential feature pixels as a function of said distance; and enhancing a periphery of said image using said average gray value. Specific applications are given for breast mammograms including the extraction of the skin line as well as correction for non-uniform exposure conditions, for hand radiographs, and for chest radiographs. Techniques include novel developments and implementations including noise filtering, local gray value range determination, modified global histogram analysis, region growing and determination of object contour. Brief Description of the Drawings A more complete appreciation of the invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: FIG. 1 is a schematic diagram illustrating the automated method for segmentation of medical images according to the invention; FIG. 2 is a schematic diagram illustrating the gray value range operator; FIGS. 3A and 3B are graphs illustrating the modified global histogram analysis; FIG. 4 is a schematic diagram illustrating a partially segmented breast image at this stage of the method; FIG. 5 is a schematic diagram illustrating determination of the object contour; FIGS. 6A and 6B are schematics illustrating a distance map image and the subsequent threshold image; FIG. 7 is a schematic diagram illustrating a segmented breast in a digital mammogram; FIG. 8 is a graph illustrating the performance of the segmentation method, evaluated on 740 mammograms. The ratings were subjectively assigned by 3 observers; FIG. 9 is a schematic diagram illustrating how the segmentation method could be incorporated within a computer-aided diagnosis scheme for mammography; FIG. 10 is a schematic diagram illustrating various uses of the segmenter when breast contour determination is necessary; FIG. 11 is a schematic diagram illustrating a segmented hand in a digital bone radiograph; FIG. 12 is a schematic diagram illustrating a segmented chest in a digital chest radiograph; FIG. 13 is a schematic diagram of threshold of an image of the hand; FIGS. 14A-14D are plots of the pixel distribution of ROIs of FIG. 13; FIG. 15 is a schematic block diagram illustrating a system for implementing the automated method for segmentation of medical images according to the invention; Best Mode for Carrying Out the Invention Referring now to the drawings, and more particularly to Figure 1 thereof, a schematic diagram of the automated method for the segmentation of breast images is shown. In this example the aim is to identify the breast region by excluding uniform dark (direct exposure) and uniform bright (unexposed) image regions. The method includes an initial acquisition of a radiograph of the breast and digitization (step 10). Noise filtering is applied to the digital image (step 20) followed by application of the gray-value range operator (step 30). Using information from the local range operator a modified global histogram analysis is performed (step 40). Region growing is performed on the threshold image using connectivity (counting pixels) in step 50, followed by a morphological erosion operation (step 60). The distance map of the image is determined (step 70) and the boundary of the segmented object in the image is then tracked to yield its contour (step 80). The contour can then be output onto the digital image or passed to other computer algorithms (step 90). Initially noise filtering using a square median filter, 3 by 3 pixels in size, is employed in order to eliminate digitizer line artifacts and spike noise. The advantage of using a median filter is that the noise reduction process does not affect the smooth edges. Figure 2 shows a schematic diagram illustrating the application of the gray-value range operator. In this example, a 7 pixel by 7 pixel ring kernel is used to find the local maximum and local minimum pixel values. The difference between the local maximum and the center pixel value, and that between the center pixel value and the local minimum are calculated as the range and stored for later reference. Pixels yielding a small local gray-value range are considered as possible "non-object" (non-breast) pixels. The range is determined on the basis of the base width of a pixel histogram, as shown in Fig. 3. Next, the global gray-value histogram of the image is determined as illustrated in Figs. 3A and 3B. The original histogram (Fig. 3A) contains gray values from all pixels in the image. The modified histogram (Fig. 3B) contains only contributions from pixels with a small local range (maximum - minimum value), which correspond to possible non-breast pixels. The criteria used in classifying a pixel as a non-breast pixel include (1) having a pixel value close to a global histogram peak, (2) having a small local gray value range and (3) being part of a large connected region. This can be thought as obtaining three sequential images. Figs. 4A-4C illustrate the effect of these three classification criteria, respectively. The direct exposure region (with black corresponding to a gray level of zero) would have pixel values in which the local minimum must be small and the non exposed region (with white being at 1023) would have pixel values in which the local maximum must be small. After the first two criteria of the method, the image is in the form of a 3-gray-level image, where one value corresponds to potential breast pixel and the other two values correspond to potential non-breast pixel (either a no exposure region or a direct exposure region). Knowledge of being in a large connected region is accomplished by region growing using connectivity, such as 4-point or 8-point. Also, another requirement is that the non-exposed region must be outside the direct exposure region (which is inside from the film edge). Figs. 4C illustrates the partially segmented breast image at this stage of the method. The darker pixels correspond to non-breast pixels. It is noted that the image may contain pixels identified as possible breast pixels in the direct exposure region and in the border. Next, the three-value image is subjected to a morphological erosion operation using a 3 pixel by 3 pixel square kernel. Such processing is necessary in order to prevent the "breast region" from including artifacts from the film digitization process, which may have gray values similar to pixels inside the breast region. The filtered binary image is then subjected to a contouring routine as illustrated in Figure 5. Note, however, (by comparing Figures 4 and 5) that rules based on knowledge of the mammographic image need to be included in the contouring in order to identify and eliminate the "transition zone" between the direct and non-exposed regions (which is included in "breast region" in Figure 4). Thus, the image becomes a four-value (2-bit) image. This is done as follows. The rules include analysis of connection points, corresponding to points with a concave angle and a short connected path to the outside, which are used in cutting across the transition zone. For examining connected paths, a distance map of the image is calculated as illustrated in Figs. 6A and 6B. Figure 6A illustrates the distance map image, and Figure 6B illustrates the subsequent threshold image obtained by thresholding Figure 6A. Here, darker pixels are closer to the film edge. The shortest connecting path of "breast object pixels" to the outside (i.e., film edge) is calculated for each pixel within the possible breast region in the 3-gray-level image. However, in the calculation, calculations of distance are only performed if the direction of the path does not cross a direct exposure region. The thresholding yields possible transition points which are then analyzed for presence of "sharp" concave angles. Then, the contouring routine need only track the pixels having the single gray value corresponding to the breast region. Figure 7 shows an example of a final segmented breast in a digital mammogram, showing an allowed connection at point A. At point B, the connection was not made since the concave angle was not sufficiently sharp. The degree of sharpness required to make the connection is empirically derived. Figure 8 is a graph illustrating the performance of the segmentation method, evaluated on 740 mammograms. The ratings were subjectively assigned by 3 observers. Note that 96% were considered acceptable for use as input to further computerized mammographic analysis methods. In the rating scale (x-axis) of Fig. 8, (1) corresponds to optimal, (2) to minor deviations, (3) to acceptable for CAD purposes, (4) to substantial deviations, and (5) to complete failure of segmentation. The segmentation method could be employed in an iterative manner as illustrated in Figure 10. In this implementation, various parameters of the method could be iteratively tried in order to segment the breast in images obtained from various film digitizers or direct digital devices. Figure 9 shows examples of how the computer-determined breast contour (found from breast segmentation) could be further used in such methods as mass detection, microcalcification detection, and skin analysis in computer-aided diagnosis schemes, and image enhancement. The segmentation can be used in other medical imaging applications including segmentation of the hand in bone radiographs as showed in Figure 11, and segmentation of the chest in images of the thorax as shown in Figure 12. In the segmentation of the hand from the directly exposed region, both global and local thresholding can be used. Local thresholding is used to segment bone from skin. As shown in Fig. 13, a number of ROIs (ROI1-ROI5, in this example) can be placed on the hand image. The corresponding pixel distributions for ROI1-ROI3 are shown in Figs. 14A-14C. As ROI1 is entirely in the directly exposed region, the pixel distribution shows a single peak with no valley (Fig. 14A). Thus the center pixel of ROI1 is set to a constant K 1 . In ROI2, a valley is found at gray level p 2 . If the center pixel in ROI2 has a gray value less than p 2 , then the center pixel is assigned a gray value of K 2 . If the center pixel in ROI2 has a gray value greater than p 2 , then the center pixel is assigned a gray value of K 3 . In ROI3, a valley is found at gray level p 3 . The center pixel of ROI3 is assigned gray value K 2 or K 3 if its gray value is less than or greater than p 3 , respectively. It should be noted that ROI4 and ROI5 will have a single peak distribution similar to Fig. 14A as ROI4 is entirely within the bone and ROI5 is entirely within the skin. The advantage of the local thresholding is that the peak shown in ROI3 may be too small to be detected on a histogram of an entire image, as shown in Fig. 14D. Figure 15 is a more detailed schematic block diagram illustrating a system for implementing the method of the invention for automated segmentation of medical images. Referring to Figure 15, radiographic images of an object are obtained from an image acquisition device 150 which could be an x-ray exposure device and a laser digitizer, and input to the system. Each breast image is digitized and put into memory 151. The image data is first passed through a noise filtering circuit 152 and a local gray-value range circuit 153 in order to determine the initial potential regions of breast and non-breast. The data is then passed to the modified global histogram analysis circuit 154 and the region growing circuit 155 in order to determine a partial segmentation. Image data are passed to the morphological erosion circuit 156, the distance map circuit 157, and the initial contouring circuit 158 which determines the contour by evaluating the thresholded image data after the distance map is obtained, in order to determine the features for input to the contour connection circuit 159. During the determination of the transition zone (as shown in Fig. 5), the data are retained in image memory 160. In the superimposing circuit 161 the results are either superimposed onto breast images, stored in file format, or shown with all non-breast regions set to a constant gray value. The results are then displayed on the display system 163 after passing through a digital-to-analog converter 162. Obviously, numerous modifications and variations of the present invention are possible in light of the above technique. It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein. Although the current application is focused on radiographic medical images, the concept can be expanded to segmentation in other images of the human body.

Claims: An automated method of segmentation of an image of an anatomic feature of a subject, comprising: determining a local gray value range for each pixel of said image of said anatomic feature (30); obtaining (40) a global histogram using said image; modifying (40) said global histogram based on said local gray value ranges to obtain a modified global histogram; and segmenting said image using said modified global histogram. A method as recited in Claim 1, wherein modifying said global histogram comprises: determining local minimum pixel values in said image; determining local maximum pixel values in said image; determining a local gray value range using said local minimum and said local maximum pixel values; and modifying said global histogram using said range. A method as recited in Claim 2, wherein segmenting said image comprises: classifying a pixel in said image as a non-feature pixel based upon said local gray value range, upon whether said pixel is in a vicinity of a gray value peak of said modified histogram, and upon whether said pixel is part of a connected region; and obtaining a first segmented image containing first pixels having a gray value corresponding to a feature pixel and second pixels having at least one gray value corresponding to said non-feature pixel. A method as recited in Claim 3, wherein classifying said pixel in said image based upon whether said pixel is part of said connected region comprises region growing. A method as recited in Claim 3, comprising: performing a morphological open operation on said first segmented image to obtain a second segmented image; and performing a contouring routine on said second segmented region. A method as recited in Claim 3, comprising: determining a distance map of said first segmented region. A method as recited in Claim 6, wherein determining said distance map comprises: determining a connection path for each first pixel in said first segmented image; and thresholding said distance map based upon said connection path. A method as recited in Claim 7, wherein determining said connection path comprises: calculating a distance from each first pixel to an edge of said first segmented image if a corresponding connection path does not cross a region of said first segmented image corresponding to a directly exposed region of said image. A method as recited in Claim 7, comprising: thresholding said distance map to generate a second segmented image having pixels corresponding to said feature pixels; and determining a contour of said feature using said second segmented region. A method as recited in Claim 9, wherein determining said contour comprises: analyzing points along said contour having a connection path to an edge of said second segmented region less than a predetermined length and a concave angle. A method as recited in Claim 9, wherein determining said contour comprises: analyzing points along said contour having a connection path to an edge of said second segmented region crossing a transition zone corresponding to an area in said image between a directly-exposed area and a non-exposed area with a length less than a predetermined length and a concave angle. A method as recited in Claim 1, comprising: obtaining a contour of said feature in said segmented image; and analyzing points along said contour where said feature connects with a transition zone corresponding to an area in said image between a directly-exposed area and a non-exposed area. A method as recited in Claim 1, comprising: digitizing said image of said feature to obtain a digitized image; and performing noise filtering on said digitized image to obtain a filtered image; wherein modifying said global histogram comprises: determining local minimum pixel values in said filtered image; determining local maximum pixel values in said filtered image; determining a local gray value range using said local minimum and said local maximum pixel values; and modifying said global histogram using said range. A method as recited in Claim 2, wherein segmenting said image comprises: determining a first segmented image by classifying first pixels in said image as non-feature pixels based upon said gray value range; determining a second segmented image by classifying second pixels in said first segmented image as said non-feature pixels based upon whether said second pixels are in a vicinity of a gray value peak of said modified histogram; and determining a third segmented region by classifying third pixels in said second segmented image as said non-feature pixels based upon whether said third pixels are part of a connected region. A method as recited in Claim 14, comprising: obtaining a fourth segmented image containing fourth pixels having a gray value corresponding to a feature pixel and fifth pixels having at least one gray value corresponding to a non-feature pixel. A method as recited in Claim 1, comprising: obtaining a radiographic breast image; and segmenting said breast in said radiographic breast image. A method as recited in Claim 16, comprising: obtaining a contour of said breast in said segmented image; and analyzing points along said contour where said breast connects with a transition zone corresponding to an area in said image between a directly-exposed area and a non-exposed area. A method as recited in Claim 1, comprising: obtaining a radiographic hand image; and segmenting said hand from said image using local thresholding. A method as recited in Claim 18, wherein said local thresholding comprises: placing a predetermined number of regions-of-interest each having a plurality of pixels on said hand image; determining a distribution of said pixels in each of said regions-of-interest; and setting selected pixels in each of said regions-of-interest to a predetermined value based upon corresponding distributions of said pixels. A method as recited in Claim 1, wherein: said step of segmenting said image produces a segmented image; and said method further comprises: determining an external skinline of said feature using said image segmented image; determining an internal skinline of said feature using said segmented image; and measuring a skin thickness of said feature using said internal and external skinlines. A method as recited in Claim 20, comprising: obtaining a gradient image of said segmented image; and determining said internal and external skinlines using said gradient image. A method as recited in Claim 20, wherein determining said internal skinline comprises: obtaining a gray value profile perpendicular to a border of said feature; and determining a local gradient minimum in said gray value profile as said internal skinline. A method as recited in Claim 20, wherein determining said external skinline comprises: determining an initial external skinline of said feature; calculating a second derivative of a dark-side Laplacian; and determining said external skinline as a ridge in said second derivative. A method as recited in Claim 23, wherein determining said external skinline as said ridge comprises: determtining a gray scale skeleton of pixels in said gradient image; and determining a maxima ridge in said skeleton as said external skinline. A method as recited in Claim 20, further comprising determining skin thickening using said internal and external skinlines. A method as recited in Claim 20, further comprising superimposing said internal and external skinlines on said image. A method as recited in Claim 1, wherein: said step of segmenting said image produces a segmented image; and said method further comprises: determining an external skinline using said segmented image; calculating a distance of pixels of said image being potential feature pixels to said external skinline; determining an average gray value of said potential feature pixels as a function of said distance; and enhancing a periphery of said image using said average gray value. A system for automodically segmenting an image of an anatomic feature of a subject, comprising: an image acquisition device (150); means for determining a global histogram of said image, connected to said image acquisition device; means for determining a local gray value range for each pixel of said image, connected to said means for determining a global histogram; means (154) for modifying said global histogram based on said local gray value ranges; means for segmenting said image based upon said modified global histogram. A system as recited in Claim 28, further comprising: means for contouring an image segmented by said means for segmenting; and a display. A system for segmenting images as recited in claim 29, further comprising: an image memory connected to said image acquisition device; a distance map circuit connected to said means for modifying said global histogram; a contour connection circuit connected to said contouring circuit. A system as recited in Claim 30, comprising: a noise filtering circuit connected between said image memory and said means for determining a local range; a region growing circuit connected to said means for modifying said global histogram ; an erosion circuit connected between said region growing circuit and said distance map circuit; and a superimposing circuit connected between said contour connection circuit and said display. A system as recited in Claim 28, comprising: a skinline identification circuit;

Inventors: BICK ULRICH, GIGER MARYELLEN L, BICK, ULRICH, GIGER, MARYELLEN L.
Applicants: ARCH DEV CORP, ARCH DEVELOPMENT CORPORATION
Application year: 1994
Main classification: G06T5
IPC codes: G06T5