Abstract:
A method for combining serial radiological images includes the projection of each of the images onto a line in a color space, the evaluation of a vector sum of the resulting projections; and the generation of a composite image from the vector sum.

Description:
FIELD OF INVENTION  
       [0001]     This invention relates to medical imaging, and in particular, to image interpretation.  
       BACKGROUND  
       [0002]     Certain illnesses are made manifest by changes in the morphology of some internal structure in the body. These structures, and many others, can be viewed using any of a variety of radiological techniques.  
         [0003]     The images that result from any of those radiological techniques are snapshots frozen in time. To identify a change in the morphology, one visually inspects two or more images of the same region of the body. While this procedure is suitable for identifying gross changes in morphology, it is easy to miss more subtle changes.  
         [0004]     For example,  FIG. 1  shows three successive MRI images representing longitudinal cross-sections of a brain taken some months apart. A close inspection of these images reveals numerous changes in brain morphology. However, these changes are subtle and easily overlooked, even by those trained in the appropriate specialty.  
       SUMMARY  
       [0005]     The invention is based on the recognition that vector addition in a color space can be used to draw attention to subtle changes in a sequence of images. In particular, the invention draws attention to changes in a sequence of images by tinting each image with a particular color. When the images are spatially registered and overlaid, colors assigned to corresponding pixels will add in such a way that static portions of the resulting image will appear in the same color (grayscale, black &amp; white), and changing portions of the resulting image will appear in other colors.  
         [0006]     In one aspect, the invention includes a method for drawing attention to changed regions in serial radiological images. The method includes tinting each of the images with a different color, spatially registering the images, and overlaying the images so that static regions and changed regions are distinguishable from each other by color.  
         [0007]     Additional practices of the invention include those in which tinting each of the images with a different color includes selecting the different colors to define an orthogonal basis of a color space. For example, this may include tinting a first image blue, tinting a second image green, and tinting a third image red. In one practice of the invention, the first image is selected to be an image obtained before the second image, and the second image is selected to be an image obtained before the third image.  
         [0008]     Other practices of the invention include those in which the images are normalized relative to each other.  
         [0009]     Another aspect of the invention includes a method for combining serial radiological images. The method includes the projection of each of the images onto a line in a color space, the evaluation of a vector sum of the resulting projections; and the generation of a composite image from the vector sum.  
         [0010]     Alternative practices of the invention include those in which projecting each of the images onto a line in a color space includes selecting the lines on which the images are projected to include axes of an orthogonal basis for the color space. For example, the axes can be selected to include a red axis, a green axis, and a blue axis.  
         [0011]     The temporal significances of the axes is arbitrary. However, in one practice of the invention, the image acquired most recently is projected onto the red axis, and the earliest-acquired image is projected onto the blue axis.  
         [0012]     Another practice of the invention features the additional step of normalizing the images relative to each other.  
         [0013]     The particular type of image can be an MRI image. However, other types of radiological images, such as computerized tomography (“CT”) or ultrasound (“US”) images can also be used.  
         [0014]     In another aspect, the invention includes a computer-readable medium having encoded thereon software having instructions for carrying out any of the foregoing methods.  
         [0015]     In yet another aspect, the invention includes a system for processing MRI images. The system includes a data storage medium for storing serial MRI images, a processor configured to process the MRI images to draw attention to changed regions therein, and a computer-readable medium in data communication with the processor. The computer-readable medium has, encoded thereon, software having instructions for executing any of the foregoing methods.  
         [0016]     Other features and advantages of the invention will be apparent from the following detailed description, and from the claims. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0017]     This patent of application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.  
         [0018]      FIG. 1  shows three serial MRI images of the same longitudinal section of a brain separated by several months;  
         [0019]      FIG. 2  shows an image acquisition system;  
         [0020]      FIG. 3  shows architecture software used to overlay images using the computer shown in  FIG. 2 ;  
         [0021]      FIG. 4  shows three tinted MRI images and a color-composite image obtained by overlaying those tinted images; and  
         [0022]      FIG. 5  is a color cube used for interpreting the color-composite image in  FIG. 4 . 
     
    
     DETAILED DESCRIPTION  
       [0023]      FIG. 2  shows an imaging system  10  having an image acquisition device  12  in data communication with a general purpose digital computer  14  having a mass storage device  16  and a display  13 . A variety of image acquisition devices  12  can be used in connection with  FIG. 2 . Exemplary image acquisition devices  12  include MRI devices, including devices for obtaining functional MRI images, ultrasonic imaging devices, CAT scanners, and PET scanners. Moreover, the image acquisition device  12  need not be one that acquires images of internal structures. For example, a digital camera might be used to take sequential images of skin tissue in an effort to identify changes in skin pigmentation that may be indicative of melanoma.  
         [0024]     Serial images  18  acquired by the image acquisition device  12  are stored on the mass storage device  16  for further processing, as shown in  FIG. 3 . These serial images  18  are obtained with identical acquisition parameters. They differ only in the times at which they were obtained.  
         [0025]     The data stored on a mass storage device  16  is in fact data representative of the serial images  18  and not the images themselves. Nevertheless, throughout this specification, the term “image” will be used to refer also to the data representative of that image.  
         [0026]     As shown in  FIG. 3 , which depicts the architecture of an image processing system, a user provides a selection process  20  with instructions on which of the serial images  18  are to be compared. Because the color space is inherently three dimensional, images are best compared three at a time. However, the selection process  20  can select two or more images for comparison.  
         [0027]     Each pixel in a serial image  18  represents a measured physical parameter. However, the serial images were taken at different times, and possibly using different image acquisition devices  12 . As a result, there may have been differences in performance and calibration of the image acquisition device  12 . These differences may result in pixels on one serial image  18  appearing different from pixels in another image even though both represent the same underlying physical parameter.  
         [0028]     Accordingly, in response to the user&#39;s selection of serial images  18  to be compared, the selection process  20  retrieves selected images  22  from the mass storage device  16  and provides them to a normalizing process  24 . The normalizing process  24  then generates three normalized images  26  from the three selected images  22 .  
         [0029]     Again, because the serial images  18  were taken at different times, there is no guarantee that the patient&#39;s position was identical each time. The normalized images  26  may therefore be spatially misaligned. In particular, the image coordinates of a structure in one normalized image  26  may not match the image coordinates of that structure in another image. To avoid this, the normalized images  26  are provided to a registration process  28  that aligns the image coordinate systems associated with each normalized image  26 . The resulting aligned images  30  have the property that the image coordinates associated with a particular structure are the same across all the aligned images  30 .  
         [0030]     Each of the three aligned images  30  is then provided to a tinting process  32  to be tinted with one of three colors. Exemplary tinted images  34  are shown on the left side of  FIG. 4 . Preferably, the aligned images  30  are tinted in red, blue, and green because those colors form an orthogonal basis for the RGB color space used in conventional computer monitors. The resulting tinted images  34  are then provided to an overlay process  36 .  
         [0031]     Each pixel on an aligned image  30  is typically a shade of gray, with white and black being considered limiting cases of gray. Accordingly, each pixel can be viewed as being mapped to a point on a line in the RGB color space, the line being characterized by equal contributions of red, green, and blue. The process of tinting an aligned image  30  effectively projects this line onto one of the three orthogonal axes of the color space.  
         [0032]     The effect of tinting is that if a pixel in one tinted image  34  is identical to its corresponding pixels in the remaining two tinted images  34 , a vector sum of the color vectors associated with each of those pixels will result in the original shade of gray that was common to all three aligned images  30 . On the other hand, if a pixel in a tinted image  34  differs from its corresponding pixels in one or both of the other two tinted images  34 , the vector sum of the color vectors associated with the three pixels will result in a color, the particular color being dependent on the nature of that difference.  
         [0033]     For each pixel location in a tinted image  34 , the overlay process  36  performs a vector sum in the RGB color space of the pixels in each of the three tinted images  34  that correspond to that location. This results in a color-composite image  38  in which regions that are gray correspond to regions that remained the same across all three images, and regions that are colored correspond to regions that changed in at least one of the three images. The particular color that results from the vector addition is indicative of when the change occurred.  
         [0034]     The right-hand image on  FIG. 4  shows a composite image  38  that results from applying the overlay process  36  to the three tinted images  34  on the left side of the figure. It is apparent from inspection of this composite image  38  that the morphology of certain areas of the brain has changed and that color has drawn particular attention to the existence of those changes. As discussed below, the particular hue and intensity of a changed portion of the image can be used to indicate when the change occurred.  
         [0035]     In interpreting the color-composite image in  FIG. 4 , it is useful to recognize that the oldest, or baseline image is tinted blue, the most recent image is tinted red, and an image taken between these two images is tinted green. Assuming this particular sequence, recent changes will tend to be in warmer colors and less recent changes will tend to be in cooler colors.  
         [0036]     The sequencing of colors (blue, green, red) can be altered to represent other imaging sequences (e.g. CT, US) and other morphological changes. This sequence is intuitive for images in which pathological changes are associated with local hyperintensities. Thus a very recent hyperintensity in image three will add more red to the mixed image and appear as a warmer color indicating recent pathology. A reversal of the order (red, green, blue) would be intuitive for images in which active changes result in hypointensity (i.e. image pixels becoming darker).  
         [0037]      FIG. 5  is a color cube suggesting how the colors in the color-composite image might be interpreted. For example, in the color-composite image in  FIG. 4 , active lesions, i.e. those that are of most recent origin, tend to be red. Recurrent lesions tend to be a combination of red and blue; resolving lesions, tend toward blue, with more recently resolving regions tending to a bluish green. Lesions that have peaked recently and have only just begun to resolve will tend toward green.  
         [0038]     The color cube  FIG. 5  provides a basis for creating a standardized interpretation system for comparison of serial images. If adopted, such an interpretation system would enhance the usefulness of the serial image analysis method disclosed herein by enabling exchange of images among sites while maintaining a common language for interpretation of those images.  
         [0039]     In the foregoing example, a change in a serial image  18  corresponds to a change in structure. However, the change being observed can be a change in any parameter that is provided by, or inferred from, a particular image. For example, in the case of a PET scan, the change can be that of an uptake rate whose change can be observed.  
         [0040]     An advantage of the foregoing implementation is that no data reduction has taken place. Because the original serial images  18  were tinted using primary colors, the original tinted images  34  can be recovered from the composite color image.  
         [0041]     In an alternative embodiment, a sequence of four or more serial images  18  is compared by tinting each image with a different color. In this case, however, information will be lost because the number of images is greater than the dimensionality of the color space.  
         [0042]     A suitable method for comparing sequences of more than four images is to process them three at a time. For example, given a sequence of images I 1 , I 2 , . . . I 1 , one can tint I 1  blue, I 2  green and I 3  red. Then, one can proceed by tinting  12  blue, tinting  13  green, and tinting I 4  red, and so on.  
         [0043]     The images being tinted need not be images taken at a particular instant. They can, for example, be images obtained by averaging or otherwise processing images that were taken at successive instants. For example, given the foregoing sequence of images, I 1 , I 2  . . . In, one might define a “synthetic” image J 1  by averaging the images I 1  . . . I 10 . One could then tint J 1  blue, and tint I 11  green and I 12  red to obtain a composite image. Or, one could define a second synthetic image, J 2 , by averaging together I 11 -I 15 , tint that image green, and tint I 16  red and obtain another composite image.  
         [0044]     It is to be understood that while the invention has been described in conjunction with the detailed description thereof, the foregoing description is intended to illustrate and not limit the scope of the invention, which is defined by the scope of the appended claims. Other aspects, advantages, and modifications are within the scope of the following claims.