Method and computer system for prefetching of images

A method and system for prefetching of data, such as, image data of medical images, for example, digital radiographic images, into a cache. The prefetching relies on workflow information. The method and system is particularly advantageous for use in screening programs where a large number of individuals are screened for a disease, such as cancer, in particular, breast cancer.

FIELD OF THE INVENTION

The present invention relates to a method and apparatus for the operation of a cache buffer store in a data processing system, such as a computer system for rendering of medical images. More particularly, the invention relates to a method and apparatus including a procedure for prefetching of images and a corresponding algorithm for replacing least recently used images from the cache.

BACKGROUND AND PRIOR ART

In many data processing systems, there is provided between the working memory of the central processing unit and the main memory a high-speed buffer storage unit that is commonly called a “cache”. This unit enables a relatively fast access to a subset of data that were previously transferred from the main storage to the cache, and thus, improves the speed of operation of a data processing system.

U.S. Pat. No. 4,807,110 discloses a prefetching mechanism for a cache and a two-level shadow directory. When an information block is accessed, a parent identifier derived from the block address is stored in a first level of the shadow directory. The address of a subsequently accessed block is stored in the second level of the shadow directory, in a position associated with the first-level position of the respective parent identifier. With each access to an information block, a check is made whether the respective parent identifier is already stored in the first level of the shadow directory. If it is found, then a descendant address from the associated second-level position is used to prefetch an information block to the cache if it is not already resident therein. This mechanism is to reduce the occurrence of cache misses.

U.S. Pat. No. 6,154,767 shows a method and apparatus for using attribute transition probability models for prefetching resources. Idle bandwidth of a client is utilized for the prefetching and resource prefetching by the resource server utilizes idle processing and/or data bus resources of the server.

U.S. Pat. No. 6,151,662 discloses a method of data transaction typing for caching and prefetching. A microprocessor assigns a data transaction type to each instruction. The data transaction type is based upon the encoding of the instruction, and indicates an access mode for memory operations corresponding to the instruction. The access mode may, for example, specify caching and prefetching characteristics for the memory operation. The access mode for each data transaction type is selected to enhance the speed of access by the microprocessor to the data, or to enhance the overall cache and prefetching efficiency of the microprocessor by inhibiting caching and/or prefetching for those memory operations.

From U.S. Pat. No. 6,098,064 a method for prefetching and caching documents according to a probability ranked need list is known. Documents are prefetched and cached on a client computer from servers located on the Internet in accordance with their computed need probability. Those documents with a higher need probability are prefetched and cached before documents with lower need probabilities. The need probability for a document is computed using both a document context factor and a document history factor. The context factor of the need probability of a document is determined by computing the correlation between words in the document and a context of the operating environment. The history factor of the need probability of a document is determined by integrating both how recently the document was used and the frequency of document use.

U.S. Pat. No. 6,154,826 discloses a method for maximizing memory system bandwidth by accessing data in a dynamically determined order. The order in which to access said information is selected based on the location of information stored in the memory. The information is repeatedly accessed from memory and stored in the temporary storage until all streamed information is accessed. The information is stored until required by the data processor. The selection of the order in which to access information maximizes bandwidth and decreases the retrieval time.

Caching is an important technique in all fields of data processing where large amounts of data have to be handled, such as image processing. A particular field of image processing is the rendering, archival, retrieval, processing, transformation, analysis and display of medical images, such as, in the field of digital radiography.

U.S. Pat. No. 6,041,135 shows an interactive off-line processing method for radiographic images. In this off-line image processing method for radiographic images an image is decomposed into detail images and multiple resolution levels and a residual image; detail images are modified up to a preset resolution level and a processed image is reconstructed by means of the modified detail images and the residual image. Interactive processing is performed with different parameter settings.

U.S. Pat. No. 6,127,669 discloses a method for computer aided determination of window and level settings for filmless radiology. Input image data is transformed into an image histogram. This histogram is then segmented into a small number of parts corresponding to structures of interest. Segmentation of the histogram is done by a Viterbi optimal runlength-constrained approximation nonlinear filter.

Window and level settings are calculated to correspond to this segmentation. A readable image of one of the structures of interest is then displayed. In another embodiment, the tool provides a menu of optimal window and level settings corresponding to the different types of information a radiologist may be interested in.

A common disadvantage of prior art workstations for processing and displaying of digital radiographic images is the latency time experiences by the user of the system when a new image is loaded.

This is due to the fact that a single digital radiographic image can be over 50 MB in size. For example, in digital mammography a case to be reviewed by a radiologist is composed of about 4 to 8 different images, each 60 MB in size for certain modalities. A complete case thus is about 500 MB of data.

Even with high-speed processors and memory components such volumes of data result in substantial latency times that are often not acceptable for the users of such workstations. As a consequence, usage of data processing systems for the archiving, transformation, retrieval and transformation of digital radiographic images has been limited so far.

SUMMARY OF THE INVENTION

Objects of the invention

It is therefore an object of the present invention to provide an improved method and apparatus for prefetching in order to reduce latency times experienced by a user, in particular with respect to applications for digital radiographic images.

It is a further object of the invention to provide a method and system having an improved “least recently used algorithm” for freeing a cache memory of image data which is no longer needed.

It is a still further object to provide such a method and system for coupling to a central picture archiving and communication system (PACS).

The invention accomplishes the above and other objects and advantages by providing a method and computer system for the prefetching of data, such as image data, for example medical images from the field of digital radiography. The invention enables maximizing system throughput and minimizing latency times experienced by a user. In brief, this is accomplished by taking a predefined workflow into consideration for the prefetching of the images. A corresponding method and system is disclosed in co-pending US Patent Application entitled A Method And Computer System For Screening Of Medical Cases, Ser. No. 09/870,380; filed May 29, 2001,

This is of particular advantage in so-called “screening programs” where a large number of medical cases is reviewed sequentially by one or a group of radiologists. Typically the cases of such screening programs are all structured in the same or in a similar way. For example each case can consist of the same sequence of image formats and views.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention will be described in detail as set forth in the preferred embodiments illustrated in the drawings. Although these embodiments depict the invention as in its preferred application, such as, for a digital radiology image system, it should be readily apparent that the invention has equal application to any type of data processing system that encounters or deals with the same or similar problems.

FIG. 1shows a data processing system1having an application program2that is coupled to a cache3.

The cache3is coupled to a data source4. The data source4can be any type of data source, such as a file system stored on a hard drive or another storage medium or the output of an image processing device, such as a computer aided diagnosis tool.

The data processing system1further has a file5for storing a workflow to be performed. The file5has a sequence6of steps S1, S2, S3, . . . Si, . . . , Sn to be sequentially performed in the workflow. The sequence6has a pointer7which points to the current step being carried out by the data processing system and/or its user. In the example considered here, the current step to which the pointer7is directed, is the step S2.

A prefetcher8has access to the file5and is coupled to the cache3for requesting of data to be prefetched.

In operation, the application program2reads the sequence6to obtain the step to which the pointer7is directed. After the application program2has obtained the next step to be carried out, it issues a corresponding request to the cache3to request data which are required for the performance of the current step.

These data are transferred from the cache3to the application program2for display and/or further processing. If a so-called “cache miss” occurs the request for the data is passed to the data source4from the cache3in order to read the data from the data source4and store the data in the cache3for access by the application program2.

A user of the application program2can also input a jump request to jump in the sequence6in a forward or backward direction. If such a jump occurs, then the pointer7is moved by a corresponding number of steps along the sequence6.

In order to prevent the occurrence of cache misses, the prefetcher8reads from the sequence6the step consecutive to the current step and all remaining images for the current step. In the example considered here the current step is the step S2and the consecutive step is the step S3. Hence, when the application program2works on the current step S2the prefetcher8reads the consecutive step S3and performs an access operation to the cache3in order to request the data which are required for the application program2to perform the step S3.

If the data for carrying out the step S3is already in the cache3, no further action is required. If the contrary is the case, the request is transferred to data source4in order to provide the cache3with the required data.

When the processing of the current step S2by the application program2is complete, the position of the pointer7is incremented by one, such that the pointer7points to the consecutive step. The consecutive step S3is read by the application program2and the application program2requests the data for the performance of the step S3—which is the new current step—from the cache3.

As the corresponding data has been requested previously from the cache3by the prefetcher8while the application program2was busy with the processing of the step S2, it is ensured that the data for the performance of the step S3is already loaded in cache3such that no cache miss occurs. Again the prefetcher8performs an access operation to the sequence6in order to read the consecutive step of the new current step S3in order to request the data being required for the performance of the new consecutive step while the current step S3is being carried out.

Alternatively the prefetcher8can also request the data for a number of consecutive steps or set of steps in the sequence6rather than just the data for the next consecutive step.

FIG. 2shows a flow chart for illustrating the operation of the data processing system1of FIG.1. In step41the application starts working on the current step Si; before or after this the prefetcher obtains the current step Siand the next consecutive step Si+1from the workflow in step42.

In step43the application requests at least a subset of the data required for the performance of the complete step Sifrom the cache. During the processing of the step Siall of the data required for the performance of step Siis requested by the application in certain discrete data packages.

After the completion of the processing of the step Sithe application goes to the next step Si+1in the workflow in step44. From there the application jumps to a step Si+1+jin the workflow in the example considered here in response to a corresponding input operation of a user. The step Si+1+jis the new current step. The jump is performed in step45.

In parallel, the data required for the performance of the current step Siand the consecutive step Si+1of the current step are requested in step46from the cache for the purposes of prefetching. In step47it is decided whether the data is already in the cache.

If this is the case an acknowledgement is sent to the prefetcher in step48. If the contrary is the case a data source is accessed in step9in order to load the data into the cache in step10. Next an acknowledgement is sent to the prefetcher in step48. It is to be noted that the step42and the steps46to48,9and10are carried out in parallel to the steps41and43to45with respect to the then current step.

FIG. 3shows the structure of a medical case consisting of a number m of images, image1to image m. It is assumed that the medical case ofFIG. 3is a standardized case having a predefined number, kinds and formats of images. Such a standardization of medical cases is advantageous for large screening programs where a large number of individuals are examined for a disease, such as cancer, especially breast cancer.

FIG. 4is a further preferred embodiment of a data processing system in which like elements are denoted with the same reference numerals as in FIG.1. File5ofFIG. 4, which is descriptive of the workflow, contains a case stack31. The case stack31contains a stack of cases, case1, case2, case3, . . . , case N to be reviewed by a radiologist. File9ais user definable and contains the sequence of images of interest to the radiologist and the corresponding required format. In the example considered here the radiologist has defined the sequences of images of interest to him in the file9aas follows:First, the image1from the current case is to be displayed in a format1;next, the same image1is to be displayed in another format2;next, the image2is to be displayed in a format3;. . .next, image m-1is to be displayed in the format1; and. . .as the last image in the sequence of images to be reviewed for an individual medical case image m is to be displayed in a format4.

It is to be noted that image3and/or other images of the m images is skipped completely—depending on the user specific initialization of the file9a.

The case stack31has a pointer10awhich points to the current case being processed by the application2. In the example considered here this is the case2. Likewise the file9ahas a pointer11pointing to the current image being processed within the current case. In the example considered here the current image is the image1(2) of the case2.

The application2is coupled to a display buffer12that is coupled to a monitor13.

Further the cache3is coupled to a data source manager14. The data source manager14has a table15relating image formats (Data Tags) to corresponding data sources. In the example, the format1is related to the data source of file reader16, the format2to the scaler17, the format3to CLAHE18for performing of a contrast limited adaptive histogram equalization, and format4to wavelet enhancement19.

In operation, the prefetcher8sequentially requests all of the images contained in the current case as indicated by the file9afrom the cache3, as well as, the images of the consecutive case. For example, when the prefetcher8requests the image1(1) of the consecutive case3, the request is provided to the cache3. If the corresponding data is already in the cache3this is acknowledged to the prefetcher8.

If this is not the case, the request is passed onwards to the data source manager14. The data source manager14identifies the appropriate data source for the request, which is the file reader16. As a consequence, the image1(1) of the case3is requested from the file reader16, which reads the corresponding data from mass storage, such as the hard drive of the data processing system1, not shown inFIG. 4, and transfers the data to the cache3.

When the prefetcher8requests the image2(3) of the case3, a situation can occur where the image2is already in the cache3, but not in the right format. As the image is not available in the required format the request is still passed onwards to the data source manager14for identification of the appropriate data source, which is CLAHE18in this case.

As a consequence, the CLAHE18is requested to provide the image2in the required format3. This is done by the CLAHE18by requesting the corresponding image data in the format1from the cache3, and then performing the contrast limited adaptive histogram equalization on the raw image data. The resulting transformed raw image data is stored in the format3in the cache3and an acknowledgement is send to the prefetcher8.

FIG. 5shows a further preferred embodiment of the data processing system. Again the same reference numerals will be used for like elements corresponding to elements ofFIGS. 1 and 4.

The application program2is coupled to the prefetcher8to inform the prefetcher8of a next case to be processed. Again the prefetcher requests the images required for the current case and a consecutive case from the image cache3.

If a requested image is not in the image cache3, the request is passed onwards to the image source20. The operation of image source20corresponds to the data source manager14. The image source20identifies an appropriate image source such as file reader16or scaler17or passes the request onwards to enhancement component21, which passes the request onwards to CLAHE18or wavelet transformer19depending on the required image enhancement.

The requested image is loaded and/or generated by the respective image source and the image is stored in the image cache3. This is acknowledged by the data source, which has loaded and/or generated the requested image by the message “Req_done” to all requestors of the image.

If a requestor of the image does not require the image anymore, a “release” message is sent to the image cache3from the requestor. Likewise a “cancel” message can be sent to the image cache3in case a request is to be cancelled before it has been completed. Such a situation can occur when the user jumps to another case in the case stack.

FIG.6. shows the format of an image file22having a header23and a field24for storage of the pictorial data.

The header23contains an entry for the case number to which the image belongs, the image number for identification of the image, the image format, the requestors, the priority of the image for the purposes of a least recently used algorithm and a lock count. The utilization of those fields will become more apparent in the following.

FIG. 7shows a flow chart for illustrating a least recently used algorithm and the generation of the header23of an image. In step70an image Ikof a case i is requested by a requester. In principle each of the components of the data processing system can take the role of a requestor of an image. This applies both to the application program as well as to the prefetcher and the data sources.

Also, the application program can have more than one requestor. For example individual program components of the application program can take the role of individual requestors of images.

In step71the corresponding request is sent to the cache. In step72the cache decides whether the requested image is already available. If this is the case the lock count is incremented in step73. In step74the priority of the image is incremented by +P2. For example, the value of P2can be equal to 100. In step75an acknowledgement is sent to all requestors of the image; the requestors are identified by the corresponding data field in the header23of the requested image.

If the image is not in the cache the control goes to step76for generation of a header of the image in the cache. The priority of the image is set to be equal to a predefined value of P1in step77. For example the value of P1can be equal to 100. In step78the lock count is set to be equal to1.

In step79it is decided whether the cache has enough capacity for loading of the pictorial data of the requested image. If this is the case the pictorial data are requested from the appropriate image source in step80. In step81the pictorial image data are loaded and/or generated by the image source.

This can require that the image source itself issues a request for the image in an appropriate format, for example to calculate the desired transformation in the requested image format. For example, if the image source is CLAHE, the CLAHE requires raw image data in the format1. Correspondingly, the CLAHE requests the image in the format1such that the steps from step70onwards are invoked.

Once the requested image is present in the cache in the raw data format1, the CLAHE image transformation is carried out to generate the image in the requested format3. This way the step81is completed and an acknowledgement is provided to all requestors of the image in the format3in step82.

If it is decided in step79that the cache does not have enough capacity, the control goes to step83. In step83the contents of the cache is examined for images having a lock count equal to0. All images having a lock count equal to0are grouped into a set S.

In step84an image ILof the set S having the lowest priority of the images in the set S is identified. This image ILis freed from the cache in step85.

In step86all priorities of the images of the set S are decremented by one but not below0. From86the control returns back to step79in order to repeat the step83to86if the available cache capacity is still insufficient.

The flow chart ofFIG. 8shows the steps performed when a requestor releases a previously requested image or cancels a request. In step80aofFIG. 8, the requestor releases the image Ikof case i because the requestor does not require the image anymore.

In step81a, a corresponding release message is received by the image cache and the header of the image Ikis identified in order to decrement the lock count in the header. This is done in step81a. In step82a, the priority is decremented by P3but not below0. For example, the value of P3can be equal to 50. This way the released image can be replaced or removed from the cache in case capacity needs to be provided. The replacement is done in accordance with the least recently used algorithm as illustrated with respect toFIG. 7, in particular steps83to84.

FIG. 9shows a system having a number of radiographic workstations25corresponding to the data processing system1ofFIG. 1,4and5. The workstations25are coupled to an image server26. The image server26has a central archive for medical images.

Each of the workstations25has the components of the data processing system1of FIG.4. In addition each workstation25has a scheduler28that is connected to the case stack31and a keyboard27for entering jump commands and other entries. Further the workstation25has a local storage29for the storage of image data provided by the image server26and a user profile30.

In operation the scheduler28identifies the case to be processed from case stack31and requests the corresponding image data from the image server26. Preferably this is done when the workstation is not in use by the radiologist such as during night time.

This way the required data can be downloaded from the image server26over night, which takes a considerable amount of time in view of the large data volume. The image data is captured and stored on the storage29locally on the workstation25. When the cases of the case stack31are reviewed by the radiologist, the required image data is obtained directly from the storage29and not from the image server26which further reduces latency times.

Although the present invention has been shown and described with respect to preferred embodiments, nevertheless, changes and modifications will be evident to those skilled in the art from the teachings of the invention. Such changes and modifications that embody the spirit, scope and teachings of the invention are deemed to fall within the purview of the invention as set forth in the appended claims.