Patent Description:
During volume rendering, a large amount of data (e.g., gigabytes (GBs) of data) is to be loaded from a storage drive. The storage drive may store datasets to be volume rendered and may be accessed by a number of workstations in communication with the storage drive. A user at a workstation may request retrieval and volume rendering of one of the stored datasets, and based on typical background load (e.g., for preprocessing jobs and Digital Imaging and Communications in Medicine (DICOM) receive jobs), a processor associated with the storage drive, for example, may access the dataset and/or volume render the dataset at a rate of <NUM>-<NUM> MB/s or less. This results in the user waiting seconds or minutes at the workstation for the volume rendered dataset to be accessed and a rendered image to be displayed at the workstation. The reference by<NPL>, DOI: <NUM>/S11548-<NUM>-<NUM>-<NUM>, discloses a pattern recognition system based on artificial neural networks with incremental learning to evaluate, from a set of usage pattern, which one fits the user behavior at a given time. Tile accuracy of the pattern recognition model in distinct training conditions was also evaluated. The reference <CIT> discloses systems and methods that allow transfer criteria to be defined based on one or more of several attributes, such as a particular user, site, or device, as well as whether individual images and/or image series are classified as thin slices, and applied to medical images in order to determine which images are downloaded, viewed, stored, and/or any number of other actions that might be performed with respect to particular images. The reference <CIT> discloses a method for remotely reviewing medical data allows medical data to be transmitted to a physician's mobile device from where they are stored or collected. The method sends data to be transmitted through a console, and a data server to reach the mobile device. After reviewing the data, physicians may then transmit their comments from their mobile device to the requestor by going through the same system. The method may use predictive algorithms regarding the next image processing steps and not regarding the preloading of the underlying image dataset.

In order to reduce a time between a user at a first computing device requesting display of an image to be rendered based on a stored dataset and the display of the image, stored datasets are preloaded at the first computing device or a second computing device in communication with the first computing device. A processor predicts which datasets are to be rendered and displayed and preloads at least one of the predicted datasets. The prediction is based on user-selected cases within a worklist, explicit selection by the user, or an automatic prediction.

In a first aspect, a method of preloading data based on which an image is to be displayed according to claim <NUM> is provided. In particular, it includes predicting, by a processor, at least one image dataset to be displayed. The processor retrieves the at least one image dataset from a memory in communication with the processor. The another memory in communication with the memory stores the at least one image dataset. After the predicting, data identifying an image to be displayed is received. The image to be displayed corresponds to an image dataset of the at least one image dataset. The processor generates the image to be displayed. The generated image is transmitted to a computing device geographically remote from the processor or the generated image is displayed based on the received data.

According to a preferred embodiment, the at least one image dataset stored in the memory is at least one dataset of a plurality of datasets stored in the memory.

The plurality of image datasets are image datasets representing volumes of one or more patients.

Further, the data identifying the image is first data and predicting the at least one image to be displayed comprises receiving sec ond data from the computing device and identifying the at least one image dataset to be displayed based on the received second data.

According to a preferred embodiment the received second data comprises a list identifying an order of image datasets to be displayed.

According to a preferred embodiment the retrieving the at least one image dataset comprises retrieving a first image dataset from the plurality of image datasets stored in the memory, the first image dataset being first in the order of the list, the generated image being a first image generated based on the first image dataset, and the method further comprises retrieving a second image dataset from the plurality of image datasets stored in the memory after the first image is transmitted to the geographically remote computing device or displayed, the second image dataset being second in the order of the list.

According to a preferred embodiment the method further comprises: generating, by the processor, a second image to be displayed based on the second image dataset; transmitting the generated second image to the geographically remote computing device or displaying the generated second image; and deleting, by the processor, the first image dataset.

According to another preferred embodiment the received second data comprises a plurality of requests from the geographically remote computing device, the plurality of requests identifying a number of image datasets of the plurality of image datasets stored in the memory.

According to a preferred embodiment the retrieving the at least one image dataset comprises retrieving at least some image datasets of the image datasets identified by the plurality of requests from the memory.

According to another preferred embodiment the predicting the at least one image dataset comprises automatically identifying, by the processor, the at least one image dataset to be displayed.

In a second aspect, a method of presenting data to be displayed includes receiving, by a first computing device, first data from a second computing device. The first data indicates one or more datasets that are preloaded at the second computing device. A display of the first computing device displays a list of datasets. The list of datasets represents a first subset of datasets and a second subset of datasets. The first subset of datasets includes the one or more preloaded datasets. The first subset of datasets is represented differently than the second subset of datasets within the list of datasets. The first computing device transmits a request for a dataset of the list of datasets to the second computing device. The first computing device receives a representation of the identified dataset from the second computing device in response to the transmitting of the request for the dataset. The display of the first computing device displays the received representation of the identified dataset. A time between the transmitting of the request and the displaying of the representation of the identified dataset is less when the identified dataset is of the first subset of datasets compared to when the identified dataset is of the second subset of datasets.

According to a preferred embodiment the method further comprises:.

receiving, by the first computing device, second data, the second data identifying the dataset of the request from the displayed list of datasets, the transmitting of the request comprising transmitting the request in response to the received second data; and receiving, by the first computing device, third data, the third data identifying at least one dataset to be displayed, the at least one dataset to be displayed being at least one dataset of a plurality of datasets stored in a memory geographically remote from the first computing device and comprising the one or more preloaded datasets.

According to a preferred embodiment the receiving the second data comprises receiving the second data from a user via an input device, and the receiving the third data comprises receiving the third data from the user or another user via the input device or another input device.

According to another preferred embodiment the at least one dataset to be displayed includes a first dataset and a second dataset, wherein the received first data indicates that the first dataset is preloaded at the second computing device, wherein the received second data identifies the first dataset from the displayed list of datasets, wherein receiving the representation of the identified dataset comprises receiving, by the first computing device, a representation of the first dataset from the second computing device in response to the transmitting of the received third data, and wherein after the receiving of the representation of the first dataset, the method further comprises receiving, by the first computing device, fourth data from the second computing device, the fourth data indicating that the second dataset is preloaded at the second computing device.

According to a preferred embodiment the received first data indicates that a plurality of datasets are preloaded at the second computing device.

According to a preferred embodiment the first subset of datasets is represented differently than the second subset of datasets in that, within a graphical user interface displayed by the display of the first computing device, representations for datasets of the first subset of datasets within the list are different colors, have different icons, have additional icons, have different text, or any combination thereof compared to representations for datasets of the second subset of datasets within the list.

According to another preferred embodiment the receiving the first data comprises receiving the first data from the second computing device without any request for the one or more datasets to be preloaded being sent from the first computing device to the second computing device.

In a third aspect, a system for preloading data to be displayed includes a memory configured to store a plurality of datasets, a plurality of images, or the plurality of datasets and the plurality of images. The system also includes a processor in communication with the memory. The processor is configured to predict at least one dataset of the plurality of datasets, based on which an image is to be displayed at a computing device geographically remote from the system. The processor is also configured to retrieve one or more datasets of the at least one dataset from the memory based on the prediction. The system also includes an interface in communication with the processor. The interface is configured to receive, after the prediction, data identifying a dataset of the one or more retrieved datasets from the computing device. The processor is further configured to transmit, via the interface, a representation of the identified dataset to the geographically remote computing device.

According to a preferred embodiment of the system the memory or another memory is configured to store the one or more retrieved datasets, and less than all storage capacity of the memory or the other memory is allocated for storage of retrieved datasets.

In a fourth aspect, a system for presenting data to be displayed is provided. In particular, it includes an input configured to receive first data from a computing device. The first data indicates one or more datasets that are preloaded at the computing device. The computing device is geographically remote from the system. The system also includes a display in communication with the input. The display is configured to display a list of datasets. The list of datasets includes a first subset of datasets and a second subset of datasets. The first subset of datasets includes the one or more preloaded datasets. The first subset of datasets is represented differently than the second subset of datasets within the list of datasets on the display. The input is configured to receive second data. The second data identifies a dataset from the displayed list of datasets. The system further includes a processor in communication with the display. The processor is configured to transmit the received second data to the geographically remote computing device. The input is configured to receive a representation of the identified dataset from the geographically remote computing device in response to the transmission of the received second data. The display is further configured to display the received representation of the identified dataset. A time between the transmission of the received second data and the receipt of the representation of the identified dataset is less when the identified dataset from the displayed list of datasets is of the first subset of datasets compared to when the identified dataset from the displayed list of datasets is of the second subset of datasets.

Data prediction is provided from a system point of view to load specific cases in main memory on a server side, so that when a user attempts to open a specific case, the case is already loaded in the main memory at the server. When the case is already loaded in the main memory at the server, the user waits a short period of time (e.g., less than one second) before an image generated based on the already loaded case is displayed to the user. When the case is not preloaded in the main memory at the server, the user waits a longer period of time (e.g., greater than five seconds) for the case to be loaded from a file system into the main memory.

<FIG> shows one embodiment of an imaging system <NUM>. The imaging system <NUM> may be used in the system and method described below. The imaging system <NUM> may include one or more imaging devices <NUM> (e.g., an imaging device) and one or more image processing systems <NUM> (e.g., an image processing system). Additional, different or fewer components may be provided. A dataset representing a two-dimensional (2D) or a three-dimensional (3D) (e.g., volumetric) region may be acquired using the imaging device <NUM> and the image processing system <NUM> (e.g., an imaging system). The 2D dataset or the 3D dataset may be obtained contemporaneously with the planning and/or execution of a medical treatment procedure or at an earlier time. Additional, different, or fewer components may be provided.

In one embodiment, the imaging system <NUM> is, for example, a CT system. The imaging system <NUM> may be used to create a patient model. The patient model may be used for diagnostic purposes and/or in the planning of a medical treatment procedure.

In other embodiments, the imaging system <NUM> may be, for example, a computed tomography (CT) system, an ultrasound system, a positron emission tomography (PET) system, and angiography system, a fluoroscopy, an x-ray system, any other now known or later developed imaging system, or any combination thereof. The imaging device <NUM> is a transmitter, a receiver, and/or a detector for scanning a patient. The image processing system <NUM> is a system or a workstation for treatment planning. The image processing system <NUM> may be a control processor, a renderer, a user interface processor of the imaging device <NUM>, a separate computer, laptop, workstation, or processor, or any combination thereof. The image processing system <NUM> receives data representing the patient or images of the patient generated by the imaging device <NUM>. The imaging system <NUM> may include more or fewer components. In one embodiment, the image processing system <NUM> includes a workstation and a server in communication with the workstation. Data, based on which an image to be displayed at the workstation is generated, for example, may be preloaded at the server and/or the workstation.

<FIG> shows the imaging system <NUM> including one embodiment of the imaging device <NUM>. The imaging device <NUM> is shown in <FIG> as a C-arm X-ray device. The imaging device <NUM> may include an energy source <NUM> and an imaging detector <NUM> connected together by a C-arm <NUM>. Additional, different, or fewer components may be provided. In other embodiments, the imaging device <NUM> may be, for example, a gantry-based CT device or an MRI device.

The energy source <NUM> and the imaging detector <NUM> may be disposed opposite each other. For example, the energy source <NUM> and the imaging detector <NUM> may be disposed on diametrically opposite ends of the C-arm <NUM>. Arms of the C-arm <NUM> may be configured to be adjustable lengthwise, so that the energy source <NUM> and the imaging detector <NUM> may be positioned optimally in a ring structure. In certain embodiments, the C-arm <NUM> may be movably attached (e.g., pivotably attached) to a displaceable unit. The C-arm <NUM> may be moved on a buckling arm robot. The robot arm allows the energy source <NUM> and the imaging detector <NUM> to move on a defined path around the patient. During acquisition of non-contrast and/or contrast scans, for example, the C-arm <NUM> is swept around the patient. During the contrast scans, contrast agent may be injected intravenously. In another example, the energy source <NUM> and the imaging detector <NUM> are connected inside a gantry.

High rotation speeds may be attained with the C-arm X-ray device <NUM>. In one embodiment, the C-arm X-ray device is operable to rotate up to <NUM> °/s or <NUM> °/s. In other embodiments, one complete scan (e.g., a <NUM> degree rotation) of the high speed C-arm X-ray device <NUM> may be conducted in less than <NUM> seconds, less than <NUM> seconds, less than <NUM> seconds, less than <NUM> second, less than <NUM> seconds, or less than <NUM> seconds.

The energy source <NUM> may be a radiation source such as, for example, an X-ray source. The energy source <NUM> may emit radiation to the imaging detector <NUM>. The imaging detector <NUM> may be a radiation detector such as, for example, a digital-based X-ray detector or a film-based X-ray detector. The imaging detector <NUM> may detect the radiation emitted from the energy source <NUM>. Data is generated based on the amount or strength of radiation detected. For example, the imaging detector <NUM> detects the strength of the radiation received at the imaging detector <NUM> and generates data based on the strength of the radiation. The data may be considered image data as the data is used to then generate an image. Image data may also include data for a displayed image.

During each rotation, the high speed C-arm X-ray device <NUM> may acquire between <NUM>-<NUM> projections, between <NUM>-<NUM> projections, or between <NUM>-<NUM> projections. In other embodiments, during each rotation, the C-arm X-ray device <NUM> may acquire between <NUM>-<NUM> projections per second, or between <NUM>-<NUM> projections per second. Any speed, number of projections, dose levels, or timing may be used.

A region <NUM> to be examined (e.g., the brain of a patient) is located between the energy source <NUM> and the imaging detector <NUM>. The region <NUM> may be all or a portion of the patient. The region <NUM> may or may not include a surrounding area. For example, the region <NUM> to be examined may include the brain and/or other organs or body parts in the surrounding area of the brain.

The data may represent a two-dimensional (2D) or three-dimensional (3D) region, referred to herein as 2D data or 3D data. For example, the C-arm X-ray device <NUM> may be used to obtain 2D data or CT-like 3D data. A computer tomography (CT) device may obtain 2D data or 3D data. In another example, a fluoroscopy device may obtain 3D representation data. The data may be obtained from different directions. For example, the imaging device <NUM> may obtain data representing sagittal, coronal, or axial planes or distribution.

The imaging device <NUM> may be communicatively coupled to the image processing system <NUM>. The imaging device <NUM> may be connected to the image processing system <NUM>, for example, by a communication line, a cable, a wireless device, a communication circuit, and/or another communication device. For example, the imaging device <NUM> may communicate the data to the image processing system <NUM>. In another example, the image processing system <NUM> may communicate an instruction such as, for example, a position or angulation instruction to the imaging device <NUM>. All or a portion of the image processing system <NUM> may be disposed in the imaging device <NUM>, in the same room or different rooms as the imaging device <NUM>, or in the same facility or in different facilities.

All or a portion of the image processing system <NUM> may be disposed in one imaging device <NUM>. The image processing system <NUM> may be disposed in the same room or facility as the imaging device <NUM>. In one embodiment, the image processing system <NUM> and the one imaging device <NUM> may each be disposed in different rooms or facilities (e.g., geographically remote from each other). The image processing system <NUM> may represent a single image processing system or a plurality of image processing systems associated with more than one imaging device <NUM>. For example, a plurality of workstations, at least one of which is associated with an imaging device <NUM>, may be in communication with a geographically remote server via a network. The server may store, in a database, datasets generated by the imaging devices <NUM> and/or may access one or more databases outside of the server that store the datasets generated by the imaging devices <NUM>.

In the embodiment shown in <FIG>, the image processing system <NUM> includes a processor <NUM>, a display <NUM> (e.g., a monitor), and a memory <NUM>. Additional, different, or fewer components may be provided. For example, the image processing system <NUM> may include an input <NUM> (e.g., an input device and/or one or more inputs via which the image processing system <NUM> receives data), a printer, and/or a network communications interface.

The processor <NUM> is a general processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array, an analog circuit, a digital circuit, an accelerated processing unit (APU), a graphics processing unit (GPU), a Manycore processor, a multicore processor, another now known or later developed processor, or combinations thereof. The processor <NUM> may be a single device or a combination of devices such as, for example, associated with a network or distributed processing. Any of various processing strategies such as, for example, multiprocessing, multi-tasking, and/or parallel processing may be used. The processor <NUM> is responsive to instructions stored as part of software, hardware, integrated circuits, firmware, microcode or the like.

The processor <NUM> may generate an image from the data. The processor <NUM> processes the data from the imaging device <NUM> and generates an image based on the data. For example, the processor <NUM> may generate one or more angiographic images, fluoroscopic images, top-view images, in-plane images, orthogonal images, side-view images, 2D images, 3D representations (e.g., renderings or volumes), progression images, multi-planar reconstruction images, projection images, or other images from the data. In another example, a plurality of images may be generated from data detected from a plurality of different positions or angles of the imaging device <NUM> and/or from a plurality of imaging devices <NUM>.

The processor <NUM> may generate a 2D image from the data. The 2D image may represent a planar slice of the region <NUM> to be examined. For example, the C-arm X-ray device <NUM> may be used to detect data that may be used to generate a sagittal image, a coronal image, and an axial image. The sagittal image is a side-view image of the region <NUM> to be examined. The coronal image is a front-view image of the region <NUM> to be examined. The axial image is a top-view image of the region <NUM> to be examined.

The processor <NUM> may generate a 3D representation (e.g., 3D reconstruction) from the data. The 3D representation illustrates the region <NUM> to be examined. The 3D representation may be generated from a reconstructed volume (e.g., by combining 2D images and/or by tomography) obtained by the imaging device <NUM> from a given viewing direction. For example, a 3D representation may be generated by analyzing and combining data representing different planes through the patient, such as a stack of sagittal planes, coronal planes, and/or axial planes. Additional, different, or fewer images may be used to generate the 3D representation. Generating the 3D representation is not limited to combining 2D images. For example, any now known or later developed method may be used to generate the 3D representation.

The processor <NUM> may be a volume renderer that generates 2D projections from the reconstructed volume. The processor <NUM> takes color and opacity into account with a transfer function when generating the 2D projections from the reconstructed volume. Values of the reconstructed volume are converted into respective RGBA values, and the RGBA values are projected on corresponding pixels. The RGBA values may be projected on the corresponding pixels in any number of ways including, for example, volume ray casting, splatting, shear warp, photon mapping, path tracing, and/or texture-based volume rendering.

The processor <NUM> may display the generated images on the monitor <NUM>. For example, the processor <NUM> may generate the 3D representation and communicate the 3D representation to the monitor <NUM>. The processor <NUM> and the monitor <NUM> may be connected by a cable, a circuit, another communication coupling, or a combination thereof. The monitor <NUM> is a monitor, a CRT, an LCD, a plasma screen, a flat panel, a projector or another now known or later developed display device. The monitor <NUM> is operable to generate images for a two-dimensional view or a rendered three-dimensional representation. For example, a two-dimensional image representing a three-dimensional volume through rendering is displayed.

The processor <NUM> may communicate with the memory <NUM>. The processor <NUM> and the memory <NUM> may be connected by a cable, a circuit, a wireless connection, another communication coupling, or a combination thereof. Images, data, and other information may be communicated from the processor <NUM> to the memory <NUM> for storage, and/or the images, the data, and the other information may be communicated from the memory <NUM> to the processor <NUM> for processing and/or routing to the display <NUM>. For example, the processor <NUM> may communicate the generated images, image data, or other information to the memory <NUM> for storage.

The memory <NUM> is a computer readable storage media (e.g., non-transitory computer-readable storage media). The computer readable storage media may include various types of volatile and non-volatile storage media, including but not limited to random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media and the like. The memory <NUM> may be a single device or a combination of devices. The memory <NUM> may be adjacent to, part of, networked with and/or remote from the processor <NUM>.

In one embodiment, the memory <NUM> includes a main memory <NUM> and another memory <NUM> including a database. The main memory <NUM> and the other memory <NUM> may be part of a same computing device (e.g., the server) or may be included in separate computing devices, respectively (e.g., the server and another computing device geographically remote from the server). The database may store datasets and/or images in the Digital Imaging and Communications in Medicine (DICOM) format.

The input <NUM> may include one or more input devices configured to allow a user to interact with any components of the image processing system <NUM>. The one or more input devices <NUM> may include a number pad, a keyboard, a cursor control device such as, for example, a mouse or a joystick, a touch screen display, a remote control, or any other device operative to interact with the image processing system <NUM>.

The image processing system <NUM> (e.g., the input <NUM>) may include one or more input and/or output interfaces via which data is received and/or sent. The one or more input devices <NUM> may be in communication with the processor <NUM> and the memory <NUM>, for example, via the one or more interfaces. For example, a first computing device (e.g., a workstation) of the image processing system <NUM> and a second computing device (e.g., a server) of the image processing system <NUM> may be in communication via respective interfaces.

<FIG> shows one embodiment of an image processing system <NUM> including a first computing device <NUM> in communication with a second computing device <NUM>. In one embodiment, the first computing device <NUM> is a workstation, and the second computing device <NUM> is a server. The server <NUM> may, for example, be geographically remote from the workstation <NUM>. For example, the workstation <NUM> may be within a hospital in which imaging devices used to generate image datasets are located, and the server <NUM> may be in a different room within the hospital, may be in a different building within the hospital, may be in a different building outside of the hospital, may be in a different city, may be in a different state, or may be in a different country.

The workstation <NUM> and the server <NUM> may each include a respective memory and processor. In one embodiment, the server <NUM> includes the main memory <NUM>, in which preloaded datasets are stored. The server <NUM> may also include the other memory <NUM> including the database, or the other memory <NUM> is external to and in communication with the server <NUM>. In another embodiment, the workstation <NUM> includes the main memory <NUM> and/or the other memory <NUM>, which includes the database. Preloading datasets may include downloading the datasets from the database of the other memory <NUM>, for example, to the main memory <NUM>. The time this downloading takes may be affected by network bandwidth, bus bandwidth, hardware parameters (e.g., hard drive speed), another parameter, or any combination thereof.

A doctor or a nurse (e.g., a user), for example, uses the workstation <NUM> to access image datasets and/or rendered images (e.g., volumes) preloaded at or provided through the server <NUM> and/or to access image datasets stored in the other memory <NUM>. The workstation <NUM> runs a graphical user interface (GUI) through which the user may request and/or view a list of image datasets stored at the server <NUM> and/or the other memory <NUM>, and/or through which the user may view images generated based on one of the image datasets. The GUI running on the workstation <NUM> may include patient-specific tabs <NUM>, for example, that allow the user to move between viewing images for different patients. As discussed below, the workstation <NUM> may send a request <NUM> to the server <NUM> to view a representation of at least one image dataset already loaded at the server <NUM>. As illustrated in <FIG>, image datasets (e.g., volumes) representing at least a portion of "Patient Jones" and datasets representing at least a portion of "Patient Smith" are already loaded. The datasets being preloaded at the server <NUM> decreases the time between the user request for an image at the workstation <NUM> and the server <NUM> rendering and sending the image to the workstation <NUM> for display, compared to the server <NUM> having to access memory within or outside of the server <NUM> to load and render an image dataset in response to the user request.

<FIG> shows a flowchart of one embodiment of a method <NUM> for preloading data (e.g., image data, a dataset, an image dataset), based on which an image is to be displayed. The image data may, for example, be CT image data or image data generated during rotation of a C-arm during X-ray imaging. Alternatively, the image data may be generated with another imaging device such as, for example, an MRI device. The method may be performed using the imaging system <NUM> shown in <FIG> (e.g., at least some of the acts of the method may be performed by the processor <NUM>) or another imaging system. For example, the acts of the method are implemented by one or more processors using instructions from one or more memories. The method is implemented in the order shown, but other orders may be used. Additional, different, or fewer acts may be provided. Similar methods may be used for preloading image data.

The method <NUM> is performed by a first computing device (e.g., a workstation) and a second computing device (e.g., a server) in communication with the workstation. The workstation may be in communication with the server, for example, via a communication line, a cable, a wireless device, a communication circuit, a network, and/or another communication device. An embodiment of the method <NUM> may be performed by a single computing device.

The workstation and the server may each include a processor, a display (e.g., a monitor), and a memory. Additional, different, or fewer components may be provided. For example, the workstation and the server may each include an input device and/or a network communications interface. In one embodiment, the workstation and/or the server form at least part of the image processing system <NUM> of <FIG>, and each of the workstation and the server may include at least a processor (e.g., the processor <NUM>), a memory (e.g., the memory <NUM>), an input (e.g., the input <NUM>), and an interface.

The memory of the server stores a plurality of image datasets. The plurality of image datasets correspond to different volumes of a single patient, volumes of different patients, datasets generated using different imaging devices, or any combination thereof. The plurality of image datasets may be generated with the imaging device <NUM> of <FIG> and/or one or more other imaging devices. In one embodiment, another memory (e.g., memory of a file system or a database) in communication with the memory of the server (e.g., main memory) stores the plurality of image datasets, and the memory of the server stores preloaded (e.g., retrieved) image datasets and/or representations (e.g., images or rendered volumes) generated based on image datasets. The memory of the server and/or the other memory may store image datasets, as generated by one or more imaging devices, and/or rendered volumes.

In act <NUM>, a processor of the server (e.g., the server <NUM>) predicts at least one dataset to be stored (e.g., loaded) in a memory of the server or another computing device (e.g., in communication with the server) for future rendering and transmission of an image for display at the workstation (e.g., the workstation <NUM>).

In a first embodiment (e.g., batch mode prediction), the processor of the server predicts which dataset of a plurality of datasets stored in the memory of the server, one or more other memories, or any combination thereof is to be used for future image rendering and display of the rendered image at the workstation based on data received from the workstation or otherwise accessed (e.g., retrieved from the workstation by the server). For example, a user of the workstation may select, with an input device (e.g., a mouse and a keyboard), one or more cases (e.g., corresponding to one or more datasets to be displayed) on which the user plans to work. The user may select cases (e.g., corresponding to datasets to be displayed) within a worklist. The worklist may be displayed and cases within the worklist may be selected via, for example, a browser application and/or any other external worklist viewing applications. The selected cases may represent a number of cases the user (e.g., a nurse or doctor) plans on working on over a predetermined amount of time. For example, a doctor may select five cases the doctor plans on reviewing before lunch or before leaving for the day.

The workstation transmits data identifying the selected cases (e.g., corresponding to datasets) to the server. The processor of the server, upon receipt of the data identifying the selected cases, predicts the at least one dataset to be displayed based on the received data. The data identifying the selected cases may identify an order in which the user plans to review the cases. Alternatively, the processor of the server may predict an order in which the user plans to review the cases. For example, the processor may order the selected cases based on an order in which the user selected the cases within the worklist, dates/times the datasets were created, the dataset the user most recently reviewed (e.g., a CT dataset for "Patient Jones"), another parameter, or any combination thereof.

In a second embodiment (e.g., manual prediction), the processor of the server again predicts the datasets based on data received from the workstation. The user, instead of selecting cases within a worklist, however, may identify a group of cases (e.g., corresponding to datasets) to be preloaded via a command within the GUI. In other words, the user may explicitly trigger the loading of the datasets on the server. The workstation transmits data identifying the group of cases (e.g., corresponding to datasets) explicitly selected by the user to the server. The processor of the server, upon receipt of the data identifying the explicitly selected cases, predicts the at least one dataset to be used for future image rendering and display of the rendered image at the workstation based on the received data. The second embodiment, manual prediction mode, may be useful for sessions where different users (e.g., different doctors, nurses, or any combination thereof) are reviewing specific cases (e.g., Oncology Board). Before a review session, the specific cases may be preloaded manually, which is further discussed below.

In a third embodiment (e.g., automatic prediction), the processor of the server automatically predicts which datasets are to be used for future image rendering and display of the rendered image at the workstation. In one example, the server is in communication with a plurality of workstations. The processor of the server analyzes user worklist entries for all users of the respective workstations. The processor may determine the first N entries in the worklists (e.g., corresponding to datasets stored in the memory of the server) until a maximum number of preloaded cases is achieved. The maximum number of preloaded cases may be based on a maximum storage space available (e.g., <NUM> GB). In another embodiment, the processor of the server analyzes study dates/times for datasets that are stored in the memory of the server, for example. The processor predicts that the datasets with most recent dates/times will be rendered displayed at the workstation. In yet another embodiment, the processor of the server uses a neural network that is trained with behaviors of the user to predict datasets to be rendered and displayed at the workstation. The processor of the server may predict the datasets to be rendered and displayed at the workstation based on outputs of the neural network.

In act <NUM>, the memory of the server preloads at least some datasets of the at least one dataset predicted in act <NUM>. One or more datasets of the at least one dataset may be a 3D dataset. The 3D dataset may be multiple gigabytes in size or larger. A redundant array of independent disks (RAID) system (e.g., part of the image processing system <NUM> as part of the server or separate from the server) may store some or all of the datasets to be preloaded at the server. A RAID system of a radiology department, for example, with a typical background load (e.g., preprocessing jobs, DICOM receive jobs) may be operable at a <NUM>-<NUM> MB/s data transfer rate. When the RAID system is idle, the RAID system may be operable at <NUM>-<NUM> MB/s. In other scenarios, the RAID system may be operable at less than <NUM> MB/s. Accordingly, the memory of the server may take minutes to preload the 3D dataset.

For the first prediction embodiment discussed above, the batch mode prediction, the processor and the memory of the server may preload a first dataset of the predicted datasets. The processor and memory may not start the process of preloaded a second dataset of the predicted datasets until the first dataset is rendered and transmitted to the workstation in act <NUM> discussed below. For the second and third prediction embodiments discussed above, the manual prediction and the automatic prediction, the processor and the memory of the server may preload at least a portion of the predicted datasets in parallel or in series, and the processor of the server may preload additional datasets of the predicted datasets when storage space becomes available. Other approaches preloading any number of datasets before or after transmission and/or memory availability may be used.

In one embodiment, only a portion of the memory is reserved for storing preloaded data (e.g., one or more datasets). For example, <NUM>% percent of storage space of the memory may be reserved for storing preloaded datasets (e.g., one or more datasets). This percentage of storage space may be defined as a maximum storage space to be used for preloaded data. Other percentages of storage space may be reserved.

If the maximum storage space is reached, the processor of the server may wait until a large enough portion of the memory of the server is available to preload the next dataset based on the predicted datasets. Once the user of the workstation, for example, requests an image based on a preloaded dataset from the server, receives the rendered image, and closes the rendered image, the processor of the server may delete the preloaded dataset from the memory of the server, for example. In one embodiment, one preloaded dataset is not deleted from the memory of the server, for example, until another preloaded dataset is transmitted from the server to the workstation. Storage space may thus be freed, allowing additional preloaded data to be stored in the memory of the server.

In act <NUM>, the server sends first data to the workstation, and the workstation receives the first data. The first data identifies one or more datasets that are loaded (e.g., generated or rendered) at the server. For example, the first data indicates that the at least one dataset preloaded in act <NUM> is available in the server. In one embodiment, the server sends the first data after the memory of the server, for example, has preloaded the dataset. In another embodiment, the server does not send the first data until all datasets of a group of datasets (e.g., all datasets of the at least one dataset) to be preloaded have been stored (e.g., loaded) in the memory of the server. In the third prediction embodiment discussed above, the computing device sends the first data to the workstation without receiving any requests for datasets to be preloaded from the workstation.

In act <NUM>, the processor of the workstation generates a list of datasets, and the display of the workstation displays the generated list within the GUI. The list of datasets may correspond to the worklist, may include a number of most recent datasets generated for the patient and/or using a particular imaging device, or another subset of the datasets. The datasets represented within the list of datasets displayed at the workstation may be stored in the memory of the server, another memory, or a combination thereof.

<FIG> illustrates one embodiment of a list <NUM> of datasets that may be displayed at the workstation. The list <NUM> may include different information that identifies, for example, a patient that was imaged, a body part that was imaged, an imaging device used, and/or other information. The list <NUM> may represent the worklist the user plans to work through.

The list <NUM> may include a first subset of datasets <NUM> and a second subset of datasets <NUM>. The first subset of datasets <NUM> may be datasets that have been preloaded at the server, and the second subset of datasets <NUM> may be datasets that have not been preloaded at the server. The first subset of datasets <NUM> may be represented differently than the second subset of datasets <NUM>. As shown in the example of <FIG>, datasets of the first subset of datasets are identified with two icons <NUM>, while the datasets of the second subset of datasets <NUM> are identified with a single icon <NUM>. Datasets that have been preloaded at the server may be identified within the list <NUM> in any number of other ways including, for example, with different colors, with different icons, with additional icons, with different text, as a separate list, or any combination thereof.

In act <NUM> of <FIG>, the processor of the workstation identifies second data. The second data identifies one or more datasets (e.g., a dataset) from the list of datasets displayed in act <NUM>. The processor of the workstation may identify the second data based on a user input such as, for example, a mouse and/or a keyboard of the workstation. For example, the user of the workstation may select a dataset from the list of datasets displayed on the display of the workstation. The processor of the workstation generates the second data, which identifies the selected dataset from the list of datasets, based on a location within the GUI the user clicked using, for example, the mouse.

In act <NUM>, the workstation transmits a request for the dataset(s) selected in act <NUM> to the server based on the second data. In act <NUM>, the processor of the server renders a representation of the selected dataset (e.g., an image generated from the selected dataset), and the server transmits the rendered representation of the selected dataset to the workstation. The processor may generate or render the representation of the selected dataset based on additional information sent by the user via the workstation and received by the server. The information may include, for example, an orientation of the representation of the selected dataset and/or a direction from which the representation of the selected dataset is viewed. The processor of the server, in act <NUM>, may generate a 2D projection of the 3D dataset using volume rendering and/or may generate a 3D image based on the 3D dataset. The transmitted image from the selected dataset is received by the workstation and displayed by the display of the workstation.

Claim 1:
A method (<NUM>) of preloading data for the display of medical images based on at least one image dataset, wherein the at least one image dataset stored in a memory (<NUM>) is at least one dataset of a plurality of datasets stored in the memory (<NUM>) with a first (<NUM>) and a second computing device (<NUM>) geographically remote from each other,
wherein the plurality of image datasets are image datasets representing volumes of one or more patients;
the method (<NUM>) comprising:
predicting (<NUM>), by a processor of the second computing device (<NUM>), at least one image dataset to be displayed;
wherein the data identifying the image to be displayed is first data, and wherein
predicting the at least one image to be displayed comprises:
receiving second data from the first computing device; and
identifying the at least one image dataset to be displayed based on the received second data;
retrieving (<NUM>), by the processor of the second computing device (<NUM>), the at least one image dataset from a memory (<NUM>) in communication with the processor;
storing (<NUM>), by another memory (<NUM>) in communication with the memory at the second computing device (<NUM>), the at least one image dataset so as to preload the at least one image dataset in the second computing device (<NUM>);
receiving (<NUM>) at the second computing device (<NUM>), after the predicting, data identifying the image to be displayed, the image to be displayed corresponding to the image dataset of the at least one image dataset;
generating (<NUM>), by the processor of the second computing device (<NUM>), the image to be displayed based on the preloaded at least one image dataset; and
transmitting (<NUM>) the generated image to the first computing device (<NUM>) geographically remote from the processor of the second computing device (<NUM>) and displaying the generated and transmitted image by the first computing device (<NUM>) based on the received data, so that a preloading of data is performed by the second computing device (<NUM>) based on the image which is to be displayed by the first computing device (<NUM>).