Patent Publication Number: US-2011050692-A1

Title: Interpolating and rendering sub-phases of a 4d dataset

Description:
TECHNICAL FIELD 
     This disclosure relates generally to volume rendering, and in particular but not exclusively, relates to animated volume rendering using deformation data. 
     BACKGROUND 
     Volume rendering enables three-dimensional (3D) volumetric data to be visualized. Volumetric data may consist of a 3D array of voxels, each voxel characterized by an intensity value which may scale via a filter function to color and opacity. As such, each voxel may be assigned a color (e.g., one of R (red), G (green), and B (blue)) and opacity, and a 2D projection of the volumetric data may be computed. Using volume rendering techniques, a viewable 2D image may be derived from the 3D volumetric data. 
     Volume rendering is widely used in many applications to derive a viewable 2D image from 3D volumetric data of an object, e.g. a target region within a patient&#39;s anatomy. The 2D image may be a 2D projection through 3D volumetric data to generate digitally reconstructed radiographs facilitating 2D-3D image registration and real-time image-guided radiosurgery referenced to the 3D volumetric data. In medical applications such as radiosurgery, an anatomical target region that moves, due to e.g. heartbeat or breathing of the patient, may need to be tracked. In these cases, a volume rendered animation of periodic motions such as respiration and heartbeat may be desirable. 
     A 3D volume dataset which varies over time may be considered to be a 4D deformable volume image. A number of methods may be used for volume rendering of 4D deformable volume images. These methods may involve one or more of the following approaches: representing a deformable volume using tetrahedrons that have freedom to move in 3D space; using a marching cube algorithm to convert volume rendering to surface rendering by finding small iso-surfaces in non-structural data; representing the volumetric dataset with a procedural mathematical function; or using a multiple volume switching method, in which all the intermediate volumes are generated before rendering. 
     Many of these approaches tend to be very time-consuming, and some require vast stores of memory. Some of these approaches do not lend themselves well for medical images, while some of these approaches are difficult to obtain smooth transitions between stages. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified. 
         FIG. 1  illustrates 3-dimensional (“3D”) images of a volume captured during different phases of deformation of an object within the volume and how voxels within the object migrate from their original coordinates locations during the deformation, in accordance with an embodiment of the invention. 
         FIG. 2A  (Prior Art) illustrates geometric based interpolation between phases of deformation of an object. 
         FIG. 2B  illustrates a content based interpolation between phases of deformation of an object, in accordance with an embodiment of the invention. 
         FIG. 3  is a flow chart illustrating a process for 4-dimensional (4D) volume rendering, in accordance with an embodiment of the invention. 
         FIG. 4  is a functional block diagram illustrating a 4D volume rendering pipeline, in accordance with an embodiment of the invention. 
         FIG. 5  is a flow chart illustrating a process for content based interpolation between consecutive phases of a 4D dataset, in accordance with an embodiment of the invention. 
         FIG. 6  illustrates an example sub-phase interpolation of a single voxel between phases  2  and  3  of a 4D dataset, in accordance with an embodiment of the invention. 
         FIG. 7  is a functional block diagram illustrating a patient treatment system for generating diagnostic images, generating a treatment plan, and delivering the treatment plan, in accordance with an embodiment of the invention. 
         FIG. 8  is a perspective view of a radiation treatment delivery system, in accordance with an embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION 
     Embodiments of a system and method for 4-dimensional (4D) volume rendering are described herein. In the following description numerous specific details are set forth to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects. 
     Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. 
       FIG. 1  illustrates 3-dimensional (“3D”) images  100 A-C of a volume  105  captured during different phases of deformation of an object  110  within volume  105  and how voxels  115  within volume  105  migrate from their original coordinates locations during the deformation.  FIG. 1  may be thought of as illustrating a 4D dataset where the fourth dimension represents time. Each 3D image  100  captures volume  105  at a different time or phase (e.g., phase  1 , phase  2 , and phase  3 ) during the deformation motion of object  110 . Volume  105  may also be referred to a “deformable volume” while the collection of 3D images  100  may be referred to as a “deformable volume dataset.” 
     As illustrated, during each phase voxels  115  migrate or deform from their original coordinate location within volume  105  to a deformed coordinate location. The selection of the “original” coordinate location may be an arbitrary choice, particularly if the deformation motion is a cyclical motion. In the illustrated embodiment, the original coordinate locations are deemed to occur during phase  1  (e.g., time t=1), which is also referred to as the primary volume  105 , primary 3D image  100 A, or primary phase. The subsequent phases (e.g., phases  2  or  3 ) are referred to as deformed phases, deformed volumes  105 , or deformed 3D images  100 . 
     Capturing and rendering a 4D dataset may have a variety of applications. In the medical profession, these operations are useful for image guided radiotherapy procedures. For example, volume  105  may represent a volume within the rib cage of a human patient, while object  110  may represent a heart deforming during the cardiac phases of motion. Alternatively, object  110  may represent a tumor within a patient which deforms during respiratory motion or cardiac motion. Each voxel  115  may represent a 3D data point or pixel including an intensity value. The intensity values may also be represented as RGBA (red, green, blue, alpha) data with the “alpha” representing an opacity value. 
     If the time lapse between consecutive or sequential phases  1 ,  2 ,  3  is sufficiently large, a rendering of object  110  may appear jittery. For image guided radiotherapy applications, it may be desirable to closely track the deformation of object  110 , even between captured 3D images  100 A-C. Embodiments of the invention enables the generation of sub-phases (e.g., sub-phase  1 . 1 , sub-phase  2 . 1 , etc.) interpolated between consecutive phases  100 A-C. By interpolating voxel data between consecutive phases, the number of 3D data images in a 4D dataset can be increased to reduce jitter and increase the quality and accuracy of the 4D dataset. 
       FIG. 2A  illustrates a conventional geometric based interpolation technique between phases of deformation of an object  205 . Phase  1  and phase  2  represent 3D images that were captured during the deformation motion of object  205 . This data may be captured using X-ray imaging, computed tomography (“CT”), magnetic resonance imaging (“MRI”), positron emission tomography (“PET”), or other 3D imaging modalities. As such, the captured 3D images have voxels  210  evenly spaced throughout the volume  215  during phases  1  and  2 . However, the geometric based interpolation uses geometric principles to interpolate voxels  220  of sub-phase  1 . 1 . The geometric principles do not preserve isotropic voxel spacing. Rather, some regions within the interpolated volume  225  end up with a thinning of interpolated voxels  220  resulting in low resolution localities. It may be undesirable, if the deformable object of interest (e.g., tumor) happens to fall within a low resolution locality of the interpolated volume  225 . The conventional geometric based interpolation is forward looking, using voxels from future phases to interpolate voxels of an earlier in time sub-phase. 
       FIG. 2B  illustrates a content based interpolation between consecutive phases of deformation of object  230 , in accordance with an embodiment of the invention. Again, phase  1  and phase  2  represent 3D images that were captured during the deformation motion of object  230 . This data may be captured using X-ray imaging, CT scanning, MRI imaging, PET scanning or other 3D imaging modalities. As such, the captured 3D images have voxels  235  evenly spaced throughout the volume  240  during phases  1  and  2 . However, the content based interpolation uses the content or dataset itself to interpolate voxels  245  of sub-phase  1 . 1 . The content interpolation preserves isotropic voxel spacing, as discussed in detail below in connection with  FIGS. 5 and 6 . As such, the interpolated deformed volume  250  is significantly more uniform and substantially eliminates high and low resolution localities. Rather, the voxel resolution, pitch, or spacing is substantially consistent throughout the interpolated volume providing a high quality, more uniform interpolated dataset. Content based interpolation is backward looking, using voxels from an earlier in-time phase (e.g., phase  1 ) to interpolate voxels in a future occurring sub-phase (e.g., sub-phase  1 . 1 ). 
       FIG. 3  is a flow chart illustrating a process  300  for 4D volume rendering, in accordance with an embodiment of the invention. Process  300  is explained with reference to  FIG. 4 . The order in which some or all of the process blocks appear in process  300  should not be deemed limiting. Rather, one of ordinary skill in the art having the benefit of the present disclosure will understand that some of the process blocks may be executed in a variety of orders not illustrated. 
     In process block  305 , 3D images  405  of a deformable volume including an object in deformation motion are acquired. 3D images  405  may be acquired using a variety of imaging modalities, such as, CT scanning, x-ray imaging, MRI imaging, PET scanning, or otherwise. One of the 3D images  405  is selected or designated as the primary 3D image or primary volume. In one embodiment, the primary 3D image is scanned or otherwise acquired to have a higher voxel resolution than the other 3D images. 
     In process block  310 , a deformation computation is executed to generate deformation matrixes  410 . In one embodiment, a separate deformation matrix  410  is generated for each 3D image  405 , including the primary 3D image. In one embodiment, deformation matrixes  410  are each a matrix of transformation vectors. Each transformation vector corresponds to a given voxel within its corresponding 3D image  405 . In other words, each transformation vector corresponds to a voxel at a deformed coordinate location within volume  105  or  240 . Furthermore, each transformation vector describes, with reference to the primary 3D image, from where the given voxel was deformed. In other words, each transformation vector describes how to return a given voxel from its deformed coordinate location to its original coordinate location within the primary 3D image prior to the deformation motion. In one embodiment, deformation matrixes  410  are 3D matrixes of transformation vectors, which are in turn deformation coordinates (e.g., X, Y, Z coordinates, polar coordinates, or otherwise) that reference or point back to their original coordinate location within primary 3D image  405 . For example, a transformation vector located at deformed coordinate location (1,2,1) of deformation matrix  410 C having a value of (1,3,0) indicates that the voxel at deformed coordinate location (1,2,1) in 3D image  405 B originated at original coordinate location 1,3,0 in primary 3D image  405 . In one embodiment, deformation matrix  410 A is an identify matrix, since deformation matrix  410 A corresponds to primary 3D image  405 . The deformation computation may be executed to generate deformation matrixes  410  using a variety of algorithms, such as and without limitation, B-Spline transformations, linear transformations. 
     In process block  315 , image filtering is applied to primary 3D image  405  to generate primary 3D texture  415 . For example and without limitation, the filter function applied by the image filtering transforms intensity values of primary 3D image  405  into RGBA values and filters out portions of 3D image  405  not of interest. The remaining RGBA values not filtered out by the filter function may correspond to a texture surface of interest. For example, if primary 3D image  405  is an image of a human anatomy, image filtering may be used to filter out soft tissue to display bone structure, or filter out healthy tissue to display a tumor. In one embodiment, the filter function is executed by a CPU. In another embodiment, the filter function is executed by a GPU after primary 3D image  405  is transferred into a GPU memory buffer. In yet another embodiment, the intensity values of primary 3D image  405  are not converted into RGBA values at all; but rather, the intensity values themselves are filtered for values or textures of interest and primary 3D texture  415  is represented as textured or filtered intensity values, as opposed to textured or filtered RGBA values. In yet another embodiment, the unfiltered intensity values of primary 3D image  405  are transferred straight into the GPU without modification. 
     In process block  320 , deformation volume textures  420  are generated based on their corresponding deformation matrixes  410 . In one embodiment, deformation volume textures  420  are created by transferring the deformation coordinates of each transformation vector into an RGB buffer of the GPU. In this embodiment, the three X, Y, Z coordinates of the transformation vector are stored into the three R, G, B, buffers, respectively (or some combination thereof). Loading the deformation coordinates into RGB buffers of the GPU leverages the parallelism of the GPU (e.g., a GPU may include 128 parallel image processors capable of operating on 128 transformation vectors in parallel) to provide hardware acceleration of the texturing and rendering process. 
     Once primary 3D texture  415  and the first set of transformation vectors have been loaded into the RGB buffers of the GPU, then rendering of phase (i) can commence. It should be appreciated that the GPU operates on one deformation matrix  410  at a time. For example, in the case of a GPU having a 128 parallel image processors, the first 128 transformation vectors of deformation matrix  410 A are loaded into the GPU, processed, and the results loaded into a frame buffer for eventual rendering to a display. Then, the next 128 transformation vectors of deformation matrix  410 A are loaded, processed, and output to the frame buffer, and so on, until an entire 3D image has been processed and loaded into the frame buffer of the GPU and ready for rendering (process block  325 ). Again, it should be appreciated that all processing steps described above, except for the final rendering, can be executed by the CPU, though without leveraging the parallelism provided by the GPU. 
     To increase the number of 3D image frames in a 4D dataset, one or more sub-phases may be interpolated between consecutive phases (i) using the content of deformation volume textures  420  (or deformation matrixes  410 ) themselves. In a process block  330 , interpolation between the transformation vectors associated with a given voxel at a given deformation coordinate location for both phases (i) and (i+1) is executed. In one embodiment, this interpolation is executed by taking a weighted average of the transformation vectors at the given deformation coordinate locations for the two consecutive phases. This interpolation is repeated for all voxels within the deformation volume textures of a pair of consecutive phases (i) and (i+1). Once a complete deformation volume texture of a given sub-phase has been completed, it may be rendered from the frame buffer (process block  335 ). 
     As mentioned above, multiple sub-phases may be interpolated between consecutive phases by adjusting a weighting factor, step (j), for each sub-phase interpolated (process block  345 ). The weighting factor skews the interpolation bias in selected increments (e.g., for ten sub-phases, the weighting factor could be incremented from 0 to 1.0 in 0.1 increments for each sub-phase) from the earlier phase to the later phase of the two consecutive phases. The interpolation, rendering, and weighting factor are described in further detail below in connection with  FIGS. 5 and 6 . 
     Once all the sub-phases between a given set of consecutive phases (i) and (i+1) have been interpolated (decision block  340 ), process  300  increments to the next set of consecutive phases (i+1) and (i+2) (process block  355 ) to interpolate sub-phases there between as described above. Finally, once the entire 4D dataset has been rendered, including all sub-phases interpolated and rendered (decision block  350 ), process  300  is completed at termination block  360 . 
       FIG. 5  is a flow chart illustrating a process  500  for content based interpolation between consecutive phases of a 4D dataset, in accordance with an embodiment of the invention. Process  500  is explained with reference to  FIG. 6 .  FIG. 6  illustrates the interpolation of a sub-phase  2 . 1  between phase  2  and phase  3  of a 4D dataset. The order in which some or all of the process blocks appear in process  500  should not be deemed limiting. Rather, one of ordinary skill in the art having the benefit of the present disclosure will understand that some of the process blocks may be executed in a variety of orders not illustrated. 
     In process block  505 , a voxel coordinate (X 0 , Y 0 , Z 0 ) is selected for interpolation. In the example of  FIG. 6 , the selected voxel is the voxel at coordinate location (1,3,0). In process block  510 , the transformation vectors at the selected voxel coordinate are retrieved from the two consecutive deformation volume textures  420  (or deformation matrixes  410  if interpolation is performed by the CPU) of phases (i) and (i+1). In accordance with the example illustrated in  FIG. 6 , the two transformation vectors, which are located at voxel coordinate (1,3,0) within deformation volume textures  420 B and  420 C are fetched (see  FIG. 4 ). In process block  515 , the weighting factor, step (j), is selected for the instant sub-phase  2 . 1  being interpolated. Step (j) is a value between 0 and 1 and is incremented for each sub-phase interpolated between consecutive phases (i) and (i+1). For example, for sub-phase  2 . 1 , step (j) may equal 0.1. 
     In process block  520 , the two transformation vectors from the consecutive phases are interpolated to generate an interpolated transformation vector. In one embodiment, interpolation is performed according to equation (1), 
       ( X   d   ,Y   d   ,Z   d )=step( j )*( X   i   ,Y   i   ,Z   i )+(1−step( j ))*( X   i+1   ,Y   i+1   ,Z   i+1 ),  (1)
 
     where (X d , Y d , Z d ) represents the interpolated transformation vector for the selected coordinate location (X 0 , Y 0 , Z 0 ), step(j) represents the weighting factor, (X i , Y i , Z i ) represents the transformation vector at coordinate location (X 0 , Y 0 , Z 0 ) for phase (i), and (X i+1 , Y i+1 , Z i+1 ) represents the transformation vector at coordinate location (X 0 , Y 0 , Z 0 ) for phase (i+1). Referring to  FIG. 6 , (X i , Y i , Z i ) corresponds to transformation vector  605  and (X i+1 , Y i+1 , Z i+1 ) corresponds to transformation vector  610 , and (X d , Y d , Z d ) corresponds to interpolated transformation vector  615 . 
     Once the interpolated transformation vector has been generated, voxel data is retrieved from primary 3D texture  415  with reference to the interpolated transformation vector (process block  525 ). However, if the interpolated transformation vector references back to an intermediate position between adjacent voxels within primary 3D texture  415 , then a second level of interpolation may be executed (decision block  522 ). In this situation, the voxel data associated with the adjacent voxels within primary 3D texture  415  is interpolated to obtain averaged voxel data, which is then retrieved in process block  525 . Referring to the example of  FIG. 6 , if interpolated transformation vector  615  referenced back to a midpoint between coordinate location (1,3,0) and (2,3,0), then the voxel data associated with the voxels at these two coordinate locations may be simply averaged. In other embodiments, a weighted average or continuous interpolation between horizontally, vertically, and diagonally adjacent voxels may be implemented. Of course, other types of interpolation or averaging may be implemented between adjacent voxels. 
     In one embodiment, the voxel data is texture data (e.g., RGBA values) pointed to by the interpolated transformation vector and retrieved from primary 3D texture  415 . In an embodiment wherein primary 3D image  405  is not filtered, the voxel data is an intensity value pointed to by the interpolated transformation vector and retrieved from primary 3D image  405  in process block  525 . 
     Referring again to the example of  FIG. 6 , it can be seen that transformation vector  605  of phase  2  originates from a coordinate location (1,3,0) during phase  2  and references back to the same coordinate location (1,3,0) within phase  1 . This means that the voxel at coordinate location (1,3,0) during phase  1  did not move during the deformation motion that occurred between phases  1  and  2 . However, transformation vector  610  of phase  3  also originates from the coordinate location (1,3,0) within the deformation volume during phase  3 , but references back to coordinate location (3,3,0) in phase  1 . This means that the voxel originally located at coordinate location (3,3,0) has deformed or migrated to coordinate location (1,3,0) by phase  3  of the deformation motion. By interpolating between (e.g., taking a weighted average) transformation vectors  605  and  610 , it is determined that the voxel at coordinate location (1,3,0) during the interpolated sub-phase  2 . 1  may have originated at the intermediate coordinate location (2,3,0) during phase  1 . Once interpolated transformation vector  615  is generated, it is referenced to fetch the texture data or intensity value associate with the voxel originally located at coordinate location (2,3,0) during phase  1 . This is illustrated graphically for all three phases  2 ,  2 . 1 , and  3  with shading. It should be appreciated that in some embodiments, primary 3D image  405  or primary 3D texture  415  may have significantly higher resolutions than the other 3D images to improve the granularity of the sub-phase interpolations. 
     In a process block  530 , the retrieved voxel data is written into the frame buffer for the current sub-phase being generated at the voxel coordinate selected in process block  505 . Process  500  continues to loop for each voxel coordinate within the volume  250  (process block  540 ) until all voxel data has been retrieved and the frame buffer filled with a complete 3D image (decision block  535 ). Finally, in a process block  545  the current sub-phase (j) can be rendered from the frame buffer to a display screen. 
       FIG. 7  is a block diagram illustrating a therapeutic patient treatment system  4000  for generating diagnostic images, generating a treatment plan, and delivering the treatment plan to a patient, in which features of the present invention may be implemented. As described below and illustrated in  FIG. 7 , system  4000  may include a diagnostic imaging system  1000 , a treatment planning system  2000  and a radiation delivery system  3000 . 
     Diagnostic imaging system  1000  may be any system capable of producing medical diagnostic images of the VOI within a patient that may be used for subsequent medical diagnosis, treatment planning and/or treatment delivery. For example, diagnostic imaging system  1000  may be a computed tomography (“CT”) system, a magnetic resonance imaging (“MRI”) system, a positron emission tomography (“PET”) system, an ultrasound system or the like. For ease of discussion, diagnostic imaging system  1000  may be discussed below at times in relation to a CT x-ray imaging modality. However, other imaging modalities such as those above may also be used. In one embodiment, diagnostic imaging system  1000  may be used to generate 3D dataset  235 . 
     Diagnostic imaging system  1000  includes an imaging source  1010  to generate an imaging beam (e.g., x-rays, ultrasonic waves, radio frequency waves, etc.) and an imaging detector  1020  to detect and receive the beam generated by imaging source  1010 , or a secondary beam or emission stimulated by the beam from the imaging source (e.g., in an MRI or PET scan). In one embodiment, diagnostic imaging system  1000  may include two or more diagnostic X-ray sources and two or more corresponding imaging detectors. For example, two x-ray sources may be disposed around a patient to be imaged, fixed at an angular separation from each other (e.g., 90 degrees, 45 degrees, etc.) and aimed through the patient toward (an) imaging detector(s) which may be diametrically opposed to the x-ray sources. A single large imaging detector, or multiple imaging detectors, can also be used that would be illuminated by each x-ray imaging source. Alternatively, other numbers and configurations of imaging sources and imaging detectors may be used. 
     The imaging source  1010  and the imaging detector  1020  are coupled to a digital processing system  1030  to control the imaging operation and process image data. Diagnostic imaging system  1000  includes a bus or other means  1035  for transferring data and commands among digital processing system  1030 , imaging source  1010  and imaging detector  1020 . Digital processing system  1030  may include one or more general-purpose processors (e.g., a microprocessor), special purpose processor such as a digital signal processor (“DSP”) or other type of device such as a controller or field programmable gate array (“FPGA”). Digital processing system  1030  may also include other components (not shown) such as memory, storage devices, network adapters and the like. Digital processing system  1030  may be configured to generate digital diagnostic images in a standard format, such as the DICOM (Digital Imaging and Communications in Medicine) format, for example. In other embodiments, digital processing system  1030  may generate other standard or non-standard digital image formats. Digital processing system  1030  may transmit diagnostic image files (e.g., the aforementioned DICOM formatted files) to treatment planning system  2000  over a data link  1500 , which may be, for example, a direct link, a local area network (“LAN”) link or a wide area network (“WAN”) link such as the Internet. In addition, the information transferred between systems may either be pulled or pushed across the communication medium connecting the systems, such as in a remote diagnosis or treatment planning configuration. In remote diagnosis or treatment planning, a user may utilize embodiments of the present invention to diagnose or treatment plan despite the existence of a physical separation between the system user and the patient. 
     Treatment planning system  2000  includes a processing device  2010  to receive and process image data. Processing device  2010  may represent one or more general-purpose processors (e.g., a microprocessor), special purpose processor such as a DSP or other type of device such as a controller or FPGA. Processing device  2010  may be configured to execute instructions for performing treatment planning operations discussed herein. 
     Treatment planning system  2000  may also include system memory  2020  that may include a random access memory (“RAM”), or other dynamic storage devices, coupled to processing device  2010  by bus  2055 , for storing information and instructions to be executed by processing device  2010 . System memory  2020  also may be used for storing temporary variables or other intermediate information during execution of instructions by processing device  2010 . System memory  2020  may also include a read only memory (“ROM”) and/or other static storage device coupled to bus  2055  for storing static information and instructions for processing device  2010 . 
     Treatment planning system  2000  may also include storage device  2030 , representing one or more storage devices (e.g., a magnetic disk drive or optical disk drive) coupled to bus  2055  for storing information and instructions. Storage device  2030  may be used for storing instructions for performing the treatment planning steps discussed herein. 
     Processing device  2010  may also be coupled to a display device  2040 , such as a cathode ray tube (“CRT”) or liquid crystal display (“LCD”), for displaying information (e.g., a 2D or 3D representation of the VOI) to the user. An input device  2050 , such as a keyboard, may be coupled to processing device  2010  for communicating information and/or command selections to processing device  2010 . One or more other user input devices (e.g., a mouse, a trackball or cursor direction keys) may also be used to communicate directional information, to select commands for processing device  2010  and to control cursor movements on display  2040 . 
     It will be appreciated that treatment planning system  2000  represents only one example of a treatment planning system, which may have many different configurations and architectures, which may include more components or fewer components than treatment planning system  2000  and which may be employed with the present invention. For example, some systems often have multiple buses, such as a peripheral bus, a dedicated cache bus, etc. The treatment planning system  2000  may also include MIRIT (Medical Image Review and Import Tool) to support DICOM import (so images can be fused and targets delineated on different systems and then imported into the treatment planning system for planning and dose calculations), expanded image fusion capabilities that allow the user to treatment plan and view dose distributions on any one of various imaging modalities (e.g., MRI, CT, PET, etc.). Treatment planning systems are known in the art; accordingly, a more detailed discussion is not provided. 
     Treatment planning system  2000  may share its database (e.g., data stored in storage device  2030 ) with a treatment delivery system, such as radiation treatment delivery system  3000 , so that it may not be necessary to export from the treatment planning system prior to treatment delivery. Treatment planning system  2000  may be linked to radiation treatment delivery system  3000  via a data link  2500 , which may be a direct link, a LAN link or a WAN link as discussed above with respect to data link  1500 . It should be noted that when data links  1500  and  2500  are implemented as LAN or WAN connections, any of diagnostic imaging system  1000 , treatment planning system  2000  and/or radiation treatment delivery system  3000  may be in decentralized locations such that the systems may be physically remote from each other. Alternatively, any of diagnostic imaging system  1000 , treatment planning system  2000  and/or radiation treatment delivery system  3000  may be integrated with each other in one or more systems. 
     Radiation treatment delivery system  3000  includes a therapeutic and/or surgical radiation source  3010  to administer a prescribed radiation dose to a target volume in conformance with a treatment plan. Radiation treatment delivery system  3000  may also include an imaging system  3020  (including imaging sources  3021  and detectors  3022 , see  FIG. 8 ) to capture inter-treatment images of a patient volume (including the target volume) for registration or correlation with the diagnostic images (e.g., DRR image  205 ) described above in order to position the patient with respect to the radiation source. Radiation treatment delivery system  3000  may also include a digital processing system  3030  to control radiation source  3010 , imaging system  3020 , and a patient support device such as a treatment couch  3040 . Digital processing system  3030  may include one or more general-purpose processors (e.g., a microprocessor), special purpose processor such as a DSP or other type of device such as a controller or FPGA. Digital processing system  3030  may also include other components (not shown) such as memory, storage devices, network adapters and the like. Digital processing system  3030  may be coupled to radiation source  3010 , imaging system  3020  and treatment couch  3040  by a bus  3045  or other type of control and communication interface. 
       FIG. 8  is a perspective view of a radiation delivery system  3000 , in accordance with an embodiment of the invention. In one embodiment, radiation treatment delivery system  3000  may be an image-guided, robotic-based radiation treatment system such as the CyberKnife® system developed by Accuray, Inc. of California. In  FIG. 8 , radiation source  3010  may be a linear accelerator (“LINAC”) mounted on the end of a source positioning system  3012  (e.g., robotic arm) having multiple (e.g., 5 or more) degrees of freedom in order to position the LINAC to irradiate a pathological anatomy (target region or volume) with beams delivered from many angles in an operating volume (e.g., a sphere) around the patient. Treatment may involve beam paths with a single isocenter (point of convergence), multiple isocenters, or with a non-isocentric approach (i.e., the beams need only intersect with the pathological target volume and do not necessarily converge on a single point, or isocenter, within the target). Treatment can be delivered in either a single session (mono-fraction) or in a small number of sessions (hypo-fractionation) as determined during treatment planning. With radiation treatment delivery system  3000 , in one embodiment, radiation beams may be delivered according to the treatment plan without fixing the patient to a rigid, external frame to register the intra-operative position of the target volume with the position of the target volume during the pre-operative treatment planning phase. 
     Imaging system  3020  (see  FIG. 7 ) may be represented by imaging sources  3021 A and  3021 B and imaging detectors (imagers)  3022 A and  3022 B in  FIG. 8 . In one embodiment, imaging sources  3021 A and  3021 B are X-ray sources. In one embodiment, for example, two imaging sources  3021 A and  3021 B may be nominally aligned to project imaging x-ray beams through a patient from two different angular positions (e.g., separated by 90 degrees, 45 degrees, etc.) and aimed through the patient on treatment couch  3040  toward respective detectors  3022 A and  3022 B. In another embodiment, a single large imager can be used that would be illuminated by each x-ray imaging source. Alternatively, other numbers and configurations of imaging sources and detectors may be used. 
     Digital processing system  3030  may implement algorithms to register images obtained from imaging system  3020  with pre-operative treatment planning images (e.g., DRR image  205 ) in order to align the patient on the treatment couch  3040  within radiation treatment delivery system  3000 , and to precisely position radiation source  3010  with respect to the target volume. Embodiments of the present invention may use the 4D imaging and interpolation techniques described above to aid in the image guidance and tracking of radiation treatment delivery system  3000 . 
     In the illustrated embodiment, treatment couch  3040  is coupled to a couch positioning system  3013  (e.g., robotic couch arm) having multiple (e.g., 5 or more) degrees of freedom. Couch positioning system  3013  may have five rotational degrees of freedom and one substantially vertical, linear degree of freedom. Alternatively, couch positioning system  3013  may have six rotational degrees of freedom and one substantially vertical, linear degree of freedom or at least four rotational degrees of freedom. Couch positioning system  3013  may be vertically mounted to a column or wall, or horizontally mounted to pedestal, floor, or ceiling. Alternatively, treatment couch  3040  may be a component of another mechanical mechanism, such as the Axum™ treatment couch developed by Accuray, Inc. of California, or be another type of conventional treatment table known to those of ordinary skill in the art. 
     Alternatively, radiation treatment delivery system  3000  may be another type of treatment delivery system, for example, a gantry based (isocentric) intensity modulated radiotherapy (“IMRT”) system or 3D conformal radiation treatments. In a gantry based system, a therapeutic radiation source (e.g., a LINAC) is mounted on the gantry in such a way that it rotates in a plane corresponding to an axial slice of the patient. Radiation is then delivered from several positions on the circular plane of rotation. In IMRT, the shape of the radiation beam is defined by a multi-leaf collimator that allows portions of the beam to be blocked, so that the remaining beam incident on the patient has a pre-defined shape. The resulting system generates arbitrarily shaped radiation beams that intersect each other at the isocenter to deliver a dose distribution to the target. In IMRT planning, the optimization algorithm selects subsets of the main beam and determines the amount of time that the patient should be exposed to each subset, so that the prescribed dose constraints are best met. 
     It should be noted that the methods and apparatus described herein are not limited to use only with medical diagnostic imaging and treatment. In alternative embodiments, the methods and apparatus herein may be used in applications outside of the medical technology field, such as industrial imaging and non-destructive testing of materials (e.g., motor blocks in the automotive industry, airframes in the aviation industry, welds in the construction industry and drill cores in the petroleum industry) and seismic surveying. In such applications, for example, “treatment” may refer generally to the application of radiation beam(s). 
     The processes explained above are described in terms of computer software and hardware. The techniques described may constitute machine-executable instructions embodied within a machine (e.g., computer) readable storage medium, that when executed by a machine will cause the machine to perform the operations described. Additionally, the processes may be embodied within hardware, such as an application specific integrated circuit (“ASIC”) or the like. 
     A computer-readable storage medium includes any mechanism that provides (i.e., stores) information in a form accessible by a machine (e.g., a computer, network device, personal digital assistant, manufacturing tool, any device with a set of one or more processors, etc.). For example, a computer-readable storage medium includes recordable/non-recordable media (e.g., read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.). 
     The above description of illustrated embodiments of the invention, including what is described in the Abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize. 
     These modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification. Rather, the scope of the invention is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation.