Abstract:
The invention relates to a system ( 100 ) for producing a representation of an object in image data, based on a template coupled to a model of the object, the system comprising a model unit for adapting the model to the object in the image data, and a template unit for adapting the template to the adapted model on the basis of the coupling between the template and the model. The template defines a representation of the object which is simpler to interpret than the model. The template may be arranged to emphasize useful features of the object. The template comprises substantially fewer degrees of freedom and thus can be efficiently adapted to the model. Because the template of the invention is coupled to the model, the position, orientation and/or shape of the template is determined by the model adapted to the object in the image data. Hence, the template is adapted to the image data. The adapted template is capable of representing the object and its individual characteristics, e.g., the shape of the object as well as the position and/or orientation of the object with respect to an external reference system defined, e.g., based on the image data.

Description:
FIELD OF THE INVENTION 
       [0001]    The invention relates to representing an object in the image data, using a simplified or schematic model of the object, and, in a particular aspect of the invention, to representing the object in a simplified or schematic image. 
       BACKGROUND OF THE INVENTION 
       [0002]    In model-based image data segmentation, results of the segmentation comprise a model adapted to the image data. Such a model typically includes many details. Therefore, when viewing the model adapted to the image data, said model cannot be easily analyzed on a display or in a report. Adapting a simplified model representing and possibly emphasizing an aspect of a structure of interest to the image data is not feasible because a simplified model does not include enough detail to be successfully adapted to the image data. On the other hand, the simplified model which is not adapted to the structure of interest does not include the structure-specific information. 
       SUMMARY OF THE INVENTION 
       [0003]    It would be advantageous to have a system that is capable of adapting a simplified model representing an aspect of a structure of interest to an object in the image data. 
         [0004]    Thus, in an aspect of the invention, there is provided a system for producing a representation of an object in image data, based on a template coupled to a model of the object, the system comprising:
       a model unit for adapting the model to the object in the image data; and   a template unit for adapting the template to the adapted model on the basis of the coupling between the template and the model.       
 
         [0007]    The template defines a representation of the object which is simpler to interpret than the model. The template may be arranged to emphasize interesting features of the object. The template comprises substantially fewer degrees of freedom and thus can be efficiently adapted to the model. Since the template of the invention is coupled to the model, the position, orientation and/or shape of the template is determined by the model adapted to the object in the image data. Hence, indirectly, the template is adapted to the image data. The adapted template is capable of representing the object and its individual characteristics, e.g., the shape of the object as well as the position and/or orientation of the object with respect to an external, i.e., independent of the adapted model, reference system. Such an external reference system may be defined, e.g., based on the image data. 
         [0008]    In an embodiment, the system further comprises an image unit for producing an image of the template. Thus, individual features of the object in the image data described by the template may be further shown in the image produced from the template. 
         [0009]    In an embodiment of the system, the template comprises a first portion and a second portion, and the image unit is arranged for producing a first portion of the image, based on the first portion of the template, using a first visual pattern, and a second portion of the image, based on the second portion of the template, using a second visual pattern. Using different visual patterns, e.g. colors or shading, one can easily see how different portions of the object are depicted in the image. 
         [0010]    In an embodiment, the template is rigid and is movably coupled to the model. For example, the template may be planar and the plane of the template may be defined by three non co-linear points defined by features of the model. The template unit may be arranged for adapting the planar template to the model such that (i) the plane of the template is determined by the three points of the adapted model, (ii) the center of the template is determined by the mass center of the three points, and (iii) an axis of the template is determined by the line obtained by applying linear regression to the three points of the adapted model. 
         [0011]    In an embodiment of the system, the template comprises a plurality of control points which are rigidly or elastically coupled to the model. For example, the positions of the plurality of control points may be based on features of the model. In the case of rigid coupling, the coordinates of the control points are fixed with respect to the model. In the case of elastic coupling, the positions of the plurality of control points are determined by their elastic interaction with the model. The elastic interaction may be described by elastic forces such as harmonic forces. The template may be a curve or a surface defined by the control points using, e.g. interpolation. The skilled person will understand that using non-elastic coupling is, in principle, also possible. 
         [0012]    In an embodiment of the system, the template is elastic. For example, the template may be implemented as a mesh comprising a plurality of nodes. The neighboring nodes may interact with each other via elastic forces. Elastic forces are easy to implement and compute. Further, elastic forces properly describe the expected deformation of the template resulting from the deformation of the model of the object. However, a person skilled in the art will appreciate that in an alternative embodiment, some or all nodes may interact with each other via non-elastic forces. 
         [0013]    In an embodiment of the system, the template comprises at least one curve or surface. For example, the template may comprise a curve or a surface which is defined, based on a section of the surface model, by a plane. 
         [0014]    In a further aspect of the invention, the system is comprised in a reporting system for creating a report, the report comprising the image of the template produced by the image unit of the system. 
         [0015]    In a further aspect of the invention, the system is comprised in an image acquisition apparatus. 
         [0016]    In a further aspect of the invention, the system is comprised in a workstation. In a further aspect of the invention, there is provided a method of producing a representation of an object in image data, based on a template coupled to a model of the object, the method comprising:
       a model step for adapting the model to the object in the image data; and   a template step for adapting the template to the adapted model on the basis of the coupling between the template and the model.       
 
         [0019]    In a further aspect of the invention, a computer program product to be loaded by a computer arrangement is provided, the computer program product comprising instructions for producing a representation of an object in image data, based on a template coupled to a model of the object, the computer arrangement comprising a processing unit and a memory, the computer program product, after being loaded, providing said processing unit with the capability to carry out the tasks of:
       adapting the model to the object in the image data,   adapting the template to the adapted model on the basis of the coupling between the template and the model.       
 
         [0022]    It will be appreciated by those skilled in the art that two or more of the above-mentioned embodiments, implementations, and/or aspects of the invention may be combined in any way deemed useful. 
         [0023]    Modifications and variations of the reporting system, of the image acquisition apparatus, of the workstation, of the method, and/or of the computer program product, which correspond to the described modifications and variations of the system, can be carried out by a person skilled in the art on the basis of the present description. 
         [0024]    A person skilled in the art will appreciate that the method may be applied to multidimensional image data, e.g., to 3-dimensional or 4-dimensional images, acquired by various acquisition modalities such as, but not limited to, standard X-ray Imaging, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), Ultrasound (US), Positron Emission Tomography (PET), Single Photon Emission Computed Tomography (SPECT), and Nuclear Medicine (NM). 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0025]    These and other aspects of the invention will become apparent from and will be elucidated with respect to the implementations and embodiments described hereinafter and with reference to the accompanying drawings, wherein: 
           [0026]      FIG. 1  schematically shows a block diagram of an exemplary embodiment of the system; 
           [0027]      FIG. 2  illustrates the template before (a) and after (b) adaptation by the system; 
           [0028]      FIG. 3  schematically shows an exemplary embodiment of the reporting system; 
           [0029]      FIG. 4  shows a flowchart of an exemplary implementation of the method; 
           [0030]      FIG. 5  schematically shows an exemplary embodiment of the image acquisition apparatus; and 
           [0031]      FIG. 6  schematically shows an exemplary embodiment of the workstation. 
       
    
    
       [0032]    Identical reference numerals are used to denote similar parts throughout the Figures. 
       DETAILED DESCRIPTION OF EMBODIMENTS 
       [0033]      FIG. 1  schematically shows a block diagram of an exemplary embodiment of the system  100  for producing a representation of an object in image data, based on a template coupled to a model of the object, the system comprising:
       a model unit  110  for adapting the model to the object in the image data; and   a template unit  120  for adapting the template to the adapted model on the basis of the coupling between the template and the model.       
 
         [0036]    The exemplary embodiment of the system  100  further comprises the following units:
       an image unit  130  for producing an image of the template;   a control unit  160  for controlling the workflow in the system  100 ;   a user interface  165  for communicating with a user of the system  100 ; and   a memory unit  170  for storing data.       
 
         [0041]    In an embodiment of the system  100 , there are three input connectors  181 ,  182  and  183  for the incoming data. The first input connector  181  is arranged to receive data coming in from a data storage means such as, but not limited to, a hard disk, a magnetic tape, a flash memory, or an optical disk. The second input connector  182  is arranged to receive data coming in from a user input device such as, but not limited to, a mouse or a touch screen. The third input connector  183  is arranged to receive data coming in from a user input device such as a keyboard. The input connectors  181 ,  182  and  183  are connected to an input control unit  180 . 
         [0042]    In an embodiment of the system  100 , there are two output connectors  191  and  192  for the outgoing data. The first output connector  191  is arranged to output the data to a data storage means such as a hard disk, a magnetic tape, a flash memory, or an optical disk. The second output connector  192  is arranged to output the data to a display device. The output connectors  191  and  192  receive the respective data via an output control unit  190 . 
         [0043]    A person skilled in the art will understand that there are many ways to connect input devices to the input connectors  181 ,  182  and  183  and the output devices to the output connectors  191  and  192  of the system  100 . These ways comprise, but are not limited to, a wired and a wireless connection, a digital network such as, but not limited to, a Local Area Network (LAN) and a Wide Area Network (WAN), the Internet, a digital telephone network, and an analog telephone network. 
         [0044]    In an embodiment of the system  100 , the system  100  comprises a memory unit  170 . The system  100  is arranged to receive input data from external devices via any of the input connectors  181 ,  182 , and  183  and to store the received input data in the memory unit  170 . Loading the input data into the memory unit  170  allows quick access to relevant data portions by the units of the system  100 . The input data may comprise, for example, the image data. The memory unit  170  may be implemented by devices such as, but not limited to, a Random Access Memory (RAM) chip, a Read Only Memory (ROM) chip, and/or a hard disk drive and a hard disk. The memory unit  170  may be further arranged to store the output data. The output data may comprise, for example, the adapted template data. The memory unit  170  may be also arranged to receive data from and/or deliver data to the units of the system  100  comprising the model unit  110 , the template unit  120 , the image unit  130 , the control unit  160 , and the user interface  165 , via a memory bus  175 . The memory unit  170  is further arranged to make the output data available to external devices via any of the output connectors  191  and  192 . Storing data from the units of the system  100  in the memory unit  170  may advantageously improve performance of the units of the system  100  as well as the rate of transfer of the output data from the units of the system  100  to external devices. 
         [0045]    Alternatively, the system  100  may comprise no memory unit  170  and no memory bus  175 . The input data used by the system  100  may be supplied by at least one external device, such as an external memory or a processor, connected to the units of the system  100 . Similarly, the output data produced by the system  100  may be supplied to at least one external device, such as an external memory or a processor, connected to the units of the system  100 . The units of the system  100  may be arranged to receive the data from each other via internal connections or via a data bus. 
         [0046]    In an embodiment of the system  100 , the system  100  comprises a control unit  160  for controlling the workflow in the system  100 . The control unit may be arranged to receive control data from and provide control data to the units of the system  100 . For example, after adapting the model to the object in the image data, the model unit  110  may be arranged to provide control data “the model is adapted” to the control unit  160 , and the control unit  160  may be arranged to provide control data “adapt the template to the model” to the template unit  120 . Alternatively, a control function may be implemented in a unit of the system  100 . 
         [0047]    In an embodiment of the system  100 , the system  100  comprises a user interface  165  for communicating with the user of the system  100 . The user interface  165  may be arranged to receive a user input for selecting a model and/or a template coupled to the model. The user interface may also provide the user with information, e.g., it may display a view of the adapted template. Optionally, the user interface may receive a user input for selecting a mode of operation of the system such as, e.g., for selecting coupling forces for coupling the template to the model. A person skilled in the art will understand that more functions may be advantageously implemented in the user interface  165  of the system  100 . 
         [0048]    In an embodiment, the system  100  is employed to model the lumbar vertebra of a patient. The model comprises a deformable mesh model of the lumbar vertebra for adapting to a lumbar vertebra object in CT image data of the patient. Such a model is described, for example, in Tobias Klinder, Cristian Lorenz, Jens von Berg, Sebastian P. M. Dries, Thomas Büllow, Jörn Ostermann:  Automated Model - Based Rib Cage Segmentation and Labeling in CT Images , MICCAI (2) 2007: pp 195-202. The template comprises a contour of the lumbar vertebra. The contour is defined by a planar cross-section of the surface of the lumbar vertebra model and described by a plurality of control points on the surface of the model. 
         [0049]    After an adaptation of the model by the model unit  110 , new positions of the control points of the template contour are found on the surface model of the vertebra by the template unit  120 . These control points define the adapted contour of the adapted template. Adapted contours are arranged to take into account individual characteristics of the patient&#39;s vertebrae and may show, for example, vertebral compression or fracture.  FIG. 2  illustrates an exemplary template of a lumbar vertebra before (a) and after (b) adaptation by the system  100  of the invention. The adapted lumbar vertebra contour illustrates that the lumbar transverse process  21  visible in the template (a) before the adaptation is fractured in the template (b) after the adaptation. 
         [0050]    The skilled person will understand that the adapted lumbar vertebra template may be non-planar, because the control points in their new positions on the surface of the adapted model do not need to be coplanar. Thus, in an embodiment, the system  100  further comprises an image unit  130  arranged to produce an image of the adapted template, i.e., a graphical representation of the modeled object. For example, the image may be an orthogonal projection of the adapted template on the plane, computed by minimizing the sum of squares of distances of the control points in their new positions to said plane. 
         [0051]    In an embodiment, the system  100  is employed to model the heart of a patient. The model comprises a deformable mesh model of the heart for adapting to a heart object in CT image data of the patient. Such a model and its adaptation is described, for example, in Cristian Lorenz, Jens von Berg,  A comprehensive shape model of the heart , Medical Image Analysis, Vol. 10, Issue 4, 2006, pp 657-670. The template comprises a surface schematically representing the left ventricle of the heart. The surface is defined by a plurality of control points on the left ventricle surface of the model using, for example, thin spline interpolation. The control points form a subset of the set of vertices of the mesh of the mesh model. 
         [0052]    After adaptation of the model by the model unit  110 , the new positions of the control points of the template on the surface of the heart model are determined based on the vertex positions of the adapted mesh model. The template unit  120  is arranged to compute the left ventricle surface, using spline interpolation, on the basis of these new control point positions. The user interface  165  of the system  100  is arranged to render a view of the template. The user may be enabled to translate and rotate the template, using the user interface  165 , to obtain more views of the template. Optionally, an image unit  130  for producing an image of the template, e.g. three orthogonal projections of the template along the principal axes of the inertia tensor of the template, may be integrated into the user interface  165 . 
         [0053]    Advantageously, the system  100  may be comprised in a reporting system  300 . Thus, views computed by the image unit  130  of the system  100  may be included in a medical report created by a report unit  310  together with annotations by a physician examining the image data. In an embodiment, the reporting system  300  comprises a reporting system first input connector  301  for obtaining data for the system  100  and a reporting system second input connector  302  for obtaining other data such as user annotations, patient name and age, other test and examination results, comments by a physician preparing the report, and so on. The reporting unit  310  is arranged to receive the adapted template from the system  100  and the other data from the second input  302  for preparing a report. The report is outputted via a reporting system output connector  303 . 
         [0054]    Those skilled in the art will further understand that other embodiments of the system  100  are also possible. It is possible, among other things, to redefine the units of the system and to redistribute their functions. Although the described embodiments apply to medical images, other applications of the system, not related to medical applications, are also possible. 
         [0055]    The units of the system  100  may be implemented using a processor. Normally, their functions are performed under the control of a software program product. During execution, the software program product is normally loaded into a memory, like a RAM, and executed from there. The program may be loaded from a background memory, such as a ROM, hard disk, or magnetic and/or optical storage, or may be loaded via a network like the Internet. Optionally, an application-specific integrated circuit may provide the described functionality. 
         [0056]      FIG. 4  shows a flowchart of an exemplary implementation of the method  400  of producing a representation of an object in image data, based on a template coupled to a model of the object. The method  400  begins with a model step  410  for adapting the model to the object in the image data. After the model step  410 , the method continues to a template step  420  for adapting the template to the adapted model on the basis of the coupling between the template and the model. In the shown embodiment of the method  100 , after the template step  420 , the method  400  continues to an image step  430  for producing an image of the template. After the image step  430  the method  400  terminates. 
         [0057]    A person skilled in the art may change the order of some steps or perform some steps concurrently using threading models, multi-processor systems or multiple processes without departing from the concept as intended by the present invention. Optionally, two or more steps of the method of the current invention may be combined into one step. For example, the model step  410  and the template step  420  may be combined into one adaptation step comprising a plurality of partial adaptation steps, wherein each partial adaptation step is arranged for adapting the model to the object in the image data, followed by adapting the template to the model, until a predetermined condition is fulfilled, e.g., until the number of partial adaptation steps is equal to a predetermined number. Optionally, a step of the method of the current invention may be split into a plurality of steps. 
         [0058]      FIG. 5  schematically shows an exemplary embodiment of the image acquisition apparatus  500  employing the system  100 , said image acquisition apparatus  500  comprising a CT image acquisition unit  510  connected via an internal connection with the system  100 , an input connector  501 , and an output connector  502 . This arrangement advantageously increases the capabilities of the image acquisition apparatus  500 , providing said image acquisition apparatus  500  with advantageous capabilities of the system  100 . 
         [0059]      FIG. 6  schematically shows an exemplary embodiment of the workstation  600 . The workstation comprises a system bus  601 . A processor  610 , a memory  620 , a disk input/output (I/O) adapter  630 , and a user interface (UI)  640  are operatively connected to the system bus  601 . A disk storage device  631  is operatively coupled to the disk I/O adapter  630 . A keyboard  641 , a mouse  642 , and a display  643  are operatively coupled to the UI  640 . The system  100  of the invention, implemented as a computer program, is stored in the disk storage device  631 . The workstation  600  is arranged to load the program and input data into memory  620  and execute the program on the processor  610 . The user can input information to the workstation  600 , using the keyboard  641  and/or the mouse  642 . The workstation is arranged to output information to the display device  643  and/or to the disk  631 . A person skilled in the art will understand that there are numerous other embodiments of the workstation  600  known in the art and that the present embodiment serves the purpose of illustrating the invention and must not be interpreted as limiting the invention to this particular embodiment. 
         [0060]    It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word “comprising” does not exclude the presence of elements or steps not listed in a claim or in the description. The word “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements and by means of a programmed computer. In the system claims enumerating several units, several of these units can be embodied by one and the same item of hardware or software. The usage of the words first, second, third, etc., does not indicate any ordering. These words are to be interpreted as names.