Patent Document

CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    The present application is a Continuation-in-Part of U.S. application Ser. No. 14/752,165, filed Jun. 26, 2015, which is a Continuation of U.S. application Ser. No. 12/846,545, filed Jul. 29, 2010, which claims priority from Korean Patent Application No. 10-2009-0070994, filed Jul. 31, 2009. The disclosure of each of these applications is incorporated herein by reference in its entirety. 
     
    
     TECHNICAL FIELD 
       [0002]    The present invention generally relates to ultrasound systems, and more particularly to an ultrasound system and method for performing sensor coordinate calibration through image-based registration between a three-dimensional ultrasound image and a computed tomography (CT) image. 
       BACKGROUND 
       [0003]    The ultrasound system has become an important and popular diagnostic tool due to its non-invasive and non-destructive nature. Modern high-performance ultrasound imaging diagnostic systems and techniques are commonly used to produce two- or three-dimensional images of internal features of patients (target objects). 
         [0004]    However, the ultrasound system suffers from inherent shortcomings of an ultrasound image such as a low signal-to-noise ratio and a limited field of view. Thus, the image registration of a CT (or MR) image onto the ultrasound image has been introduced in order to compensate for deficiencies of the ultrasound image. A sensor has been used to perform the image registration of a CT (or MR) image onto the ultrasound image. Researches have been introduced to calibrate the sensor to match coordinates of the CT image and coordinates of the sensor. 
         [0005]    Conventionally, after outer markers are attached on a surface of a target object, a CT image and an ultrasound image for the target object with the markers are acquired. Thereafter, the calibration is carried out by using a relationship between coordinates of the markers in the CT and ultrasound image. That is, the outer markers should be attached to the surface of the target objects before obtaining the CT image and the ultrasound image and be maintained in the same position until completing the acquisition of the ultrasound image. Moreover, a sensor must sense the positions of the respective outer markers. 
         [0006]    Further, the registration between the coordinate of the CT image and the coordinate of the sensor has been performed by manually inputting inner markers on the CT image. Thus, a user of the ultrasound system had to input the inner markers, which causes the registration between the coordinate of the CT image and the coordinate of the sensor to be wrong. 
       SUMMARY 
       [0007]    An embodiment for forming a plurality of three-dimensional ultrasound images is disclosed herein. In one embodiment, by way of non-limiting example, an ultrasound system, includes: an ultrasound image forming unit including a ultrasound probe and being configured to form a three-dimensional ultrasound image of a target object; a sensor coupled to the ultrasound probe; a memory configured to store a three-dimensional computed tomography (CT) image of the target object and position information on a position between the three-dimensional ultrasound image and the sensor; and a processor configured to perform image registration between the three-dimensional CT image and the three-dimensional ultrasound image to thereby form a first transformation function for transforming a position of the sensor to a corresponding position on the three-dimensional CT image and perform calibration of the sensor by applying the position information to the first transformation function. 
         [0008]    In another embodiment, a method of performing a calibration of a sensor, includes: a) obtaining a three-dimensional ultrasound image of a target object obtained by the ultrasound system and a three-dimensional CT image; b) calculating a position information on a position between the three-dimensional ultrasound image and the sensor; c) performing registration between the three-dimensional ultrasound image and the three-dimensional CT image to obtain a first transformation function for transforming a position of the sensor to a corresponding position on the three-dimensional CT image; and d) performing calibration of the sensor by applying the position information to the first transformation function. 
         [0009]    The Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in determining the scope of the claimed subject matter. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0010]      FIG. 1  is a block diagram showing an illustrative embodiment of an ultrasound system. 
           [0011]      FIG. 2  is a block diagram showing an illustrative embodiment of an ultrasound image forming unit. 
           [0012]      FIG. 3  is an illustrative embodiment of an ultrasound probe. 
           [0013]      FIG. 4  is a block diagram showing an illustrative embodiment of a processor. 
           [0014]      FIG. 5  is a schematic diagram showing an example of eigenvalues in the Hessian matrix. 
       
    
    
     DETAILED DESCRIPTION 
       [0015]    A detailed description may be provided with reference to the accompanying drawings. One of ordinary skill in the art may realize that the following description is illustrative only and is not in any way limiting. Other embodiments of the present invention may readily suggest themselves to such skilled persons having the benefit of this disclosure. 
         [0016]      FIG. 1  is a block diagram showing an illustrative embodiment of an ultrasound system. The ultrasound system  100  may include an ultrasound image forming unit  110 , a sensor  120 , a memory  130 , a processor  140  and a display unit  150 . 
         [0017]    The ultrasound image forming unit  110  may be configured to transmit ultrasound signals to a target object (not shown) and receive ultrasound echo signals reflected from the target object. The ultrasound image forming unit  110  may be further configured to form a three-dimensional ultrasound image of the target object based on the received ultrasound echo signals. 
         [0018]      FIG. 2  is a block diagram showing an illustrative embodiment of an ultrasound image forming unit  110 . The ultrasound image forming unit  110  may include a transmit (Tx) signal generating section  111 , an ultrasound probe  112  including a plurality of transducer elements (not shown), a beam former  113 , an ultrasound data forming section  114  and an image forming section  115 . 
         [0019]    The Tx signal generating section  111  may generate Tx signals according to an image mode set in the ultrasound system  100 . The image mode may include a brightness (B) mode, a Doppler (D) mode, a color flow mode, etc. In one exemplary embodiment, the B mode may be set in the ultrasound system  100  to obtain a B mode ultrasound image. 
         [0020]    The ultrasound probe  112  may receive the Tx signals from the Tx signal generating section  111  and generate ultrasound signals, which may travel into the target object. The ultrasound probe  112  may further receive ultrasound echo signals reflected from the target object and convert them into electrical receive signals. In such a case, the electrical receive signals may be analog signals. The ultrasound probe  112  may be a three-dimensional probe, a two-dimensional probe, a one-dimensional probe or the like. 
         [0021]      FIG. 3  is an illustrative embodiment of an ultrasound probe  112 . At least one transducer element (not shown) of the ultrasound probe  112  generates an image plane IP, which is used to scan a region of interest ROI. The image plane IP may be one of slice planes of the three-dimensional ultrasound image. The sensor  120  is attached to the housing of the ultrasound probe  112  to determine the position and orientation of the image plane IP. The ultrasound system  100  coupled with the ultrasound probe  112  via the probe cable  105  can use the data generated by the sensor  120  to determine the position and orientation of the sensor  120  and/or the image plane IP, as described below. 
         [0022]    In this preferred embodiment, the sensor  120  is a magnetic sensor that monitors the free-hand movement of the ultrasound probe  112  in six degrees of freedom with respect to a transducer element  170 . As shown in  FIG. 3 , the sensor  120  and the transducer element  170  each define an origin ( 122 ,  172 , respectively) defined by three orthogonal axes (X′, Y′, Z′ and X″, Y″, Z″, respectively). The sensor  120  monitors the translation of the origin  122  with respect to the origin  172  of the transducer element  170  to determine position and monitor the rotation of the X′, Y′, Z′ axes with respect to the X″, Y″, Z″ axes of the transducer element  170  to determine orientation. 
         [0023]    The position and orientation of the sensor  120  can be used to determine the position and orientation of the image plane IP. As shown in  FIG. 3 , the image plane IP defines an origin OR defined by three orthogonal axes X, Y, Z, which are preferably aligned with the origin of a center acoustic line generated by the ultrasound probe  112 . The position of the origin  122  and the orientation of axes X′, Y′, Z′ of the sensor  120  may not precisely coincide with the position of the origin OR and the orientation of the axes X, Y, Z of the image plane IP. For example, in  FIG. 3 , the origin OR of the image plane IP is offset from the origin  122  of the sensor  120  by a distance z 0  along the Z-direction and a distance of y 0  along the Y-direction. In  FIG. 3 , there is no offset along the X-direction nor is there a rotational offset in the orientation of the axes. Accordingly, the position and orientation of the sensor  120  do not directly describe the position and orientation of the image plane IP. 
         [0024]    To determine the position an orientation of the image plane IP from the position and orientation of the sensor  120 , sensor calibration data is used to transform the position and orientation of the sensor  120  to the position and orientation of the image plane 
         [0025]    IP. For simplicity, the term “position and orientation” is used to broadly refer to position and/or orientation. Accordingly, if the sensor  120  has the same orientation as the image plane IP, then the position and orientation calibration data may not contain any orientation calibration data. Similarly, as shown in  FIG. 3 , the sensor  120  may not have a positional offset with respect to one or more axes of the image plane IP. 
         [0026]    There are a number of ways of defining the image plane/sensor offset. One method of calibrating at least some types of sensors use three orthogonal linear dimension offsets in X, Y, Z and three rotation angles about each of these axes. Other methods include using a position transformation matrix or quaternions, which are described in the user manual for the mini Bird™ and the Flock of Bird™ systems by Ascension Technology Corp. 
         [0027]    As described above, the ultrasound probes with position and orientation sensors are typically used only with ultrasound systems that contain the calibration data for the probe/sensor pair. Conventionally, the probe/sensor pair is calibrated, and the calibration data is stored in the ultrasound system  100 , which will be used in conjunction with the probe/sensor pair. If the probe/sensor pair is to be used with a different ultrasound system, then the probe/sensor pair typically needs to be re-calibrated on that different ultrasound system. Since sonographers are often unable or unwilling to perform probe/sensor pair calibration, probe/sensor pairs are often used only with the ultrasound system for which the probe/sensor pair was initially calibrated. 
         [0028]    Referring back to  FIG. 2 , the beam former  113  may convert the electrical receive signals outputted from the ultrasound probe  112  into digital signals. The beam former  113  may further apply delays to the digital signals in consideration of the distances between the transducer elements and focal points to thereby output receive-focused signals. 
         [0029]    The ultrasound data forming section  114  may form a plurality of ultrasound data by using the receive-focused signals. In one embodiment, the plurality of ultrasound data may be radio frequency (RF) data or IQ data. The image forming section  115  may form the three-dimensional ultrasound image of the target object based on the ultrasound data. 
         [0030]    Referring back to  FIG. 1 , the sensor  120  may be mounted on one side of the ultrasound probe  112 . In one embodiment, by way of non-limiting examples, the sensor  120  may be built in the ultrasound probe  112  to be away from the plurality of transducer elements (not shown) by a predetermined distance. Alternatively, the sensor  120  may be externally mounted on the ultrasound probe  112  to be away from the plurality of transducer elements. The sensor  120  may include three-dimensional sensor, which can detect a three-dimensional position and an angle of the ultrasound probe  112 . 
         [0031]    The memory  130  may store a three-dimensional CT image of the target object. In one embodiment, by way of non-limiting examples, the three-dimensional CT image may be a three-dimensional CT image of a liver in which a diaphragm and a blood vessel are extracted. The memory  130  may store information on a position between the three-dimensional ultrasound image and the sensor  120  (hereinafter, referred to as “position information”). The position information may include information on a distance between the transducer elements (not shown) and the sensor  120 . In one embodiment, by way of non-limiting examples, the memory  120  may include at least one of a random access memory (RAM), a hard disk drive or the like. 
         [0032]    The processor  140  may be configured to perform registration between the three-dimensional CT image and the three-dimensional ultrasound image, thereby forming a transformation function (T probe ) for representing the ultrasound probe  112  on the three-dimensional CT image. Furthermore, the processor  140  may perform calibration of the sensor  120  to match coordinates of the three-dimensional CT image (not shown) and coordinates of the sensor  120  based on the position information and the transformation function. 
         [0033]      FIG. 4  is a block diagram showing an illustrative embodiment of the processor  140 . The processor  140  may include a diaphragm extracting section  141 , a vessel extracting section  142 , a diaphragm refining section  143 , a registration section  144 , a calibration section  145  and an image processing section  146 . 
         [0034]    The diaphragm extracting section  141  may be configured to extract a diaphragm from the three-dimensional ultrasound image formed in the ultrasound image forming unit  110 . In one embodiment, the diaphragm extracting section  141  may perform a Hessian matrix based flatness test upon the three-dimensional ultrasound image to extract the diaphragm. The diaphragm may be considered as a curved surface in the three-dimensional ultrasound image. Thus, regions in which a voxel intensity change in a normal direction at a surface is greater than a voxel intensity change in a horizontal direction at the surface may be extracted as the diaphragm.  FIG. 5  is a schematic diagram showing an example of eigenvalues λ1, λ2 and λ3 in the Hessian matrix. 
         [0035]    Hereinafter, an operation of the diaphragm extracting section  141  will be described in detail. The diaphragm extracting section  141  may be configured to select voxels having a relatively high flatness value. The flatness μ(v) may be defined as the following equation (1). 
         [0000]      μ( v )=Φ 1 ( v )Φ 2 ( v )Φ 3 ( v )/Φ 3     max   ( v )   (1)
 
         [0000]    wherein Φ 1 (v), Φ 2 (v) and Φ 3 (v) in the equation (1) may be represented as the following equation (2). 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         φ 
                         1 
                       
                        
                       
                         ( 
                         v 
                         ) 
                       
                     
                     = 
                     
                       
                         ( 
                         
                           1 
                           - 
                           
                             
                               
                                 λ 
                                 1 
                               
                                
                               
                                 ( 
                                 v 
                                 ) 
                               
                             
                             
                               
                                 λ 
                                 3 
                               
                                
                               
                                 ( 
                                 v 
                                 ) 
                               
                             
                           
                         
                         ) 
                       
                       2 
                     
                   
                   , 
                   
                     
                       
                         φ 
                         2 
                       
                        
                       
                         ( 
                         v 
                         ) 
                       
                     
                     = 
                     
                       
                         ( 
                         
                           1 
                           - 
                           
                             
                               
                                 λ 
                                 2 
                               
                                
                               
                                 ( 
                                 v 
                                 ) 
                               
                             
                             
                               
                                 λ 
                                 3 
                               
                                
                               
                                 ( 
                                 v 
                                 ) 
                               
                             
                           
                         
                         ) 
                       
                       2 
                     
                   
                   , 
                   
                     
                       
                         φ 
                         3 
                       
                        
                       
                         ( 
                         v 
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         i 
                       
                        
                       
                           
                       
                        
                       
                         
                           
                             λ 
                             i 
                           
                            
                           
                             ( 
                             v 
                             ) 
                           
                         
                         2 
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein λ 1 , λ 2  and λ 3  denote eigenvalues of the Hessian matrix at voxel v. The flatness μ(v) may be normalized to have values of ranging 0-1. A flatness map may be formed based on the flatness values obtained from all of the voxels according to the equations (1) and (2). Thereafter, the voxels having a relatively high flatness value are selected. In one embodiment, the voxels having the flatness over 0.1 may be selected. 
         [0036]    The diaphragm extracting section  141  may be further configured to perform the morphological opening upon the selected voxels to remove small clutters therefrom. The morphological opening may be carried out by sequentially performing erosion and dilation. That is, a predetermined number of the voxels are removed in the edges of the area in which the voxels exist, and thus, the area becomes contracted (erosion). In this manner, it becomes possible to remove small clutters. Thereafter, the edges of the area are expanded by the predetermined number of the voxels (dilation). These erosion and dilation may be performed by one or more voxels. 
         [0037]    The diaphragm is the largest surface in the three-dimensional ultrasound image. The largest surface may be selected among candidate surfaces obtained by the intensity-based connected component analysis (CCA) for the voxles and the selected surface may be regarded as the diaphragm in the three-dimensional ultrasound image. The voxel-based CCA is one of the methods of grouping regions in which voxel values exist. For example, the number of voxels connected to each of the voxels through a connectivity test by referring to values of voxels neighboring the corresponding voxel (e.g., 26 voxels) may be computed. The voxels, of which connected voxels are greater than the predetermined number, are selected as candidate groups. Since the diaphragm is the widest curved surface in the ROI, the candidate group having the most connected voxels may be selected as the diaphragm. The surface of the diaphragm may be smoothened. 
         [0038]    The vessel extracting section  142  may be configured to perform vessel extraction upon the three-dimensional ultrasound image. The vessel extracting section  142  may be configured to perform the vessel extraction through ROI masking, vessel segmentation and classification. 
         [0039]    To avoid mis-extraction of the vessels due to mirroring artifacts, the ROI masking may be applied to the three-dimensional ultrasound image by modeling the diaphragm as a polynomial curved surface. For example, the ROI masking may be used to model the diaphragm as the polynomial curved surface by using the least means square. However, if all of the lower portions of the modeled polynomial curved surface are eliminated, then meaningful vessel information may be lost at a portion of regions due to an error of the polynomial curved surface. To avoid losing the vessel information, the lower portion of the modeled polynomial curved surface may be eliminated with a marginal distance. For example, the marginal distance may be set to about 10 voxels at a lower portion of the ROI mask. 
         [0040]    Subsequently, the vessel extracting section  142  may be further configured to segment a vessel region and a non-vessel region. To exclude non-vessel high intensity regions such as the diaphragm and vessel walls, a low intensity bound value having a less reference bound value in the ROI masked three-dimensional ultrasound image may be set as a reference bound value. Thereafter, voxels with a higher intensity value than the reference bound value may be removed. The remaining regions may be binarized by using an adaptive threshold value. Then, the binarized segments may be labeled as vessel candidates. 
         [0041]    Next, the vessel extracting section  142  may be further configured to remove non-vessel-type clutters from the binarization image to form real vessel regions from the vessel candidates. In one embodiment, the vessel classification may include a size test, which evaluates the goodness of fit to a cylindrical tube, for filtering out tiny background clutters, a structure-based vessel test for removing non-vessel type clutters, i.e., an initial vessel test, a gradient magnitude analysis, and a final vessel test for precisely removing the clutters. Although some clutters are not removed through the structure-based vessel test, an initial threshold may be marginally set so that all vessels may be included. For example, a threshold value of the initial vessel test may be set to 0.6. At the final vessel test, clutters, which may be formed by small shading artifacts having low gradient magnitudes, may be precisely removed by considering variation of voxel values, i.e., gradient magnitudes, to thereby extract vessel data. In one embodiment, a threshold of the final vessel test may be set to 0.4. 
         [0042]    The diaphragm refining section  143  may be configured to refine the diaphragm region by removing the clutters with the extracted vessel regions. The clutters are mainly placed near the vessel walls. Especially, the vessel walls of inferior vena cava (IVC) are more likely to be connected to the diaphragm and cause clutters. These clutters may degrade the accuracy of the feature based registration, and thus, it may be necessary to refine the diaphragm region. To refine the diaphragm, the vessel regions are extracted according to the vessel extraction mentioned above, the extracted vessel regions may be dilated, and then the dilated vessel regions may be subtracted from the initially extracted diaphragm region to estimate vessel walls. The estimated vessel walls may be removed from the diaphragm region. Finally, the diaphragm region may be extracted by applying CCA and the size test. 
         [0043]    The registration section  144  may be configured to perform the image registration between the three-dimensional ultrasound and CT image. The registration section  144  may extract sample points from the vessel regions and the diaphragm region, respectively, among the features extracted from the three-dimensional ultrasound image. Also, after the vessel regions and the diaphragm region are extracted from the CT image, the registration section  144  may extract sample points from the vessel and the diaphragm region, respectively. The image registration between the three-dimensional ultrasound and CT image may be performed based on the extracted sample points to thereby form the transformation function (T probe ) between the three-dimensional CT image and the three-dimensional dimensional ultrasound image. The transformation function (T probe ) may be given by a matrix and used to transform a position of the ultrasound probe  112  to a corresponding position on the three-dimensional CT image. 
         [0044]    The calibration section  145  may perform the calibration of the sensor  120  based on the transformation matrix (T probe ) from the registration section  144  and the position information stored in the memory  130 . More particularly, the calibration section  145  may form a transformation matrix (T sensor ) between the sensor  120  and the three-dimensional ultrasound image, i.e., a transformation matrix representing a position of the sensor  120  with respect to the three-dimensional ultrasound image. The transformation matrix (T sensor ) may be given by a matrix. The transformation matrix (T sensor ) may be defined as the following equation (3). 
         [0000]                    T   sensor     =                r                 11           r                 12           r                 13         x             r                 21           r                 22           r                 23         y             r                 31           r                 32           r                 33         z           0       0       0       1                      (   3   )                 r 11=cosθ y *cosθ z +sinθ x *sinθ y *sinθ z  
 
         [0000]        r 12=sinθ z *cosθ y −sinθ x *sinθ y *cosθ z  
 
         [0000]        r 13=cosθ x *sinθ y, r 21=sinθ z *cosθ x  
 
         [0000]        r 22=cosθ z *sinθ x ,  r 23=sinθ x  
 
         [0000]        r 31=sinθ z *sinθ x *cosθ y −cosθ z *sinθ y  
 
         [0000]        r 32=cosθ z *sinθ x *cosθ y −sinθ z *sinθ y  
 
         [0000]        r 33=cosθ x *cosθ y  
 
         [0000]    wherein, x denotes coordinate of a lateral direction of the sensor  120 , y denotes coordinate of an elevation direction of the sensor  120 , z denotes an axial direction of the sensor  120 , θ x  denotes an angle of the sensor  120  from the x-axis, θ y  denotes an angle of the sensor  120  from the y-axis, and θ z  denotes an angle of the sensor  120  from the z-axis. The elevation direction may be a swing direction of the transducer elements, the axial direction may be a scan line direction from the transducer elements and the lateral direction may be a longitudinal direction of the transducer elements. 
         [0045]    The calibration section  145  may perform the calibration based on the transformation matrix (T probe ) and the transformation matrix (T sensor ). The calibration section  145  may form a transformation matrix (T) representing the position of the sensor  120  on the three-dimensional CT image. In one embodiment, the calibration section  145  may form the transformation matrix (T) through matrix multiplication of the transformation matrix (T probe ) and the transformation matrix (T sensor ). 
         [0046]    The image processing section  146  may apply the transformation matrix (T) to the three-dimensional CT image to thereby form a two-dimensional CT image according to a two-dimensional ultrasound image. 
         [0047]    Referring back to  FIG. 1 , the display unit  150  may display the two-dimensional CT image, which is provided from the processor  140 . Furthermore, the display unit  150  may display the three-dimensional ultrasound image and the three-dimensional CT image. 
         [0048]    Although the exemplary embodiments above described in detail a CT image as an image to be aligned with the ultrasound image, this is not limiting. For example, the image to be aligned with the ultrasound image may be obtained by various modalities and may be, for example, an optical coherence tomography (OCT) image, a magnetic resonance (MR) image, a single-photon emission computed tomography (SPECT) image, a PET image, a PET-CT image, a PET-MR image, a fluoroscopic image, an X-ray image including an image obtained by a stationary X-ray, a mobile X-ray, a C-arm X-ray, etc., and an image of any other appropriate imaging modality. 
         [0049]    The above-discussed images may be aligned and/or fused in various ways. Further, any two images or an ultrasound image and the fused image may be displayed side by side, partially overlapping, etc. 
         [0050]    Any reference in this specification to “one embodiment,” “an embodiment,” “example embodiment,” “illustrative embodiment,” etc. means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment. Further, when a particular feature, structure or characteristic is described in connection with any embodiment, it is submitted that it is within the purview of one skilled in the art to affect such feature, structure or characteristic in connection with other ones of the embodiments. 
         [0051]    Although embodiments have been described with reference to a number of illustrative embodiments thereof, it should be understood that numerous other modifications and embodiments can be devised by those skilled in the art that will fall within the spirit and scope of the principles of this disclosure. More particularly, numerous variations and modifications are possible in the component parts and/or arrangements of the subject combination arrangement within the scope of the disclosure, the drawings and the appended claims. In addition to variations and modifications in the component parts and/or arrangements, alternative uses will also be apparent to those skilled in the art.

Technology Category: 3