Abstract:
A system for visualizing and guiding a surgical device having a first imaging device of a first type and has a first image output. The first imaging device is positioned to image an area being subject to surgery. A second imaging device of a second type has a second image output. The second imaging device is positioned to image an area being subject to surgery. A computer is coupled to receive the first and second image outputs and a computer software program, resident in the computer receives and displays information received from the surgical device and/or for guiding the operation of the surgical device and for generating a graphic user interface including selectable menu and submenu items. The surgical device is coupled to the computer.

Description:
FIELD OF THE INVENTION 
       [0001]    The invention relates to a system for the control and visualization of medical devices positioned in a patient&#39;s body for ablation of a tumor, such as a uterine fibroid and, more particularly, to a user interface for visualizing an ultrasound image and dynamic 3D avatar guidance of an ablation probe during a surgical procedure, enabling the operator to make operational decisions based on those visualizations. 
       CROSS REFERENCE 
     Background of the Invention 
       [0002]    Advances in technology have resulted in graphical user interfaces that allow a practitioner or other medical professional to visualize a plurality of images obtained from multiple medical devices such as a laparoscopic cameras and ultrasound probes together on one screen. Current systems use simple picture in picture technology to view smaller laparoscopic camera images and ultrasound images on a larger guidance system screen. 
     
    
     
       SUMMARY OF THE INVENTION 
       Brief Description of the Drawings 
         [0003]      FIG. 1  illustrates an overview of the operating room layout useful with the inventive system; 
           [0004]      FIG. 2  illustrates an ablation system incorporating computer controls in accordance with the inventive system; 
           [0005]      FIG. 3  shows the inventive graphical user interface screen in which the display is divided into two separate screens with an associated text box and a control tab; 
           [0006]      FIG. 4  illustrates the inventive user interface being displayed from the surgeon&#39;s point of view with the ultrasound beam facing the patient&#39;s head; 
           [0007]      FIG. 5  illustrates the inventive user interface being displayed from the surgeon&#39;s point of view with the ultrasound beam facing the patient&#39;s feet; 
           [0008]      FIG. 6  illustrates the inventive user interface being displayed from the laparoscopic point of view with the ultrasound beam facing the patient&#39;s right hand; 
           [0009]      FIG. 7  illustrates the inventive user interface being displayed from the laparoscopic point of view with the ultrasound beam facing the patient&#39;s left hand; 
           [0010]      FIG. 8  shows an overhead view of a patient&#39;s body divided into four quadrants and the icon depictions associated with each quadrant; 
           [0011]      FIG. 9  shows the graphical user interface screen in which the navigational tool has scrolled to the tool icon control button; 
           [0012]      FIG. 10  shows the graphical user interface in which the tool icon control button has been selected; 
           [0013]      FIG. 11  shows the graphical user interface screen in which the navigational tool has scrolled to the three dimension icon control button; 
           [0014]      FIG. 12  shows the graphical user interface screen in which the navigational screen is displayed in three dimension; 
           [0015]      FIG. 13  shows the graphical user interface in which the navigational tool has scrolled to the control button; 
           [0016]      FIG. 14  shows the graphical user interface in which the navigational tool has scrolled to the grid icon control button; 
           [0017]      FIG. 15  shows the graphical user interface in which a grid is displayed on the GUI; 
           [0018]      FIG. 16  shows the graphical user interface in which the navigational tool has scrolled to ultrasound probe icon control button; 
           [0019]      FIG. 17  shows the graphical user interface in which the ultrasound data has been made full screen and the navigational screen has been eliminated. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0020]      FIG. 1  illustrates an overview of the typical operating room layout for the present invention. The surgeon  10  is standing on the left hand side of patient  12  who is lying on operating table  14 . Surgeon  10  may also stand on the right hand of patient  12 , whichever is most convenient. Two screens  16 ,  18  are located on the opposite side of patient  12  from surgeon  10 . Screen  16  displays video from a laparoscopic camera and screen  18  displays the inventive user interface. Screens  16  and  18  can be any type of display monitor such as a computer monitor, television, handheld device screen, etc. 
         [0021]      FIG. 2  illustrates an exemplary system for implementing the present invention. Computer  20  may be any control device, such as a microprocessor, personal computer, or a more powerful or less powerful computer with typical computer-type operating system. Computer  20  includes display screens  16  and  18 , which may optionally be a touchscreen to provide a second means of navigation. 
         [0022]    Personal computer  20  also incorporates software  22 . Software  22  may be of any type for use on any suitable computing device, and which may be easily written by a programmer of ordinary skill in the art who is informed by this specification. The software is responsive to produce images illustrated in the drawings and stored in memory  24  of computer  20 . The software performs navigation functions by being responsive to touchscreen entry on display screen  18 . Likewise, in response to operation by touching display screen  18 , computer  20  may cause the screen to change in one of the ways described in full detail below. 
         [0023]    Computer  20  communicates with a plurality of medical devices  26  through an interface board  28 . Medical devices  26  include an ablation instrument, laparoscopic camera, and ultrasound probe, or any other instrument useful in imaging and treating uterine fibroids or other pelvic tumors. At the same time, medical devices  26  provide information to interface  28  which in turn provides this information to personal computer  20  for display on display screens  16  and  18 . 
         [0024]      FIG. 3  illustrates the preferred embodiment of the inventive user interface. Software takes data from transducers located on an ablation probe, and data from transducers on an ultrasound probe and displays it to a user via user interface  100  on display screens  18 . Screen  18  displays the inventive user interface  100 . Video from a laparoscopic camera is communicated directly to display screen  16 . User interface has an overlay  102  that provides the appearance of a physical bezel that divides the display into two screens  104 ,  106 . Overlay  102  has a control tab  108  located on the left hand side of overlay  102 . Control tab  108  has five selections to choose from  110 ,  112 ,  114 ,  116 ,  118 . 
         [0025]    Overlay  102  includes two text boxes  120 ,  122 . Text box  120  is located under screen  104  and displays information relating to what is displayed on screen  104 . For example, text box  120  may display the depth of the ultrasound. 
         [0026]    Text box  122  is located under screen  106  and displays information relating to what is displayed on screen  106 . For example, text box  122  may display the point of view of the image displayed on screen  106 . 
         [0027]    Overlay  102  has a meter  124  located in between screens  104 ,  106 . Meter  124  indicates the distance between the tip of an ablation probe to the plane of an ultrasound scan. Meter  124  has hash marks  126  to indicate the distance from the tip of the ablation probe to the plane of the ultrasound scan. The central hash mark  128  is green and indicates that the tip of the ablation probe is in line with the plane of the ultrasound scan. The hash marks above central hash mark  128  are blue and indicate that the ablation needle is behind the plane of the ultrasound scan. The space between the hash marks increases the further away from central hash mark  128  you get. The hash marks below central hash mark  128  are yellow and indicate that the ablation needle is in front of the plane of the ultrasound scan. Meter  124  also has a sliding hash mark  130  that slides up and down meter  124  indicating the dynamic location of the tip of the ablation needle relative to the plane of the ultrasound scan. Sliding hash mark  130  is translucent, allowing the user to see the blue, yellow, or green hash mark underneath. 
         [0028]    Screen  104  is used to display a virtually complete ultrasound screen in two dimensions. Screen  104  contains a photorealistic avatar of ultrasound shaft  132 . Ultrasound shaft  132  is placed above the display of ultrasound beam  134  in order to provide orientation for ultrasound beam  134  to the user. This enables the user to easily see the direction the ablation probe is entering the ultrasound beam. 
         [0029]    Screen  106  also displays icon  136 , which indicates the direction the ultrasound beam is pointing in relation to the uterus of patient  12 . 
         [0030]    Icon  136  may be a depiction of a right hand with the letter “R” to indicate that ultrasound beam  134  is pointing towards the right hand of patient  12 . 
         [0031]    Icon  136  may be a depiction of a left hand with the letter “L” to indicate that ultrasound beam  134  is pointing towards the left hand of patient  12 . 
         [0032]    Icon  136  may be a depiction a person&#39;s head to indicate that ultrasound beam  134  is pointing towards the head of patient  12 . 
         [0033]    Icon  136  may also be a depiction of feet to indicate that ultrasound beam  134  is pointing towards the feet of patient  12 . 
         [0034]    Icon  136  will change in real time between these depictions depending on the direction ultrasound beam  134  is pointing. For instance, if a user moves an ultrasound beam from pointing towards the feet of a patient towards to the right hand of the patient, icon  136  will change from a depiction of feet to a depiction of a right hand with the letter R. This is done automatically without any input from the surgeon. The surgeon also has the capability to freeze the orientation view if desired. 
         [0035]      FIG. 4  illustrates the inventive user interface with the ultrasound beam facing the patient&#39;s head in the surgeon&#39;s point of view. Icon  136  displays as a depiction of a human face in profile. 
         [0036]      FIG. 5  illustrates the inventive user interface with the ultrasound beam facing the patient&#39;s feet in the surgeon&#39;s point of view. Icon  136  displays as depiction of a human foot. 
         [0037]      FIG. 6  illustrates the inventive user interface with the ultrasound beam facing the patient&#39;s right hand in the laparoscopic point of view. Icon  136  displays as a depiction of a right human hand with the letter “R.” The letter “R” can be displayed inside the depiction of the human hand or alternatively, in the immediate surrounding area of Icon  136 , such as to the right, to the left, on top of, or beneath, Icon  136 . 
         [0038]      FIG. 7  illustrates the inventive user interface with the ultrasound beam facing the patient&#39;s left hand in the laparoscopic point of view. Icon  136  displays as a depiction of a left human hand with the letter “L.” The letter “L” can be displayed inside the depiction of the human hand or alternatively, in the immediate surrounding area of Icon  136 , such as to the right, to the left, on top of, or beneath, Icon  136 . 
         [0039]    The display of ultrasound beam  134  will also rotate direction depending upon the orientation of ultrasound beam  134 . For example, in  FIG. 4 , ultrasound beam  134  is pointed at the patient&#39;s head, thus icon  136  will be a depiction of a head ultrasound beam  134  will point to the right. When the ultrasound transducer is physically rotated so that it points to the patient&#39;s right hand, the transducer shaft  132  will occlude the image of ultrasound beam  134 . In order to provide a user with a view of the system, the system will virtually rotate ultrasound beam  134  back to the right and icon  136  will display a right hand, to signify that ultrasound beam  134  is pointing to the patient&#39;s right hand, as seen in  FIG. 6 . This happens whenever ultrasound shaft  132  is physically rotated 90 degrees as shown in  FIG. 8 . 
         [0040]      FIG. 8  illustrates which graphical depiction Icon  136  will display when the ultrasound probe is in four defined quadrants of patient  12 . The depiction is based on the location of the ultrasound probe in patient  12  in relation to the uterus of patient  12 . Patient  12  is divided by a centerline  310  y-axis with the patient&#39;s head being 0 degrees and the patient&#39;s feet being 180 degrees. Patient is then divided by an x-axis through the uterus. Patient  12  is then further divided into four 90 degree quadrants  302 ,  304 ,  306 , and  308 . 
         [0041]    Quadrant  302  is defined between 315 degrees to 45 degrees measured from the uterus. When the ultrasound probe is in quadrant  302 , Icon  136  will display a depiction of a human foot since to view the uterus the ultrasound beam would have to be directed towards the patient&#39;s feet. 
         [0042]    Quadrant  304  is defined between 45 degrees and 135 degrees. When the ultrasound probe is in quadrant  304 , Icon  136  will display a depiction of a right human hand with the letter “R”. 
         [0043]    Quadrant  306  is defined between 135 degrees and 225 degrees. When the ultrasound probe is in quadrant  306 , Icon  136  will display a depiction of a human face in profile. 
         [0044]    Quadrant  308  is defined between 225 degrees and 315 degrees. When the ultrasound probe is in quadrant  308 , Icon  136  will display a depiction of a left human hand with the letter “L”. 
         [0045]    Referring back to  FIG. 3 , screen  106  is used to display ablation probe guidance information in either two dimensions or three dimensions. 
         [0046]    Again referring to  FIG. 3 , overlay  102  has a control tab  108  containing controls  110 ,  112 ,  114 ,  116 , and  118  located on the left side of overlay  102  which can be used by a surgeon or medical professional while maintaining the sterile field. 
         [0047]    Control  110  has a depiction of a tool, and when selected allows a medical professional to change the display of the system. 
         [0048]    Control  112  displays “3D,” and when selected allows a medical professional to view the guidance information on screen  106  in three dimension. 
         [0049]    Control  114  has a depiction of a lock, and when selected allows a medical professional to freeze the point of view being displayed. 
         [0050]    Control  116  has a depiction of a grid, and when selected displays a grid over the ultrasound data, which decreases the need for measurement of the dimensions of any tumors or masses being imaged. 
         [0051]    Control  118  has a depiction of an ultrasound probe, and when selected brings screen  104  into full-screen, eliminating screen  106 . 
         [0052]    Selecting control  110  ( FIG. 9 ) with navigational tool  111  causes the system to exit the display of  FIG. 2  and go to the display of  FIG. 10 . 
         [0053]    Selecting control  112  ( FIG. 11 ) with navigational tool  111  causes the system to exit the display of  FIG. 2  and go to the display of  FIG. 12 . The display of  FIG. 12  is substantially the same as the display of  FIG. 2 , except that screen  106  is now viewed as three-dimensional as opposed to two-dimensional. 
         [0054]    Selecting control  114  ( FIG. 13 ) with navigational tool  111  causes the display in  FIG. 2  to become locked in the currently displayed point of view. Thus, the display in screen  104  will not automatically switch point of views based on the placement of the ultrasound probe. Instead, the screen will stay locked in the point of view at the time of selection of control  114 . 
         [0055]    Selecting control  116  ( FIG. 14 ) with navigational tool  111  causes grid  138  to be displayed over screen  104  as illustrated in  FIG. 15 . Grid  138  decreases the need for measurement of the dimensions of any tumors or masses being imaged. 
         [0056]    Selecting control  118  ( FIG. 16 ) with navigational tool  111  causes the system to exit the display of  FIG. 3  and go to the display of  FIG. 17 .  FIG. 17  illustrates a full screen view of the ultrasound data of screen  104 , eliminating everything else from the display except for overlay  142 , which contains control tab  108  with controls  110 ,  112 ,  114 ,  116 , and  118 . Overlay  142  is substantially similar to overlay  102  in appearance except without the division into two screens. 
         [0057]    It will be appreciated by those skilled in the art that changes can be made to the embodiments described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the particular embodiments disclosed, but it is intended to cover modifications that are within the spirit and scope of the invention, as defined by the appended claims.