Patent Publication Number: US-8543338-B2

Title: System and method for performing computerized simulations for image-guided procedures using a patient specific model

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application is a Continuation-in-part application of U.S. application Ser. No. 12/224,314, filed Aug. 22, 2008, which is a National Phase Application of PCT International Application No. PCT/IL2008/000056, International filing date Jan. 13, 2008, claiming priority of U.S. Provisional Patent Application No. 60/880,415, filed Jan. 16, 2007. 
    
    
     BACKGROUND OF THE INVENTION 
     The clinical practice is replacing, wherever possible, traditional open surgical procedures with less invasive techniques that require, however, indirect, image-based feedback. In image-guided procedures, such as vascular catheterization, angioplasty and stent placement the physician needs to identify anatomical structures in images. These procedures are difficult to master without extensive training, yet training on humans may be fatal. Accordingly, simulation systems for image-guided procedures may train a physician without unnecessary risk and may also serve as pre-operative planning tool or post-operative assessment tool. Most of the simulation systems are based on predefined models of anatomical structures and are not patient specific. Accordingly, such systems cannot be used for accurately planning an operation on a particular patient prior to performing the operation or for post assessment of the operation. A more progressive simulation system is a patient-specific simulation system that uses patient-specific medical image data. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanied drawings in which: 
         FIG. 1  shows an exemplary system for simulating an image-guided procedure according to embodiments of the invention; 
         FIG. 2  shows an illustration of an exemplary anatomical structure helpful in understanding embodiments of the present invention; 
         FIG. 3  is a flowchart diagram illustrating a method for simulating an image-guided procedure according to some embodiments of the present invention; 
         FIGS. 4A and 4B  show graphical illustrations of an exemplary 3D digital model produced according to embodiments of the invention; 
         FIG. 5  is a flowchart diagram illustrating an exemplary method for generating an extended model for client-specific simulation for image-guided procedures according to some embodiments of the present invention; 
         FIG. 6  is a graphical illustration of an exemplary 3D generic model for extended models according to embodiments of the invention; and 
         FIGS. 7A and 7B  show graphical illustrations of an exemplary 3D digital model produced according to embodiments of the invention. 
     
    
    
     It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements. 
     DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 
     In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those of ordinary skill in the art that the invention may be practiced without these specific details. In other instances, well-known methods, procedures, components, modules, units and/or circuits have not been described in detail so as not to obscure the invention. 
     Embodiments of the invention are directed to patient-specific computerized simulations for image-guided procedures. The method may include producing a digital model of an anatomical structure based on medical image data received from a scan of a subject. The subject may be for example a patient that is about to undergo an image-guided procedure. Usually the medical image data received, for example, from a CT system or any other suitable imaging system does not include or cover the entire area of interest for simulation but rather a more limited area or region. For example, a scan performed prior to a stenting procedure may typically cover areas in close vicinity to the area about to be treated. As will be understood to a person skilled in the art, usually, a scan of a patient is performed as a diagnostic tool to diagnose, for example, pathology in a specific anatomical region and therefore usually such a scan covers only the area that diseased, damaged or modified and its immediate vicinity. An operation or procedure for treating the pathology may require passage through other anatomical regions not covered by the scan. A physician may wish to practice on the entire procedure prior to performing the operation and/or to perform post-procedure assessment. Accordingly, in order to enable a comprehensive simulation of the procedure, an anatomical model of additional regions may be desired. 
     According to embodiments of the invention, the method may include producing an extended model based on the digital model and extrapolated data representing adjacent sections of the anatomical structures not included in the medical image data. A graphical representation of the extended model may be displayed on a monitor and the extended model may be viewed with additional information, such as models of tools. A physician may perform a computerized simulation of the image-guided procedure using the extended model as a pre-procedure of the actual surgery. 
     The medical image data may be patient specific medical images obtained from an imaging system such as computed tomography (CT), CT-fluoroscopy, fluoroscopy, magnetic resonance imaging (MRI), Ultrasound, positron emission tomography (PET) and X-ray. Embodiments of the invention may use as input the medical image data to produce 3D or 4D model of the anatomical structure, organ, and system and further an extended model including regions that are not presented in the medical images obtained from the imaging system. 
     The 3D model generated based on the image data may be, for example, a polygonal mesh representing the three-dimensional (3D) surfaces of the anatomical structure, a voxel mask of the structure volume or a patch surface, such as a 2D B-spline or the like. For ease of explanation, embodiments of the invention are described with respect to polygonal meshes. It should be however understood to a person skilled in the art that the invention is not limited in to such models and other models are within the scope of the invention. 
     Reference is made to  FIG. 1  showing an exemplary system  100  for simulating an image-guided procedure according to embodiments of the invention. System  100  may include an input unit  105 , an output unit  120 , a model generation unit  110 , a simulation unit  115  and a management unit  135 . System  100  may further include a memory  130  and a controller  131 . Input unit  105  may receive medical image data and may deliver the medical image data to model generation unit  110  for producing a patient-specific model for simulation. The model may include extended portions representing areas of the anatomical structures that are were not presented in the medical image data used as input data. When a user performs a simulation, for example as a pre-procedure for an image-guided procedure, using simulation unit  115 , a graphical representation of the model and the simulation process may be displayed on a monitor (not shown) of output unit  120 . 
     Input unit  105  may interface with or receive medical image data from an imaging system (not shown) such as X-ray system, CT system, MRI system and/or an ultrasound imaging system. Input unit  105  may include a mouse, a keyboard, a touch screen or pad or any suitable input devices. Alternatively or additionally, input unit  105  may include a wired or wireless network interface card (NIC) that may receive data, for example, from the imaging system. According to some embodiments, input unit  105  may communicate with a system or server storing medical images such as a picture archiving communication system (PACS) and may obtain any relevant imaging information, data or parameters from such a system, server or application. 
     Model generation emit  110  may include components or modules for generating a digital model and its graphical representation, e.g., a 3D or 4D anatomical model of an anatomical structure, such as an organ vessel system or any other area of interest within the body. The model may be generated by model generation unit  110  according to information received from an imaging system, for example, a medical image received from a CT system via input unit  105 . Simulation unit  115  may include components for generating a simulation of an image-guided procedure. According to embodiments of the invention, system  100  may further include a force feedback device (not shown) to simulate the perception of force when manipulating physical tools within the body. 
     Output unit  120  may include components for interfacing with a display screen to enable visual output or optionally a speaker or another audio device to enable audible output. Output unit  120  may include one or more displays, speakers and/or any other suitable output devices. Output unit  120  may communicate with any other component or unit of system  100  and accordingly may enable such units to communicate with external systems. Units  105 ,  110 ,  115  and  120  may be or may comprise software, hardware, firmware or any suitable combination thereof. 
     Controller  131  may be any suitable controller or processing unit, e.g., a central processing unit processor (CPU). Memory  130  may be any suitable memory component, device, chip or system and may store applications or other executable codes that may be executed by controller  131 . For example, applications or modules implementing model generation and/or simulation may be loaded into memory  130  and executed by controller  131 . 
     It will be recognized that system  100  as described herein is an exemplary system. According to embodiments of the invention, system  100  may be implemented on a single computational device or alternatively, in a distributed configuration, on two or more different computational devices. For example, model generation unit  110  may operate on a first computational device and managed by a first management unit whereas simulation unit  115  may operate on another computational device and managed by a second management unit that communicates with the first management unit. In another exemplary embodiment, management unit  135  may operate on a computational device, model generation unit  110  may operate on a second computational device and simulation unit  115  may operate on a third computational device. 
     Reference is made to  FIG. 2  showing an exemplary illustration of an anatomical structure of a cardiovascular system helpful in understanding embodiments of the invention. Performing an image-guided procedure simulation, such as carotid stenting simulation, the user may practice navigation in the vasculature while experimenting with various catheters. Typically, the scan of the patient for diagnostic purposes is performed only in the direct vicinity of the lesion. Accordingly, a model for simulation that would use as input data such scans may not encompass all the areas through which that the physician should navigate to reach the lesion. 
     The illustration of  FIG. 2  shows a top region  220  representing an area that was scanned by an exemplary imaging system and a bottom region  230  that was not scanned. Accordingly, the medical image data would include only the image of region  220 . According to embodiments of the invention, in order to perform a full and/or comprehensive simulation of the procedure, an extended model for simulation may be produced representing both the scanned region  220  and the un-scanned region  230 . According to embodiments of the invention, the portion of the model representing the scanned region may be generated from the medical image data of a specific subject and the model may be extrapolated to include representation of the un-scanned region. The extrapolation may include selecting from a set of pre-designed generic models representing anatomical structures the best match to represent the desired extended portion. 
     For example, in an angioplasty procedure, such as carotid stenting, in order to reach the left vertebral at point  235  where a lesion may exist, a catheter may be inserted into the aorta at the lower part of the patient&#39;s body and guided through point  240  on the way to the left vertebral. Accordingly, an extended model that includes the aorta may be desired for the simulated procedure. 
     Reference is made to  FIG. 3 , which is an exemplary flowchart describing a method for simulating an image-guided procedure according to some embodiments of the present invention. Reference is additionally made to  FIGS. 4A and 4B  showing graphical illustrations of an exemplary 3D digital model produced according to embodiments of the invention. As shown by box  310 , the method may include receiving medical image data of a subject. The medical image data may be received from an imaging or scanning system such as for example, CT or MRI scanners. As shown by box  315 , the method may include processing the received image data and generating based on the processed data a model of the anatomical structure depicted in the medical image, which is patient specific. A model that includes only anatomical structure that is represented in the medical image data is also termed herein a basic model or an image-based model.  FIG. 4A  shows an exemplary image-based model  410  processed from CT data of a portion of the vasculature. 
     According to exemplary embodiments, the processing may include segmenting the medical image data. Segmentation involves partitioning of the image domain into non-overlapping regions that correspond to distinct anatomical structures and identifying the desired anatomical structures in the medical images. Other segmentation techniques, such as soft segmentation, probabilistic or Bayesian segmentations may enable overlapping regions. It will be recognized that embodiments of the invention are not limited in that respect, and any applicable segmentation or soft segmentation method may be used. 
     The segmentation process may be implemented using fully-automated algorithms, semi-automated algorithms with minimal input from the user, tailor-made algorithms where the user can explicitly specify the desired segmentation or by manual segmentation, using for example CAD tools. The output of the segmentation process includes identifying a portion of the image data as a set of voxels (a mask) that represents the desired anatomical structure volume. The processing of the data may further include generating the discretisized surfaces of this volume, also referred to as a boundary-representation (B-rep). These surfaces are commonly represented by polygonal meshes, although embodiments of the invention are not limited in this respect and other representations, such as spline surface patches or constructive solid geometry (CSG) representation or any mix thereof are likewise possible. 
     The processing may further include computation of center-lines of approximate-tubular sections of the polygonal mesh, e.g., blood vessels, intestines or colon. In the exemplary image-based or basic model of  FIG. 4A , the center-lines represent the center-lines of different vessels. The vessel center-lines may be defined by cubic splines, e.g., Catmul-Rom splines or any other suitable mathematical function. The processing may further include, additionally or alternatively, computation of the radii of the various vessels. 
     According to some embodiments, generating the image-based model of the anatomical structure depicted in the medical image may include the process of registration. Registration may be defined as the alignment of multiple data sets into a single coordinate system such that the spatial locations of corresponding points would coincide. The segmented portions of the anatomical structure may be properly located and rotated with respect to other models representing the anatomy of the patient. Such other models may be generic or patient-specific models as described herein. For example, registering a simulated blood vessel may include properly locating it with respect to digital models of the bone structure, internal organs or lungs of the patient. Registration may enable presenting a real-life image of the subject being treated. Registration may further enable rotating or otherwise relocating or repositioning in space a simulated model while preserving the relative position of organs, sections, regions or parts in the simulated model. 
     According to some embodiments, the procedures, tasks and/or functions associated with the segmentation, generation of polygonal meshes, computing the centerlines and/or radii may be performed in the segmentation stage or at simulation start up by either model generation unit  110  or simulation unit  115 . It should be understood by a person skilled in the art that although the described herein are typically performed, other implementations may exist. For example, simulation unit  115  may generate a model using as input the set of voxels without performing surface rendering and/or computation of centerlines or radii and/or registration. It will be recognized that embodiments of the invention are not limited by the exemplary method or system described for generating an image-based basic model of an anatomical structure and other methods may be used for generating such a model without departing from the scope of the invention. 
     Returning back to  FIG. 3 , according to embodiments of the invention, the method, may include generating or calculating an extended model of the anatomical structure by extrapolating boundaries of the basic model depicting the medical image data (box  320 ). Then, the method may include performing a patient-specific simulation of an image-guided procedure according to the extended model (box  325 ). According to embodiments of the invention, parts, sections or regions missing from a medical image data of a subject may be simulated or modeled by extrapolating a model calculated from the medical image data.  FIG. 4B  shows an extended model produced by identifying to boundaries in basic model  410  of  FIG. 4  and extrapolating the basic model  410  and adding extrapolated sections  420 ,  421  as explained in detail with respect to  FIGS. 5 ,  6 ,  7 A and  7 B. 
     Reference is made to  FIG. 5  which is an exemplary flowchart describing a method of generating an extended model for patient-specific simulation of an image-guided procedure according to embodiments of the invention. Reference is additionally made to  FIGS. 6 ,  7 A and  7 B showing graphical illustrations of exemplary polygonal meshes produced according to embodiments of the invention. Embodiments of the invention may be applicable to various anatomical structures, systems, sections, regions, organs or parts thereof. Accordingly, the method described herein with reference to  FIG. 5  may be applicable to generating an extended model of any such anatomical structure. However, for the sake of simplicity and clarity the discussion below refers to as an exemplary illustration to the vasculature. It will be noted that the embodiments of the invention are not limited in this respect. For ease of explanation, embodiments of the invention are described with respect to polygonal meshes. It should be however understood to a person skilled in the art that the invention is not limited in to such models and other models are within the scope of the invention. 
     According to the exemplary embodiment of  FIG. 5 , the input to the extrapolation phase is an image-based model that is represented as a polygonal mesh representing 3D surfaces of the image-based model, properly registered and rotated including further the vessel centerlines represented by splines. As shown by box  510 , the method may include identifying one or more boundaries of the polygonal mesh, such as mesh  410  of  FIG. 4A  or mesh  710  of  FIG. 7A . For example, the boundaries of the polygonal mesh may be detected by detecting edges associated with only one polygon. The detected edges may be grouped into one or more connectivity components where each component is associated with an open boundary of a single vessel that may potentially require extrapolation. 
     According to an exemplary embodiment of the invention, the method may include selecting for each boundary that requires extrapolation a generic pre-designed model represented for example as a polygonal mesh, such as mesh  610  of  FIG. 6 , to be used as a model for the extended section (box  515 ). The generic model may be selected from a set of pre-designed models to best match the open boundary edge according to certain parameters and rules. The generic model may be represented by a polygonal mesh to be used as an approximation or estimation of a region or section not included in an image-based model generated according to the medical image data. 
     The selected generic model, e.g., polygonal mesh, may be retrieved from a database, library or any other repository of generic meshes. Such repository may store a large number of polygonal meshes or other models, each corresponding, for example, to a particular vessel at a particular anatomical location and has known physical characteristics such as length and radius. According to embodiments of the invention, a repository of pre-generated models or polygonal meshes may be accessed in order to obtain a polygonal mesh closest to or best matching an open boundary of the polygonal mesh representing the basic or image-based model. For example, polygonal mesh  610  may be chosen as a basis or starting point for generating an extended section  720  for section  710  in boundary  715  as illustrated in  FIGS. 7A and 7B . The selection of the best match may be performed manually or automatically based on a predefined set of geometric attributes of the segmented model. Alternatively, the selection may be made by using both manual and automated operations. According to embodiments of the invention, the system may send a user a recommendation to select a specific generic model and the user may them manually confirm of deny the recommendation. 
     Then, as shown by box  525 , the exemplary embodiment may include positioning the selected mesh in alignment with the boundary. An exemplary alignment method may utilize centerlines associated with a library of predefined extrapolation meshes and may include computing a tangent vector to the centerline at the edge of the segmented region near the boundary. Such tangent vector may be computed with relation to a point at the edge or boundary detected. Next, a point on a centerline of the predefined or generated polygonal mesh may be located. Such point may be related to or have a tangent vector that is close or identical to the tangent value computed for the boundary of the simulated organ. Using points with similar tangent vector may enable positioning the predefined polygonal mesh in correct orientation with respect to the simulated organ or the polygonal mesh representing it. 
     Then, as shown by box  530 , the exemplary embodiment may include connecting the centerline of the polygonal mesh representing the image-based model with the centerline of the selected predefined polygonal mesh as illustrated by  FIG. 7A . The connection of the centerlines may be done in accordance with the computed tangent values as described herein. The centerlines may be joined, bonded or welded such that the points with close, similar or identical tangent value coincide to ensure that the basic model and the added section may be similarly positioned or orientated in a 3D space. 
     Next, as shown by box  535 , the exemplary embodiment may include removing of redundant sections. As the registration of the boundary of the basic model represented by the basic polygonal mesh is known, a virtual box, sphere or other structured volume may be defined such that it may contain the polygonal mesh. Accordingly, parts of the added section extending beyond or outside such virtual volume may be removed as they may be redundant. Alternatively, only vertices of the selected a mesh which lie inside such virtual box or volume may be kept for further processing as described herein. 
     As shown by box  540 , the exemplary embodiment may include scaling the radius of the selected generic mesh to better match the radius of the basic polygonal mesh representing the basic model. The shape of the selected predefined polygonal mesh may not fit or perfectly match that of the boundary or open end to which it is connected. Moreover, although aligned or orientated as described herein, the centerlines of the polygonal mesh representing the basic model and the selected polygonal mesh representing the extended section may still be spatially apart. Additionally, for example, in the case of a simulated blood vessel, a cross-section of the predefined, extrapolated section being matched to an open end may not be a perfect circle, nor does its centerline guaranteed to be an exact mathematical centerline of a cylinder. Accordingly, a different scaling factor, possibly for every angle around the centerline may be applied. 
     Then, as shown by box  545 , the exemplary embodiment may include welding the surface of selected polygonal mesh with the surface of the basic polygonal mesh. For example, the basic polygonal mesh and the selected predefined polygonal mesh may be joined such that a continuous surface is produced as illustrated by  FIG. 7B . Vertices from the boundary of the basic polygon mesh and from the corresponding selected mesh of the extended section may be joined and represented by a single vertex while redundant vertices may be removed from the model. The exemplary method described above may be repeated for each open end, boundary or edge of the basic model that needs to be extended. 
     It should be understood to a person skilled in the art that the predefined generic models representing the extended anatomy may be represented in many methods and models, different or similar to the basic model. According to some embodiments, the model representing the extended regions may be different than the basic model. Accordingly, the model representing an extrapolated region may be converted to another format or representation to comply with the basic model. The extrapolated region in the converted representation may then be welded or otherwise connected to the basic model. 
     Alternatively, according to other embodiments of the invention, a polygonal mesh or another suitable model representing the extrapolated section may be generated procedurally. The procedural generation of the extrapolated sections or regions may include identifying key attributes of the basic model such as the boundaries to be extended, relevant radii, centerlines and the like. For example, centerlines of a generated basic model of a blood vessel may be extrapolated according to a first set of rules. A non-limiting example for the first set of rules may include rules directed to smoothly curving the extrapolation towards a pre-determined direction. Then, the radii function of the simulated blood vessel model may be extrapolated, for example, by using a second set of rules. A non-limiting example for the second set of rules may include varying the radius smoothly from the identified value of the basic model toward a predefined value (e.g. 1 cm) for a predefined length and then smoothly changing the radius to zero at a predefined distance from the tip (e.g. 2 mm from the tip). Then, using the extrapolated centerline and radius-function, a unified polygonal mesh representing both the basic model and its extrapolation may be generated. 
     Embodiments of the invention may include an article such as a computer or processor readable medium, or a computer or processor storage medium, such as for example a memory, a disk drive, or a USB flash memory, encoding, including or storing instructions, e.g., computer-executable instructions, which when executed by a processor or controller, carry out methods disclosed herein. 
     Although embodiments of the invention are not limited in this regard, the terms “plurality” and “a plurality” as used herein may include, for example, “multiple” or “two or more”. The terms “plurality” or “a plurality” may be used throughout the specification to describe two or more components, devices, elements, units, parameters, or the like. 
     Unless explicitly stated, the method embodiments described herein are not constrained to a particular order or sequence. Additionally, some of the described method embodiments or elements thereof can occur or be performed at the same point in time or overlapping points in time. As known in the art, an execution of an executable code segment such as a function, task, sub-task or program may be referred to as execution of the function, program or other component. 
     Although embodiments of the invention are not limited in this regard, discussions utilizing terms such as, for example, “processing,” “computing,” “calculating,” “determining,” “establishing”, “analyzing”, “checking”, or the like, may refer to operation(s) and/or process(es) of a computer, a computing platform, a computing system, or other electronic computing device, that manipulate and/or transform data represented as physical (e.g., electronic) quantities within the computer&#39;s registers and/or memories into other data similarly represented as physical quantities within the computer&#39;s registers and/or memories or other information storage medium that may store instructions to perform operations and/or processes. 
     While certain features of the invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents may occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the invention.