Abstract:
A method includes receiving an intra-procedure image of a region of interest of an object. The intra-procedure image includes a representation of an instrument in the object. The method further includes identifying a location of the instrument in the object in the intra-procedure image. The method further includes combining the intra-procedure image with a pre-procedure image of the region of interest. The pre-procedure image includes a planned trajectory for the instrument from a surface of the object to a target in the object and segmented structure(s) to be avoided by the instrument during the procedure. The method further includes segmenting the same structure(s) in the intra-procedure image relative to the instrument using the segmented structure(s) in the pre-procedure image. The method further includes computing a risk of the instrument contacting the structure(s) along the trajectory with the identified location, the trajectory, and the segmented structure(s) in the intra-procedure image.

Description:
TECHNICAL FIELD 
       [0001]    The following generally relates to an imaging and more particularly to real-time image based risk assessment for an instrument along a path to a target in an object, and is described with particular application to ultrasound imaging, but is also amenable to other imaging modalities. 
       BACKGROUND 
       [0002]    Imaging has been used to guide procedures such as biopsies, resections, cryotherapy, ablations, etc. For instance, cancerous tumors have been treated using a needle that is inserted into the tumor, where the needle is used to ablate a region around a tip of the needle. The insertion of the needle to the tumor has been guided with an image. For example, the physician first views the image to determine how far the needle should travel and an appropriate trajectory by eye before the treatment commences. During the procedure, the clinician cognitively assess a risk of injury to neighboring anatomical structures such as the lungs, heart and other organs or major blood vessels as the needle is advanced to the target tissue along the trajectory. Unfortunately, due to a variety of factors including patient motion and needle deflection, it may be difficult for the clinician to determine if the needle is coming close to any of the neighboring anatomical structures solely based on the cognitive assessment as the needle is advanced. Furthermore, the clinician may have to advance the needle in small incremental steps and continuously review and interpret the image, which is a tedious and time consuming process that may increase in overall time, radiation dose when under X-ray guidance and cost of the procedure. 
       SUMMARY 
       [0003]    Aspects of the application address the above matters, and others. 
         [0004]    In one aspect, a method includes receiving an intra-procedure image of a region of interest of an object. The intra-procedure image includes a representation of an instrument in the object. The method further includes identifying the location of the instrument in the object in the intra-procedure image. The method further includes combining the intra-procedure image with a pre-procedure image of the region of interest of the object. The pre-procedure image includes a planned trajectory for the instrument from a surface of the object to a target in the object and segmented structure(s) to be avoided by the instrument during the procedure. The method further includes segmenting the same structure(s) in the intra-procedure image relative to the instrument using the segmented structure(s) in the pre-procedure image from the combined images. The method further includes computing a risk of the instrument contacting the structure(s) along the trajectory with the identified location of the instrument, the trajectory, and the segmented structure(s) in the intra-procedure image. The method further includes presenting the risk. 
         [0005]    In another aspect, an apparatus includes an instrument identifier that receives a real-time image generated during a procedure and identifies a position of an instrument in the real-time image. The apparatus further includes a segmentor that segments a predetermined set of structures in the real-time image using previously segmented structure in a previously generated image. The apparatus further includes a risk assessor that computes a risk of the instrument damaging the structures from the identified position, the segmented structures in the real-time image, and a planned path for the instrument in the previously generated image. The apparatus further includes a plan evaluator that evaluates the plan using the computed risk and presents information indicating the risk along the planned path. 
         [0006]    In another aspect, a non-transitory computer readable medium is encoded with computer executable instructions, which, when executed by a computer processor, causes the processor to: determine a location of a needle in an object in an intra-procedure image, fuse the intra-procedure image with a pre-procedure image, segment structure in the intra-procedure image with structure previously segmented in the pre-procedure image, and determine a risk of the needle coming into contact with the structure based on a pre-planned needle path to tissue of interest in the object, the location of the needle, and the segmented structure in the intra-procedure image. 
         [0007]    Those skilled in the art will recognize still other aspects of the present application upon reading and understanding the attached description. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]    The application is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements and in which: 
           [0009]      FIG. 1  schematically illustrates an example ultrasound imaging system in connection with procedure planning and evaluation components; 
           [0010]      FIG. 2  illustrates an example method in accordance with an embodiment herein; 
           [0011]      FIG. 3  shows an example pre-operative image; 
           [0012]      FIG. 4  shows an example trajectory to a target structure in the pre-operative image; 
           [0013]      FIG. 5  shows example contours of segmented structures in a pre-operative image; 
           [0014]      FIG. 6  shows an example instrument location in a real-time image; and 
           [0015]      FIG. 7  shows segmented structure in a real-time image relative to the instrument location. 
       
    
    
     DETAILED DESCRIPTION 
       [0016]    The following describes an approach for assessing risk of contact of an instrument with structure(s) in an object, during a procedure in which the instrument is advanced in the object to a target destination, based on segmentation and tracking of objects using feedback based on analysis of real-time images. By non-limiting example, this includes assessing a risk of damage to an anatomical structure located in a vicinity of a needle trajectory as the needle is guided towards target tissue of interest such as a lesion or tumor during an image guided procedure and also when the treatment is performed. 
         [0017]    Initially referring to  FIG. 1 , an ultrasound imaging system  100  includes a probe  102  housing a transducer array  104  having at least one transducer element  106 . The at least one transducer element  106  is configured to convert electrical signals to an ultrasound pressured field and vice versa respectively to transmit ultrasound signals into a field of view and receive echo signals, generated in response to interaction with structure in the field of view, from the field of view. The illustrated transducer array  104  can include one or more arrays, including linear, curved (e.g., concave, convex, etc.), circular, etc. arrays, which are fully populated or sparse, etc. 
         [0018]    Transmit circuitry  108  generates a set of pulses (or a pulsed signal) that are conveyed, via hardwire (e.g., through a cable) and/or wirelessly, to the transducer array  104 . The set of pulses excites a set (i.e., a sub-set or all) of the at least one transducer element  106  to transmit ultrasound signals. Receive circuitry  110  receives a set of echoes (or echo signals) generated in response to a transmitted ultrasound signal interacting with structure in the field of view. A switch (SW)  112  controls whether the transmit circuitry  108  or the receive circuitry  110  is in electrical communication with the at least one transducer element  106  to transmit ultrasound signals or receive echoes. 
         [0019]    A beamformer  114  processes the received echoes by applying time delays to echoes, weighting echoes, summing delayed and weighted echoes, and/or otherwise beamforming received echoes, creating beamformed data. In B-mode imaging, the beamformer  114  produces a sequence of focused, coherent echo samples along focused scanlines of a scanplane. The scanplanes correspond to the imaging planes of the transducer array  104 . The beamformer  114  may also process the scanlines to lower speckle and/or improve specular reflector delineation via spatial compounding, and/or perform other processing such as FIR filtering, IIR filtering, edge enhancement, etc. 
         [0020]    A user interface (UI)  116  includes an input device(s) (e.g., a physical button, a touch screen, etc.) and/or an output device(s) (e.g., a touch screen, etc.), which allow for interaction between a user and the ultrasound imaging system  100 . A display  118  is configured to display an image and/or other information. A controller  120  controls one or more of the components  102 - 114  of the ultrasound imaging system  100 . Such control includes controlling one or more of the components to perform the functions described herein and/or other functions. 
         [0021]    Pre-procedure image memory  122  stores a pre-procedure image. The memory  122  can be local to the ultrasound imaging system  100 , remote therefrom, e.g., memory of a data repository such as a picture arching and communication system (PACS), a radiology information system (RIS), etc., distributed between the ultrasound imaging system  100  and one or more remote memories, etc. The pre-procedure image can include 2-D and/or 3-D image data, and can be generated by the ultrasound imaging system  100 , another ultrasound imaging system and/or another imaging modality such as a magnetic resonance imaging (MRI) system, a computed tomography (CT) imaging system, and/or other imaging system. 
         [0022]    A trajectory and structure of interest identifier  124  processes the pre-procedure image and identifies a trajectory to a target structure of interest, which represents a path for an instrument (e.g., a needle) to the target structure of interest. In one example, the trajectory is automatically generated in response to an identification of the target structure of interest on the pre-procedure image, which can be identified based on a user input and/or otherwise. In another instance, the trajectory is semi-automatically or manually identified by a user, e.g., via freehand drawing, etc. In either instance, the user can reject, modify or accept the trajectory. In addition, the identifier  124  can recommend candidate trajectories and/or modifications to trajectories. 
         [0023]    A segmentor  126  segments other predetermined structure(s) from the pre-procedure image such as one or more structures that are not to be traversed by the identified trajectory. In the context of a tumor biopsy, the other structure may include organs, blood vessels, etc. Such structure(s) can be identified via a user input, a default set of structures for the structure of interest, a clinician customized set of structure, and/or other structure. Known and/or other segmentation algorithms can be employed. The segmentation algorithm can be automatic, semi-automatic or manual. In any instance, the user can reject, modify or accept the segmentation, and/or add a structure to segment and/or remove a segmentation. 
         [0024]    A risk assessor  128  processes the pre-procedure image along with the identified trajectory and the segmented other structure(s) and computes a risk for the entire length of the trajectory or multiple risks, each for a sub-segment of the entire trajectory. The risk identifies a risk of contacting the structure and/or otherwise affecting (e.g., damaging, etc.) the structure with the instrument. Known and/or other risk assessment algorithms can be employed. For example, risk can be determined based on distance between the trajectory and the segmented other structure at one or more points along the trajectory. This includes determining different levels of risk for different points along the trajectory one point on the trajectory may be associated with greater risk relative to another point. Additionally or alternatively, the risk is computed based on the area that is affected by the treatment. 
         [0025]    A plan creator  130  creates an examination procedure plan based on the pre-procedure image, the identified trajectory, the segmented other structure(s), and the risk. This can be achieved automatically and/or semi-automatically. This may include evaluating the current risk based on a predetermined set of rules, a predetermined risk threshold, etc. to determine if the current risk level is satisfactory. Where the risk level is not satisfactory, the trajectory can be discarded and a new one generated by components  124 - 128  or modified by a user, e.g., both based on different input and/or processing parameters. 
         [0026]    In one instance, the risk information is presented, visually, audibly, etc. for observation. For example, the risk level can be identified through color coding the trajectory and/or a region around the trajectory, e.g., where red represents high risk, yellow represents medium risk and green represents low risk. In an alternative configuration, the risk level is provided as a numerical value in conjunction with a range from low to high risk and optional explanatory text on the image about the trajectory and/or otherwise. Other risk representations are also contemplated herein. 
         [0027]    With this information, a user can accept or reject a current trajectory. This may include accepting a trajectory the plan creator  130  recommends to reject, rejecting a trajectory the plan creator  130  recommends to accept, as well as accepting the recommendation of the plan creator  130 , where the plan creator  130  is configured to provide a recommendation. Rejection may invoke starting the process over again and/or modifying the existing rejected trajectory. 
         [0028]    A plan memory  132  stores accepted plans. One or more of the planning components ( 122 - 132 ) can be part of the system  100  and/or an apparatus separate therefrom. 
         [0029]    During a procedure, an instrument identifier  134  processes an intra-procedure real-time image and identifies a location of an instrument therein. As utilized herein, a real-time image refers to a currently or presently generated image, generated from currently or presently acquired data (i.e., ultrasound echoes). In the illustrated example, the real-time image is a real-time ultrasound image, which can be a 2-D and/or a 3-D image. In another embodiment, the real-time image can alternatively or additionally be an MRI, CT, etc. 2-D and/or a 3-D image. Known and/or other risk assessment algorithms can be employed. For example, the instrument can be identified via a tracking device (e.g., an electromagnetic sensor thereon), pixel intensity, a change in pixel intensity, object shape in the image, etc. 
         [0030]    A plan evaluator  136  combines (e.g., fuses, registers, overlays, etc.) the intra-procedure real-time image with the location of the instrument identified therein with the pre-procedure image where the previously segmented structure was identified. This can be through a registration algorithm and/or otherwise. From this, the plan evaluator  136  automatically locates current locations of the structure in the real-time image relative to the location of the instrument in the real-time image based on the previously segmented structure and computes a new risk. Similar to planning, the risk can be based on the proximity of the instrument with respect to the identified structure along the trajectory and the risk can be presented and/or evaluated based on rules, thresholds, etc. 
         [0031]    Where the risk level is satisfactory, the instrument can be advanced and another real-time image can be generated and evaluated to evaluate the plan. Where the risk level is not satisfactory, in one instance, the procedure is terminated and the planning process is revisited. This may involve removing the instrument and re-scheduling the procedure. In another instance, the plan is instead modified during the procedure, and the procedure is finished. The modification may include changing the trajectory, identifying additional or fewer structures to avoid, etc. For a procedure involving a treatment, once the instrument is guided to the target tissue and the treatment is performed, the plan evaluator  136  can generate a final risk, e.g., to determine an extent of the treatment. 
         [0032]    With the configuration described herein, the previously identified structures are automatically located in the intra-procedure real-time image relative the current location of the instrument and real-time trajectory risk is provided. As such, during the procedure the clinician does not have to cognitively assess a risk of injury to neighboring anatomical structures as the needle is inserted and moved in the patient. Furthermore, one or more of the intra-procedure components ( 134  and  136 ) can be part of the system  100  and/or an apparatus separate therefrom. Furthermore, the plan evaluator  136  can employ the components  126 - 130  (as shown in the illustrated example) or have and/or utilize one or more of its own corresponding components. 
         [0033]    In the illustrated example, at least one of the components of the system  100  can be implemented via one or more computer processors (e.g., a microprocessor, a control processing unit, a controller, etc.) executing one or more computer readable instructions encoded or embodied on computer readable storage medium (which excludes transitory medium), such as physical computer memory, which causes the one or more computer processors to carry out the various acts and/or other functions and/or acts described herein. Additionally or alternatively, the one or more computer processors can execute instructions carried by transitory medium such as a signal or carrier wave. 
         [0034]      FIG. 2  illustrates an example method in accordance with an embodiment herein. 
         [0035]    It is to be appreciated that the ordering of the above acts is not limiting. As such, other orderings are contemplated herein. In addition, one or more acts may be omitted and/or one or more additional acts may be included. 
         [0036]    At  202 , a pre-procedure image is obtained, as described herein and/or otherwise.  FIG. 3  shows an example pre-procedure image. 
         [0037]    At  204 , a target trajectory to a target structure is identified in the pre-procedure image, as described herein and/or otherwise.  FIG. 4  shows an example trajectory  402  to a target structure  404 . 
         [0038]    At  206 , structures to be avoided during the procedure are identified in the pre-procedure image, as described herein and/or otherwise. 
         [0039]    At  208 , the identified structures are segmented in the pre-procedure image, as described herein and/or otherwise.  FIG. 5  shows example contours  502  and  504  of such structures. 
         [0040]    At  210 , a risk is computed for the trajectory  402  and the segmented structures  502  and  504 , as described herein and/or otherwise. 
         [0041]    At  212 , an instrument is located in a real-time intra-procedure image, as described herein and/or otherwise.  FIG. 6  shows example in which an instrument  602  is located in a real-time image. 
         [0042]    At  214 , the real-time image is combined with the pre-procedure image where structures were previously identified, as described herein and/or otherwise. 
         [0043]    At  216 , the segmented structures in the pre-procedure image are located in the real-time image relative to the instrument location, as described herein and/or otherwise.  FIG. 7  shows current locations of structure  702  and  704  relative to the current location of the instrument  602 . 
         [0044]    At  218 , the risk is updated based on the real-time image, as described herein and/or otherwise. 
         [0045]    At  220 , acts  214 - 218  are repeated for subsequent real-time images until the instrument is at the target, as described herein and/or otherwise. 
         [0046]    Optionally, after the procedure at the target, a final risk is computed to determine an extent of treatment. 
         [0047]    At least a portion of one or more of the methods discussed herein may be implemented by way of computer readable instructions, encoded or embedded on computer readable storage medium (which excludes transitory medium), which, when executed by a computer processor(s), causes the processor(s) to carry out the described acts. Additionally or alternatively, at least one of the computer readable instructions is carried by a signal, carrier wave or other transitory medium. 
         [0048]    The application has been described with reference to various embodiments. Modifications and alterations will occur to others upon reading the application. It is intended that the invention be construed as including all such modifications and alterations, including insofar as they come within the scope of the appended claims and the equivalents thereof.