Patent Description:
Separately, registration is used to align the coordinate systems of two separate devices and/or systems. For example, registration from an OSS device to an X-Ray imaging system can be accomplished via a transform TOX from the OSS device to the X-Ray imaging system. Registration from an ultrasound imaging system to the X-Ray imaging system can be accomplished via a transform TUX from the ultrasound imaging system to the X-Ray imaging system. Registration from the OSS device to the ultrasound imaging system can be accomplished via a transform TOU from the OSS device to the ultrasound imaging system.

Additionally, segmentation is used in medical imaging systems to represent surfaces of structures as three-dimensional models.

Currently, registration between the OSS device and the X-Ray imaging system may accumulate a noticeable error, such as when the proximal end of the OSS device (i.e., nearest the user) is moved several centimeters. Correction of the error requires re-registration between the OSS device and the X-Ray imaging system, which in turn requires two new offset X-Ray projections. The additional X-Ray projections may interrupt the flow of the interventional medical procedure, may extend the time of the interventional medical procedure, and subject the patient and clinician to additional X-Ray dosages.

Registration between the OSS device and the ultrasound imaging system may also accumulate a notice error. Re-registration between the OSS device and the ultrasound imaging system may interrupt the flow of the interventional medical procedure and extend the time of the interventional medical procedure while image analysis software searches through the latest ultrasound imagery for the OSS device. The image analysis software may require a designation of the OSS device in the ultrasound imagery, such as from a designation of a tip of the OSS device in the ultrasound imagery by a user, in order to constrain a search for the OSS device in the ultrasound imagery. Even with an initial constraint, the OSS device must be fully identified and located in the ultrasound imagery for the re-registration between the OSS device and the ultrasound imaging system, and this interrupts the flow of the interventional medical procedure and extends the time of the interventional medical procedure. <CIT> discloses a method to efficiently register an OSS device to an imaging system by matching over time a stable curvature of an OSS device with a curvature from an imaging system and aligns the matched curvatures.

The invention is solely defined by the appended claims.

According to an aspect of the present disclosure, a system for tracking location of an interventional medical device in an interventional medical procedure includes an interface and a controller. The interface interfaces an optical shape sensing device which has a shape that conforms to a shape of the interventional medical device during the interventional medical procedure. The controller includes a memory that stores instructions and a processor that executes the instructions. When executed by the processor, the instructions cause the system to identify a shape of the optical shape sensing device using optical shape sensing signals received from the optical shape sensing device via the interface, and identify the interventional medical device in a first coordinate space of a first imaging system that images the interventional medical device in a first imaging mode during the interventional medical procedure, based on identifying the shape of the optical shape sensing device. The instructions also cause the system to register the interventional medical device to the first coordinate space, identify the interventional medical device in a second coordinate space of a second imaging system that images the interventional medical device in a second imaging mode during the interventional medical procedure, and register the first coordinate space of the first imaging system to the second coordinate space of the second imaging system. The instructions further cause the system to segment the interventional medical device in the second coordinate space to obtain a segmented representation of the interventional medical device in the second coordinate space, register the interventional medical device to the second coordinate space using the segmented representation of the interventional medical device, and re-register the interventional medical device to the first coordinate space based on registering the interventional medical device to the second coordinate space.

According to another aspect of the present disclosure, a tangible non-transitory computer readable storage medium stores a computer program. The computer program, when executed by a processor, causes a system that includes the tangible non-transitory computer readable storage medium to identify a shape of the optical shape sensing device using optical shape sensing signals received via an interface from an optical shape sensing device which has a shape that conforms to a shape of the interventional medical device during the interventional medical procedure, and to identify the interventional medical device in a first coordinate space of a first imaging system that images the interventional medical device in a first imaging mode during the interventional medical procedure, based on identifying the shape of the optical shape sensing device. The instructions also cause the system to register the interventional medical device to the first coordinate space, identify the interventional medical device in a second coordinate space of a second imaging system that images the interventional medical device in a second imaging mode during the interventional medical procedure, and register the first coordinate space of the first imaging system to the second coordinate space of the second imaging system. The instructions further cause the system to segment the interventional medical device in the second coordinate space to obtain a segmented representation of the interventional medical device in the second coordinate space, register the interventional medical device to the second coordinate space using the segmented representation of the interventional medical device, and re-register the interventional medical device to the first coordinate space based on registering the interventional medical device to the second coordinate space.

According to yet another aspect of the present disclosure, a method for tracking location of an interventional medical device in an interventional medical procedure includes identifying a shape of the optical shape sensing device using optical shape sensing signals received via an interface from an optical shape sensing device which has a shape that conforms to a shape of the interventional medical device during the interventional medical procedure, and identifying the interventional medical device in a first coordinate space of a first imaging system that images the interventional medical device in a first imaging mode during the interventional medical procedure, based on identifying the shape of the optical shape sensing device. The method also includes registering the interventional medical device to the first coordinate space, identifying the interventional medical device in a second coordinate space of a second imaging system that images the interventional medical device in a second imaging mode during the interventional medical procedure, and registering the first coordinate space of the first imaging system to the second coordinate space of the second imaging system. The method further includes segmenting the interventional medical device in the second coordinate space to obtain a segmented representation of the interventional medical device in the second coordinate space, registering the interventional medical device to the second coordinate space using the segmented representation of the interventional medical device, and re-registering the interventional medical device to the first coordinate space based on registering the interventional medical device to the second coordinate space.

The example embodiments are best understood from the following detailed description when read with the accompanying drawing figures. It is emphasized that the various features are not necessarily drawn to scale. In fact, the dimensions may be arbitrarily increased or decreased for clarity of discussion. Wherever applicable and practical, like reference numerals refer to like elements.

In the following detailed description, for the purposes of explanation and not limitation, representative embodiments disclosing specific details are set forth in order to provide a thorough understanding of an embodiment according to the present teachings. Descriptions of known systems, devices, materials, methods of operation and methods of manufacture may be omitted so as to avoid obscuring the description of the representative embodiments. Nonetheless, systems, devices, materials and methods that are within the purview of one of ordinary skill in the art are within the scope of the present teachings and may be used in accordance with the representative embodiments. It is to be understood that the terminology used herein is for purposes of describing particular embodiments only and is not intended to be limiting. The defined terms are in addition to the technical and scientific meanings of the defined terms as commonly understood and accepted in the technical field of the present teachings.

It will be understood that, although the terms first, second, third etc. may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another element or component. Thus, a first element or component discussed below could be termed a second element or component without departing from the teachings of the inventive concept.

The terminology used herein is for purposes of describing particular embodiments only and is not intended to be limiting. As used in the specification and appended claims, the singular forms of terms 'a', 'an' and 'the' are intended to include both singular and plural forms, unless the context clearly dictates otherwise. Additionally, the terms "comprises", and/or "comprising," and/or similar terms when used in this specification, specify the presence of stated features, elements, and/or components, but do not preclude the presence or addition of one or more other features, elements, components, and/or groups thereof.

Unless otherwise noted, when an element or component is said to be "connected to", "coupled to", or "adjacent to" another element or component, it will be understood that the element or component can be directly connected or coupled to the other element or component, or intervening elements or components may be present. That is, these and similar terms encompass cases where one or more intermediate elements or components may be employed to connect two elements or components. However, when an element or component is said to be "directly connected" to another element or component, this encompasses only cases where the two elements or components are connected to each other without any intermediate or intervening elements or components.

The present disclosure, through one or more of its various aspects, embodiments and/or specific features or sub-components, is thus intended to bring out one or more of the advantages as specifically noted below. For purposes of explanation and not limitation, example embodiments disclosing specific details are set forth in order to provide a thorough understanding of an embodiment according to the present teachings. Moreover, descriptions of well-known apparatuses and methods may be omitted so as to not obscure the description of the example embodiments. Such methods and apparatuses are within the scope of the present disclosure.

As described herein, interventional medical device tracking may leverage three-dimensional segmentation of interventional medical devices in an ultrasound volume and registration of an ultrasound imaging system to an X-Ray imaging system. Interventional medical device tracking may enable accurate maintenance of registration of an OSS device to both the ultrasound imaging system and the X-Ray imaging system throughout the duration of an interventional medical procedure and without repeating the X-Ray imaging.

As also described herein, use of the shape of an OSS device may enhance a process for identifying the OSS device in imagery such as from an ultrasound imaging system, and this in turn may enhance the registration processes described below.

<FIG> illustrates a system for interventional medical device tracking, in accordance with a representative embodiment.

<FIG> illustrates a tracking system <NUM>. The tracking system <NUM> includes a console with a controller <NUM>, an interface <NUM> and a touch panel <NUM>. The controller <NUM> includes at least a memory <NUM> that stores instructions and a processor <NUM> that executes the instructions. The controller <NUM> controls one or more aspects of methods described herein. The processor <NUM> retrieves or otherwise receives instructions from the memory <NUM> via a bus (not shown). When executed by the processor <NUM>, the instructions cause the controller <NUM> to implement one or more aspects of the methods shown in and described with respect to <FIG>, <FIG>, <FIG> and <FIG>. The interface <NUM> provides an interface between the console that includes the controller <NUM> and an optical shape sensing device <NUM>. The interface <NUM> is representative of interfaces between elements and components of the tracking system <NUM>. The touch panel <NUM> includes buttons, keys and any other touch surfaces that can be used to input instructions from a user to the tracking system <NUM>.

The tracking system <NUM> also includes a monitor <NUM>, an X-Ray imaging system <NUM>, and an ultrasound imaging system <NUM>. The monitor <NUM> may be used to display images from the X-Ray imaging system <NUM> and the ultrasound imaging system <NUM>. As a non-limiting example, the X-Ray imaging system <NUM> may perform fluoroscopic imaging during the interventional medical procedure. Also as a non-limiting example, the ultrasound imaging system <NUM> may perform transesophageal echocardiography (TEE) or other forms of ultrasound imaging. The X-Ray imaging system <NUM> performs imaging in a three-dimensional coordinate space that may be centered at an isocenter of a C-arm of the X-Ray imaging system <NUM>. The ultrasound imaging system <NUM> performs imaging in another three-dimensional coordinate space. The three-dimensional coordinate space of the ultrasound imaging system <NUM> and other three-dimensional coordinate spaces may be registered to the three-dimensional coordinate space of the X-Ray imaging system <NUM> so that the isocenter of the C-arm of the X-Ray imaging system <NUM> becomes the origin of all such registered coordinate spaces.

The tracking system <NUM> also includes an interventional medical device <NUM> integrated with the optical shape sensing device <NUM>. The optical shape sensing device <NUM> may be flexible and may have a shape that flexibly conforms to a shape of the interventional medical device <NUM> during the interventional medical procedure. In the descriptions herein, references to the interventional medical device <NUM> are also to the optical shape sensing device <NUM> insofar as the optical shape sensing device <NUM> is integrated with the interventional medical device <NUM>. On the other hand, references to the optical shape sensing device <NUM> may be particular to the optical shape sensing device <NUM> independent of the interventional medical device <NUM> insofar as the optical shape sensing device <NUM> independently interfaces the controller <NUM> via the interface <NUM> to provide optical shape sensing signals generated by the optical shape sensing device <NUM>.

The elements and components of the tracking system <NUM> in <FIG> may be provided together or may be distributed. For example, the controller <NUM>, the monitor <NUM> and the touch panel <NUM> may be provided as an integrated computer system that is provided separately from the X-Ray imaging system <NUM>, the ultrasound imaging system <NUM> and the interventional medical device <NUM>. The X-Ray imaging system <NUM>, the ultrasound imaging system <NUM> and the interventional medical device <NUM> may be provided separately from one another, and may be integrated together via an integrated computer system that includes the controller <NUM>, the monitor <NUM> and the touch panel <NUM>.

The controller <NUM> may include one or more input interface(s) in addition to the interface <NUM>. The interface <NUM> and other input interfaces (not shown) of the controller <NUM> may include cables, adapters, ports, disk drives, antennas for wireless communications, and other forms of interfaces specifically used to connect elements and components of the tracking system <NUM>. The input interfaces may further connect user interfaces, such as a mouse, a keyboard, a microphone, a video camera, a touchscreen display, or another element or component to the controller <NUM>. The interfaces of the tracking system <NUM> may connect the controller <NUM> to the monitor <NUM>, to the X-Ray imaging system <NUM>, and to the ultrasound imaging system <NUM>. For example, the controller <NUM> may be connected to the monitor <NUM> via a local wired interface such as an Ethernet cable or via a local wireless interface such as a Wi-Fi connection.

The monitor <NUM> may be a computer monitor, a display on a mobile device, a television, an electronic whiteboard, or another screen configured to display electronic imagery. The monitor <NUM> may also include one or more input interface(s) such as those noted above that may connect other elements or components to the monitor <NUM>. The monitor <NUM> may also include a touch screen that enables direct input via touch.

In one set of embodiments, the tracking system <NUM> tracks the interventional medical device <NUM> during an interventional medical procedure. The X-Ray imaging system <NUM> may be a first imaging system that images the interventional medical device <NUM> during the interventional medical procedure, and the ultrasound imaging system <NUM> may be a second imaging system that images the interventional medical device <NUM> during the interventional medical procedure. When executed by the processor <NUM>, the instructions stored in the memory <NUM> cause the tracking system <NUM> to track locations of the interventional medical device <NUM> during the interventional medical procedure. The process by which the interventional medical device <NUM> is tracked may include identifying a shape of the optical shape sensing device <NUM> using optical shape sensing signals received from the optical shape sensing device <NUM> via the interface <NUM>. The process in this set of embodiments may also include identifying the interventional medical device <NUM> in a first coordinate space of the X-Ray imaging system <NUM>, based on identifying the shape of the optical shape sensing device <NUM>. The interventional medical device <NUM> is then registered to the first coordinate space of the X-Ray imaging system <NUM>. The process may also include identifying the interventional medical device <NUM> in a second coordinate space of the ultrasound imaging system <NUM>. The first coordinate space of the X-Ray imaging system <NUM> is registered to the second coordinate space of the ultrasound imaging system <NUM>. The process of tracking location of the interventional medical device <NUM> in this set of embodiments may also include segmenting the interventional medical device <NUM> in the second coordinate space of the ultrasound imaging system <NUM> to obtain a segmented representation of the interventional medical device <NUM> in the second coordinate space. The interventional medical device <NUM> is then registered to the second coordinate space of the ultrasound imaging system <NUM> using the segmented representation of the interventional medical device. Afterwards, the interventional medical device <NUM> is re-registered to the first coordinate space of the X-Ray imaging system <NUM> based on registering the interventional medical device <NUM> to the second coordinate space using the segmented representation. The re-registration of the interventional medical device <NUM> to the first coordinate space of the X-Ray imaging system <NUM> is performed without requiring additional X-Ray imaging of the patient. The process performed in this set of operations may be performed on-demand, periodically, or once movement of the interventional medical device <NUM> beyond a threshold is detected.

The controller <NUM> may perform some of the operations described herein directly and may implement other operations described herein indirectly. For example, the controller <NUM> may directly control displays of the monitor <NUM>, and indirectly control imaging by the X-Ray imaging system <NUM> and/or imaging by the ultrasound imaging system <NUM>. Accordingly, the process implemented by the tracking system <NUM> when the processor <NUM> executes instructions from the memory <NUM> may include steps not directly performed by the controller <NUM>.

In another set of embodiments using the tracking system <NUM>, registration may be performed using a predetermined shape of the interventional medical device <NUM>. For example, a predetermined shape of the interventional medical device <NUM> may be stored as a template in the memory <NUM> and retrieved from the memory <NUM> for use in searching the ultrasound space for the interventional medical device <NUM>. A shape of the interventional medical device <NUM> may also be dynamically obtained from the optical shape sensing device <NUM>. In this set of embodiments, the ultrasound imaging system <NUM> may be a first imaging system and the X-Ray imaging system <NUM> may be a second imaging system. The interventional medical device <NUM> may be registered to the ultrasound space (first coordinate space) based on the shape of the interventional medical device <NUM> identified using the optical shape sensing signals and based on the shape of the interventional medical device <NUM> identified in the ultrasound space (first coordinate space). A process for tracking the interventional medical device <NUM> may include identifying a shape of the optical shape sensing device <NUM> using optical shape sensing signals received from the optical shape sensing device <NUM> via the interface <NUM>, and identifying a shape of the interventional medical device <NUM> in a first coordinate space of the ultrasound system (first imaging system) that images the interventional medical device <NUM> in a first imaging mode during the interventional medical procedure. The interventional medical device <NUM> is registered to the ultrasound space (first coordinate space) based on the shape of the interventional medical device <NUM> identified using the optical shape sensing signals and based on the shape of the interventional medical device <NUM> identified in the ultrasound space (first coordinate space). In this set of embodiments, preliminary registration of the X-Ray imaging system to the interventional medical device <NUM> and the ultrasound imaging system <NUM> is not required in order to register the interventional medical device <NUM> to the ultrasound imaging system <NUM> using the known shape of the interventional medical device <NUM>.

Before proceeding to the description of <FIG>, the concepts of registration and segmentation will be more fully explained next. Registration involves aligning disparate three-dimensional coordinate systems. In <FIG>, the X-Ray imaging system <NUM>, the ultrasound imaging system <NUM> and the optical shape sensing device <NUM> may each have its own three-dimensional coordinate system. A common three-dimensional coordinate system is provided by aligning the disparate three-dimensional coordinate systems such as by sharing a common origin and set of axes. Registration may include first adjusting the origin of one coordinate system to the origin of another coordinate system, and then aligning axes of the one coordinate system to the axes of the other coordinate system. Registration typically involves calculating and applying transformation matrices based on observations of common three-dimensional elements in the two coordinate systems.

Segmentation produces a representation of the surface of structures such as anatomical features and the interventional medical device <NUM>. The segmented representation consists for example of a set of points in three-dimensional (<NUM>-D) coordinates on the surfaces of the structure, and triangular plane segments defined by connecting neighboring groups of three points, such that the entire structure is covered by a mesh of non-intersecting triangular planes. A three-dimensional model of the interventional medical device <NUM> is obtained by segmenting. Segmenting may also involve performing segmentation on anatomy structures, and/or other structures present in a three-dimensional ultrasound volume.

<FIG> illustrates registration of an interventional medical device to an X-Ray imaging system in interventional medical device tracking, in accordance with a representative embodiment.

In <FIG> an optical fiber is integrated into an interventional medical device <NUM>. An example of the optical fiber in <FIG> is the optical shape sensing device <NUM> in <FIG> The optical fiber provides the position and orientation of the interventional medical device <NUM>. An example of the interventional medical device <NUM> in <FIG> is a catheter with a guidewire. In <FIG>, the optical fiber may be integrated into the interventional medical device <NUM> (e.g., into a guidewire of a catheter) in a vascular branch on the right side. The interventional medical device <NUM> with the integrated optical fiber is overlaid on an X-ray (fluoroscopy) image of a vascular phantom produced by the X-Ray imaging system <NUM>.

In <FIG>, a shape of the optical fiber may be identified using optical shape sensing signals received from the optical fiber via an interface such as the interface <NUM>. The X-Ray imaging system <NUM> may be a first imaging system that produces the X-ray image in a first coordinate space particular to the X-Ray imaging system <NUM>. The interventional medical device <NUM> is registered to the X-Ray imaging system <NUM> by assigning locations of the interventional medical device <NUM> from the optical shape sensing signals to coordinates of the interventional medical device <NUM> in the first coordinate system based on the X-ray image. The interventional medical device <NUM> may be registered to the X-Ray coordinate space using two X-Ray projection images offset by <NUM> degrees or more. An operator identifies the tip of the interventional medical device <NUM> in each X-Ray image and the visible portion of the interventional medical device <NUM> is automatically detected. The transform from the interventional medical device <NUM> and the X-Ray coordinate space is determined from the two X-Ray projections and the reconstruction of the interventional medical device <NUM> based on the optical shape sensing signals from the optical fiber. The interventional medical device <NUM> is identified in the first coordinate space of the X-Ray imaging system <NUM> during the interventional medical procedure, based on identifying the shape of the optical shape sensing device <NUM> using the optical shape sensing signals from the optical fiber.

<FIG> illustrates registration of an ultrasound system to an X-Ray imaging system <NUM> in interventional medical device tracking, in accordance with a representative embodiment.

In <FIG>, registration between the ultrasound imaging system <NUM> and the X-Ray imaging system <NUM> is accomplished through image fusion platforms. An example of an image fusion platform is EchoNavigator. A registration algorithm provided by EchoNavigator is based on taking a fluoroscopic image from the X-Ray imaging system <NUM>. The fluoroscopic image contains the probe head of the ultrasound imaging system <NUM>. As an example, the ultrasound imaging system <NUM> may be a transesophageal echocardiography (TEE) ultrasound system. From the pose of the probe head in the X-Ray image, the transform relating ultrasound space to X-Ray space (TUX) can be calculated.

<FIG> illustrates registration of an interventional medical device to an ultrasound system in interventional medical device tracking, in accordance with a representative embodiment.

As shown in <FIG>, multiple registrations between different coordinate systems can be integrated so that three or more coordinate systems are aligned. In <FIG>, the optical shape sensing coordinate system (OSS Space) can be registered to the X-Ray imaging system coordinate system (X-Ray Space) using a program executed by the controller <NUM>. The X-Ray imaging system coordinate system (X-Ray Space) can be registered to the local environment (Patient Space) which includes the X-Ray imaging system <NUM> using a program such as EchoNavigator executed by the controller <NUM>. Separately, the ultrasound imaging system coordinate system (US Space) can be registered to the X-Ray imaging system coordinate system (X-Ray Space) using a program such as EchoNavigator executed by the controller <NUM>.

Once the interventional medical device <NUM> and the ultrasound imaging system <NUM> have both been registered to the X-Ray space, the interventional medical device <NUM> can be rendered in the ultrasound imaging system coordinate system (US Space) via transforms outlined in <FIG>.

<FIG> illustrates registrations of an interventional medical device to an ultrasound system and to a X-Ray system in interventional medical device tracking, in accordance with another representative embodiment.

In <FIG>, a guidewire <NUM> is shown in the ultrasound space on the left and in the X-Ray space on the right. Since the interventional medical device <NUM> is registered to the ultrasound space and to the X-Ray space, and the ultrasound space is registered to the X-Ray space, the images shown in <FIG> reflect the same coordinate system even though the perspective differs between the two images.

In <FIG>, reconstruction of the interventional medical device <NUM> from the optical shape sensing using the optical shape sensing device <NUM> is shown as the OSS reconstruction <NUM>. The image-based segmentation of the optical shape sensing device <NUM> in the ultrasound space is shown as the segmented representation <NUM>. As shown, in the registration process described herein, the OSS reconstruction <NUM> can be registered to the segmented representation <NUM> in the ultrasound space. Transformation TOU relates the current location of the OSS reconstruction <NUM> to the segmented representation <NUM> of the interventional medical device <NUM> location in ultrasound space.

In a first set of embodiments described herein, registration may be accomplished by identifying a location of an interventional medical device <NUM> in medical imagery such as by user-designation, tracking of a sensor integrated to a tip of the interventional medical device <NUM>, or otherwise. Three-dimensional segmentation of the interventional medical device <NUM> in ultrasound imagery may be used to update the registration in the first set of embodiments. Three-dimensional segmentation of tube-like interventional medical devices in ultrasound may be achieved through image processing techniques combined with sensor tracking technology. Examples of three-dimensional segmentation of interventional medical devices are explained in <CIT>. Examples of tube-like interventional medical devices that are readily subject to three-dimensional segmentation include guidewires and catheters. Alternative mechanisms for identifying the interventional medical devices in the ultrasound images include initializations by users clicking on the locations of the tips of the interventional medical devices in the ultrasound images, as well as by deep learning using artificial intelligence based on previous instantiations of identifications of interventional medical devices in ultrasound images.

The updated registrations address errors known to occur when optical shape sensing is registered to X-Ray space and/or ultrasound space. That is, while optical shape sensing provides highly accurate reconstructions of the local shape of an interventional medical device <NUM>, optical shape sensing may be prone to errors in registration offset due to the accumulation of error along the length of the interventional medical device <NUM>. For example, while the accuracy may be very good immediately after registration of the interventional medical device <NUM> to the X-Ray space is completed, if the proximal end of the interventional medical device <NUM> is moved several centimeters, the registration may accumulate a noticeable error. Using the teachings of the first set of embodiments provided herein, the error can be corrected by re-registration without requiring additional exposures to X-Ray projections, and thus without increasing the X-Ray dose exposure to the patient and clinicians. Moreover, registration of the interventional medical device <NUM> to the X-Ray imaging system <NUM> can be updated continually, on-demand of the clinician (e.g., when the clinician notices an error) or automatically (e.g., when the tracking system <NUM> detects that misalignment exceeds a predetermined threshold). By leveraging segmentation of the interventional medical device <NUM> in the ultrasound space, and registrations as described herein, the registration of the interventional medical device <NUM> can be continually and accurately updated throughout the duration of the procedure.

The interventional medical device <NUM>, which has been roughly registered to X-Ray space and three-dimensional ultrasound space, can maintain an automatically fine-tuned registration based on the segmented shape of the interventional medical device <NUM> in three-dimensional ultrasound. The shape of the interventional medical device <NUM> in three-dimensional ultrasound is determined via image processing or deep learning techniques. A rigid point-to-point transform is then calculated from the corresponding portion of the interventional medical device <NUM> to the three-dimensional segmentation of the interventional medical device <NUM> in the ultrasound coordinate system. Accurate registration of the interventional medical device <NUM> can be maintained throughout the procedure by automatically segmenting the interventional medical device <NUM> in the image and aligning the reconstruction from the optical shape sensing device <NUM>.

In a second set of embodiments, registration of an interventional medical device <NUM> may be achieved and updated using a known shape of the interventional medical device <NUM>, and this may involve a simplified workflow compared to the first set of embodiments. For example, when a shape of the interventional medical device <NUM> is known, such as from a template and/or from the optical shape sensing device <NUM>, the shape can be identified in the ultrasound coordinate space, and the registration between optical shape sensing and the ultrasound space can be achieved without first registering the interventional medical device <NUM> to the X-Ray imaging system <NUM>. A template of the shape of the interventional medical device <NUM> may be obtained from a library of templates stored in a memory such as the memory <NUM>. The template may include a template of a portion of the shape of the interventional medical device <NUM>, such as a template of the shape of a distal tip of the interventional medical device <NUM>. When the template is a template of a portion of the shape, the remainder of the shape of the interventional medical device <NUM> may be identified based on image analysis software searching for the remainder of the shape of the interventional medical device <NUM> in areas proximate to the portion of the shape identified in ultrasound imagery from the template of the portion of the shape.

Additionally, in the second set of embodiments, registration between the ultrasound coordinate space and the X-Ray coordinate space may be performed without requiring an X-Ray image of the probe head of the ultrasound imaging system <NUM>. For example, the common shape of the interventional medical device <NUM> in both coordinate systems may be used as the mechanism to register the two coordinate systems. When the interventional medical device <NUM> is already registered to the X-Ray space, and the interventional medical device <NUM> may be registered to the ultrasound space using a template of the shape of the interventional medical device <NUM>, then the ultrasound coordinate system may be registered to the X-Ray coordinate system by calculating the transformation from the segmentation of the interventional medical device <NUM> in the ultrasound space to the corresponding interventional medical device <NUM> in the X-Ray space.

In the second set of embodiments, the portion of the shape may be used as a constraint in an initial search for the shape of the interventional medical device <NUM> in the ultrasound imagery. Artificial intelligence may be applied to analyze the ultrasound imagery in the ultrasound coordinate space. The search may be initially constrained by the tip of the interventional medical device <NUM>, and once the tip of the interventional medical device <NUM> is identified in the search, artificial intelligence may be applied to find the remainder of the shape of the interventional medical device <NUM> based on characteristics and parameters identified from previous instantiations of the interventional medical device <NUM> in previous searches of ultrasound imagery.

In both the first set of embodiments and the second set of embodiments, metrics may be generated to show a correlation between identifications of the interventional medical device <NUM> in different coordinate spaces. For example, a metric may be generated based on a correlation between an existing location of the segmented representation of the interventional medical device <NUM> in the ultrasound space and a newly-identified location of the interventional medical device <NUM> in the ultrasound coordinates. The correlation may be an estimate of confidence as to the accuracy of the identification, and may be based on, for example, a quantity of discrepancies between the segmented representation and the proposed newly-identified locations of the interventional medical device <NUM> in the ultrasound coordinates.

<FIG> illustrates a method for interventional medical device tracking, in accordance with a representative embodiment.

In <FIG>, the method starts at S510 by identifying a shape of an optical shape sensing device. The optical shape sensing device may be the optical shape sensing device <NUM> in the embodiment of <FIG>, and may comprise an optical fiber. The optical shape sensing device may be identified using optical shape sensing technology described above.

At S520, the method of <FIG> includes identifying an interventional medical device in a first coordinate space. The first coordinate space may be the coordinate space of a first imaging system operating in a first imaging mode, such as of the X-Ray imaging system <NUM> operating in an X-Ray imaging mode. The identification at S520 may be by a user designating the interventional medical device <NUM> in an X-Ray image, or by image analysis software identifying the interventional medical device <NUM> in an X-Ray image.

At S530, the method of <FIG> includes registering the interventional medical device <NUM> to the first coordinate space. The registration at S530 may be of the interventional medical device <NUM> to the X-Ray space of the X-Ray imaging system <NUM>. The registration at S530 may be based on the shape of the interventional medical device <NUM> identified at S510 from the conforming shape of the optical shape sensing device <NUM>. The registration at S520 may also be based on the shape of the interventional medical device <NUM> identified at S520 from the X-Ray image.

At S540, the method of <FIG> includes identifying the interventional medical device <NUM> in a second coordinate space. The second coordinate space may be the coordinate space of a second imaging system operating in a second imaging mode, such as of the ultrasound imaging system <NUM> operating in an ultrasound imaging mode. The identification at S540 may be by a user designating the interventional medical device <NUM> in an ultrasound image, or by image analysis software identifying the interventional medical device <NUM> in an ultrasound image. Although not shown in <FIG>, S540 may be performed between S550 and S560 (described below), so as to register the first coordinate space to the second coordinate space before fine-tuning the registration of the interventional medical device <NUM> in the second coordinate space (i.e., in the ultrasound space).

In an embodiment, a user may identify a tip of the interventional medical device <NUM> in an ultrasound image, and image analysis software may constrain a search for the remainder of the interventional medical device <NUM> to the area around the designated tip.

In another embodiment, a sensor on the tip of the interventional medical device <NUM> may be a passive ultrasound sensor that responds to emissions from the ultrasound imaging system <NUM>. Sensor-based tracking of an interventional medical device <NUM> is described in <CIT>.

In the embodiment using the sensor on the tip of the interventional medical device <NUM>, image analysis software may constrain a search for the remainder of the interventional medical device <NUM> to the area around the tip identified from a signal from the sensor. The constraint may be based on identification of the tip of the interventional medical device <NUM> by a user, identification based on a signal from a passive ultrasound sensor, or identification from image analysis software that is trained by artificial intelligence to recognize the tip of the interventional medical device.

In another embodiment, the interventional medical device <NUM> may be roughly registered to the ultrasound space via the TOX transform from the interventional medical device <NUM> to the X-Ray space and via the TUX transform from the ultrasound space to the X-Ray space. The ultrasound image-based device segmentation described below at S560 is then continually calculated throughout the acquisition, using the tip of the interventional medical device <NUM> as a rough estimate to constrain the search space of the image processing algorithm. The transform TOU from the interventional medical device <NUM> to the ultrasound space may be calculated on each ultrasound frame, and the registration of the interventional medical device <NUM> to the ultrasound space is updated continuously or at fixed intervals throughout the procedure.

At S550, the first coordinate space is registered to the second coordinate space. Registration at S550 may be performed by imaging a head of an ultrasound probe in the ultrasound imaging system <NUM> using the X-Ray imaging system <NUM>. As noted above, in some embodiments, S550 may be performed before S540.

At S560, the interventional medical device in the second coordinate space is segmented to produce a segmented representation of the interventional medical device. In an embodiment, the segmentation at S560 is initialized by a user identifying the interventional medical device <NUM> in an ultrasound image at S540. An image-processing algorithm searches for the interventional medical device <NUM> in the image in the region identified by the user. A rigid transform TOU from the interventional medical device <NUM> to the ultrasound space is calculated such that the distal portion of the interventional medical device <NUM> corresponding to the length of the ultrasound device segmentation is rotated/translated to most closely match the segmented representation in ultrasound. The rigid transform TOU is then applied to the entire length of the reconstruction from the optical shape sensing device <NUM>.

At S565, the segmented representation of the interventional medical device <NUM> is rendered, such as on the monitor <NUM> in <FIG>. The segmented representation of the interventional medical device <NUM> may be overlaid on an ultrasound image that, in turn, is overlaid on an X-Ray image.

At S570, the interventional medical device is registered to the second coordinate space of the ultrasound imaging system <NUM>. The registration at S570 may be the initial registration of the interventional medical device <NUM> to the ultrasound space, or may be a repeated registration of the interventional medical device <NUM> to correct an earlier registration that has become outdated.

At S580, the interventional medical device <NUM> is re-registered to the first coordinate space of the X-Ray imaging system <NUM>. The re-registration at S580 may correct an outdated earlier registration, and does not require additional imaging by the X-Ray imaging system <NUM>.

In an embodiment, the interventional medical device <NUM> is segmented in the ultrasound volume based on artificial intelligence from previous identifications of the interventional medical device <NUM> in ultrasound images. In this embodiment, initialization from a user identification or from a sensor is not necessarily required to constrain the search space. The transform TOU from the interventional medical device <NUM> to the ultrasound space may be calculated on each ultrasound frame and the registration from the interventional medical device <NUM> to the ultrasound space is continuously updated throughout the procedure.

Although not shown in <FIG>, the first coordinate space may be re-registered to the second coordinate space by calculating the transformation from the segmented representation of the interventional medical device <NUM> in the first coordinate space to a shape of the interventional medical device <NUM> identified based on optical shape sensing in the second coordinate system. According to the second set of embodiments described herein, the re-registration may be performed without requiring an X-Ray image of the probe head of the ultrasound imaging system <NUM>. When the interventional medical device <NUM> is already registered to the X-Ray space and the interventional medical device <NUM> is already registered to any ultrasound space based on the template of the shape, the ultrasound space may be registered to the X-Ray space by calculating the transformation from the segmented representation of the interventional medical device <NUM> in the ultrasound space to the corresponding section of the interventional medical device <NUM> in the X-Ray space.

In the method of <FIG>, a selection of an interventional medical device <NUM> is detected in the second coordinate space of the ultrasound imaging system <NUM>. The selection may be a user selection of a tip of the interventional medical device <NUM>.

At S640, the interventional medical device <NUM> is identified in the second coordinate space. The identification at S640 is based on the selection at S635, and may involve using image analysis software to search the area around where the user selects for the remainder of the interventional medical device <NUM>.

At S645, the transform TOU from the existing location of the interventional medical device <NUM> to the segmented representation is calculated. Using the new TOU transform, the previous location of the interventional medical device <NUM> in the ultrasound space can be updated to the new location of the segmented representation.

At S648, the second coordinate space of the ultrasound imaging system <NUM> is registered to the first coordinate space of the X-Ray imaging system based on the segmented representation of the interventional medical device <NUM> in the second coordinate space of the ultrasound imaging system <NUM>. By this registration, the previous location of the interventional medical device <NUM> in the X-Ray space is updated to account for any error due, for example, to movement of the interventional medical device <NUM>.

The embodiment of <FIG> may be supplemental to the embodiment of <FIG>, and includes functions that supplement the functions described with respect to <FIG>.

In the description of the embodiment of <FIG> above, identification of the interventional medical device <NUM> in the ultrasound space is based on user designation of a tip of the interventional medical device <NUM>. In another embodiment, a known shape of the interventional medical device <NUM> can be searched in the ultrasound space without requiring knowledge of the shape of the distal tip of the interventional medical device <NUM>. A search for the known shape of the interventional medical device <NUM> may be based on information from optical shape sensing signals from the optical shape sensing device <NUM>, and eliminates a need for a rough registration of the interventional medical device <NUM> to the ultrasound space via the transform TOU.

In <FIG>, the interventional medical device <NUM> is identified in the second coordinate space of the ultrasound imaging system <NUM> at S736. The identification at S736 may be based on the user selecting a location of the tip of the interventional medical device <NUM> or by a signal from a passive ultrasound sensor on the tip of the interventional medical device <NUM>.

At S760, the interventional medical device <NUM> is segmented in the ultrasound image to produce a segmented representation of the interventional medical device <NUM>.

At S770, the interventional medical device <NUM> is registered to the second coordinate space of the ultrasound imaging system <NUM>. The registration at S770 may be based on at least the transform TOU.

At S780, the interventional medical device is registered to the first coordinate space of the X-Ray imaging system <NUM>. The transform at S780 may be based on all three of the transforms TOX, TUX and TOU.

In the description of the embodiment of <FIG> above, identification of the interventional medical device <NUM> in the ultrasound space is based on user or sensor identification of a tip of the interventional medical device <NUM>. In another embodiment, a known shape of the interventional medical device <NUM> can be retrieved from a template or from the information from optical shape sensing signals from the optical shape sensing device <NUM>. Registration based on a template or knowledge of the shape of the interventional medical device <NUM> can be used to define or maintain the registration between the ultrasound space and the X-Ray space, particularly if the ultrasound probe of the ultrasound imaging system <NUM> is at an angle that is difficult to detect in the X-Ray imaging system <NUM>. Assuming the transform TOU from the interventional medical device <NUM> to the ultrasound space is known and the transform TOX from the interventional medical device <NUM> to the X-Ray space is known, the transform TUX from ultrasound space to the X-Ray space can be determined, loosely, as (TUX = inv(TOU) * TOX). As in other embodiments described herein, an embodiment which maintains a registration by repeatedly updating the transform TUX may not require additional exposures to X-Rays.

In <FIG>, the method starts at S836 by identifying the interventional medical device <NUM> in the second coordinate space. The identification may be made based on a user designation or based on a signal from a sensor on the tip of the interventional medical device <NUM>. The identification at S836 may also be made on-demand, periodically on a continuous basis during the interventional medical procedure, or based on detection of movement of the interventional medical device <NUM> compared to a previous registration.

At S837, a determination is made whether a position of the interventional medical device deviates from an existing segmented representation by more than a threshold. The determination at S837 may be based on detection of movement of the interventional medical device <NUM> compared to a previous registration. If any deviation does not exceed a threshold (S837 = No), the method returns to S836, and otherwise proceeds to S860.

At S860, the interventional medical device <NUM> is segmented in the ultrasound space.

At S870, the interventional medical device <NUM> is registered to the second coordinate space based on the segmentation at S860.

At S880, the second coordinate system is re-registered to the first coordinate system. The re-registration at S880 is provided without requiring another exposure to X-Rays from the X-Ray imaging system <NUM>.

After S880, the process returns to S836. Accordingly, the process of <FIG> is recursive, and may involve repeatedly checking whether a position of the interventional medical device <NUM> deviates from the last existing segmented representation by more than a threshold and correcting the deviation by updating registrations of the interventional medical device <NUM> to the X-Ray space and the ultrasound space.

In the description of the embodiment of <FIG> above, identification of the interventional medical device <NUM> in the ultrasound space is again based on knowledge of the tip of the interventional medical device <NUM>. In another embodiment, irregularly-shaped therapeutic devices deployed with optical shape sensing may be delivered via a an OSS-enabled delivery sheath or catheter. A therapeutic device of known irregular shape may be detected in the ultrasound space via image-based segmentation or via artificial analysis applied to previous instantiations of identifications of the OSS delivery sheath or catheter. Segmentation of the irregularly shaped therapeutic device may be used to locate the distal end of the OSS-enabled delivery device in ultrasound for repeated updating of the registration between the interventional medical device <NUM> and the ultrasound space.

In additional embodiments, registration methods described above may be triggered automatically. For example, an interventional medical device <NUM> may be continually segmented in ultrasound space in background processing. The existing registration of the interventional medical device <NUM> may be updated to most closely match the interventional medical device <NUM> in ultrasound space on every frame, on every nth frame, or only when a metric describing the correlation and/or offset between the location of the interventional medical device <NUM> and the location of the segmented representation of the interventional medical device <NUM> in ultrasound space exceeds a predetermined threshold.

Additionally, registration methods described above may be triggered on-demand. A user interface may include a metric describing the correlation and/or offset between the location of the interventional medical device <NUM> and the location of the interventional medical device <NUM> in ultrasound space. The user may then select an "update registration" soft button when the offset metric exceeds the desired error limit, or anytime the user wishes to update the current registration based on visual inspection.

Furthermore, a user interface may provide a metric describing success of registration after the registration has been performed. The metric may contain information about the correlation between the shape of the interventional medical device <NUM> in the re-registration and the shape of the interventional medical device <NUM> in the segmented representation in the ultrasound space. Alternatively, the metric may include a confidence level that the correct shape of the interventional medical device <NUM> has been detected.

<FIG> illustrates a computer system, on which a method for interventional medical device tracking is implemented, in accordance with some representative embodiments.

The computer system <NUM> of <FIG> shows a complete set of components for a communications device or a computer device. However, a "controller" as described herein may be implemented with less than the set of components of <FIG>, such as by a memory and processor combination. The computer system <NUM> may include some or all elements of one or more component apparatuses in a system for interventional medical device tracking herein, although any such apparatus may not necessarily include one or more of the elements described for the computer system <NUM> and may include other elements not described.

Referring to <FIG>, the computer system <NUM> includes a set of software instructions that can be executed to cause the computer system <NUM> to perform any of the methods or computer-based functions disclosed herein. The computer system <NUM> may operate as a standalone device or may be connected, for example, using a network <NUM>, to other computer systems or peripheral devices. In embodiments, a computer system <NUM> performs logical processing based on digital signals received via an analog-to-digital converter.

In a networked deployment, the computer system <NUM> operates in the capacity of a server or as a client user computer in a server-client user network environment, or as a peer computer system in a peer-to-peer (or distributed) network environment. The computer system <NUM> can also be implemented as or incorporated into various devices, such as the controller <NUM> in FIG. 1A, a stationary computer, a mobile computer, a personal computer (PC), a laptop computer, a tablet computer, or any other machine capable of executing a set of software instructions (sequential or otherwise) that specify actions to be taken by that machine. The computer system <NUM> can be incorporated as or in a device that in turn is in an integrated system that includes additional devices. In an embodiment, the computer system <NUM> can be implemented using electronic devices that provide voice, video or data communication. Further, while the computer system <NUM> is illustrated in the singular, the term "system" shall also be taken to include any collection of systems or sub-systems that individually or jointly execute a set, or multiple sets, of software instructions to perform one or more computer functions.

As illustrated in <FIG>, the computer system <NUM> includes a processor <NUM>. The processor <NUM> may be considered a representative example of the processor <NUM> of the controller <NUM> in <FIG> and executes instructions to implement some or all aspects of methods and processes described herein. The processor <NUM> is tangible and non-transitory. As used herein, the term "non-transitory" is to be interpreted not as an eternal characteristic of a state, but as a characteristic of a state that will last for a period. The term "non-transitory" specifically disavows fleeting characteristics such as characteristics of a carrier wave or signal or other forms that exist only transitorily in any place at any time. The processor <NUM> is an article of manufacture and/or a machine component. The processor <NUM> is configured to execute software instructions to perform functions as described in the various embodiments herein. The processor <NUM> may be a general-purpose processor or may be part of an application specific integrated circuit (ASIC). The processor <NUM> may also be a microprocessor, a microcomputer, a processor chip, a controller, a microcontroller, a digital signal processor (DSP), a state machine, or a programmable logic device. The processor <NUM> may also be a logical circuit, including a programmable gate array (PGA), such as a field programmable gate array (FPGA), or another type of circuit that includes discrete gate and/or transistor logic. The processor <NUM> may be a central processing unit (CPU), a graphics processing unit (GPU), or both. Additionally, any processor described herein may include multiple processors, parallel processors, or both. Multiple processors may be included in, or coupled to, a single device or multiple devices.

The term "processor" as used herein encompasses an electronic component able to execute a program or machine executable instruction. References to a computing device comprising "a processor" should be interpreted to include more than one processor or processing core, as in a multi-core processor. A processor may also refer to a collection of processors within a single computer system or distributed among multiple computer systems. The term computing device should also be interpreted to include a collection or network of computing devices each including a processor or processors. Programs have software instructions performed by one or multiple processors that may be within the same computing device or which may be distributed across multiple computing devices.

The computer system <NUM> further includes a main memory <NUM> and a static memory <NUM>, where memories in the computer system <NUM> communicate with each other and the processor <NUM> via a bus <NUM>. Either or both of the main memory <NUM> and the static memory <NUM> may be considered representative examples of the memory <NUM> of the controller <NUM> in FIG. 1B, and store instructions used to implement some or all aspects of methods and processes described herein. Memories described herein are tangible storage mediums for storing data and executable software instructions and are non-transitory during the time software instructions are stored therein. As used herein, the term "non-transitory" is to be interpreted not as an eternal characteristic of a state, but as a characteristic of a state that will last for a period. The term "non-transitory" specifically disavows fleeting characteristics such as characteristics of a carrier wave or signal or other forms that exist only transitorily in any place at any time. The main memory <NUM> and the static memory <NUM> are articles of manufacture and/or machine components. The main memory <NUM> and the static memory <NUM> are computer-readable mediums from which data and executable software instructions can be read by a computer (e.g., the processor <NUM>). Each of the main memory <NUM> and the static memory <NUM> may be implemented as one or more of random access memory (RAM), read only memory (ROM), flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable disk, tape, compact disk read only memory (CD-ROM), digital versatile disk (DVD), floppy disk, blu-ray disk, or any other form of storage medium known in the art. The memories may be volatile or non-volatile, secure and/or encrypted, unsecure and/or unencrypted.

"Memory" is an example of a computer-readable storage medium. Computer memory is any memory which is directly accessible to a processor. Examples of computer memory include, but are not limited to RAM memory, registers, and register files. References to "computer memory" or "memory" should be interpreted as possibly being multiple memories. The memory may for instance be multiple memories within the same computer system. The memory may also be multiple memories distributed amongst multiple computer systems or computing devices.

As shown, the computer system <NUM> further includes a video display unit <NUM>, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid-state display, or a cathode ray tube (CRT), for example. Additionally, the computer system <NUM> includes an input device <NUM>, such as a keyboard/virtual keyboard or touch-sensitive input screen or speech input with speech recognition, and a cursor control device <NUM>, such as a mouse or touch-sensitive input screen or pad. The computer system <NUM> also optionally includes a disk drive unit <NUM>, a signal generation device <NUM>, such as a speaker or remote control, and/or a network interface device <NUM>.

In an embodiment, as depicted in <FIG>, the disk drive unit <NUM> includes a computer-readable medium <NUM> in which one or more sets of software instructions <NUM> (software) are embedded. The sets of software instructions <NUM> are read from the computer-readable medium <NUM> to be executed by the processor <NUM>. Further, the software instructions <NUM>, when executed by the processor <NUM>, perform one or more steps of the methods and processes as described herein. In an embodiment, the software instructions <NUM> reside all or in part within the main memory <NUM>, the static memory <NUM> and/or the processor <NUM> during execution by the computer system <NUM>. Further, the computer-readable medium <NUM> may include software instructions <NUM> or receive and execute software instructions <NUM> responsive to a propagated signal, so that a device connected to a network <NUM> communicates voice, video or data over the network <NUM>. The software instructions <NUM> may be transmitted or received over the network <NUM> via the network interface device <NUM>.

In an embodiment, dedicated hardware implementations, such as application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic arrays and other hardware components, are constructed to implement one or more of the methods described herein. One or more embodiments described herein may implement functions using two or more specific interconnected hardware modules or devices with related control and data signals that can be communicated between and through the modules. Accordingly, the present disclosure encompasses software, firmware, and hardware implementations. Nothing in the present application should be interpreted as being implemented or implementable solely with software and not hardware such as a tangible non-transitory processor and/or memory.

In accordance with various embodiments of the present disclosure, the methods described herein may be implemented using a hardware computer system that executes software programs. Virtual computer system processing may implement one or more of the methods or functionalities as described herein, and a processor described herein may be used to support a virtual processing environment.

Accordingly, interventional medical device tracking enables updated registrations to correct locations of an interventional medical device <NUM>. Nevertheless, interventional medical device tracking is not limited as an application to specific details described herein, and instead is applicable to additional embodiments in which other types of medical imaging systems and interventional medical devices are used.

Although interventional medical device tracking has been described with reference to several exemplary embodiments, it is understood that the words that have been used are words of description and illustration, rather than words of limitation. Although interventional medical device tracking has been described with reference to particular means, materials and embodiments, interventional medical device tracking is not intended to be limited to the particulars disclosed; rather interventional medical device tracking extends to all functionally equivalent structures, methods, and uses such as are within the scope of the appended claims.

The illustrations are not intended to serve as a complete description of all of the elements and features of the disclosure described herein.

Moreover, although specific embodiments have been illustrated and described herein, it should be appreciated that any subsequent arrangement designed to achieve the same or similar purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all subsequent adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent to those of skill in the art upon reviewing the description.

The Abstract of the Disclosure is provided to comply with <NUM> C. §<NUM>(b) and is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, various features may be grouped together or described in a single embodiment for the purpose of streamlining the disclosure. This disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter may be directed to less than all of the features of any of the disclosed embodiments. Thus, the following claims are incorporated into the Detailed Description, with each claim standing on its own as defining separately claimed subject matter.

Claim 1:
A system for tracking location of an interventional medical device (<NUM>, <NUM>) in an interventional medical procedure, comprising:
an interface (<NUM>) to an optical shape sensing device (<NUM>) which has a shape that conforms to a shape of the interventional medical device (<NUM>, <NUM>) during the interventional medical procedure; and
a controller (<NUM>) comprising a memory (<NUM>) that stores instructions and a processor (<NUM>) that executes the instructions, wherein, when executed by the processor (<NUM>), the instructions cause the system to:
identify a shape of the optical shape sensing device (<NUM>) using optical shape sensing signals received from the optical shape sensing device (<NUM>) via the interface (<NUM>);
identify the interventional medical device (<NUM>, <NUM>) in a first coordinate space of a first imaging system that images the interventional medical device (<NUM>, <NUM>) in a first imaging mode during the interventional medical procedure, based on identifying the shape of the optical shape sensing device (<NUM>);
register the interventional medical device (<NUM>, <NUM>) to the first coordinate space;
identify the interventional medical device (<NUM>, <NUM>) in a second coordinate space of a second imaging system that images the interventional medical device (<NUM>, <NUM>) in a second imaging mode during the interventional medical procedure;
register the first coordinate space of the first imaging system to the second coordinate space of the second imaging system;
segment the interventional medical device (<NUM>, <NUM>) in the second coordinate space to obtain a segmented representation (<NUM>) of the interventional medical device (<NUM>, <NUM>) in the second coordinate space;
register the interventional medical device (<NUM>, <NUM>) to the second coordinate space using the segmented representation (<NUM>) of the interventional medical device (<NUM>, <NUM>); and
re-register the interventional medical device (<NUM>, <NUM>) to the first coordinate space based on registering the interventional medical device (<NUM>, <NUM>) to the second coordinate space.