Patent Description:
Aspects of the present disclosure generally relate to medical devices and operations. Particular aspects relate to guidance systems and associated methods.

Elongated devices are commonly used to access remote regions and cavities of the body for diagnostic, surgical, and/or therapeutic purposes. For example, endoscopes may use naturally formed lumens or orifices, and/or surgical incisions to access the colon, esophagus, stomach, urethra, bladder, ureter, kidneys, lungs, bronchi, uterus, and other organs; and catheters may use the circulatory system as pathways to access treatment sites near the heart, or the urinary canal to access urinary regions. These elongated devices may be introduced into the body through a natural body path, such as large artery, including those found in the groin or in the neck. The same device also may be introduced through artificial paths formed in the body, such as a tunnel formed by a tunneling tool.

Navigation through some body paths requires the elongated device to be passed through ever-narrowing diameters until a distal end of the device reaches an operative site inside the body. Locating the distal end at the operative site can be challenging. In some instances, a fluoroscope may be employed to guide the distal end through the body in one plane of movement. Because X-rays are applied externally, in a single imaging plane, they often cannot account for localized aspects of body paths, especially paths extending deep into the body. Additional sensing technologies are needed to guide medical devices through a greater range of body paths, and reduce the risks associated with such navigation. Aspects of the devices, methods, and systems described herein are provided to address these issues.

Document <CIT> discloses method which comprises navigating a patient's anatomy with a medical instrument, the instrument comprising a sensing tool. The method further includes correlating a position of the instrument with a model of the patient's anatomy. The method further includes, while navigating the patient's anatomy, updating the model based on data obtained by the sensing tool.

Document <CIT> describes a method and system for scene parsing and model fusion in laparoscopic and endoscopic 2D/<NUM>. 5D image data. A current frame of an intra-operative image stream including a 2D image channel and a <NUM>. 5D depth channel is received. A 3D pre-operative model of a target organ segmented in pre-operative 3D medical image data is fused to the current frame of the intra-operative image stream. Semantic label information is propagated from the pre-operative 3D medical image data to each of a plurality of pixels in the current frame of the intra-operative image stream based on the fused pre-operative 3D model of the target organ, resulting in a rendered label map for the current frame of the intra-operative image stream. A semantic classifier is trained based on the rendered label map for the current frame of the intra-operative image stream.

Document <CIT> describes a method for performing a percutaneous operation on a patient to remove an object from a cavity within the patient. The method includes advancing a first alignment sensor into the cavity through a patient lumen. The first alignment sensor provides its position and orientation in free space in real time. The alignment sensor is manipulated until it is located in proximity to the object. A percutaneous opening is made in the patient with a surgical tool, where the surgical tool includes a second alignment sensor that provides the position and orientation of the surgical tool in free space in real time. The surgical tool is directed towards the object using data provided by both the first and the second alignment sensors.

Document <CIT> describes systems, devices, and methods for navigating a tool inside a luminal network. An exemplary method includes receiving image data of a patient's chest, identifying the patient's lungs, determining locations of a luminal network in the patient's lungs, identifying a target location in the patient's lungs, generating a pathway to the target location, generating a three-dimensional (3D) model of the patient's lungs, the 3D model showing the luminal network in the patient's lungs and the pathway to the target location, determining a location of a tool based on an electromagnetic (EM) sensor included in the tool as the tool is navigated within the patient's chest, displaying a view of the 3D model showing the determined location of the tool, receiving cone beam computed tomography (CBCT) image data of the patient's chest, updating the 3D model based on the CBCT image data, and displaying a view of the updated 3D model.

The invention is defined in appended claim <NUM>. Aspects, embodiments and examples of the present disclosure which do not fall under the scope of the appended claims are merely provided for illustrative purposes. Surgical methods described herein do not form part of the claimed invention. Aspects of the present disclosure relate to guidance devices, systems, and methods. Numerous aspects of the present disclosure are now described.

One aspect of the present disclosure is a method comprising: receiving macroscan data including first images of a body and first location data associated with each first image; combining the first images according to the first location data to generate a body information model; positioning a navigation component in the body; generating microscan data with the navigation component, the microscan data including second images of the body and second location data associated with each second image; correlating the first location data with the second location data at a target location in the body information model; and/or modifying the body information model by combining the first images with the second images at the target location.

According to this aspect, the navigation component may include an elongated shaft and an internal scanner located on a distal portion of the elongated shaft, and the method may comprise positioning the internal scanner in the body, and generating the microscan data with the internal scanner. The internal scanner may include a probe configured to generate wave energy images of the body and the second images may include the wave energy images, the method further comprising generating the wave energy images with the probe. The probe may include one or more ultrasound transducers configured to generate ultrasound images of the body, and generating the wave energy images may comprise generating ultrasound images with the one or more ultrasound transducers. The probe may include one or more laser sources configured to generate laser images of the body, and generating the wave energy images may comprise generating laser images with the one or more laser sources.

In some aspects, the internal scanner may include a plurality of imaging elements configured to generate graphical images and the second images may include the graphical images, the method further comprising generating the graphical images at one or more frequencies of the electromagnetic spectrum with the plurality of imaging elements. Generating the graphical images may comprise generating the graphical images in a visual light range. The internal scanner includes a plurality of light sources, and the method may further comprise operating the plurality of light sources independent of or together with the plurality of imaging elements. Generating the microscan data may include generating second images and location data according to a predetermined sequence. For example, the second images may be generated at a rate of <NUM> or greater in the predetermined sequence.

In other aspects, the navigation component may include an elongated shaft, and positioning a navigation component in the body may comprise moving a distal portion of the elongated shaft through the body. For example, the distal portion of the elongated shaft may include a tracking sensor responsive to a magnetic field, and generating second location data may comprise moving the sensor in the field.

Another aspect of the present disclosure is a method comprising: receiving macroscan data prior to an operation; generating a body information model from the macroscan data; positioning a navigation component in the body during the operation; generating microscan data with the navigation component; correlating a location in the macroscan data with a location in the microscan data in the body information model; and/or modifying the body information model by combining an image in the microscan data with an image in the macroscan data.

According to this aspect, the method may comprise outputting a portion of the body information model as a navigation image. For example, the method may comprise: identifying a path through the body in the navigation image; identifying a location of the navigation component in the navigation image; and guiding the navigation component along the path in the navigation image. The method may further comprise: locating objects in the body relative to the path; and guiding the navigation component to the located objects. In some aspects, the method includes determining characteristics of the located objects. The method also may include: selecting one of the located objects based on its characteristics; performing a treatment on the selected object; and modifying the body information model to indicate that the selected object has been treated.

Yet another aspect of the present disclosure is a method comprising: receiving macroscan data prior to an operation, the macroscan data including first images of a body and first location data associated with each first image; combining the first images according to the first location data to generate a body information model; positioning a navigation component in the body during the operation; generating microscan data with the navigation component, the microscan data including second images of the body and second location data associated with each second image; correlating the first location data with the second location data at a target location in the body information model; modifying the body information model by combining the first images with the second images at the target location; outputting a portion of the body information model as a navigation image; identifying a path through the body in the navigation image; identifying a location of the navigation component in the navigation image; and/or guiding the navigation component along the path in the navigation image.

It may be understood that both the foregoing summary and the following detailed descriptions are exemplary and explanatory only, neither being restrictive of the inventions claimed below.

The accompanying drawings are incorporated in and constitute a part of this specification. These drawings illustrate aspects of the present disclosure that, together with the written descriptions herein, serve to explain this disclosure. Each drawing depicts one or more aspects of this disclosure as follows:.

Aspects of the present disclosure are now described with reference to exemplary guidance systems and associated methods. Some aspects are described with reference to operations where an elongated device is guided through a body path until a distal end of the elongated body is located in a body cavity (or other target site in a body). For example, an endoscope may include an elongated body that is guided through a natural body path including a urethra, bladder, and/or ureter until a distal end of endoscope is located in an interior cavity of a kidney. References to a particular type of operation or procedure, such as medical; body path, such as natural or artificial; or body cavity, such as the interior cavity of a kidney, are provided for convenience and not intended to limit the present disclosure unless claimed. Accordingly, the concepts described herein may be utilized for any analogous device or method - medical or otherwise, kidney-specific or not.

Numerous axes are described. Each axis may be transverse, or even perpendicular, with the next so as to establish a Cartesian coordinate system with an origin point O (or O'). One axis may extend along a longitudinal axis extending through an origin point defined within an element or body path. The directional terms "proximal" and "distal," and their respective initials "P" and "D," may be used to describe relative components and features in relation to these axes. Proximal refers to a position closer to the exterior of the body or a user, whereas distal refers to a position closer to the interior of the body or further away from the user. Appending the initials "P" or "D" to an element number signifies a proximal or distal location, and appending P or D to an arrow in a figure signifies a proximal or distal direction along one or more axes. The term "elongated" may be used to describe any aspect extending proximally or distally along any axis. Unless claimed, these terms are provided for convenience and not intended to limit the present disclosure to a particular location, direction, or orientation.

As used herein, the terms "comprises," "comprising," or like variation, are intended to cover a non-exclusive inclusion, such that a device or method that comprises a list of elements does not include only those elements, but may include other elements not expressly listed or inherent thereto. Unless stated otherwise, the term "exemplary" is used in the sense of "example" rather than "ideal. " Conversely, the terms "consists of" and "consisting of" are intended to cover an exclusive inclusion, such that a device or method that consists of a list of elements includes only those elements. As used herein, terms such as "about," "substantially," "approximately," or like variations, may indicate a range of values within +/- <NUM>% of a stated value.

Aspects of the present disclosure pertain to an exemplary guidance system <NUM>, examples of which are depicted in <FIG> with reference to an exemplary body <NUM>. Body <NUM> may be a patient's body (e.g., a human body) or a portion thereof. In <FIG>, for example, body <NUM> is depicted as a medial-lateral cross-section of a human torso including: an exterior surface <NUM>; an entry point <NUM> on surface <NUM>; a body cavity <NUM>; and a body path <NUM> extending between entry point <NUM> and cavity <NUM>. A body axis B-B extends through an origin point O in body <NUM> (e.g., <FIG>), and a body path P-P extends through an origin point O' in body path <NUM> (e.g., <FIG>).

System <NUM> may include a mapping component <NUM> configured to perform macroscans of body <NUM> (e.g., <FIG>) and output macroscan data <NUM> (e.g., <FIG>). The system <NUM> includes a navigation component <NUM> configured to perform microscans of body <NUM> (e.g., <FIG>) and output microscan data <NUM> (e.g., <FIG>); and a controller <NUM> (e.g., <FIG>) configured to generate a pre-operative body information model <NUM> (e.g., <FIG>) from macroscan data <NUM>, operatively update information model <NUM> with microscan data <NUM>, and/or output portions of model <NUM> as a navigation image <NUM> (e.g., <FIG>). The terms "macroscan(s)" and "microscan(s)" are utilized herein to describe aspects of system <NUM>. In this application, a macroscan may be obtained from an externally-applied scanning medium (e.g., X-rays from a CT scanner), while a microscan may be obtained from an internally-applied scanning medium (e.g., sound waves from a transducer).

Mapping component <NUM> is configured to perform macroscans of body <NUM>, and output macroscan data <NUM>. An exemplary mapping component <NUM> is depicted in <FIG> as comprising an external scanner <NUM> configured to take one or more macroscans of body <NUM>, and a transmitter <NUM> configured to output macroscan data <NUM>. As shown in <FIG>, macroscan data <NUM> include: imaging data 23ID (e.g., a wave energy image) generated from the macroscan; and location data 23LD (. e.g., the location of the wave energy image along body axis B-B) generated when the macroscan is performed.

Imaging data 23ID may include any number of two- or three-dimensional images generated by external scanner <NUM> in any format, at any resolution, using any scanning medium. For example, scanner <NUM> may be configured to direct a wave energy (e.g., light, sound, X-rays, etc.) toward exterior surface <NUM> of body <NUM>, receive a reflected portion of the wave energy, and generate imaging data 23ID including wave energy images of body <NUM>. In some aspects, scanner <NUM> may be a CT scanner configured to direct and receive X-rays along an imaging axis I-I, and generate imaging data 23ID including a plurality of two-dimensional cross-sectional X-ray images taken at one or more angles and/or positions relative to body <NUM>. A representative cross-sectional X-ray image is depicted in <FIG> as imaging data 23ID generated by a CT scanner. Similar images may be created with other wave energies. In other aspects, external scanner <NUM> may include a magnetic resonance imaging device configured to use magnetic fields, radio waves, and field gradients to generate imaging data 23ID including a plurality of three-dimensional images.

Location data 23LD may be used to locate imaging data 23ID in body <NUM> with respect to natural and/or artificial markers. In <FIG>, an imaging plane I-I extends through origin point O and body axis B-B extending therethrough. Origin point O may be defined at the centroid of a vertebrae of body <NUM> (e.g., <FIG>), allowing body axis B-B to coincide with a longitudinal axis extending through the spine of body <NUM>, and provide a common point of reference in each macroscan. Artificial markers may alternatively be placed in or on body <NUM> to define the location of origin point O and/or body axis B-B. For example, an artificial marker may be placed on exterior surface <NUM>, inside of body cavity <NUM>, and/or on bed <NUM>; and include radiopaque elements shaped to define the location of point O and/or axis B-B in a cross-sectional X-ray image of body <NUM>.

Transceiver <NUM> may comprise any wired or wireless technology configured to output macroscan data <NUM>. For example, transceiver <NUM> may be configured to output data <NUM> and/or otherwise communicate with a transceiver <NUM> of controller <NUM> using any known data communication technology, including Bluetooth®, fiber optic cables, Wi-Fi, and the like. If mapping component <NUM>, navigation component <NUM>, and controller <NUM> are all part of the same device, then transceiver <NUM> may simply be a direct or wired connection between the respective components of such a device.

With reference to <FIG>, navigation component <NUM> is configured to perform microscans of body <NUM>, and output microscan data <NUM>. Numerous aspects of navigation component <NUM> are described. As depicted for example in <FIG>, navigation component <NUM> comprises an elongated shaft <NUM> steerable through body path <NUM> and an internal scanner or sensor <NUM> configured to perform one or more microscans as shaft <NUM> is moved along path <NUM>. Navigation component <NUM> may further comprise: a magnetic field generator <NUM> configured to generate a magnetic field about path <NUM>; a tracking sensor <NUM> configured to locate scanner <NUM> in the magnetic field; and a transceiver <NUM> configured to output microscan data <NUM>. As shown in <FIG>, microscan data <NUM> include imaging data 43ID (e.g., a wave energy image) generated from the microscan, and location data 43LD (e.g., the location of the wave energy image along axis P-P) generated when the microscan is performed.

Elongated shaft <NUM> of <FIG>may include at least one channel <NUM> extending therethrough, and any number of articulating portions and/or steering mechanisms operable together with channel <NUM>. An exemplary steering mechanism may comprise a handle attached to a proximal end of shaft <NUM>, and a plurality of pull wires extending through shaft <NUM>, wherein the wires are configured to articulate shaft <NUM> in response to a physical force applied to an actuator on the handle, as in <CIT>. In other aspects, shaft <NUM> may include a plurality of electrically responsive articulation sections configured to articulate shaft <NUM> in response to a control signal from controller <NUM>, as in <CIT>. For example, the control signal may be generated in response to a user input, such as the movement of a joystick or like input device, including those described in <CIT>.

Imaging data 43ID may include any number of two- or three-dimensional images generated by internal scanner <NUM> in any format, at any resolution, using any scanning medium. For example, scanner <NUM> may be configured to generate imaging data 43ID including three-dimensional images generated with ultrasound or like techniques. The depth, quality, and/or resolution of imaging data 43ID may be different and/or greater than that of imaging data 23ID, as indicated by exemplary grid patterns depicted in <FIG> and <FIG> respectively. An exemplary scanner <NUM> is depicted in <FIG>. As shown, scanner <NUM> may be disposed on a distal portion 41D of shaft <NUM>, and include a plurality of imaging elements <NUM>; a plurality of light sources <NUM>; and a probe <NUM>. Each element of exemplary scanner <NUM> is now described.

The plurality of imaging elements <NUM> may be configured to generate a portion of imaging data 43ID. For example, data 43ID may be generated continuously and/or when shaft <NUM> is moved along body path axis P-P. Imaging elements <NUM> may be operable to generate images of body <NUM> at one or more frequencies of the electromagnetic spectrum, including frequencies in and beyond the visible light range. As shown in <FIG>, image elements <NUM> may include digital camera circuits configured to generate two-dimensional graphic images (e.g., photographic, topographic, and the like) of the interior of body <NUM>. For example, the plurality of imaging elements <NUM> may be spaced apart annularly around an exterior surface of shaft <NUM> so that the resulting images may be combined to form a continuous panoramic graphical image of the interior of body path <NUM> and/or body cavity <NUM>.

Plurality of light sources <NUM> may comprise light emitting diodes configured to generate light at one or more wavelengths. Light sources <NUM> may be operable with imaging elements <NUM> (e.g., activated together) to provide illumination for imaging data 43ID. Light sources <NUM> may be limited to imaging purposes. For example, because tracking sensor <NUM> may be responsive to a magnetic field, as described below, navigation component <NUM> may be configured for use in the dark, i.e., without any navigational light, or in the presence of light that might challenge visibility, such as light generated by rapidly pulsed laser energy. In some aspects, plurality of light sources <NUM> may be configured to generate a photosensitive response from the interior surfaces of body <NUM>. For example, a photosensitive material may be delivered through lumen <NUM>, body path <NUM> and/or body cavity <NUM>, and then illuminated by light sources <NUM> at a wavelength configured to produce a photosensitive reaction with targeted tissues within path <NUM> and/or cavity <NUM>. In some aspects, characteristics of this reaction (e.g., density, intensity, size, etc.) may be captured by imaging elements <NUM>, output in microscan data <NUM> as imaging data 43ID, and/or processed by controller <NUM>.

Probe <NUM> may be operable independent of or together with imaging elements <NUM> (e.g., activated together) to generate another portion of imaging data 43ID. For example, probe <NUM> may be configured to direct a wave energy (e.g., light, sound, X-rays, etc.) toward the interior surfaces of body <NUM>, receive a reflected portion of the wave energy, and generate imaging data 43ID including two-or three-dimensional wave energy images of body <NUM>. Probe <NUM> of <FIG>, for example, may be an intravascular ultrasound (or "IVUS") probe including a plurality of forward-facing and/or side-facing transducers configured to direct and receive sound waves along path axis P-P and/or in other directions transverse therewith (e.g., <FIG>), and generate imaging data 43ID including a plurality of two-dimensional cross-sectional ultrasound images (e.g., <FIG>). A representative IVUS ultrasound image of body path <NUM> is depicted in <FIG>, for example, as an axial ultrasound image generated from an IVUS probe. In other aspects, probe <NUM> may include a laser scanner configured to discharge a laser energy (e.g., a pulsed or continuous laser beam) into body <NUM>, receive a reflected portion of the laser energy, and generate imaging data 43ID including three-dimensional images of path <NUM> and/or cavity <NUM>.

Tracking sensor <NUM> may be operable with magnetic field generator <NUM> to generate location data 43LD, for example, each time at least one of the plurality of imaging elements <NUM> and/or probe <NUM> are activated. In <FIG>, exemplary sensor <NUM> includes a housing mounted on distal portion 41D of shaft <NUM>, and a sensor coil (not shown) mounted in the housing. Magnetic field generator <NUM> is shown in <FIG> and <FIG> as a planar element with a plurality of field generation elements (not shown) that may be placed, for example, underneath a bed <NUM> supporting body <NUM> (e.g., as in <FIG>), or directly underneath body <NUM> (e.g., as in <FIG>), and configured to generate a magnetic field (not shown) extending about body path <NUM>. The coil in tracking sensor <NUM> may output, in real-time, location data 43LD including a continuous locator signal (e.g., an analog signal) responsive to the magnetic field produced by generator <NUM>.

Transceiver <NUM> may comprise any wired or wireless technology configured to output microscan data <NUM>. For example, transceiver <NUM> may be configured to output data <NUM> and/or otherwise communicate with transceiver <NUM> of controller <NUM> using any known data communication technology, including Bluetooth®, fiber optic cables, Wi-Fi, and the like. Transceiver <NUM> of <FIG>, for example, is coupled to a proximal portion of shaft <NUM> and located outside of body <NUM>. As shown, transceiver <NUM> is configured to receive microscan data <NUM> from internal scanner <NUM> via a wired connection, and transmit microscan data <NUM> to transceiver <NUM> via a wireless connection with controller <NUM>.

Controller <NUM> of <FIG> is configured to generate a pre-operative body information model <NUM> (e.g., <FIG>) from macroscan data <NUM>, operatively update model <NUM> with microscan data <NUM>, and/or output portions of information model <NUM> in pre-operative or updated state, in real-time, as a navigation image <NUM> (e.g., <FIG>). Accordingly, controller <NUM> may include: one or more processors <NUM>; a memory <NUM> including computer-readable instructions <NUM> executable by one or more processors <NUM>; a transceiver <NUM>; sensor interface unit <NUM>; localization unit <NUM>; and a signal station <NUM>.

These elements of controller <NUM> may be organized in a single device, as shown in <FIG>, or distributed throughout system <NUM>, each element being capable of reciprocal communication with the next, and/or the components <NUM> and <NUM>, however arranged. One or more processors <NUM>, for example, may be part of controller <NUM>, distributed throughout system <NUM>, and/or located remotely from system <NUM> and placed in communication therewith. Processors <NUM> are configured to perform any computational function described herein responsive to computer-readable instructions <NUM>. Memory <NUM> may include any technology configured to store instructions <NUM> as well as any other data and/or information described herein. Like processors <NUM>, memory <NUM> also may be located anywhere relative to system <NUM>.

Transceiver <NUM> may comprise any wired or wireless technology configured to receive macroscan data <NUM> and microscan data <NUM>. For example, transceiver <NUM> may be configured to receive data <NUM> and <NUM>, and/or otherwise communicate with transceiver <NUM> of navigation component <NUM> and transceiver <NUM> of navigation component <NUM> using any known data communication technology, including Bluetooth®, fiber optic cables, Wi-Fi, and the like. Transceiver <NUM> may be located anywhere relative to system <NUM>.

Processor <NUM> may be configured to generate location data 43LD any time, such as whenever imaging dating 43ID is generated. For example, sensor interface unit <NUM> (e.g., <FIG>) may be configured to receive the locator signal from tracking sensor <NUM>, and convert (e.g., condition, amplify, and/or digitize) the locator signal into a digital location signal including raw magnetic data. Localization unit <NUM> (e.g., <FIG>) may be configured to receive the digital location signal, and generate instantaneous location data 43LD based on the raw magnetic data, each time at least when one of the plurality of imaging elements <NUM> and/or probe <NUM> are activated. Signal station <NUM> is configured to output location data 43LD, for example, for display within navigation image <NUM>.

Body information model <NUM> may be generated with controller <NUM> by combining two- or three-dimensional images within macroscan data <NUM> according to location data 23LD to define a three-dimensional data set or data mesh. For example, as shown in <FIG>, macroscan data <NUM> may include a first cross-sectional data set <NUM>', a second cross-sectional data set <NUM>", and a third cross-sectional data set <NUM>‴ spaced apart at predetermined intervals along body axis B-B. First data set <NUM>' may be generated by positioning external scanner <NUM> at a first location along axis B-B, generating imaging data (e.g., 23ID'), and associating location data (e.g., 23LD') therewith. External scanner <NUM> may then be moved to second and third positions on axis B-B to generate second data set <NUM>" and third data set <NUM>‴. In this example, controller <NUM> is configured to generate the three-dimensional data set by combining (e.g., stitching) the images within data sets <NUM>', <NUM>", and <NUM>‴ along body axis B-B. Similar methods may be used to combine three-dimensional data sets.

In other aspects, body information model <NUM> may be generated by another device and delivered to controller <NUM> without external scanner <NUM>. For example, information model <NUM> may be included with the patient's electronic medical records and delivered to processor <NUM> via transceiver <NUM>. Body information model <NUM> also may be created with microscan data <NUM> using similar methods.

Controller <NUM> may define at least a general representation of body cavity <NUM>, body path <NUM>, and path axis P-P in this three-dimensional data set. For example, controller <NUM> may automatically define the boundary conditions of body cavity <NUM> and body path <NUM>, and/or plot body path axis P-P therethrough. To enhance this general representation of body <NUM>, controller <NUM> is configured to operatively update (e.g., in real-time, during an operation) a pre-operative version of body information model <NUM> by correlating location data 43LD with location data 23LD at target locations in body <NUM>, and combining imaging data 23ID with a correlated portion of imaging data 43ID at the target locations. For example, because body path axis P-P is defined in model <NUM>, location data 43LD may be correlated with location 23LD by locating a portion of imaging data 43ID at a first location along path axis P-P, geometrically aligning (e.g., resizing, shifting, rotating, etc.) a boundary condition defined by imaging data 23ID at the first location with a boundary condition defined by imaging data 43ID at that location, and then repeating these steps at a plurality of second locations along path axis P-P. A diameter of body path <NUM> may, for example, be used as the boundary condition.

Imaging data 23ID and 43ID may be combined using a variety of techniques. In one aspect, portions of imaging data 43ID are overlaid onto or stitched together with portions of imaging data 23ID at a plurality of locations along axis P-P. For example, controller <NUM> may be configured to select a target location in pre-operative body information model <NUM>, identify imaging data 23ID and 43ID associated with the target location, and overlay aspects of data 43ID onto data 23ID at that location. In accordance with the invention, a cross-sectional image included in imaging data 43ID at the target location is overlaid onto a cross-sectional image included in imaging data 23ID at the same location or a three-dimensional image included in data 43ID is stitched together with a three-dimensional image included in data 23ID at the target location.

Body information model <NUM> may be enhanced by combining imaging data 23ID with imaging data 43ID. For example, because imaging data 43ID may be generated at a higher resolution than imaging data 23ID, owing to the different capabilities of scanner <NUM> and its location within body <NUM>, the resolution of body information model <NUM> may be increased when imaging data 23ID and imaging data 43ID are combined. To provide even more detail, controller <NUM> may be further configured to overlay graphical images captured by the plurality of imaging elements <NUM> onto representation of body cavity <NUM> and/or body path <NUM> in body information model <NUM>, resulting in enhanced or even photo-realistic depictions of the interior surfaces cavity <NUM> and/or path <NUM>. These graphical images may aligned and oriented within body information model <NUM> (e.g., resized, shifted, rotated, etc.) according to location data 23LD and/or 43LD. If a photosensitive material has been delivered into body path <NUM>, and activated by plurality of light sources <NUM>, for example, then any resulting photosensitive response may thus be represented in body information model <NUM>, allowing for identification and/or diagnosis of those responses. Other conditions may be identified and diagnosed using similar methods, with or without a photosensitive material.

When generated by controller <NUM> according to these aspects, body information model <NUM> may be a data-rich environment that is far more detailed than would otherwise be possible with just external scanner <NUM> or internal scanner <NUM>. To further leverage the capabilities of this environment, controller <NUM> may be configured to output aspects of model <NUM> as a two- or three-dimensional navigation image <NUM>. Any aspect of body information model <NUM> described herein may be represented in image <NUM>. For example, in <FIG>, navigation image <NUM> is a three-dimensional representation that has been visually enhanced, for example, by adjusting contrast and color, adding boundary lines, creating a wireframe model, and truncating extraneous anatomy. Any graphical technique may be used. As shown in <FIG>, for example, controller <NUM> may use location data 23LD and 43LD to output navigation image <NUM> as a three-dimensional representation of body <NUM> depicting external surface <NUM>, entry point <NUM>, interior cavity <NUM>, body path <NUM>, and body path axis P-P in relation to internal scanner <NUM>. For example, cavity <NUM> of <FIG> may be the interior of a kidney, and navigation image <NUM> may provide a three-dimensional representation depicting scanner <NUM> moving through a natural body path extending into the kidney through a urethra, bladder, and/or ureter.

Whether two- or three-dimensional, numerous aspects of navigation image <NUM> may be realized in system <NUM>. For example, controller <NUM> may be configured to overlay microscan data <NUM> onto macroscan data <NUM>, and iteratively define the boundaries of body path <NUM> and/or body cavity <NUM>. These boundaries may then be displayed on navigation image <NUM> to provide, for example, a highly detailed two- or three-dimensional map of body <NUM>. With reference to these boundaries, controller <NUM> of <FIG> may be configured to plot a course through body path <NUM>, and/or display the course on navigation image <NUM>, for example, as a two- or three-dimensional pathway. Image <NUM> may be configured to guide shaft <NUM> through body path <NUM> along this pathway. For example, because the location of scanner <NUM> in body <NUM> may be determined from location data 43ID, navigation image <NUM> may include graphical indicators (e.g., directional arrows) configured to guide scanner <NUM> through body path <NUM> along axis P-P.

Responsive to navigation image <NUM>, a user may guide scanner <NUM> into a specific portion of body cavity <NUM> (e.g., a calyx of a kidney) by, for example, manipulating an actuator on the handle of an endoscope so as to articulate and/or advance the distal end 41D of shaft <NUM> through a tortuous portion of body path <NUM>, and/or around a tight corner inside of cavity <NUM>, as shown by the movements between <FIG> and <FIG>. Direct visualization of body path <NUM> may be provided by imaging elements <NUM>, but is not required. For example, image <NUM> may be used to guide distal end 41D toward a kidney stone located in body cavity <NUM>, even if the stone is located in a narrow calyx and surrounded by a turbid fluid; regardless of bio-impedance changes, respiratory patterns, or fluid status; and without reliance on a positional electrodes or visual images. Controller <NUM> may be configured to provide additional navigation and cue features to further guide distal end 41D, such as generating audio or tactile feedback signals.

Controller <NUM> may be configured to identify one or more targeted objects or locations in body <NUM>, and/or determine characteristics of said objects or locations. Navigation image <NUM> may allow for rotation and zoom, permitting optimal views of the targeted objects or locations; and/or the selection of particular objects or locations. Accordingly, controller <NUM> may allow a user to plan an operation with navigation image <NUM>, and annotate image <NUM> during the operation, for example, by marking portions of body cavity <NUM> that have been treated. As a further example, the wave energy generated by probe <NUM> may be utilized to identify kidney stones inside of body cavity <NUM>, determine the location of each kidney stone in body information model <NUM>, and ascertain characteristics of each stone (e.g., composition, density, fragility, and/or size).

Controller <NUM> may be further configured to identify stones having a targeted characteristic, associate the targeted stones with one or more tags or icons in navigation image <NUM>, determine whether a user has removed the targeted stones (or missed one), and/or validate that cavity <NUM> has been cleared of all the stones. Each targeted stone may, for example, be associated with a different tag or icon to facilitate rapid identification. These tags or icons may indicate a size of the stone, such as a maximum width of the stone. Controller <NUM> may be configured to compare the maximum width of each stone with a predetermined maximum size of a retrieval device (e.g., the maximum diameter of a retrieval basket or channel), and indicate whether each stone can removed with said device, allowing the user to determine whether additional treatment is required. For example, a stone may be captured in a retrieval basket, sized to determine a maximum width, and associated with a tag or icon indicating whether the captured stone can be removed in said retrieval basket without further treatment. These tags or icons may, for example, indicate whether the captured stone may be removed if released and recaptured to a lower profile orientation, and/or treated with laser energy to decrease the size of the capture stone.

Similar techniques may be used to determine characteristics (e.g., a size, surface area, or volume) of body cavity <NUM> and/or body path <NUM>, and associate cavity <NUM> and/or path <NUM> with said characteristics. For example, various natural markers within body cavity <NUM> and/or body path <NUM> may be associated with tags and icons in navigation image <NUM> to communication the respective volumes of cavity <NUM> and path <NUM>.

Additional or alternative aspects of navigation component <NUM> and controller <NUM> are now described with reference to <FIG>. System <NUM> may include any combination of these aspects with any other aspects described herein, each possible combination being part of the present disclosure.

According to some additional aspects, controller <NUM> is configured to perform microscan sequences. For example, the plurality of imaging elements <NUM>, plurality of light sources <NUM>, and probe <NUM> may be activated in a predetermined sequence, and localization unit <NUM> may generate location data 43LD each time these elements are activated in the predetermined sequence. Accordingly, system <NUM> may, for example, be switchable between: a first mode with a continuous activation rate corresponding with a desired refresh rate of image <NUM> (e.g., <NUM> or <NUM>); a second mode with a pulsed and/or staggered activation rate corresponding with a desired imaging or treatment method (e.g., photosensitive reactions); and/or a third mode where microscan data <NUM> is automatically generated each time internal scanner <NUM> is moved a predetermined distance along body path axis P-P (e.g., <<NUM>).

Magnetic field generator <NUM> is depicted as a planar element in <FIG> and <FIG>, but may assume any shape, and include any number of field generation elements. For example, an alternative magnetic field generator <NUM> is depicted in <FIG> as a flexible element that is directly attached (e.g., adhered) to a portion body <NUM> (e.g., the back) so as to provide a fixed frame of reference for tracking sensor <NUM>, even if body <NUM> is moved during the time period between each macroscan and/or microscan. Generator <NUM> also may be formed integral with bed <NUM> (<FIG>), which also may provide a reference location for body <NUM>.

In some aspects, tracking sensor <NUM> is operable with additional and/or alternative tracking devices. For example, sensor <NUM> may be operable with a sensor belt <NUM>, depicted in <FIG> as being wrapped around body <NUM>. As shown, belt <NUM> includes three external sensors <NUM>, although any number of sensors may be provided. Belt <NUM> of <FIG> is wrapped around a medial-lateral axis of body <NUM> so that sensors <NUM> are arranged about body axis B-B of body <NUM> in a generally triangular formation. In this configuration, sensors <NUM> may generate location data 43LD for tracking sensor <NUM> as it moved through an interior diameter of belt <NUM>. For example, in FIG. 5B, tracking sensor <NUM> includes an RFID sensor or antenna located on a distal portion of an elongated element, and sensors <NUM> are operable with the RFID sensor to generate location data 43LD when the elongated element is moved through belt <NUM>. Belt <NUM> may have a narrow width (e.g., <NUM>-<NUM> (<NUM>-<NUM> inches)); alternatively, belt <NUM> may be elongated to define a sleeve. In some aspects, belt <NUM> includes tension elements (e.g., elastic, buckles, etc.) configured to hold belt <NUM> in a fixed position relative to body <NUM>.

In still other aspects, internal scanner <NUM> may include a shape-sensing mechanism configured to determine location data 43LD from the shape of shaft <NUM>. For example, the shape-sensing mechanism may be configured to measure the stress and deflection of elongated shaft <NUM> at various positions along its length, and determine location data 43LD therefrom independent of any temperature or loading applied to shaft <NUM>. In some aspects, the shape-sensing mechanism may be included within an optical fiber extending within shaft <NUM>, such as a Fiber Bragg Grating (FBG) fiber. Other exemplary shape-sensing fibers may include those sold by Luna Inventions® (see, e.g.,<NPL>)); and/or those described in <CIT>.

Although not required, controller <NUM> may be configured to steer shaft <NUM> at least partially through body path <NUM> and/or into body cavity <NUM> responsive to body information model <NUM>, with or without navigation image <NUM>. For example, shaft <NUM> may include a plurality of electrically responsive articulation sections, and controller <NUM> may include a steering module <NUM> (shown in dotted lines on <FIG>) configured to generate a steering signal responsive to a user input, such as the movement of a joystick in a proximal or distal direction and/or a confirmation as to location of body cavity <NUM>. Steering module <NUM> may use the steering signal to selectively actuate the plurality of electrically responsive articulation sections of shaft <NUM>. The user input need not be exact or precise. For example, because body information model <NUM> includes a detailed three-dimensional data set, steering module <NUM> may be configured to automatically plot a precise course through a portion of body path <NUM> responsive to a generic user input (e.g., a forward joystick movement), and/or steer shaft <NUM> through body path <NUM> on that course without exact corrections from the user.

Exemplary methods are depicted in <FIG> with reference to various elements of system <NUM> described herein. Numerous aspects of these methods are now described. According to one aspect, shown in <FIG>, an exemplary method <NUM> may comprise: receiving macroscan data <NUM> (<NUM>); generating a pre-operative body information model <NUM> with macroscan data <NUM> (<NUM>); receiving microscan data <NUM> (<NUM>) during an operation; and updating the body information model by combining a portion of microscan data <NUM> with a correlated portion of macroscan data <NUM> (<NUM>).

In method <NUM>, macroscan data <NUM> may include pre-diagnostic and/or pre-operative images taken in advance of an operation. Each image may be generated by external scanner <NUM>. For example, method <NUM> at <NUM> may further comprise: directing a wave energy (e.g., light, sound, X-rays, etc.) toward exterior surface <NUM> of body <NUM>, receiving a reflected portion of the wave energy, and generating imaging data 23ID including wave energy images of body <NUM>. The wave energy may be directed along imaging axis I-I, such that each wave energy image includes a cross-sectional image of body <NUM> (e.g., <FIG>). Method step <NUM> may be repeated at a plurality of positions along body axis B-B of body <NUM>. For example, step <NUM> may comprise positioning external scanner <NUM> at a plurality of positions along a body axis B-B of body <NUM>, generating macroscan data <NUM> including imaging data 23ID and location data 23LD at each of the plurality of positions, and outputting macroscan data <NUM> to controller <NUM>.

Method <NUM> at step <NUM> comprises generating a pre-operative body information model <NUM> with macroscan data <NUM>. Body <NUM> may be moved after the macroscans are completed, such that method step <NUM> may occur anytime after method <NUM>. In keeping with previous examples, method step <NUM> may comprise combining macroscan data <NUM> according to location data 23LD to define a three-dimensional data set. Step <NUM> may further comprise providing at least a general representation of body cavity <NUM>, body path <NUM>, and path axis P-P in this three-dimensional data set.

As noted above, body information model <NUM> may be generated by another device and deliver to controller <NUM> without external scanner <NUM>, in which case, method <NUM> at steps <NUM> and <NUM> may comprise receiving model <NUM> with transceiver <NUM>. For example, method <NUM> may comprise receiving body information model <NUM> from a patient's electronic medical record, modifying aspects of model <NUM> as described herein, and/or updating the electronic medical record accordingly.

Microscan data <NUM> may include two- or three-dimensional images taken during an operation. Each of these images may be generated by internal scanner <NUM>. For example, because of probe <NUM>, method <NUM> at step <NUM> may include: directing a wave energy (e.g., light, sound, X-rays, etc.) toward the interior surfaces of body <NUM>, receiving a reflected portion of the wave energy, and generating imaging data 43ID including wave energy images of body <NUM>. Each wave energy image may, for example, include a cross-sectional image of body path <NUM> (e.g., <FIG>). Graphical images from imaging elements <NUM> may likewise may be included in imaging data 43ID. Accordingly, method step <NUM> may comprise positioning internal scanner <NUM> at a plurality of positions along path axis P-P, generating microscan data <NUM> including imaging data 43ID and location data 43LD at each of the plurality of positions, and outputting data <NUM>.

Method step <NUM> of <FIG> comprises updating the pre-operative body information model <NUM> by combining a portion of microscan data <NUM> with a correlated portion of macroscan data <NUM>. In some aspects, information model <NUM> is updated in real-time as internal scanner <NUM> is moved through body <NUM>. For example, step <NUM> may include: correlating location data 23LD with location data 43LD at a target location in body <NUM>, and combining imaging data 23ID with imaging data 43ID at the target location. Location data 23LD and 43LD may be correlated in body information model <NUM> using any technique. Any means of combining macroscan data <NUM> with microscan data <NUM> may likewise be employed in step <NUM>. For example, a two-dimensional cross-sectional ultrasound image included in imaging data 43ID may be overlaid onto a two-dimensional cross-sectional X-ray image included in imaging data 23ID to update the boundary conditions of body cavity <NUM> and/or body path <NUM>. Likewise, portions of a three-dimensional image in data 23ID may be overlaid with a two-dimensional image in data <NUM>, or stitched together with portions of a three-dimensional image in data 23ID.

Additional steps for method <NUM> are depicted in <FIG> as comprising: outputting a portion of body information model <NUM> as navigation image <NUM> (<NUM>), and guiding shaft <NUM> through body <NUM> responsive to or as function of image <NUM> (<NUM>). Navigation image <NUM> may be a two- or three-dimensional image generated with any technique. For example, method step <NUM> may comprise graphically rendering the three-dimensional data set included in body information model <NUM>. Navigation image <NUM> may be enhanced in step <NUM>. For example, step <NUM> may comprise associating the boundary conditions of body cavity <NUM> and/or body path <NUM> with a particular color or line weight. As a further example, step <NUM> may further comprise superimposing graphical images from imaging elements <NUM> onto these boundary conditions.

Method <NUM> at step <NUM> may include guiding steps. For example, a plurality of body paths <NUM> may be defined through body <NUM>, such that method step <NUM> includes obtaining user input regarding an optimal path <NUM>. Step <NUM> may further comprise generating one or more graphical indictors (e.g., directional arrows) configured to guide elongated shaft <NUM> through path <NUM>. Additional navigation and cue features may be provided at step <NUM>, such as generating audio or tactile feedback signals. Additional targets in body <NUM> may be identified in method <NUM>. For example, step <NUM> may further include identifying one or more targeted objects or locations in body cavity <NUM>, determining characteristics of said one or more objects or locations, and guiding the distal end 41D of elongated shaft <NUM> towards a particular object or location based on its characteristics. Although not shown in <FIG>, additional method steps may be provided with respect to any aspect of system <NUM> described herein.

Claim 1:
A guidance system (<NUM>) comprising:
a controller (<NUM>) including a transceiver (<NUM>), wherein the transceiver is configured to receive macroscan data (<NUM>) including first imaging data (23ID) comprising first images of a body (<NUM>), the macroscan data (<NUM>) further including first location data (23LD) associated with each first image, wherein the controller (<NUM>) is configured to combine the first images according to the first location data (23LD) to generate a body information model (<NUM>);
a navigation component (<NUM>) including an elongated shaft (<NUM>) and an internal scanner (<NUM>) located on a distal portion (41D) of the elongated shaft (<NUM>), wherein the internal scanner (<NUM>) is configured to be positioned in the body (<NUM>) and to generate microscan data (<NUM>), the microscan data (<NUM>) including second imaging data (43ID) comprising second images of the body (<NUM>), the microscan data (43ID) further including second location data (43LD) associated with each second image;
wherein the controller (<NUM>) is configured to correlate the first location data (23LD) with the second location data (43LD) at a target location in the body information model (<NUM>) and to modify the body information model (<NUM>) by combining the first images with the second images at the target location, wherein the controller (<NUM>) is configured to overlay a cross-sectional image included in the second imaging data (43ID) onto a cross-sectional image included in the first imaging data (23ID) at the target location or to stitch a three-dimensional image included in the second imaging data (43ID) together with a three-dimensional image included in the first imaging data (23ID) at the target location.