Patent Description:
Minimally invasive medical techniques are intended to reduce the amount of tissue that is damaged during medical procedures, thereby reducing patient recovery time, discomfort, and harmful side effects. Such minimally invasive techniques may be performed through natural orifices in a patient anatomy or through one or more surgical incisions. Through these natural orifices or incisions, an operator may insert minimally invasive medical tools to reach a target tissue location. Minimally invasive medical tools include instruments such as therapeutic, diagnostic, biopsy, and surgical instruments. Medical tools may be inserted into anatomic passageways and navigated toward a region of interest within a patient anatomy. Navigation may be assisted using images of the anatomic passageways. Improved systems and methods are needed to accurately perform registrations between medical tools and images of the anatomic passageways.

Disclosed are devices, systems, methods, and computer program products for mitigating oversampling of data points collected by a medical device when steered to particular regions of an anatomic structure for surveying the anatomic structure, such as airways in regions of the lungs and bronchial tubes, e.g., in advance of a medical procedure.

In some embodiments, for example, a system for mitigating oversampling of data points includes a medical device comprising a sensor, wherein the medical device is insertable in an anatomic passageway of a patient such that the sensor is operable to detect one or both of a position and a motion of the medical device when inserted in the anatomic passageway; and a computing device in communication with the medical device, the computing device comprising a processor, and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving data points detected by the sensor of the medical device, the received data points associated with one or both of the detected position and the detected motion of the medical device; analyzing a set of the received data points to determine a motion parameter associated with a movement or change in position of the sensor of the medical device in a region of the anatomic passageway, wherein the motion parameter includes a change of one or both of a translational motion and a rotational motion of the sensor; comparing the motion parameter to a threshold to determine whether to accept the set of data points when the motion parameter satisfies the threshold or to reject the set of data points when the motion parameter does not satisfy the threshold; and recording the accepted set of data points in a survey point cloud usable to register the medical device in an anatomic frame of reference space.

In another embodiement, a system is defined in claim <NUM>.

In another embodiment, a system is defined in claim <NUM>.

It is to be understood that both the foregoing general description and the following details description are exemplary and explanatory in nature and are intended to provide an understanding of the present disclosure without limiting the scope of the present disclosure.

Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale. Instead, emphasis is placed on illustrating clearly the principles of the present disclosure. The drawings should not be taken to limit the disclosure to the specific embodiments depicted, but are for explanation and understanding only.

The system and techniques disclosed herein may be used to register a medical instrument reference frame to an image frame of reference for an intra-operative anatomic image that includes an image of the medical instrument, such as a catheter. Often, anatomical motion can result in intra-operative images that are too distorted to clearly isolate and segment the catheter and in medical instrument position data that is agitated. By representing the intra-operative image of the medical instrument as a cloud of points (also referred to as a "image point cloud") and the shape of the medical instrument (obtained by a sensor during the image capture period) as a cloud of points (also referred to as a "sensor point cloud"), point matching registration techniques, such as an iterative closest point (ICP) technique, can be used to register the sensor point cloud and the image point cloud. The robustness of this registration technique allows the image frame of reference to be registered to the medical instrument frame of reference, despite data spread caused by patient anatomical motion.

Specific details associated with several embodiments of the present technology are described herein, some with reference to <FIG>. Although some of the embodiments are described with respect to particular medical systems and devices in the context of navigating and performing medical procedures within lungs of a patient, other applications and other medical system and medical device embodiments in addition to or alternative to those described herein are within the scope of the present technology. For example, unless otherwise specified or made clear from context, the devices, systems, methods, and computer program products of the present technology can be used for various image-guided medical procedures, such as medical procedures performed on, in, or adjacent hollow patient anatomy, and, more specifically, in procedures for surveying, biopsying, ablating, or otherwise treating tissue within and/or proximal the hollow patient anatomy. Thus, for example, the systems, devices, methods, and computer program products of the present disclosure can be used in one or more medical procedures associated with other patient anatomy, such as the bladder, urinary tract, and/or heart of a patient.

It should be noted that other embodiments in addition to those disclosed herein are within the scope of the present technology. Further, embodiments of the present technology can have different configurations, components, and/or procedures than those shown or described herein. Moreover, a person of ordinary skill in the art will understand that embodiments of the present technology can have configurations, components, and/or procedures in addition to those shown or described herein and that these and other embodiments can be without several of the configurations, components, and/or procedures shown or described herein without deviating from the present technology.

As used herein, the term "physician" shall be understood to include any type of medical personnel who may be performing or assisting a medical procedure and, thus, is inclusive of a doctor, a nurse, a medical technician, other similar personnel, and any combination thereof. Additionally, or alternatively, as used herein, the term "medical procedure" shall be understood to include any manner and form of diagnosis, treatment, or both, inclusive of any preparation activities associated with such diagnosis, treatment, or both. Thus, for example, the term "medical procedure" shall be understood to be inclusive of any manner and form of movement or positioning of a medical device in an anatomical chamber. As used herein, the term "patient" should be considered to include human and/or non-human (e.g., animal) patients upon which a medical procedure is being performed.

Point matching registration techniques, like ICP technique, used to register collected data points in a point cloud are generally robust, in that, implementations of such techniques can provide reliable registration data for establishing a frame of reference to track a medical instrument relative to the patient's anatomy within which it is inserted. However, any point matching technique, including ICP, are susceptible to some degree of error as a result of inaccuracy in the collected point sets. This is due to misalignments between the real anatomical structure of the patient's anatomy and a model of the anatomical structure that are inherent. Commonly, this error is the result of physical deformation (e.g., patient breathing, motion, shifting) relative to the previously-acquired model of the patient anatomy, e.g., created from previously-acquired data such as in a pre-operation image of the patient's anatomy to produce an initial model of the anatomical structure. This variability in physical deformation can result in variabilities in the level of misalignment in different parts of the anatomical structure. In general, an optimal registration is considered that which minimizes the misalignment between real and model-based reference frames.

Registration techniques such as ICP can mathematically compute this optimal alignment by minimizing the error between collected and model-based point sets. However, because ICP is susceptible to inherent physiological misalignments, the resulting error can be made worse by sampling processes that magnify or give more weight to some data over others. Such sampling processes can be affected by the way a medical instrument is manipulated to collect or sample point data from various regions. One example of worsening the error in point matching registration is "overdriving" of the insertable medical instrument where a sensor associated with the insertable medical instrument is used to disproportionally survey an area or areas of the patient's anatomy as compared to other areas. Overdriving results in the oversampling and over-representation of points for the over-surveyed areas, which causes undue weighting of the data points in the point matching registration process.

As an example, some medical systems may implement a registration protocol that requires the system user to position the sensor (e.g., associated with the system's insertable medical device) in a plurality of anatomic regions. The registration protocol facilitates the collection of data points for a point cloud to be registered with a global data set (e.g., created from a pre-procedural image), referred to as "survey data. " For example, the registration protocol may request or require the system user to move the medical device with the associated sensor device to a first area of the anatomic structure (e.g., determined from the pre-procedural image), to a second area, to a third area, and so forth. However, the medical system may have little to no control on where, when or how the system user "drives" the medical device with the associated sensor device during the registration protocol, which makes the registration prone to oversampling data in regions of the anatomic structure where the user may "overdrive" the device more frequently with respect to other regions of interest. This may cause inaccuracies in the registration of survey data with the global data set.

One way to deal with this issue is to not collect any survey data when the medical device is stationary and only collect data when the medical device is in motion relative to the anatomic structure. However, this technique is insufficient as data would still be oversampled when the device is moved repetitively by the user in a given area of the anatomic structure. What is needed is an effective and convenient (e.g., non-taxing of computing resources) way to mitigate oversampling of survey data when a medical device performs registration.

In some embodiments in accordance with the present technology, a computer-implemented method for mitigating oversampling of data points collected by a sensor associated with a medical device includes analyzing (i) parameter(s) of the sensor (of the medical device) and (ii) parameter(s) of the sampled data points, and, in real-time, comparing one or both of the analyzed sensor parameter(s) and/or data point parameter(s) to a threshold value, respectively, where individual data points among the sampled data points are recorded in a registration point cloud when the respective parameter(s) satisfies the threshold. An example embodiment of such a method is described in connection with <FIG>.

<FIG>, for example, is a flow diagram illustrating a method <NUM> for mitigating oversampling of data points in accordance with various embodiments of the present technology. Various embodiments of the method <NUM> can be based on a point sampling technique to mitigate oversampling and/or on a density normalization technique to mitigate oversampling. All or a subset of the steps of the method <NUM> can be implemented by a computing device, such as a control system of a medical system or device, including various components or devices of a robotic or teleoperated system. The method <NUM> includes a set of operations or processes <NUM>-<NUM>.

The computing device for implementing the method <NUM> includes one or more processors coupled to one or more memory devices storing instructions that, when executed by the one or more processors, cause the computing device to perform operations in accordance with the processes <NUM>-<NUM>. In some implementations where the computing device is included in a robotic or teleoperated medical system, the computing device is in data communication with a medical instrument system, which includes the medical device and the sensor, and receives sensor data for mitigating oversampling of data points. The sensor is configured to generate position sensor data and/or motion sensor data during a registration protocol where the medical device is driven in an anatomical structure or structures of the patient (e.g., driven through anatomic passageway(s) of the patient). In this manner, the position sensor data is associated with one or more positions of the medical device within the anatomic passageway, and the motion sensor data is associated with the translational motion and/or the rotational motion of the medical device within the anatomic passageway. Optionally, in some embodiments, the medical instrument system includes an image capture device configured to capture image data of patient anatomy within the anatomic passageway during the data sampling of the anatomic structure(s). The method <NUM> is described below with reference to an exemplary robotic or teleoperated medical system <NUM> ("medical system <NUM>"), discussed later in connection with <FIG> and <FIG>.

At process <NUM>, the method <NUM> receives, at the computing device, data points that correspond to a sampled survey point cloud detected by a sensor of a medical device (e.g., the shape sensor <NUM> and/or of the position measuring device <NUM> of the medical instrument system <NUM> shown in <FIG> and <FIG>) during data sampling by the sensor of an anatomic structure or structures of a patient. The received data points at the computing device can be associated with a position and/or a motion of the sensor, e.g., thereby of the medical device.

At process <NUM>, the method <NUM> determines, at the computing device, a first parameter associated with the medical device (e.g., the sensor and/or other component of the medical device) and/or a second parameter associated with the received data points. In some implementations of the process <NUM>, the first parameter includes a motion parameter associated with the medical device. In such implementations, determining the first parameter can include determining a change of the translational motion and/or rotational motion of the medical device, such as a change in a roll value or a pitch value and/or a yaw value of the sensor of the medical device (e.g., such as the medical instrument system <NUM>, at the tip of the shape sensor <NUM>). In some implementations of the process <NUM>, the second parameter can include a point distance parameter and/or a point density parameter associated with the received data points. In such implementations, determining the second parameter can include determining (i) a distance from a data point to its nearest neighbor within the sampled survey point cloud, and/or (ii) a density of the data points, e.g., within a predefined subset of the sampled survey point cloud corresponding to a sub-region of the anatomic structure. One, some or all of the above example features may be implemented by the process <NUM>.

At process <NUM>, the method <NUM> analyzes, at the computing device, the first parameter and/or the second parameter by comparing the first parameter to a first threshold and/or by comparing the second parameter to a second threshold, respectively. For example, the first threshold and second threshold can each include a threshold value or range of values. As an example, the first threshold value or range of values can include a velocity (or velocity range) that the sensor exhibited by movement from the previous sample. As another example, the second threshold value or range of values can include a minimum distance or distance range that the sensor was translated or rotated from the previous sample, e.g., the previous sample taken temporally. In some implementations of the process <NUM>, where the second parameter includes a determined distance from a data point to its nearest neighbor within the sampled survey point cloud, the determined distance can be compared to a distance threshold. In some implementations of the process <NUM>, where the second parameter includes a determined density value of data points, the determined density value can be compared to a density threshold. One, some or all of the above example features may be implemented by the process <NUM>.

At process <NUM>, the method <NUM> records, at the computing device, an individual data point (among the received data points) in a registration point cloud when the first parameter and/or the second parameter satisfies the respective threshold. In this manner, for example, the identified individual data point(s) from the received data points can be added to the recorded coordinate points that form positional point cloud data representing a shape of the medical device within an anatomic region. In some implementations of the process <NUM>, the received data points are initially recorded in the registration point cloud, after which the process <NUM> rejects any individual data point when the determined second parameter satisfies the threshold value. Yet, in some implementations of the process <NUM>, the process <NUM> includes only adding an individual data point when the determined second parameter satisfies the threshold value. Yet, in some implementations, prior to recording the individual data points, the process <NUM> can be implemented to decrease a weighting value of a data point when the determined density of data points (as the second parameter) exceeds a threshold density.

In some embodiments, the method <NUM> provides a motion collection-based technique for mitigation of oversampled data. Whereas, in some embodiments, the method <NUM> provides a point distance rejection-based technique for mitigation of oversampled data. Yet, in some embodiments, the method <NUM> provides a point density rejection-based technique for mitigation of oversampled data.

<FIG> is a flow diagram depicting an example of a motion collection-based oversampling mitigation method <NUM> in accordance with some embodiments of the method <NUM>. For example, the method <NUM> can be used to limit the collection of survey data until motor encoder values from either IO or pitch/yaw have changed enough to qualify as motion of the medical device, such as a catheter. All or a subset of the steps of the method <NUM> can be implemented by the computing device, e.g., such as the control system <NUM> of the medical system <NUM> described later, or various other components or devices of a robotic or teleoperated system. In various implementations of the method <NUM>, for example, the sensor can include the shape sensor <NUM> and/or the position measuring device <NUM> of the medical instrument system <NUM>, and the method <NUM> can be implemented during surveying of an anatomic structure or structures of a patient, such as in a registration protocol during an implementation of the sensor system <NUM>.

At process <NUM>, the method <NUM> receives survey data points detected by the sensor of the medical device (e.g., shape sensor <NUM>) for determining when the medical device is moved in one or more particular translational and/or rotational motions, e.g., roll motion (delta ϕ) or pitch or yaw motions. During surveying by the sensor (process <NUM>), the method <NUM> includes a process <NUM> to determine a change of the translational and/or rotational motion (e.g., delta IO or pitch/yaw values). At process <NUM>, the method compares the change to a threshold (e.g., threshold value or range of values) associated with the translational and/or rotational motion. At process <NUM>, the method <NUM> records survey data points in the point cloud when the determined change in motion meets the threshold, and not record (e.g., discard) the survey data points in the point cloud when the determined change in motion does not meet the threshold. In this manner, for example, the method <NUM> can limit the collection of survey data that will be included in the point cloud based on the sensor (e.g., encoder) values of a particular magnitude, such as a substantial change in IO or pitch/yaw, to qualify as motion of the medical device within the anatomic region during registration-not just simple movement of the medical device.

In an example implementation of the method <NUM>, the shape sensor <NUM> and/or the position measuring device <NUM> of the medical instrument system <NUM> is driven in one or multiple anatomic passageways of the patient. At the process <NUM>, the control system <NUM> of the medical system <NUM> receives all of the data generated by the shape sensor <NUM> and/or the position measuring device <NUM>. At the process <NUM>, the control system <NUM> determines whether there is a change in movement and/or position of the shape sensor <NUM> and/or the position measuring device <NUM>; and if there is a determined change, the control system <NUM> determines a value of the change, i.e., a delta of the movement and/or a delta of the position. If no change is determined, the control system <NUM> assigns a delta of zero to the movement and/or position parameter (e.g., the first parameter). At the process <NUM>, the control system <NUM> compares the determined value of the change to a threshold value (or range of threshold values) for determining whether to accept or reject the received survey data sampled from the medical instrument system <NUM>. In one non-limiting example, the threshold value is <NUM> in a position change from the previously collected point. The threshold value (or threshold range) can be predetermined and stored in the memory of the control system <NUM>. At the process <NUM>, the control system <NUM> records the survey data points in the point cloud when it is determined at the process <NUM> that the determined value of the change meets the threshold value. For example, when the delta is zero or less than the threshold (or outside of any threshold range), the survey data will be rejected at the process <NUM>. For example, when the delta is at or greater than the threshold (or within a threshold range), the survey data will be accepted at the process <NUM>. In this manner, the method <NUM> mitigates potential oversampling by the system <NUM> by using only the accepted data to register the medical instrument system <NUM> in anatomic space (e.g., which corresponds with an image space from a pre-operation image).

<FIG> is a flow diagram depicting an example of a point distance rejection-based oversampling mitigation method <NUM> in accordance with some embodiments of the method <NUM>. All or a subset of the steps of the method <NUM> can be implemented by the computing device, such as the control system <NUM> of the medical system <NUM>, or various other components or devices of a robotic or teleoperated system. In various implementations of the method <NUM>, for example, the sensor can include the shape sensor <NUM> and/or of the position measuring device <NUM> of the medical instrument system <NUM>, and the method <NUM> can be implemented during surveying of an anatomic structure or structures of a patient, e.g., such as in a registration protocol during an implementation of the sensor system <NUM>.

At process <NUM>, the method <NUM> receives survey data points detected by the sensor of the medical device (e.g., shape sensor <NUM> at tip and/or body), which are recorded to a sampled survey point cloud. At process <NUM>, the method <NUM> determines, e.g., in real-time during surveying by the sensor (e.g., at process <NUM>), a distance from a data point to its nearest neighbor within the sampled survey point cloud. At process <NUM>, the method <NUM> compares the determined distance to a threshold distance, e.g., threshold distance value or range of distance values. At process <NUM>, the method <NUM> rejects a data point from the recorded sampled survey point cloud when the determined distance of that data point is within the threshold distance of the nearest neighbor. In this manner, for example, the method <NUM> adds the surveyed data points to the point cloud and rejects those data points whose distance are determined to be too close to nearest neighbors, e.g., in a real-time evaluation during a registration protocol of the medical device.

In an example implementation of the method <NUM>, the shape sensor <NUM> and/or the position measuring device <NUM> of the medical instrument system <NUM> is driven in one or multiple anatomic passageways of the patient. At the process <NUM>, the control system <NUM> of the medical system <NUM> receives all of the data generated by the shape sensor <NUM> and/or the position measuring device <NUM> and initially records all of the data to the point cloud. At the process <NUM>, the control system <NUM> examines at least a set of the recorded data to the point cloud by determining a distance of a data point or data points within the set to other nearest neighbor data points within the set. At the process <NUM>, the determined distance between each data point and its nearest neighbors is compared to a threshold (e.g., a threshold value or a threshold range), e.g., which provides the control system <NUM> with a 'degree of closeness' of the data point to its nearest neighbors. In implementations of the processes <NUM> and <NUM>, for example, the control system <NUM> can calculate a set of K nearest neighbor distances and evaluate the point using any number of nearest neighbors. In one case, the number of nearest neighbors, K, may be specified by the user or software. Alternatively, the number of nearest neighbors may be determined as the set of all points that lie within a specified distance from the point in question. At the process <NUM>, the data points determined to be 'too close' to their nearest neighbors (i.e., its distance is within the threshold distance of its nearest neighbor(s)), are rejected from the point cloud.

<FIG> is a flow diagram depicting an example of a point density rejection-based oversampling mitigation method, <NUM> in accordance with some embodiments of the method <NUM>. All or a subset of the steps of the method <NUM> can be implemented by the computing device, such as the control system <NUM> of the medical system <NUM>, or various other components or devices of a robotic or teleoperated system. In various implementations of the method <NUM>, for example, the sensor can include the shape sensor <NUM> and/or of the position measuring device <NUM> of the medical instrument system <NUM>, and the method <NUM> can be implemented during surveying of an anatomic structure or structures of a patient, e.g., such as in a registration protocol during an implementation of the sensor system <NUM>.

At process <NUM>, the method <NUM> receives survey data points detected by the sensor of the medical device (e.g., shape sensor <NUM> at tip and/or body), which are recorded to a sampled survey point cloud. At process <NUM>, the method <NUM> determines, e.g., in real-time during surveying by the sensor (e.g., at process <NUM>), a density of the data points, e.g., within a subset of the sampled survey point cloud corresponding to a sub-region of the anatomic structure (e.g., predefined subset). At process <NUM>, the method <NUM> compares the determined density to a threshold density, e.g., threshold density value or range of density values. At process <NUM>, the method <NUM> rejects a data point from the recorded sampled survey point cloud when the determined density of data points (that encompasses that data point) is within the threshold density of the sub-region. In this manner, for example, the method <NUM> adds the surveyed data points to the point cloud and rejects them in real-time upon evaluation with respect to a point density threshold, e.g., which can be a point density threshold within a region or regions (of various sizes, e.g., predefined) of the anatomic structure.

In an example implementation of the method <NUM>, the shape sensor <NUM> and/or the position measuring device <NUM> of the medical instrument system <NUM> is driven in one or multiple anatomic passageways of the patient. At the process <NUM>, the control system <NUM> of the medical system <NUM> receives all of the data generated by the shape sensor <NUM> and/or the position measuring device <NUM> and initially records all of the data to the point cloud. At the process <NUM>, the control system <NUM> begins examining the density of data points within a set of the recorded data to the point cloud. For example, at process <NUM>, the control system <NUM> determines a density of data points within the set that includes an analysis of the data points within the set with respect to their nearest neighbor data points. At the process <NUM>, the determined density of data points within the set is compared to a threshold (e.g., a threshold value or a threshold range), e.g., which provides the control system <NUM> with a 'degree of denseness' of the data points with respect to their nearest neighbors within the set. In implementations of the processes <NUM> and <NUM>, for example, the control system <NUM> can calculate a set of K nearest neighbor distances and evaluate each point using any number of nearest neighbors. When the set is determined to be 'too dense' within data points, the control system <NUM> can reject data point(s) to mitigate the oversampling. At the process <NUM>, the data points determined to be in a 'dense set' with respect to their nearest neighbors are rejected from the point cloud.

<FIG> is a flow diagram depicting an example of a survey density normalization-based oversampling mitigation method <NUM> in accordance with some embodiments of the method <NUM>. Like the methods described above, all or a subset of the steps of the method <NUM> can be implemented by a computing device, such as the control system <NUM> of the medical system <NUM>, or various other components or devices of a robotic or teleoperated system. In various implementations of the method <NUM>, the sensor can include the shape sensor <NUM> and/or of the position measuring device <NUM> of the medical instrument system <NUM>, and the method <NUM> can be implemented during surveying of an anatomic structure or structures of a patient, e.g., such as in a registration protocol during an implementation of the sensor system <NUM>.

In some examples of the method <NUM>, survey data points would be collected by the sensor, but certain data points might be removed if over-sampled in a given region. For example, a straight-forward approach would be to keep all points but to decrease the per-point weighting within densely surveyed regions, which can be implemented as an augmentation to the ICP algorithm because weighting is already a variable employed by standard ICP algorithms. Within an example ICP algorithm, a registration is computed at each step using the cumulative set of nearest-neighbor matches between the surveyed point cloud and the comparative data set of the anatomic structure, e.g., a pre-operative image data set of an airway tree. In implementations of the method <NUM>, for example, by reducing the weighting applied to point matches within a given region, the method can effectively down-weight or correct for over-sampling in that region. The method <NUM> includes a set of operations or processes <NUM>-<NUM> described below.

At process <NUM>, the method <NUM> receives survey data points detected by the sensor of the medical device (e.g., shape sensor <NUM>), which can be recorded to a sampled survey point cloud based on the outcomes of the processes <NUM> and <NUM>. At process <NUM>, the method <NUM> determines, e.g., in real-time during surveying by the sensor (e.g., at process <NUM>), a density of the data points, e.g., within a subset of the sampled survey point cloud corresponding to a sub-region of the anatomic structure (e.g., predefined subset). In some implementations of the process <NUM>, for example, the determined density is based on distance parameters from medical device's location within the anatomic region. At process <NUM>, the method <NUM> compares the determined density to a threshold density for the sub-region, e.g., threshold density value or range of density values.

The method <NUM> includes a process <NUM> to (i) record the collected survey data points to the survey point cloud and (ii) decrease the weighting value associated with a data point within the sub-region when the determined density exceeds the threshold density (e.g., referred to as oversampled sub-region). In some implementations of the process <NUM>, for example, the weighting value is normalized for weighting values associated with data points in the over-sampled sub-region, where the normalization process includes dividing weighting by total number of matches to nearest survey points. As an illustrative example, the process <NUM> can be implemented where the weighting would be normalized such that points in the anatomic structures, like the pulmonary airway tree, that are nearest to multiple survey points would have their weighting divided by the total number of matches. In some examples, matches could be down-weighted or up-weighted depending on the local density of points. In such cases, for example, the density can be computed based on the number of survey points in a given volume.

Yet, in some implementations of the process <NUM>, the weighting value is normalized by smoothing data points along a length line traversing at least a portion of the over-sampled sub-region. As an illustrative example, the process <NUM> can be implemented where the density is normalized by computing the number of matches that occur along a given length of the anatomic structure (e.g., an airway in a pulmonary airway tree). In such a case, for example, larger areas of over-sampling can be smoothed out by normalizing the local weighting density of all survey points along each airway. The result would be that the registration is balanced over the total length of driven airways.

<FIG> is a schematic representation of a robotic or teleoperated medical system <NUM> ("medical system <NUM>") configured in accordance with various embodiments of the present technology. As shown, the medical system <NUM> includes a manipulator assembly <NUM>, a medical instrument system <NUM>, a master assembly <NUM>, and a control system <NUM>. The manipulator assembly <NUM> supports the medical instrument system <NUM> and drives the medical instrument system <NUM> at the direction of the master assembly <NUM> and/or the control system <NUM> to perform various medical procedures on a patient <NUM> positioned on a table <NUM> in a surgical environment <NUM>. In this regard, the master assembly <NUM> generally includes one or more control devices that can be operated by an operator <NUM> (e.g., which can be a physician) to control the manipulator assembly <NUM>. Additionally, or alternatively, the control system <NUM> includes a computer processor <NUM> and at least one memory <NUM> for effecting control between the medical instrument system <NUM>, the master assembly <NUM>, and/or other components of the medical system <NUM>. The control system <NUM> can also include programmed instructions (e.g., a non-transitory computer-readable medium storing the instructions) to implement any one or more of the methods described herein, including instructions for providing information to a display system <NUM> and/or processing data for registration of the medical instrument <NUM> for various medical procedures on the patient by the medical system <NUM> (as described in greater detail below). The manipulator assembly <NUM> can be a teleoperated, a non-teleoperated, or a hybrid teleoperated and non-teleoperated assembly. Thus, all or a portion of the master assembly <NUM> and/or all or a portion of the control system <NUM> can be positioned inside or outside of the surgical environment <NUM>.

In some embodiments, to aid the operator <NUM> in controlling the manipulator assembly <NUM> and the medical instrument system <NUM>, the medical system <NUM> further includes a sensor system <NUM>, an endoscopic imaging system <NUM>, an imaging system <NUM>, a virtual visualization system <NUM>, and/or the display system <NUM>. In some embodiments, the sensor system <NUM> includes a position/location sensor system (e.g., an electromagnetic (EM) sensor system) and/or a shape sensor system for determining position, orientation, speed, velocity, pose, and/or shape of the medical instrument system <NUM> (e.g., while the medical instrument system <NUM> is within the patient <NUM>). In these and other embodiments, the endoscopic imaging system <NUM> includes one or more image capture devices (not shown) (e.g., such as an imaging scope assembly and/or an imaging instrument) that records endoscopic image data, including concurrent or real-time images (e.g., video, still images, etc.) of patient anatomy. Images captured by the endoscopic imaging system <NUM> may be, for example, two or three-dimensional images of patient anatomy captured by an imaging instrument positioned within the patient <NUM>, and are referred to hereinafter as "real navigational images.

In some embodiments, the medical instrument system <NUM> may include components of the sensor system <NUM> and/or of the endoscopic imaging system <NUM>. For example, components of the sensor system <NUM> and/or components of the endoscopic imaging system <NUM> can be integrally or removably coupled to the medical instrument system <NUM>. Additionally, or alternatively, the endoscopic imaging system <NUM> can include a separate endoscope (not shown) attached to a separate manipulator assembly (not shown) that can be used in conjunction with the medical instrument system <NUM> to image patient anatomy. The sensor system <NUM> and/or the endoscopic imaging system <NUM> may be implemented as hardware, firmware, software, or a combination thereof that interact with or are otherwise executed by one or more computer processors, such as the computer processor(s) <NUM> of the control system <NUM>.

The imaging system <NUM> of the medical system <NUM> may be arranged in the surgical environment <NUM> near the patient <NUM> to obtain real-time and/or near real-time images of the patient <NUM> before, during, and/or after a medical procedure. In some embodiments, the imaging system <NUM> includes a mobile C-arm cone-beam computerized tomography (CT) imaging system for generating three-dimensional images. For example, the imaging system <NUM> can include a DynaCT imaging system from Siemens Corporation or another suitable imaging system. In these and other embodiments, the imaging system <NUM> can include other imaging technologies, including magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and/or the like.

In these and other embodiments, the control system <NUM> further includes the virtual visualization system <NUM> to provide navigation assistance to the operator <NUM> when controlling the medical instrument system <NUM> during an image-guided medical procedure. For example, virtual navigation using the virtual visualization system <NUM> can be based upon reference to an acquired pre-operative or intra-operative dataset (e.g., based upon reference to data generated by the sensor system <NUM>, the endoscopic imaging system <NUM>, and/or the imaging system <NUM>) of anatomic passageways of the patient <NUM>. In some implementations, for example, the virtual visualization system <NUM> processes image data of the patient anatomy captured using the imaging system <NUM> (e.g., to generate an anatomic model of an anatomic region of the patient <NUM>). The virtual visualization system <NUM> can register the image data and/or the anatomic model to data generated by the sensor system <NUM> and/or to data generated by the endoscopic imaging system <NUM> to (i) determine position, pose, orientation, shape, and/or movement of the medical instrument system <NUM> within the anatomic model (e.g., to generate a composite virtual navigational image), and/or (ii) determine a virtual image (not shown) of patient anatomy from a viewpoint of the medical instrument system <NUM> within the patient <NUM>. For example, the virtual visualization system <NUM> can register the anatomic model to positional sensor data generated by the positional sensor system <NUM> and/or to endoscopic image data generated by the endoscopic imaging system <NUM> to (i) map the tracked position, orientation, pose, shape, and/or movement of the medical instrument system <NUM> within the anatomic region to a correct position within the anatomic model, and/or (ii) determine a virtual navigational image of virtual patient anatomy of the anatomic region from a viewpoint of the medical instrument system <NUM> at a location within the anatomic model corresponding to a location of the medical instrument system <NUM> within the patient <NUM>.

The display system <NUM> can display various images or representations of patient anatomy and/or of the medical instrument system <NUM> that are generated by the sensor system <NUM>, by the endoscopic imaging system <NUM>, by the imaging system <NUM>, and/or by the virtual visualization system <NUM>. In some embodiments, the display system <NUM> and/or the master assembly <NUM> may be oriented so the operator <NUM> can control the manipulator assembly <NUM>, the medical instrument system <NUM>, the master assembly <NUM>, and/or the control system <NUM> with the perception of telepresence.

As discussed above, the manipulator assembly <NUM> drives the medical instrument system <NUM> at the direction of the master assembly <NUM> and/or the control system <NUM>. In this regard, the manipulator assembly <NUM> can include select degrees of freedom of motion that may be motorized and/or teleoperated and select degrees of freedom of motion that may be non-motorized and/or non-teleoperated. For example, the manipulator assembly <NUM> can include a plurality of actuators or motors (not shown) that drive inputs on the medical instrument system <NUM> in response to commands from the control system <NUM>. The actuators can include drive systems (not shown) that, when coupled to the medical instrument system <NUM>, can advance the medical instrument system <NUM> into a naturally or surgically created anatomic orifice. Other drive systems may move a distal portion (not shown) of the medical instrument system <NUM> in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). Additionally, the actuators can be used to actuate an articulable end effector of the medical instrument system <NUM> (e.g., for grasping tissue in the jaws of a biopsy device and/or the like).

<FIG> is a schematic representation of a manipulator assembly <NUM>, a medical instrument system <NUM>, and an imaging system <NUM> in a surgical environment <NUM> and configured in accordance with various embodiments of the present technology. In some embodiments, the manipulator assembly <NUM>, the medical instrument system <NUM>, and/or the imaging system <NUM> are the manipulator assembly <NUM>, the medical instrument system <NUM>, and/or the imaging system <NUM>, respectively, of <FIG>. As shown, the surgical environment <NUM> illustrated in <FIG> has a surgical frame of reference (XS, YS, ZS) in which a patient <NUM> is positioned on a table <NUM>, and the medical instrument system <NUM> illustrated in <FIG> has a medical instrument frame of reference (XM, YM, ZM) within the surgical environment <NUM>. During the medical procedure, the patient <NUM> may be stationary within the surgical environment <NUM> in the sense that gross patient movement can be limited by sedation, restraint, and/or other means. In these and other embodiments, cyclic anatomic motion of the patient <NUM>, including respiration and cardiac motion, may continue unless the patient <NUM> is asked to hold his or her breath to temporarily suspend respiratory motion.

The manipulator assembly <NUM> includes an instrument carriage <NUM> mounted to an insertion stage <NUM>. In some embodiments, the insertion stage <NUM> is fixed within the surgical environment <NUM>. Alternatively, the insertion stage <NUM> can be movable within the surgical environment <NUM> but have a known location (e.g., via a tracking sensor or other tracking device) within the surgical environment <NUM>. In these alternatives, the medical instrument frame of reference (XM, YM, ZM) is fixed or otherwise known relative to the surgical frame of reference (XS, YS, ZS). In the illustrated embodiment, the insertion stage <NUM> is linear, while in other embodiments, the insertion stage <NUM> is curved or has a combination of curved and linear sections.

The medical instrument system <NUM> of <FIG> includes an elongate device <NUM>, a medical instrument <NUM>, an instrument body <NUM>, a sensor system <NUM>, and an endoscopic imaging system <NUM>. In some embodiments, the elongate device <NUM> is a flexible catheter that defines a channel or lumen <NUM>. The channel <NUM> can be sized and shaped to receive the medical instrument <NUM> (e.g., via a proximal end <NUM> and/or an instrument port (not shown) of the elongate device <NUM>) and facilitate delivery of the medical instrument <NUM> to a distal portion <NUM> of the elongate device <NUM>. As shown, the elongate device <NUM> is coupled to the instrument body <NUM>, which in turn is coupled and fixed relative to the instrument carriage <NUM> of the manipulator assembly <NUM>.

In operation, for example, the manipulator assembly <NUM> can control insertion motion (e.g., proximal and/or distal motion along an axis A) of the elongate device <NUM> into the patient <NUM> via a natural or surgically created anatomic orifice of the patient <NUM> to facilitate navigation of the elongate device <NUM> through anatomic passageways of the patient <NUM> and/or to facilitate delivery of the distal portion <NUM> of the elongate device <NUM> to a target location within the patient <NUM>. For example, the instrument carriage <NUM> and/or the insertion stage <NUM> may include actuators (not shown), such as servomotors, that facilitate control over motion of the instrument carriage <NUM> along the insertion stage <NUM>. Additionally, or alternatively, the manipulator assembly <NUM> in some embodiments can control motion of the distal portion <NUM> of the elongate device <NUM> in multiple directions, including yaw, pitch, and roll rotational directions (e.g., to navigate patient anatomy). To this end, the elongate device <NUM> may house or include cables, linkages, and/or other steering controls (not shown) that the manipulator assembly <NUM> can use to controllably bend the distal portion <NUM> of the elongate device <NUM>. For example, the elongate device <NUM> can house at least four cables that can be used by the manipulator assembly <NUM> to provide (i) independent "up-down" steering to control a pitch of the distal portion <NUM> of the elongate device <NUM> and (ii) independent "left-right" steering of the elongate device <NUM> to control a yaw of the distal portion <NUM> of the elongate device <NUM>.

The medical instrument <NUM> of the medical instrument system <NUM> can be used for medical procedures, such as for survey of anatomical passageways, surgery, biopsy, ablation, illumination, irrigation, and/or suction. Thus, the medical instrument <NUM> can include image capture probes, biopsy instruments, laser ablation fibers, and/or other surgical, diagnostic, and/or therapeutic tools. For example, the medical instrument <NUM> can include an endoscope having one or more image capture devices <NUM> positioned at a distal portion <NUM> of and/or at other locations along the medical instrument <NUM>. In these embodiments, the image capture device <NUM> can capture one or more real images or video (e.g., a sequence of one or more real navigation image frames) of anatomic passageways and/or other patient anatomy while the medical instrument <NUM> is within the anatomic region of the patient <NUM>.

As discussed above, the medical instrument <NUM> can be deployed into and/or be delivered to a target location within the patient <NUM> via the channel <NUM> defined by the elongate device <NUM>. In embodiments in which the medical instrument <NUM> includes an endoscope or other medical device having the image capture device <NUM> at the distal portion <NUM> of the medical instrument <NUM>, the image capture device <NUM> can be advanced to the distal portion <NUM> of the elongate device <NUM> before, during, and/or after the manipulator assembly <NUM> navigates the distal portion <NUM> of the elongate device <NUM> to a target location within the patient <NUM>. In these embodiments, the medical instrument <NUM> can be used as a survey instrument to capture real images and/or video of anatomic passageways and/or other patient anatomy, and/or to aid the operator (e.g., a physician) to navigate the distal portion <NUM> of the elongate device <NUM> through anatomic passageways to the target location.

As another example, after the manipulator assembly <NUM> positions the distal portion <NUM> of the elongate device <NUM> proximate a target location within the patient <NUM>, the medical instrument <NUM> can be advanced beyond the distal portion <NUM> of the elongate device <NUM> to perform a medical procedure at the target location. Continuing with the above example, after all or a portion of the medical procedure at the target location is complete, the medical instrument <NUM> can be retracted back into the elongate device <NUM> and, additionally or alternatively, be removed from the proximal end <NUM> of the elongate device <NUM> or from another instrument port (not shown) along the elongate device <NUM>.

In the example embodiment shown in <FIG>, the sensor system <NUM> of the medical instrument system <NUM> includes a shape sensor <NUM> and a position measuring device <NUM>. In some embodiments, the sensor system <NUM> includes all or a portion of the sensor system <NUM> of <FIG>. In these and other embodiments, the shape sensor <NUM> of the sensor system <NUM> includes an optical fiber extending within and aligned with the elongate device <NUM>. In one embodiment, the optical fiber of the shape sensor <NUM> has a diameter of approximately <NUM>. In other embodiments, the diameter of the optical fiber may be larger or smaller.

The optical fiber of the shape sensor <NUM> forms a fiber optic bend sensor that is used to determine a shape of the elongate device <NUM>. In some embodiments, optical fibers having Fiber Bragg Gratings (FBGs) can be used to provide strain measurements in structures in one or more dimensions. Various systems and methods for monitoring the shape and relative position of an optical fiber in three dimensions are described in further detail in <CIT>) (disclosing fiber optic position and shape sensing device and method relating thereto); <CIT>) (disclosing fiber-optic position and shape sensing device and method relating thereto); <CIT>), (disclosing fiber-optic position and/or shape sensing based on Rayleigh scatter); and <CIT>) (disclosing optical fiber bend sensor), which are all incorporated by reference herein in their entireties. In these and other embodiments, sensors of the present technology may employ other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and Fluorescence scattering. In these and still other embodiments, the shape of the elongate device <NUM> may be determined using other techniques. For example, a history of the pose of the distal portion <NUM> of the elongate device <NUM> can be used to reconstruct the shape of elongate device <NUM> over an interval of time.

In some embodiments, the shape sensor <NUM> is fixed at a proximal point <NUM> on the instrument body <NUM> of the medical instrument system <NUM>. In operation, for example, the shape sensor <NUM> measures a shape in the medical instrument reference frame (XM, YM, ZM) from the proximal point <NUM> to another point along the optical fiber, such as the distal portion <NUM> of the elongate device <NUM>. The proximal point <NUM> of the shape sensor <NUM> may be movable along with instrument body <NUM> but the location of proximal point <NUM> may be known (e.g., via a tracking sensor or other tracking device).

The position measuring device <NUM> of the sensor system <NUM> provides information about the position of the instrument body <NUM> as it moves along the insertion axis A on the insertion stage <NUM> of the manipulator assembly <NUM>. In some embodiments, the position measuring device <NUM> includes resolvers, encoders, potentiometers, and/or other sensors that determine the rotation and/or orientation of actuators (not shown) controlling the motion of the instrument carriage <NUM> of the manipulator assembly <NUM> and, consequently, the motion of the instrument body <NUM> of the medical instrument system <NUM>.

<FIG> is a schematic representation of a portion of the medical instrument system <NUM> of <FIG> extended within an anatomic region <NUM> (e.g., human lungs) of the patient <NUM> in accordance with various embodiments of the present technology. In particular, <FIG> illustrates the elongate device <NUM> of the medical instrument system <NUM> extending within branched anatomic passageways <NUM> of the anatomic region <NUM>. The anatomic passageways <NUM> include a trachea <NUM> and bronchial tubes <NUM>.

As shown in <FIG>, the elongate device <NUM> has a position, orientation, pose, and shape within the anatomic region <NUM>, all or a portion of which (in addition to or in lieu of movement, such as speed or velocity) can be captured by the shape sensor <NUM> and/or the position measuring device <NUM> of the sensor system <NUM> to survey the anatomic passageways <NUM> of the anatomic region <NUM>. In particular, the shape sensor <NUM> and/or the position measuring device <NUM> of the sensor system <NUM> can survey the anatomic passageways <NUM> by gathering positional information of the medical instrument system <NUM> within the anatomic region <NUM> in the medical instrument frame of reference (XM, YM, ZM). The positional information may be recorded as a set of two-dimensional or three-dimensional coordinate points. In the example of the anatomic region <NUM> being human lungs, the coordinate points may represent the locations of the distal portion <NUM> of the elongate device <NUM> and/or other portions of the elongate device <NUM> while the elongate device <NUM> is advanced through the trachea <NUM> and the bronchial tubes <NUM>. In these and other embodiments, the collection of coordinate points may represent the shape(s) of the elongate device <NUM> while the elongate device <NUM> is advanced through the anatomic region <NUM>. In these and other embodiments, the coordinate points may represent positional data of other portions (e.g., the medical instrument <NUM>) of the medical instrument system <NUM>.

The coordinate points may together form positional point cloud data. For example, <FIG> illustrates a plurality of coordinate points <NUM> forming positional point cloud data <NUM> representing a shape of the elongate device <NUM> while the elongate device <NUM> is within the anatomic region <NUM> (previously shown in <FIG>) in accordance with various embodiments of the present technology. In particular, the positional point cloud data <NUM> is generated from the union of all or a subset of the recorded coordinate points <NUM> of the shape sensor <NUM> (previously shown in <FIG> and <FIG>) and/or of the position measuring device <NUM> (previously shown in <FIG>) during a data acquisition period by the sensor system <NUM>. The positional point cloud data <NUM> can be generated by implementation of the disclosed example embodiments of the method <NUM>.

In some embodiments, a point cloud (e.g., the point cloud <NUM>) can include the union of all or a subset of coordinate points recorded by the sensor system <NUM> during an image capture period that spans multiple shapes, positions, orientations, and/or poses of the elongate device <NUM> within the anatomic region <NUM>. In these embodiments, the point cloud can include coordinate points captured by the sensor system <NUM> that represent multiple shapes of the elongate device <NUM> while the elongate device <NUM> is advanced or moved through patient anatomy during the image capture period. Additionally, or alternatively, because the configuration, including shape and location, of the elongate device <NUM> within the patient <NUM> may change during the image capture period due to anatomical motion, the point cloud in some embodiments can comprise a plurality of coordinate points <NUM> captured by the sensor system <NUM> that represent the shapes of the elongate device <NUM> as the elongate device <NUM> passively moves within the patient <NUM>. A point cloud of coordinate points captured by the sensor system <NUM> can be registered to different models or datasets of patient anatomy. For example, the positional point cloud data <NUM> can be used in registration with different models of the branched anatomic passageways <NUM>.

Referring again to <FIG>, the endoscopic imaging system <NUM> of the medical instrument system <NUM> includes one or more image capture devices configured to capture one or more images and/or video (e.g., a sequence of image frames) of anatomic passageways (e.g., the anatomic passageways <NUM> of <FIG>) and/or other patient anatomy while the elongate device <NUM> and/or the medical instrument <NUM> is within the patient <NUM>. For example, the endoscopic imaging system <NUM> can include (i) the image capture device <NUM> positioned at the distal portion <NUM> of the medical device <NUM> and/or (ii) one or more other image capture devices (not shown) positioned at other locations along the medical device <NUM>. In these and other embodiments, the endoscopic imaging system <NUM> can include one or more image capture devices (not shown) positioned at the distal portion <NUM> and/or other locations along the elongate device <NUM>. In some embodiments, the endoscopic imaging system <NUM> can include all or a portion of the endoscopic imaging system <NUM> of <FIG>.

As shown in <FIG>, the image capture device <NUM> of the medical instrument <NUM> is positioned at the distal portion <NUM> of the elongate device <NUM>. In this embodiment, the image capture device <NUM> surveys the anatomic passageways <NUM> by capturing real images of the anatomic passageways <NUM> while the elongate device <NUM> is advanced through the trachea <NUM> and the bronchial tubes <NUM> of the anatomic region <NUM>.

<FIG> is an example of an endoscopic video image frame <NUM> (e.g., a real image, such as a still image, an image frame of a video, etc.) of patient anatomy of the anatomic region <NUM> such as the anatomic passageways <NUM> of <FIG> captured using the image capture device <NUM> of the medical instrument system <NUM>. As shown, the real image <NUM> illustrates a branching point <NUM> of two bronchial tubes <NUM> (within the anatomic region <NUM> illustrated in <FIG>) from a viewpoint of the medical instrument <NUM>. In this example, the viewpoint is from the distal tip of the medical instrument <NUM>, such that the medical instrument <NUM> is not visible within the real image <NUM>. In other embodiments, the image capture device <NUM> can be positioned at another location along the medical instrument <NUM> and/or along the elongate device <NUM> such that the real image <NUM> is taken from another viewpoint of the medical instrument <NUM> and/or from another viewpoint of the elongate device <NUM>. A portion of the medical device <NUM> and/or of the elongate device <NUM> may be visible within the real image <NUM> depending on the positions of the medical instrument <NUM> and the elongate device <NUM> relative to one another.

Referring again to <FIG>, the real images captured by the endoscopic imaging system <NUM> can facilitate navigation of the distal portion <NUM> of the elongate device <NUM> through anatomic passageways (e.g., the anatomic passageways <NUM> of <FIG>) of the patient <NUM> and/or delivery of the distal portion <NUM> of the elongate device <NUM> to a target location within the patient <NUM>. In these and other embodiments, the real images captured by the endoscopic imaging system <NUM> can facilitate (i) navigation of the distal portion of the medical instrument <NUM> beyond the distal portion <NUM> of the elongate device <NUM>, (ii) delivery of the distal portion of the medical instrument <NUM> to a target location within the patient <NUM>, and/or (iii) visualization of patient anatomy during a medical procedure. In some embodiments, each real image captured by the endoscopic imaging system <NUM> can be associated with a time stamp and/or a position within an anatomic region of the patient <NUM>.

As illustrated in <FIG>, the imaging system <NUM> can be arranged near the patient <NUM> to obtain three-dimensional images of the patient <NUM>. In some embodiments, the imaging system <NUM> includes one or more imaging technologies, including CT, MRI, fluoroscopy, thermography, ultrasound, OCT, thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and/or the like. The imaging system <NUM> is configured to generate image data of the patient <NUM> before, during, and/or after the elongate device <NUM> is extended within the patient <NUM>. Thus, the imaging system <NUM> can be configured to capture preoperative, intraoperative, and/or postoperative three-dimensional images of the patient <NUM>. In these and other embodiments, the imaging system <NUM> may provide real-time or near real-time images of the patient <NUM>.

<FIG> illustrates such intra-operative image data <NUM> of a portion <NUM> of the anatomic region <NUM> of <FIG> captured during an image capture period by the imaging system <NUM> while the elongate device <NUM> of the medical instrument system <NUM> is extended within the anatomic region <NUM>. As shown, the image data <NUM> includes graphical elements <NUM> representing the elongate device <NUM> and graphical elements <NUM> representing the anatomical passageways <NUM> of the anatomic region <NUM>.

All or a portion of the graphical elements <NUM> and <NUM> of the image data <NUM> can be segmented and/or filtered to generate (i) a three-dimensional model of the anatomical passageways <NUM> of the portion <NUM> of the anatomic region <NUM>, and/or (ii) an image point cloud of the elongate device <NUM> within the anatomic region <NUM>. During the segmentation process, pixels or voxels generated from the image data <NUM> may be partitioned into segments or elements or be tagged to indicate that they share certain characteristics or computed properties such as color, density, intensity, and texture. The segments or elements may then be converted to a model and/or a point cloud. Additionally, or alternatively, the segments or elements can be used to locate (e.g., calculate) and/or define a center line running along the anatomical passageways <NUM>. The generated anatomic models and/or point clouds may be two or three-dimensional and may be generated in an image reference frame (XI, YI, ZI).

As discussed above with respect to <FIG>, the display system <NUM> of the medical system <NUM> can display various images or representations of patient anatomy and/or of the medical instrument system <NUM> based on data captured and/or generated by the positional sensor system <NUM>, by the endoscopic imaging system <NUM>, by the imaging system <NUM>, and/or by the virtual visualization system <NUM>. In various implementations, the images and/or representations can be utilized by the system to aid the operator <NUM> in conducting an image-guided medical procedure.

<FIG> is a schematic representation of an example display <NUM> produced by the display system <NUM> in accordance with various embodiments of the present technology. As shown, the display <NUM> includes a real navigational image <NUM>, a composite virtual navigational image <NUM> (also referred to as "composite virtual image <NUM>"), and a virtual navigational image <NUM>. The real navigational image <NUM> can be substantially the same as the real navigational image <NUM> of <FIG>. Thus, for example, the real navigational image <NUM> can be captured by the endoscopic imaging system <NUM> (<FIG>) and provided to the display system <NUM> to be presented on the display <NUM> in real-time or near real-time. In the illustrated embodiment, the real navigational image <NUM> illustrates real patient anatomy, e.g., such as a real image of a branching point or carina <NUM> at which an anatomic passageway branches into the two bronchial tubes <NUM> and/or anatomic passageways <NUM>) from a viewpoint oriented distally away from the distal portion <NUM> of the medical instrument <NUM>.

The composite virtual image <NUM> of <FIG> is displayed in the image reference frame (XI, YI, ZI) and includes an anatomic model <NUM> generated from image data (e.g., of the anatomic region <NUM> of <FIG>) captured by the imaging system <NUM>. The anatomic model <NUM> is registered (i.e., dynamically referenced) with a point cloud of coordinate points (e.g., the point cloud <NUM> of <FIG>) generated by the positional sensor system <NUM> to display a representation <NUM> within the anatomic model <NUM> of the tracked position, shape, pose, orientation, and/or movement of embodiments of the medical instrument system <NUM> (e.g., such as of the elongate device <NUM> of <FIG>) within the patient <NUM>. In some embodiments, the composite virtual image <NUM> is generated by the virtual visualization system <NUM> (<FIG>) of the control system <NUM> (<FIG>). Generating the composite virtual image <NUM> involves registering the image reference frame (XI, YI, ZI) with the surgical reference frame (XS, YS, ZS) and/or to the medical instrument reference frame (XM, YM, ZM). This registration may rotate, translate, or otherwise manipulate by rigid and/or non-rigid transforms coordinate points of the point cloud (e.g., the coordinate points <NUM> of the point cloud <NUM> of <FIG>) captured by the positional sensor system <NUM> to align the coordinate points with the anatomic model <NUM>. The registration between the image and surgical/instrument frames of reference may be achieved, for example, by using a point-based iterative closest point (ICP) technique as described in <CIT> and No. <CIT>, which are both incorporated by reference herein in their entireties. In other embodiments, the registration can be achieved using another point cloud registration technique.

Based at least in part on the registration, the virtual visualization system <NUM> can additionally or alternatively generate virtual navigational images (e.g., the virtual navigational image <NUM>) that include a virtual depiction of patient anatomy from a viewpoint of a virtual camera on the representation <NUM> of the medical instrument system <NUM> (<FIG>) within the anatomic model <NUM>. In the embodiment illustrated in <FIG> of the representation <NUM> of the medical instrument system <NUM> shown in <FIG>, the virtual camera is positioned at the distal portion <NUM> of representation <NUM> (e.g., of the medical instrument <NUM>) such that (i) the viewpoint of the virtual navigational image <NUM> (shown in <FIG>) is directed distally away from the distal portion <NUM> of the representation <NUM> and (ii) the representation <NUM> is not visible within the virtual navigational image <NUM>. In other embodiments, the virtual visualization system <NUM> can position the virtual camera (i) at another location along the representation <NUM> and/or (ii) in a different orientation such that the virtual navigational image <NUM> has a corresponding virtual viewpoint. In some embodiments, depending on the position and orientation of the virtual camera and the positions of the elongate device <NUM> and the medical instrument <NUM> relative to one another when within the patient <NUM>, the virtual visualization system <NUM> can render a virtual representation (not shown) of at least a portion of the elongate device <NUM> and/or of the medical instrument <NUM> into the virtual navigational image <NUM>.

In some embodiments, the virtual navigational image <NUM> can optionally include a navigation stripe <NUM>. In some implementations, for example, the navigation stripe <NUM> is used to aid the operator <NUM> to navigate the medical instrument system <NUM> through anatomic passageways to a target location within a patient <NUM>. For example, the navigation stripe <NUM> can illustrate a "best" path through patient anatomy for the operator <NUM> to follow to deliver the distal portions <NUM> and/or <NUM> of the medical instrument <NUM> and/or of the elongate device <NUM>, respectively, to a target location within an anatomic region. In some embodiments, the navigation stripe <NUM> can be aligned with a centerline of or another line along (e.g., the floor of) a corresponding anatomic passageway.

In some embodiments, the virtual visualization system <NUM> can place the virtual camera within the anatomic model <NUM> at a position and orientation corresponding to the position and orientation of the image capture device <NUM> within the patient <NUM>. As further shown in <FIG>, the virtual navigational image <NUM> illustrates virtual patient anatomy from substantially the same location at which the real navigational image <NUM> is captured by the image capture device <NUM>, e.g., showing carina <NUM> marking a branching point of two anatomic passageways <NUM> of the anatomic model <NUM>. Thus, the virtual navigational image <NUM> provides a rendered estimation of patient anatomy visible to the image capture device <NUM> at a given location within the anatomic region <NUM> of <FIG>. Because the virtual navigational image <NUM> is based on the registration of a point cloud generated by the positional sensor system <NUM> and image data captured by the imaging system <NUM>, the correspondence between the virtual navigational image <NUM> and the real navigational image <NUM> provides insight regarding the accuracy and/or efficiency of the registration and can be used to improve the registration, as described in greater detail below. Furthermore, the real navigational images (e.g., the real navigational images <NUM> and <NUM>) captured by the endoscopic imaging system <NUM> can (a) provide information regarding the position and orientation of the medical instrument system <NUM> within the patient <NUM>, (b) provide information regarding portions of an anatomic region actually visited by the medical instrument system, and/or (c) help identify patient anatomy (e.g., branching points or carinas of anatomic passageways) proximate the medical instrument system <NUM>, any one or more of which can be used to improve the accuracy and/or efficiency of the registration as described in greater detail below.

Several aspects of the present technology are set forth in the following examples. Although several aspects of the present technology are set forth in examples directed to systems, computer-readable mediums, and methods, any of these aspects of the present technology can similarly be set forth in examples directed to any of systems, computer-readable mediums, and methods in other embodiments.

In some embodiments in accordance with the present technology (example <NUM>), a system for mitigating oversampling of data points includes a medical device comprising a sensor, wherein the medical device is insertable in an anatomic passageway of a patient such that the sensor is operable to detect one or both of a position and a motion of the medical device when inserted in the anatomic passageway; and a computing device in communication with the medical device, the computing device comprising a processor, and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving data points detected by the sensor of the medical device, the received data points associated with one or both of the detected position and the detected motion of the medical device; analyzing a set of the received data points to determine a motion parameter associated with a movement or change in position of the sensor of the medical device in a region of the anatomic passageway, wherein the motion parameter includes a change of one or both of a translational motion and a rotational motion of the sensor; comparing the motion parameter to a threshold to determine whether to accept the set of data points when the motion parameter satisfies the threshold or to reject the set of data points when the motion parameter does not satisfy the threshold; and recording the accepted set of data points in a survey point cloud usable to register the medical device in an anatomic frame of reference space.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the sensor is configured to generate one or both of position sensor data and motion sensor data during data sampling of the anatomic passageway of the patient, wherein the position sensor data is associated with one or more positions of the medical device within the anatomic passageway, and wherein the motion sensor data is associated with one or both of the translational motion and the rotational motion of the medical device within the anatomic passageway.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the change of one or both of the translational motion and rotational motion of the sensor includes a change in one or more of (i) a roll value, (ii) a pitch value, or (iii) a yaw value.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the threshold includes a motion value or a range of motion values associated with the one or both of the translational motion and the rotational motion of the sensor.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM>, or <NUM> wherein the system is configured to perform further operations include generating a registration between the accepted set of data points in the survey point cloud and image data points derived from a previously-acquired image of the anatomic passageway of the patient.

In some embodiments in accordance with the present technology (example <NUM>), a system for mitigating oversampling of data points includes a medical device comprising a sensor, wherein the medical device is insertable in an anatomic passageway of a patient such that the sensor is operable to detect one or both of a position and a motion of the medical device when inserted in the anatomic passageway; and a computing device in communication with the medical device, the computing device comprising a processor, and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving data points detected by the sensor of the medical device, the received data points associated with a detected position of the medical device; analyzing the received data points to determine a distance parameter associated with a distance between a data point and one or more nearest neighbors of the data point; comparing the distance parameter to a threshold to determine whether to accept the data point among the received data points when the distance parameter satisfies the threshold or to reject the data point among the received data points when the distance parameter does not satisfy the threshold; and recording accepted data points in a survey point cloud usable to register the medical device in an anatomic frame of reference space.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the threshold includes a distance value or a range of distance values.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the received data points are initially recorded in the survey point cloud, and the recording the accepted data points in the survey point cloud includes deleting rejected data points that do not satisfy the threshold.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the system is configured to perform further operations that include storing the received data points in a temporary storage, and deleting rejected data points that do not satisfy the threshold from the temporary storage.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the system is configured to perform further operations include generating a registration between the recorded non-rejected data points in the survey point cloud and image data points derived from a previously-acquired image of the anatomic passageway of the patient.

In some embodiments in accordance with the present technology (example <NUM>), a system for mitigating oversampling of data points includes a medical device comprising a sensor, wherein the medical device is insertable in an anatomic passageway of a patient such that the sensor is operable to detect one or both of a position and a motion of the medical device when inserted in the anatomic passageway; and a computing device in communication with the medical device, the computing device comprising a processor, and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving data points detected by the sensor of the medical device, the received data points associated with a detected position of the medical device; analyzing the received data points to determine a density parameter associated with a density of one or more data points to nearest neighbors data points; comparing the density parameter to a threshold to determine whether to accept the one or more data points among the analyzed data points when the density parameter satisfies the threshold or to reject the one more data points when the density parameter does not satisfy the threshold; and recording accepted data points in a survey point cloud usable to register the medical device in an anatomic frame of reference space.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the threshold includes a density value or a range of density values.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the system is configured to perform further operations that include generating a registration between the recorded non-rejected data points in the survey point cloud and image data points derived from a previously-acquired image of the anatomic passageway of the patient.

In some embodiments in accordance with the present technology (example <NUM>), a system for mitigating oversampling of data points includes a medical device comprising a sensor, wherein the medical device is insertable in an anatomic passageway of a patient such that the sensor is operable to detect one or both of a position and a motion of the medical device when inserted in the anatomic passageway; and a computing device in communication with the medical device, the computing device comprising a processor, and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising: receiving data points detected by the sensor of the medical device, the received data points associated with a detected position of the medical device; analyzing the received data points to determine a density parameter associated with a density of one or more data points to nearest neighbors data points; comparing the density parameter to a threshold to determine whether to alter a weighting value of the one or more data points within the analyzed data points; when the density parameter meets the threshold, altering the weighting value of the one or more data points; and recording the data points to register the medical device in an anatomic frame of reference space.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the altering the weighting value includes normalizing the weighting values.

Example <NUM> includes the system of any of examples <NUM>, <NUM>, <NUM> or <NUM> wherein the anatomic passageway includes pulmonary airway passages of lungs.

The above detailed descriptions of embodiments of the technology are not intended to be exhaustive or to limit the technology to the precise form disclosed above. Although specific embodiments of, and examples for, the technology are described above for illustrative purposes, various equivalent modifications are possible within the scope of the technology, as those skilled in the relevant art will recognize. For example, while steps are presented in a given order, alternative embodiments can perform steps in a different order. Furthermore, the various embodiments described herein can also be combined to provide further embodiments.

Implementations of the subject matter and the functional operations described in this patent document can be implemented in various systems, digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Implementations of the subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible and non-transitory computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing unit" or "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.

Computer readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices.

From the foregoing, it will be appreciated that specific embodiments of the technology have been described herein for purposes of illustration, but well-known structures and functions have not been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the technology. To the extent any materials incorporated herein by reference conflict with the present disclosure, the present disclosure controls. Where the context permits, singular or plural terms can also include the plural or singular term, respectively. Moreover, unless the word "or" is expressly limited to mean only a single item exclusive from the other items in reference to a list of two or more items, then the use of "or" in such a list is to be interpreted as including (a) any single item in the list, (b) all of the items in the list, or (c) any combination of the items in the list. As used herein, the phrase "and/or" as in "A and/or B" refers to A alone, B alone, and both A and B. Where the context permits, singular or plural terms can also include the plural or singular term, respectively. Additionally, the terms "comprising," "including," "having" and "with" are used throughout to mean including at least the recited feature(s) such that any greater number of the same feature and/or additional types of other features are not precluded.

Furthermore, as used herein, the term "substantially" refers to the complete or nearly complete extent or degree of an action, characteristic, property, state, structure, item, or result.

Claim 1:
A system for mitigating oversampling of data points, the system comprising:
a medical device comprising a sensor, wherein the medical device is insertable in an anatomic passageway of a patient such that the sensor is operable to detect one or both of a position and a motion of the medical device when inserted in the anatomic passageway; and
a computing device in communication with the medical device, the computing device comprising a processor, and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the system to perform operations comprising:
receiving data points detected by the sensor of the medical device, the received data points associated with one or both of the detected position and the detected motion of the medical device;
analyzing a set of the received data points to determine a motion parameter associated with a movement or change in position of the sensor of the medical device in a region of the anatomic passageway, wherein the motion parameter includes a change of one or both of a translational motion and a rotational motion of the sensor;
comparing the motion parameter to a threshold to determine whether to accept the set of data points when the motion parameter satisfies the threshold or to reject the set of data points when the motion parameter does not satisfy the threshold; and
recording the accepted set of data points in a survey point cloud usable to register the medical device in an anatomic frame of reference space.