Patent Description:
However, in some situations it may be difficult for a surgeon to determine when a surgical instrument is in physical contact with patient tissue. For example, during a minimally-invasive surgical procedure a surgeon may be unable to determine from an endoscopic view of a surgical site when a surgical instrument is in physical contact with patient tissue at or near the surgical site.

<CIT> discusses an intubation system including a tracheal tube, a heat source coupled to the tracheal tube, and a temperature sensor disposable in a patient's trachea to detect a temperature within the patient's trachea. The heat source is adapted to generate heat when the tracheal tube is disposed in the airway of the patient. A temperature control system coupled to the heat source is adapted to monitor the detected temperature and to control generation of heat from the heat source based on the detected temperature.

<CIT> discusses a method wherein tissue ablation is carried out using an insertion tube having at least one ablation electrode, a first temperature sensor disposed on the distal portion sufficiently proximate the ablation electrode to detect heat generated during the ablation procedure, a second temperature sensor disposed on the distal portion sufficiently distant from the ablation electrode to be unable to detect the heat, and electronic logic circuitry linked to the first temperature sensor and the second temperature sensor and programmed to compute a temperature differential between respective temperatures sensed by the first temperature sensor and the second temperature sensor when conveying the electromagnetic energy.

<CIT> discusses systems, devices and methods for modulating targeted nerve fibers (e.g., hepatic neuromodulation) or other tissue. The systems may be configured to access tortuous anatomy of or adjacent hepatic vasculature. The systems may be configured to target nerves surrounding (e.g., within adventitia of or within perivascular space of) an artery or other blood vessel, such as the common hepatic artery.

<CIT> discusses systems for facilitating consistent radiometric tissue contact detection independent of orientation which comprises a medical instrument having an antenna positioned at a distal end, an energy delivery element positioned at a distal end of the medical instrument, a radiometer, an impedance transformation network positioned between the antenna and the radiometer, and a processor configured to receive an input signal from the radiometer and provide an output indicative of an amount of tissue contact based upon the input signal. The impedance transformation network may be tuned to provide a substantially uniform radiometric response independent of an orientation of the antenna with respect to the tissue as the antenna is brought into contact with the tissue of a subject.

The present invention provides a tissue contact detection system and associated method as defined in the appended independent claims. Optional features are defined in the appended dependent claims.

The following description presents a simplified summary of one or more aspects of the methods and systems described herein in order to provide a basic understanding of such aspects. Its sole purpose is to present some concepts of one or more aspects of the methods and systems described herein in a simplified form as a prelude to the more detailed description that is presented below.

An exemplary system may comprise a memory storing instructions and a processor communicatively coupled to the memory and configured to execute the instructions to track, over time during a surgical procedure, a temperature of a surgical instrument associated with a surgical system used for the surgical procedure; determine, based on the tracked temperature of the surgical instrument, that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount; determine, based on the determination that the temperature of the surgical instrument changes from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue; and perform, in response to the determination that the surgical instrument is in physical contact with patient tissue, a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue.

Another exemplary system may comprise a temperature sensor configured to detect, during a surgical session, a temperature of a surgical instrument included in a surgical system used for the surgical session; and a processor communicatively coupled to the temperature sensor and configured to execute instructions to track, over time during the surgical session, the temperature of the surgical instrument detected by the temperature sensor; determine, based on the tracked temperature of the surgical instrument, that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount; determine, based on the determination that the temperature of the surgical instrument changes from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue; and perform, in response to the determination that the surgical instrument is in physical contact with patient tissue, a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue.

An exemplary method may comprise tracking, by a tissue contact detection system over time during a surgical session, a temperature of a surgical instrument included in a surgical system used for the surgical session; determining, by the tissue contact detection system based on the tracked temperature of the surgical instrument, that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount; determining, by the tissue contact detection system based on the determination that the temperature of the surgical instrument changes from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue; and performing, by the tissue contact detection system in response to the determination that the surgical instrument is in physical contact with patient tissue, a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue.

Tissue contact detection systems and methods are described herein. As will be described below in more detail, a tissue contact detection system may track, over time during a surgical session, a temperature of a surgical instrument included in a surgical system used for the surgical session. Based on the tracked temperature of the surgical instrument, the tissue contact detection system may determine that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount. Based on the determination that the temperature of the surgical instrument changes from the first temperature to the second temperature, the tissue contact detection system may determine that the surgical instrument is in physical contact with patient tissue. In response to the determination that the surgical instrument is in physical contact with patient tissue, the tissue contact detection system may perform a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue.

To illustrate, during a minimally-invasive surgical procedure an endoscope included in a computer-assisted surgical system may be used to illuminate a surgical area within a patient and provide images of the surgical area for display and use by a surgeon in performing the surgical procedure. During the surgical procedure the endoscope may operate at a temperature (e.g., <NUM>) that is higher than a nominal temperature of patient tissue (e.g., <NUM>). When a shaft of the endoscope physically contacts the patient tissue, the patient tissue acts as a conductive heat sink and the temperature of the endoscope drops (e.g., decreases from <NUM> to <NUM>).

During the surgical session a tissue contact detection system may track, over time, the temperature of the endoscope and determine that the temperature of the endoscope changes (e.g., decreases from <NUM> to <NUM>). The tissue contact detection system may determine that this temperature change exceeds at least a predetermined amount (e.g., <NUM>) and thus determine that the endoscope is in physical contact with patient tissue. To mitigate the physical contact of the surgical instrument with the patient tissue the tissue contact detection system may present, or direct the surgical system to present, a notification (e.g., a visual notification, a warning message, an audible notification, etc.) of the physical contact of the surgical instrument with the patient tissue. The surgeon may respond to the notification by removing the endoscope from the physical contact with the patient tissue. Additionally or alternatively, the tissue contact detection system may decrease, or direct the surgical system to decrease, an intensity of illumination emitted from the endoscope and/or turn off auxiliary illumination (e.g., fluorescence excitation illumination).

The systems and methods described herein may provide various benefits. For example, the systems and methods described herein may reduce the risk of injury to patient tissue caused by prolonged physical contact with a surgical instrument. Additionally, the determination that the surgical instrument is in physical contact with patient tissue is highly reliable because it is based on a detected temperature change of the surgical instrument, which change is detectable, has low noise, and is not affected by electrical noise in the operating environment. Furthermore, the systems and methods described herein enable detection of physical contact by a surgical instrument with patient tissue located outside of an endoscopic view. These and other benefits of the systems and methods described herein will be made apparent in the description that follows.

<FIG> illustrates an exemplary tissue contact detection system <NUM> ("system <NUM>") that may be configured to determine that a surgical instrument is in physical contact with patient tissue. System <NUM> may be included in, implemented by, or connected to any surgical systems or other computing systems described herein. For example, system <NUM> may be implemented by a computer-assisted surgical system. As another example, system <NUM> may be implemented by a stand-alone computing system communicatively coupled to a computer-assisted surgical system.

As shown, system <NUM> includes, without limitation, a storage facility <NUM> and a processing facility <NUM> selectively and communicatively coupled to one another. Facilities <NUM> and <NUM> may each include or be implemented by hardware and/or software components (e.g., processors, memories, communication interfaces, instructions stored in memory for execution by the processors, etc.). For example, facilities <NUM> and <NUM> may be implemented by any component in a computer-assisted surgical system. In some examples, facilities <NUM> and <NUM> may be distributed between multiple devices and/or multiple locations as may serve a particular implementation.

Storage facility <NUM> may maintain (e.g., store) executable data used by processing facility <NUM> to perform any of the operations described herein. For example, storage facility <NUM> may store instructions <NUM> that may be executed by processing facility <NUM> to perform any of the operations described herein. Instructions <NUM> may be implemented by any suitable application, software, code, and/or other executable data instance. Storage facility <NUM> may also maintain any data received, generated, managed, used, and/or transmitted by processing facility <NUM>.

Processing facility <NUM> may be configured to perform (e.g., execute instructions <NUM> stored in storage facility <NUM> to perform) various operations associated with detecting that a surgical instrument is in physical contact with patient tissue. For example, processing facility <NUM> may be configured to track, over time during a surgical session, a temperature of a surgical instrument included in a surgical system used for the surgical session. Based on the tracked temperature of the surgical instrument, processing facility <NUM> may determine that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount. Based on the determination that the temperature of the surgical instrument changes from the first temperature to the second temperature, processing facility <NUM> may determine that the surgical instrument is in physical contact with patient tissue. In response to the determination that the surgical instrument is in physical contact with patient tissue, processing facility <NUM> may perform, or direct the surgical system to perform, a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue. These and other operations that may be performed by processing facility <NUM> are described herein. In the description that follows, any references to operations performed by system <NUM> may be understood to be performed by processing facility <NUM> of system <NUM>.

As mentioned, system <NUM> may be configured to track, over time during a surgical procedure, a temperature of a surgical instrument used in the surgical procedure. <FIG> illustrates a functional diagram of an exemplary environment in which a surgical procedure is performed. As shown, a surgical instrument <NUM> is used during a surgical procedure to perform one or more operations with respect to a patient. As a result, surgical instrument <NUM> may at various times be positioned near patient tissue <NUM> located at a surgical area associated with the patient. A surgical procedure may include any procedure in which manual and/or instrumental techniques are used to perform operations on a patient to investigate, diagnose, and/or treat a physical condition of the patient. For example, surgical instrument <NUM> may be used to perform a minimally invasive surgical procedure on tissue internal to a patient. In other examples, surgical instrument <NUM> may be used to perform an open surgical procedure, such as when part of the surgical area (e.g., tissue being operated on) is internal to the patient while another part of the surgical area (e.g., a space around the tissue where one or more surgical instruments may be disposed) is external to the patient.

Surgical instrument <NUM> may be implemented by any suitable surgical tool (e.g., a tool having tissue-interaction functions), medical tool, monitoring instrument (e.g., an endoscope), sensing instrument (e.g., a force-sensing surgical instrument), diagnostic instrument, or the like that may be used for a surgical procedure (e.g., a computer-assisted surgical procedure in which the surgical instrument is at least partially inserted into a patient and manipulated to perform a minimally-invasive surgical procedure on the patient). Surgical instrument <NUM> is associated with (e.g., connected to, integrated into, or implemented by) a surgical system <NUM>. In some examples surgical system <NUM> is connected to, integrated into, or implemented by a computer-assisted surgical system that utilizes robotic and/or teleoperation technology to control surgical instrument <NUM> to perform a surgical procedure (e.g., a minimally invasive surgical procedure). An exemplary computer-assisted surgical system is described below in more detail. In some examples system <NUM> is implemented entirely by surgical instrument <NUM> and/or surgical system <NUM>. As another example, system <NUM> may be implemented, in whole or in part, by a stand-alone computing system communicatively coupled to surgical instrument <NUM> and/or surgical system <NUM> (e.g., by way of a network).

Surgical instrument <NUM> includes a temperature sensor <NUM> configured to measure a temperature of surgical instrument <NUM>. Temperature sensor <NUM> may implemented by any suitable temperature sensing device, such as but not limited to a thermocouple, a resistance thermometer, a thermistor, a semiconductor-based temperature sensor, a fiber optic temperature probe, and the like. In some examples temperature sensor <NUM> may be connected to, integrated with, or included in system <NUM>.

Temperature sensor <NUM> may be positioned at any position on surgical instrument <NUM> as may suit a particular implementation. In some examples, as shown in <FIG>, temperature sensor <NUM> is positioned on a shaft wall <NUM> of surgical instrument <NUM>. Alternatively, temperature sensor <NUM> may be positioned on a distal end <NUM> of surgical instrument <NUM>. In some examples temperature sensor <NUM> is configured to measure a temperature of an external surface of surgical instrument <NUM>. Accordingly, temperature sensor <NUM> may be positioned in contact with the external surface of surgical instrument <NUM>. For instance, temperature sensor <NUM> may be positioned externally on shaft wall <NUM>. In alternative examples, temperature sensor <NUM> may be positioned internally within surgical instrument <NUM>. In some examples where temperature sensor <NUM> is not in contact with the external surface of surgical instrument <NUM>, the temperature value measured by temperature sensor <NUM> may be correlated to a temperature of the external surface of surgical instrument <NUM>.

As shown in <FIG>, temperature sensor <NUM> is integrated with surgical instrument <NUM>. For example, temperature sensor <NUM> is built into surgical instrument <NUM>, and temperature data generated by temperature sensor <NUM> is transmitted to surgical system <NUM> by way of surgical instrument <NUM> (e.g., by wiring located within surgical instrument <NUM>). Alternatively, temperature sensor <NUM> may be independent of surgical instrument <NUM>. <FIG> illustrates an exemplary embodiment in which a temperature sensor is separate from a surgical instrument. As shown, a temperature sensor <NUM> and wiring <NUM> are integrated into a sleeve <NUM>. Sleeve <NUM> is formed of a material having high thermal conductivity, such as a metal, a polymer with thermally-conductive additives, and the like. Sleeve <NUM> is configured to be positioned on a shaft of a surgical instrument (e.g., around shaft wall <NUM> of surgical instrument <NUM>). In some examples sleeve <NUM> is selectively removable from surgical instrument <NUM>. In this way temperature sensor <NUM> can be used with various different surgical instruments as the need may arise. In other examples sleeve <NUM> may be permanently secured to surgical instrument <NUM>. Wiring <NUM> is configured to transmit temperature data to a computing system (e.g., to surgical system <NUM>). Sleeve <NUM> enables tracking of the temperature of legacy surgical instruments that do not include an integrated temperature sensor. Additionally, sleeve <NUM> facilitates and improves the tracking of the temperature of surgical instruments having a low thermal conductivity (e.g., surgical instruments having a non-metallic shaft).

In some examples a temperature sensor may be integrated with surgical instrument <NUM> (e.g., on an exterior surface of shaft wall <NUM>), and a thermally-conductive sleeve (similar to sleeve <NUM>) is provided over surgical instrument <NUM> to improve the detection of tissue contact.

Referring again to <FIG>, surgical instrument <NUM> may include one or more other sensors (not shown), such as displacement transducers, orientational sensors, positional sensors, etc., for generating kinematics information (hereinafter "surgical instrument sensors"). Kinematics information may include information such as pose (e.g., position and/or orientation), movement (e.g., velocity, direction, acceleration, etc.), state (e.g., open, closed, stowed, etc.), and/or other attributes of surgical instrument <NUM>. System <NUM> and/or surgical system <NUM> may be configured to use the kinematics information to track (e.g., determine poses, movements, and/or states of) and/or control surgical instrument <NUM>. Surgical instrument <NUM> may also include other sensors configured to generate other information as may suit a particular implementation. As will be explained below in more detail, system <NUM> may also be configured to track, based on parameters sensed by the surgical instrument sensors, operations of surgical instrument <NUM>, and determine whether surgical instrument <NUM> is in physical contact with patient tissue <NUM> based on the tracked operations.

The foregoing embodiments have described use of a single temperature sensor. In alternative embodiments surgical instrument <NUM> and/or sleeve <NUM> may include a plurality of temperature sensors configured to measure the temperature of surgical instrument <NUM> at distinct locations on surgical instrument <NUM>. For example, as shown in <FIG> a surgical instrument <NUM> includes multiple temperature sensors <NUM> (e.g., temperature sensors <NUM>-<NUM>, <NUM>-<NUM>, and <NUM>-<NUM>) positioned at distinct locations along a shaft <NUM> of surgical instrument <NUM>. For instance, temperature sensor <NUM>-<NUM> is positioned at about <NUM> from a distal end of surgical instrument <NUM>, temperature sensor <NUM>-<NUM> is positioned at about <NUM> from a distal end of surgical instrument <NUM>, and temperature sensor <NUM>-<NUM> is positioned at about <NUM> from a distal end of surgical instrument <NUM>. While three temperature sensors <NUM> are shown in <FIG>, surgical instrument <NUM> may include fewer or more temperature sensors as may suit a particular implementation. Additionally, temperature sensors <NUM> may be positioned at any locations on surgical instrument <NUM> as may suit a particular implementation.

As mentioned above, system <NUM> may be configured to track the temperature of a surgical instrument (e.g., surgical instrument <NUM> and/or surgical instrument <NUM>). The tracking may be performed in any suitable way. In some examples the tracking includes collecting and/or storing temperature data representative of a measured temperature of the surgical instrument. For example, system <NUM> may collect temperature data from a temperature sensor (e.g., temperature sensor <NUM>, temperature sensor <NUM>, and/or a temperature sensor <NUM>) at regular intervals (e.g., every second, every <NUM> seconds, etc.) and store the collected temperature data (e.g., in storage facility <NUM>). In examples where the surgical instrument includes multiple temperature sensors, the tracking includes collecting and/or storing temperature data for each temperature sensor. In some examples the tracking may also include processing the collected temperature data, such as to remove noise, amplify signals, convert temperature data (e.g., convert a measured temperature value of an internal temperature of surgical instrument <NUM> to a temperature value of an external surface of surgical instrument <NUM>), generate a temperature profile, and the like.

Based on the tracked temperature of the surgical instrument, system <NUM> may detect that the surgical instrument is in physical contact with patient tissue. Various examples of detecting that the surgical instrument is in physical contact with patient tissue based on the tracked temperature of the surgical instrument will now be described with reference to <FIG> illustrates an exemplary temperature profile <NUM> of a surgical instrument based on the tracked temperature data. Temperature profile <NUM> plots the tracked temperature of the surgical instrument over time.

In some examples, system <NUM> is configured to determine that the surgical instrument is in physical contact with patient tissue when system <NUM> determines that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount. In some examples the first temperature and/or the second temperature are each an instantaneous temperature value. For instance, the first temperature may be a most recent maximum temperature, such as a maximum temperature occurring during a predetermined time interval (e.g., <NUM> seconds). Similarly, the second temperature may be a most recent minimum temperature, such as a minimum temperature occurring during the predetermined time interval. Alternatively, the first temperature and/or the second temperature may each be an average of multiple temperature values sampled during a predetermined time interval (e.g., <NUM> seconds). In yet further examples that will be explained below, the first temperature and/or the second temperature are each a steady-state temperature (e.g., a temperature of the surgical instrument when the temperature of the surgical instrument is in a steady-state).

The predetermined amount may be set in any suitable manner, and is generally set at a value indicative of tissue contact. For example, the predetermined amount may be a predetermined temperature difference value (e.g., <NUM>) or a percentage of the first temperature (e.g., <NUM>%). If the change in temperature is equal to at least the predetermined amount, system <NUM> determines that the surgical instrument is in physical contact with patient tissue. If, however, the change in temperature is less than the predetermined amount, system <NUM> does not determine that the surgical instrument is in physical contact with patient tissue.

To illustrate, as shown in <FIG> the temperature of the surgical instrument at time t<NUM> is approximately <NUM> and at time t<NUM> is approximately <NUM>. If the predetermined amount is a temperature difference value of <NUM>, then system <NUM> infers that the temperature drop is due to the surgical instrument being in physical contact with patient tissue because the change in temperature is <NUM>, which is greater than the predetermined temperature difference value of <NUM>. Thus, system <NUM> determines that the surgical instrument is in physical contact with patient tissue at time t<NUM>. Similarly, the temperature of the surgical instrument at time t<NUM> is approximately <NUM> and at time t<NUM> is approximately <NUM>. Accordingly, system <NUM> determines that the surgical instrument is in physical contact with patient tissue at time t<NUM> because the change in temperature is <NUM>, which is greater than the predetermined temperature difference value of <NUM>. On the other hand, the temperature of the surgical instrument at time t<NUM> is approximately <NUM> and at time t<NUM> is approximately <NUM>. Accordingly, system <NUM> does not determine that the surgical instrument is in physical contact with patient tissue at time t<NUM> because the change in temperature is <NUM>, which is less than the predetermined temperature difference value of <NUM>.

In some examples system <NUM> determines that the surgical instrument is in physical contact with patient tissue only when the temperature of the surgical instrument changes in a direction toward the temperature of the patient tissue (e.g., nominal body temperature of <NUM>). In other words, if the first temperature of the surgical instrument is greater than the temperature of the patient tissue, then only a decrease in temperature of the surgical instrument (i.e., the second temperature of the surgical instrument is less than the first temperature) will indicate physical contact with patient tissue. On the other hand, if the first temperature of the surgical instrument is less than the temperature of the patient tissue, then only an increase in temperature of the surgical instrument (i.e., the second temperature of the surgical instrument is greater than the first temperature) will indicate physical contact with patient tissue.

In additional or alternative examples system <NUM> may determine that the surgical instrument is in physical contact with patient tissue based on a determination that the first temperature and/or the second temperature of the surgical instrument is in a steady-state. System <NUM> may determine that the temperature of the surgical instrument is in a steady-state in any suitable way. In some examples the temperature of the surgical instrument is in a steady-state when the temperature of the surgical instrument does not vary by more than a predetermined steady-state amount over a predetermined steady-state time period. The predetermined steady-state amount may be any suitable amount and may be specified as a temperature difference value (e.g., ± <NUM>) or a percentage (e.g., ± <NUM>%). The predetermined steady-state time period may be any suitable time period (e.g., <NUM> seconds).

To illustrate, as shown in <FIG> the surgical instrument is initialized (e.g., connected to a surgical system, docked, turned on, first operated, etc.) at time t<NUM> and operates in an initialization phase from time t<NUM> to time t<NUM>. During the initialization phase the temperature of the surgical instrument ramps-up and fluctuates (e.g., due to manual handling of the surgical instrument, navigation of the surgical instrument to a surgical area within the patient, activation of optical illumination, movement of the surgical instrument, etc.). Thus, the temperature of the surgical instrument does not reach a steady-state between time t<NUM> and time t<NUM>. Accordingly, although the temperature change from time t<NUM> to time t<NUM> may be greater than the predetermined amount, system <NUM> does not determine that the surgical instrument is in physical contact with patient tissue because the temperature of the surgical instrument has not reached a steady state. However, the temperature of the surgical instrument reaches a steady-state between time t<NUM> to time t<NUM> and between time t<NUM> to time t<NUM>. Thus, system <NUM> may determine that the surgical instrument is in physical contact with patient tissue between time t<NUM> to time t<NUM> if the second temperature varies from the first temperature by at least the predetermined amount.

In some examples the determination that the surgical instrument is in physical contact with patient tissue is conditioned at least on the second temperature of the surgical instrument being in a steady-state. For instance, system <NUM> does not determine that the surgical instrument is in physical contact with patient tissue at any time between time t<NUM> and time t<NUM> because the temperature of the surgical instrument is not in a steady-state. On the other hand, provided that the second temperature varies from the first temperature by at least the predetermined amount, system <NUM> determines that the surgical instrument is in physical contact with patient tissue between time t<NUM> and t<NUM> because the temperature of the surgical instrument is in a steady-state from time t<NUM> to time t<NUM>.

In additional or alternative examples the determination that the surgical instrument is in physical contact with patient tissue is conditioned at least on the first temperature of the surgical instrument being in a steady-state. For instance, system <NUM> does not determine that the surgical instrument is in physical contact with patient tissue at any time between time t<NUM> and time t<NUM> because the temperature of the surgical instrument is not in a steady-state. On the other hand, provided that the second temperature varies from the first temperature by at least the predetermined amount, system <NUM> determines that the surgical instrument is in physical contact with patient tissue between time t<NUM> and t<NUM> because the temperature of the surgical instrument is in a steady-state from time t<NUM> to time t<NUM>.

In some examples the determination that the surgical instrument is in physical contact with patient tissue is based on a determination of a non-contact state temperature of the surgical instrument. A non-contact state temperature is a normal operating temperature of the surgical instrument when the surgical instrument is not in contact with patient tissue. The non-contact state temperature of the surgical instrument may be determined in any suitable way. In some examples the non-contact state temperature may be the first instance of a steady-state temperature during use of the surgical instrument (e.g., the temperature from time t<NUM> to time t<NUM> in <FIG>). Alternatively, the non-contact state temperature may be predetermined (e.g., based on empirical analysis).

As mentioned, system <NUM> is configured to determine that the surgical instrument is in physical contact with patient tissue when system <NUM> determines that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount. In some examples the non-contact state temperature is set as the first temperature for all real-time determinations of tissue contact during the surgical procedure. To illustrate, system <NUM> may determine that the first instance of a steady-state temperature occurs from time ts to time t<NUM> and thus determine that the non-contact state temperature of the surgical instrument is the temperature at time t<NUM> to t<NUM>, i.e., approximately <NUM>. System <NUM> may set the first temperature to the non-contact state temperature (e.g., <NUM>). The temperature of the surgical instrument changes from approximately <NUM> at time t<NUM> to approximately <NUM> at time t<NUM>. If the predetermined amount is set at <NUM>, the temperature change from time t<NUM> to time t<NUM> is <NUM>, which is less than the predetermined amount. However, system <NUM> determines that the surgical instrument is in physical contact with patient tissue at time t<NUM> because the temperature of the surgical instrument changes from the first temperature (the non-contact state temperature, i.e., <NUM>) to a second temperature (approximately <NUM> at time t<NUM>) that varies from the first temperature by <NUM>, which is greater than the predetermined amount. In this way physical contact with tissue may be detected even though the surgical instrument may not have completely returned to its non-contact state temperature after a prior decrease in temperature.

In additional or alternative embodiments, system <NUM> determines whether the surgical instrument is in physical contact with patient tissue based on a detected rate of temperature change. The rate of temperature change may indicate whether the surgical instrument is in physical contact with patient tissue or whether the temperature change is due to some other cause. For example, the surgical instrument may change temperature faster when in physical contact with patient tissue than when in contact with another surgical instrument operating at a temperature above the patient tissue temperature.

System <NUM> may determine the rate of temperature change in any suitable way. In some examples system <NUM> determines an average rate of temperature change over a period of time, such as over a regular period of time (e.g., every <NUM> seconds), over a period of time between a most recent maximum temperature and a most recent minimum temperature (e.g., between time t<NUM> and time t<NUM>), over a period of time between successive steady-state temperatures (e.g., between time t<NUM> and time t<NUM>), and the like. System <NUM> may determine that the surgical instrument is in physical contact with patient tissue if the measured rate of temperature change is greater than or equal to a rate change minimum value (e.g., <NUM>/minute), less than or equal to a rate change maximum value (e.g., <NUM>/minute), or within a particular rate change range (e.g., between <NUM>/minute and <NUM>/minute). The rate change minimum value, maximum value, and/or range may be predetermined (e.g., based on empirical analyses) or it may be set based on the detected non-contact state temperature of the surgical instrument.

To illustrate, as shown in <FIG> the temperature change from time t<NUM> to time t<NUM> may be due to initialization of the surgical instrument and/or navigation of the surgical instrument to the surgical area within the patient, while the temperature change from time t<NUM> to time t<NUM> may be due to contact with patient tissue. If the rate change minimum value is <NUM>/minute, the rate change maximum value is <NUM>/minute, the rate of temperature change from time t<NUM> to time t<NUM> is <NUM>/minute, and the rate of temperature change from time t<NUM> to time t<NUM> is <NUM>/minute, system <NUM> determines that the surgical instrument is in physical contact with patient tissue at time t<NUM> but not at time t<NUM> because only the rate of temperature change from time t<NUM> to time t<NUM> falls between the rate change minimum and maximum values.

Alternatively to tracking a rate of temperature change, system <NUM> may determine that the surgical instrument is in physical contact with patient tissue when system <NUM> determines that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount within a predetermined time interval. For instance, system <NUM> may determine that the surgical instrument is in physical contact with patient tissue at time t<NUM> because the temperature change of the surgical instrument from time t<NUM> to time t<NUM> exceeds the predetermined amount of temperature change and the amount of time elapsed from time t<NUM> to time t<NUM> is within the predetermined time interval (e.g., within <NUM> seconds).

In additional or alternative embodiments, system <NUM> determines that the surgical instrument is in physical contact with patient tissue if the second temperature is within a contact-state range of a contact-state temperature. The contact-state temperature of the surgical instrument refers to a steady-state temperature of the surgical instrument when the surgical instrument is in contact with patient tissue. Since a temperature of patient tissue is generally constant (approximately <NUM>), the contact-state temperature of the surgical instrument may be measured empirically or calculated or based on the non-contact state temperature of the surgical instrument.

The contact-state range refers to a temperature range within which the temperature of the surgical instrument may vary while the surgical instrument remains in contact with patient tissue, and may be represented as a temperature value (e.g., ± <NUM>) or a percentage (e.g., ± <NUM>%). For instance, if the non-contact state temperature of the surgical instrument is <NUM> (see <FIG>), system <NUM> may determine (e.g., calculate or determine from pre-stored data) that the contact state temperature of the surgical instrument is <NUM>. Accordingly, system <NUM> may determine that the surgical instrument is in physical contact with patient tissue if the measured temperature of the surgical instrument is within the contact-state range, e.g., <NUM> ± <NUM> or <NUM> ± <NUM>%, as is the case from time t<NUM> to t<NUM> but not from time t<NUM> to t<NUM>.

In some cases the non-contact state temperature of the surgical instrument may not be high (or low) enough to generate an easily detectable temperature change when the surgical instrument physically contacts patient tissue. Accordingly, system <NUM> may be configured to maintain the non-contact state temperature of the surgical instrument at a predetermined level (e.g., at <NUM>), or at such a level that the non-contact state temperature varies from the tissue temperature by at least a predetermined non-contact state difference amount (e.g., <NUM>), such as by passively applying heat to or removing heat from the surgical instrument. The non-contact state temperature of the surgical instrument may be increased or decreased such that the non-contact state temperature of the surgical instrument varies from the temperature of the patient tissue by at least the predetermined non-contact state difference amount. In some examples system <NUM> is configured to adjust the non-contact state temperature of the surgical instrument in response to a determination that the non-contact state temperature of the surgical instrument varies from the temperature of patient tissue by less than the predetermined non-contact state difference amount. By adjusting the non-contact state temperature of surgical system as just described, system <NUM> may ensure that physical contact with patient tissue can be accurately detected.

In the foregoing embodiments system <NUM> determines that the surgical instrument physically contacts patient tissue based on a tracked temperature of the surgical instrument. In the embodiments described above the temperature of the surgical instrument is tracked based on the temperature value measured by a temperature sensor (e.g., temperature sensor <NUM> or temperature sensor <NUM>) on the surgical instrument. In embodiments where the surgical instrument includes multiple temperature sensors (see, e.g., <FIG>), the temperature of the surgical instrument is tracked at each location on the surgical instrument where a temperature sensor is located. Accordingly, in some examples system <NUM> may determine that the surgical instrument physically contacts patient tissue if system <NUM> determines that, at any one or more of the locations on the surgical instrument, the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least the predetermined amount. Moreover, system <NUM> may determine, based on the tracked temperature at each location on the surgical instrument, the particular location on the surgical instrument that is in physical contact with patient tissue. In alternative examples, system <NUM> may determine that the surgical instrument physically contacts patient tissue only if system <NUM> determines that, at two or more of the locations on the surgical instrument, the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least the predetermined amount.

In embodiments in which the surgical instrument includes multiple temperature sensors, the sensitivity of the tissue contact detection may also be dependent on the location of the temperature sensors on the surgical instrument. For instance, the predetermined amount may be set to be larger for temperature data obtained from a temperature sensor located close to the distal end of an endoscope (e.g., at <NUM> from the distal end) than for a temperature sensor located farther (e.g., <NUM>) from the distal end.

In the foregoing embodiments system <NUM> determines that the surgical instrument physically contacts patient tissue based on a tracked temperature of the surgical instrument. In other embodiments, system <NUM> may determine that the surgical instrument physically contacts patient tissue based also on tracked operations of the surgical instrument. Tracked operations of the surgical instrument may be used by system <NUM> in conjunction with the tracked temperature of the surgical instrument to better determine when the surgical instrument physically contacts patient tissue.

As used herein, operations of the surgical instrument may include any mechanical, electrical, optical, hardware, and/or software-based operations as may serve a particular implementation. For example, operations of the surgical instrument may include movement of the surgical instrument, operation of a functional feature of the surgical instrument (e.g., energizing a cautery instrument, opening and closing forceps or scissors, firing a stapling instrument, activating fluorescence excitation illumination, etc.), adjustment of a surgical instrument setting (e.g., adjusting an exposure level or a zoom level of an endoscope, etc.), detection of reflected illumination by an image sensor, detection of a system fault or error (e.g., detection of a collision of the surgical instrument with another surgical instrument, etc.), generation of a fault code, and the like.

System <NUM> may track operations of the surgical instrument in any suitable way. In some examples the tracking includes collecting surgical session data representative of operations of the surgical instrument during a surgical session and/or processing the surgical session data (e.g., to reduce noise, sort and classify events, apply timestamps, etc.). The surgical session data may be generated by system <NUM>, the surgical instrument, a surgical system associated with the surgical instrument, and/or by any other device associated with the surgical instrument as may serve a particular implementation. Surgical session data generated during a surgical session may include various types of data. For example, surgical session data generated during a surgical session may include kinematic data, image data, sensor data, surgical instrument data, and/or any other type of data as may serve a particular implementation.

Kinematic data may be representative of a pose of the surgical instrument, movement of the surgical instrument, and any other positional and/or motion-based information as may suit a particular implementation. Image data may be representative of one or more images captured by the surgical instrument. For example, image data may be representative of one or more still images and/or video captured by an imaging device (e.g., a stereoscopic endoscope). Sensor data may include any data generated by surgical instrument sensors included in or associated with the surgical instrument. Sensor data may be representative of any sensed parameter as may serve a particular implementation. In some examples, certain kinematic data and image data may be generated by and/or based on parameters sensed by surgical system sensors. Accordingly, sensor data may include such kinematic data and image data. Surgical instrument data may include any other data generated or maintained by the surgical instrument, such as an identification ("ID") of the surgical instrument, an operational state of the surgical instrument (e.g., open, closed, electrically charged, idle, etc.), a fault code of the surgical instrument, and the like.

As mentioned, system <NUM> may be configured to determine that the surgical instrument is in physical contact with patient tissue further based on the tracked operations of the surgical instrument. For example, as shown in <FIG> system <NUM> may determine that the temperature of the surgical instrument changes from a first temperature at time t<NUM> to a second temperature at time t<NUM> that varies from the first temperature by more than a predetermined amount. However, system <NUM> may further determine, based on the tracked operations of the surgical instrument, that the surgical instrument is an endoscope and that an illumination source that provides illumination via the surgical instrument is turned off at time t<NUM>. Accordingly, system <NUM> may infer that the decrease in temperature of the surgical instrument is due to the decrease in illumination output by the surgical instrument. Accordingly, system <NUM> does not determine that the surgical instrument is in physical contact with patient tissue at time t<NUM>.

As another example, system <NUM> may determine that the temperature of the surgical instrument changes from a first temperature at time t<NUM> to a second temperature at time t<NUM> that varies from the first temperature by more than a predetermined amount. Additionally, system <NUM> may further determine that the surgical instrument has moved immediately prior to the change of the temperature (e.g., at or immediately prior to time t<NUM>). Accordingly, system <NUM> may determine that, because the change in temperature has occurred immediately after movement of the surgical instrument, the surgical instrument is in physical contact with patient tissue at time t<NUM>.

As mentioned, system <NUM> may determine, based on the tracked temperature of the surgical instrument, that the surgical instrument is in physical contact with patient tissue when the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount. In some examples the predetermined amount may vary based on the tracked operations of the surgical instrument. For instance, the predetermined amount may be decreased when the tracked operations of the surgical instrument suggest that physical contact with patient tissue is likely (e.g., movement prior to the change in temperature, an increased intensity of reflected illumination detected by an imaging device, increased saturation in the imaging device, surgical instrument sensor data indicating contact with another object, etc.). Similarly, the predetermined amount may be increased when the tracked operations of the surgical instrument indicate that physical contact with patient tissue is not likely (e.g., no movement of the surgical instrument, a decreased intensity of reflected illumination detected by an imaging device, decreased brightness of an imaged captured by the imaging device, detection of a collision of the surgical instrument with another surgical instrument, etc.).

In the foregoing embodiments the surgical instrument has been described as having a non-contact state temperature above the patient tissue temperature, and thus system <NUM> detects physical contact with patient tissue when the temperature of the surgical instrument decreases. In like manner the systems and methods described herein also apply to a surgical instrument that has a non-contact state temperature below the patient tissue temperature. In such cases system <NUM> detects physical contact with patient tissue when the temperature of the surgical instrument increases.

The preceding examples provide various exemplary methods and criteria for determining, based on a tracked temperature of a surgical instrument and/or based on tracked operations of the surgical instrument, that the surgical instrument is in physical contact with patient tissue. However, system <NUM> is not limited to the specific methods and criteria described in the examples above, but may include variations and modifications of the methods and criteria as may suit a particular implementation. Additionally, system <NUM> may utilize any combination or sub-combination of methods and criteria for determining that the surgical instrument is in physical contact with patient tissue. System <NUM> may combine various methods and criteria in any suitable way that facilitates system <NUM> discerning when the surgical instrument is in physical contact with patient tissue.

As mentioned above, system <NUM> may be configured to perform, in response to a determination that the surgical instrument is in physical contact with patient tissue, a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue. Mitigation operations include any operations configured to reduce the likelihood of injury to patient tissue caused by the physical contact of the surgical instrument with the patient tissue. In some examples system <NUM> performs a mitigation operation by directing the surgical instrument and/or a surgical system associated with the surgical instrument to perform another mitigation operation.

In some examples mitigation operations include any suitable operations configured to notify an operator of the surgical instrument that the surgical instrument is in physical contact with patient tissue. For example, system <NUM> may direct surgical system <NUM> to present a notification that the surgical instrument is in physical contact with patient tissue. The notification may be in any format, such as but not limited to visual (e.g., a warning light, a warning icon displayed on a graphical user interface viewed by a user, a message displayed on the graphical user interface, etc.), audio (e.g., a warning tone, an audible warning message, etc.), and haptic (e.g., vibration of a manual controller of the surgical instrument, etc.). Additionally, the notification may be provided by any suitable device, including but not limited to a display device included in surgical system <NUM> (e.g., a stereo viewer in a surgeon console, an auxiliary display device, a mobile device associated with a user, etc.), a speaker (e.g., a speaker included in the surgeon console, a speaker on a mobile device, etc.), a manual control device (e.g., a controller for manually controlling operation of the surgical instrument, etc.), and the like.

<FIG> illustrates an exemplary notification that may be presented to a user (e.g., a surgeon, a technician, etc.). As shown, an image <NUM> presented by an image display system included in a surgical system (e.g., surgical system <NUM>) shows an endoscopic view of a surgical area as captured by an endoscope. Image <NUM> includes a message bubble <NUM> superimposed on the endoscopic view of the surgical area. As shown, message bubble <NUM> includes a message that states: "Endoscope is in contact with tissue. " It will be appreciated that message bubble <NUM> may be positioned at any location and may provide any information as may suit a particular implementation. In some examples message bubble <NUM> may include a selectable option (not shown) configured to dismiss or minimize message bubble <NUM> upon selection by the user. As is evident from <FIG>, system <NUM> is configured to detect when a surgical instrument (e.g., an endoscope) included in a surgical system is in physical contact with patient tissue located outside of an endoscopic field of view during a minimally-invasive surgical procedure.

In situations where multiple surgical instruments are located at the surgical area, the notification may identify the particular surgical instrument that is in physical contact with patient tissue. In some examples a warning message may name the particular surgical instrument that is in physical contact with patient tissue (see, e.g., <FIG>). Additionally or alternatively, an endoscopic image may graphically identify (e.g., highlight, point to, mark with an icon, etc.) the particular surgical instrument. The particular surgical instrument that is in physical contact with patient tissue may be identified in any suitable way. In some examples system <NUM> may obtain (e.g., from the surgical instrument that is in physical contact with patient tissue) an ID of the surgical instrument and provide the ID of the surgical instrument to the surgical system. The surgical system may track the location of the surgical instruments located at the surgical area (e.g., utilizing kinematic data, marker-based computer vision tracking, image object recognition, etc.) and, based on the ID of the surgical instrument, identify the particular surgical instrument that is in physical contact with patient tissue.

<FIG> illustrates an exemplary notification that may be presented to a user to identify the particular surgical instrument that is in physical contact with patient tissue. As shown, an image <NUM> presented by way of an image display system included in a surgical system (e.g., surgical system <NUM>) shows an endoscopic view of a surgical area as captured by an endoscope, including a view of an ultrasound instrument <NUM> and a scissors instrument <NUM>. When system <NUM> determines that ultrasound instrument <NUM> is in physical contact with patient tissue a message bubble <NUM> is displayed on the endoscopic view of the surgical area and points to ultrasound instrument <NUM>. As shown in <FIG>, message bubble <NUM> includes a message that states: Instrument is in contact with tissue. " It will be appreciated that message bubble <NUM> may be positioned at any location and may provide any information as may suit a particular implementation. With this configuration a user may quickly identify which surgical instrument is in physical contact with patient tissue and take corrective action. Additionally, since ultrasound instrument <NUM> must be in physical contact with patient tissue in order to capture and generate ultrasound images, the features described herein may help the user to know when ultrasound instrument <NUM> is in physical contact with patient tissue.

In examples where system <NUM> identifies a particular location on the surgical instrument that is in physical contact with patient tissue, the notification may show or identify the location on the surgical instrument that is in physical contact with patient tissue. The particular location on the surgical instrument that is in physical contact with patient tissue may be identified in any suitable way. As mentioned above, system <NUM> may use the tracked temperature of each location on the surgical instrument to determine the particular location on the surgical instrument that is in physical contact with patient tissue. System <NUM> may provide to a surgical system (e.g., surgical system <NUM>) information identifying the particular location. As also mentioned above, the surgical system may track the location of the surgical instrument within the surgical area. Based on the tracked location of the surgical instrument and the particular location on the surgical instrument, the surgical system may configure the notification to identify the particular location on the surgical instrument that is in physical contact with patient tissue.

<FIG> illustrates an exemplary notification that may be presented to a user to identify the particular location on the surgical instrument that is in physical contact with patient tissue. As shown, an image <NUM> presented by way of an image display system included in a surgical system (e.g., surgical system <NUM>) shows an endoscopic view of a surgical area as captured by an endoscope, including a view of an ultrasound instrument <NUM> located at the surgical area. When system <NUM> determines that a distal end region of ultrasound instrument <NUM> is in physical contact with patient tissue a message bubble <NUM> is displayed on the endoscopic view of the surgical area and points to the distal end region of ultrasound instrument <NUM>. As shown, a message in message bubble <NUM> states: "Tissue contact detected here. " It will be appreciated that message bubble <NUM> may be positioned at any location and may provide any information as may suit a particular implementation.

Additionally or alternatively to notifications, mitigation operations may include any operations configured to decrease the temperature of the surgical instrument and thereby reduce the risk of injury to the patient tissue. To this end, system <NUM> may adjust, or direct the surgical instrument and/or a surgical system associated with the surgical instrument to adjust, the operation of the surgical instrument. For example, where surgical instrument <NUM> is implemented by an imaging device (e.g., an endoscope), system <NUM> may direct surgical system <NUM> to adjust the output of illumination provided to the surgical area by way of surgical instrument <NUM>. The output of illumination may be adjusted, for example, by decreasing an intensity of light provided to the surgical instrument, intermittently discontinuing the output of light provided to the surgical instrument, or turning off a light source (e.g., fluorescence excitation illumination source, a blue light source, etc.). As another example, system <NUM> may direct surgical system <NUM> to adjust the operation of an imaging device (e.g., decrease resolution, decrease frame rate, etc.) to thereby reduce heat generated by the imaging device. In other examples system <NUM> may direct surgical system <NUM> to decrease the amount of passive heat applied to the surgical instrument, increase cooling of the surgical instrument, decrease an amount of energy (e.g., cauterizing energy) provided by way of the surgical instrument, and the like.

<FIG> illustrates an exemplary method <NUM> of performing mitigation operations. While <FIG> illustrates exemplary operations according to one embodiment, other embodiments may omit, add to, reorder, and/or modify any of the operations shown in <FIG>.

In operation <NUM>, system <NUM> determines whether a surgical instrument is in physical contact with patient tissue. Operation <NUM> may be performed in any of the ways described herein. If system <NUM> does not determine that the surgical instrument is in physical contact with patient tissue, system <NUM> returns to the start and continues monitoring for tissue contact. However, if system <NUM> determines that the surgical instrument is in physical contact with patient tissue, system <NUM> proceeds to operation <NUM>.

In operation <NUM> system <NUM> checks whether a predetermined amount of time has elapsed since the tissue contact was first detected. The predetermined amount of time may be any time period as may suit a particular implementation (e.g., <NUM> minutes). In some examples the predetermined amount of time is set by system <NUM>. Additionally or alternatively, the predetermined amount of time is configurable by a user. If system <NUM> determines that the predetermined amount of time has not elapsed since the tissue contact was first detected, system <NUM> proceeds to operation <NUM>. In operation <NUM> system <NUM> presents, or directs the surgical system to present, a notification that the surgical instrument is in physical contact with patient tissue. Operation <NUM> may be performed in any of the ways described herein.

If, however, system <NUM> determines that the predetermined amount of time has elapsed since the tissue contact was first detected, system <NUM> proceeds to operation <NUM>. In operation <NUM> system <NUM> adjusts, or directs the surgical system to adjust, an operation of the surgical instrument to mitigate the physical contact of the surgical instrument with the patient tissue. Operation <NUM> may be performed in any of the ways described herein.

After operation <NUM> and operation <NUM> system <NUM> returns to operation <NUM> to determine if the surgical instrument is still in physical contact with patient tissue. If system <NUM> determines that the surgical instrument is no longer in physical contact with patient tissue system <NUM> returns to the start to monitor for physical contact with patient tissue. System <NUM> may also dismiss, or direct the surgical system to dismiss, the notification and/or resume, or direct the surgical system to resume, normal operation of the surgical instrument.

In method <NUM> a notification that the surgical instrument is in physical contact with patient tissue is presented, but operation of the surgical instrument is not adjusted unless a predetermined amount of time has elapsed. In this way system <NUM> may allow a user to perform an action to mitigate the tissue contact before the surgical system automatically performs operations to mitigate the tissue contact. However, the processing for performing mitigation operations is not limited to the foregoing sequence. In alternative embodiments system <NUM> may be configured to present, or direct the surgical system to present, the notification only after the surgical instrument has remained in physical contact with the patient tissue for another predetermined amount of time (e.g., three minutes). In such embodiments it may be presumed that physical contact with patient tissue is only temporary and that the surgical instrument will likely be moved before the patient tissue is injured. In other alternative embodiments system <NUM> may be configured to provide, or direct the surgical system to provide, the notification and adjust, or direct the surgical system to adjust, the operation of the surgical instrument simultaneously. In yet further embodiments any predetermined amounts of time and/or the order of performing mitigation operations may be set based on the type of surgical instrument. For instance, for an endoscope that operates at a higher non-contact state temperature than a scissors instrument, the predetermined amount of time considered in operation <NUM> may be set shorter for the endoscope than for the scissors instrument.

<FIG> illustrates an exemplary computer-assisted surgical system <NUM> ("surgical system <NUM>"). As described herein, system <NUM> may be implemented by surgical system <NUM>, implemented by a component included in surgical system <NUM>, connected to surgical system <NUM>, and/or otherwise used in conjunction with surgical system <NUM>.

As shown, surgical system <NUM> may include a manipulating system <NUM>, a user control system <NUM>, and an auxiliary system <NUM> communicatively coupled one to another. Surgical system <NUM> may be utilized by a surgical team to perform a computer-assisted surgical procedure on a patient <NUM>. As shown, the surgical team may include a surgeon <NUM>-<NUM> , an assistant <NUM>-<NUM>, a nurse <NUM>-<NUM>, and an anesthesiologist <NUM>-<NUM>, all of whom may be collectively referred to as "surgical team members <NUM>. " Additional or alternative surgical team members may be present during a surgical session as may serve a particular implementation.

While <FIG> illustrates an ongoing minimally invasive surgical procedure, it will be understood that surgical system <NUM> may similarly be used to perform open surgical procedures or other types of surgical procedures that may similarly benefit from the accuracy and convenience of surgical system <NUM>. Additionally, it will be understood that the surgical session throughout which surgical system <NUM> may be employed may not only include an operative phase of a surgical procedure, as is illustrated in <FIG>, but may also include preoperative, postoperative, and/or other suitable phases of the surgical procedure.

As shown in <FIG>, manipulating system <NUM> may include a plurality of manipulator arms <NUM> (e.g., manipulator arms <NUM>-<NUM> through <NUM>-<NUM>) to which a plurality of surgical instruments may be coupled. While manipulating system <NUM> is depicted and described herein as including four manipulator arms <NUM>, it will be recognized that manipulating system <NUM> may include only a single manipulator arm <NUM> or any other number of manipulator arms as may serve a particular implementation.

Manipulator arms <NUM> and/or surgical instruments attached to manipulator arms <NUM> may include one or more displacement transducers, orientational sensors, and/or positional sensors used to generate raw (i.e., uncorrected) kinematics information. One or more components of surgical system <NUM> may be configured to use the kinematics information to track (e.g., determine positions and orientations of) and/or control the surgical instruments. Additionally, surgical instruments attached to manipulator arms <NUM> may include one or more temperature sensors used to generate temperature data. System <NUM> may be configured to use the temperature data to track the temperature of the surgical instruments during the surgical session.

User control system <NUM> may be configured to facilitate control by surgeon <NUM>-<NUM> of manipulator arms <NUM> and surgical instruments attached to manipulator arms <NUM>. For example, surgeon <NUM>-<NUM> may interact with user control system <NUM> to remotely move or manipulate manipulator arms <NUM> and the surgical instruments. To this end, user control system <NUM> may provide surgeon <NUM>-<NUM> with images (e.g., high-definition 3D images, composite medical images, etc.), such as images <NUM>, <NUM>, and/or <NUM>, of a surgical area associated with patient <NUM> as captured by an imaging system (e.g., an endoscope implemented by surgical instrument <NUM>). In certain examples, user control system <NUM> may include a stereo viewer having two displays where stereoscopic images of a surgical area associated with patient <NUM> and generated by a stereoscopic imaging system may be viewed by surgeon <NUM>-<NUM>. Surgeon <NUM>-<NUM> may utilize the images to perform one or more procedures with one or more surgical instruments attached to manipulator arms <NUM>.

To facilitate control of surgical instruments, user control system <NUM> may include a set of master controls. These master controls may be manipulated by surgeon <NUM>-<NUM> to control movement of surgical instruments (e.g., by utilizing robotic and/or teleoperation technology). The master controls may be configured to detect a wide variety of hand, wrist, and finger movements by surgeon <NUM>-<NUM>. In this manner, surgeon <NUM>-<NUM> may intuitively perform a procedure using one or more surgical instruments.

Auxiliary system <NUM> may include one or more computing devices configured to perform primary processing operations of surgical system <NUM>. In such configurations, the one or more computing devices included in auxiliary system <NUM> may control and/or coordinate operations performed by various other components (e.g., manipulating system <NUM> and user control system <NUM>) of surgical system <NUM>. For example, a computing device included in user control system <NUM> may transmit instructions to manipulating system <NUM> by way of the one or more computing devices included in auxiliary system <NUM>. As another example, auxiliary system <NUM> may receive, from manipulating system <NUM>, and process image data representative of images (e.g., images <NUM>, <NUM>, and/or <NUM>) captured by an imaging device attached to one of manipulator arms <NUM>.

In some examples, auxiliary system <NUM> may be configured to present visual content to surgical team members <NUM> who may not have access to the images provided to surgeon <NUM>-<NUM> at user control system <NUM>. To this end, auxiliary system <NUM> may include a display monitor <NUM> configured to display one or more user interfaces, such as images (e.g., 2D images, composite medical images, etc.) of the surgical area, information associated with patient <NUM> and/or the surgical procedure, and/or any other visual content as may serve a particular implementation. For example, display monitor <NUM> may display images of the surgical area together with additional content (e.g., graphical content, contextual information, etc.) concurrently displayed with the images. In some embodiments, display monitor <NUM> is implemented by a touchscreen display with which surgical team members <NUM> may interact (e.g., by way of touch gestures) to provide user input to surgical system <NUM>.

Manipulating system <NUM>, user control system <NUM>, and auxiliary system <NUM> may be communicatively coupled one to another in any suitable manner. For example, as shown in <FIG>, manipulating system <NUM>, user control system <NUM>, and auxiliary system <NUM> may be communicatively coupled by way of control lines <NUM>, which may represent any wired or wireless communication link as may serve a particular implementation. To this end, manipulating system <NUM>, user control system <NUM>, and auxiliary system <NUM> may each include one or more wired or wireless communication interfaces, such as one or more local area network interfaces, Wi-Fi network interfaces, cellular interfaces, etc..

<FIG> shows an exemplary method <NUM> of detecting physical contact of a surgical instrument with patient tissue. While <FIG> illustrates exemplary operations according to one embodiment, other embodiments may omit, add to, reorder, combine, and/or modify any of the steps shown in <FIG>. One or more of the operations shown in in <FIG> may be performed by system <NUM>, any components included therein, and/or any implementation thereof.

In operation <NUM>, a tissue contact detection system tracks, over time during a surgical procedure, a temperature of a surgical instrument associated with a surgical system used for the surgical procedure. Operation <NUM> may be performed in any of the ways described herein.

In operation <NUM>, the tissue contact detection system determines, based on the tracked temperature of the surgical instrument, that the temperature of the surgical instrument changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount. Operation <NUM> may be performed in any of the ways described herein.

In operation <NUM>, the tissue contact detection system determines, based on the determination that the temperature of the surgical instrument changes from the first temperature to the second temperature, that the surgical instrument is in physical contact with patient tissue. Operation <NUM> may be performed in any of the ways described herein.

In operation <NUM>, the tissue contact detection system performs, in response to the determination that the surgical instrument is in physical contact with patient tissue, a mitigation operation configured to mitigate the physical contact of the surgical instrument with the patient tissue. Operation <NUM> may be performed in any of the ways described herein.

<FIG> illustrates an exemplary computing device <NUM> that may be specifically configured to perform one or more of the processes described herein. Any of the systems, units, computing devices, and/or other components described herein may be implemented by computing device <NUM>.

As shown in <FIG>, computing device <NUM> may include a communication interface <NUM>, a processor <NUM>, a storage device <NUM>, and an input/output ("I/O") module <NUM> communicatively connected one to another via a communication infrastructure <NUM>. While an exemplary computing device <NUM> is shown in <FIG>, the components illustrated in <FIG> are not intended to be limiting. Additional or alternative components may be used in other embodiments. Components of computing device <NUM> shown in <FIG> will now be described in additional detail.

Claim 1:
A system (<NUM>) comprising:
a memory (<NUM>) storing instructions (<NUM>); and
a processor (<NUM>) communicatively coupled to the memory (<NUM>) and configured to execute the instructions (<NUM>) to:
track, over time during a surgical procedure, a temperature of a surgical instrument (<NUM>) associated with a surgical system (<NUM>) used for the surgical procedure,
determine, based on the tracked temperature of the surgical instrument (<NUM>), that the temperature of the surgical instrument (<NUM>) changes from a first temperature to a second temperature that varies from the first temperature by at least a predetermined amount,
determine, based on the determination that the temperature of the surgical instrument (<NUM>) changes from the first temperature to the second temperature, that the surgical instrument (<NUM>) is in physical contact with patient tissue (<NUM>), and
perform a mitigation operation in response to the determination that the surgical instrument (<NUM>) is in physical contact with patient tissue (<NUM>).