Patent Description:
Surgical robots may assist a surgeon or other medical provider in carrying out a surgical procedure, or may complete one or more surgical procedures autonomously. Reference markers on an object, or the object itself may be used by a medical provider to detect and/or track the object during the surgical procedure. The reference markers may be disposed on any object such as a robotic arm, an end effector, an instrument, or a tool.

<CIT>, <CIT> and <CIT> show systems for determining a tool pose of a tool according to the preamble of claim <NUM>.

Some embodiments are shown in the dependent claims. Example aspects of the present disclosure include:
A system for determining a tool pose according to at least one embodiment of the present disclosure comprises a tracking device mounted to a tool and comprises: a plurality of faces; a plurality of markers defining a plurality of sets of markers, each set of markers comprising one or more markers of the plurality of markers, each set of markers disposed on a corresponding face; at least one processor; and a memory storing data for processing by the at least one processor that, when processed, the data causes the at least one processor to: receive information about a set of markers of the plurality of markers, determine a face of the plurality of faces having the set of markers disposed thereon, and determine a pose of the tool based on information and the determined face, wherein each adjacent set of markers share at least two markers, wherein the at least two markers being shared are included in both adjacent faces corresponding to the adjacent set of markers.

Any of the aspects herein, wherein the information includes a detected position of each marker of the set of markers.

Any of the aspects herein, wherein a first set of markers is arranged in a different pattern than a second set of markers.

Any of the aspects herein, wherein the information is received from a camera.

Any of the aspects herein, wherein the tool is supported by a robotic arm.

Any of the aspects herein, wherein the markers are infrared light emitting diodes.

Any of the aspects herein, further comprising: a controller configured to selectively cause one or more markers of the set of markers to emit infrared light.

Any of the aspects herein, wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: cause the controller to activate only one marker from each set of markers, in sequence; receive information about a detected activated marker; and cause the controller to activate the set of markers comprising the detected activated marker.

Any of the aspects herein, further comprising: a sensor configured to detect the set of markers, wherein the information is received from the sensor.

Any of the aspects herein, wherein the plurality of faces extend around a circumference of the tool.

A tracking device according to at least one embodiment of the present disclosure comprises a body; a first ring supported by the body; a second ring supported by the body, the second ring axially aligned with the first ring and spaced from the first ring; and a first plurality of markers disposed on the first ring and a second plurality of markers disposed on the second ring, the first plurality of markers and the second plurality of markers defining a plurality of sets of markers, each set of markers comprising one or more markers of each of the first plurality of markers and the second plurality of markers, each set of markers define a corresponding face of a plurality of faces.

Any of the aspects herein, wherein a first set of markers of the plurality of sets of markers is disposed in a first pattern and a second set of markers of the plurality of sets of markers is disposed in a second pattern.

Any of the aspects herein, wherein the second pattern is an inverse pattern of the first pattern.

Any of the aspects herein, wherein the first pattern and the second pattern are each a trapezoid.

Any of the aspects herein, wherein each set of markers includes at least three markers.

Any of the aspects herein, wherein the plurality of faces includes eight adjacent faces, each face adjacent to two other faces.

A system for tracking multiple objects according to at least one embodiment of the present disclosure comprises a first tracking device and a second tracking device, wherein the first tracking device and the second tracking device each comprise: a plurality of faces, and a plurality of markers defining a plurality of sets of markers, each set of markers comprising one or more markers of the plurality of markers, each set of markers disposed on a corresponding face, a controller configured to selectively activate one or more markers of the plurality of markers; at least one processor; and a memory storing data for processing by the at least one processor that, when processed, the data causes the at least one processor to: cause the controller to selectively activate one or more markers of the plurality of markers on the first tracking device and one or more markers of the plurality of markers on the second tracking device.

Any of the aspects herein, wherein each marker on the first tracking device has a first arrangement and each marker on the second tracking device has a second arrangement.

Any of the aspects herein, wherein the first arrangement different from the second arrangement.

Any of the aspects herein, wherein the second arrangement is identical to the first arrangement.

Any of the aspects herein, wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: cause the controller to activate the markers on the first tracking device in a first sequence, and cause the controller to activate the markers on the second tracking device in a second sequence.

Any of the aspects herein, wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: cause the controller to activate the markers on the first tracking device in a first wavelength, and cause the controller to activate the markers on the second tracking device in a second wavelength.

A system for determining a tool pose according to at least one embodiment of the present disclosure comprises a tracking device mounted to a tool and comprising: a plurality of segments, and a plurality of markers defining a plurality of sets of markers, each set of markers comprising one or more markers of the plurality of markers, each set of markers disposed on a corresponding segment, a controller configured to selectively activate one or more markers of the plurality of markers on each segment; a sensor having a field of view and configured to detect activated markers within the field of view; at least one processor; and a memory storing data for processing by the at least one processor that, when processed, the data causes the at least one processor to: cause the controller to selectively activate one or more markers of the plurality of markers on the tracking device; and determine an orientation of the tool based on data from the sensor corresponding to the one or more activated markers, the one or more activated markers disposed on a single segment of the plurality of segments, wherein markers not on the single segment are not activated.

Any of the aspects herein, wherein causing the controller to selectively activate one or more markers of the plurality of markers causes the controller to activate one or more markers on a predicted segment, and wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: determine, based on a known pose of the robotic arm, the predicted segment of the plurality of segments within the field of view; identify, using the sensor, at least one marker of the one or more activated markers; determine an actual segment based on the identified at least one marker; and compare the actual segment to the predicted segment to confirm the actual segment matches the predicted segment.

Any of the aspects herein, wherein causing the controller to selectively activate one or more markers of the plurality of markers causes the controller to activate, in sequence, a single marker on each of the plurality of segments, and wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: receive an indication from the sensor that an activated marker has been detected; and cause the controller to activate any remaining markers on the same segment as the activated marker.

A system for determining a tool pose according to at least one embodiment of the present disclosure comprises a tracking device mounted to a tool and comprising: a plurality of segments, and a plurality of markers defining a plurality of sets of markers, each set of markers comprising one or more markers of the plurality of markers, each set of markers disposed on each segment, a controller configured to selectively activate one or more markers of the plurality of markers on each segment; a sensor having a field of view and configured to detect activated markers within the field of view; at least one processor; and a memory storing data for processing by the at least one processor that, when processed, the data causes the at least one processor to: cause the controller to activate all of the markers on a first set of segments of the plurality of segments, and determine, based on data from the sensor, whether an activated marker has been detected; when an activated marker has been detected, cause the controller to activate a first subset of markers on the first set of segments of the plurality of segments; and when an activated marker has not been detected, cause the controller to activate all of the markers on a second set of segments of the plurality of segments, the first set and the second set of the plurality of segments comprising all of the plurality of segments.

Any of the aspects herein, wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: when an activated marker of the first subset of markers on the first set of segments of the plurality of segments has not been detected, cause the controller to activate a second subset of markers on the first set of segments of the plurality of segments.

Any of the aspects herein, wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: when an activated marker has been detected from markers on the second set of segments of the plurality of segments, cause the controller to activate a first subset of markers on the second set of segments of the plurality of segments.

Any of the aspects herein, wherein the memory stores additional data for execution by the at least one processor that, when processed, further cause the at least one processor to: when an activated marker of the first subset of markers on the second set of segments of the plurality of segments has not been detected, cause the controller to activate a second subset of markers on the second set of segments of the plurality of segments.

Any aspect in combination with any one or more other aspects.

Any one or more of the features disclosed herein.

Any one or more of the features as substantially disclosed herein.

Any one or more of the features as substantially disclosed herein in combination with any one or more other features as substantially disclosed herein.

Any one of the aspects/features/embodiments in combination with any one or more other aspects/features/embodiments.

Use of any one or more of the aspects or features as disclosed herein.

It is to be appreciated that any feature described herein can be claimed in combination with any other feature(s) as described herein, regardless of whether the features come from the same described embodiment.

It should also be understood that, depending on the example or embodiment, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, and/or may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the disclosed techniques according to different embodiments of the present disclosure). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a computing device and/or a medical device.

In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. Alternatively or additionally, functions may be implemented using machine learning models, neural networks, artificial neural networks, or combinations thereof (alone or in combination with instructions).

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple A11, A12, A12X, A12Z, or A13 Bionic processors; or any other general purpose microprocessors), graphics processing units (e.g., Nvidia GeForce RTX <NUM>-series processors, Nvidia GeForce RTX <NUM>-series processors, AMD Radeon RX <NUM>-series processors, AMD Radeon RX <NUM>-series processors, or any other graphics processing units), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry.

The terms proximal and distal are used in this disclosure with their conventional medical meanings, proximal being closer to the operator or user of the system, and further from the region of surgical interest in or on the patient, and distal being closer to the region of surgical interest in or on the patient, and further from the operator or user of the system.

Part of a robotic guided spine surgery may involve ascertaining a position or pose of a robot used for the spine surgery, though it will be appreciated that the position or pose of a robot may be ascertained in other types of surgical procedures. One suggested method to perform a desired validation of the robot position or pose uses, for example, optical tracking of active markers on a multi-faced tool. The active markers may include infrared light emitting diodes (IREDs) mounted on a ceramic base, for example. A position sensor (such as, for example, a camera or an infrared camera) may detect the infrared light emitted by activated markers. A system such as a navigation system or a computing system, for example, can report the positions of the markers individually, and can calculate a position and orientation of one or more tools that incorporate the markers.

One embodiment of a multi-faced tool may be a tool with markers arranged into faces of the tool. In some examples, the position sensor may track only one face at a time. In some instances, the markers on a multi-faced tool may not all be detected simultaneously by the position sensor. A multi-faced tool may be beneficial if an intended application requires the tool to rotate beyond a maximum marker angle. The integration of the active wired tool with the robotic arm may allow for continuous tracking of the position of the robotic arm and its orientation.

Active tools may be physically connected to the system such as, for example, the navigation system or a computing system, and incorporate the active markers. To track an active tool, the position sensor detects, for example, infrared light emitted by the active markers. The system may control the markers (to selectively emit, for example, infrared light) through a wired connection to a system control unit of the system. To identify the tool, the system control unit may trigger the IREDs or LEDs in synchronization (e.g., according to a predetermined pattern) until the system control unit finds enough IREDs or LEDs to calculate a tool's transformation. If the system control unit tries to track a unique geometry tool, the IREDs or LEDs may be sequenced all at the same time since the tool orientation can be detected by the unique shape.

For a multi-faced tool, in instances where the tool may go missing, the system control unit may illuminate one marker at a time until the marker is in view of the position sensor. The position sensor may then lock on the face that the marker is a part of (and may sequence each IRED or LED in the face to find an orientation of the face). To reduce a lock-on time it is possible to use a combination of active wired markers and unique geometry. Instead of illuminating one marker at a time for unique geometry tools, the system control unit illuminates an entire face.

In some instances, if a view of the tool has been blocked from the position sensor and then comes back into a field of view of the position sensor, the position sensor has no knowledge of which face it is looking at so it systemically determines which face the position sensor is viewing. For example, if the position sensor tracks a face and the face goes missing from a field of view of the sensor, the sequencing may start from the adjacent face to determine which face is now in the field of view of the sensor. The system control unit or controller may accomplish this by activating markers sequentially until a marker is detected by the sensor.

For an illustrative tool, a smart algorithm to reduce the lock-on time and increase efficiency of the searching process may be employed. Instead of firing/activating the IREDs or LEDs by the natural order (e.g., <NUM>, <NUM>, <NUM>, etc.), the system control unit can fire/activate the IREDs or LEDs by each face by firing a primary IRED or LED of each face. When the position sensor identifies the IRED or LED, it then sequences then the IREDs or LEDs on the face to identify its orientation. For example, in the conventional manner, if a tool has <NUM> LEDs that is divided into <NUM> faces (each face shares <NUM> LEDs with its adjacent face), to detect the last face at least <NUM> LEDs would need to be fired using a conventional algorithm (<NUM> LEDs for a face and to get a transformation <NUM> LEDs are enough). In the suggested method using the smart algorithm, only <NUM> LEDs would need to be fired (<NUM> faces plus <NUM> LEDs in the last face), which may produce a difference of, for example, a decrease of approximately <NUM> in the lock-on time for a <NUM> refresh rate.

For both types of tools - a standard geometry tool and/or a unique geometry tool - instead of firing/activating the faces in an order, another approach may be used. The approach may divide the faces into two adjacent groups - a first group and a second group - and activate all of the markers in the first group. If the position sensor detects IREDs or LEDs, then the first group is divided into two sub-groups and markers of each sub-group is activated until the position sensor detects IREDs or LEDs. If the position sensor does not detect IREDs or LEDs, then the markers in the second group are activated. In such embodiments, the lock-on time may be reduced. For example, for a tool with eight faces, to detect the last face, markers on eight faces would need to be activated, i.e. <NUM> frames of activation. If the suggested algorithm is used, only six frames are needed to detect the faces, which may result in a difference of, for example, a decrease of approximately <NUM> in the lock-on time for a <NUM> refresh rate.

In some embodiments, the multi-face active wired tool may be positioned on a robotic arm. This connection may provide the position sensor a prediction of which face is in the field of view of the sensor, and as a result may reduce the lock-on time significantly. This algorithm or sequence utilizes this communication such that whenever a tool goes missing, the system control unit will calculate the predicted face within the field of view of the sensor according to the solver information and illuminate only the relevant IREDs or LEDs instead of starting a standard, conventional sequencing. For example, for a standard geometry tool, having <NUM> LEDs that is divided into eight faces (each face shares <NUM> LEDs with its adjacent face), to detect the last face using a conventional algorithm, at least <NUM> LEDs (<NUM> LEDs for a face and to get a transformation, <NUM> LEDs are enough) would need to be activated. Using the algorithm or sequence, only <NUM> LEDs would need to be activated, which may result in a difference of about <NUM> lock-on time for a <NUM> refresh rate.

Embodiments of the present disclosure provide technical solutions to one or more of the problems of (<NUM>) determining a pose of a tool based on a tracking device mounted to the tool, (<NUM>) improving a lock-on time of a sensor to a tracking device, (<NUM>) tracking a tool based on a tracking device mounted to the tool, (<NUM>) validating a pose of a robotic arm during a surgical procedure, and (<NUM>) reducing an overall operating time.

Turning first to <FIG>, a block diagram of a system <NUM> according to at least one embodiment of the present disclosure is shown. The system <NUM> may be used to determine a pose of an object (e.g., a tool, an instrument, an end effector, a robotic arm, an anatomical element of a patient, or the like) and/or carry out one or more other aspects of one or more of the methods disclosed herein. The system <NUM> comprises a computing device <NUM>, one or more sensors <NUM>, a robot <NUM>, one or more controllers <NUM>, one or more tracking devices <NUM>, <NUM>, <NUM>, a navigation system <NUM>, a database <NUM>, and/or a cloud or other network <NUM>. Systems according to other embodiments of the present disclosure may comprise more or fewer components than the system <NUM>. For example, the system <NUM> may not include the robot <NUM>, the navigation system <NUM>, the controller <NUM>, one or more components of the computing device <NUM>, the database <NUM>, and/or the cloud <NUM>.

The computing device <NUM> comprises a processor <NUM>, a memory <NUM>, a communication interface <NUM>, and a user interface <NUM>. Computing devices according to other embodiments of the present disclosure may comprise more or fewer components than the computing device <NUM>.

The processor <NUM> of the computing device <NUM> may be any processor described herein or any similar processor. The processor <NUM> may be configured to execute instructions stored in the memory <NUM>, which instructions may cause the processor <NUM> to carry out one or more computing steps utilizing or based on data received from the sensor <NUM>, the robot <NUM>, the controller <NUM>, the tracking device <NUM>, <NUM>, <NUM>, the navigation system <NUM>, the database <NUM>, and/or the cloud <NUM>.

The memory <NUM> may be or comprise RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other tangible, non-transitory memory for storing computer-readable data and/or instructions. The memory <NUM> may store information or data useful for completing, for example, any step of the methods <NUM>, <NUM>, <NUM>, <NUM>, and/or <NUM> described herein, or of any other methods. The memory <NUM> may store, for example one or more sequencing algorithms <NUM> and/or instruction(s) <NUM>. Such instructions <NUM> or algorithms <NUM> may, in some embodiments, be organized into one or more applications, modules, packages, layers, neural networks, or engines. The algorithms <NUM> and/or instructions <NUM> may cause the processor <NUM> to manipulate data stored in the memory <NUM> and/or received from or via the sensor <NUM>, the robot <NUM>, the controller <NUM>, the tracking device <NUM>, <NUM>, <NUM>, the navigation system <NUM>, the database <NUM>, and/or the cloud <NUM>.

The computing device <NUM> may also comprise a communication interface <NUM>. The communication interface <NUM> may be used for receiving sensor data or other information from an external source (such as the sensor <NUM>, the robot <NUM>, the controller <NUM>, the tracking device <NUM>, <NUM>, <NUM>, the navigation system <NUM>, the database <NUM>, the cloud <NUM>, and/or any other system or component not part of the system <NUM>), and/or for transmitting instructions, images, or other information to an external system or device (e.g., another computing device <NUM>, the sensor <NUM>, the robot <NUM>, the controller <NUM>, the tracking device <NUM>, <NUM>, <NUM>, the navigation system <NUM>, the database <NUM>, the cloud <NUM>, and/or any other system or component not part of the system <NUM>). The communication interface <NUM> may comprise one or more wired interfaces (e.g., a USB port, an ethernet port, a Firewire port) and/or one or more wireless transceivers or interfaces (configured, for example, to transmit and/or receive information via one or more wireless communication protocols such as <NUM>. 11a/b/g/n, Bluetooth, NFC, ZigBee, and so forth). In some embodiments, the communication interface <NUM> may be useful for enabling the device <NUM> to communicate with one or more other processors <NUM> or computing devices <NUM>, whether to reduce the time needed to accomplish a computing-intensive task or for any other reason.

The computing device <NUM> may also comprise one or more user interfaces <NUM>. The user interface <NUM> may be or comprise a keyboard, mouse, trackball, monitor, television, screen, touchscreen, and/or any other device for receiving information from a user and/or for providing information to a user. The user interface <NUM> may be used, for example, to receive a user selection or other user input regarding any step of any method described herein. Notwithstanding the foregoing, any required input for any step of any method described herein may be generated automatically by the system <NUM> (e.g., by the processor <NUM> or another component of the system <NUM>) or received by the system <NUM> from a source external to the system <NUM>. In some embodiments, the user interface <NUM> may be useful to allow a surgeon or other user to modify instructions such as the instructions <NUM> to be executed by the processor <NUM> according to one or more embodiments of the present disclosure, and/or to modify or adjust a setting of other information displayed on the user interface <NUM> or corresponding thereto.

Although the user interface <NUM> is shown as part of the computing device <NUM>, in some embodiments, the computing device <NUM> may utilize a user interface <NUM> that is housed separately from one or more remaining components of the computing device <NUM>. In some embodiments, the user interface <NUM> may be located proximate one or more other components of the computing device <NUM>, while in other embodiments, the user interface <NUM> may be located remotely from one or more other components of the computer device <NUM>.

The system <NUM> includes the one or more sensor(s) <NUM>. The sensor <NUM> is operable to detect a plurality of markers <NUM> disposed on the tracking device <NUM>, <NUM>, <NUM> (described in detail with respect to <FIG>). The sensor <NUM> may be, for example, an optical camera; an infrared camera; a 3D camera system; a stereoscopic vision system; an imaging device; or any other sensor that can detect the plurality of markers <NUM>. In some examples, the markers <NUM> may be active markers and the sensor <NUM> may be configured to sense the markers <NUM> when the markers <NUM> are activated (by the controller <NUM>, for example). In such examples, the markers <NUM> may be an infrared light emitting diode and the sensor <NUM> may be an infrared camera configured to sense when the markers <NUM> emit infrared light.

The sensor <NUM> may comprise a dedicated processor for executing instructions stored in a dedicated memory of the sensor <NUM>, or the sensor <NUM> may simply be configured to transmit data collected therewith to the computing device <NUM> or to another component of the system <NUM>. Although shown in <FIG> as being in communication only with the computing device <NUM>, in some embodiments, the sensor <NUM> may be in communication with any one or more of the tracking device <NUM>, <NUM>, <NUM>, the computing device <NUM>, the robot <NUM>, the controller <NUM>, the navigation system <NUM>, the database <NUM>, and/or the cloud <NUM>. Also, in some embodiments, the computing device <NUM> may comprise the sensor <NUM>, while in other embodiments, the navigation system <NUM> may comprise the sensor <NUM>. In still other embodiments, the robot <NUM> may comprise the sensor <NUM>.

The sensor <NUM> may be positioned directly above an operating table or portion thereof, or above and to one side of an operating table or portion thereof, or in another convenient position within an operating room or other room. The sensor <NUM> may be positioned at a location selected to provide the sensor <NUM> with a clear and/or unobstructed view of the tracking device <NUM>, <NUM>, <NUM> (and thus of one or more markers <NUM> fixedly secured to the tracking device <NUM>, <NUM>, <NUM>) during operation thereof. In some embodiments, the sensor <NUM> is fixed, while in other embodiments, the sensor <NUM> may be precisely movable (whether manually or automatically) in one or more directions.

The sensor <NUM> may be configured to capture data regarding sensed markers <NUM> only at a given moment in time. For example, where the sensor <NUM> is a camera, the sensor <NUM> may be configured to capture still images comprising one or more markers <NUM>. The sensor <NUM> may be configured to capture such data at periodic intervals, or when commanded by a user (e.g., via a user interface <NUM>), or upon a signal (generated either autonomously or in response to user input) from the controller <NUM>, the computing device <NUM>, the robot <NUM>, and/or the navigation system <NUM>.

The sensor <NUM> may additionally or alternatively be operable to capture data corresponding to the plurality of markers <NUM> continuously, in real-time. In such embodiments, the sensor <NUM> may provide a stream of real-time sensor data to the computing device <NUM>, which may continuously process the sensor data to detect the markers <NUM> therein. In some embodiments, the sensor <NUM> may comprise more than one sensor <NUM>.

Still referring to <FIG>, the robot <NUM> may be any surgical robot or surgical robotic system. The robot <NUM> may be or comprise, for example, the Mazor X™ Stealth Edition robotic guidance system. The robot <NUM> may be configured to position the tracking device, <NUM>, <NUM>, <NUM>, for example, at one or more precise position(s) and orientation(s), and/or to return the tracking device, <NUM>, <NUM>, <NUM> to the same position(s) and orientation(s) at a later point in time. The robot <NUM> may additionally or alternatively be configured to manipulate a surgical tool (whether based on guidance from the navigation system <NUM> or not) to accomplish or to assist with a surgical task. In some embodiments, the robot <NUM> may be configured to hold and/or manipulate an anatomical element during or in connection with a surgical procedure. The robot <NUM> may comprise one or more robotic arms <NUM>. In some embodiments, the robotic arm <NUM> may comprise a first robotic arm and a second robotic arm, though the robot <NUM> may comprise more than two robotic arms. In some embodiments, one or more of the robotic arms <NUM> may be used to hold and/or maneuver the tracking device, <NUM>, <NUM>, <NUM>. In embodiments where two tracking devices <NUM>, <NUM>, <NUM> are used, one robotic arm <NUM> may hold a first tracking device, and another robotic arm <NUM> may hold a second tracking device. Each robotic arm <NUM> may be positionable independently of the other robotic arm. It will be appreciated that any number of robotic arms <NUM> may be used to support or hold any number of tracking device <NUM>, <NUM>, <NUM>. The robotic arms may be controlled in a single, shared coordinate space, or in separate coordinate spaces.

The robot <NUM>, together with the robotic arm <NUM>, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Further, the robotic arm <NUM> may be positioned or positionable in any pose, plane, and/or focal point. The pose includes a position and an orientation. As a result, a tracking device <NUM>, <NUM>, <NUM>, a surgical tool, or other object held by the robot <NUM> (or, more specifically, by the robotic arm <NUM>) may be precisely positionable in one or more needed and specific positions and orientations.

The robotic arm(s) <NUM> may comprise one or more sensors that enable the processor <NUM> (or a processor of the robot <NUM>) to determine a precise pose in space of the robotic arm (as well as any object or element held by or secured to the robotic arm).

In some embodiments, reference markers (i.e., navigation markers) or markers <NUM> (shown in <FIG>), may be placed on the robot <NUM> (including, e.g., on the robotic arm <NUM>) or any other object in the surgical space. The markers <NUM> may be tracked by the navigation system <NUM>, and the results of the tracking may be used by the robot <NUM> and/or by an operator of the system <NUM> or any component thereof. In some embodiments, the navigation system <NUM> can be used to track other components of the system and the system can operate without the use of the robot <NUM> (e.g., with the surgeon manually manipulating one or more surgical tools, based on information and/or instructions generated by the navigation system <NUM>, for example).

The navigation system <NUM> may provide navigation for a surgeon and/or a surgical robot during an operation. The navigation system <NUM> may be any now-known or future-developed navigation system, including, for example, the Medtronic StealthStation™ S8 surgical navigation system or any successor thereof. The navigation system <NUM> may include one or more cameras or other sensor(s) such as the sensor <NUM> for tracking the markers <NUM>, navigated trackers, or other objects within the operating room or other room in which some or all of the system <NUM> is located. The one or more cameras may be optical cameras, infrared cameras, or other cameras. In some embodiments, the navigation system <NUM> may comprise one or more electromagnetic sensors. In various embodiments, the navigation system <NUM> may be used to track a position and orientation (i.e., pose) of the robot <NUM> and/or robotic arm <NUM>, and/or one or more surgical tools (or, more particularly, to track a pose of a navigated tracker attached, directly or indirectly, in fixed relation to the one or more of the foregoing). The navigation system <NUM> may include a display for displaying one or more images from an external source (e.g., the computing device <NUM>, sensor <NUM>, or other source) or for displaying an image and/or video stream from the one or more cameras or other sensors of the navigation system <NUM>. In some embodiments, the system <NUM> can operate without the use of the navigation system <NUM>. The navigation system <NUM> may be configured to provide guidance to a surgeon or other user of the system <NUM> or a component thereof, to the robot <NUM>, or to any other element of the system <NUM> regarding, for example, a pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, and/or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan.

In the illustrated embodiment, the system <NUM> includes the controller <NUM>, though in some embodiments the system <NUM> may not include the controller <NUM>. The controller <NUM> may be an electronic, a mechanical, or an electro-mechanical controller. The controller <NUM> may comprise or may be any processor described herein. The controller <NUM> may comprise a memory storing instructions for executing any of the functions or methods described herein as being carried out by the controller <NUM>. In some embodiments, the controller <NUM> may be configured to simply convert signals received from the computing device <NUM> (e.g., via a communication interface <NUM>) into commands for operating the tracking device <NUM>, <NUM>, <NUM> (and more specifically, for activating markers <NUM> fixedly secured to the tracking device <NUM>, <NUM>, <NUM>), the sensor <NUM>, the navigation system <NUM>, and/or the robot <NUM>. In other embodiments, the controller <NUM> may be configured to process and/or convert signals received from the tracking device <NUM>, <NUM>, <NUM>, the sensor <NUM>, the navigation system <NUM>, and/or the robot <NUM>. Further, the controller <NUM> may receive signals from one or more sources (e.g., the tracking device <NUM>, <NUM>, <NUM>, the sensor <NUM>, the navigation system <NUM>, and/or the robot <NUM>) and may output signals to one or more sources.

The controller <NUM> is operable to cause one or more markers of a plurality of markers <NUM> to selectively activate. The controller <NUM> may cause the one or more markers to activate for any duration of time, at any intensity, and/or at any wavelength. In some embodiments, the controller <NUM> may cause one or more markers to selectively activate in sequence. In other embodiments, the controller <NUM> may cause one or more markers to activate one at a time. In still other embodiments, the controller <NUM> may cause one or more sets of markers to activate at the same time, while deactivating markers not in the one or more sets of markers.

The system <NUM> also includes a tracking device <NUM>, <NUM>, <NUM> described in detail below with respect to <FIG>. The tracking device <NUM>, <NUM>, <NUM> may be mounted to a tool or an end effector supported by a robotic arm such as the robotic arm <NUM>. By mounting the device <NUM>, <NUM>, <NUM> to the tool, the end effector, or any other object, a pose of such tool, end effector, or object may be determined based on a pose of the device <NUM>, <NUM>, <NUM>. In some embodiments, the pose of the tracking device <NUM>, <NUM>, <NUM> may be used to determine and validate a pose of a robotic arm such as the robotic arm <NUM>. In such embodiments, the pose of the robotic arm <NUM> as determined by the tracking device <NUM>, <NUM>, <NUM> may be compared to the pose of the robotic arm <NUM> as determined from the robotic arm <NUM> (using, for example, sensors disposed or integrated with the robotic arm <NUM>) to validate the pose of the robotic arm <NUM>.

Turning to <FIG>, a tracking device <NUM> is shown. The tracking device <NUM> includes a body <NUM>. The body <NUM> may be formed from any material such as, for example, ceramic, aluminum, steel, or the like. In the illustrated embodiment, the body <NUM> a cross-section of the body <NUM> is cylindrical. In other embodiments, the cross-section of the body <NUM> may be any shape such as a square, an oval, a rectangle, or the like. In some embodiments the body <NUM> has a height of about <NUM>. In other embodiments, the body may have a height less or greater than <NUM>. In some embodiments, the body has a diameter of about <NUM>. In other embodiments, the body may have a diameter less or greater than <NUM>.

The body <NUM> includes an outer surface <NUM>, an inner surface <NUM>, and an interior space <NUM>. The outer surface <NUM> may be smooth, as illustrated, or may have any texture. The outer surface <NUM> may also be curved, as shown, or may have any shape or form. A plurality of faces or segments <NUM> may span across the outer surface <NUM> and extend around a circumference of the body <NUM>. In some embodiments, the plurality of faces <NUM> may include eight adjacent faces. In such embodiments, each face is adjacent to two other faces. In other embodiments, the plurality of faces <NUM> may include fewer or more than eight faces. In yet other embodiments, some faces may be adjacent to one, two, or more than two faces, and/or some faces may not be adjacent to any faces. A boundary of four faces of the plurality of faces <NUM> is shown in dotted lines in <FIG> for illustrative purposes. In some embodiments, each face may have a length of about <NUM>. In other embodiments, each face may have a length less or greater than <NUM>.

In the illustrated embodiment, a plurality of markers <NUM> are fixedly secured to the outer surface <NUM>. The plurality of markers <NUM> are distributed around a circumference of the body <NUM> and provide <NUM> degrees of markers on the device <NUM> so that at least one marker <NUM> is visible or in a field of view of a sensor such as the sensor <NUM> at any rotational position of the device <NUM>. In some embodiments, the plurality of markers <NUM> include <NUM> markers. In other embodiments, the plurality of markers <NUM> include fewer or more than <NUM> markers.

The plurality of markers <NUM> may be spaced so that at least one marker may be visible in a field of view of a sensor such as the sensor <NUM> at any given position, so long as a line of sight between the device <NUM> and the sensor <NUM> is clear or not blocked. As shown, the plurality of markers <NUM> may include a first row of markers <NUM> and a second row of markers <NUM>. Each marker in the first row <NUM> and the second row <NUM> may be spaced equidistance and symmetrically around the body <NUM>, though as will be shown and described below, each marker may be spaced at different distances and/or non-symmetrically from each other. In some embodiments, a marker in the first row <NUM> and the second row <NUM> may be spaced about <NUM> degrees from an adjacent marker in the same row and relative to a center axis <NUM> of the body <NUM>. For example, marker 216A,B may be spaced about <NUM> degrees from marker 216B,C. In other embodiments, a marker may be spaced less or greater than <NUM> degrees from an adjacent marker.

The plurality of markers <NUM> may be active markers. In some embodiments, the plurality of markers <NUM> are infrared light emitting diodes. In other embodiments, the plurality of markers <NUM> may be any type of active markers. For example, the plurality of markers <NUM> may emit light in any wavelength. In some embodiments, the plurality of markers <NUM> may comprise different types of active markers. In other embodiments, the plurality of markers <NUM> may comprise the same type of active markers. In other embodiments, the plurality of markers <NUM> may be passive markers. For example, the plurality of markers <NUM> may be reflective spheres.

The plurality of markers <NUM> define a plurality of sets of markers <NUM> and each set of markers <NUM> is disposed on a corresponding face of the plurality of faces <NUM>. For example, in the illustrated embodiments the set of markers 216A and 216A,B correspond to face 214A. As shown, each set of markers <NUM> includes four markers. In other embodiments, each set of markers <NUM> includes three markers. In still other embodiments, each set of markers <NUM> may include fewer or more than four markers. Each set of markers <NUM> may include the same number of markers. In other embodiments, a set of markers <NUM> may include a different number of markers than another set of markers <NUM>. In the illustrated embodiment, each set of markers <NUM> is arranged in a square. In other embodiments, each set of markers <NUM> may be arranged in any shape or pattern. In still other embodiments, a set of markers <NUM> may be arranged in a different shape or pattern than another set of markers <NUM>. For example, a first set of markers can be arranged in a first pattern or unique geometry and a second set of markers can be arranged in a second pattern or unique geometry different from the first pattern. Such patterns or unique geometries may aid in determining a position of the device <NUM> (and thus, for example, the corresponding robotic arm <NUM>) based on the corresponding face of the pattern detectable by the sensor <NUM>.

In the illustrated embodiment, each face <NUM> is defined by the corresponding set of markers <NUM> and adjacent faces <NUM> may share two markers <NUM>. In other words, a first set of markers and a second set of markers may share two markers, while each of the first set and the second set include four markers total. For example, as shown, face 214A includes the set of markers 216A and 216A,B and face 214B includes the set of markers 216A,B and 216B,C. In other embodiments, each face <NUM> (and thus, the corresponding set of markers) may not share any markers with an adjacent face (and thus, the adjacent set of markers).

Turning to <FIG>, the tracking device <NUM> is shown. A plurality of wires <NUM> may be stored in the interior space <NUM> of the body <NUM> and one or more wires of the plurality of wires <NUM> may be secured to the inner surface <NUM> of the body <NUM>. It will be appreciated that any hardware may be stored or secured in the interior space <NUM>. Each wire <NUM> may provide or deliver power to a corresponding marker <NUM> to selectively activate the corresponding marker <NUM> when power is delivered to the corresponding marker <NUM>. In embodiments where the marker <NUM> is an infrared emitting light diode, activating the marker <NUM> causes the marker <NUM> to emit infrared light. Conversely, the marker <NUM> does not emit infrared light when power is not delivered to the marker <NUM> and the marker <NUM> is not activated.

Though not illustrated, the tracking device <NUM> may be mounted to, installed on, or otherwise secured to any object such as an end effector, a robotic arm (including, for example, an end of the robotic arm, a joint of the robotic arm, a segment of the robotic arm, or any portion of the robotic arm), or any other component.

Turning to <FIG>, a tracking device <NUM> according to at least one embodiment is shown. The tracking device <NUM> may be the same as or similar to the tracking device <NUM> described above. In the illustrated embodiment, a plurality of markers <NUM> are fixedly secured to an outer surface <NUM> of a body <NUM>. The plurality of markers <NUM> define a plurality of sets of markers <NUM> and each set of markers <NUM> is disposed on a corresponding face of the plurality of faces <NUM>. As shown, each set of markers <NUM> includes four markers. In other embodiments, each set of markers <NUM> may include fewer or more than four markers. In still other embodiments, a set of markers <NUM> may include a different number of markers than another set of markers <NUM>.

In the illustrated embodiment, each set of markers <NUM> defines a trapezoid, shown in broken lines for illustrative purposes. In some embodiments, a first segment of the trapezoid may be about <NUM>, a second segment of the trapezoid may be about <NUM>, a third segment of the trapezoid may be about <NUM>, and a fourth segment of the trapezoid may be about <NUM>. In other embodiments, the first segment may be less or greater than <NUM>, the second segment may be less or greater than <NUM>, the third segment may be less or greater than <NUM>, and the fourth segment may be less or greater than <NUM>. Also shown in the illustrated embodiment, a first set of markers 316A and 316A,B corresponding to a face 314A is rotated <NUM> degrees from an adjacent and second set of markers 316B and 316A,B corresponding to a face 314B. In other words, the set of markers <NUM> may be an inverse pattern of a pattern of the adjacent set of markers <NUM> (and as illustrated, for example, an inverse trapezoid of the adjacent trapezoid). In other embodiments, each set of markers <NUM> may be configured in any shape or pattern. In yet other embodiments, a set of markers <NUM> may be configured in a different shape or pattern than another set of markers <NUM>. In the illustrated embodiment, each face <NUM> is defined by the corresponding set of markers <NUM> and adjacent faces <NUM> may share two markers <NUM>. In other words, a first set of markers and a second set of markers may share two markers, while each of the first set and the second set include four markers total. In other embodiments, each face <NUM> (and thus, the corresponding set of markers) may not share any markers with an adjacent face (and thus, the adjacent set of markers).

Turning to <FIG>, a tracking device <NUM> according to at least one embodiment is shown. The tracking device <NUM> may be the same as or similar to the tracking device <NUM> described above except for the tracking device <NUM> includes a first ring <NUM> spaced from a second ring <NUM>. In some embodiments, the first ring <NUM> is spaced from the second ring <NUM> a distance of about <NUM>. In other embodiments the first ring <NUM> may be spaced from the second ring <NUM> a distance of less or greater than <NUM>. Though the first ring <NUM> and the second ring <NUM> are shown having a cross-section that is circular, it will be appreciated that the first ring <NUM> and the second ring <NUM> may have any cross-section with any shape such as, for example, a rectangle, a square, or an oval. The device <NUM> also includes a base <NUM> spanning a distance between the first ring <NUM> and the second ring <NUM>. The base <NUM> may support, for example, hardware, wires, a connector <NUM> (for connecting the device <NUM> to an end effector or robotic arm, for example), or the like.

A first plurality of markers 404A is disposed on the first ring <NUM> and a second plurality of markers 404B is disposed on the second ring <NUM>. The first plurality of markers 404A and the second plurality of markers 404B define a plurality of sets of markers <NUM>, wherein each set of markers <NUM> comprises one or more markers of each of the first plurality of markers 404A and the second plurality of markers 404B. Each set of markers <NUM> may belong to or define a corresponding face of a plurality of faces <NUM>. In the illustrated embodiment, one face <NUM> is shown in dotted lines for illustrative purposes.

In some embodiments the first plurality of markers 404A includes the same number of markers as the second plurality of markers 404B. In other embodiments, the first plurality of markers 404A may have more or fewer markers than the second plurality of markers 404B. Similarly, in some embodiments, the first plurality of markers 404A may be disposed on the first ring <NUM> in the same arrangement as the second plurality of markers 404B is disposed on the second ring <NUM>. In other embodiments, the first plurality of markers 404A may be disposed on the first ring <NUM> in a different arrangement as the second plurality of markers 404B disposed on the second ring <NUM>. In the illustrated embodiment - where the first ring <NUM> and the second ring <NUM> are identical (in other words, the first plurality of markers 404A and the second plurality of markers 404B are disposed in the same arrangement on the first ring <NUM> and the second ring <NUM>, respectively) - the first ring <NUM> and the second ring <NUM> are aligned so as to offset the first plurality of markers 404A and the second plurality of markers 404B. In such embodiments, each face of the plurality of faces may be shaped as, for example, a parallelogram. In other embodiments, the first ring <NUM> and the second ring <NUM> may be aligned so as to align the first plurality of markers 404A and the second plurality of markers 404B. In such embodiments, each face of the plurality of faces may be shaped as, for example, a rectangle, or, depending on the spacing between the first ring <NUM> and the second ring <NUM>, a square. It will be appreciated that the first plurality of markers 404A and the second plurality of markers 404B may be disposed in any arrangement to define each face of the plurality of faces in any shape.

Though the device <NUM>, <NUM> includes a body <NUM>, <NUM> with respect to <FIG>, and <FIG> and the device <NUM> includes a first ring <NUM>, a second ring <NUM>, and a base <NUM>, it will be appreciated that any embodiment can include any combination of or any number of a body, a base, and a ring. For example, the device <NUM> may include a first ring and a second ring instead of a body. In another example, the device <NUM> may include a body instead of a first ring and a second ring.

The system <NUM> or similar systems of <FIG> and the tracking device <NUM>, <NUM>, <NUM> or similar tracking devices of <FIG> may be used, for example, to carry out one or more aspects of any of the methods <NUM>, <NUM>, <NUM>, <NUM> and/or <NUM> described herein. The system <NUM> or similar systems and/or the tracking device <NUM>, <NUM>, <NUM> or other similar tracking devices may also be used for other purposes.

<FIG> depicts a method <NUM> that may be used, for example, for determining a tool pose.

The method <NUM> (and/or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) <NUM> of the computing device <NUM> described above. The at least one processor may be part of a robot (such as a robot <NUM>) or part of a navigation system (such as a navigation system <NUM>). A processor other than any processor described herein may also be used to execute the method <NUM>. The at least one processor may perform the method <NUM> by executing instructions such as instructions <NUM> stored in a memory such as the memory <NUM>. The instructions may correspond to one or more steps of the method <NUM> described below. The instructions may cause the processor to execute one or more algorithms, such as a sequencing algorithm <NUM>.

The method <NUM> comprises causing a controller to activate one marker from a set of markers (step <NUM>). The controller may be the same as or similar to the controller <NUM> and the set of markers may be the same as or similar to the set of markers <NUM>, <NUM>, <NUM>. The set of markers may be part of a plurality of sets of markers. Each set of markers may comprise one or more markers of a plurality of markers (such as the plurality of markers <NUM>, <NUM>, 404A, 404B) fixedly secured to a tracking device such as the tracking device <NUM>, <NUM>, <NUM>. Each set of markers may define a corresponding face of a plurality of faces such as the plurality of faces <NUM>, <NUM>, <NUM>.

The tracking device may be mounted to a tool. In some embodiments, the tool is supported by a robotic arm such as the robotic arm <NUM>. In other embodiments, the tool is an end effector supported by a robotic arm. During use, a pose of the tracking device may be determined as described herein. By determining a pose of the tracking device, a pose of the tool can also be determined relative to the pose of the tracking device.

In some embodiments where each of the plurality of markers is an infrared light emitting diode, activating a marker causes the marker to emit infrared light. The infrared light may be detected by a sensor such as the sensor <NUM>, which may be, for example, an infrared camera. In other embodiments, activating the marker may cause the marker to emit a light of any wavelength. Activating the marker may include using, for example, a processor such as the processor <NUM> to generate instructions such as the instructions <NUM> and transmitting the instructions to the controller. Further, the controller may activate one or more markers in sequence using an algorithm such as the sequencing algorithm <NUM>. In some embodiments, the controller activates one marker from each set of markers, in sequence until an activated marker is detected by the sensor. By activating one marker at a time, a face corresponding to the activated marker can be determined, as described below.

The method <NUM> also comprises receiving information about a detected activated marker (step <NUM>). The information may be received from the sensor configured to detect an activated marker. The sensor may be a camera such as, for example, an infrared camera or an optical camera. The information may include, for example, a detected position of the detected activated marker.

The method <NUM> also comprises causing the controller to activate the set of markers comprising the detected activated marker (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> described above with respect to activating one or more markers. In some embodiments, each marker of the set of markers may be activated in sequence. In other embodiments, the entire set of markers may be activated at the same time. The detected activated marker corresponds to a face of the plurality of faces and thus, the set of markers comprising the activated marker corresponds to the face as well. By activating the set of markers corresponding to the face, an orientation of the face may be determined, as described below.

The method <NUM> also comprises receiving information about the activated set of markers (step <NUM>). The information may be received from the sensor, which may be a camera such as, for example, an infrared camera or an optical camera. The information may include a detected position of each marker of the activated set of markers. The information may also include a pattern or arrangement of the activated set of markers.

In some embodiments, the information about the activated set of markers is information about the activated set of markers activated in step <NUM>. In other embodiments, the information about the activated set of markers may be obtained from other methods or steps described herein. In other words, the steps <NUM> and <NUM> below describing determining a pose of the tool may be applied to information about an activated set of markers obtained using methods other than the steps <NUM>-<NUM> described above. For example, the information may be received from the steps <NUM>-<NUM>, for example.

The method <NUM> also comprises determining a face of a plurality of faces having the at least one set of markers (step <NUM>). The face may be determined based on the detected position of each marker of the activated set of markers. Further, an orientation of the face may be determined based on the detected position of each marker of the activated set of markers. In embodiments where a corresponding set of markers is arranged in a unique pattern on each face, the face may also be determined based on the pattern of the activated set of markers.

The method <NUM> also comprises determining a pose of the tool and/or the robotic arm based on the information and the determined face (step <NUM>). More specifically, the pose of the tool and/or the robotic arm may be based on the determined face and the orientation of the determined face.

The method <NUM> also comprises validating a pose of the tool and/or robotic arm based on the determined pose of the tool and/or robotic arm (step <NUM>). The pose of the tool and/or the robotic arm may be validated by comparing the determined pose of the tool and/or robotic arm as determined in, for example, step <NUM> to pose information received from the robotic arm. In some embodiments, the robotic arm may include one or more sensors configured to sense a pose of the robotic arm and yield pose information. When the pose information received from the robotic arm matches the determined pose determined from the tracking device, then this indicates that the pose of the tool and/or robotic arm is validated. When the pose information received from the robotic arm does not match the determined pose determined from the tracking device, then this indicates that an error has occurred. The error may be related to, for example, a navigation system such as the navigation system <NUM> and/or a registration process.

The present disclosure encompasses embodiments of the method <NUM> that comprise more or fewer steps than those described above, and/or one or more steps that are different than the steps described above.

<FIG> depicts a method <NUM> that may be used, for example, for tracking multiple objects.

The method <NUM> (and/or one or more steps thereof) may be carried out or otherwise performed, for example, by at least one processor. The at least one processor may be the same as or similar to the processor(s) <NUM> of the computing device <NUM> described above. The at least one processor may be part of a robot (such as a robot <NUM>) or part of a navigation system (such as a navigation system <NUM>). A processor other than any processor described herein may also be used to execute the method <NUM>. The at least one processor may perform the method <NUM> by executing instructions such as the instructions <NUM> stored in a memory such as the memory <NUM>. The instructions may correspond to one or more steps of the method <NUM> described below. The instructions may cause the processor to execute one or more algorithms, such as a sequencing algorithm <NUM>.

The method <NUM> comprises causing a controller to selectively activate one or more markers of a first tracking device and one or more markers of a second tracking device (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> with respect to activating one or more markers. Each of the first tracking device and the second tracking device may be the same as or similar to the tracking device <NUM>, <NUM>, <NUM>. In some embodiments, the first tracking device is identical to the second tracking device. In other embodiments, the first tracking device may be different from the second tracking device. The controller may be the same as or similar to the controller <NUM>. In some embodiments, one controller may control each of the first tracking device and the second tracking device. In other embodiments, a first controller may control the first tracking device and a second controller may control the second tracking device.

The one or more markers of the first tracking device may be releasably or fixedly secured to the first tracking device in a first arrangement and the one or more markers of the second tracking device may be fixedly secured to the second tracking device in a second arrangement. In some embodiments, the first arrangement is the same as the second arrangement. In other embodiments, the first arrangement may be different from the second arrangement.

The method <NUM> also comprises causing the controller to activate the markers on the first tracking device in a first configuration (step <NUM>). In some embodiments, the first configuration may be a first sequence and the controller may use an algorithm such as the sequencing algorithm <NUM> to cause the markers on the first tracking device to activate in the first sequence. For example, each marker of the one or more markers on the first tracking device may be activated one at a time in sequence, two or more markers may be activated together in sequence, or two or more markers may be activated together in a pattern. In other embodiments, the first configuration may be a first wavelength. The wavelength may include, for example, infrared light, visible light, or ultraviolet light. In some embodiments, the wavelength may be between about <NUM> to <NUM>. In other embodiments, the wavelength may be about <NUM>. In still other embodiments, the wavelength may be less than <NUM> or greater than <NUM>. In still other embodiments, the wavelength may be any wavelength within the infrared spectrum. In some embodiments, the first configuration may include a combination of sequences and/or wavelengths.

The method <NUM> also comprises causing the controller to activate the markers on the second tracking device in a second configuration (step <NUM>). In some embodiments, the second configuration may be a second sequence and the controller may use an algorithm such as the sequencing algorithm <NUM> to cause the markers on the second tracking device to activate in the second sequence. The second sequence may be the same as or similar to the first sequence. In such instances, the second sequence may be activated before or after the first sequence to distinguish the second tracking device from the first tracking device. In other instances, the second sequence may be different from the first sequence. In such instances, the second sequence may be unique to the second tracking device so as to distinguish the second tracking device from the first tracking device. In other embodiments, the second configuration may be a second wavelength. The second wavelength may be the same as or similar to the first wavelength. In other instances, the second wavelength may be different from the first wavelength. In still other embodiments, the first configuration may include a combination of sequences and/or wavelengths.

In other examples, a pose of the first tracking device and the second tracking device may be determined from, for example, a first robotic arm and a second robotic arm that the first tracking device and the second tracking device are mounted to, respectively. As such, the second tracking device can be distinguished from the first tracking device based on a known pose of the first robotic arm and the second robotic arm. In such examples, the markers on each of the first tracking device and the second tracking device may be used to confirm an orientation of the first tracking device and the second tracking device.

Turning to <FIG>, methods <NUM>, <NUM>, <NUM> relate to determining a tool or object pose based on sequencing one or more markers on a tracking device to which the tool or object is coupled or mounted thereon. The methods <NUM>, <NUM>, <NUM> may be used, for example, in instances where a tracking device such as the tracking device <NUM>, <NUM>, <NUM> exits a field of view of a sensor such as the sensor <NUM> and re-enters the field of view. In such instances, the sensor <NUM> no longer has information describing which face of a plurality of faces of the tracking device is within the field of view. By sequencing the one or more markers in the methods <NUM>, <NUM>, <NUM> described below, a lock-on time of a sensor to the face of the tracking device may be reduced. Such reduction in lock-on time may reduce an overall time of a surgical procedure.

The method <NUM> comprises determining a predicted segment (or face) of a plurality of segments of a tracking device within a field of view of a sensor (step <NUM>). The tracking device may be the same as or similar to the tracking device <NUM>, <NUM>, <NUM>, the plurality of segments may be the same as or similar to the plurality of segments <NUM>, <NUM>, <NUM>, and the sensor may be the same as or similar to the sensor <NUM>. The tracking device may be supported by a robotic arm such as the robotic arm <NUM>. In some embodiments, the tracking device is mounted to, for example, an object or a tool supported by the robotic arm. Determining the predicted segment may be based on a known pose of the robotic arm and in some instances, a known pose of the sensor. More specifically, a field of view can be determined based on a known pose of the sensor and the predicted segment within the field of view can be determined based on the known pose of the robotic arm within the field of view.

The method <NUM> also comprises causing a controller to activate one or more markers of a plurality of markers on a tracking device (step <NUM>). The tracking device may be the same as or similar to the tracking device <NUM>, <NUM>, <NUM> and the plurality of markers may be the same as or similar to the plurality of markers, <NUM>, <NUM>, 404A, 404B. The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> with respect to activating one or more markers. The step <NUM> further includes activating one or more markers on the predicted segment identified in step <NUM> described above.

The method <NUM> also comprises identifying at least one marker of the one or more activated markers (step <NUM>). The identified at least one marker may be identified by the sensor, for example. Identifying the at least one marker may include identifying a position of the at least one marker.

The method <NUM> also comprises determining an actual segment based on the identified at least one marker (step <NUM>). The actual segment may be determined based the position of the at least one marker and may be based on a position of the tracking device. The actual segment corresponds to the segment in the field of view of the sensor.

The method <NUM> also comprises comparing the actual segment to the predicted segment (step <NUM>). In embodiments where the actual segment matches the predicted segment, such a match confirms that the predicted segment is in the field of view of the sensor and the sensor may then lock onto and track the tracking device. In other embodiments, where the actual segment does not match the predicted segment, the steps <NUM>-<NUM> may be repeated until the actual segment matches the predicted segments. In further embodiments, where the actual segment does not match the predicted segment, another method such as method <NUM> and/or <NUM> may be utilized to determine the pose of the tool or object.

Such method <NUM> may reduce a lock-on time of the sensor to the tracking device. For example, if the tracking device is no longer being tracked because the tracking device was no longer in a field of view of the sensor, then re-enters the field of view of the sensor, the method <NUM> may be utilized to determine a predicted segment within a field of view of the sensor and confirm that the predicted segment is in fact within the field of view of the sensor. Thus, the tracking device can be identified and tracked again.

The method <NUM> comprises causing a controller to activate one or more markers of a plurality of markers on a tracking device (step <NUM>). The tracking device may be the same as or similar to the tracking device <NUM>, <NUM>, <NUM> and the plurality of markers may be the same as or similar to the plurality of markers, <NUM>, <NUM>, 404A, 404B. The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> with respect to activating one or more markers. The step <NUM> further includes activating a single marker on each segment of a plurality of segments such as the plurality of segments <NUM>, <NUM>, <NUM> in sequence. By activating a single marker on each segment (until, for example, a marker is detected as described in step <NUM> below), a face corresponding to a detected single marker can be identified without activating each marker of the plurality of markers in sequence.

The method <NUM> also comprises receiving an indication from the sensor that an activated marker has been detected (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of step <NUM> with respect to identifying or detecting an activated marker.

The method <NUM> also comprises causing the controller to activate any remaining markers on the same segment as the activated marker (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> with respect to activating one or more markers. In some embodiments, the markers may be activated in sequence. In other embodiments, the markers may be activated at the same time. In yet other embodiments, some markers may be activated at the same time and other markers may be activated in sequence.

The method <NUM> also comprises determining an orientation of a tool based on the data from the sensor corresponding to the one or more markers (step <NUM>). The remaining markers may be activated in step <NUM> to determine an orientation of the segment and thus, an orientation of the tool. More specifically, the data received from the sensor may be a position of each marker. The position of each marker may then be used to determine the orientation of the segment, and thus, based on the segment and the orientation of the segment, the orientation of the tool may be determined.

The method <NUM> comprises causing a controller to activate all markers on a first set of segments of a plurality of segments (step <NUM>). The controller may be the same as or similar to the controller <NUM>. The plurality of segments may be the same as or similar to the plurality of segments <NUM>, <NUM>, <NUM>. The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> with respect to activating one or more markers. When the markers on the first set of segments are activated, the markers on the second set of segments may not be activated.

The first set of segments and a second set of segments may comprise all segments of the plurality of segments. In some embodiments each of the first set of segments and the second set of segments includes half of the segments of the plurality of segments. In other embodiments, the first set of segments may include more or fewer segments than the second set of segments.

The method <NUM> also comprises determining whether an activated marker has been detected (step <NUM>). Determining whether the activated marker has been detected may be based on data received from a sensor such as the sensor <NUM>. The sensor <NUM> may also determine a position of the detected activated marker.

The method <NUM> also comprises causing the controller to activate a first subset of markers on the first set of segments (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> described above with respect to activating one or more markers.

The first subset of markers on the first set of segments may be activated when an activated marker has been detected in step <NUM>, which indicates that markers on the first set of segments is within a field of view of the sensor. In such instances, markers on the first set of segments may be divided into the first subset and a second subset. By further dividing the markers, a face of a plurality of faces of the tool may be identified by honing in on markers in a field of view of the sensor. When the first subset of markers is activated, all other markers may not be activated (such as the second subset of markers and the markers of the second set of segments).

The method <NUM> also comprises causing the controller to activate a second subset of markers on the first set of segments (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> described above with respect to activating one or more markers.

The second subset of markers on the first set of segments may be activated when an activated marker is not detected in the first subset of markers activated in step <NUM>. When the second subset of markers on the first set of segments is activated, all other markers may not be activated (such as the first subset of markers and the markers of the second set of segments).

The method <NUM> also comprises causing the controller to activate all markers on a second set of segments of the plurality of segments (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> described above with respect to activating one or more markers.

The markers on the second set of segments may be activated when an activated marker has not been detected in step <NUM>. In other words, when an activated marker is not detected, this may indicate that a portion of the tracking device corresponding to the first set of segments is not in the line of sight or field of view of the sensor. Thus, a portion of the tracking device corresponding to the second set of segments may be in the light of sight or field of view of the sensor. When the markers on the second set of segments is activated, the markers on the first set of segments may not be activated.

The method <NUM> also comprises causing the controller to activate a first subset of markers on the second set of segments (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> described above with respect to activating one or more markers.

The first subset of markers on the second set of segments may be activated when an activated marker has been detected in step <NUM>. When the first subset of markers on the second set of segments is activated, all other markers may not be activated (such as a second subset of markers and the markers of the first set of segments).

The method <NUM> also comprises causing the controller to activate a second subset of markers on the second set of segments (step <NUM>). The step <NUM> may be the same as or similar to the step <NUM> of method <NUM> described above with respect to activating one or more markers.

The second subset of markers on the second set of segments may be activated when an activated marker is not detected in the first subset of markers activated in step <NUM>. When the second subset of markers on the second set of segments is activated, all other markers may not be activated (such as the first subset of markers and the markers of the first set of segments).

It will be appreciated that the steps <NUM>-<NUM> and/or the steps <NUM>-<NUM> may be repeated with smaller subsets until an individual segment of the plurality of segments is identified.

It will be appreciated that when no active markers are detected by a sensor using any of the methods <NUM>, <NUM>, <NUM>, <NUM>, <NUM> described above, that a line of sight to the sensor may be obstructed. In such instances, a notification may prompt a user (such as a surgeon or other medical provider) to check that the line of sight is unobstructed and clear.

As noted above, the present disclosure encompasses methods with fewer than all of the steps identified in <FIG>, <FIG>, <FIG>, <FIG> and <FIG> (and the corresponding description of the methods <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>), as well as methods that include additional steps beyond those identified in <FIG>, <FIG>, <FIG>, <FIG>, and <FIG> (and the corresponding description of the methods <NUM>, <NUM>, <NUM>, <NUM>, and <NUM>). The present disclosure also encompasses methods that comprise one or more steps from one method described herein, and one or more steps from another method described herein. Any correlation described herein may be or comprise a registration or any other correlation.

The foregoing is not intended to limit the disclosure to the form or forms disclosed herein. In the foregoing Detailed Description, for example, various features of the disclosure are grouped together in one or more aspects, embodiments, and/or configurations for the purpose of streamlining the disclosure. The features of the aspects, embodiments, and/or configurations of the disclosure may be combined in alternate aspects, embodiments, and/or configurations other than those discussed above. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed aspect, embodiment, and/or configuration. Thus, the following claims are hereby incorporated into this Detailed Description, with each claim standing on its own as a separate preferred embodiment of the disclosure.

Claim 1:
A system (<NUM>) for determining a tool pose of a tool, the system (<NUM>) comprising:
a tracking device (<NUM>, <NUM>, <NUM>) mounted to the tool and comprising:
a plurality of faces (<NUM>, 214A, 214B, 214C, 314A, 314B, <NUM>);
a plurality of markers (<NUM>) defining a plurality of sets of markers (<NUM>), each set of markers (<NUM>) comprising one or more markers (<NUM>, 216A, 216A,B, 216B,C, 216C, 316A, 316A,B, 316B, <NUM>) of the plurality of markers (<NUM>), each set of markers (<NUM>) disposed on a corresponding face (<NUM>, 214A, 214B, 214C, 314A, 314B, <NUM>);
at least one processor (<NUM>); and
a memory (<NUM>) storing data for processing by the at least one processor (<NUM>) that, when processed, the data causes the at least one processor (<NUM>) to:
receive information about one of the sets of markers (<NUM>) of the plurality of markers (<NUM>),
determine the face (<NUM>, 214A, 214B, 214C, 314A, 314B, <NUM>) of the plurality of faces (<NUM>, 214A, 214B, 214C, 314A, 314B, <NUM>) having the set of markers (<NUM>) disposed thereon, and
determine the tool pose based on the information and the determined face (<NUM>, 214A, 214B, 214C, 314A, 314B, <NUM>),
characterized in that
each adjacent set of markers (<NUM>) share at least two markers (216A,B, 216B,C, 316A,B), wherein the at least two markers being shared are included in both adjacent faces (<NUM>, 214A, 214B, 214C, 314A, 314B, <NUM>) corresponding to the adjacent set of markers (<NUM>).