Patent ID: 12193768

DETAILED DESCRIPTION

It should be understood that various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example or embodiment, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, and/or may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the disclosed techniques according to different embodiments of the present disclosure). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a computing device and/or a medical device.

In one or more examples, the described methods, processes, and techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors (e.g., Intel Core i3, i5, i7, or i9 processors; Intel Celeron processors; Intel Xeon processors; Intel Pentium processors; AMD Ryzen processors; AMD Athlon processors; AMD Phenom processors; Apple A10 or 10X Fusion processors; Apple A11, A12, A12X, A12Z, or A13 Bionic processors; or any other general purpose microprocessors), application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

Before any embodiments of the disclosure are explained in detail, it is to be understood that the disclosure is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. The disclosure is capable of other embodiments and of being practiced or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Further, the present disclosure may use examples to illustrate one or more aspects thereof. Unless explicitly stated otherwise, the use or listing of one or more examples (which may be denoted by “for example,” “by way of example,” “e.g.,” “such as,” or similar language) is not intended to and does not limit the scope of the present disclosure.

Turning first toFIG.1, a block diagram of a system100according to at least one embodiment of the present disclosure is shown. The system100may be used to process image data, execute a target detection algorithm, make a position and/or location determination, and/or carry out other aspects of one or more of the methods disclosed herein. The system100comprises a computing device102, an imaging device112, a navigation system114, and/or a robot126. The robot126may comprise a robotic arm that may be holding a reference target128, or on which a reference target128may be disposed. The reference target128may be disposed on an end of the robotic arm in some examples, while in other examples the reference target128may be disposed on any portion of the robotic arm. In some embodiments, the reference target128may be useful for aligning, mapping, or otherwise registering a robotic coordinate space with a patient coordinate space.

The computing device102comprises a processor104, a memory106, a communication interface108, and a user interface110. Systems such as the system100according to other embodiments of the present disclosure may comprise more or fewer components than the system100.

The processor104of the computing device102may be any processor described herein or any similar processor. The processor104may be configured to execute instructions stored in the memory106, which instructions may cause the processor104to carry out one or more computing steps utilizing or based on data received from the imaging device112, and/or the navigation system114.

The memory106may be or comprise RAM, DRAM, SDRAM, other solid-state memory, any memory described herein, or any other non-transitory memory for storing computer-readable data and/or instructions. The memory106may store information or data useful for completing any step of any of the methods200,400,500and/or600described herein. The memory106may store, for example, one or more target detection algorithms118, one or more image processing algorithms120, one or more position and/or location determination algorithms122, and/or one or more positioning instructions124. Such instructions or algorithms may, in some embodiments, be organized into one or more applications, modules, packages, layers, or engines. The algorithms and/or instructions may cause the processor104to manipulate data stored in the memory106and/or received from the imaging device112, and/or the navigation system114.

The computing device102may also comprise a communication interface108. The communication interface108may be used for receiving image data or other information from an external source (such as the imaging device112, the robot126, and/or the navigation system114), and/or for transmitting instructions, images, or other information to an external source (e.g., the navigation system114, another computing device102, and/or the robot126). The communication interface108may comprise one or more wired interfaces (e.g., a USB port, an ethernet port, a Firewire port) and/or one or more wireless interfaces (configured, for example, to transmit information via one or more wireless communication protocols such as 802.11a/b/g/n, Bluetooth, NFC, ZigBee, and so forth). In some embodiments, the communication interface108may be useful for enabling the device102to communicate with one or more other processors104or computing devices102, whether to reduce the time needed to accomplish a computing-intensive task or for any other reason.

The computing device102may also comprise one or more user interfaces110. The user interface110may be or comprise a keyboard, mouse, trackball, monitor, television, touchscreen, and/or any other device for receiving information from a user and/or for providing information to a user. The user interface110may be used, for example, to receive a user selection or other user input regarding first instructions configured to cause a robot to position a reference target; to receive a user selection or other user input regarding determining whether all, some, or none of a reference target is represented in the image data; to receive a user selection or other user input regarding determining whether all, some, or none of an anatomical element is represented in the image data; to receive user input regarding generating a repositioning requirement based on the imaging device, the reference target, and/or the anatomical element; to receive a user selection or other user input regarding second instructions for repositioning at least one of the imaging device and the reference target; to display the image data received from the imaging device112; to display the first instructions; and/or to display second instructions for moving the imaging device112or causing the imaging device112to move, and/or configured to cause the reference target128to move (e.g., by causing the robotic arm supporting the reference target128to move). In some embodiments, the user interface110may be useful to allow a surgeon or other user to modify the first instructions, the second instructions, or other information displayed.

Although the user interface110is shown as part of the computing device102, in some embodiments, the computing device102may utilize a user interface110that is housed separately from one or more remaining components of the computing device102. In some embodiments, the user interface110may be located proximate one or more other components of the computing device102, while in other embodiments, the user interface110may be located remotely from one or more other components of the computer device102.

The imaging device112is operable to image an anatomy of a patient (e.g., a spine region) to yield image data (e.g., image data depicting or corresponding to a spinal column of a patient). “Image data” as used herein refers to the data generated or captured by an imaging device, including in a machine-readable form, a graphical form, and in any other form. In various examples, the image data may correspond to a complete anatomical feature of a patient, or to a portion thereof (e.g., the entire spinal column of the patient or to a portion thereof). The imaging device112may be a device for obtaining X-ray images and/or image data (e.g., a fluoroscope, a CT scanner, or other X-ray machine), but may alternatively be a magnetic resonance imaging (MRI) scanner, an ultrasound scanner, an optical computed tomography scanner, or any other imaging device suitable for obtaining images of an anatomical element of a patient. The imaging device112may be, for example, a C-arm imaging device. The imaging device112may take an image from an underside of the patient, in which event the X-rays or other electromagnetic signals or waves pass through the patient first and then through the reference target128positioned above the patient.

During the surgical operation, the navigation system114may provide navigation for a surgeon and/or a surgical robot. In other embodiments, the surgical operation may not use a navigation system. The navigation system114may be any known or future navigation system, including, for example, the Medtronic StealthStation™ S8 surgical navigation system. In various embodiments, the navigation system114may be used to track a position of the imaging device112(or, more particularly, of a navigated tracker attached to the imaging device112), and of the reference target128(which may be attached to the robot126or may be separate from the robot126). The navigation system114may include a camera or other sensor(s) for tracking one or more reference markers, navigated trackers, or other objects within the operating room. The navigation system114may also track one or more reference targets in some embodiments. The navigation system114may include a display for displaying one or more images from an external source (e.g., the computing device102, imaging device112, or other source) or a video stream from the camera or other sensor of the navigation system114. The navigation system114may be, for example, the same as or similar to a navigation system described by U.S. Pat. No. 7,366,562, entitled “Method and Apparatus for Surgical Navigation, filed Oct. 17, 2003, and assigned to Medtronic Navigation Inc., the entirety of which is hereby incorporated by reference herein.

Turning now toFIG.2, a method200for aligning an imaging device112in connection with a surgical procedure on an anatomical feature of a patient according to embodiments of the present disclosure may be executed in whole or in part on a computing device102. The alignment is performed during initial setup for a surgical operation. The surgery may be performed by a surgical robot, a surgeon, or a combination of both.

The method200comprises transmitting positioning instructions124comprising first instructions configured to cause the robot126to position the reference target128proximate a body of a patient (step202), and more specifically proximate a portion of the body of the patient containing an anatomical element that is the subject of the planned surgery. The reference target128may also be positioned directly on the patient. The reference target128may be or comprise one or more optical targets that can be detected in image data. In other embodiments, the reference target128may be navigation markers or spheres, and may be configured for detection by a navigation system such as the navigation system114, which may be an optical (visual wavelength) navigation system, an infrared navigation system, an electromagnetic navigation system, or any other navigation system. The first instructions may be transmitted to the robot126from the communication interface108to cause the robot126to automatically position the reference target128. In other examples, the first instructions may be displayed on the user interface110, and a user may cause the robot126to move the robotic arm, and thus the reference target128, based on the displayed first instructions. The first instructions may be based on a surgical plan and/or may be based on preoperative images taken prior to the procedure. Additionally or alternatively, the first instructions may be based on user-provided input regarding an observed location (or at least an estimated location based on one or more observations of the patient) of the anatomical feature that is the subject of the planned surgery.

The method200further comprises receiving image data from the imaging device112(step204). The image data may, for example, be machine-readable and may be useful for displaying an image on a user interface110or elsewhere. The image data may correspond to (e.g., may contain data representing) the anatomical feature that is the subject of the planned surgery, which may be any anatomical element—for example, a spinal column or spinal element, an appendage, a cranial element, or the like. In some examples, the image data may comprise or correspond to a two-dimensional image. The image data may correspond to an image taken of the spinal column of the patient using an imaging device112, such as an MRI scanner, a CT scanner, a fluoroscopy device, or another imaging device. In various embodiments, the image data may contain data for an entire spinal column of the patient or for a portion of the spinal column of the patient. The image data is generated by the imaging device112, but may be received directly from the imaging device112or indirectly via any other source. The image data may be received via the communication interface108. Processing of the image data may include applying the image processing algorithm120to the image data (or, alternatively, inputting the image data to the image processing algorithm120), which algorithm120may apply one or more filters to the image data300to prepare the image data300for further processing.FIGS.3A-3Dprovide a visual depiction of four different possible sets of image data300.

The method200further comprises determining whether all, some, or none of an anatomical element is represented in the image data300to yield a first determination (step206). The anatomical element302, as shown inFIGS.3A-3D, may be identified by executing the image processing algorithm120by the processor104. The image processing algorithm120may, in some embodiments, use a neural network, machine learning, artificial intelligence, or the like, to process the image data300. Identifying the anatomical element302may include segmenting the image data300and evaluating the resulting segments. In some embodiments, the image processing algorithm120may comprise a feature identification algorithm that identifies objects in the image data300and compares them to one or more known shapes to determine whether the identified objects correlate to a known shape and can therefore be identified as a known anatomical element302. In other embodiments, the image processing algorithm120may be generated by a machine learning engine based on training data. The training data may be or comprise, for example, a plurality of images of anatomical features that have been marked so that the machine learning engine can identify correlations between different images of the same anatomical feature and thus learn to identify the anatomical feature in question.

In other embodiments, a surgeon or other user may identify the anatomical element302by providing one or more inputs via a user interface110. In such embodiments, the identification of the anatomical element302may be based on the image data300and/or additional information obtained from the user.

The first determination corresponds to whether some, all, or none of the anatomical element302is represented in the image data300. The first determination also corresponds to whether the imaging device112is misaligned. For example, if some or none of the anatomical element302is not represented in the image data300, as shown inFIGS.3C and3D, then the imaging device112is not properly aligned. Conversely, if all of the anatomical element302is in the image data300, as shown inFIGS.3A and3B, then the imaging device112is properly aligned.

The method200further comprises determining whether all, some, or none of the reference target128is represented in the image data300to yield a second determination (208). The reference target128may be identified using a target detection algorithm118executed by the processor104. In some embodiments, the algorithm118may detect the reference target128by comparing the image data300to preoperative image data with or without a reference target128. In other embodiments, the algorithm118may identify one or more shapes corresponding to the reference target128in the image data300based on input to the algorithm118regarding the shape(s) of the reference target. For example, the algorithm118may receive input (e.g., from a user via the user interface110) that the reference target128is a square and may search for a square or substantially square shape in the image data300. In other embodiments, the shape may include, but is not limited to, a circle, an oval, a start, a pentagon, or the like.

In still other embodiments, the target detection algorithm118may search the image data300for data that corresponds to a readily distinguishable objects included in the reference target128. Thus, for example, if a reference target128is known to comprise an oblong shape, then the target detection algorithm118may analyze the image data300to determine whether some or all of the oblong shape is represented in the image data300. If the oblong shape is represented in the image data300, the target detection algorithm118may determine that the entire reference target128is in the image data300. If some of the oblong shape is in the image data300, then the target detection algorithm118may determine that only a portion of the reference target128is in the image data300. And, if the oblong shape is not represented in the image data300, the target detection algorithm118may determine that the reference target128does not appear in the image data300.

Similarly, in other embodiments, if a reference target128is known to comprise four spheres mounted to a frame, then the target detection algorithm118may analyze the image data300to determine whether four spheres are represented in the image data300. If all four spheres are represented in the image data300, the target detection algorithm118may determine that the entire reference target128is in the image data300. If at least one but fewer than four spheres are in the image data300, then the target detection algorithm118may determine that only a portion of the reference target128is in the image data300. And, if no spheres are represented in the image data300, the target detection algorithm118may determine that the reference target128does not appear in the image data300.

In still other embodiments, a surgeon or other user may identify the reference target128in the image data300by providing one or more inputs via a user interface110. In such embodiments, the identification of the reference target128may be based on the image data and/or additional information obtained from user.

The second determination corresponds to whether some, all, or none of the reference target128is represented in the image data300. The second determination also corresponds to whether the reference target128is misaligned. For example, if less than all of the reference target128is represented in the image data300, as shown inFIGS.3B and3D, then the reference target128is not properly aligned. Conversely, if all of the reference target128is represented in the image data300, as shown inFIGS.3A and3C, then the reference target128is properly aligned.

The method200also comprises generating a repositioning requirement based on the first determination and the second determination (step210). The repositioning requirement may be calculated using the repositioning algorithm122. The algorithm122determines if the imaging device112and/or the reference target128requires repositioning based on whether the imaging device112and/or the reference target128are misaligned, and if so, calculates a new position for the imaging device112and/or the reference target128. The repositioning requirement may be based on a geometric comparison of a position of the imaging device112(which is movable), the anatomical element (which is stationary), and a position of the reference target128(which is movable).

The repositioning requirement may be based at least in part on a portion of the anatomical element302or the reference target128that is not represented in the image data300. For example, a portion of the anatomical element302(e.g., a spine region) inFIGS.3C and3Dis not represented in the image data300. Based on a portion of the anatomical element302that is represented in the image data300, a direction or predicted position of the portion of the anatomical element302not represented in the image300can be determined based on the known portion of the anatomical element302that is represented in the image data300. For example, inFIGS.3C and3D, the portion of the anatomical element302not represented in the image data300is to a left side of a frame of the image data300and thus, the predicted position of the portion not represented in the image data300would be to the left of the frame. The algorithm122can calculate a needed change in position or new position for the imaging device112based on the direction or predicted position of the portion of the anatomical element302not represented in the image data300.

In another example, generating the repositioning requirement includes determining a needed change in position or new position of the reference target128. For example, a portion of the reference target128inFIGS.3B and3Dis not represented in the image data300. Based on the portion of the reference target128that is represented in the image data300, a direction or predicted position of the portion not represented in the image data300can be generated. For example, inFIGS.3B and3D, the portion of the reference target128not represented in the image data300is to a right side of a frame of the image data300and thus, the predicted position of the portion not represented in the image data300would be to the right of the frame. As a result, movement of the reference target128to the left would enable the reference target128to be positioned entirely within the image data300. The algorithm122can calculate the repositioning requirement of the reference target128based on the predicted position of the portion of the reference target128not represented in the image data300. In some embodiments, the repositioning requirement may reflect a needed change in position of the reference target128toward the center of an image represented by the image data300.

The repositioning requirement may correspond to both the reference target128and the imaging device112when neither the reference target128nor the anatomical element302are identified or visible in the image data300or, as shown inFIG.3D, when the image data300lacks a portion of both the reference target128and the anatomical element302. In such circumstances, the repositioning requirement identifies a needed change in position or new position for both the imaging device112and the reference target128.

The method200further comprises transmitting positioning instructions124comprising second instructions for repositioning the imaging device112and/or the reference target128based on the repositioning requirement (step212). As explained above, the repositioning requirement may identify a needed change in position or new position for the imaging device112and/or the reference target128. The second instructions may be configured to reposition the imaging device112based on the repositioning requirement. Additionally or alternatively, the second instructions may be configured to reposition the reference target128based on the repositioning requirement. In some embodiments, the second instructions are configured to reposition the imaging device112first, until all of the anatomical element302is represented in the image data300, and then to reposition the reference target128until all of the reference target128is included in the image data300. In other embodiments, the second instructions are configured to reposition the reference target128first, then the imaging device112.

The repositioning of the imaging device112and/or of the reference target128may occur iteratively by repeating the steps204through212until an entirety of both the anatomical element302and the reference target128is represented in the image data.

If an entirety of both the anatomical element302and the reference target128are represented in the image data300, then no repositioning requirement is calculated and alignment of the imaging device112and the reference target128is complete.

In various examples, the second instructions may be machine readable to cause both the imaging device112to automatically move and/or to cause the robot126to automatically move the reference target128. In other examples, the second instructions may comprise both human-readable instructions displayed on the user interface110instructing the user to move (and, in some embodiments, how to move) the imaging device112, and machine-readable instructions to cause the robot126to automatically move the reference target128. In still other examples, the second instructions may comprise both machine-readable instructions to cause the imaging device112to automatically move and human-readable instructions displayed on the user interface110instructing the user to move (and, in some embodiments, how to move) the robot126and/or the reference target128. In still further examples, the second instructions may comprise human-readable instructions displayed on the user interface110instructing the user to move (and, in some embodiments, how to move) the imaging device112and the robot126and/or the reference target128.

As may be appreciated based on the foregoing disclosure, if one of the imaging device112and the reference target128is fully represented within the image data300, then the second instructions may not include instructions for moving the one of the imaging device112and the reference target128. On the other hand, if for example the anatomical element302is not fully represented in the image data300and the reference target128is fully represented in the image data300, but movement of the imaging device112to capture the entirety of the anatomical element302may or will cause some or all of the reference target128to not be represented in the image data300, then the second instructions may include instructions for moving both the imaging device112and the reference target128.

Turning now toFIG.4, a method400of further aligning the reference target128comprises receiving or obtaining a second set of image data (step402). The received second image data may be received or obtained in the same manner as or in a similar manner to the image data received in step204of the method200. The second image data may correspond to the anatomical feature302that is the subject of the planned surgery, which may be any anatomical element—for example, a spinal column or spinal element, an appendage, a cranial element, or the like. In some examples, the second image data may comprise or correspond to a two-dimensional image. The second image data may correspond to a second image taken of the spinal column of the patient using the imaging device112, such as an MRI scanner, a CT scanner, a fluoroscopy device, or another imaging device. In various embodiments, the second image data may contain data for an entire spinal column of the patient or for a portion of the spinal column of the patient. The second image data is generated by the imaging device112, but may be received directly from the imaging device112or indirectly via any other device or system (e.g., a database, the cloud or another network), and may be received via the communication interface108. Processing of the second image data may include applying the image processing algorithm120, which may apply one or more filters to the second image data to prepare the second image data for further processing.

The method400also comprises determining that only a portion or none of the reference target128is not represented within the second image data to yield a third determination (step404). The step404may be the same as or similar to the step208of the method200. The reference target128can be identified using the target detection algorithm118. For example, in some embodiments, the algorithm118may detect the reference target128by comparing the second image data to preoperative image data without a reference target128. In other embodiments, the algorithm118may identify a shape corresponding to the reference target128in the second image data based on input to the algorithm118regarding the shape. For example, the algorithm118may receive input (e.g., from a user via the user interface110) that the reference target128is a square and may search for a square or substantially square shape in the second image data. In other embodiments, the shape may include, but is not limited to, a circle, an oval, a start, a pentagon, or the like.

In still other embodiments, the target detection algorithm118may search the second image data for data that corresponds to a readily distinguishable objects included in the reference target128. Thus, for example, if a reference target128is known to comprise an oblong shape, then the target detection algorithm118may analyze the image data300to determine whether some or all of the oblong shape is represented in the image data300. If the oblong shape is represented in the image data300, the target detection algorithm118may determine that the entire reference target128is in the image data300. If some of the oblong shape is in the image data300, then the target detection algorithm118may determine that only a portion of the reference target128is in the image data300. And, if the oblong shape is not represented in the image data300, the target detection algorithm118may determine that the reference target128does not appear in the image data300.

Similarly, in other embodiments, if a reference target128is known to comprise four navigation spheres mounted to a frame, then the target detection algorithm118may analyze the image data to determine whether four navigation spheres are represented in the image data. If all four navigation spheres are represented in the data, the target detection algorithm118may determine that the entire reference target128is in the image. If at least one but fewer than four navigation spheres are in the image, then the target detection algorithm118may determine that only a portion of the reference target128is in the image. And, if no navigation spheres are represented in the data, the target detection algorithm118may determine that the reference target128does not appear in the image.

In still other embodiments, a surgeon or other user may identify the reference target128by providing one or more inputs via a user interface110. In such embodiments, the identification of the reference target128may be based on the second image data and/or additional information obtained from user.

The third determination corresponds to whether only a portion or none of the reference target128is represented in the second image data and whether the reference target128is misaligned. If the reference target128is not entirely represented in the second image data, as shown inFIGS.3B and3D, then the reference target128is not properly aligned.

The method400also comprises generating, based on the third determination, a second repositioning requirement (step406). The step406may be the same as or similar to the step210of the method200. The second repositioning requirement can be generated using the repositioning algorithm122(e.g., in one or more of the ways described above with respect to the step210of the method200). The second repositioning requirement includes repositioning the reference target128in a direction of a portion of the reference target128not represented in the image data300. For example, a portion of the reference target128inFIGS.3B and3Dare not represented in the image data300. Based on a portion of the reference target128that is represented in the image data, a direction or predicted position of the portion not represented in the image data300can be generated based on the portion of the reference target128that is represented in the image data300. For example, inFIGS.3B and3D, the portion of the reference target128not represented in the image data300is to a right side of a frame of the image data300and thus, the predicted position of the portion not represented in the image data300would be to the right of the frame. The algorithm122can calculate the repositioning requirement of the reference target128based on the predicted position of the portion of the reference target128not represented in the image data300, and/or based on a change in position of the reference target128that would place the reference target128closer to the center of an image represented by the second image data.

The method400further comprises transmitting positioning instructions124comprising third instructions for repositioning the reference target128(step408). The step408may be the same as or similar to the step212of the method200. In some examples, the third instructions may be configured to automatically cause the robot126to reposition the reference target128. In other embodiments, the third instructions may be displayed on a user interface110or other device so that a user can manually reposition the reference target128(and the robotic arm to which the reference arm128is attached). In various embodiments, the second instructions are configured to reposition the reference target128toward a center of an image represented by the second image data.

The method400may be repeated until the reference target128is properly aligned, as indicated by identification of the reference target128in its entirety in the second image data or subsequent image data.

Turning now toFIG.5, a method500of further aligning the imaging device112includes receiving or obtaining second image data (step502). The receiving or obtaining of the second image data may be accomplished in the same manner as or in a similar manner to step402of the method400and/or step204of the method200. The second image data may correspond to the anatomical feature302that is the subject of the planned surgery, which may be any anatomical element—for example, a spinal column or spinal element, an appendage, a cranial element, or the like. In some examples, the second image data may comprise or correspond to a two-dimensional image. The second image data may correspond to a second image taken of the spinal column of the patient using the imaging device112, such as an MM scanner, a CT scanner, a fluoroscopy device, or another imaging device. In various embodiments, the second image data may contain data for an entire spinal column of the patient or for a portion of the spinal column of the patient.

The second image data is generated by the imaging device112, but may be received directly from the imaging device112or indirectly via any other device or system (e.g., a database, the cloud or another network), and may be received via the communication interface108. Processing of the second image data may include applying the image processing algorithm120, which may apply one or more filters to the second image data to prepare the second image data for further processing.

The method500also comprises determining that only a portion or none of the anatomical element302is represented in the second image data300to yield a third determination (step504). The step504may be the same as or similar to the step206of the method200. As in the step206described above, the anatomical element302, as shown inFIGS.3A-3D, may be identified by executing the image processing algorithm120using the processor104. The image processing algorithm120may, in some embodiments, use a neural network, machine learning, artificial intelligence, or the like, to process the image data. Identifying the anatomical element302may include segmenting the image data and evaluating the resulting segments. In some embodiments, the image processing algorithm120may comprise a feature identification algorithm that identifies objects in the image data and compares them to one or more known shapes to determine whether the identified objects correlate to a known shape and can therefore be identified as a known anatomical element302. In other embodiments, the image processing algorithm120may be generated by a machine learning engine based on training data. The training data may be or comprise, for example, a plurality of images of anatomical features that have been marked so that the machine learning engine can identify correlations between different images of the same anatomical feature and thus learn to identify the anatomical feature in question.

In other embodiments, a surgeon or other user may identify the anatomical element302by providing one or more inputs via a user interface110. In such embodiments, the identification of the anatomical element302may be based on the image data and/or additional information obtained from the user.

The third determination corresponds to whether only a portion or none of the anatomical element302is represented in the image data300. The third determination also corresponds to whether the imaging device112is misaligned. If some or none of the anatomical element302is not represented in the image data300, as shown inFIGS.3C and3D, then the imaging device112is not properly aligned.

The method500also comprises generating, based on the third determination, a second repositioning requirement (step506). The step506may be the same as or similar to the step210of the method200. The second repositioning requirement can be generated using the repositioning algorithm122(e.g., in one or more of the ways described above with respect to the step210of the method200). The second repositioning requirement may be based on a portion of the anatomical element302or the reference target128that is not represented in the image data. For example, a portion of the anatomical element302(e.g., a spine region) inFIGS.3C and3Dis not represented in the second image data. Based on a portion of the anatomical element302that is represented in the second image data, a direction or predicted position of the portion of the anatomical element302not represented in the second image can be generated based on the known portion of the anatomical element302that is represented in the second image data. For example, inFIGS.3C and3D, the portion of the anatomical element302not represented in the second image data is to a left side of a frame of the second image data and thus, the predicted position of the portion not represented in the second image data would be to the left of the frame. The algorithm122can calculate the repositioning requirement of the imaging device112based on the direction or predicted position of the portion of the anatomical element302not represented in the second image data.

The method500further comprises transmitting positioning instructions124comprising third instructions for repositioning the imaging device based on the second repositioning requirement (step508). The step508may be the same as or similar to the step212of the method200. In some examples, the third instructions may be configured to automatically cause the imaging device112to be repositioned. In other embodiments, the third instructions may be displayed on a user interface110or other device so that a user can manually reposition the imaging device112. In various embodiments, the third instructions are configured to reposition the imaging device112based on a portion of the anatomical element302not represented in the second image data, as described above with respect to step210.

The method500may be repeated until the imaging device112is properly aligned, as indicated by identification of the anatomical element302in its entirety in the second image data or subsequent image data.

Turning now toFIG.6, a method600of further aligning the imaging device112and the reference target128includes receiving or obtaining second image data (step602). The receiving or obtaining the second image data may be accomplished in the same manner as or in a similar manner to step502of the method500, step402of the method400, and/or step204of the method200. The second image data may correspond to the anatomical feature302that is the subject of the planned surgery, which may be any anatomical element—for example, a spinal column or spinal element, an appendage, a cranial element, or the like. In some examples, the second image data may comprise or correspond to a two-dimensional image. The second image data may correspond to a second image taken of the spinal column of the patient using the imaging device112, such as an MM scanner, a CT scanner, a fluoroscopy device, or another imaging device. In various embodiments, the second image data may contain data for an entire spinal column of the patient or for a portion of the spinal column of the patient. The second image data is generated by the imaging device112, but may be received directly from the imaging device112or indirectly via any other device or system (e.g., a database, the cloud or another network), and may be received via the communication interface108. Processing of the second image data may include applying the image processing algorithm120, which may apply one or more filters to the second image data to prepare the second image data for further processing.

The method600also comprises determining that only a portion or none of both the reference target128and the anatomical element302is represented in the second image data to yield a third determination (step604). A lack of some or all of the anatomical element302can be identified using the image processing algorithm120(e.g., in one or more of the ways described above with respect to step206of the method200or step504of the method500) and a lack of some or all of the reference target128can be identified using the target detection algorithm118(e.g., in one or more of the ways described above with respect to step208of the method200and/or step404of the method400).

The third determination corresponds to whether only a portion or none of both the anatomical element302and the reference target128is represented in the second image data. The third determination also corresponds to whether the imaging device112and the reference target128is misaligned. If only a portion or none of both the anatomical element302and the reference target128is represented in the second image data, as shown inFIG.3D, then the imaging device112and the reference target128are not properly aligned.

The method600also comprises generating, based on the third determination, a second repositioning requirement (step606). The step606may be the same as or similar to the step210of the method200. For example, the second repositioning requirement can be generated using the repositioning algorithm122. As another example, the second repositioning requirement may be based on a geometric comparison of a position of the imaging device112(which is movable), the anatomical element (which is stationary), and a position of the reference target128(which is movable).

As yet another example, the second repositioning requirement may correspond to both the reference target128and the imaging device112when neither the reference target128nor the anatomical element302are identified or visible in the second image data or, as shown inFIG.3D, when the second image data lacks a portion of both the reference target128and the anatomical element302.

The method600further comprises transmitting positioning instructions124comprising third instructions for repositioning the imaging device112and/or the reference target128based on the repositioning requirement (step608). The step608may be the same as or similar to the step212of the method200. In some embodiments, the third instructions are configured to reposition the imaging device112first, until all of the anatomical element302is represented in the second image data, and then to reposition the reference target128until all of the reference target128is included in the second image data. In other embodiments, the third instructions are configured to reposition the reference target128first, then the imaging device112.

In various examples, the third instructions may be machine readable to cause both the imaging device112to automatically move and/or to cause the robot126to automatically move the reference target128. In other examples, the third instructions may comprise both human-readable instructions displayed on the user interface110instructing the user to move (and, in some embodiments, how to move) the imaging device112, and machine-readable instructions to cause the robot126to automatically move the reference target128. In still other examples, the third instructions may comprise both machine-readable instructions to cause the imaging device112to automatically move and human-readable instructions displayed on the user interface110instructing the user to move (and, in some embodiments, how to move) the robot126and/or the reference target128. In still further examples, the third instructions may comprise human-readable instructions displayed on the user interface110instructing the user to move (and, in some embodiments, how to move) the imaging device112and the robot126and/or the reference target128.

The method600may be repeated until the imaging device112and the reference target128are properly aligned, as indicated by identification of the reference target128and the anatomical element302in their entirety in the second image data or subsequent image data.

The methods and systems described provide an efficient method for aligning an imaging device for robotic surgery. Further, the methods and systems described herein reduce the amount of imaging needed for each iteration, thereby reducing the amount of radiation to which a patient is exposed. The method is simple to implement and portions of or the entire method may be automated, thereby reducing initial setup time and overall operation time.

As may be appreciated based on the foregoing disclosure, the present disclosure encompasses methods with fewer than all of the steps identified inFIGS.2and4-6(and the corresponding description), as well as methods that include steps from more than one ofFIGS.2and4-6(and the corresponding description) and methods that include one or more steps disclosed herein in combination with one or more steps not disclosed herein.

One or more aspects of the present disclosure may be the same as or similar to one or more corresponding aspects described in U.S. patent application Ser. No. 16/854,011, filed contemporaneously herewith by the same applicant, entitled “System and Method for Positioning an Imaging Device” and naming the same inventor as the present application, which is hereby incorporated herein by reference in its entirety.

The foregoing discussion has been presented for purposes of illustration and description. The foregoing is not intended to limit the disclosure to the form or forms disclosed herein. In the foregoing Detailed Description, for example, various features of the disclosure are grouped together in one or more aspects, embodiments, and/or configurations for the purpose of streamlining the disclosure. The features of the aspects, embodiments, and/or configurations of the disclosure may be combined in alternate aspects, embodiments, and/or configurations other than those discussed above. This method of disclosure is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed aspect, embodiment, and/or configuration. Thus, the following claims are hereby incorporated into this Detailed Description, with each claim standing on its own as a separate preferred embodiment of the disclosure.

Moreover, though the description has included description of one or more aspects, embodiments, and/or configurations and certain variations and modifications, other variations, combinations, and modifications are within the scope of the disclosure, e.g., as may be within the skill and knowledge of those in the art, after understanding the present disclosure. It is intended to obtain rights which include alternative aspects, embodiments, and/or configurations to the extent permitted, including alternate, interchangeable and/or equivalent structures, functions, ranges or steps to those claimed, whether or not such alternate, interchangeable and/or equivalent structures, functions, ranges or steps are disclosed herein, and without intending to publicly dedicate any patentable subject matter.