Method For Stylus And Hand Gesture Based Image Guided Surgery

A system is disclosed that allows for determining a position of an instrument in an object space. The position may include a three-dimensional location and at least one degree of freedom of orientation, or any appropriate number of degrees of freedom. The tracked position may be based on imaging the instrument, including imaging an external contour of the instrument. The movement of the instrument can be determined based on a three-dimensional determination of a movement of the instrument contour

FIELD

The subject disclosure is related to determining a location of an instrument relative to an object, where the object may be a living or non-living object, and for displaying the location of the instrument relative to the object on a display device.

BACKGROUND

In performing a procedure on an object, a user may often need to move an instrument relative to the object while the instrument is covered or internal to the object. Various imaging techniques can be used to obtain images of an internal portion of an object, but generally optical systems or human eyesight cannot see through an opaque exterior of the object. Objects can include humans, fuselages, mechanical systems (e.g., engines, condensers, and other systems) that include internal components that may require maintenance over time. It may be desirable, therefore, to have a system that allows for determining the location of an instrument relative to an internal component of the object based upon an image that is acquired with an imaging system that is able to image an internal portion of the object.

SUMMARY

A system is disclosed that allows for determining the location of an instrument relative to an object space (which also may be referred to as patient space or subject space) which includes a three-dimensional location and three-dimensional orientation, or any appropriate number of dimensions, in real space. The position of an instrument can be tracked with a tracking localizer that allows for determining the location of the instrument within the object by tracking at least a portion of the instrument or a tracking device connected to the instrument. Further, the tracking system can be used to illustrate a projected line from the instrument into the object based upon the current tracked position of the object. Further, it is understood, that the position of the instrument can include both a location that can include a three-dimensional coordinate location of the instrument and an orientation that can include a six degree of freedom orientation at the tracked location. The combination of the location and the orientation may be referred to as a position of the instrument, which can be determined with the tracking system, as discussed further herein.

The tracking system may include a substantially portable localizer element system that may be selectively connected to a mounting or holding system that is associated with the object. By positioning the localizer relative to the object, the localizer can be used to track the position of the instrument relative to the object. When tracked, the position of the instrument may be displayed with a display device relative to an image of the object, including internal portions of the object. The localizer can be a relatively small and movable system to assist in determining a position of the instrument. The localizer may operate by various techniques such as an optical tracking technique that may use stereoscopic cameras or multiple camera lenses to image the instrument in space to allow for a determination of the position of the instrument.

DETAILED DESCRIPTION

A tracking system that may register image space (generally defined by an image that is displayed) to object space (generally the space defined in and around a selected object) can include a STEALTHSTATION® TRIA®, TREON®, and/or S7™ Navigation System having an optical localizer, similar to the optical localizer40, sold by Medtronic Navigation, Inc. having a place of business in Louisville, Colo. In various embodiments, object or subject space and image space can be registered by identifying matching points or fiducial points in the object space and related or identical points in the image space. When the position of an imaging device (not illustrated) is known, either through tracking or its “known” position (e.g. O-arm® imaging device sold by Medtronic, Inc.), or both relative to the object during imaging, the image data is generated at a precise and known position. This can allow image data that is automatically or “inherently registered” to the object being imaged upon acquisition of the image data.

Alternatively, manual or automatic registration can occur by matching fiducial points in image data with fiducial points on the object. Registration of image space to object space allows for the generation of a translation map between the object space and the image space. According to various embodiments, registration can occur by determining points that are substantially identical in the image space and the object space. The identical points can include anatomical fiducial points or implanted fiducial points. Exemplary registration techniques are disclosed in U.S. Pat. No. 8,842,893 and U.S. Pat. App. Pub. No. 2010/0228117, both incorporated herein by reference.

Once registered, a navigation system, as discussed herein, can be used to perform selected procedures. Selected procedures can use the image data generated or acquired with the imaging system. Further, the imaging system can be used to acquire image data at different times relative to a procedure. As discussed herein, image data can be acquired of the object subsequent to a selected portion of a procedure for various purposes, including confirmation of the portion of the procedure.

According to various embodiments, an object20can be placed in a three-dimensional space. The space contained within the object and a portion of space near the object may be referred to as an object space. The object20may define or include an interior volume22which may include an internal object or component24. An opening or portal26may be provided or made in the object20into which an instrument30may be inserted. The instrument30, which may be a stylus, a drill, an awl, etc., can be tracked relative to the object20with a localizer system40. The localizer system40can include various components including a first lens42and a second lens44. Both of the lenses42,44may be connected to a single camera or each of the lenses may be part of separate cameras, therefore two separate cameras may be included in the localizer40. Additionally, illumination structures or the like46can be provided that may illuminate the instrument30to assist in allowing the lenses42,44to capture an image of the instrument30.

The lenses42,44can image the instrument30and the objects20to determine a relative position. The two lenses42,44may view and be used to determine a depth or three-dimensional image or position of an instrument using stereoscopic techniques, generally understood in the art. Alternatively, as is generally understood in the art as discussed above, registration may occur between the object20in object space and an image of the object, including the internal object portion24. An image can include an image60illustrated on a display device66which may include a monitor screen of a system70, such as a computer system including a laptop computer, tablet computer, or the like.

The computer system may be a navigation computer70including at least a processor system70aand a memory70b. Both the memory system70band the processor system70amay be incorporated with the computer system or be accessed by the system70. Further, the processor70amay be a general purpose processor that executes instructions in the form of code to complete selected tasks. The processor70amay alternatively be an application specific integrated circuit (ASIC) or include portions that are application specific. The memory system70bmay be any appropriate type of memory such as a solid state, random access, removable disk, or the like.

The display66may also display an icon30′ that illustrates the tracked position of the instrument30′ relative to the object20as a position of the instrument icon30′ relative to the object, including the internal object24. The icon30′ of the instrument may substantially appear as the instrument on the display. Further, the icon30′ may include a three-dimensional rendering of the instrument30. Accordingly, a rendering or an icon24′ can be illustrated in the image60. Further, the icon30′ of the instrument can be superimposed on the object image24′ when it is determined that the instrument30is over or in contact with the object24. Further, a projected line, path or trajectory of the instrument30can be illustrated and superimposed on the object24′ as well.

In various embodiments, the localizer system40can include a motion tracking device that can track the instrument30or other appropriate portions without a specific tracking member affixed thereto. Various systems can include the LEAP MOTION® motion sensing device sold by Leap Motion, Inc. having a place of business at San Francisco, Calif., USA. Generally, the LEAP MOTION® motion sensing device includes a first camera lens and a second camera lens that may image or view an object in a field of view of the LEAP MOTION® motion sensing device. The LEAP MOTION® motion sensing device can identify the portion to be tracked and identify movements of the tracked portion in space. For example, the instrument30can be positioned relative to the localizer40, which may be the LEAP MOTION® motion sensing device, and the localizer can image the instrument30to track and/or identify the instrument and the navigation computer70can identify the instrument30in space and determine its position and movements in space. Software can be executed by the navigation computer70and include instructions embodied in code that is executed by the processor70a, the software may be stored on the memory70b.

The localizer device40, may further include illuminating elements46. The illuminating elements46may include infrared (IR) emitting elements. The emitting elements46may be appropriate elements, such as light emitting diodes (LEDs) or other appropriate emitting elements. The emitting elements46can insure proper illumination of the instrument30, or other appropriate portion to be tracked during a selected procedure. The localizer40may include sensors, such as sensors of the cameras, which are sensitive to IR wavelengths or other appropriate wavelength.

With reference toFIG. 2and various embodiments, including a specific example, a procedure may occur on a human subject or object100. The human subject or object100may be positioned in an operating room and on a table or support102. Further, the patient or subject100may be held or partially held in the selected orientation. For example, a head104of the subject100may be held with a holder or fixation system110. The holder110may include a MAYFIELD® skull clamp, such as those sold by Integra LifeSciences Corporation, that may be selectively attached to the head104to hold the head104in a selected position. Associated with the holder110may be the localizer40.

The localizer40may be similar to that described above, including the LEAP MOTION® motion sensing device as discussed above. The localizer40can be used to track a position of a surgical instrument or intervention instrument120. The intervention instrument120, which may be a stylus, a pointer, a drill, an awl, etc. may include a manipulable portion or handle122. The handle122may be held by a user, such as a physician130. Further, the instrument120may include an intervention or operating end124. The operating end124may be positioned within the head104of the subject100by the user130. The instrument120can be used for various purposes such as biopsy (i.e., removal of selected material), shunt placement, deep-brain stimulation, or other appropriate procedures. It is further understood that non-brain procedures may also occur and the instrument120may be positioned relative to a selected portion of a patient, such as a spinal column for a spinal neural procedure, a chest cavity for a cardiac procedure, and other appropriate procedures. Nevertheless, the instrument120can be tracked with the localizer system40as discussed further herein. Also, the instrument120may be used to create gestures that are tracked with the localization system40for either performing the procedure and/or for interacting with the navigation system. Gestures may, for example, be used to change perspective of the imager, zoom of the image, opacity of the icon or rendering of the instrument120, etc.

In the particular example, image data can be acquired of the subject100using various imaging techniques. For example, x-ray imaging, fluoroscopic imaging, magnetic resonance imaging (MRI), computer tomography (CT) imaging, and other appropriate imaging systems may be used to acquire or obtain the image data of the subject100. The image data may be two-dimensional image data, three-dimensional image data, or two- or three-dimensional image data acquired over time to show change. For example, fluoroscopic or MRI image data can be acquired of the patient over time to illustrate motion of the various anatomical and physiological features, such as a heart rhythm. The image data can be saved and/or immediately transferred for display on a display device, such as the display device66. The image60may include a direct image or a rendering based upon image data of a selected portion of the subject100, such as a rendering or display of a brain150.

An icon124′ (which may be a representation of the instrument120, which may further include a rendering of the instrument120including a three-dimensional rendering) can be used to illustrate a position of at least a selected portion, such as the intervention portion124of the instrument120. The icon124′ may be a three-dimensional rendering of the instrument120or selected portion thereof, such as only the intervention portion124. The determination of the position of the intervention portion124can be made by tracking all or a portion of the instrument120and determining a location of the intervention portion124, as discussed further herein. Therefore, the user130can view the display device66and the image60to understand the position of the intervention portion124by viewing the icon124′ relative to the image of the brain150, which is based upon the image data. As discussed above, registration of a position of the subject100, including the head104relative to the localizer device40, can be used to assist in appropriately illustrating the location of the icon124′ relative to the brain rendering150. Registration techniques can include those discussed above and others that are generally known in the art.

A location of the intervention portion124of the instrument120can be based upon known or input measurements of the instrument120. With additional reference toFIG. 3, the instrument120is illustrated in greater detail. As noted above, the instrument120can include the handle122and the interventional portion124. The handle122can be designed and provided in an appropriate configuration, such as an ergonomic shape including indentations and soft portions for efficient and proper grasping by the user130. The handle122has an external surface contour or profile that may be imaged with the localizer40. The external surface contour or profile can be recognized and tracked in the object space using the stereoscopic lens/camera of the localizer40.

The intervention portion124can include a dimension160between a terminal distal end162of the intervention portion124and a distal terminal end164of the handle122. The distance160can be used to determine the location of the distal terminal end162and other portions of the intervention portion124, such as a portal or opening166. The portal or opening166can include a portion that allows for resection or biopsy of a selected tissue. Further, the portal166may represent a stimulating region of an electrode or other operative portion.

Nevertheless, the localizer40may identify and track the handle122including the location of the distal terminal end164of the handle122. Therefore, even if all or a portion of the intervention portion124, including the distal terminal end162, is not directly viewable or imageable by the localizer40, a determination can be made, based upon the distance160, of the position of the unviewed portion. Thus, the navigation computer70can determine the location of the portal166and the distal terminal end162and may display the icon124′ at the position relative to the image66that represents the position of the intervention portion124relative to the subject100.

The user130may enter into the system70, such as with the user input portion71, the distance160. Alternatively, or in addition, the user may identify the instrument120and the system70may recall specific dimensions of the instrument120, such as from the memory70b. The memory70bmay include a database, such as a look-up table, of dimensions of specific instruments. Thus, the user130may identify the instrument, as discussed further herein. Also, the database may include an instrument's external surface contour. Thus, the external surface contour and the position of selected portions of the intervention portions relative to selected points, such as the distal terminal end164of the handle122, may be saved in the database for tracking. Further, generally known techniques may be used to determine the location of a portion of the intervention portion124relative to the handle122.

Accordingly, the localizer40can track the instrument120, including a portion of the intervention portion124, without a separate tracking member associated with the instrument120. For example, a reflector portion or other tracking member need not be attached to the instrument120. The instrument120may be a standard instrument that is not otherwise augmented or enhanced for use with a tracking system. The localizer40can be used to specifically identify a portion of the instrument120, such as the handle122, and track its position and the subject or object space so that the position of the instrument120can be determined as it is positioned relative to the subject100.

A specific tracking member, such as a specifically positioned reflector or emitting device is not needed to be attached to the instrument120for tracking the instrument120. This may allow for an efficient tracking of the instrument120during a procedure. Further, the tracking of the instrument120may occur without requiring additional attachments to the instrument, thus, the instrument120may be easily and efficiently used by the user130and the possibility of moving a tracking device relative to the instrument120is eliminated. Also, as no tracking member is required, no calibration of the position of the tracking member is required.

Further, as discussed above, and illustrated inFIG. 2, the localizer40may be provided as a single localizer element or portion. It is understood, however, as also illustrated inFIG. 2, that a second localizer40a(illustrated in phantom), which is optional, may be provided. The second localizer40amay be positioned near the first localizer40or may be positioned at another appropriate position. Exemplarily illustrated inFIG. 2both the first localizer40and the second localizer40aare positioned on the holder110. Each of the respective localizers40,40acan have separate field of views180,182, respectively. The field of views180,180may be co-extensive, such as to provide redundancy to the navigation system, may partially overlap to provide redundancy and a greater field of view, or may have entirely separate field of views such that an entire field of view of the tracking and navigation system may be increased relative to only a single of the localizers40,40a. Both of the localizers40and40acan be connected to the navigation computer system70with one or more communication lines186. The communication lines186can be wired, wireless, or other appropriate communications between the respective localizers40and40aand the navigation computer70. As discussed above, the tracking information can be sent from the localizer40and40ato the navigation computer70.

With additional reference toFIG. 4, a procedure200using the localizer40and/or40amay be performed. Generally, the procedure may start in start block210and include selecting a procedure in block212. Selecting a procedure can include selecting performing a biopsy, placing a stent, or other appropriate procedures. As noted above, various instruments can be used to perform selected procedures, such as biopsy of brain tissue or other tissues for placing of an implant such as a stent, catheter, vascular implant, or the like. The procedure can be selected to assist in determining the appropriate placement of the localizers, selection of an instrument or preparation of instruments and/or implants, and other procedural requirements.

As noted above, image60can be displayed on the display device66. The image60may be a rendered model or may be raw image data that is displayed on the display device66. Nevertheless, image data may be acquired and/or loaded in block220. The acquisition of the image data can be performed with various imaging systems, including those discussed above, for example an MRI. The acquired image data may be stored in a patient storage system and may be loaded into the navigation computer70for the procedure200. Additionally, models based upon the image data may be loaded or acquired for performing the procedure200. The loaded image data or models may relate to the portion of the patient being operated on, including neurological models, heart models, or the like. The acquired image data may be displayed for illustrating the location of the instruments120relative to the subject100by displaying the instrument icon124′ relative to the image60on the display device66.

The subject may be prepared in block224at an appropriate time, such as before or after acquiring the image data or models of block220. Preparation of the subject in block224may include fixing or holding the subject relative to the localizers40,40asuch as with the holder110. In exemplary embodiments, as discussed above, the holder110may be a Mayfield® skull clamp and may be used to fix the head104of the subject100in a selected location so that it is substantially immobile relative to the localizer40. Preparation of the subject100may also include general surgical preparation such as cleaning, forming a burr hole, forming incisions, and other appropriate subject preparation.

Instruments may be selected in block230and the instruments or the selected instruments may then be identified or input into the navigation computer70in block232. As discussed above, the instruments may include selected dimensions that are known relative to trackable or identifiable portions of the various instruments, including a selected external contour. As noted above, a distal end164of the handle122of the instrument120may be at the known distance160from the distal end162and/or or the portal166of the intervention portion124. Therefore, selecting and inputting the instruments, in blocks230,232respectively, may allow for the navigation computer70to identify position of various portions of the instrument120relative to other portions of the instrument120.

As noted above, inputting the instruments in block232may include inputting specific dimensions of the instrument120for determining locations of portions thereof or other inputs, by the user130. Further, inputting an identifying feature of the instrument120may allow the navigation computer70to load, such as from a database in the memory system70b, predetermined dimensions of the instrument120, including the dimension160. The determination of the dimensions, however, may be performed in any appropriate manner. Also, the localizer40may image the instrument120to determine at least an external surface contour for tracking and at least the dimension160. Thus, the localizer40may be used to determine dimensions for tracking in addition to or separate from any stored in a database. Also, this may allow for new or additional instruments to be used that are pre-identified and measured for storage in the database.

Once the subject is prepared in block224, the subject may also be registered in block240. Registration may proceed according to any appropriate registration technique, including those discussed above. Generally, registration allows for mapping of points in the subject space or object space to points in the image60or image space. Accordingly, registration may include identifying one or more points on the subject100, such as on the head104, and identifying the same points in the image60to allow for a mapping between the points or locations in the object space and points or locations in the image60. Registration can allow for illustrating the instrument icon124′ at the appropriate position relative to the image60based upon its tracked position relative to the subject100, including the head104.

During a procedure, registration may be maintained as the portion of the patient being acted upon on and relative to which the instrument120is being tracking is held fixed and substantially immobile (i.e. less than or equal to a tolerance of the system which may include a tracking accuracy) relative to the localization system40. As illustrated inFIG. 2, the localization system40is fixed to the clamp110and the head104is fixed relative to the clamp110. Thus, the head104is fixed relative to the localization system40, and registration may be maintained. This may also reduce or eliminate the need of a patient tracking device to track the patient during a procedure to maintain registration.

Once registration occurs, navigation of the procedure with the instruments and the localizer may occur in block242. Navigation may include illustrating the icon124′ relative to the image60on the display device66. Therefore, the user130may view on the display device66the position of the instrument120by viewing the icon124′ representing the instant relative to the image60. Thus, the user130may know the position of the instrument120, including the operative end124relative to the subject100without directly viewing the operative end124within the subject100.

The patient100, including the head104or only the head104is generally held relative to the localizer40,40awith the holding device110. Holding the head104relative to the localizer40maintains registration of the image space relative to the subject space. If the head104, or other registered portion of the patient100, moves relative to localizer40then the mapping of points between the patient100and the image60is no longer proper and navigation or illustration of the icon124′ relative to the image is not correct. Thus, if the registered portion, such as the head104, moves relative to the localizer40registration may need to occur again. Alternatively, a tracking member may be placed on the head104, or other registered portion, to be tracked to maintain registration during a procedure. Further, the localizer40may be able to identify an external surface contour of the head104, or other portion to be registered, and track the head during the procedure to maintain registration even with movement relative to the localizer40.

The procedure may then be completed in block250. Completion of the procedure may be obtaining the biopsy material, fixing a deep brain simulation probe, or completing other appropriate portions of a procedure. It may also include withdrawing an instrument and confirming that a selected procedure has been completed. The procedure may then end in block260.

Therefore, the localizer, including the localizer40and/or40amay be used to track and navigate the procedure relative to the subject100. As noted above, the subject100need not be a human subject, and can be any appropriate object including that illustrated inFIG. 1. Nevertheless, the localizer40may be substantially small and efficiently positioned during a procedure. The localizer40may also be provided or supplied from general merchandise suppliers to allow for a cost-effective navigation system.

The instruments also may be provided in a manner that does not require a specific tracking member to be affixed thereto. Thus, the presently disclosed system allows for standard and unaltered instruments to be used to perform a navigated procedure, such as a navigated surgical procedure. Although the instruments may include inherent features (e.g., unique handle geometries), the inherent features may be substantially immovable relative to operative portions of the instrument. Therefore, placing and/or fixing tracking members at positions on the instrument, such as on the instrument120, need not be considered. Therefore, navigating an instrument relative to an object, such as tracking the instrument120relative to the subject100and illustrating it via the icon124′ relative to the image60, is disclosed. The system allows for efficient and compact navigation of the instrument120relative to the subject, or other appropriate instruments relative to an object.

According to various embodiments, the localizer40may image all or a portion of the instrument120. Once imaged, that portion may become the external surface contour that is tracked during a procedure. As discussed above, the localizer40includes lenses and or cameras that may image the instrument. Thus, the image of the instrument may be used to determine the external surface contour that is tracked during the procedure. For example, the external surface contour may include the transition point or contour from the handle122to the intervention portion124at the distal end164of the handle122. This transition portion that defines the external surface contour that is used for tracking and allows the instrument120to be tracked without a separate tracking member attached to the instrument120. Further, external surface contours may be predetermined and saved in the database stores in the memory system70b. Thus, when the user130inputs the instrument, such as in block232, the navigation computer70may recall the external surface contour of the instrument for proper tracking thereof.

Example embodiments are provided so that this disclosure will be thorough, and will fully convey the scope to those who are skilled in the art. Numerous specific details are set forth such as examples of specific components, devices, and methods, to provide a thorough understanding of embodiments of the present disclosure. It will be apparent to those skilled in the art that specific details need not be employed, that example embodiments may be embodied in many different forms and that neither should be construed to limit the scope of the disclosure. Further, the various disclosed embodiments may be combined or portions of one example may be combined with another example in an appropriate manner. In some example embodiments, well-known processes, well-known device structures, and well-known technologies are not described in detail.