Patent Description:
A dental implant procedure generally involves several aspects: making the diagnosis, planning the position of the implant, surgically milling the anatomy and placing the implant (i.e., an anchoring post), and then attaching the abutment and crown (i.e., prosthetic tooth / teeth). In some instance, such a procedure may be automated, for example, via a robotic surgical system. Some surgical systems, such as dental implantation surgical systems, may implement planning of the surgical procedure based on non-invasive imaging of the planned surgical site (i.e., via a CT scan). The result of the plan, namely the collected image(s) of the surgical site or patient anatomy, can thus be used, for example, for diagnosis, for surgical training, for pre-operative planning, for manufacturing a drill guide, or for guiding haptic or autonomous robotic systems.

In this regard, some such navigated and/or guided medical devices may require pre-operative planning to determine the intended course of action during surgery. Some current planning systems may, for instance, use 2D screens and interactive tools such as a mouse and keyboard in order to accomplish such a "virtual" pre-operative planning procedure. In some instances, a three-dimensional arrangement may be implemented, using, for example, a three-dimensional (3D) display arrangement, which incorporates a stereoscopic screen and 3D glasses, combined with a pen-like stylus that can be used as a pointer in the virtual space of the 3D display arrangement (see, e.g., <FIG>). However, the manipulation of a virtual object (i.e., a dental implant or a surgical instrument) using such systems, whether by manipulating a two-dimensional (2D) object in a 2D environment, or manipulating a 3D object in a 3D environment, may not necessarily be "natural" to the user. That is, it may not necessarily be intuitive or ergonomically agreeable to manipulate the virtual object using a mouse or a stylus in a conventional manner, and various additional actions may be required by the user to impart the desired motion/movement to the virtual object (i.e., an inconvenience).

In a dental implant planning procedure using a conventional two-dimensional planning arrangement (i.e., a desktop or laptop computer), a user can, for example, generally click and drag a mouse associated with the computer to drag and drop a virtual dental implant in a desired location in a virtual space on a display screen. In some instances, the user may be required to line up circles/crosshairs with a target on each virtual object being manipulated, or the location or other object with respect to which the virtual object is being manipulated. However, such conventional manipulation of a mouse may not necessarily be effective when the planning procedure is executed by a 3D planning arrangement. Though a stylus may be used to address this issue, it remains that manipulation of the virtual object in the 3D space, using functions associated with the tip of the stylus, may not necessarily be intuitive, ergonomically agreeable, or convenient.

As such, it may be desirable to provide a planning and/or training arrangement and method, for example, for a surgical robotic system, with such a planning and/or training arrangement and method implementing a more intuitive, ergonomically agreeable, and convenient system for manipulating a virtual object, such as a dental implant or surgical instrument, in a two-dimensional or three-dimensional virtual environment, whether or not the user is significantly experienced with the system.

The above and other needs are met by aspects of the present disclosure which, in one particular aspect, provides a system for planning a procedure in accordance with the invention and as defined in claim <NUM>. Such a system comprises a display device configured to display a first virtual element, the first virtual element comprising a surgical apparatus including one of a dental implant and a surgical instrument configured to prepare a site on a jaw structure to receive the dental implant; a controller device having a processor, and being configured to be in communication with the display device, so as to direct the display device to display the first virtual element. A physical control element is in communication with the controller device, and is configured as a corresponding scale model of the first virtual element and having six degrees of freedom of movement, wherein the processor is configured to analyze an actual manipulation of the control element, and wherein the actual manipulation of the physical control element in each degree of freedom of movement is the same as a virtual manipulation of the first virtual element displayed, via the analysis by the processor of the controller device and on the display device.

Another aspect of the present disclosure provides a method for planning a procedure in accordance with the invention and as defined in claim <NUM>. Such a method comprises displaying a first virtual element on a display device with a controller device having a processor and configured to be in communication with the display device, the first virtual element comprising a surgical apparatus including one of a dental implant and a surgical instrument configured to prepare a site on a jaw structure to receive the dental implant; performing an actual manipulation of a physical control element in communication with the controller device and configured as a corresponding scale model of the first virtual element and having six degrees of freedom of movement, the actual manipulation being analyzed by the processor; and displaying, via the analysis by the processor of the controller device and on the display device, a virtual manipulation of the first virtual element, the virtual manipulation of the first virtual element corresponding to and being the same as the actual manipulation of the physical control element in each degree of freedom of movement.

Yet another aspect provides a method for planning a procedure, comprising displaying a first virtual element via a display device; analyzing, via a processor, physical manipulation of a control element interface configured to correspond to the first virtual element; and displaying, in response to the analysis of the physical manipulation of the control element interface, a response of the first virtual element corresponding to the physical manipulation of the control element interface.

Still another aspect provides a system comprising processing circuitry operatively coupled with a control element interface, wherein the processing circuitry is configured to cause the system to at least display a first virtual element on a display device; analyze physical manipulation of a control element interface configured to correspond to the first virtual element; and display, in response to physical manipulation of the control element interface, a response of the first virtual element corresponding to the physical manipulation of the control element interface.

A further aspect provides a computer program product according to the invention and as defined in claim <NUM> comprising at least one non-transitory computer readable storage medium having computer readable program instructions stored thereon. The computer readable program instructions comprise program instructions which, when executed by at least one processor implemented on a system for planning a procedure, said system according to any one of claims <NUM> to <NUM>, cause the system to perform a method according to any one of claims <NUM> to <NUM>.

These and other features, aspects, and advantages of the present disclosure will be apparent from a reading of the following detailed description together with the accompanying drawings, which are briefly described below. The present disclosure includes any combination of two, three, four, or more features or elements set forth in this disclosure, regardless of whether such features or elements are expressly combined or otherwise recited in a specific embodiment description herein. This disclosure is intended to be read holistically such that any separable features or elements of the disclosure, in any of its aspects and embodiments, should be viewed as intended, namely to be combinable, unless the context of the disclosure clearly dictates otherwise.

It will be appreciated that the summary herein is provided merely for purposes of summarizing some example aspects so as to provide a basic understanding of the disclosure. As such, it will be appreciated that the above described example aspects are merely examples and should not be construed to narrow the scope or spirit of the disclosure in any way. It will be appreciated that the scope of the disclosure encompasses many potential aspects, some of which will be further described below, in addition to those herein summarized. Further, other aspects and advantages of such aspects disclosed herein will become apparent from the following detailed description taken in conjunction with the accompanying drawings which illustrate, by way of example, the principles of the described aspects.

The present disclosure now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all aspects of the disclosure are shown. Indeed, the disclosure may be embodied in many different forms and should not be construed as limited to the aspects set forth herein; rather, these aspects are provided so that this disclosure will satisfy applicable legal requirements.

Various aspects of the present disclosure may be at least partially based on a guided surgical robotic system and method such as that disclosed, for example, in <CIT> and assigned to Neocis, also the assignee of the present application.

In this regard, a dental implantation procedure generally involves an invasive incision into the gum of the patient in order to allow the practitioner to view the underlying jawbone structure. A hole is then drilled into the jawbone structure, into which a dental implant is placed (see, e.g., <FIG>). In some instances, the dental implant may be shaped, for example, like a screw or other threaded member. Once the dental implant is inserted into the jawbone structure, an external post is attached to the dental implant (see, e.g., <FIG>), and a prosthetic cap (i.e., a crown or tooth reproduction) attached to the post (see, e.g., <FIG>). With computerized tomography (CT) and other imaging scans becoming more common, the practitioner may be able to graphically visualize the jawbone structure, without or before the invasive incision. However, the alignment of the dental implant with respect to the jawbone structure and/or relative to other implants or teeth may be an important factor in determining, for example, the life of the dental implant, the appearance thereof, and the comfort to the patient. If the dental implant is poorly or otherwise not optimally placed, the dental implant can undesirably fail (or at least have a shorter service life), may undesirably cause other teeth or dental implants to be compromised, and/or damage proximal nerves.

<FIG> and <FIG> thus illustrate various aspects of a dental implantation system according to the present disclosure, the system being generally indicated by the numeral <NUM>. As previously indicated, current dental implantation procedures generally involve an imaging step, wherein CT or other appropriate images of the patient's jaw structure are obtained, and any anomalies diagnosed (i.e., whether the patient requires bone grafts to prepare the implant area). The practitioner then corrects any anomalies and proceeds with the invasive implant procedure based on the conditions associated with the patient's jaw structure, once the appropriate incisions have been made in the patient's gum. In this regard, one skilled in the art will appreciate that, though the present disclosure provides some exemplary aspects of the various systems and methods implemented with respect to the jaw structure of a patient, the various systems and method disclosed herein may be readily applicable, or otherwise readily adaptable, to other surgical procedures that are proximal to or otherwise capable of being correlated with the fiducial marker associated with the engagement between a splint or other engaging member, and the jaw structure of the patient, as otherwise disclosed herein (i.e., brain surgery, skull surgery, ENT surgery, or any other surgical procedure associated with the head/skull structure of the patient).

A dental implantation system <NUM> according to various aspects of the present disclosure addresses particular subjective aspects of current dental implantation procedures by providing a guided patient-interacting device <NUM> (otherwise referred to herein as a "cutting device" or "drilling device" or "site preparation device" or "implantation device" depending on the particular instrument <NUM> engaged with the patient-interacting device <NUM> so as to configure the patient-interacting device <NUM> for a particular corresponding purpose or procedure) configured to be guided with respect to the invasive portion, or at least the patient-interacting portion, of the dental implant procedure (i.e., to "prepare" the site within or otherwise engage the patient's mouth). That is, the patient-interacting device <NUM> is operably engaged with a guiding device <NUM>, such as, for example, an articulating arm member <NUM> (i.e., a robotic arm). The guiding device <NUM> is adapted to operably engage or otherwise be in communication with the mouth of the patient, for example, by way of a splint <NUM> or other engaging member, forming or otherwise defining a fiducial marker. That is, in one instance, the splint <NUM> is configured to engage the patient's mouth in a "firm" or secure interaction (i.e., the splint <NUM> is engaged with the patient's teeth and does not move with respect to the patient's mouth). Since the splint <NUM> does not move with respect to the patient's mouth, the disposition of the splint <NUM> is known, and thus can be configured to provide a fiducial marker (i.e., a known origin or coordinate formed by the secure interaction with or otherwise associated with or attached to the splint <NUM>) which can be used, for instance, to guide the patient-interacting device / instrument, via the guiding device <NUM>, to prepare the site in the patient's mouth in association with the dental implant <NUM> (see, e.g., <FIG>).

In one aspect, the splint <NUM> is configured to be "universally applicable" (i.e., capable of forming the secure engagement with the mouth of any patient), or at least applicable across a particular range of patients (i.e., one size fits a certain size or age of patient). In order to determine a reference associated with the fiducial marker, according to one aspect of the disclosure, the splint <NUM> may be engaged with the patient's teeth, and the patient's jawbone structure then imaged using, for example, CT or any other suitable imaging technique such as, for instance, MRI. The fiducial marker can thus be established, for instance, as a reference origin of a relative coordinate system.

One skilled in the art will appreciate that the splint <NUM> may be configured in many different manners to accomplish the desired function as discussed herein. For example, the splint <NUM> may be rigidly attached to the patient's mouth in an appropriate manner depending on the condition of the patient. That is, if the patient has some strong teeth capable of supporting the splint <NUM>, the splint <NUM> can be attached to the teeth with an adhesive or with a suitable clamp. For edentulous patients (i.e., without teeth), bone pins may be drilled through the splint <NUM> and into the patient's jawbone structure to fasten the splint <NUM> securely into place. The splint <NUM> may also be attached to the jawbone structure of any patient using, for example, appropriate bone screws. In one aspect, the positioning of the splint <NUM> with respect to the patient's mouth may not be critical or important, as long as the splint <NUM> remains rigidly in place. A fiducial marker (not shown) may then be formed by the secure engagement, or may otherwise be attached to, or incorporated into, or associated with the splint <NUM>, wherein the fiducial marker may be configured to have a geometry or other characteristic or feature that uniquely defines the fiducial marker in a three-dimensional space (i.e., such that the fiducial marker is readily identified in images of the patient's jawbone structure, or is otherwise detectable and trackable using a mechanical arrangement, an electrical arrangement, an electromechanical arrangement, an optical arrangement, a magnetic arrangement, or any other suitable detection/tracking arrangement, or combination thereof). In such instances, the fiducial marker may be comprised of, for example, a radiopaque material that can be clearly defined in the image obtained, e.g., by CT or MRI.

In one aspect, the patient-interacting device <NUM> may be engaged with the guiding device <NUM>, for example, in the form of an articulating arm member or a robotic arm <NUM>, which is configured to determine a range of motion of the patient-interacting device <NUM>/instrument <NUM> (i.e., translation in a particular direction (horizontal and/or vertical), and/or rotation about an axis). In some instances, the functionality of the guiding device <NUM> may be included in the configuration and arrangement of the articulating arm member <NUM>, itself. For example, the articulating arm member <NUM> or portions thereof may include or be engaged with one or more actuators configured and arranged to cooperate to guide a distal end of the articulating arm member <NUM> in a desired direction and orientation, upon manipulation by the user to accomplish the surgical procedure.

In some instances, the guiding device <NUM> may further comprise a communication element <NUM> in communication between the splint <NUM> and the patient-interacting device <NUM> and/or between the splint <NUM> and the arm member <NUM>. For example, the communication element <NUM> may comprise a mechanical linkage connecting the splint <NUM> to the patient-interacting device <NUM> or to the arm member <NUM>. That is, the communication element <NUM> may comprise, for example, a mechanically- or physically-tracked arm which attaches to the splint <NUM> engaged with the patient. In some instances, the arm (communication element <NUM>) may be attached to the splint <NUM> (rigidly and in a known, repeatable manner) with an attachment mechanism comprising a kinematic mount (i.e., a kinematic mount may be engaged between the arm and the splint <NUM>). Attached to the patient in this manner via the attachment mechanism and the splint <NUM>, the communication element <NUM> may be tracked or otherwise monitored to provide data (whether constantly, selectively, or otherwise as necessary) about the position of the patient (i.e., with respect to the fiduciary or fiducial marker) to the patient-interacting device <NUM> and/or to the arm member <NUM>, while still providing for accurate guidance of the patient-interacting device <NUM> and/or the arm member <NUM>, in the event that the patient moves during the surgical procedure.

However, one skilled in the art will appreciate that the splint <NUM> and/or the fiducial marker determined thereby may be communicated to the patient-interacting device <NUM> and/or to the arm member <NUM> in many different manners. For example, instead of or in addition to the physical arm (communication element <NUM>), the fiducial marker may be communicated via a communication element <NUM> comprising a wireless transceiver, a hardwire connection, an optical communication system (i.e., a camera or other video device), an acoustic tracking system, or any other suitable mechanism, whether electrical, mechanical, electromechanical, acoustic, or optical in nature. That is, in various instances, the kinematic mount, itself, may comprise an attachment point for a tracking portion (and/or the tracking arm or other tracking provision) associated with the guidance system for the surgical robot (i.e., wherein, for instance, reflective markers may be mounted to the attachment point for optical tracking of the fiducial marker or the splint device itself, or the attachment point may include a securing site for forming a mechanical connection therewith for mechanical tracking of the fiducial marker, or the attachment point may otherwise be configured to receive an appropriate element associated with any other suitable tracking arrangement for the fiducial marker, whether electrical, mechanical, electromechanical, acoustic, or optical in nature). In other aspects, the kinematic mount may be configured or otherwise arranged to function as a fixed mounting site for particular tracking devices such as, for example, one or more markers that may be permanently affixed to the kinematic mount <NUM> and configured to be trackable by an optical-type tracking device (i.e., an optical tracking marker).

In any instance, the system <NUM> may be further configured to include a controller device <NUM> (i.e., a computer device as shown in <FIG>) for determining, controlling, or tracking the fiducial marker with respect to the image of the patient's mouth having the splint <NUM> disposed therein. The controller device <NUM> may also be configured and arranged to appropriately communicate the fiducial marker to the patient-interacting device <NUM> and/or to the arm member <NUM>. In some aspects, the system <NUM> or the controller device <NUM> may also comprise a planning device or otherwise include planning functionality for allowing a user to develop the virtual implantation plan, as otherwise disclosed herein, in conjunction with the hardware and/or software of the system <NUM>.

In one aspect, the controller device <NUM> may be further configured to receive the image of the patient's jawbone structure (having the splint <NUM> therein). In some instances, the controller device <NUM>, which includes a processor, may be further configured to be capable of executing a planning routine that may comprise software, hardware, or a combination thereof (i.e., a planning device and/or planning functionality). The planning routine thus allows the practitioner to create, for example, a virtual implantation plan based on the captured image(s), whether in two dimensions or three dimensions, and to manipulate the image(s) of the patient's jawbone structure in conjunction with a "virtual implant" in order to develop the virtual implantation plan or placement determination of the prosthesis for the patient, in conjunction with a computerized model based on the image(s). In some aspects, the planning routine, virtual implantation plan, and/or placement determination may be created in relation, for example, to a coordinate system (relative or absolute), as will be appreciated by one skilled in the art, configured to associate the planning parameters with the fiducial marker. In other aspects, the controller device <NUM> and/or the planning device associated therewith may include a peripheral device (i.e., a trackball or joystick in conjunction with, for example, 3D goggles, all not shown) to assist with or otherwise permit virtual manipulation of the placement of the virtual implant(s) with respect to the image(s) of the patient's jaw structure in order to, for example, align the implant(s) relative to each other or relative to adjacent teeth, to align the implant(s) relative to the affected nerve, and/or to align the implant(s) relative to the jawbone structure. The controller device <NUM> and/or the planning device may be further configured to perform such manipulation manually, automatically, or semi-automatically, as necessary or desired. Because the virtual implant(s) may be manipulated in a similar manner to the image(s), the orientation or placement of the virtual implant(s) may represent the desired actual placement of the implant with respect to the patient's jawbone structure, thus providing an intuitive interface for planning the implantation procedure.

In aspects where the splint <NUM>/fiducial marker approach is used, the patient is automatically registered with the system <NUM> / controller device <NUM> once the communication element <NUM> is attached to or otherwise engaged or in communication with the splint <NUM> via the kinematic mount of the attachment mechanism. That is, the fiducial marker is automatically determined from the image(s) of the patient's jawbone structure, and the alignment and location thereof in physical space is known due to the kinematic mount connecting the communication element <NUM> (i.e., arm) to the splint <NUM>. One skilled in the art will appreciate, however, that other alignment approaches may be implemented that do not necessarily require a fiducial marker. For example, in some instances, a surface matching technique can be implemented. More particularly, the patient's jawbone structure may be manipulated into a 3D configuration in the captured image(s). A suitable scanning device (i.e., a physical pointer or other imaging device such as an ultrasound transducer or OCT (optical coherence tomography) scanner may be attached to an end effector (i.e., tip) of the arm member <NUM> such that the tip of the arm member <NUM> is capable of scanning the patient's jawbone structure to "surface match" the captured and manipulated image(s) with an actual scan of the jawbone structure, as administered, for example, via the controller device <NUM>.

One skilled in the art will further appreciate that the association of the fiducial marker with the patient's anatomy, via the controller device <NUM>, may be accomplished in different manners. For example, with respect to the registration of the image (e.g., CT scan) to the fiducial marker, one method could involve the jaw structure of the patient being imaged with the fiducial marker in place, as previously discussed, wherein the patient would then be substantially immediately subjected to the implantation procedure. Such a scheme may be beneficial, for example, in reducing the number of visits to the practitioner by the patient. However, in some instances, the practitioner may not have the imaging capabilities at hand, or may prefer to carefully determine the virtual implantation plan before carrying out the implantation procedure. In both such instances, the patient will likely be required to return to the practitioner at a later time. Accordingly, in such situations, a pre-operative imaging procedure (e.g., CT scan) may be performed on the jaw structure of the patient, without a fiducial marker in place (i.e., a "normal" scan by which the practitioner can determine the virtual implantation plan). This pre-operative imaging procedure can thus be performed, for example, at the practitioner's site, or at a dedicated scanning/imaging center. Subsequently, immediately prior to the implantation procedure being performed, and with the fiducial marker(s) engaged with the jaw structure of the patient, the practitioner may capture another image (e.g., CT scan, panoramic x-ray, or two single x-rays) of the patient's jaw structure. The controller device <NUM> / planning device may thus also be configured to correlate the pre-operative image (used to determine the virtual implantation procedure) with the "day of" image so as to register the fiducial marker(s) with respect to the original pre-operative image. Such a registration or correlation procedure may be implemented in hardware, software, or a combination thereof, as will be appreciated by one skilled in the art. The implantation procedure could then proceed as otherwise disclosed herein.

In any instance, the communication element <NUM> may be configured to communicate with the arm member <NUM> in a manner known to the system <NUM>, such that the position/movement characteristics of the end effector /tip thereof are also known. This engagement between the communication element <NUM> and the arm member <NUM> thus allows the patient-interacting device <NUM>/instrument <NUM> (i.e., the end effector / tip) to be registered with respect to the fiducial marker (or other reference with respect to the patient) attached to the patient via the splint <NUM>, the kinematic mount, the communication element <NUM>, and the arm member <NUM>. In this manner, the virtual implantation process, planned through the controller device <NUM> / planning device, may be accomplished in relation to the fiducial marker (or other reference with respect to the patient) and thus translated or otherwise communicated to the system <NUM> for directing the patient-interacting device <NUM>/instrument <NUM> via the guiding device <NUM> and the arm member <NUM>. As previously disclosed, and as will be appreciated by one skilled in the art, the communication element <NUM> may, in some instances, be configured to communicate between the splint <NUM> / kinematic mount and the controller device <NUM> / planning device (and/or between the splint <NUM> / kinematic mount and the patient-interacting device <NUM>/instrument <NUM>), based upon the premise of establishing a known association of the fiducial marker with the patient's anatomy, in relation to which the guiding device <NUM> is used to guide the patient-interacting device <NUM>/instrument <NUM> via the arm member <NUM> during the implantation procedure.

The patient-interacting device <NUM>/instrument <NUM> may comprise, be disposed in, or otherwise engaged with the end effector of the arm member <NUM> (robotic arm). The arm member <NUM> may be configured, for example, to provide six degrees of freedom and can also be configured to restrict or otherwise control the movement of the patient-interacting device <NUM>/instrument <NUM>. Further, in some instances, the arm member <NUM> may have a miniature parallel structure to which the patient-interacting device <NUM> is secured and allowed to have full freedom of movement when not in cutting/preparation/implantation mode. Since the patient-interacting device <NUM>/instrument <NUM> comprises or is attached to the end effector of the arm member <NUM>, the patient interacting portion (i.e., the cutting/drilling tip) is the instrument <NUM> (see, e.g., <FIG>) of the patient-interacting device <NUM>, and the instrument <NUM> thus must be in a known position (i.e., known to the system <NUM>/controller device <NUM>) relative to the arm member <NUM>. In some aspects, in order to calibrate the interacting portion/instrument <NUM> of the patient-interacting device <NUM> with respect to the fiducial marker, a calibration element may be engaged with the patient-interacting device <NUM> via a kinematic coupling (i.e., rigidly mounted thereto in a known, repeatable manner). One skilled in the art will thus appreciate that the interacting portion/instrument <NUM> of the patient-interacting device <NUM> can then be calibrated with various tip calibrating methods (i.e., invariant point, etc.). Once calibrated, the calibration element is replaced with a cutting/drilling element (instrument <NUM>) in the patient-interacting device <NUM>, in a known and repeatable manner, so that the calibration parameters (i.e., the position of the distal-most point and axis of cutting/drilling) associated with the interacting portion/instrument <NUM> are maintained as calibrated.

With the alignment with respect to the patient established and known by the system <NUM> / controller device <NUM>, and the virtual implantation plan developed through the controller device <NUM> / planning device, the implantation procedure (i.e., cutting/drilling/insertion) can then be initiated by the practitioner moving the patient-interacting device <NUM>/instrument <NUM> toward the patient's mouth (having the splint <NUM> engaged therewith). In such instances, the controller device <NUM> / planning device is configured to control, restrict, or otherwise modulate the movement of (or the practitioner's ability to move) the patient-interacting device <NUM> via the arm member <NUM> such that the action of the practitioner merely moves the interacting portion/instrument <NUM> (i.e., the cutting/drilling element) to the appropriate starting position for the implantation procedure, with respect to the patient's jawbone structure, as determined by the controller device <NUM> / planning device and dictated by the virtual implantation plan. Once the cutting/drilling element is in the position dictated by the controller device <NUM> / planning device, the invasive portion of the procedure can then be initiated, wherein the controller device <NUM> / planning device may further dictate other parameters of the implantation device <NUM>/instrument <NUM> such as, for example, the orientation of the path of the cutting/drilling element (instrument <NUM>) and the cutting/drilling distance along that path from the cutting/drilling origin, also according to the virtual implantation plan. In some instances, the system <NUM> disclosed herein may be configured such that the patient-interacting device <NUM> is not guided by the practitioner, but is only urged by the practitioner along a procedural route determined via the virtual implantation plan and implemented via the controller device <NUM> / planning device and the arm member <NUM>. That is, the system <NUM> may be configured to restrict the practitioner to performing the implantation procedure with respect to the patient, as determined via the virtual implantation plan and implemented via the controller device <NUM> / planning device and the arm member <NUM>, whereby the controller device <NUM> / planning device controls the allowable movement of the arm member <NUM> (and thus the patient-interacting device <NUM>/instrument <NUM>) in accordance with the virtual implantation plan created from the image(s) of the patient's jawbone structure. For instance, the system <NUM> may be configured for restricted movement of the arm member <NUM>/ patient-interacting device <NUM>/instrument <NUM>, as communicated to the practitioner through tactile/haptic feedback, where, for example, the arm member <NUM> / patient-interacting device <NUM> / instrument <NUM> may be easier to move according to the virtual implantation plan, and more difficult to move if deviating from the virtual implantation plan.

One skilled in the art will also appreciate, however, that the physical structure of the arm member <NUM> / patient-interacting device <NUM> / instrument <NUM> may not necessarily be configured to provide full and absolute controlled movement according to the virtual implantation plan (i.e., due to vibration, flexing of components, gravity, and/or excessive force applied by the practitioner) and, as such, the system <NUM> / controller device <NUM> may be further configured to provide other manners of feedback to the practitioner such as, for example, via a deviation warning indicia, haptic feedback, or any other suitable audio and/or visual and/or any other suitable mechanism. Therefore, the system <NUM> / controller device <NUM> includes provisions for actually implementing the virtual implantation plan, and thus facilitates a more accurate implantation procedure, rather than merely warning the practitioner if any procedural parameters may be inaccurate. One skilled in the art will also appreciate, however, that, in some instances, the system <NUM> may be further configured to autonomously accomplish the virtual implantation plan, without the manipulation of the practitioner, through automatic manipulation of the arm member <NUM> / patient-interacting device <NUM> / instrument <NUM> via the controller device <NUM> / planning device.

In one exemplary surgical procedure using a dental implantation system <NUM>, as disclosed herein, the splint <NUM> (i.e., mouthpiece) is first attached to the patient's teeth, and thus provides or is associated with a fiducial marker. The patient's jawbone structure is then imaged (with the splint <NUM> in place and engaged with the patient's teeth) using, for example, CT or any other appropriate imaging technique (e.g., MRI), and the image(s) communicated to the controller device <NUM>. The controller device <NUM> may be further configured to be capable of executing an implantation routine, thus allowing the practitioner to develop an implantation plan for the patient, for example, by manipulating a virtual implant with respect to the captured image(s). Once the virtual implantation plan is created, the communication element <NUM> is engaged with (i.e., attached to the patient's mouth, with the patient being positioned in a suitable position to initiate the procedure) or otherwise placed into communication with the splint <NUM> (i.e., via the kinematic mount). The arm member <NUM>, patient-interacting device <NUM>, and interacting portion / instrument <NUM> thereof, are then calibrated by the practitioner (or automatically by the controller device <NUM>), before the actual cutting/drilling element (instrument <NUM>) of the patient-interacting device <NUM> is used by the practitioner (or autonomously via the controller device <NUM> / planning device), via the patient-interacting device <NUM> as guided by the arm member <NUM> and the controller device <NUM>, to accomplish the implantation procedure as planned and dictated by the virtual implantation plan.

In some aspects, as previously discussed, the controller device <NUM>, which includes a processor, may be configured to be capable of executing a planning routine or procedure that may comprise software, hardware, or a combination thereof (i.e., a planning device, planning arrangement, and/or planning functionality). The planning routine thus allows the practitioner to create, for example, a virtual implantation plan based on the captured image(s) of the patient's jawbone structure, whether in two dimensions or three dimensions, and to manipulate the image(s) of the patient's jawbone structure in conjunction with a "virtual implant" in order to develop the virtual implantation plan or placement determination of the prosthesis for the patient, in conjunction with a computerized model based on the image(s) (i.e., wherein the virtual planning procedure could be displayed to the user on a display or display device <NUM> associated or otherwise in communication with the controller device <NUM>). In some aspects, the planning routine, virtual implantation plan, and/or placement determination may be created in relation, for example, to a coordinate system (relative or absolute), as will be appreciated by one skilled in the art, configured to associate the planning parameters with the fiducial marker. In further aspects, the planning routine may also involve manipulation of the surgical instrument(s) to be used in performing the dental implant procedure, for example, as an instructional or training tool via the display device <NUM>.

In other aspects, as shown, for example, in <FIG>, the controller device <NUM> and/or the planning device/arrangement <NUM> associated therewith may include a peripheral device or control element <NUM> (i.e., a trackball, joystick, or stylus, in conjunction with, for example, 3D goggles (as shown, for example, in <FIG>)) to assist with or otherwise permit virtual manipulation of the placement of the virtual implant(s) with respect to the image(s) of the patient's jaw structure in order to, for example, align the implant(s) relative to each other or relative to adjacent teeth, to align the implant(s) relative to the affected nerve, to align the implant(s) relative to the jawbone structure, and/or otherwise to perform a "test fitting" of the implant(s). In some instances, the control element <NUM> may be implemented to assist with or otherwise permit virtual manipulation of the surgical device (i.e., the patient-interacting device <NUM>/instrument <NUM>) with respect to the implantation site, for instance, as a teaching or training tool. In yet other aspects, the peripheral device or control element <NUM> may be more generally configured to correspond to a first virtual element <NUM> which, in some instances, may broadly comprise a surgical apparatus, wherein such a surgical apparatus may include one of a dental implant, a surgical instrument configured to prepare a site on a jawbone structure to receive the dental implant, or any other appropriate apparatus associated with a surgical procedure, whether dental-related or not.

As also previously discussed, however, if the peripheral device / control element <NUM> is implemented in a "conventional manner" (i.e., to click, drag, and drop like a conventional mouse or similarly with a conventional stylus), such an arrangement may not necessarily be intuitive, ergonomically agreeable, or convenient for a user to manipulate a virtual object, such as a dental implant or surgical instrument, in a two-dimensional or three-dimensional virtual environment, whether or not the user is significantly experienced with the system, while viewing such manipulation on the display device <NUM>.

As such, since the orientation and/or placement of the virtual implant (i.e., a first virtual element <NUM>) may represent the desired actual placement of the implant with respect to the patient's physical jawbone structure, or since the manipulation of a virtual surgical instrument can represent the actual use of that surgical instrument with respect to the patient's physical jawbone structure, aspects of the present disclosure are directed to a system for planning a procedure (i.e., a planning device / arrangement <NUM>), wherein the display device <NUM> is configured to display the first virtual element <NUM> (i.e., a dental implant comprising an implant element and a prosthetic member, or a surgical instrument represented by the patient-interacting device <NUM>/instrument <NUM>). The controller device <NUM>, having a processor, is configured to be in communication with the display device <NUM>, wherein the controller device <NUM> is further configured to direct the display device <NUM> to display the first virtual element <NUM>. In such aspects of the present disclosure, the physical control element <NUM>, in communication with the controller device <NUM>, may be particularly configured to correspond to the first virtual element <NUM>. As such, in particular aspects, an actual or physical manipulation of the control element <NUM> by the user is displayed (i.e., via cooperation between the processor of the controller device <NUM>, the control element <NUM>, and the display device <NUM>) on the display device <NUM>, wherein the actual manipulation of the physical control element <NUM> by the user is manifest and represented by a response of the first virtual element <NUM> displayed on the on display device <NUM>, as if the first virtual element <NUM> was subjected to the actual manipulation of the control element <NUM>. One skilled in the art will appreciate that such aspects are fundamentally different from manipulating a virtual object on a display device in a conventional manner using, for example, a mouse / keyboard or stylus pointer. In particular aspects, the correlation / contemporaneousness between the manipulation of the control element <NUM> and the response of the first virtual element <NUM> displayed on the display device <NUM> may be arranged to occur in real time or substantially real time, such that the immediacy of the response of the first virtual element <NUM> to the manipulation of the physical control element <NUM> is apparent to the user.

For example, as shown in <FIG>, if the first virtual element <NUM> was configured as a threaded post (i.e., a dental implant configured as a threaded screw for forming a threaded engagement with a bore formed in the patient's jawbone structure), the control element <NUM> may be generally configured as an elongate member (i.e., a stylus), but may also be specifically configured as a threaded screw, or configured as any suitable corresponding object representative of a threaded screw (see, e.g., <FIG>). Such a threaded post, in its actual functionality, must be rotated about the longitudinal axis thereof in order to form the threaded engagement, and the virtual representation thereof thus retains the same functionality for the planning purposes disclosed herein. As such, the control element <NUM> (i.e., the stylus) may be configured to cooperate with the controller device <NUM> such that actual rotation of the stylus (control element <NUM>) about the longitudinal axis thereof is manifest or otherwise realized on the display device <NUM> as the first virtual element <NUM> (i.e., the threaded post) rotating about the longitudinal axis thereof, as if it were being manipulated in the same manner as the control element <NUM>. Moreover, any other manipulation of the control element <NUM> would be manifest and realized as a corresponding response of the first virtual element <NUM>. For instance, if translational motion (i.e., motion in any, all, or any combination of the x-, y-, and z-axes in a three-dimensional environment) or rotational motion (i.e., rotation about any, all, or any combination of the x-, y-, and z-axes in a three-dimensional environment) is imparted to the control element <NUM>, the display device <NUM> would display the first virtual element <NUM> (i.e., the threaded post) as translating and/or rotating in the same manner as the control element <NUM>, as if it were being manipulated in the same manner as the control element <NUM>.

Such aspects can be described in different manners. For example, the control element <NUM> may be arranged as a proxy for the first virtual element <NUM> in regard to the manipulation of the first virtual element <NUM>. As such, the control element <NUM> is the recipient of the manipulation or the object being manipulated, while the first virtual element <NUM> displayed on the display device <NUM> demonstrates or otherwise reflects the response of the first virtual element <NUM> to the manipulation (i.e., as if the first virtual element itself is being manipulated by the user, instead of the control element <NUM>). In other instances, the physical manipulation of the control element <NUM> is replicated by the controller device <NUM> as the response of the first virtual element <NUM> displayed on the display device <NUM>. In embodiments of the claimed invention, the control element <NUM> has six degrees of freedom (i.e., configured as a body in a three-dimensional environment, wherein the body is free to move forward/backward, up/down, left/right (i.e., translation in three perpendicular axes) combined with rotation about the three perpendicular axes (i.e., pitch, yaw, and roll), with the first virtual element <NUM> correspondingly displayed to have the same degrees of freedom as the control element <NUM>. As such, one skilled in the art will appreciate that the control element <NUM> (i.e., a physical object) is used to manipulate a corresponding virtual object (i.e., the first virtual element <NUM>) in the virtual environment represented by the display device <NUM>.

In some aspects, a plurality of virtual elements may be displayed or selectively displayed (i.e., by user selection) via the controller device <NUM> on the display device <NUM>. For example, in addition to the first virtual element <NUM>, a second virtual element <NUM> (see, e.g., <FIG>) may be displayed by the controller device <NUM> on the display device <NUM>. In one instance, the first virtual element <NUM> may comprise a dental implant, as previously discussed, while the second virtual element <NUM> may comprise a surgical apparatus, such as a surgical instrument for performing a surgical procedure (i.e., used to prepare the site within the patient's mouth for receiving the dental implant). In such instances, a selector device <NUM> may be operably engaged with the control element <NUM>, wherein the selector device <NUM> may be configured to direct the controller device <NUM> to associate the control element <NUM> with the first virtual element <NUM> and/or the second virtual element <NUM>, or to dissociate the control element <NUM> from the first virtual element <NUM> and/or the second virtual element <NUM>. That is, the selector device <NUM> may be used to indicate to the controller device <NUM> as to the virtual element displayed on the display device <NUM> that is to be associated with or dissociated from the control element <NUM> (i.e., to establish or disestablish correspondence between the control element <NUM> and one or more of the virtual elements displayed on the display device <NUM>). In some aspects, the first virtual element <NUM> and/or the second virtual element <NUM> may be configured to interact with the other. For example, in some aspects, the second virtual element may comprise, for instance, a visual representation, whether in two dimensions or three-dimensions, of the patient's jawbone structure (see, e.g., element <NUM> in <FIG>). Accordingly, in such aspects, the first virtual element <NUM> may be manipulated via the control element <NUM> such that the first virtual element <NUM> (i.e., a dental implant or a surgical instrument) interacts with the patient's jawbone structure <NUM> (i.e., a second virtual element).

In particular aspects where the control element <NUM> comprises, for example, a stylus, the selector device <NUM> may be accomplished, for example, by contacting a tip of the stylus with the display device <NUM> in the portion of the display device <NUM> where the virtual element being selected or deselected is displayed. However, the selector device <NUM> may be implemented in many different manners, as will be appreciated by one skilled in the art. In some instances, the selector device <NUM> may be operably engaged with the control element <NUM>, and may comprise, for example, one of a touch-sensitive actuator, a pressure-sensitive actuator, an optical actuator, and an acoustic actuator, associated with the control element <NUM>. For example, the user's selection of a virtual element displayed on the display device <NUM> may be signified or otherwise actuated by way of, for example, a selector device <NUM> comprising a camera, an infrared sensor, an acoustic sensor, a range-finder, or other appropriate sensor that is engaged with or in proximity to the control element <NUM> to determine if an actual selection is made by the user. In embodiments of the invention the selector device <NUM> comprises an appropriate pressure switch or sensor mounted on the control element <NUM>.

The control element <NUM> and/or the selector device <NUM> may be established in communication with the controller device <NUM> and/or other components in the system <NUM>, for example, by way of an electrical communication system, a mechanical communication system, an electromechanical communication system, an optical communication system, and combinations thereof, or through any other suitable communication arrangement, as will be appreciated by one skilled in the art, wherein the communication system or communication arrangement may, for example, comprise a wireless communication system and/or a wired communication system.

Upon selection of the one or more virtual elements displayed on the display device <NUM>, the controller device <NUM> may be further configured to establish a correlation between the attributes of the control element <NUM>, and the characteristics of the selected virtual element on the display device <NUM> (see, e.g., <FIG>). For instance, reverting to the threaded post (i.e., dental implant) example, on selection of the virtual threaded post displayed on the display device <NUM>, the controller device <NUM> may be configured to correlate the longitudinal axes of the virtual threaded post and the stylus, as well as to ascertain the orientation of each (i.e., that the engaging end of the threaded portion of the threaded post corresponds to the tip of the stylus). This correlation may be established in different manners. For example, the control element <NUM> may be provided with one or more location and/or motion sensors (i.e., accelerometers or trackable location indicia incorporated in the control element <NUM>) that are arranged to be in communication with the controller device <NUM>, such that the location, orientation, or other data indicative of the attributes of the control element <NUM> are made known to the controller device <NUM> for correlation with the characteristics of the selected virtual element on the display device <NUM>. One skilled in the art will appreciate, however, that the location, orientation, or other data indicative of the attributes of the control element <NUM> may be known or otherwise determined in different manners. For instance, an optical arrangement (i.e., a stereoscopic camera or other imaging device) may be implemented to optically analyze the control element <NUM>, in conjunction with the controller device <NUM>, to evaluate the location, orientation, or other data indicative of the attributes of the control element <NUM> for correlation with the characteristics of the selected virtual element on the display device <NUM>.

According to various aspects, the control element <NUM> may have any suitable shape or form to be in correspondence to the selected virtual element, such that the virtual element may be moved around in the virtual environment, which tracking the movement of the control element <NUM> in an actual physical environment. Moreover, the concepts disclosed herein may be extended to other instances outside of dental implant procedures. For example, in relation to pre-operative planning applications, the concepts herein can be applied to any implant situation such as, for instance, knee and hip implants. Even though such implants may be significantly different from a dental implant discussed herein, a basic aspect may be applied in that a physical representation of the implant can be manipulated in actual physical space by a user, while a virtual representation of that implant is moved in the virtual environment in the same manner. In some particular aspects, any object can be, for example, printed as a scale model (i.e., using a 3D printer), and the manipulation thereof can be tracked (i.e., tracking the manipulation of the scale model could be used as the manipulation basis of the control element) to create an even more seamless interaction with the virtual representation of the object in the virtual environment (see, e.g., <FIG>). One skilled in the art will also appreciate that such concepts may also be implemented in regard to 2D display environments (i.e., standard computer screens) as well, even in instances, where a pseudo-three-dimensional representation may be provided in that two-dimensional environment.

Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these disclosed embodiments pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that embodiments of the invention are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the invention, as long as they fall within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe example embodiments in the context of certain example combinations of elements and/or functions, it should be appreciated that different combinations of elements and/or functions may be provided by alternative embodiments without departing from the scope of the disclosure.

In this regard, for example, different combinations of elements and/or functions than those explicitly described above are also contemplated within the scope of the disclosure. For example, aspects of the present disclosure may provide a method for planning a procedure, comprising displaying a first virtual element via a display device; analyzing, via a processor, physical manipulation of a control element interface configured to correspond to the first virtual element; and displaying, in response to the analysis of the physical manipulation of the control element interface, a response of the first virtual element corresponding to the physical manipulation of the control element interface.

Another aspect may provide a system comprising processing circuitry operatively coupled with a control element interface, wherein the processing circuitry is configured to cause the system to at least display a first virtual element on a display device; analyze physical manipulation of a control element interface configured to correspond to the first virtual element; and display, in response to physical manipulation of the control element interface, a response of the first virtual element corresponding to the physical manipulation of the control element interface.

A further aspect may provide a computer program product comprising at least one non-transitory computer readable storage medium having computer readable program instructions stored thereon. The computer readable program instructions comprise program instructions which, when executed by at least one processor implemented on a system for planning a procedure, cause the system to perform a method comprising displaying a first virtual element via a display device; analyzing, via a processor, physical manipulation of a control element interface configured to correspond to the first virtual element; and displaying, in response to the analysis of the physical manipulation of the control element interface, a response of the first virtual element corresponding to the physical manipulation of the control element interface.

In particular aspects implementing a stylus as the control element, it may also be observed that, by the user grasping the stylus as if it was, for example, a dental implant (i.e., rather than in a conventional manner wherein the stylus is used as a pointer), the controller device can correlate the virtual representation of the dental implant with the physical attributes of the stylus, such that the virtual dental implant displayed on the display device can be manipulated by physically moving the stylus as if it was the virtual dental implant in a 3D virtual environment. In such a manner, the user in a 3D environment may now have a full 3D interactive experience by being able to visualize in real time the interactivity of the virtual dental implant in the 3D environment by moving a physical 3D object, which is similar to the virtual object being virtually represented in the 3D environment.

Claim 1:
A system for planning a procedure, comprising:
a display device (<NUM>) configured to display a first virtual element (<NUM>), the first virtual element (<NUM>) comprising a surgical apparatus including one of a dental implant and a surgical instrument configured to prepare a site on a jaw structure to receive the dental implant;
a controller device (<NUM>) having a processor and being configured to be in communication with the display device (<NUM>), the controller device (<NUM>) being further configured to direct the display device (<NUM>) to display the first virtual element (<NUM>); and
a physical control element (<NUM>) in communication with the controller device (<NUM>), the physical control element (<NUM>) being configured as a corresponding scale model of the first virtual element (<NUM>) and having six degrees of freedom of movement, wherein the processor is configured to analyze an actual manipulation of the physical control element (<NUM>), and wherein the actual manipulation of the physical control element (<NUM>) in each degree of freedom of movement is the same as a virtual manipulation of the first virtual element (<NUM>) displayed, via the analysis by the processor of the controller device (<NUM>), on the display device (<NUM>) as a corresponding response to the actual manipulation of the physical control element (<NUM>); characterized by:
a selector device (<NUM>) in the form of an appropriate pressure switch or an appropriate sensor mounted on the physical control element (<NUM>), the selector device (<NUM>) being configured to direct the controller device (<NUM>) to one of associate and dissociate the physical control element (<NUM>) with the first virtual element (<NUM>).