METHOD FOR ESTABLISHING A THREE-DIMENSIONAL REPRESENTATION OF BONE X-RAY IMAGES

Abstract: Method for establishing a virtual three-dimensional representation of a bone or bone fragment from X-ray images comprising the steps of A) taking at least one medical image (11) of a patient's bone (21) or bone fragment; B) storing the at least one medical image (11) as a set of data in a data storage means (9) of a computer (10); C) display the at least one medical image (11) at a display means (32) connected to the computer (10); D) creating a three-dimensional virtual representation of the bone (21) or bone fragment by means of using a control means (26) connected to the computer (10); and E) adjusting the size of the three-dimensional virtual representation to the size of the bone (21) or bone fragment by means of using the control means (26).

Description: Method for establishing a three-dimensional representation of bone x-ray images The invention relates to a method for establishing a virtual three-dimensional representation of a bone or bone fragment from X-ray images according to the concept of claim 1. In order to perform minimal invasive surgery quite often computer assisted surgery systems (CAS systems) or image guided surgery systems that are provided with a computer and a position measurement device in order to measure the position of surgical instruments, devices and a body portion of the patient are used. Such computer assisted surgery systems (CAS-systems) are disclosed e.g. in EP 0 359 773 SCHLONDORFF, US 5,383,454 BUCHHOLZ and in US 5,682,886 DELP. Often these CAS - systems comprise a memory means apt to store medical images such as e.g. X-rays, Computertomographs or MR images (Magnetic Resonance images). Thereby the medical images may be gathered pre-operatively or intraoperatively Currently, there are three classes of computer assisted orthopaedic surgery systems a) CT based systems, which use a preoperative CT (Computertomogram) of a bone or bone fragment to establish a three-dimensional anatomical model that is referenced with the intraoperative respective bone or bone fragment through landmark based or surface based registration or matching; b) CT based and fluoroscopy systems, which use the same method as CT based systems to establish a three-dimensional anatomical model, whereby the preoperative CT of a bone or bone fragment is registered or matched to the intraoperative respective bone or bone fragment through using a surface model of the bone or bone fragment and its projections in the planes of the fluoroscopic images; BESTATIGUNGSKOPIE c) fluoroscopy based systems, which use calibrated fluoroscopes to generate undistorted images of a bone or bone fragment and virtual geometric representations of the projection of surgical tools. A method of generating tomographic images of a body using penetrating radiation is known from EP 1 004 272 LIN. This known method comprises the steps of - cycling the radiation source among a plurality of positions relative to the region of interest of the body such that radiation from the radiation source passing through each of a plurality of focal planes, which are parallel to the detector plane and within the region of interest, impinge upon the detector plane superimposed and offset from each other; - Shifting the electronic views for a first selected focal plane, such that the radiation which passes through each incremental element of the first selected focal plane contributes to a common pixel of the electronic views; and - Finally, summing the electronic views such that the pixels of each electronic view corresponding to the incremental element on the first selected focal plane are summed to generate a slice image taken through the first selected focal plane. This known method has the disadvantage that CT-scanning delivers cross-sectional images of the patient body whereas fluoroscopic X-ray images deliver two-dimensional images and therefore many X-ray images are required in several parallel planes and under several different angles of the radiation source to obtain a three-dimensional model. On this point, the invention intends to provide remedial measures. The invention is based on the objective of providing a method that allows to establish a virtual three- dimensional representation of a bone or bone fragment based on fluoroscopic images. Therewith: - the radiation exposure is reduced; - three-dimensional images instead of projected two-dimensional images may be used; and - closed (minimally invasive) surgical operation in reality is turned to open surgical operation in a virtual world. The invention solves the posed problem with a method that displays the features of claim 1. The method according to the invention concerns the establishment of a virtual three- dimensional representation (3D model) from X-ray images and essentially comprises A) positioning an X-ray device such that the region of interest at the patient's body may be mapped on the plane of projection of the X-ray device; B) taking at least one medical image of the region of interest mapped on the plane of projection and storing this first image as a set of data in a data storage means of a computer. Today, image data acquisition is computerized and the images are delivered in digital form as a matrix of currently typical 128 2 to 1024 2 picture elements; C) display the at least one medical image at a display means connected to the computer, whereby the display means may be the display of the computer or e.g. a head mounted display; D) creating a three-dimensional virtual representation of the bone or bone fragment by means of using a control means connected to the computer; and E) adjusting the size of the three-dimensional virtual representation to the size of the bone or bone fragment by means of using the control means. The creation and adjustment of the virtual three-dimensional representation using the control means may be performed visually through using a keyboard or virtual keyboard, a mouse or a pointer as control means and determining the shape and dimensions visually at the display means. The preferred application of the method according to the invention further comprises the steps of: F) taking a second medical image of a patient's bone or bone fragment at an angle with respect to the first medical image; G) storing the second medical image as a set of data in the data storage means of the computer; H) display the first and second medical image at the display means; I) creating a three-dimensional virtual representation of the bone or bone fragment by means of using a control means connected to the computer; and K) adjusting the size of the three-dimensional virtual representation to the size of the bone or bone fragment by means of using the control means. Furthermore, the method according to the invention preferably comprises the step of reconstructing an axis, e.g. the central axis of the at least one bone or bone fragment by - defining respective points on the at least one medical image represented on the display means, e.g. the display of the computer. The definition of these points may be performed e.g. with a keyboard or virtual keyboard, a mouse or a pointer. Alternatively, an automatic axis identification by a computing means might be considered, e.g. by chamfer methods; - automatic surface or volume methods (chamfer methods); identification by a computing means might be considered, e.g. by chamfer methods; or - identification of anatomical points. The size of the virtual representation is thereby adjusted until the projection boundary size of the virtual representation is as close to the projected image boundary of the The reconstruction of an axis of the bone or bone fragment that is to be represented through the respective virtual representation is achieved by reconstructing two points on the specific axis. Each point is reconstructed through choosing projection points from at least two acquired images. Preferably, the computer is provided with an algorithm that allows the projection of the X-ray beam for each selected point on the other images through a thin line. This allows verification of the selected point and facilitates the identification of the other point that must be chosen on the displayed line. The definition and adjustment of the virtual representation is preferably performed in a perspective representation determined from the projected images taken with the C-arm X-ray device. After the virtual representation is defined and adjusted to the images of the bone or bone fragments a check is performed whether the projected images of the bone or bone fragment are covered by the projected boundary of the virtual representation which is calculated through a specific C-arm X-ray projection model. Thereby, the projected images correspond to the images taken by means of the C-arm X-ray device under step B) and F). Since the virtual representation is defined from images of the bone or bone fragment that have a different axis of projection a three-dimensional representation with a central axis coinciding with the longitudinal axis of the bone or bone fragment may be established. Therewith, a more precise manipulation of the bone or bone fragments is possible. In a further development of the method according to the invention it comprises representing surgical tools and surgical implants at the display means connected to the computer together with the virtual representations of the respective bone or bone fragments. Furthermore, in order to plan the surgical action by means of particular interface means, e.g. a keyboard or virtual keyboard, a mouse or a pointer the method allows displacing the virtual bone representations, the surgical tools and surgical implants on the display means. Therefore, geometrical models of the surgical tools and implants that are provided by their manufacturers may be used. Furthermore, the method according to the invention may be used as a planning method for a subsequent surgical action using surgical navigation to perform the surgical action. Therefore, in order to use surgical navigation to perform the surgical action as planned with the above steps the method further comprises the steps of A) attaching a reference means at each bone or bone fragment involved in the surgical action; B) measuring the position and orientation of each reference means with respect to a system of coordinates by means of a position measurement device; C) measuring the position and orientation of a third reference means fixedly attached to a C-arm X-ray device having an axis of projection and whereof the position and orientation of the plane of projection with respect to the system of coordinates defined by the third reference means is known; and D) measuring the position and orientation of a fourth reference means fixedly attached to a surgical tool or implant. Fluoroscopy based systems allow superimposed real-time virtual line graphics visualization of the projection of surgical tools and bone fragments relative to the acquired C-arm X-ray images. Therefore, a comprehensive calibration procedure has to be applied where all physical parameters are determined that specify imaging attributes. These parameters are then recalled and combined with the C-arm X-ray device's current spatial position and orientation with respect to the patient's anatomy. Advantageously, the pre-calibration of the C-arm X-ray device is divided into three steps: extrinsic, intrinsic and mechanic calibration. The extrinsic calibration provides external geometric parameters describing the X-ray projection such as the initial focal point (X-ray emitter) and image plane positions. The intrinsic calibration extracts parameters quantifying the distortions caused by the electronic optics of the image intensifier. It allows a correction for these distortions in every acquired image. As most C-arm X-ray devices are subject to significant elastic frame deformations when their position is changed, a mechanic calibration compensates the related variations of the focal point. The deformations of the C-arm X-ray device are mapped over the whole range of motion to be able to interpolate between these values. The reference means preferably comprise at least three markers that are non-collinearly arranged. The markers as well as the detectors of the position measurement device may be acoustic or electromagnetic effective means such as energy emitting, receiving or reflecting means. For instance as energy emitting means: - Light sources, particularly light emitting diodes (LED's); - Infrared light emitting diodes (IRED's); or - Acoustic transmitters or as energy receiving means: - Photodiodes; or - Microphones may be used. Other position measurement devices contain coils as energy emitting means and Hall-effect components as energy receiving means may be used as well. Each reference means defines a local system of coordinates with a fixed mathematical relation to every point of the body e.g. the bone, bone fragment, surgical tool or surgical implant where the reference means is attached to. The position measurement device may be connected to a computer. By means of the position measurement device coordinate transformations between any of the above systems of coordinates may be performed. A medical image taken by means of the C-arm X-ray device reflects to momentary position of the bone or bone fragment and must therefore be registered to each reference means attached at the bone or bone fragments. Therefore, matrices allowing coordinate transformations between the system of coordinates of the reference means at the C-arm X-ray device and the reference means attached at the bone or bone fragments are obtained by measuring the positions and orientations by means of the position measurement device and then stored at the acquisition time of the respective image in the data storage means of the computer. In case of optoelectronic position measurement, a custom optoelectronic position measurement device may be used e.g. an OPTOTRAK 3020 System, Northern Digital, Waterloo, On., Canada. It preferably comprises - an OPTOTRAK 3020 Position Sensor consisting of three one-dimensional charge- coupled devices (CCD) paired with three lens cells and mounted on a stabilised bar. Within each of the three lens cells, light from an infrared marker is directed onto a CCD and measured. All three measurements together determine - in real time - the three- dimensional location of the marker; - a system control unit; - a computer interface card and cables; - data collection and display software; and - a strober and marker kit. The advantages achieved by the invention are essentially to be seen in the fact that, thanks to the method according to the invention a three-dimensional representation of a bone or bone fragment may be established from two-dimensional C-arm X-ray images. This allows a higher precision when manipulating the bone or bone fragment during the surgical action than the present use of two-dimensional images produced by the C-arm X-ray device. Furthermore, the necessary amount of fluoroscopic images to be taken of the patient's body and therewith the time of exposure to radiation may be significantly reduced. It allows an optimal selection of the shape of surgical implants that shall be implanted into the patient's body by means of the planned surgical action e.g. size and length of an intramedullary nail, bending of a plate or position of screws. The method according to the invention is explained in even more detail with reference to the partially schematic illustrations. Shown are: Fig. 1 the performance of the method according to the invention by means of a C-arm X-ray device connected to a computer; Fig. 2 the establishment of the virtual three-dimensional representation by means of a computer; Fig. 3 the performance of a computer assisted close reduction of a fractured bone; and Fig. 4 the adjustment of the virtual three-dimensional representation to a bone or bone fragment using the method according to the invention. Fig. 1 depicts the steps of positioning a mobile X-ray device 6 (C-arm X-ray device) in different positions with respect to a bone 21 such that a region of interest of the bone 21 is mappable under different angles of view on the plane of projection 7 of the X-ray device 6, taking a first and second image 11 ;13 (Fig. 2) and storing the images 11 ;13 as a first and second set of data in a data storage means 9 of the computer 10. The images 11 ;13 which are taken may be displayed at a display means 32 which is e.g. the display 20 of the computer 10. Other display means such as a head mounted display may be used as well. Optionally, a position measurement device 4 may be applied. By means of the position measurement device 4 the position of the markers 29 (Fig. 3) attached at the reference means 5 at the X-ray device 6 with respect to an on-site three-dimensional system of coordinates 3 may be measured. This additional step allows to determine the position and orientation of the plane of projection 7,7' in each selected position of the X-ray device 6 with respect to the on-site system of coordinates 3. If the selected virtual representation is a symmetrical representation as e.g. a virtual cylinder coaxial to the specified axis (Fig. 2) of the bone 21 the position measurement of the X-ray device 6 is not necessary. Fig. 2 depicts the establishment of the three-dimensional representation at two images 11 ;13 of a fractured bone 21 , whereby the image 11 is taken at an anterior-posterior view and the image 13 is taken at a lateral-medial view of the region of interest of the bone 21. Using a mouse as interface means 26 a first point 14';14" and a second point 15';15" may be selected in each image 11 ;13 therewith specifying a distinctive axis 16';16" of the bone 21. Once the specific axis 16 of the bone 21 is determined a virtual representation is selected. Here, the virtual representation is a virtual cylinder 17, whereby the anterior-posterior projected virtual cylinder 17" is represented on the first image 11 and the lateral-medial projected virtual cylinder 17' is represented on the second image 13. With a mouse as interface means 26 the size of the virtual cylinder 17 may be adjusted to the size of the bone 21 by adjusting one or both of the projected virtual cylinders 17'; 17" to the displayed bone 21. Fig. 3 depicts a computer assisted closed reduction of a fractured bone 21. After a virtual representation of the proximal and distal bone fragments 18; 19 is established and displayed at e.g. the display 20 of the computer 10 a set of image data of a surgical tool 22, e.g. a drill drive is loaded into the processor 27 (Fig. 2) of the computer 10. A forth reference means 28 is attached at the drill drive so that the relative position of the drill drive with respect to the first and second reference means 1 ;2 may be determined by measuring the position of the markers 29 on each of the reference means 1 ;2;28 with respect to the three-dimensional system of coordinates 3 by means of the position measurement device 4. In order to register the images 11 ;13 taken by the X-ray device 6 the surgeon manually identifies an anatomical landmark at the bone fragment 18; 19 on the two images 11 ;13 (Fig. 4) represented at a display means and then the related three-dimensional position of the anatomical landmark relative to the corresponding reference base 1 ; 2 is determined. This step is performed through coordinate transformations between the system of coordinates affixed to the images, the system of coordinates affixed to the plane of projection 7, the system of coordinates affixed to the third reference means 5 attached at the X-ray device 6 and the on-site system of coordinates 3. Such that the virtual representation may be used to intraoperatively visualize the position of the virtual representation directly and based on the measured positions of the bone fragments 18; 19. Therewith the surgeon may perform the alignment of the bone fragments 18; 19 with direct visual control. As the surgeon displaces the bone fragment 18;19 the corresponding virtual representation is moved as well on the display 20. Similarly, a surgical tool 22 or surgical implant 23, e.g. an intramedullary nail may be intraoperatively guided with direct visual control at the display 20.

Claims: Claims 1. Method for establishing a virtual three-dimensional representation of a bone or bone fragment from X-ray images comprising the steps of A) taking at least one medical image (11) of a patient's bone (21) or bone fragment; B) storing the at least one medical image (11 ) as a set of data in a data storage means (9) of a computer (10); characterized in that the method further comprises the steps of C) display the at least one medical image (11 ) at a display means (32) connected to the computer (10); D) creating a three-dimensional virtual representation of the bone (21) or bone fragment by means of using a control means (26) connected to the computer (10); and E) adjusting the size of the three-dimensional virtual representation to the size of the bone (21 ) or bone fragment by means of using the control means (26). 2. Method according to claim 1 , characterized in that it further comprises the steps of: F) taking a second medical image (13) of a patient's bone (21 ) or bone fragment at an angle with respect to the at least one medical image (11 ); G) storing the second medical image (13) as a set of data in the data storage means (9) of the computer (10); H) display the first and second medical image (11 ;13) at a display means (32); I) creating a three-dimensional virtual representation of the bone (21 ) or bone fragment by means of using a control means (26) connected to the computer (10); and K) adjusting the size of the three-dimensional virtual representation to the size of the bone (21) or bone fragment by means of using the control means (26). 3. Method according to claim 1 or 2, characterized in that it further comprises the step of reconstructing an axis (16) of the bone (21 ) or bone fragment (18; 19) at the display means (32) by using a control means (26) and by defining at least two respective points (14; 15) on at least one of the images (11 ;13) represented on the display means (32). 4. Method according to claim 1 or 2, characterised in that it further comprises the step of reconstructing an axis (16) of the bone (21) or bone fragment (18; 19) through an automatic axis identification. 5. Method according to claim 3 or 4, characterized in that the virtual representation comprises a enveloping surface (25) of a bone (21 ) or bone fragment (18; 19), said enveloping surface (25) being aligned to the axis (16). 6. Method according to claim 3 or 4, characterised in that the virtual representation comprises a body representing a bone (21) or bone fragment (18; 19), said body being aligned to the axis (16). 7. Method according to claim 1 , characterised in that it further comprises the steps of a) loading a set of image data of a surgical tool (22) into a processor (27) of the computer (10); b) representing the surgical tool (22) at the display means (32) together with the virtual representation of a respective bone (21) or bone fragment (18; 19); and c) displacing the virtual representation and/or the representation of the surgical tool (22) on the display means (32) by using a control means (26) connected to the computer (10).. 8. Method according to claim 1 or 7, characterised in that it further comprises the steps of d) loading a set of image data of a surgical implant (23) into the processor (27) of the computer (10); e) representing the surgical implant (23) at the display means (32) together with the virtual representation of a respective bone (21 ) or bone fragments (18; 19); and f) displacing the virtual representation and the representation of the surgical implant (23) on the display means (32) by using a control means (26). 9. Method according to one of the claims 1 to 8, characterised in that it further comprises the step of rotating the virtual representation on the display means (32) by using a control means (26) in order to receive a perspective view of the region of interest of a bone (21 ) or bone fragments (18; 19). 10. Method according to claim 8 or 9, characterised in that it further comprises the step of simulating a surgical action at the display means (32) by means of adjusting the virtual representation and arranging the surgical implant (23) at its desired position. 11. Method according to one of the claims 8 to 10, characterised in that it further comprises the steps of g) attaching a reference means (1 ;2) at each bone (21) or bone fragment (18;19) of interest; h) measuring the position and orientation of each reference means (1 ;2) with respect to a system of coordinates (3) by means of a position measurement device (4); i) measuring the position and orientation of a further reference means (5) fixedly attached to a mobile X-ray device (6) having an axis of projection (12) and a plane of projection (7), whereby the position and orientation of said plane of projection (7) with respect to the system of coordinates (8) defined by the third reference means (5) is known; and j) measuring the position and orientation of a fourth reference means (28) fixedly attached to a surgical tool (22).

Inventors: NOLTE LUTZ-PETER, ZHENG GUOYAN, NOLTE, LUTZ-PETER, ZHENG, GUOYAN
Applicants: NOLTE LUTZ-PETER, SYNTHES AG, SYNTHES USA, ZHENG GUOYAN, NOLTE, LUTZ-PETER, SYNTHES (U.S.A.), SYNTHES AG CHUR, ZHENG, GUOYAN
Application year: 2001
Main classification: A61B19
IPC codes: A61B6, A61B19