Source: https://patents.google.com/patent/DE69636176T2/en
Timestamp: 2020-03-31 17:31:59
Document Index: 797546650

Matched Legal Cases: ['art;\n7', 'art;\n8', 'art;\n9', 'art;\n10', 'art;\n19', 'art 125', 'art 503', 'art 503', 'art 504', 'art 508', 'art 508', 'art 505', 'art 508', 'art 505', 'art 508', 'art 504', 'arts 505', 'art 503', 'art 503', 'art 608', 'art 608', 'art 505', 'art 505', 'art 508', 'art 508', 'art 508', 'art 508', 'art 508', 'art 508']

DE69636176T2 - System for supporting remote controlled surgery - Google Patents
System for supporting remote controlled surgery
DE69636176T2
DE69636176T2 DE69636176T DE69636176T DE69636176T2 DE 69636176 T2 DE69636176 T2 DE 69636176T2 DE 69636176 T DE69636176 T DE 69636176T DE 69636176 T DE69636176 T DE 69636176T DE 69636176 T2 DE69636176 T2 DE 69636176T2
DE69636176T
DE69636176D1 (en
Masakatsu Ushiku-shi Fujie
Isao Nishiibaraki-gun Nakajima
Fujio Niihari-gun Tajima
Hiroshi Matsudo-shi Takeuchi
Norihiko Musashimurayama-shi Wada
1995-02-16 Priority to JP02839195A priority Critical patent/JP3539645B2/en
1995-02-16 Priority to JP2839195 priority
2007-07-12 Publication of DE69636176T2 publication Critical patent/DE69636176T2/en
The The invention relates to a system for assisting remote-controlled surgery, In particular, a system for assisting remote surgery that suitable is surgical surgeons during surgical treatment of the brain, the nervous system, the eyeballs or the like remote-controlled surgical instruments or therapeutic instruments to assist by means of a manipulator with a high degree of freedom.
So far is as a system of implementation an operation on cranial nerves or the like, in which the data the measuring instruments are visibly reconstructed, a treatment system with surveillance known on the basis of magnetic resonance, in which the position of a Catheter automatically controlled by a positioning device when a surgical surgeon's position and insertion of the catheter while entering he sees an MRI image (Magnet Resonance Imaging). Such a thing System is disclosed, for example, in JP-A-4-53533.
Of Further is as a system for brain surgery a manipulator for stereotactic puncture known, for example, in the Japanese patent application No. Hei. 3-121064 is disclosed. As a system for carrying out a Surgery by remote control is a system with a remote-controlled surgical manipulator in a double-tube probe as disclosed, for example, in JP-A-4-146097.
Of Further, a micro-handling system is known whose construction it allows the Turning and moving a manipulator and a table degrees of freedom assign. Such a system is from a collection of publications known, Symposium on Robotics Mechatoronics in 1993, The Japan Society of Mechanical Engineers, pages 693-696.
The The system described in JP-A-4-53533 is primarily intended to to carry out a treatment at the main A catheter is used and it is difficult with this system to perform an operation the direct treatment of diseased areas with advanced technical skills requires, such as the removal of a tumor, the on a blood capillary or a nerve hangs.
Besides that is It is difficult to increase the compliance to work to treatment to allow soft tissue as an ultrasonic motor (piezoelectric element) as an actuator is used for operation in a static magnetic field.
The System is as well more or less incomplete in terms of how the data will be made available there the modality in question is just magnetic resonance, so that it will carry out Craniotomy difficult changes in the shape of a skull bone to understand the measured contents show only forms and it is implemented no functional measurement, and the measured and represented Contents show a two-dimensional tomographic image, not that is intuitive.
The in the above mentioned JP-A-3-121064 described system is used for the stereotactic and is only able to perform punctures. It However, there are considerable difficulties with cerebral nerves that are exclusively due to Puncture can not be treated well by surgery, and the require a plurality of higher degree of freedom devices, to treat the tissue of diseased areas. The above-mentioned system is unable to provide such treatments. Furthermore is nothing relative disclosed further parts for controlling the manipulator for puncture.
The in the above mentioned JP-A-4-146097 disclosed system isolated the surgical surgeon Completely from the patient so that it will have difficulty on one Emergency response and even if it were able, it could considerably too late be.
The in the collection of publications of the Symposium on Robotics Mechatoronics, The Japan Society of Mechanical Engineers, 1993, is constructed such that the work can not be started, as long as not an object placed on the table. In addition, this system is for actual surgical work not suitable.
In addition, will in all the above-described examples of improvement of works and the simplification of the control paid no attention.
In In the known examples described above, the data is converted exclusively of the surgical surgeon will get to the end, regardless of whether before or during the operation.
In addition, show they do not have a surgical simulation function and can not benefit from the result, regardless of whether it is in the Object of the surgical procedure around an actual "dummy" living tissue or a model in one Computer acts.
WO 93/13916 describes a telemani telepresence pulsator system which can be used for surgical procedures and which has left and right hand controls for controlling a manipulator, respectively for the right and left hand, through the use of a servo which includes a computer. Video cameras monitor a workspace from different angles to produce a stereoscopic signal as output on lines. Based on the signal output of the camera, a 3D image is generated and reflected in the direction of the surgeon's eyes. A virtual image is generated in the vicinity of the control arms, which is seen by the operator looking towards the control arms. By placing the working space image close to the control arms, the operator is made to feel as if the end effectors carried by the manipulator arms and the control arms are substantially integral, and the feeling that he controls the end effectors directly by hand.
Various Concepts of a robotic workstation for computer-aided surgery are by P. Dario, S. Matelli and A.M. Sabatini in the article 'An experimental set-up for investigating sensorbased tele-operated surgery procedures in Proceedings of the Twelfth Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Volume 12, 1990, Page 1942 published has been.
In The journal Surgical Enthroscopy 7 (1993) No. 3 is on the Pages 203 to 205 a publication from R.M. Satawa entitled "Virtual Reality Surgical Simulator " which has an overview of the Development of surgical simulators for Virtual Reality as an example of use the computer simulation in the context of surgery.
Accordingly It is the object of the invention to provide a system for supporting remote-controlled surgery to provide for support of surgical operators in the surgical treatment of Brain, nervous system, eyeballs or the like remote-controlled surgical instruments or therapeutic instruments by means of a manipulator with high degree of freedom while keeping the data different See measuring instruments in visually reconstructed form.
A Another object of the invention is a system for supporting remotely controlled Provide surgery of the "master / slave" type, that's the implementation allows surgical work, the one high degree of freedom and a plurality of cooperation within the limited space at the terminal.
A Another object of the invention is a system for supporting remotely controlled To provide surgery that reduces the loss of work skills of the surgical surgeon compensated by deterioration of sight and deterioration of terminal resolution Aging is caused.
It It is another object of the invention to provide a system for supporting remotely controlled To provide surgery that involves a blood infection between the patient and the surgeon prevented.
It It is another object of the invention to provide a system for supporting remotely controlled To provide surgery that involves surgical treatment less intervention, mainly by exploiting the Degeneration of tissue.
It It is another object of the invention to provide a system for supporting remotely controlled To provide surgery that is consistent with each other execution from surgical training over the surgical simulation, with the consent of the informed Patients the results of this can be used until towards the surgical operation.
These Tasks are performed by a system to support remote-controlled surgery solved that having the features of claim 1.
The remote surgery assist system of the present invention for assisting surgical procedures of surgical operators remotely controlling a surgical tool or therapeutic instrument includes means for manipulating diseased tissue consisting of the surgical tools or therapeutic instruments; an in-vivo data measuring device for measuring in vivo data by periodically applying one or more fluctuating magnetic fields, electromagnetic waves and / or ultrasonic waves to a diseased area and surrounding area before and during the operation and measuring the penetrated one Signal or the resonance signal; a measurement data processing device for generating measurement data images in 3D representation from the in-vivo data measured by means of the in-vivo data measurement device; means for acquiring data in the work environment for acquiring image data of the diseased area and detecting an approach state and a touch force of the diseased tissue manipulation means on the diseased area; means for generating reality control data for synthesizing and processing the output of the means for acquiring data in the Working environment and those of the measurement data processing device to present them to each operating surgeon as reality control data; Input means for entering action commands for entering actions presented by each operating surgeon on the basis of the reality control data presented to each surgeon operating by the means for generating reality control data; and manipulation command generating means for converting the action command outputted from the action command input means into manipulation command data for transmitting said data to said diseased tissue manipulation means and for transmitting touch force detected by said work environment data sensing means to said means for Manipulation of diseased tissue.
A Device for positioning the surgical tools or the therapeutic instruments of the manipulator for treatment Diseased tissue of the present invention is made of a material manufactured and engineered with a drive gauge that less are sensitive to the magnetic field.
The Realism control data that from the reality control data generator of the present invention further comprise at least a dataset from the following group: a virtual image that presented to the surgical surgeon by synthesizing with the image data acquired by the working environment data detecting means be recorded, and the measured image data obtained by the Measurement data processing device are generated; a virtual one Sound field presented to the surgical surgeon as sound data becomes; and virtual force reflection data representative of the surgical Operator by combining with the contact force in the manipulation command generating means to get presented.
The Manipulation command generating means of the present invention transmits a synthesized force reflection by combining the through the Working environment data detecting means detected force sensor data and by the reality control data generator generated virtual force reflection was obtained over the Action command input device to each surgical operator.
According to the invention is further the device for manipulating diseased tissue via the surgical Instrument or the therapeutic instrument on the diseased Part positioned when the device is the manipulation command as input receives and generates deformation, destruction or degeneration of the diseased tissue by generating or transmitting of at least one energy from the kinetic energy group, Light energy, electrical energy and thermal energy.
The System for support Remote-controlled surgery of the present invention also includes a Data storage means for storing by the reality control data generating means generated reality control data and / or the image data generated by the measurement data processing means to provide an additional Training function for Surgical surgeons by simulation of surgical work under Use of data stored in the memory device or to add a function for displaying the stored in the Speicherein direction Data to explain the condition of the disease.
In addition, will according to the present invention Models for generating the reality control data and / or the measured image data stored to produce a Training function of surgical operators by simulation the surgical work by using this model or for generating a function for representing the memory device stored data for explanation of the disease state.
Furthermore includes a device for assisting remote surgery for the support Surgical surgeons doing surgical work on one Diseased part by operation one with a surgical tool or a therapeutic one Instrument equipped "slave" manipulators Manipulate a "master" manipulator, the Actuation of a "slave" manipulator a combined command obtained by specifying the output of the Action command from the "master" manipulators of surgical operators with a predetermined weighting factor is multiplied and these are then added.
As As described above, the present invention is capable of Remote-assisted system To realize surgery to assist surgical operators in the surgical treatment of the brain, the nervous system, the eyeballs or the like by remote controlled surgical instruments or therapeutic instruments by means of a manipulator with high Degree of freedom while the data of different measuring instruments are visually reconstructed.
In addition, the present invention can realize a "master / slave" type remote control assisted surgery system having a Performing surgical work that requires a high degree of freedom and a variety of collaborations at the terminal within a limited space allowed.
Furthermore allows the present invention, the realization of a system for supporting remotely controlled Surgery involving the loss of surgical skill Surgeons compensated by the deterioration of eyesight and the deterioration of terminal resolution caused by aging becomes.
The present invention enables also the realization of a system for supporting remote-controlled surgery, that is a blood infection between the patient and the surgical Operator prevented.
The present invention enables the realization of a system for supporting remote-controlled surgery, which realizes a surgical treatment with less intervention, mainly by exploiting the degeneration of tissue.
The present invention enables the realization of a system for supporting remote-controlled surgery, that allows surgical operators to do their work skills to transfer with each other.
The present invention enables the realization of a system for supporting remote-controlled surgery, consistent with each other performing surgical training over the surgical simulation, with the consent of the informed patient The results of this can be used up to surgical Operation allows.
1 Fig. 12 is a block diagram showing an embodiment of the remote surgery assisting system according to the present invention;
2 Fig. 10 is a block diagram showing a structural example of the reality control data generating means;
3 Fig. 10 is a block diagram showing a structural example of a manipulation command generating means;
4 Fig. 10 is a block diagram showing a structural example of a work environment data detecting means;
5 Fig. 10 is a block diagram showing a whole structure of a diseased tissue treatment manipulating device;
6 Fig. 10 is a schematic diagram showing a structural example of a coarse moving part;
7 Fig. 10 is a schematic diagram showing a fine movement part;
8th Fig. 10 is a schematic diagram showing a micro-moving part;
9 Fig. 12 is a schematic diagram showing a first joint of the micro-moving part;
10 is a sectional view of two parts of the first joint;
11 is an enlarged view of the area A in FIG 11 ;
12 is an enlarged view of area B in FIG 10 ;
13 Fig. 10 is a schematic diagram showing a main drive gage of the first link;
14 is a schematic diagram of 2 from the top view from above;
15 Fig. 4 is a flowchart showing an example of an algorithm for generating visualized data of a voltage or distance distribution;
16 Fig. 10 is a flowchart showing an example of an algorithm for transferring the movement of the manipulator into color data;
17 Fig. 4 is a flowchart showing an example of an algorithm for converting the manipulator's movement into graphic data;
18 Fig. 10 is a flowchart showing an example of an algorithm for generating visualized data of a temperature distribution of a diseased part;
19 Fig. 10 is a flowchart showing an example of an algorithm for generating audible data of the voltage or distance distribution;
20 Fig. 10 is a flowchart showing an example of an algorithm for implementing the movement of the manipulator in sound field data;
21 Fig. 10 is a flowchart showing an example of an algorithm for generating audible data of the temperature distribution of the diseased part shows;
22 Fig. 10 is a flowchart showing an example of an algorithm for converting a movement of the manipulator into an actual sound;
23 Fig. 10 is a flowchart showing an example of an algorithm for generating a virtual force reflection from the distance values;
24 Fig. 10 is a flowchart showing an example of an algorithm for generating a virtual force reflection from the temperature distribution;
25 FIG. 10 is a flowchart showing an algorithm for operating a force sensor data operation unit; FIG.
26 Fig. 10 is a flowchart showing an algorithm for synthesized power reflection;
27 Fig. 10 is a flow chart showing an algorithm for operating a command conversion unit;
28 Fig. 15 is a diagram showing a data format of the manipulation command data;
29 Fig. 10 is a flowchart showing an algorithm for operating a virtual switch;
30 Fig. 10 is a block diagram showing a structural example of an action command input means;
31 Fig. 10 is a flowchart showing an algorithm for processing signals of the force sensor;
32 Fig. 10 is a flowchart showing an algorithm for controlling an actuator for controlling a parallax angle;
33 Fig. 10 is a flowchart showing an algorithm for controlling lighting;
34 Fig. 10 is a flowchart showing an example of an algorithm for performing an interpretation;
35 Fig. 10 is a flowchart showing an example of an algorithm of a magnetic force control device;
36 Fig. 10 is a schematic diagram showing an example in which a display is assigned to a plurality of surgical operators;
37 Fig. 10 is a schematic diagram showing an example in which a display is assigned to a surgical operator;
38 Fig. 15 are schematic drawings showing examples of a handle portion of the action command input device;
39 FIG. 10 is a flowchart showing an operating algorithm of a manipulated force calculating element by others; FIG.
40 Fig. 10 is a block diagram showing an in-vivo data measuring device; and
41 Fig. 10 is a block diagram showing a measurement data processing device.
The present invention will be explained below with reference to preferred embodiments in detail. 1 FIG. 10 is a block diagram showing an embodiment of a remote surgery assisting system according to the present invention, which includes a reality control data generation device 101 to output virtual force reflection data 108 , a synthesized processed image 109 and a virtual sound field, a manipulation device 102 for diseased tissue, part of which consists of a "slave" manipulator for output of "slave" manipulator position data 115 , a manipulation command generating means 103 for outputting manipulation command data 111 and compound force reflection 112 , a work environment data acquirer 104 for output of force sensor data 105 , Proximity sensor data 106 , visual sensor data 107 and magnification ratio data 116 , an in-vivo data measuring device 117 for outputting an input signal for measurement within the organism and internal measurement data of the interior of the organism 121 and for receiving an output signal 120 which has passed through or reflected in the interior of the organism, a measurement data processing means 118 for the output of visualized in vivo data 122 , which are reconstructed in 3D format or the like, and an action command input device 114 to issue an action command 113 is composed.
The work environment data detecting means 104 has sensor parts at the tip and around the "slave" manipulator, which is a component of the manipulation device 102 represents diseased tissue and the diseased areas and the surrounding areas as detection data 124 detected by an optical sensor, and a force sensor and a distance sensor on End of the mentioned manipulator.
The reality control data generation device 101 processes and synthesizes the work environment data acquisition means 104 acquired data, the in vivo reconstructed in vivo image data obtained by the measurement data acquisition device 118 and the "slave" manipulator position data 115 for generating image, sound and virtual power reflection. This shows the condition of the patient surgical surgeons.
The data of the force sensor and the distance sensor also become the manipulation command generating means 103 transfer. The actual force reflection sensed by the force sensor is converted to a magnitude that can be sensed by each surgical operator. The data generated by the reality control data generator 101 generated virtual force reflection is synthesized (combined) with the area-converted actual force reflection and the manipulation force of other surgical operators and sent to each surgical operator via the action command input device 114 transfer.
Each surgical operator gives on the basis of the reality control data generating means 101 data shown an action command via the action command input device 114 to the manipulation device 102 for diseased tissue. The action command is issued by the manipulation command generator 103 in manipulation command data 111 implemented.
The manipulation device 102 for diseased tissue interprets and solves the manipulation command data 111 with a parameter of magnification ratio data 116 to manipulate the diseased tissue ( 123 ).
At the same time gives the in vivo data measuring device 117 the measurement pulse signal 119 to the diseased part 125 periodically and receives an output signal ( 120 ) that passes through or is reflected by the diseased part. The signal is digitized and sent to the measurement data processing device 118 sent as measured data.
The measurement data processing device 118 periodically processes the obtained measured data and reconstructs the result as 3D image data.
The structure and operation of each device will now be explained in detail. 2 Fig. 13 shows a structural example of the reality control data generation means 101 , It includes a working environment data processor 201 , a binocular field of view controller 202 , a display 203 for displaying the processed image and a virtual sound field reproduction device 210 and outputs the synthesized image data 211 that interfere with the work environment data processor 201 generated virtual image data were synthesized, and the visual sensor data 107 which give an actual image and a control signal 213 of the binocular field of view controller 202 and the virtual sound field 110 outward by a number of surgical operators.
The working environment data processor 201 is with a secondary data storage section 214 equipped to receive data and is able to record time series data or the like from the data of each sensor of the group of optical, force and distance sensors. As will be described later, these data are used in a simulation and training of surgery.
14 is a drawing that depicts the representation of 2 viewed from above. The hands beyond the wrists of each surgical surgeon appear as if they were spikes 1401 of the "slave" manipulator.
When a majority of surgical operators are busy, it is beneficial to have a display 203 arrange so that it, as in 36 shown, can be used from multiple directions or to make an arrangement in which each display 203 associated with every surgical surgeon, as in 37 shown.
36 shows an arrangement for a relatively small number of surgical operators. In this case, each individual "slave" manipulator is associated with each surgical operator, and each surgical operator may feel as if both his hands extend into the display and continue.
In addition, the surgical surgeons refer to each other as the same image if you have a visually magnified image of the diseased part (as if a lens were present) and can talk directly to each other.
While in the arrangement of the 37 Each surgical operator feels as though his own hands are extending and continuing into the display, the actual "slave" manipulator is controlled by the manipulative power of each surgical operator, which is weighted and combined. This will be described later.
In the arrangement of 37 it is not necessary, the binocular field of view controller 202 to wear, because the display every surgical surgeon and it is desirable to use a method of realizing stereoscopy by displaying different images in both eyes, for example using lenticular lenses.
An example of the operation of the working environment data processor 201 is described below with reference to the 15 to 24 described. A process for generating the virtual image data is first based on the 15 to 18 explained.
In 15 will first in the steps 1501 and 1502 through the force sensor data 105 and the distance sensor data 106 detects and determines whether the tip of the "slave" manipulator is in contact or not. If the "slave" manipulator is not in contact, in step 1505 a distance from the target position of the tip of the "slave" manipulator based on the optical sensor data 107 and the distance sensor data 106 and the amount of distance is calculated using one or more representations from the type of color group (for example, further distance in red to nearer areas in violet), brightness of the color (for example, further distance with higher brightness toward closer areas lower brightness) and saturation of the color (for example, further distance with higher saturation towards closer areas with lower saturation) in step 1506 shown.
there is the proximity sensor for determining the distance data, for example by measuring a intensity a reflected ultrasonic or light wave and a corresponding one Reflection time formed.
When the "slave" manipulator is in contact, in step 1503 a voltage distribution on the diseased part based on the data 107 the visual sensor and the data 106 of the proximity sensor is estimated and the amount of voltage in each image is determined in step 1504 by the type of color (for example higher voltage in red and lower voltage in violet), and / or the brightness of the color (for example higher voltage with higher brightness and lower voltage with lower brightness) and saturation of the color (for example higher voltage with higher saturation towards lower voltage with lower saturation).
When the process in step 1504 or step 1506 ends, gets in step 1507 a virtual image composed of the color or the like which is representative of the distance or the voltage represented by superposition on the original image. That is, the location where the distance between the diseased part and the "slave" manipulator is shorter or at which more tension occurs is displayed more vividly or colored with a particular color.
The above-mentioned image may be superimposed on the in vivo reconstructed 3D data 122 or without overlaying it. In this case, it is possible to prepare different representations or to display these representations by opening several windows on the same display.
By the way said it is necessary to indicate whether the representation type, the Brightness and saturation the color representative for the Distance or voltage are when they are overlaid. As far as this is concerned, an arrangement is preferably used in this example, by an illustrated color of the manipulator itself can be distinguished.
16 Fig. 12 shows a case where the movement of the tip of the "slave" manipulator is displayed according to the display mode, brightness and saturation of the color. First, in step 1601 a positional deviation, a velocity and an acceleration of the "slave" manipulator based on the data 115 the peak position of the "slave" manipulator containing reference values and response values.
Next will be in step 1602 the values of the positional deviation, the velocity and the acceleration by the chromaticity (for example, higher values in red towards lower values in violet), the color brightness (for example higher values with higher brightness towards smaller values with lower brightness) and the saturation of the Color (for example, larger values with higher saturation and smaller values with lower saturation). At this time, the relationship between the positional deviation, the speed and the acceleration and the kind, brightness and saturation of the color is arbitrary and various ways are conceivable.
Last will be in step 1603 the performing colors are displayed around the tip of the manipulator in the original image by being superimposed on the original image. It can be overlaid on the in vivo reconstructed in vivo data 122 or without overlaying it.
17 Fig. 10 shows a case where an image converted from an infrared wavelength region into a visible light wavelength region is displayed. First, an infrared component at each point within the original image becomes the data 107 of the visual sensor in step 1701 collected. After that, in step 1702 only the wavelength of the infrared component of each point is shifted into the visible range while leaving the light intensity of the component. Last will be in step 1703 the picture is shown overlaid on the original picture. In this way, the above-described display method makes it possible to specify in advance the diseased area by visualizing an originally invisible temperature distribution, since the temperature of diseased areas of the tissue often differs fundamentally from that of the surrounding areas.
It is possible, too, the in-vivo data reconstructed as image, as for example by MRI, X-ray computed tomography (X-ray CT) and ultrasound computer tomography are given to use.
Preferably can also change the picture by overlaying of arrows and effect lines to indicate the movement of the "slave" manipulator within the environment understandable to do under explanation the situation with luminous letters onomatopoetic, d. H. mimicking words and mimetic words.
18 shows a corresponding example. First, in step 1801 a time deviation of a position vector of the peak or a deviation vector between the recorded times from the data 115 calculated from the peak position of the "slave" manipulator. Then in step 1802 the amount and the direction of the vector are indicated by a transparent arrow and the arrow becomes in step 1803 drawn from the top of the manipulator in the original image.
The through the in the 15 to 18 Virtual image data generated or synthesized as described above is presented to the operator (surgical operator) as a stereoscopic image by the binocular field of view control means 202 and the display 203 be operated synchronously to display the synthesized image.
If, as in 36 shown a display 203 from different directions, we could either provide a display mode in which the view of the left and right eyes are alternately obstructed, and the left and right eye images are displayed synchronously or provide a display mode in which two images are slightly parallaxed can be displayed by dividing a display into two displays that are viewed through special lenses.
If the display as in 37 shown assigned to each surgical operator, it is possible to realize a stereoscopic system in which no device must be worn, such as the lenticular lens system described above.
The in vivo reconstructed in vivo data 122 can also be superimposed on the image type described above. In this case, the image may be displayed by providing another display or by opening another window on the same display.
As described above, allow in the 15 to 18 shown algorithms that more reality is added in the actual image and in this way the operability of each surgical surgeon is increased.
In addition, can a gradually higher one Decision to be made as in the past by adding to same time on the in vivo data image and the actual one Picture is taken.
In addition, will allows, accurately grasp the position of the diseased area so that a accurate surgical operation on a barely visible morbid Tissue performed can be.
The process of creating the virtual sound field 110 through the working environment data processor 201 is described below with reference to the 19 to 22 explained. In 19 be first in the steps 1901 and 1902 through the force sensor data 105 and the proximity sensor data 106 detects and determines whether the tip of the "slave" manipulator is in contact or not.
If the "slave" manipulator is not in contact, the distance will be based on the data 107 the visual sensor and the data 106 of the proximity sensor in step 1905 calculated and the amount of distance will be in step 1906 by one or more types of representation from the group volume (for example, further distance with higher volume towards closer range with lower volume), timbre (the time change of the volume) (for example further distance with larger change to closer ranges with less change), musical interval (sound level) (for example, further distance with higher musical interval towards closer areas with lower musical interval) and timbre (the distribution of components of fundamental waves and higher vibrations) (for example, further distance with fewer components towards closer areas with more Components).
When the "slave" manipulator is in contact with the diseased area, a Stress distribution at the diseased area in step 1903 based on the data 107 the visual sensor and the data 106 of the proximity sensor and the magnitude of the voltage to a point closest to the tip of the manipulator point of the tissue is in step 1904 using one or more types from the group volume or volume (for example higher voltage with higher volume to lower voltage with smaller volume), timbre (the time change of the volume) (for example further distance with larger change towards closer ranges with lesser change), musical interval (sound level) (for example, further distance with higher musical interval towards closer areas with lower musical interval) and timbre (the distribution of components of first harmonic fundamental) (for example, further distance with fewer components towards closer areas with more components).
When the process with the step 1904 or 1906 ends up being in step 1907 the sound data described above by the device for reproducing the virtual sound field 210 reproduced. That is, the volume increases, the musical interval becomes higher, the timbre brightens, or the tone becomes metallic at the point where the distance between the diseased area and the slave manipulator is lower or where a higher voltage occurs , Incidentally, since it is necessary to be able to discriminate whether the virtual sound field represents the voltage distribution or the distance, it can be easily done by, for example, changing the musical interval or interrupting the sound.
20 Fig. 12 shows a flow chart in the case where the movement of the tip of the "slave" manipulator is displayed in accordance with the volume, the timbre, the musical interval and the timbre. First, in step 2001 a positional deviation, a velocity and an acceleration of the "slave" manipulator from the data 115 the peak position of the "slave" manipulator containing reference values and response values.
Next will be in step 2002 the values of the position deviation, the speed and the acceleration by the volume (for example higher values with higher volume towards lower values with lower volume), the timbre (change of time of the volume) (for example higher values with larger change towards smaller values with minor change), the musical interval (noise level) (for example higher values with higher musical interval towards smaller values with lower musical interval) and the timbre (distribution of the components of the first harmonic fundamental and higher vibrations) (for example higher values with less Components towards smaller values with more components). At this time, the relationship between the positional deviation, the speed and the acceleration and the volume, the timbre, the musical interval and the timbre is arbitrary and various ways are conceivable.
Last will be in step 2003 the representative sound through the device 210 reproduced to reproduce the virtual sound field.
21 Fig. 10 shows a process in which a temperature distribution obtained from an image in the infrared wavelength region is made to correspond to the change of the sound. First, in step 2101 from the data 107 of the visual sensor collected an infrared component at each point within the original image. Next will be in step 2102 a light intensity at the point closest to the "slave" manipulator is interpreted as a temperature and brought in accordance with any representation from the group volume, timbre, musical interval or timbre. Lastly, this light intensity will be in step 2103 through the device 210 reproduced to reproduce the virtual sound field.
On this way it will be possible Specify the diseased area from the outset by the originally invisible temperature distribution is made audible, as a Temperature of the diseased area of the tissue is often fundamentally different from that distinguishes the surrounding areas.
For the specification of the diseased area, it is also considered effective, the Brightness of the tip area of the "slave" manipulator from the image reconstructed in vivo data, such as by MRI, X-ray CT and ultrasound CT given image data, to collect and make them audible.
Preferably, a sound effect may also be generated to make the movement of the "slave" manipulator within the environment understandable, and to generate onomatopoetic words and imitative words simultaneously to explain the situation. 22 shows such an example.
In 22 will be in step 2201 a time deviation of the position vector of the peak or a deviation vector between accumulated times from the position data 115 calculated at the top of the "slave" manipulator. Then in step 2202 the The magnitude and magnitude of the vector represented by the volume, timbre, musical interval, timbre, and image orientation of the wind-cutting sound are displayed and in step 2203 through the device 2101 reproduced to reproduce the virtual sound field.
This is done by the working environment data processor as described above 201 generated virtual sound field is the operator through the device for playing the virtual sound field 210 presents. In this way, the use of the sound field allows more naturalness to be added and any surgeon to operate more easily.
A process for generating the virtual power reflection data 108 is explained below with reference to the 23 and 24 explained. The 23 shows a corresponding example. First, in the steps 2301 and 2302 from the data 105 and 106 of the force sensor and determines whether the tip of the "slave" manipulator is in contact or not. In step 2303 no action is taken if the "slave" manipulator is not in contact. When the "slave" manipulator is in contact, in step 2304 the distance r between the diseased area and the peak position and its differential value m-th order or difference value m-th order (m = 1, 2, ...) calculated:
wherein t n is a n (= 0, 1, 2, ...) - multiple sampling time and (·) t n has a value of a variable in the time t n denote.
Next will be in step 2305 the virtual force reflection Fv (r, r (1) , r (2) ...) that operates between the tip of the manipulator and the diseased area is calculated.
Then in step 2306 the data of the virtual force reflection 108 to the device for generating a manipulation command 103 transfer. For example, a potential is established that generates a high virtual recoil force when the distance between the diseased area and the "slave" manipulator is small.
On this way it will be possible a discontinuity the force reflection between the contact state and the non-contact state, which the surgeon otherwise might feel avoid, so that the surgeon can handle the manipulator without worrying about the transition between the contact state and the non-contact state be, and the operability will be raised.
A process of generating a virtual force reflection corresponding to a temperature distribution will be described below with reference to FIGS 24 explained.
An intensity distribution of light of the infrared range is in step 2401 from the data of the visual sensor 107 added.
Next will be in step 2402 taking into account that the intensity distribution is equal to the temperature distribution, a virtual force reflection Fv is calculated which corresponds to the light intensity at each point of the image and whose depth direction is opposite. It is then sent to the device for generating a manipulation command 103 transfer. In this way, contactless scanning is enabled in which a degree of temperature is sensed by a magnitude of force reflection.
A method for calculating the virtual force reflection Fv corresponds to the method described above. However, it is impossible to distinguish which value is displayed when the generated virtual force reflection and that due to the expiration in 23 obtained virtual force reflection at the same time generated and output. Accordingly, it is preferably decided in advance, for example, by switching the modes, which value is displayed.
The Providing the virtual force reflection of the tip of the manipulator or the virtual force reflection indicative of a temperature of the diseased area for the surgical surgeon allows it, more reality add and the surgeon can handle the manipulator more easily.
In addition, can an operation known as so-called contactless scanning can be performed by in vivo data, which are reconstructed in image form, for example in the form of brightness at the top of the manipulator by means of MRI, X-ray CT and ultrasound CT given image data, instead of the light intensity in the infrared wavelength range and by converting that data into force reflection through the above be removed described method. In the aspect of Specification of the diseased area and the speed of the Surgery, this is very effective.
As described above, the working environment data processor overlays and processes 201 the force sensor data 105 , the proximity sensor th 106 , the optical sensor data 107 , the data of the position of the tip of the "slave" manipulator 115 and the visualized in 3D form reconstructed in vivo data 122 , Based on this data, the working environment data processor transforms 201 the quality of this data or generates new data.
That is, the working environment data processor 201 transforming quality into something that can not be grasped by human sensitive organs into something that can be grasped by humans, transforms a domain by modifying a physical quantity that is out of range, that is human Sensitive organs can be detected to a value within the range and, so to speak, replace the sensitive organs by converting a variable - though detectable by human sensitive organs - into a size that is difficult to grasp intuitively into a size that is more easily grasped by detecting it in a different way becomes. Thereby, the naturalness of the operation can be controlled and the surgical capabilities of each surgical surgeon can be increased.
Next is the facility below 103 for generating a manipulation command explained. 3 is a block diagram showing a structure of the device 103 for generating a manipulation command showing a virtual switch 307 for controlling a transmission of an action command, a force sensor data controller 308 to strengthen the force sensor data 105 to convert it to an appropriate range and to perform a recursive calculation corresponding to the removal of noise, to actual power reflection data 309 to issue a command converter 310 for setting an action mode / control mode from the action command data 113 and for receiving all joint data from the device 114 for entering an action command, an A / D converter 318 , a D / A converter 319 , a calculation element 320 the manipulative power of other surgeons to provide a value obtained by adding and converting a weighted amount of manipulation (e.g., manipulation force) of each surgical operator out of themselves to this surgical operator as a force reflection and a force / torque converter 322 , The arrows in the figure indicate the flow of signals or data.
There are a number of channels for the signals of the synthesized force reflection 112 and the action command 113 which is equal to a number of the action command input devices 114 and the force sensor data 105 . 106 and the data 108 The virtual force reflection has a number of channels equal to the number of "slave" manipulators in the device 102 for the manipulation of diseased tissue and multiplied.
The operation of the force sensor data operating unit 308 , the calculation element 320 the manipulation force of the other and the force / torque converter 322 is explained below with reference to the 25 and 26 described.
As in 25 shown, converts the force sensor data operating unit 308 the size of the force sensor data 105 in step 2501 in an appropriate level for human muscle power and leads in step 2502 a recursive calculation corresponding to a low-pass filter for noise removal.
After in step 2503 every degree of freedom is weighted in step 2504 to generate the synthesized power reflection data, an output 321 of the calculation element 320 the manipulative power of the others and the value of the virtual power reflection data 108 and the generated synthesized force reflection data is added in step 2505 entered into the force / torque converter.
The force / torque converter 322 walks in step 2601 the synthesized force reflection data into each joint torque value of a force reflection generation unit of the action command input device 114 um and gives, as in 26 shown this through the D / A converter 319 as analog data out.
The output data is sent to the action command input device 114 transmitted and become a torque command of the drive of each Gelenkaktuators in step 2603 ,
The process described above is performed by a number of action command input devices 114 , ie a number of channels performed. The operation of the calculation element 320 the manipulative power of the others will be explained later in detail.
The operation of the command converter 310 is described below with reference to the 27 explained.
In step 2701 reads the command converter 310 a mode selection signal included in the signal from the action command input device 114 is included. He puts in step 2702 an action mode and gives in step 2703 the action mode to the manipulation device 102 for diseased tissue.
Then he puts in step 2704 using the action mode determines a control mode and gives the Control mode in step 2705 to the manipulation device 102 for diseased tissue. Depending on the action mode, some control modes are not allowed, so the control mode in step 2704 by automatic selection from other than the non-permissible control modes according to a correspondingly determined algorithm or by input from the input device 114 for action commands.
After setting the control mode and its output to the manipulation device 102 for diseased tissue, she receives in step 2706 all joint angle data through the A / D converter 319 and translate these in step 2707 in a work coordinate system.
After output of the reference value data of the tip position of the manipulator to the manipulation device 102 for diseased tissue in step 2708 check the action mode in step 2709 and puts it in a stop mode when the action mode is "Stop", and if not, it returns to the steps 2710 and 2711 to the step 2706 back.
One to the manipulation device 102 Diseased tissue transmitted data train includes a header 2801 , an action mode 2802 , a control mode 2803 and a data train of position and orientation data 2804 until an arbitrary time t n . These are sequentially sent to the manipulation device 102 transmitted to diseased tissue.
Next is the operation of a virtual switch 307 with reference to the 29 which shows a corresponding working algorithm. Referring to the figure will be in step 2901 detected a distance between the position of the tip of the manipulator and the diseased area and in step 2902 controls whether it is smaller than a certain value. If it is not less than a certain value, the process returns to step 2901 back.
If it is smaller than a certain value, the position and the speed of the tip of the manipulator in the vicinity of the diseased area by the proximity sensor in step 2903 detected.
Next, the magnitude of the velocity vector and the magnitude of each of its components, the magnitude of the virtual force reflection and the magnitude of each of its components and the magnitude of a velocity command vector and the magnitude of each of its components when the velocity-servo control mode is controlled, and if so all are less than a certain value, the process returns to step 2901 back, and if any of the conditions is not met, the control mode is switched to position control and the current position is set in the steps 2904 to 2910 set as a command value.
Of the The process described above is for all channels carried out. In this way, the command value is not changed when an abnormality occurs. Therefore, the safety of the work is increased.
The action command input device 114 is described below with reference to the 30 explained. Even if a plurality of action command input devices 114 is used, the structure of these devices is the same. It includes a magnet 3001 for coupling the handle and the force reflection generation unit, a control signal 3002 a solenoid for limiting the freedom of movement of the handle and a control signal 3003 a current of a solenoid for coupling, a handle restricting solenoid 3004 , a switchable switch 3006 for the action mode, a handle 3007 , a spherical coupler 3008 , spherical joints 3009 and direct-acting cylindrical electrostatic actuators 3010 ,
The synthesized power reflection decomposed per actuator 112 generated by the manipulation command generator 103 is output to each actuator as a control input 3011 fed.
Every actuator 3010 is thereby driven and generates a required force reflection as a whole. The displacement of each actuator 3010 is detected by a displacement sensor, not shown, and is called Output 3012 of the displacement sensor.
Every surgical surgeon holds the handle 3007 and, while sensing the power reflection, moves it to enter action data. The mode is activated by pressing the switch 3006 for switching the action mode on the handle to output a signal 3005 to set an action mode. At this time, the spherical coupling becomes 3008 between the handle 3007 and the force reflection generating unit coupled by magnetic force of the electromagnet.
The coupler 3008 is constructed such that it is controlled by the control signal 3002 of the solenoid for limiting the grip and by the control signal 3003 for the current of the electromagnet for coupling from the magnetic force control device 3001 in accordance with the action mode and the size of the input, and such that it is capable of changing a restriction on the degree of freedom.
35 is a flow chart illustrating the Be drove the magnetic force control device 3001 shows. Referring to the figure, the magnetic force control device reads 3001 in step 3501 the action mode and, if there is an action mode, which is a mode that in step 3502 just pretend the position, it triggers in step 3503 only the magnetic force of the electromagnet out.
Because the coupler 3008 , as in 30 is spherical, it is free with respect to its orientation. In other words, only three degrees of freedom in position can be achieved by operating the handle 3007 be entered.
On the other hand, a change in orientation by clamping the lower part of the handle by the solenoids 3004 are entered when all six degrees of freedom of position and orientation are to be entered, as in step 3504 in 35 shown.
When an excess force or moment is applied in the state in which the from the coupler 3008 and the solenoids 3004 existing connection is fixed, the compulsion decreases. This makes it possible to prevent an instruction with excessive force from being input, thereby increasing the safety of the operation. The direct acting cylindrical electrostatic actuator 3010 will be explained later.
On this way, no command value is sent to the manipulation manipulator transferred from diseased tissue, if an abnormality occurs, whereby the safety of the work is increased.
It should be noted that the handle 3007 as in 38 shown can be trained in addition to the 37 shown handle of the pen type. 38a shows a grip of the tweezers type, 38b a ligate-type handle and 38c a grip of the clamping type. The surgeon can operate any of these handles while passing through the actuator 38001 to generate a grip force reflection produced power reflection.
In the 38b and 38c One side or both sides of the part of the grip for hooking the fingers rotate along a rotation axis 3802 , The surgeon can handle 3007 at this time by loosening the clamp and that by the solenoids 3004 exchange generated magnetic force. That is, the operator can attach or remove the handle according to the circumstances of use for entering action commands.
When a lead is caught in the joint, when the surgeon grasps the handle 3007 exchanges, an opening signal corresponding to the handle shape to the actuator 102 for manipulation of diseased tissue via the manipulation command generating means 103 and a manipulator carrying a corresponding tool is set as the effector. Accordingly, it is only necessary to provide various types of tools for this part, and it is not necessary to prepare the action command input device in advance with a number of tools.
hereby the surgeon is not confused as to which tool by using which input device can be used. Furthermore can be hand-available Space be wide open as an input device for a hand sufficient.
The calculation element 320 for the manipulative power of the other will be described below with reference to 39 explained. A case is assumed in which a plurality of action command input means 114 assigned to a "slave" manipulator. If this number is n, the weights W 1 to W n first in step 3901 set on each action command input device: W 1 + W 2 + ... W n = 1; W k > = 0 (k = 1 to n).
Assuming a case where the force feedback is implemented, the weight fraction with respect to the force reflection from a work object is also set. This is considered Wr 1 to Wr n and set similarly to the above as follows: Wr 1 + Wr 2 + ... Wr n = 1; Wr k > = (k = 1 to n).
The Size of wk indicates which action command input device prevails is.
Next will be in step 3902 the action command data input from each action command input means 113 converted into power. While it is not necessary to convert it when the mode is a force command mode at this time, a value obtained by inputting an error between a command and an answer to an adequate transfer function is used as the peak driving force command value when the command value Position, speed or the like. These values are vector quantities having an amount and a direction and are set as f1 to fn. At this time, in step 3909 within the force reflection given to the kth (k = 1 to n) action command input means, one calculated by the operation force of the others is calculated. If this is set as Fref k , Fref k can be described as follows: Fref k = W 1 f 1 + W 2 f 2 + ... + W k-1 f k-1 + W k + 1 f k + 1 + ... + W n f n
When force feedback is implemented, in step 3904 the force reflection Fext k = Wr k * Fext given to the kth (k = 1 to n) action command input device is calculated. Here Fext stands for the force sensor data 105 and is a vector size.
It is usually assumed that it is natural to set the above-described sequence W k = Wr k (k = 1 ... n). That is, it is a method of returning a high proportion of force reflection to more prevalent input devices. However, it is not always necessary that basically W k = Wr k and anisotropy may be given with respect to the direction. That is, W k and Wr k can be described by a matrix. In this case follows: W 1 + W 2 + ... W n = I; W k t W k > = 0, where the superscript letter t denotes a transformation of the matrix and I a unit matrix. The same applies to Wr k . Finally, Fref k and Fext k are added and in step 3905 output. The value comes with the virtual force reflection data 108 synthesized, ie combined.
Of the The algorithm described above allows the use of various input devices Action Commands giving surgeons perform work while they the force generated by others and the force reflection from the object feel.
On this way it is made possible to teach how, for example, a surgery is handled "physically", and above In addition, it is possible to surgical Transfer techniques by combining with surgical simulations.
The detection device 104 for work environment data is described below with reference to 4 described.
The detection device 104 for work environment data includes optical sensors 402 , an assembly unit 403 for optical sensors, a passive rotary joint 404 for connecting the two, a linear actuator 405 to adjust the parallax angle, a lighting 410 for illuminating the diseased area, a force sensor and a preprocessing circuit 412 for the force sensor signal, a proximity sensor, and a preprocessing circuit 413 for the proximity sensor attached to the tip section of the "slave" manipulator. It also includes an operation control device 401 for receiving a picture signal for the right eye 406 and an image signal for the left eye 407 for generating and outputting the data 107 of the optical sensor, for generating and outputting control signals 408 and 409 of the linear actuator 405 and the lighting 410 , for receiving a force sensor signal 414 and a proximity sensor signal 416 for controlling the sensors by a control signal 415 the preprocessing circuit of the force sensor signal and a control signal 417 the preprocessing circuit of the proximity sensor signal and for outputting the force sensor data 105 and the proximity sensor data 106 ,
The optical sensors 402 take an image of the diseased area and give the image signals for the right and the left eye 406 and 407 out. The optical sensors 402 are with the mounting unit 403 over the passive rotary joint 404 connected.
The picture signals 406 and 407 are digitized and in the operating control device 401 into the data 107 of the optical sensor. The force sensor signal 414 and the proximity sensor signal 416 are also converted to digital values to become the force sensor data 105 and the proximity sensor data 106 to change. At this time, the operation control device controls 401 each part through an algorithm as shown below.
If, as in 32 is shown, the work environment detection device 104 in step 3201 first a size of a movement of a coarse movement part 503 (please refer 5 which will be described later) of the actuator 102 detected for manipulation of diseased tissue in the depth direction of the diseased area, it sends the control signal 408 to the linear actuator 405 so that the parallax angle of the right and left optical sensors in step 3202 becomes equal to a value at the reference position. Then the parallax angle is kept constant while the linear actuator 405 is moved out or collapsed, and the right and the left optical sensor 402 rotate with centering on the passive rotation connection 404 fine and even.
The working environment detecting device 104 also controls the brightness of the lighting 410 by detecting a distance between the optical sensor and the diseased area in step 3301 in 33 to step in 3302 to investigate: a = (distance to the diseased area) / (standard distance).
Then she determines in step 3303 : b = (current magnification) / (default magnification).
Last, she puts in step 3304 the amount of light is proportional to a2 × b2. The in the 32 and 33 The above-described process makes it possible to adjust the parallax angle and the brightness of the diseased area adaptively according to the distance between the diseased area and the optical sensor and the magnification as they change.
The force sensor and its corresponding preprocessing circuit 412 and the proximity sensor and its corresponding preprocessing circuit 413 are in the area of the top 411 of the "slave" manipulator. Proximity sensors and the low force sensors and corresponding signal processing circuitry can be fabricated by micromachining technology. While the force sensor, the force sensor signal 414 and the proximity sensor receives the proximity sensor signal 416 output, sends the operation control device 401 the control signals 415 and 417 to each processing circuit corresponding to a signal level for changing the amplification factor at that time. A digital signal of more bits having a higher voltage compared to a noise level which is supposed to occur will be used for the control signal, and a control process as described in US Pat 31 will be shown.
An amplifier output within the above-mentioned processing circuit is sampled and in step 3101 held at 0th order and in step 3102 converted from analog to digital. Then in step 3103 c = (standard value of the sensor signal) / (value of the sensor signal) determined.
Next, the operation control device 401 the preprocessing circuit in step 3104 to multiply the magnification factor by c. The processing time between steps 3101 and 3103 is very short and it is assumed that the value of the signal does not change during that time.
The above-mentioned amplifier output becomes in step 3105 again sampled and held at 0th order and in step 3106 converted from analog to digital. Then the digitized value in step 3107 expressed by a real number and divided by c. That is, if the signal level is low, the gain of the preprocessing circuit is increased to prevent the signal from being masked by the noise that is mixed in until it enters the operational controller 401 is inputted, and when the signal level is large, the amplification factor for preventing the saturation of the signal is lowered. In this way, it becomes possible to reduce the influence of the noise from the environment and the actuator and to reduce the effect of quantization due to the digital sampling.
The actuating device 102 for the manipulation of diseased tissue will now be explained.
As in 5 As shown, it includes a manipulation command interpretation and control device 502 , the part 503 for rough movement, the part 504 for fine movement and the part 508 for very fine movement as a whole.
each Part becomes under exclusive Use of a material that compared to other often used building material no forces from the magnetic field receives such as polymeric materials, for example plastic.
This allows the diseased tissue to be manipulated without being affected by the magnetic field, even if MRI is one of the structural elements of the in vivo data measuring device described later 117 is used. Or, conversely, since the in vivo data generated by the MRI is generated during surgery, ie while the diseased tissue is being manipulated, it becomes possible to track any deformation in the tissue surrounding the diseased area in real time and to perform operations while measuring functions in addition to the shape, for example, especially in craniotomy.
The part 508 for the very fine movement has a plurality of manipulators and various treatment effectors can be attached to its tip.
It is in advance by the signal from the manipulation command generating means 103 determine which manipulator should be activated in certain circumstances. The manipulation command 111 includes a command whose degree of abstraction is high, such as "grabbing", a control mode, and a train of time series motion command data from the tip of a manipulator.
Upon receipt of the data train, the interpretation and control device interprets 502 for manipulation commands this and generates motion commands of each joint of the part 505 for fine movement and the manipulator (a plurality of manipulators, if necessary) of the part 508 for the Feinstbewegung and makes a Servolevel-Primitivsteuerung at the same time from the gripping command, the Control mode and the motion command value of a manipulator.
The control input 506 for each joint of the part 505 for the fine movement and the control input 509 for each joint of the part 508 for the microfinishing and the end effector, using the above-mentioned manipulation command, the displacement sensor data at each part 504 . 507 and 510 and the force sensor data 105 certainly.
34 Fig. 10 shows a flow of the manipulation instruction interpretation and control device 502 ,
First, it reads the control mode and the action mode from data obtained from the manipulation command generating means in the steps 3401 and 3402 in the in 28 sequence shown and interpreted in step 3403 the train of position and orientation data based on the control mode.
Of the Control mode specifies a control scheme, such as position control / speed control / impedance control, a commanded number of degrees of freedom and the existence of Anisotropy in terms of the degree of freedom.
Assuming that the above-mentioned commands and specifications are peak position and orientation command values of a single specific manipulator, it converts them into command values for each joint angle of the parts 505 and 508 around. In this case, a size of a displacement of a directly acting joint is also included as an angle.
Next, the controller performs 502 in step 3405 a joint servo control of the specific manipulator on the basis of the command value. The process branches here in step 3402 according to the action mode, and when the mode is MOVE, the process is finished.
If the action mode is grasp, in step 3407 a virtual power of attraction placed between the tips of the specific manipulator (the manipulator manipulated by the train of position and orientation data) and the other manipulator, and a balanced point of potential (a point where the attraction becomes zero) ) will be in step 3408 within a space created by the tip of each manipulator.
The other manipulator breaks the virtual force acting on the tip into a torque of each joint to control the torque in step 3409 ,
If the action mode is ENABLE, the action is almost the same as for GRIVE, except that the adjusted point of the potential in step 3410 is placed outside the above-mentioned room.
Although in the above statements only three action modes have been described as examples, it is in the reality necessary, some additional Types of basic To provide action modes. Then a mode in which a "master" manipulator and a "slave" manipulator himself in a 1: 1 ratio in terms of position / force control, which is conventionally practiced , and a mode in which a plurality of "master" manipulators correspond to a "slave" manipulator, provided and switched according to circumstances.
The part 503 to the coarse movement will now be with reference to the 6 explained. The part 503 for the coarse movement includes a pedestal 601 , a first connection 602 , a first joint 603 , a second connection 604 , a second joint 605 , a third connection 606 and a third joint 607 and a part 608 for fine movement is at the third joint 607 arranged.
The base 601 is with the first connection 602 coupled with a linear motion rail and a mechanism of the first connection, and accordingly can move horizontally along the transverse direction of the pedestal 601 to be moved. One degree of freedom of this part is set manually and a mechanical lock is provided so that it can be secured in any position.
By Construction of the mechanism of the first connection and, accordingly, that it can be moved manually makes it possible to quickly respond to an emergency to respond, such as power failure, so that safety is increased.
Although the shape of the first joint 602 is semicircular, it is not always necessary that it is semicircular, as long as it does not interfere with the mechanism of the second connection and subsequently.
The second connection 604 is with the first connection 602 over the first joint 603 coupled and rotates axially about the center line of the first joint 603 on both sides. The first joint is also constructed to be manually rotated and locked, for the same reason as in the case of the manual linear motion rail. The shape of the second joint 604 is semicircular.
The third joint 606 is with the second joint 604 over the second connection 605 coupled and rotates axially about the center line of the second joint 605 , The third joint 606 is also designed to be manually rotated and mechanically locked for added safety. The shape of the third joint 606 is also semicircular.
The part 608 for fine movement is with the third compound 606 over the third joint 607 coupled. The third joint 607 moves directly in the normal direction of the third joint 606 ,
The displacement sensor data 504 of each joint are sent to the manipulation command interpretation and control device 502 sent with reference to the 5 has been described.
By The above-described construction can make this coarse movement subsystem the mechanism and the structure of the degree of freedom of one degree of freedom the parallel motion and three degrees of freedom of the spherical Coordinate system that has the shape of a skull bone a lying patient is adjusted, and that a rough positioning the patient at the beginning of surgery and a quick and rapid removal, for example in case of an emergency, allows.
The construction of the part 505 will now be with reference to the 7 explained.
The part 505 for fine movement includes a base connection 704 , a first joint 705 , a second joint 706 , a second connection 707 , a third connection 708 , a third joint 709 and a base 712 of the part for fine movement.
The first to third joint are all hinges and are constructed in such a way that only one orientation of the whole Feinstbewegteilils is changed.
It was pointed out that in the case of fine work the change depending on the orientation from the scaling is. This means, as long as there is a small value in relation to the distance to the work object is the fine work in terms of the change of orientation the same as with normal work. Accordingly, the Degree of freedom of position and orientation are separated and the same drive method and mechanism as in the case of normal scaling can be used in terms of orientation become.
The change of the orientation of the part 508 for fine movement is with the optical sensor 402 connected. In this way, becomes a focal point of the optical sensor 402 always approximately near the working space of the manipulator of the part 508 positioned for fine movement. Although the part for fine movement 505 in 7 has a gimbal shape, a mechanism such as a Stewart platform can be used.
The part 508 for fine movement is described below with reference to the 8th described.
The part 508 for fine movement includes columns 801 , a ring band 802 , a first joint 803 , a first connection 804 , a second joint 805 , a second connection (the tip of a "slave" manipulator) 411 and a Peltier effect element 809 ,
The first joint 803 Moves in two degrees of freedom of linear motion in the direction of the center line of the ring rail 802 and by rotation around the midline. The second joint 805 is cylindrical and rotates about its midline.
The structure described above makes it possible to make the entire manipulator compact. While the part 508 For fine movement is constructed such that it has three degrees of freedom, it is possible to change the degree of freedom by exchanging the structures of the first compound 804 and the second connection 411 to increase in this way the same structure as with the first joint 803 train. In this embodiment, by adding the degrees of freedom of the part 508 realized for Feinstbewegung to the three degrees of freedom of rotation of the part for fine movement more than six degrees of freedom.
The Peltier effect element 809 attached to the tip of the manipulator is an element for realizing thermoelectric cooling by the Peltier effect, which can be realized by micro-machining technology.
If the manipulator itself provided with a mechanical force is organic tissues such as brain tissue, Ner ven and destroy blood vessels, it gets very dangerous when she causes an accident, such as an outlier. Then it is necessary to cut and remove diseased tissue, that was done in the past with mechanical force, through the manipulation, which causes tissue degeneration Control of an energy flow generated in the fine surgery, too replace.
In addition, the reduction in required mechanical forces provides advantages in terms of the design of the manipulator, and the manipulator driving the actuator can be further miniaturized or required specifications of the manipulator and the actuator can be simplified, and this is very advantageous.
So far became a maximum temperature method (by devices such as a laser cutter or other) as Tissue regeneration procedure using energy control The surrounding areas are used considerably by its radiation which leads to some concerns about the application in the performed fine-surgical operation. In contrast, it causes degeneration and destruction of Tissues by freezing only that the manipulated area reliable degenerate, because almost no heat is transmitted, as long as no Contact exists.
Besides that is it is not necessary over to think about the problem of radiation (although the temperature difference in this case is opposite), since this procedure - compared with the laser - no size Temperature difference compared to the environment has.
The Part of microfinance that is less invasive and less influential caused in the environment can, as described above, realized become.
It It is noted that, although it is in relation to many points advantageous is a degeneration and spallation of the tissue through to cause the peltier effect element it is not realistic perform all surgical treatments by cryo-spallation. Besides that is the treatment effect not limited to the Peltier effect element, besides, you can Tools, as they are known, with this in combination be used. This means, The manipulator can with a laser knife, an ultrasonic knife or an electric knife or a small one Clamp, a knife or a clip can be attached to it.
Besides that is a plurality of manipulators exist so that it is possible to equip them with different treatment effectors.
9 shows that off 8th taken first joint 803 , The first joint 803 includes an inner stator 901 ie a small part of the ring rail 802 , a multi-degree-of-freedom movement device 902 , an outer stator 903 to which the first joint 804 is coupled, a drive circuit 904 for controlling an electrode voltage of the outer stator 903 , Drive circuits 905 and 907 for controlling the electrode voltages of the outer surface and the inner surface of the multi-degree-of-freedom movement means 902 , a drive circuit 906 for controlling an electrode voltage of the inner stator 901 and a main controller 908 , The materials of the moving means and the stator are polyimide and adhesive. A conductive polymer composition consisting mainly of carbon molecules is used as an electrode.
The annular electrodes are on the outer surface of the inner stator 901 arranged vertically to the axis of the cylinder. The electrodes are also on the inner surface of the multi-degree-of-freedom movement device 902 parallel to the inner stator 901 arranged and a large number of linear electrodes is vertical to the inner stator 901 arranged on the outer surface. Although not shown, flanges are on both sides of the multi-degree-of-freedom movement device 902 for limiting the degree of freedom of the outer stator 903 attached only to a rotation about the centerline of the cylinder. A large number of linear electrodes are on the inner surface of the outer stator 903 parallel with the electrodes on the outer surface of the multi-degree-of-freedom movement device 902 arranged.
10 is a sectional view of an in 9 shown first joint, cut along a plane that includes the central axis and an orthogonal to this surface. 11 is an enlarged view of the section in area A of FIG 10 and 12 is an enlarged view of the section in area B.
In 11 is the outer stator 903 with an inner cover 1110 and an outer cover layer 1113 covered on both sides and an adhesive or adhesive 1112 is inserted between the two layers. electrodes 1111 of a carbonaceous polymer composition are arranged equidistantly to provide a line of intersection of the cylinder and the plane including the center axis of the cylinder. This section corresponds to a dot-dash line, as in 12 will be shown.
The structure of the outer portion of the multi-degree-of-freedom movement device 902 is the same as that of the outer stator 903 and the structure of the inner portion of the stator 903 is the same as that of the inner stator 901 , Insulating fluids 1108 and 1109 are between the inner stator 901 and the multi-degree-of-freedom movement device 902 filled as well as between the multi-degree-of-freedom movement device 902 and the outer stator 903 ,
12 is an enlarged sectional view of area B in 10 and the same parts are designated by the same reference numerals. In this sectional view, the direction of the electrodes is opposite to that in FIG 11 because the direction of the section is orthogonal to that in 11 is.
Next is the basic working principle of the first joint 803 with reference to the 13 explained. The 13 is a sectional view showing a combination of the outer stator 903 and the peripheral portion of the multi-degree-of-freedom movement device 902 shows. Three-phase AC voltages become the electrode 1111 of the outer stator 903 and a peripheral electrode 1303 the multi-degree-of-freedom movement device 902 in a set of three electrodes, over the wires 1301 and 1304 fed. When the electrodes 1111 and 1303 supplied voltages are out of phase or if their frequency is different, a driving force between the moving means and the stator is generated, which is the moving means 902 moves in the direction of the axis of the cylinder. The same applies to the inner area of the multi-degree-of-freedom movement device 902 and the inner stator 901 to.
In addition, since the set of electrodes on the inner region is orthogonal to the set of the electrodes of the outer region, the set of inner electrodes microscopically generates a driving force in the tangential direction of a circular section perpendicular to the axis of the cylinder. The integration of this driving force in the circumferential direction transforms it into a rotational force about the axis, and the multi-degree-of-freedom movement means 902 rotates.
In addition, the above-described movements in the two directions are orthogonal to each other and a movement generated by a combination will not change the positional relationship between the electrodes of another combination. Accordingly, the first joint 803 in the axial direction of the cylinder and at the same time rotate about the axis.
On an explanation of the second joint 805 is omitted here, since its construction is the same in combination with the multi-degree-of-freedom movement device 902 and the outer stator 903 of the 9 ,
The measuring device 117 for in vivo data is described below with reference to 40 explained. The in vivo data measuring device 117 includes a measurement signal transmission unit 4001 and a measurement signal receiving unit 4002 ,
The measuring signal receiving unit 4001 is equipped with an X-ray tube, a superconducting magnet and magnetic coils, an ultrasonic transducer and the like to be able to transmit measurement signals in various modalities such as X-ray, magnetic field and ultrasound.
The measuring signal receiving unit 4002 is equipped with receiving accessories that correspond to each modality, for receiving an input signal 119 which passes through or is reflected by the body of the patient and as an output signal 120 exit.
From the X-ray tube emitted X-rays, through the body is through X-ray sensors with two-dimensional Array arrangement received. Signals representing a three-dimensional CT image can generate in a very short time by rotation of these transmitting and receiving units to be received by the patient. This system is called a cone beam CT designated.
One is generated by the magnetic coil alternating magnetic field through a high-frequency sensor receive. He uses the same principle as one called MRI Measuring instrument. A transmitting unit and a receiving unit of a Ultrasonic transducer is arranged in the form of a two-dimensional array.
hereby be by X-ray CT Signals are generated that reflect the data of the patient's environment including the bones generate, and the MRI generates signals that data the Form, visualized data of the difference between tissues that for the nakedness Eye are indistinguishable, and visualized data of brain function produce. The ultrasound generates signals, the data of the form and Generate data indicating local abnormal blood flows and dyskinesia.
The measurement by means of the above-described multiplexing modalities is performed in a very short period of time, irrespective of whether before or during the operation, and the obtained signal data is digitized and received in the signal receiving unit 4002 preprocessed. The digitized measurement data 121 are sent to the measurement data processing device 118 Posted.
Next, the measurement data processing device 118 with reference to the 41 described. The measurement data processing device 118 includes an image processing work unit 4101 , a main memory 4102 , a secondary data store 4103 and an address and a data bus 1104 ,
Preferably, the image processing and operating unit 4101 built like a massive parallel computer. The main memory 4102 is a normal store and the secondary store 4103 is a mass storage, such as an optical, magnetic, disc-shaped storage medium. These devices 4101 to 4103 are through the address and data bus 4101 closely linked to enable super high speed image processing and operation.
The measured data 121 are in main memory 4102 stored and run through the image processing and operating unit 4101 , They are processed and as three-dimensional image data by the image processing and operating unit 4101 reconstructed. The reconstructed data 122 are sent to the reality control data generator 101 sent to be synthesized with the other images, ie combined.
The above-described construction reconstructs and visualizes the in-vivo data in three-dimensional form, whether before or during the operation, and presents it to the surgeon and updates at short intervals. The reconstructed three-dimensional image data at that time becomes the secondary data store 4103 saved. These data are used in addition to use during the surgical operation to perform simulations and training.
The above-described explanation is not limited to the explanation of the detailed construction of the embodiment of FIG 1 and apply to the support of the surgical operation, but also to the training by simulating surgery and explaining the surgical operation to the patient (informed consent).
The Training function by simulation is explained below.
First, a virtual diseased area is acquired by the reality control data generation device 101 generated. Image data of actual diseased areas generated by reconstruction in 3D from the measured data and in the secondary data store 214 of the 2 or the second memory 4103 of the 41 are used, or virtual image data of a diseased area are generated by a model. These are then displayed and virtual force reflection data 108 are generated and sent to the manipulation command generating means 103 Posted. Since there is no force reflection from the actual world at this time, the virtual force reflection is equal to the synthesized force reflection.
The virtual force reflection is based on one in the secondary data store 214 in the working environment data processor 201 stored dynamic model calculated. The result of the calculation is sent to the action command input device of each surgical operator for transmitting the virtual force reflection.
The surgical operators operate the "slave" manipulators in the real world within the virtual environment. The manipulator moves following the intension of the surgical surgeon and captures all sensor data. Out of the group of these, only the optical sensor is sent to the reality control data generating means 101 sent for combination with the image data of the virtual diseased area.
At this time, the simulation may be performed by one person, or the "physical" training may be performed using the above-described "multi-to-one master / slave" function. The secondary data store 214 within the working environment data processor 201 Also includes all time data series of this simulation of surgery. Accordingly, the recorded result can be reflected on another occasion to evaluate actions and decisions.
When next the case is described in which the surgical operation the Patients explained becomes. Because the simulation results described above are included were, can the corresponding contents of the operation by reproducing the results explained become. In this way the patient can be understanding of the Deepen operation and put his trust in the surgical surgeon and increase the surgery itself.
The case where the explanation is made using actual non-virtual shooting data of the operation is made as follows. The data in the secondary data store within the work environment data processor and the data in the secondary data store 4103 the measured data processing device 118 contain data for synchronization. These data are combined and on display 203 the reality control data generator 101 reproduced from these data. It is possible to reproduce only the actual image or merely to reproduce the measurement data reconstructed as a 3D image. The surgical surgeon can explain the operation to patients having similar symptoms by displaying the image.
On In this way, the patient can deepen his understanding of the operation and his confidence in the surgical surgeon and the surgical operation itself similar increase as in the case described above.
Device for surgical operation, comprising: - a device for manipulating diseased tissue ( 102 ) designed to be remotely controllable by a variety of operators and having a surgical instrument or instrument for treatment; A detection device for measuring and detecting the diseased tissue ( 125 ) and the institution ( 102 ) for manipulation during an operation for obtaining data ( 105 . 106 . 107 . 120 ) concerning the diseased tissue ( 125 ) and the manipulation device ( 102 ), wherein the detection device is an in-vivo data measuring device ( 117 ) for measuring in vivo data by periodically applying a fluctuating magnetic field or a plurality of fluctuating magnetic fields and / or electromagnetic waves and / or ultrasonic waves to a diseased area and surrounding body area before and during the operation and by measuring the intruded signal or the resonance signal, as well as a device ( 104 ) for acquiring data in the work environment that captures imaging data of the diseased body part and an approach state and a touch force of the device ( 102 ) for the manipulation of diseased tissue against the diseased part of the body ( 125 ), - a data feed device ( 118 . 101 ), which conveys the data acquired by the detection device to the surgeons, wherein the data supply device has a measurement data processing device ( 118 ) for generating a measured data image in 3D representation from the by means of the in vivo data measuring device ( 117 ) measured in vivo data ( 121 ) and a facility for the generation of real estate tax data ( 101 ) for linking and / or processing the output signals of the device ( 104 ) for acquiring data in the work environment and the output signals from the measurement data processing device ( 118 ) to present them to each operating surgeon as reality control data, - a plurality of input devices for entering action commands ( 114 ) for inputting actions performed by each operating surgeon on the basis of that provided by the device for generating reality control data ( 101 ) reality control data presented to each operating surgeon, and - means for generating manipulation commands ( 103 ) for converting from each of the input devices to input action commands ( 114 ) issued action command ( 113 ) in manipulation command data ( 111 ) for the transmission of this data to the device for the manipulation of diseased tissue ( 102 ) and for the transmission of one from the work environment data collection facility ( 104 ) touched force to the device for the manipulation of diseased tissue ( 102 ).
Surgical surgery device according to claim 1, characterized in that each of the input devices is capable of entering action commands ( 114 ) Signals ( 113 ) generated with a plurality of command inputs from the plurality of operators.
Device for surgical operations according to one of claims 1 or 2, characterized in that each of the input devices for inputting action commands ( 114 ) the signals ( 113 ) generated with the plurality of command inputs under weighting and addition in a predetermined manner.
DE69636176T 1995-02-16 1996-02-16 System for supporting remote controlled surgery Expired - Lifetime DE69636176T2 (en)
JP02839195A JP3539645B2 (en) 1995-02-16 1995-02-16 Remote surgery support device
JP2839195 1995-02-16
DE69636176T2 true DE69636176T2 (en) 2007-07-12
DE69623674A Expired - Fee Related DE69623674D1 (en) 1995-02-16 1996-02-16 Remote surgery support system
DE69636176A Expired - Fee Related DE69636176D1 (en) 1995-02-16 1996-02-16 System for supporting remote controlled surgery
DE69636176T Expired - Lifetime DE69636176T2 (en) 1995-02-16 1996-02-16 System for supporting remote controlled surgery
DE69623674T Expired - Lifetime DE69623674T2 (en) 1995-02-16 1996-02-16 Remote surgery support system
EP (2) EP0732082B1 (en)
JP4655175B2 (en) * 2000-12-19 2011-03-23 ソニー株式会社 Manipulator system, master manipulator, slave manipulator, control method thereof, and recording medium
CA2443996A1 (en) * 2001-04-13 2002-10-24 First Genetic Trust Methods and systems for managing informed consent processes
CA2477758A1 (en) 2002-03-28 2003-10-09 Scimed Life Systems, Inc. Polymer welding using ferromagnetic particles
EP1631191A4 (en) * 2002-08-06 2009-04-22 Stereotaxis Inc Virtual device interface control for medical devices
JP4696307B2 (en) 2003-07-24 2011-06-08 学校法人慶應義塾 Position / force control device
US10555775B2 (en) 2005-05-16 2020-02-11 Intuitive Surgical Operations, Inc. Methods and system for performing 3-D tool tracking by fusion of sensor and/or camera derived data during minimally invasive robotic surgery
JP4876246B2 (en) * 2006-03-03 2012-02-15 国立大学法人長岡技術科学大学 Haptic control method and tactile control device
AT491405T (en) * 2006-07-25 2011-01-15 Alcon Inc Surgical console for reproducing multimedia content
WO2010048160A2 (en) * 2008-10-20 2010-04-29 The Johns Hopkins University Environment property estimation and graphical display
EP2854686A4 (en) 2012-06-01 2016-01-27 Intuitive Surgical Operations Surgical instrument manipulator aspects
EP2983069B1 (en) * 2013-03-27 2018-12-19 Olympus Corporation Operation input device and master-slave system
WO2015143067A1 (en) * 2014-03-19 2015-09-24 Intuitive Surgical Operations, Inc. Medical devices, systems, and methods using eye gaze tracking
JP6576042B2 (en) * 2015-01-14 2019-09-18 キヤノン株式会社 Display control apparatus and method, and program
EP3484401A1 (en) * 2016-07-14 2019-05-22 Intuitive Surgical Operations Inc. Systems and methods for controlling a surgical instrument
CN109195544A (en) * 2016-07-14 2019-01-11 直观外科手术操作公司 Secondary instrument control in computer-assisted remote operating system
US20190209199A1 (en) * 2018-01-11 2019-07-11 Misonix, Incorporated Ultrasonic surgical system for osseous transection
WO2019188022A1 (en) * 2018-03-28 2019-10-03 ソニー株式会社 Control device, control method, and master/slave system
WO2020003728A1 (en) * 2018-06-28 2020-01-02 ソニー株式会社 Encoding device, encoding method, decoding device, decoding method, and program
WO2020021870A1 (en) * 2018-07-26 2020-01-30 ソニー株式会社 Information processing device, information processing method, and program
1995-02-16 JP JP02839195A patent/JP3539645B2/en not_active Expired - Fee Related
1996-02-16 US US08/601,356 patent/US5855553A/en not_active Expired - Fee Related
1996-02-16 DE DE69623674A patent/DE69623674D1/en not_active Expired - Fee Related
1996-02-16 DE DE69636176A patent/DE69636176D1/en not_active Expired - Fee Related
1996-02-16 EP EP96102360A patent/EP0732082B1/en not_active Expired - Lifetime
1996-02-16 DE DE69636176T patent/DE69636176T2/en not_active Expired - Lifetime
1996-02-16 DE DE69623674T patent/DE69623674T2/en not_active Expired - Lifetime
1996-02-16 EP EP01111968A patent/EP1125557B1/en not_active Expired - Lifetime
EP1125557A3 (en) 2004-01-02
JPH08215211A (en) 1996-08-27
EP1125557B1 (en) 2006-05-24
JP3539645B2 (en) 2004-07-07
US5855553A (en) 1999-01-05
EP0732082B1 (en) 2002-09-18
DE69623674D1 (en) 2002-10-24
EP1125557A2 (en) 2001-08-22
EP0732082A3 (en) 1996-11-20
DE69636176D1 (en) 2006-06-29
DE69623674T2 (en) 2003-05-22
EP0732082A2 (en) 1996-09-18
JP2017512554A (en) 2017-05-25 Medical device, system, and method using eye tracking
CN104840253B (en) 2019-07-12 Master control set and its control method for operating robot