Abstract:
A teleoperator system with telepresence is shown which includes right and left hand controllers ( 72 R and  72 L) for control of right and left manipulators ( 24 R and  24 L) through use of a servomechanism that includes computer. The teleoperator system comprises a surgical system suited for endoscopic surgery. The surgical system includes a surgical instrument, a servomechanism and a controller. The surgical includes an insertion section and a control section. The insertion section comprises a forearm, a wrist and an end effector in the form of a surgical instrument head selected from the group consisting of retractors, electrosurgical cutters, electrosurgical coagulators, forceps, needle holders, scissors, blades and irrigators. The control section comprises a plurality of motors and linkages which operate to insert and retract the forearm, rotate the forearm, pivot the forearm, and pivot the wrist link about the wrist joint.

Description:
This is a Continuation of application Ser. No. 07/823,932 filed Jan. 21, 1992, now abandoned. 
    
    
     FIELD OF THE INVENTION 
     This invention relates generally to teleoperator method and apparatus, and particularly to those which include means for providing the operator of remote apparatus with the same sense as working directly with his hands at the worksite. 
     BACKGROUND OF THE INVENTION 
     Teleoperating, which is well known, includes the human performance of tasks at a remote location using manipulators. Telepresense includes providing the teleoperator with the same feedback and control that he would have were he actually at the worksite carrying out the operation with his own hands. Telepresence operation generally includes use of a stationary visual display, particularly a stereographic visual display of the remote workspace. Stereoscopic television systems are well known as shown, for example, in U.S. Pat. Nos. 4,562,463 and 4,583,117 and in U.K. Patent Application GB 2,040,134. 
     Remote manipulators employing stereoscopic TV viewing together with force feedback also are well known as shown, for example, in an article entitled, “Controlling Remote Manipulators Through Kinesthetic Coupling,” Bejczy et al, Computers in Mechanical Engineering, July 1983, pps. 48–60, and in an article entitled, “Stereo Advantage for a Peg-In-Hole Task Using a Force-Feedback Manipulator” by E. H. Spain, SPIE Vol. 1256 Stereoscopic Displays and Applications, 1990, pps. 244–254. In the Bejczy et al article, force-torque feedback is disclosed. Also, in U.S. Pat. No. 3,921,445, a manipulator which includes force, torque and slip sensors of a type which may be employed with the present invention is shown. 
     Even though the operator of prior art manipulators is provided with a stationary three-dimensional image of the workspace, and manual controllers for control of the manipulators are provided with feedback, the operator is not provided with a sense of actually being present at the worksite. The present invention is directed to a viewing arrangement for use in a remote manipulation system which substantially adds to the operator&#39;s sense of presence at the remote manipulator site. 
     SUMMARY AND OBJECT OF THE INVENTION 
     An object of this invention is the provision of an improved teleoperator system and method which include an improved viewing system to enhance the operator&#39;s sense of presence at remote manipulators controlled by the operator from a remote location. 
     An object of this invention is the provision of an improved teleoperator system and method of the above-mentioned type wherein an image of manipulator end effectors for viewing by the operator are sensed by the operator as comprising an integral part of hand-controllers used by the operator to control the end effectors, thereby giving the operator a strong sense of presence at the worksite. 
     An object of this invention is the provision of an improved teleoperator system and method of the above-mentioned type which is well adapted for use in a wide variety of applications including military, industrial, biomedical, and the like. 
     The present invention includes manipulators located at a worksite and which are controlled by hand-operated means at a remote operator control station. End effectors at the manipulators are used for manipulating objects located in a workspace at the worksite, and force-torque feedback is employed for transmitting back to the operator mechanical resistance encountered by the end effectors. Stereographic visual display means provide the operator with an image of the workspace. In accordance with the present invention, the image is located adjacent the hand-operated means so that the operator looks in the direction of the hand-operated means for viewing the image adjacent the hand-operated means. Either a real or virtual image of the workspace may be provided adjacent the hand-operated means. Display means for display of a real image may be located adjacent the hand-operated means for direct viewing of the real image by the operator. For display of a virtual image of the workspace, a mirror is located between the operator&#39;s eyes and the hand-operated means. In this case, display means provide a real image which is inverted from top to bottom, which inverted image is viewed via the mirror, which mirror inverts the image and provides the operator with a virtual image of the workspace, which appears to be located adjacent the hand-operated means. By locating the image of the workspace adjacent the hand-operated means the operator is provided with a sense that the end effectors and hand-operated means are substantially integral despite the fact the end effectors are located at the worksite and the hand-operated means are located at the remote operator&#39;s station. A stereophonic sound system may be included to provide the operator with stereophonic sound from the worksite. Video camera means are provided for viewing the workspace from which an image of the workspace is obtained. Various other sensors and associated responders may be located at the worksite and operator&#39;s station, respectively, for transmission of pressure, tactile, heat, vibration and similar information for enhanced telepresence operation. 
     Depending upon the application, different scaling may be provided in the transmission of information between the operator&#39;s station and worksite. For example, for microassembly, microsurgery and like operations involving small part manipulation, optical and/or video magnification may be employed to provide an enlarged 3-dimensional image for viewing by the operator. With similar scaling between the hand operated means and manipulators, the perception of the operator is substantially that which a miniature operator would have were he at the worksite. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The invention, together with other objects and advantages thereof, will be better understood from the following description considered with the accompanying drawings. It will be understood that the drawings are for purposes of illustration and example only, and that the invention is not limited thereto. In the drawings, wherein like reference characters refer to the same parts in the several views, 
         FIG. 1  is a diagrammatic showing of a teleoperator system embodying the present invention including side elevational views of a worksite and remote control operator&#39;s station; 
         FIG. 2  is an enlarged rear elevational view of the operator&#39;s station taken substantially along line  2 — 2  of  FIG. 1 ; 
         FIG. 3  is an enlarged rear elevational view of the worksite taken substantially along line  3 — 3  of  FIG. 1 ; 
         FIG. 4  is a simplified side elevational view which is similar to  FIG. 1  and showing dimensional relationships between elements at the worksite and elements at the operator&#39;s station; 
         FIG. 5  is a diagrammatic view to illustrate visual perception by a miniature virtual eye, and 
         FIG. 6  is a diagrammatic view to illustrate visual perception by the operator when image magnification is employed; 
         FIG. 7  is a diagrammatic view which is similar to that of  FIG. 1  but showing the teleoperator system used for telepresence surgery; 
         FIG. 8  is a rear elevational view of the operator&#39;s station shown in  FIG. 7 ; 
         FIG. 9  is a rear elevational view of the worksite shown in  FIG. 7 ; 
         FIGS. 10 and 11  are fragmentary side elevational views of modified forms of operator&#39;s station and manipulator, respectively, having increased degrees of freedom; 
         FIG. 12  is a side elevational view of a modified form of operator&#39;s station wherein display means are positioned for direct viewing by the operator; 
         FIG. 13  is a rear elevational view of the modified form of operator&#39;s station shown in  FIG. 12 ; and 
         FIG. 14  shows a fragmentary portion of the insertion portion of an endoscope for use with the present invention. 
     
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     Reference now is made to  FIGS. 1–3  wherein the teleoperator system is shown to include an operator&#39;s station  20  ( FIGS. 1 and 2 ) and worksite  22  ( FIGS. 1 and 3 ). An operator  18  at the operator&#39;s station controls manipulator means  24  at the remote worksite. Manipulator means  24 , comprising right and left manipulators  24 R and  24 L, respectively, are used for manipulating objects, such as object  26  which is shown located on a platform, or base,  28  within a workspace  30  shown in broken lines. For purposes of illustration only, and not by way of limitation, the right manipulator  24 R is shown to comprise a housing  32 R affixed to base  28  and from which housing a telescopic arm  34 R extends. The inner end  34 R 1  of arm  34 R is mounted for pivotal movement in any pivotal direction using conventional mounting means. For example, the inner end of arm  34 R may be mounted for pivotal movement about a horizontal pivot axis  36  which pivot axis, in turn, is adapted for pivotal movement about vertical axis  38 . 
     Arm  34 R includes telescopic inner section  34 R 1  and outer section  34 R 2 , which outer section is adapted both for axial movement into and out of inner section  34 R 1  and for rotation about its longitudinal axis. An end effector  40 R is carried at the outer end of the arm which, for purposes of illustration, is shown to comprise a gripper. Motor means, not shown, control pivotal movement of arm  34 R about pivot axes  36  and  38 , axial and rotary movement of outer arm section  34 R 2  along and about the longitudinal axis of the arm, and opening and closing of gripper  40 R. The motor means, together with motor control circuits for control of the motors, may be included in housing  32 R. The motors are under control of a computer  42  connected thereto through right manipulator interface  44 R and the above-mentioned motor control circuits. 
     The left manipulator  24 L is of substantially the same design as the right manipulator  24 R and the same reference numerals, but with the suffix L instead of R, are used to identify similar parts. For purposes of illustration, the left end effector  40 L, shown in  FIG. 3 , is seen to comprise cutting blades which operate to cut in the manner of a pair of scissor blades. 
     The worksite is provided with a pair of video cameras  46 R and  46 L for viewing workspace  30  from different angles from production of stereoscopic signal outputs therefrom at lines  48 R and  48 L. The angle γ between the optical axes of the cameras shown in  FIG. 3  is substantially equal to the operator&#39;s interocular viewing angle γ of an image of the workspace as shown in  FIG. 2 . 
     The video camera outputs at lines  48 R and  48 L are supplied to an image memory  50  for momentary storage of video fields of right and left images from the cameras. Fields of right and left images from image memory  50  are alternately supplied through left/right switch means  52  to visual display means  54 , such as a television monitor, for alternate display of the two images at the face  54 A of the monitor. Timing and control means  56  provide timing and control signals to various elements of the system, including elements included in the stereographic display system, for signal timing and control of the system. If digital storage means  50  are employed, then conversion of the camera signal outputs to digital signal form by analog to digital converter means prior to storage, and conversion of the digital signal output from left/right switch means to analog signal form in preparation for display at monitor  54  may be employed. 
     An electrooptical device  58  at the face of the display means  54  controls polarization of light received from display means  54  under control of a left/right synchronizing signal from timing and control unit  56 . The left and right image fields are viewed by operator  18  wearing a pair of passive polarized glasses  60  having right and left polarizing elements  62  and  64  polarized in orthogonal directions. The polarization of light from display  54  through electrooptical device  58  is synchronized field by field such that the right field is occluded from the left eye and the left field is occluded from the right eye for stereographic viewing by the operator. Other means for stereographic viewing of left and right image fields are well known, including, for example, those using active stereographic glasses, which may be used in the practice of this invention to provide the operator with a stereoscopic view of the remote workspace. 
     The vertical deflection coil connections for monitor  54  are reversed, causing the monitor to scan from bottom to top thereby creating a top-to-bottom inverted image  30 I of workspace  30 . Letters a, b, c and d are used to identify corresponding corners of the workspace  30  and inverted workspace image  30 I. The inverted workspace image  30 I is viewed by the operator via a mirror  66  at the top of a table  68 , which mirror inverts image  30 I to return the image as viewed by the operator to an upright position. Looking downwardly in the direction of the mirror, the operator views a virtual image  30 V of workspace  30 . In accordance with one aspect of the present invention, the image viewed by the operator, which in the  FIG. 1-3  embodiment comprises a virtual image, is located adjacent controller means  70  used by the operator for control of manipulator means  24  at the worksite. 
     Controller means  70  are shown located beneath the table top  68  and include right and left controllers  72 R and  72 L for control of the respective right and left manipulators  24 R and  24 L. The right and left controllers are of substantially the same design so that a description of one applies to both. As with the manipulators, the suffixes R and L are used to distinguish elements of the right controller from those of the left controller. For purposes of illustration, and not by way of limitation, the right controller  72 R is shown to comprise a housing  74 R affixed to the bottom of table top  68  and from which hand-operated means  76 R in the form of a telescopic control arm, or stick, extends. 
     The right and left control arms  76 R and  76 L are provided with the same degrees of freedom as the associated manipulator arms  34 R and  34 L, respectively. For example, the inner end of control arm  76 R is mounted for pivotal movement about a horizontal pivot axis, corresponding to manipulator pivot axis  36 , which axis, in turn, is adapted for pivotal movement about an intersecting vertical axis, corresponding to manipulator axis  38 . Control arm  76 R also includes inner section  76 R 1  and outer section  76 R 2 , which outer section is adapted both for axial movement into and out of inner section  76 R 1  and for rotation about its longitudinal axis. It will be apparent that the control arm  76 R is provided with the same four degrees of freedom as the associated manipulator arm  34 R. Additionally, sensor means  78 R are located adjacent the outer end of outer arm section  76 R 2  for use in controlling gripping action of gripper  40 R. Similar sensor means  78 L adjacent the outer end of control arm  76 L are adapted for use in controlling operation of scissor blades  40 L. 
     Right and left controllers  72 R and  72 l are included in a servomechanism system wherein mechanical motion of control arms  76 R and  76 L controls the position of manipulator arms  34 R and  34 L, and pressure on sensor means  78 R and  78 L controls opening and closing of end effectors  40 R and  40 L, respectively. In  FIG. 1 , right and left hand controller interfaces  80 R and  80 L, respectively, are shown for connection of the controllers to computer  42 . Servomechanisms for control of mechanical motion at a remote location are well known, including those which provide force and torque feedback from the manipulator to the hand-operated controller means. Any suitable prior art servomechanism may be used in the practice of the present invention, with those incorporating force and torque feedback being particularly preferred for telepresence operation of the system. In the illustrated system, right and left microphones are included at the worksite, outputs from which microphones are amplified by right and left amplifiers and supplied to right and left speakers at the operators&#39; station for providing a stereophonic sound output to provide the operator with an audio perspective present at the workspace. In  FIG. 1 , only the right channel of the stereophonic system is shown including right microphone  82 R, right amplifier  86 R and right speaker  88 R. The left microphone and speaker are located directly behind the respective right microphone and speaker at the worksite and operator&#39;s control station as viewed in  FIG. 1 . Obviously, earphones may be provided for use by the operator in place of the speakers which would help to block out external noises at the operator&#39;s control station. Also, in  FIG. 1  a light shield  54 B at the monitor is shown for blocking direct viewing of the monitor face by the operator. 
     Reference now is made to  FIG. 4  wherein a simplified diagrammatic view of the system illustrated in  FIGS. 1–3  is shown and wherein various lengths and angular positions are identified by reference characters. In  FIG. 4 , the optical path length between the cameras and a point F at the workspace is identified by reference character L. A corresponding path length between the operator&#39;s eyes and point F at the virtual image of the workspace is identified by the distance a+b, where a is the distance from the eyes of the operator to mirror  66 , and b is the distance from the mirror to point F at the virtual image. Other dimensions shown include the height G of the cameras above the pivot point of manipulator arm  34 R and corresponding height g of the operator&#39;s eyes above the pivot point of control arm  76 R. With the control arm  76 R at length d, the manipulator arm  34 R adjusts to length D. Similarly, with the control arm  76 R at an angle β A  with the vertical, the manipulator arm  34 R is positioned at the same angle from vertical. The angle from vertical at which the cameras view the workspace and the eyes view the virtual image of the workspace is identified by α. 
     Between elements of the worksite and operator station, the following relationships pertain:
 
 a+b−kL,    (1)
 
d=kD, and   (2)
 
g=kG   (3)
         where k is a scale factor constant.
 
When k equals 1 such that a+b=L, d=D and g=G, no scaling of worksite dimensions is required.
       

     Any scale factor may be employed, the invention not being limited to full-scale manipulation. For example, the worksite can be small, including microscopic in size, in which case the optical parameters, including distance to object, interocular distance and focal length, and mechanical and dimensional paramaters are appropriately scaled. 
     By using appropriate scaling and image magnification and force and torque feedback, and by locating the image  30 V of the workspace  30  adjacent hand-operated control means  76 R and  76 L, the operator is provided with a strong sense of directly controlling the end effectors  40 R and  40 L. The operator is provided with a sense that the end effectors  40 R and  40 L and respective control arms  76 R and  76 L are substantially integral. This same sense of togetherness of the hand-operated control means and end effectors is not provided in prior art arrangements wherein the image viewed by the operator is not located adjacent the hand-operated control means. Even where the prior art includes stereoscopic viewing and force and torque feedback, there is a feeling of disconnectedness of the hand motions from the visual image object being worked upon. The present invention overcomes this sense of disconnectedness by locating the workspace image where the operator&#39;s hands appear to exercise direct control over the end effectors. 
     For small-scale manipulation, such as required for surgical applications, it is desired to replicate the visual experience that a miniature observer would have were he closely adjacent the actual worksite. In  FIG. 5 , the virtual eye  90  of a hypothetical miniature observer is shown viewing an actual workspace. Light from a source at a point X, Y, Z in the actual workspace produces a stimulus on the miniature observer&#39;s eye  90  at a point identified as X′/M. In  FIG. 6 , an eye  92  of an actual operator is shown viewing an enlarged image of the virtual workspace produced by means of a video camera  94  used to view the actual workspace. The illustrated camera includes a light-receiving lens  96  and solid state imaging device such as a charge-coupled-device (CCD) array  98  where the point light source at X, Y, Z is shown imaged at point X i , Y i , Z i . With correct scaling, a corresponding light source is produced at point MX i , MY i , MZ i  at either the real or apparent position of the face of the visual display which, due to stereoscopic operation of the system appears to the operator to originate from point MX, MY, MZ corresponding to point X,Y,Z at the actual workspace. At the retina of the actual eye  92 , a stimulus is produced at point X′ at proportionately the same position as point X′/M at eye  90  of the hypothetical observer. This relationship is ensured by selecting a correctly scaled camera distance and lens focal length such that the optical magnification M o =M/M v  where M is the desired overall magnification and M v  is the video magnification. A typical video magnification, M v , which equal the ratio of the CCD-array  98  width to the display width, is about 40. 
     Reference now is made to  FIGS. 7 through 9  wherein a modified form of this invention is shown for medical use. Here, right and left manipulators  100 R and  100 L are shown which are under control of right and left controllers  102 R and  102 L, respectively. Elements of the imaging system are substantially the same as those employed in the imaging system illustrated in  FIGS. 1–3  described above except that an enlarged virtual image  104 V of actual workspace  104  is provided for viewing by the operator. Also, servomechanism elements for connection of the right and left controllers  102 R and  102 L to the respective manipulators  100 R and  100 L are substantially the same as those described above with reference to  FIGS. 1–3 . In the illustrated arrangement, the right and left manipulators are of substantially the same construction as are the right and left controllers, such that a description of one manipulator and one controller applied to both. Again, suffixes R and L are used to distinguish between right and left elements thereof. 
     The manipulators include outer control sections  100 RA and  100 LA end insertion sections  100 RB and  100 LB, which insertion sections are adapted for insertion into a body cavity through cylindrical tubes, or cannulas, not shown. For purposes of illustration, the manipulators are shown inserted through the abdomen wall  106  of a subject. As is well understood, for laparoscopic surgical procedures, wall  106  is separated from internal organs by insufflation wherein a gas is introduced into the abdomen by any suitable means not shown. Manipulator motors and associated motor control circuits are contained in the outer control sections  100 RA and  100 LA of the manipulators for control of the insertion section. The manipulators, together with a laparoscope  108  for viewing organs within the cavity, are carried by a fixed rail  110  forming part of a surgical table upon which the subject is supported. 
     The insertion sections  100 RB and  100 LB of the manipulators may be of substantially the same design as manipulator arms  34 R and  34 L described above with reference to the  FIGS. 1–3  embodiment. The insertion sections are of relatively small size for use inside the body. Insertion section  100 RB includes telescopic inner section  112 R 1  and outer section  112 R 2 , which outer section is adapted for both axial movement into and out of inner section  112 R 1  and for rotation about its longitudinal axis. End effectors  114 R and  114 L are carried at the outer ends of the respective right and left sections  112 R 2  and  112 L 2  for manipulation of organ  116 . The inner section  112 R 1  is adapted for pivotal movement about intersecting perpendicular axes located substantially at point P where the insertion section intersects wall  106 . Exclusive of operation of end effectors  114 R and  114 L the manipulator arms each are provided with four degrees of freedom, the same as in the embodiment shown in  FIGS. 1–3 . End effectors  114 R and  114 L simply may comprise, essentially, microsurgical instruments with their handles removed including, for example, retractors, electrosurgical cutters and coagulators, microforceps, microneedle holders, dissecting scissors, blades, irrigators, and sutures. 
     Laparoscope  108  for viewing the workspace  104  is shown comprising an outer operating section  108 A and insertion section  108 B. The outer end section  120  of insertion section  108 B is axially and rotatably movable within the inner end  122  thereof, and is provided with a pair of image transmission windows  124 , 124  for stereoscopic viewing of workspace  104 . The laparoscope also is provided with illuminating means, not shown for illuminating the workspace, and with liquid inlet and outlet means, not shown, for flow of liquid past the windows. Video camera means within section  108 A are responsive to light received through the viewing windows for generation of left and right electronic images at output lines  48 R and  48 L for connection to image memory  50 . A magnified 3-dimensional image  104 I is produced at display means  54  for viewing by the operation wearing cross-polarized glasses  60  via mirror  66 . As with the embodiment shown in  FIGS. 1–3 , a virtual image  104 V of the workspace  104  is produced adjacent control arms  130 R and  130 L of controllers  102 R and  102 L. Control arms  130 R and  130 L are of the same type as control arms  76 R and  76 L included in the  FIGS. 1–3  embodiment described above. They include telescopic inner and outer sections  132 R 1  and  132 R, and  132 L 1  and  132 L 2 . Sensor means  134 R and  134 L located adjacent the outer ends of the control arms control operation of end effectors  114 R and  114 L, respectively, in the manner described above with reference to  FIGS. 1–3 . It here will be noted that the angle from vertical at which the image is viewed by the operator need not equal the angle from vertical at which the object is viewed by the cameras. In the arrangement illustrated in  FIGS. 7–9 , the operator is shown to view the image  104 V at an angle θ from vertical ( FIG. 7 ) whereas the object  116  is shown as viewed directly downwardly. With no external reference, the sense of vertical within a body is not particularly great, and no confusion is produced in the mind of the operator as a result of the different observer and camera viewing angles relative to vertical. 
     With the  FIGS. 7–9  embodiment, not only is a magnified virtual image  104 V of the workspace provided for viewing by the operator, but control arms  130 R and  130 L of greater length than the length of the manipulator insertion sections  100 RB and  100 LB are employed. Servomechanism scaling of axial movement of the telescopic control arms is provided such that axial extension or retraction thereof results in a smaller extension or retraction of the telescopic insertion sections. Angular pivotal motion of the control arms  130 R and  130 L produces the same angular pivotal motion of insertion sections  100 RB and  100 LB, and rotational movement of the end sections  132 R 2  and  132 L 2  of the control arms produces the same rotational motion of end sections  112 R 2  and  112 L 2  of the insertion sections of the right and left manipulators, without scaling. This embodiment of the invention, with its magnified image, is of particular use in the area of microsurgery, and especially in those cases where the surgeon cannot reach an area by hand because of size constraints. 
     The present invention is not limited to use with manipulators having any particular number of degrees of freedom. Manipulators with different degrees of freedom which are well known in the art may be used in the practice of this invention. In  FIGS. 10 and 11 , to which reference now is made a controller  140  and manipulator  142 , respectively, are shown which include a wrist joint to provide the same with additional freedom of movement. The illustrated controller  140  includes a housing  144  affixed to the bottom of table top  68  upon which table mirror  66  is located. An enlarged virtual image  146 V of actual workspace  146  is provided adjacent the operator&#39;s hand  148  viewable by the operator when looking downwardly onto the mirror  66  in a manner described above. 
     A control arm  150 L comprising inner and outer sections  150 L 1  and  150 L 2 , respectively, is mounted within housing  144  for pivotal movement in any pivotal direction as indicated by intersecting double-headed arrows  152  and  154 . The outer section  150 L 2  is adapted for axial movement into and out of inner section  150 L 1  in the direction of double-headed arrow  156 . It also is adapted for rotation about its longitudinal axis in the direction of double-headed arrow  158 . In this embodiment, the control arm includes an end section  160  pivotally attached to outer section  150 L 2  by wrist joint  162  for pivotal movement in the direction of double-headed arrow  164 . End section  160  comprises axially aligned inner and outer sections  160 A and  160 B, the outer section  160 B of which is rotatable about its longitudinal axis in the direction of double-headed arrow  166 . As with the above-described arrangements, sensor means  168  are located adjacent the free end of the control arm for operation of an end effector  170  at manipulator  142  shown in  FIG. 11 . 
     Referring to  FIG. 11 , end effector  170  is shown to comprise a pair of movable jaws attached to a wrist  172  comprising axially aligned links  172 A and  172 B. Outer link  172 B is rotatable about its longitudinal axis relative to inner link  172 A by motor means, not shown, in the direction of double-headed arrow  166 M in response to rotation of section  160 B of the hand-operated control unit in the direction of arrow  166 . Wrist link  172 A is pivotally attached to a shaft or manipulator forearm  174  for pivotal movement in the direction of double-headed arrow  164 M in response to pivotal movement of end section  160  of the hand-operated control means about pivot axis  162 . Forearm  174  is longitudinally axially movable in the direction of double-headed arrow  156 M in response to axial movement of outer section  150 L 2  of control arm  150 L in the direction of double-headed arrow  156 . It also is rotatable about its longitudinal axis in the direction of double-headed arrow  156 M in response to rotation of outer section  150 L 2  of control arm  150 L in the direction of double-headed arrow  158 . Additionally, it is pivotally movable about point  176  in the directions of double-headed arrows  152 M and  154 M in response to pivotal movement of control arm  150 L in the directions of double-headed arrows  152  and  154 , respectively. For biomedical use, such as remote laparoscopic surgery, pivot point  176  is substantially located at the level of abdominal wall  178  through which the manipulator extends. In  FIG. 11 , manipulator arm  174  is shown extending through a cannula  180  which penetrates the abdominal wall. 
     The outer operating end of the manipulator is adapted for attachment to a supporting rail, not shown, of the surgery table upon which the subject is supported. It includes an end effector drive motor  182  for opening and closing of gripper  170 . Wrist drive motor  184  controls pivotal movement of wrist  172  in the direction of double-headed arrow  164 M, and extension drive motor  186  controls axial movement of manipulator arm  174  in the direction of double-headed arrow  156 M. Forearm pivotal control motors and linkages, identified generally by reference numeral  188 , provide for pivotal movement of arm  174  about pivot point  176  in the directions of arrows  152 M and  154 M. Pivotal motion about point  176  is provided by simultaneous lateral movement of the outer operating end of the manipulator and pivotal movement of arm  174 . Movements are coordinated such that the center of rotation of forearm  174  is fixed in space at point  176  at the level of the abdominal wall. 
     Controller  140  and manipulator  142  are included in a system such as shown in  FIGS. 7 ,  8  and  9  which includes a second controller and manipulator for use by the operator&#39;s right hand, and associated servomechanism means of any suitable type, not shown, for remote control of the manipulators by the hand-operated controllers. Video camera means at the worksite, such as shown in  FIG. 9 , together with display means, such as shown in  FIG. 7 , are employed for providing the operator with an image of the workspace at a location adjacent the left and right hand-operated control means. By using manipulators with a wrist joint, an added degree of freedom is provided for increased maneuverability and usefulness thereof. However, as noted above, the present invention is not limited to use with manipulators with any particular degree of freedom. 
     Reference now is made to  FIGS. 12 and 13  wherein a modified form of this invention is shown which provides for direct viewing of a 3-dimensional image  240 I of a workspace, not shown. In  FIGS. 12 and 13 , only the operator&#39;s station is shown, which includes right and left controllers  242 R and  242 L and associated right and left hand-operated means  244 R and  244 l which may be of the same type as controllers and control arms described above. The operator&#39;s station is adapted for remote control of manipulators which also may be of the above-described type. The 3-dimensional image  240 I of the workspace is provided by visual display means  246  in conjunction with electrooptical device  58  at the face of the display means and cross-polarized glasses  60  worn by the operator, to which display means left and right video fields from left and right video cameras that view the workspace are alternately supplied, all in the manner described in detail above. End effector and object images  248  and  250 , respectively, are shown within the workspace image as viewed by video cameras at the worksite. The display means  246  is located adjacent the left and right hand-operated means  244 R and  244 L for direct viewing by the operator. With this arrangement, the end effector and object images together with the hand-operated means  244 R and  244 L are simultaneously viewable by the operator. Since the hand-operated means also are visible, the operator is provided with a visual sense of connection between the end effector means and hand-operated means whereby they appear substantially as being integral. 
     Reference now is made to  FIG. 14  wherein the distal end portion, or tip,  260  of the insertion section of an endoscope is shown which is of substantially the same type as shown in the above-mentioned publication entitled “Introduction to a New Project for National Research and Development Program (Large-Scale Project) in TY 1991” which endoscope may be used in the practice of the present invention. The insertion end of the endoscope includes a pair of spaced viewing windows  262 R and  262 L and an illumination source  264  for viewing and illuminating workspace to be observed. Light received at the windows is focused by objective lens means, not shown, and transmitted through fiber-optic bundles to a pair of cameras at the operating end of the endoscope, not shown. The camera outputs are converted to a 3-dimensional image of the workspace which image is located adjacent hand-operated means at the operator&#39;s station, not shown. Right and left steerable catheters  268 R and  268 L pass through accessory channels in the endoscope body, which catheters are adapted for extension from the distal end portion, as illustrated. End effectors  270 R and  270 L are provided at the ends of the catheters which may comprise conventional endoscopic instruments. Force sensors, not shown, also are inserted through the endoscope channels. Steerable catheters which include control wires for controlling bending of the catheters and operation of an end effector suitable for use with this invention are well known. Control motors for operation of the control wires are provided at the operating end of the endoscope, which motors are included in a servomechanism of a type described above for operation of the steerable catheters and associated end effectors from a remote operator&#39;s station. As with the other embodiments, the interfacing computer in the servomechanism system remaps the operator&#39;s hand motion into the coordinate system of the end effectors, and images of the end effectors are viewable adjacent the hand-operated controllers in a manner described above. With this embodiment, the operator has the sensation of reaching through the endoscope to put his hands directly on the end effectors for control thereof. Endoscopes of different types may be employed in this embodiment of the invention so long as they include one or more accessory channels for use in control of end effector means, and suitable viewing means for use in providing a visual display of the workspace. For example, gastric, colonscopic, and like type, endoscopes may be employed. 
     The invention having been described in detail in accordance with requirements of the Patent Statutes, various other changes and modifications will suggest themselves to those skilled in this art. For example, as noted above, the invention may include the use of tactile feedback to provide the subtle sensations for palpation and for manipulating tissues and instruments. To provide this feedback, tactile sensor arrays may be included on the end effectors which are coupled to tactile sensor stimulator arrays on the hand-operated control means, which reproduce the tactile sensation on the operator&#39;s hands. A variety of transduction technologies for teleoperator tactile sensing are known including resistive/conductive, semiconductor, piezoelectric capacitive and photoelectric. Hand-operated control means and manipulators of different types may be employed using a wide variety of well-known mechanisms and electromechanical elements including, for example, gimbals, linkages, pulleys, cables, drive belts and bands, gears, optical or electromagnetic position encoders, and angular and linear motors. Force feedback to the operator requires use of body contact with hand-operated control means. Both hand grip type hand controllers such as those illustrated, and control brace type hand controllers are well adapted for use with the present invention for force feedback to the operator. Control brace hand controllers include use of structures with positive sensors mounted on the operator at joints for measuring joint angles. Force feedback then can be applied to each joint. Similarly, light fabric gloves with variable-resistance or fiber-optic flex sensors mounted on the joints for measuring bending of individual fingers may be used. Gloves of this type also may be provided with force feedback to provide for telepresence interaction with real objects. Regardless of the type of hand-operated control means employed, an image of the workspace is produced adjacent thereto to provide the operator with a sense that the end effector means and hand-operated control means are substantially integral. Also, as noted above, servomechanisms of many different types are well known in the robotic and teleoperator system arts, and the invention is not limited to any particular type. Those that include force and torque feedback to the operator are preferred to contribute to a telepresence sense of operation. In addition, many different means for producing a stereoscopic image of the workspace are known. For example, instead of using two cameras, a single camera may be employed together with switched cross-polarizing elements in the image receiving path. In this case, a pair of spaced stereoscopic lenses are used for viewing the workspace from different angles and providing first and second images thereof to the camera. In the  FIG. 9  arrangement, wherein a laparoscope is shown, other types of endoscopes may be used for viewing the workspace. As noted above, the invention is not limited to any particular application or use. In the biomedical field, uses include, for example, open surgery, including surgery from a remote location, microsurgery, and minimum invasive surgery such as laparoscopic and endoscopic surgery. Laboratory use including microscopic manipulation also is contemplated. Industrial use of the invention include, for example, hazardous materials handling, remote operations, microassembly, and the like. Military and undersea use of the teleoperator system of this system are apparent. It is intended that the above and other such changes and modifications shall fall within the spirit and scope of the invention defined in the appended claims.