Patent ID: 12213755

DETAILED DESCRIPTION

One or more implementations described herein relate to actuated grips of a controller. In some implementations, a controller includes one or more grip members, each moveable in a respective degree of freedom, a shaft coupled to the one or more grip members, and an actuator coupled to the shaft and operative to output actuator forces on the shaft. The actuator forces cause grip forces to be applied via the shaft to the grip members in the respective degrees of freedom. For example, the degree of freedom of each grip member can be a rotary degree of freedom, and the shaft is coupled to the grip member via at least one rotary coupling such that the grip member is rotatable relative to the shaft. The actuator forces can be output to the shaft along a longitudinal axis of the shaft, and the shaft is decoupled in rotation from the actuator about the longitudinal axis of the shaft.

Various other features are also disclosed. For example, the actuator can be a linear actuator outputting active linear forces, or can be a rotary actuator outputting rotary forces. Rotary forces from a rotary actuator can be converted to linear forces through the use of a transmission, e.g., a ballscrew mechanism or a crank and linkage. Some implementations can include two grip members that rotate in respective degrees of freedom, e.g., toward each other or away from each other in a pincher-type of movement. In some examples, link members can couple the grip members to the shaft, and in some implementations the link members can be attached to the shaft at rotational couplings located a further distance from their respective linked grip member than the rotational coupling of the other link member. Some implementations can provide multiple grip members, where a grip member receives forces from an associated actuator independently of the other grip members. A spring can be coupled between the shaft and a controller body, which can provide resistive force to a closing motion of the grip members. In some examples, the grip members and the shaft can together be rotated about the lengthwise axis of the shaft, and forces can be applied in this degree of freedom by a second actuator, where the shaft is decoupled in rotation from the second actuator.

Described features also include sensing positions of one or more grips of a controller in one or more respective degrees of freedom, where the positions are used to control movement of an end effector of a slave device in communication with the controller, and applying force to the one or more grips in the grip degrees of freedom using one or more actuators. The force can be applied according to at least one force profile associated with a type of the end effector controlled by the grips. For example, a force profile can be selected from multiple force profiles associated with different types of end effectors usable with the slave device. The force profiles can define different grip forces for different grip positions, such as different linearly-changing forces at different grip positions. Some implementations can use the grip forces to constrain or hold one or more of the grips in a particular position, such as a closed position of the grip members, and/or to match the positions of the grips to current positions of components of the controlled end effector.

Features described herein provide forces on one or more grips of a controller and provide several advantages. For example, decoupling the rotation of a transmission shaft from an actuator providing linear forces on the shaft allows the grips to be rotated about an axis of rotation of the shaft without having to rotate the actuator. A spring can assist actuator forces on the grips, e.g., by resisting motion of the grip members in particular directions, allowing the active actuator to be sized smaller. The grip forces can provide different assistive forces on the controller grips for different types of end effectors of a controlled slave device, such as different types of surgical instruments. This can provide more effective control over these different types of instruments. For example, forces can guide the user with respect to particular grip positions that correspond to particular instrument positions for particular types of end effector instruments (e.g., a grip position to close a held clip in a clip applier, a closed or static grip position for an instrument not using grip motion, etc.). Positions of the grips can be matched to a current position of an instrument component of a controlled slave device, providing the user with an intuitive sense of control over the instrument immediately after contacting the grips. Forces can be output on the grips as informational assistance to the user, e.g., to indicate particular interactions of a controlled slave device and/or events occurring in during the control procedure. For example, vibrations can be output directly on grips contacted by a user's fingers using described features, rather than or in addition to outputting a vibration in other, less directly-experienced degrees of freedom of the controller or vibrating the entire master controller system.

Features thus allow a user to operate a controller more easily, accurately, and intuitively, thus providing more accurate results in procedures performed using the controller. For example, medical procedures performed using the controller and slave devices can be accurately performed with less user training required.

The terms “center,” “parallel,” “perpendicular,” “aligned,” or particular measurements in degrees, Hertz, or other units as used herein need not be exact and can include typical engineering tolerances.

FIG.1is a diagrammatic illustration of an example teleoperated surgical system100which can be used with one or more features disclosed herein. Teleoperated surgical system100includes a master control workstation (e.g., surgeon's console)102and a manipulator slave device104.

In this example, the master control workstation (e.g., surgeon's console)102includes a viewer213(shown inFIG.2) where an image of a worksite is displayed during an operating procedure using the system100. For example, the image can be displayed by a display device such as one or more display screens, depict a surgical site during a surgical procedure. A support110is provided on which a user112, e.g., an operator such as a surgeon, can rest his or her forearms while gripping two master controllers210and212(shown inFIG.2), one in each hand. The master controllers are positioned in a workspace114disposed inwardly beyond the support110. When using the workstation102, the user112can sit in a chair in front of the workstation, position his or her eyes in front of the viewer and grip the master controllers, one in each hand, while resting his or her forearms on the support110. Additional details are described below with reference toFIG.2.

A manipulator slave device104is also included in the teleoperated system100. During a surgical procedure, the slave device104can be positioned close to a patient (or simulated patient) for surgery, where it can remain stationary until a particular surgical procedure or stage of a procedure is completed. Manipulator slave device104can include one or more arm assemblies120. In some examples, one or more of the arm assemblies120can be configured to hold an image capturing device, e.g., an endoscope122, which can provide captured images of a portion of the surgical site. In some implementations, the captured images can be transmitted to the viewer of the workstation102and/or transmitted to one or more other displays, e.g., a display124coupled to the slave device120. In some examples, each of the other arm assemblies120may include a surgical tool126. Each surgical tool can include a surgical end effector, e.g., for treating tissue of the patient.

In this example, the arm assemblies120can be caused to move and articulate the surgical tools126in response to manipulation of the master controllers210and212at the workstation102by the user112, e.g., so that the user112can direct surgical procedures at internal surgical sites through minimally invasive surgical apertures. For example, one or more actuators coupled to the arm assemblies120can output force to cause links or other portions of the arm assemblies to move in particular degrees of freedom in response to control signals received from the workstation102. The workstation102can be used within a room (e.g., an operating room) with the slave device104or can be positioned more remotely from the slave device102, e.g., at a different location than the slave device.

Some implementations of the teleoperated system100can provide different modes of operation. In some examples, in a non-controlling mode (e.g., safe mode) of the teleoperated system100, the controlled motion of the manipulator slave device104is disconnected from the master controllers of the workstation102in disconnected configuration, such that movement and other manipulation of the master controls does not cause motion of the manipulator slave device104. In a controlling mode of the teleoperated system (e.g., following mode), motion of the manipulator slave device104can be controlled by the master controls210and212of the workstation102such that movement and other manipulation of the master controllers causes motion of the manipulator slave device104, e.g., during a surgical procedure.

Some implementations can be or include a teleoperated medical system such as a da Vinci® Surgical System (e.g., a Model IS3000 or IS4000, marketed as the da Vinci® Si® or da Vinci® Xi® Surgical System), commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. However, features disclosed herein may be implemented in various ways, including teleoperated and, if applicable, non-teleoperated (e.g., locally-controlled) implementations. Implementations on da Vinci® Surgical Systems are merely exemplary and are not to be considered as limiting the scope of the features disclosed herein. For example, different types of teleoperated systems having slave devices at worksites can make use of actuated controlled features described herein. Other, non-teleoperated systems can also use one or more described features, e.g., various types of control systems and devices, peripherals, etc.

In some implementations, a controlled slave manipulator device can be a virtual representation of device, e.g., presented in a graphical simulation provided by a computing device coupled to the teleoperated system100. For example, a user can manipulate the master controls210and212of the workstation102to control a displayed representation of an end effector in virtual space of the simulation, similarly as if the end effector were a physical object coupled to a physical slave device.

FIG.2is a front elevational view of an example master control workstation102as described above forFIG.1. Master control workstation102includes a viewer213, where an image of a worksite can be displayed during a procedure using the teleoperated system100. For example, images depicting a surgical site can be displayed during a surgical procedure. The viewer213can be positioned within a viewing recess211in which the user can position his or her head to view images displayed by the viewer213. When using the workstation102, the user112can sit in a chair in front of the workstation and position his or her head within the recess211such that his or her eyes are positioned in front of the viewer213.

In some implementations, one or more user presence sensors214can be positioned at one or more locations of the master control workstation102to detect the presence of a user located next to or near to the workstation102. In this example, the user presence sensors214can sense a presence of a user's head within the recess211. For example, an optical sensor can be used for a presence sensor, where the optical sensor includes an emitter216and a detector218. A beam of infrared or other wavelength of light is emitted from one side of the recess211by the emitter216, and the beam is detected on the other side of the recess by the detector218. If the beam is interrupted from detection by the detector, the system determines that a user's head is within the recess and that the user is in a proper position to use the master controllers of the master control workstation102. Additional or alternative types of presence sensors can be used in various implementations.

Two master controllers210and212are provided for user manipulation. In some implementations, each master controller210and212can be configured to control motion and functions an associated arm assembly120of the manipulator slave device104. For example, a master controller210or212can be moved in a plurality of degrees of freedom to move a corresponding end effector of the slave device104in corresponding degrees of freedom. The master controllers210and212are positioned in workspace114disposed inwardly beyond the support110. For example, a user112can rest his or her forearms while gripping the two master controllers210,212, with one controller in each hand. The user also positions his or her head within the viewing recess211to view the viewer213as described above while manipulating the master controllers210and212. Various examples of master controller portions are described below.

Some implementations of workstation102can include one or more foot controls220positioned below the master controls210and212. The foot controls220can be depressed, slid, and/or otherwise manipulated by a user's feet to input various commands to the teleoperated system while the user is sitting at the master control workstation102.

FIG.3is a perspective view of an example portion300of a master controller which can include one or more features described herein. In some implementations, master controller portion300can be used as a portion of a master controller210or212as described above with reference toFIGS.1and2. In some implementations, the master controller portion300includes one or more gimbal mechanisms.

Master controller portion300includes a handle302which is contacted by a user to manipulate the master controller300. In this example, the handle302includes two grips that each include a finger loop304and a grip member306. The two grip members306are positioned on opposite sides of a central portion303of the handle302, where the grip members306can be grasped, held, or otherwise contacted by a user's fingers. The two finger loops304are attached to grip members306and can be used to secure a user's fingers to the associated grip members306. The user may also contact other portions of handle302while grasping the grip members306. The grip members306are pivotally attached to the central portion303of the handle302. Each grip member306and finger loop304can be moved in an associated degree of freedom308by a user. For example, the grip members306can be moved simultaneously in a pincher-type of movement (e.g., toward or away from each other). In various implementations, a single grip member306and finger loop304can be provided, or only one of the grip members306can be moved in the degree of freedom308while the other grip member306can be fixed with reference to the handle302.

One or more sensors (not shown) coupled to the handle302can detect the positions of the grip members306in their degrees of freedom308and send signals describing the positions to one or more control circuits of the teleoperated system100. The control circuits can provide control signals to the slave manipulator device104, an example of which is described with reference toFIG.23. For example, the positions of the grip members306in degrees of freedom308can be used to control any of various degrees of freedom of an end effector of the slave manipulator device104, some examples of which are described below. Some implementations of the controller300can provide one or more passive actuators (e.g., springs) between the grip members306and the central portion303of the handle302to provide resistance in particular directions of the grips (e.g., movement in directions toward each other in degree of freedom308). Various implementations can provide one or more active actuators (e.g., motors, voice coils, etc.) to output active forces on the grip members306in the degree of freedom308. For example, a sensor and/or actuator can be housed in central portion303or in housing309and coupled to the grip members306by a transmission.

The handle302of example master controller portion300can additionally be provided with a rotational degree of freedom310about an axis312extending approximately along the center of the central portion303of handle302. A user can rotate the grip members306as a single unit around the axis312to provide control of, e.g., an end effector of the manipulator slave device104or other element of the slave device.

One or more sensors (not shown) can be coupled to the handle302to detect the rotation and/or position of the handle302in the rotational degree of freedom310. For example, the sensor can send signals describing the position to one or more control circuits of the teleoperated system100which can provide control signals to the slave device104similarly as described above. For example, degree of freedom310can control a particular degree of freedom of an end effector of the slave device that is different than a slave degree of freedom controlled by degree of freedom308of the grip members306.

Some implementations of the controller300can provide one or more actuators to output forces on the handle302(including grip members306and finger loops304) in the rotational degree of freedom310. For example, a sensor and/or actuator can be housed in housing309and coupled to the handle302by a shaft extending through the central portion303of the handle302.

In various implementations, the handle302can be provided with additional degrees of freedom. For example, a rotational degree of freedom320about an axis322can be provided to the handle302at a rotational coupling between an elbow shaped link324and a link326, where the elbow shaped link324is coupled to the handle302(e.g., at housing309). For example, axis322can be similar to axis232shown inFIG.2. Additional degrees of freedom can similarly be provided. For example, link326can be elbow-shaped and a rotational coupling can be provided between the other end of link326and another link (not shown). A rotational degree of freedom328about an axis330can be provided to the handle302at the rotational coupling. For example, axis330can be similar to axis230shown inFIG.2. In some examples, the master controller300can allow movement of the handle302within the workspace114of the master control workstation102with a plurality of degrees of freedom, e.g., six degrees of freedom including three rotational degrees of freedom and three translational degrees of freedom. This allows the handle302to be moved to any position and any orientation within its range of motion. One or more additional degrees of freedom can be sensed and/or actuated similarly as described above for the degrees of freedom308and310. In some implementations, each additional degree of freedom of the handle302can control a different slave degree of freedom (or other motion) of an end effector of the slave device104.

One or more features described herein can be used with other types of master controllers. For example, ungrounded master controllers can be used, which are free to move in space and disconnected from ground. In some examples, one or more handles similar to handle302and/or grip members306can be coupled to a mechanism worn on a user's hand and which is ungrounded, allowing the user to move grips freely in space. In some examples, the positions of the grips relative to each other and/or to other portions of the handle can be sensed by a mechanism coupling the grips together and constraining their motion relative to each other. Some implementations can use glove structures worn by a user's hand. Furthermore, some implementations can use sensors coupled to other structures to sense the grips within space, e.g., using video cameras or other sensors that can detect motion in 3D space. Some examples of ungrounded master controllers are described in U.S. Pat. Nos. 8,543,240 and 8,521,331, both incorporated herein by reference. The detection of user touch described herein can be used with ungrounded master controllers. For example, vibration can be applied to a handle (e.g., grip) by one or more actuators coupled to the handle, and this vibration can be sensed similarly as described herein to determine if the handle is contacted or grasped by the user.

FIG.4is a perspective view andFIG.5is a side elevational view of an example implementation of a portion400of a controller including one or more features described herein. In some implementations, controller portion400can be used as a portion of a master controller210or212as described above with reference toFIGS.1and2. In some implementations, the controller portion400includes one or more gimbal mechanisms. In this example implementation, the controller portion400can provide forces in the degrees of freedom of the grips of the controller.

Controller portion400can include several elements similar to controller portion300shown inFIG.3. For example, a handle402can be contacted by a user to manipulate the master controller400. In this example, the handle402includes two grips that each include a grip member406aor406b. The two grip members406aand406bare positioned on opposite sides of a central portion403of the handle402, where the grip members406aand406bcan be grasped, held, or otherwise contacted by a user's fingers. For example, finger contacts407aand407bcan be connected or formed at the unconnected end of the grip members406aand406b, respectively, to provide surfaces to contact the user's fingers. Finger loops (not shown) similar to finger loops304ofFIG.3can be attached to the grip members in some implementations, e.g., to secure a user's fingers to the associated grip members406aand406b.

The grip members406aand406bare coupled to the central portion403of the handle402at rotational couplings409aand409b, respectively, allowing rotational movement of the grip members with respect to the central portion. Each grip member406aand406bcan be moved in an associated degree of freedom408aand408b, respectively (seeFIG.5), e.g., by a user contacting the grip members. For example, in some implementations the grip members406aand406bcan be moved simultaneously in a pincher-type of movement (e.g., toward or away from each other). For example, the first and second grip members can move simultaneously and in coordination, e.g., move in opposing directions and by the same angular amount in their respective degrees of freedom in response to motion of the main shaft. In various implementations, a single grip member406aor406bcan be provided, or only one of the grip members406aor406bcan be moved in the associated degree of freedom408aor408bwhile the other grip member406bor406acan be fixed with reference to the handle402. In other implementations, the grip members can be coupled to the handle with other mechanisms and can be moved in linear degrees of freedom, e.g., in linear directions toward and away from the central portion403of the handle402.

One or more sensors (not shown inFIGS.4-6) can be coupled to the handle402and/or other components of the controller portion400and can detect the positions of the grip members406aand406b. The sensors can send signals describing sensed positions and/or motions to one or more control circuits of the teleoperated system100. In some modes or implementations, the control circuits can provide control signals to the slave manipulator device104. For example, the positions of the grip members406aand406bin degrees of freedom408aand408bcan be used to control any of various degrees of freedom of an end effector of the slave manipulator device104, some examples of which are described herein.

An active actuator (e.g., motor, voice coil, etc.)411can be coupled to the grip members406aand406band can output active forces on the grip members in the either or both of degrees of freedom408aand408bbased on control signals received by the actuator411. For example, the actuator411can be coupled to the grip members406aand406bby a main shaft and/or a transmission. Some examples of such couplings are described below.

A sensor413can be used to sense motion of the grip members406aand406b. Sensor413can sense the position of a moving portion of actuator411in its linear range of motion (described below), which indicates the position of the grip members406aand406bin their rotary degrees of freedom. The sensor413can be any of a variety of types of sensors, e.g., a magnetic sensor (e.g., magnetic incremental linear position sensor, Hall Effect sensor, etc.), optical sensor, encoder, resistance sensor, etc.

Some implementations of the controller400can provide one or more passive actuators (e.g., springs, brakes, etc.) coupled to the grip members406aand406band the central portion403of the handle402to provide greater resistance in particular directions of the grips (e.g., movement in directions toward each other in degrees of freedom408aand408b) than in other directions (e.g., movement in directions away from each other in degrees of freedom408aand408b). Passive actuators can provide resistance in rotation of handle402about axis412. Some examples of a passive actuator are described below.

The handle402of example master controller portion400can additionally be provided with a rotational degree of freedom410about an axis412extending approximately along the center of the central portion403of handle402. A user can rotate the grip members406aand406bas a single unit around the axis412to provide control of, e.g., an end effector of the manipulator slave device104or other component of the slave device.

An active actuator414can be coupled to the handle402and output forces on the handle402(including grip members406) in the rotational degree of freedom410. Some examples of the transmission of force from actuator414to the handle402are described with respect toFIG.6. One or more sensors can be coupled to the handle402to detect the rotation and/or position of the handle402in the rotational degree of freedom410. For example, the sensor can send signals describing the position to one or more control circuits of the teleoperated system100which can provide control signals to the slave device104similarly as described above. In some examples, a sensor (e.g., a rotary encoder) can be coupled with actuator414to sense rotation of the actuator shaft of actuator414and sense rotation of the handle about axis412.

In various implementations, the handle402can be provided with additional degrees of freedom. For example, a rotational degree of freedom420about an axis422can be provided to the handle402at a rotational coupling between an elbow shaped link424and another link (not shown), similarly as shown for elbow shaped link324and link326of controller portion300ofFIG.3. Additional degrees of freedom can similarly be provided as described above forFIG.3. In some examples, the master controller400can allow movement of the handle402within the workspace114of the master control workstation102with a plurality of degrees of freedom, e.g., six degrees of freedom including three rotational degrees of freedom and three translational degrees of freedom. This allows the handle402to be moved to any position and any orientation within its range of motion. One or more additional degrees of freedom can be sensed and/or actuated similarly as described above for the degrees of freedom. In some implementations, each additional degree of freedom of the handle402can control a different slave degree of freedom of an end effector of the slave device104.

In some implementations, handle402can also include one or more switches or buttons440, e.g., coupled to the central portion403or to mechanisms within central portion403. For example, two buttons440can each be positioned on opposite sides of axis412, or additional buttons can be provided. In some examples, button440can slide parallel to the axis412, e.g., as directed by a user's finger, or the button can be depressed. The button440can be moved to various positions to provide particular command signals, e.g., to select functions, options, or modes of the control console and/or master controller (e.g., a controlling mode or non-controlling mode as described below), to command a slave device or other system in communication with the master controller, etc. In an example implementation, button440can be coupled to a magnet. For example, button440can be coupled to a rod that extends parallel to the axis412, where the rod can include a magnet at its end. The magnet is sensed by a magnetic sensor coupled to a plate430, where the plate430is rigidly coupled to the central portion403of the handle402. When the button440is activated by the user, e.g., slid by a user parallel to axis412, the magnet is moved into a range sensed by the magnetic sensor. Other types of sensors can alternatively be used, such as optical sensors, mechanical switches, etc.

In some implementations, a touch-sensitive sensing surface can be provided on the handle402to sense a user's touch using any of a variety of types of sensors such as capacitive sensors, resistive sensors, optical sensors, etc. In some examples, one or more such sensing surfaces can be provided on the central portion403of the handle402. In another example, a sensing surface can be provided on a portion of plate430. The sensing surface can be tapped by a user's finger to provide selections or commands, and/or various gestures of the user's finger(s) over the sensing surface can be sensed to provide different selections or commands (e.g., a swipe, pinch, fingers moving away from each other, etc.).

FIG.6is a perspective view of master controller portion400ofFIG.4showing the controller portion from a different perspective. In this example, actuator411is shown coupled to a main shaft602which is linearly moved by the actuator to provide forces to the grip members406aand406b. The axis412can be the longitudinal axis of the main shaft602, for example. In this example implementation, actuator411is a linear voice coil actuator outputting linear forces. A moving portion604of the actuator411is forced linearly along the axis412with respect to the grounded portion606of the actuator411. The grounded portion606is coupled to the structure610, which is rigidly coupled to the link424. For example, in some implementations the moving portion604can include a coil holder having a coil. When electric current is provided in the coil, the moving portion604is caused to move based on the magnetic field induced with a magnet of the grounded portion606. Alternatively, the coil can be provided in the grounded portion606and the magnet provided in the moving portion604.

Moving portion604of the actuator411can move linearly along a guide rail612that is coupled to the grounded structure610. A groove or slot (not shown) is provided in the moving portion604to engage with the guide rail612and align its movement with the guide rail612. Other mechanisms can be used in other implementations to guide the moving portion604along axis412. For example, a linear rail can be provided on the moving portion604and a groove or slot can be provided in the grounded structure610. Sensor413can sense the position and/or motion of the moving portion604to determine the position or motion of the grip members406aand406b, which are coupled to the linear motion of moving portion604through main shaft602.

The second actuator414can be a rotary actuator coupled to the structure610. In this example, the actuator414is positioned such that its axis of rotation416is offset from the central axis412. The actuator414can output rotational forces on its shaft to drive the rotation of the handle402about axis412. For example, a belt620can grip the rotating shaft of the actuator414and can also grip a pulley622that is configured to rotate about axis412. This configuration allows the actuator414to rotate the pulley622. The pulley622can be rigidly coupled to a member including plate430(shown inFIG.4), where the member and plate430are rigidly coupled to the central portion403of the handle402. Thus, the pulley622can transmit rotational forces to the handle402around axis412. In some implementation, the belt620can be a toothed belt that engages a toothed circumferential surface of pulley622to provide traction between belt and pulley.

FIGS.7A and7Bare side elevational views of the interior of controller portion400ofFIG.4and showing different positions of the grip members406aand406b.

InFIG.7A, the grip members406aand406bare in an “open” position, e.g., the grip members are at a position in which their disconnected ends are furthest away from each other as allowed by the coupled mechanism. To obtain this position from a position in which the links are closer to each other, the actuator411provides a force in the direction702away from the grip members406aand406b. For example, the moving portion604of the actuator411can be moved in direction702. Moving portion604is coupled to the main shaft602at a coupling704. Coupling704couples the main shaft602to the moving portion604of the actuator411along the linear directions of axis412, and decouples the shaft602and moving portion604in the rotational directions around axis412so that the shaft602can rotate without rotating the moving portion604.

The main shaft602is also decoupled in rotation from the pulley622and the central portion403of the handle402so that the shaft can rotate without rotating these elements. The tip706of the main shaft602is connected to one end of a spring708that extends along the axis412, where the other end of the spring708is connected to a pin710that is connected to the central portion403of the handle402. Spring708compresses in the direction opposite to direction702. Depending on the rest position of the spring708in different implementations and the current position of the shaft602along axis412, the spring708can bias the movement of the main shaft602in either direction702or in the opposite direction. In some implementations, pin710can be made adjustable by a user. For example, the pin710can include screw threads to move the pin710along the axis412when rotated, thus adjusting the tension in spring708.

Two intermediate links714aand714bare connected at one of their ends to the main shaft602at rotational couplings716aand716b, respectively, allowing the intermediate links to rotate with respect to the main shaft602. The intermediate links714aand714bare connected at their other ends to the grip members406aand406bat rotational couplings718aand718b, allowing the intermediate links to rotate with respect to the grip members406aand406b.

In response to the main shaft602being moved along axis412, force is transmitted from the shaft602along the intermediate links714aand714bto the grip members406aand406b. This causes force in the rotational degrees of freedom of the grip members406aand406b, e.g., around the axes at couplings409aand409b, respectively. In the described implementation, the rotational coupling716ais positioned on the opposite side of axis412from the grip member406athat receives force from the link714aconnected to that rotational coupling716a. Similarly, the rotational coupling716bis positioned on the opposite side of axis412from the grip406bthat receives force from the link714bconnected to that rotational coupling716b. These connections cause the intermediate links714aand714bto cross in a scissor configuration when viewed from the side as inFIGS.7A and7B. In some implementations, the shaft end of intermediate link member714ais coupled to the shaft602at a first location of the shaft at coupling716athat is spaced further from the grip member406athan a second location of the shaft at coupling716b. The shaft end of the intermediate link member714bis coupled to the shaft602at the second location of the shaft at coupling716bthat is spaced further from the grip member406bthan the first location of the shaft at coupling716a.

In some implementations, the intermediate links714aand714bcan rotate in respective planes approximately parallel to each other, e.g., close to and not touching each other. For example, the intermediate links714aand714bcan be positioned in planes offset to one side of the axis412such that these planes do not intersect the axis412. In some implementations, the intermediate links714aand714bcan be positioned on opposite sides of the axis412.

InFIG.7B, the grip members406aand406bhave been positioned in a “closed” position, e.g., the grip members are at a position in which their disconnected ends are closest to each other. To obtain this position from the position shown inFIG.7A, the actuator411provides a force in the direction703toward the grip members406aand406b. For example, the moving portion604of the actuator411can be moved in direction703. Moving portion604causes the main shaft602to move in direction703.

The movement of main shaft602in direction703causes the rotational couplings716aand716bto be moved in that same direction. This causes the intermediate links714aand714bto exert force on the grip members406aand406bin the directions in which they are rotated towards each other. The grip members406aand406bcan be rotated in this manner until reaching the closed end position as shown inFIG.7B.

The spring708can be tensioned to have a rest position such that force is exerted on the main shaft602in the direction702(shown inFIG.7A) when the grip members are in the closed position shown inFIG.7B. This causes a force that biases the grip members406aand406bto move away from each other in some or all of the movement range of the grip members406aand406b. In some implementations, multiple springs can be connected to the main shaft602to provide multiple different resistances for the degrees of freedom of the grip members406aand406b. For example, concentric springs can be provided, where one of the springs is providing resistance in an initial position range of the grip members, until a second position range of the grip members in which the second spring is encountered such that both springs provide resistance to the grip member motion. Additional springs can also be used for additional resistance at other position ranges.

In some implementations, other mechanisms can be used. For example, the intermediate links714aand714bcan be provided in different configurations connecting the main shaft602to the grip members406aand406b. In some examples, the rotational couplings716aand716b, and/or718aand718b, can be positioned in different locations on their respective components. In some implementations, rotational couplings716aand716bcan be co-located to rotate about the same axis.

FIG.8is a diagrammatic illustration of an example arm assembly800or portion thereof, which can be used for one or more of the arm assemblies120of the manipulator slave device104shown inFIG.1, and which in some implementations can be controlled by master controller implementations described herein. Arm assembly800can include multiple links802,804, and806coupled to each other by rotational couplings. For example, link member802can be coupled to a grounded structure, link member804can be coupled to link member802, and link member806can be coupled to link member804. Each link member can be coupled to the other link member(s) at rotational axes sensed and driven by sensors and actuators, allowing portions of arm assembly800to be actuated and sensed about rotational axes810,812, and814. Some implementations can provide additional actuated and/or sensed motion of the arm assembly, e.g., about axes extending lengthwise through the links802,804, and806, thus allowing rotation about axes820,822, and824. One example of a surgical manipulator arm is a da Vinci® surgical system instrument manipulator arm available from Intuitive Surgical, Inc. of Sunnyvale, Calif.

An end effector mechanism840can be coupled to the end of link member806and provides an end effector842at its distal end. The end effector842is provided the degrees of freedom provided by the rotation of the link members802,804, and806as described above. End effector mechanism840additionally can provide linear motion to the end effector842along a linear axis844. Furthermore, end effector mechanism840can provide rotational and other degrees of freedom to the end effector842as described below with reference toFIG.9. In some examples, actuators and sensors included in a mechanism846of the end effector mechanism840can provide such degrees of freedom to the end effector842.

In some implementations, components in the arm assembly800can function as force transmission mechanisms to receive teleoperated servo actuation forces and redirect the received forces to operate components of the end effector842. In some examples, end effector842receives multiple separate actuation inputs from the end effector mechanism840and/or other arm assembly components, e.g., where the number of actuation inputs depend on the number of instrument features to be controlled. In other examples, the end effector842can include one or more motors or other actuators that operate associated features of the end effector. Some implementations can control end effector features such as the pitch, yaw, and/or roll of the end effector842, opening jaws of the end effector842, the output of material transported through a connecting tube and out of end effector842(e.g., liquid or other fluids), suction forces provided by end effector842, and/or any of a multiple of other end effector functions (e.g., moving a blade, etc.).

FIG.9is a perspective view of one example of an end effector900. For example, end effector900can be used as end effector842of the arm assembly800as referenced above with respect toFIG.8. End effector900is an example surgical instrument that can operate as forceps in a surgical procedure to grasp tissue, objects, etc. Other types of surgical instruments and end effectors can be used in other implementations as described elsewhere herein.

End effector900can be provided at a distal end of a main tube910which can be coupled to another portion of the end effector mechanism840shown inFIG.8, for example. A proximal clevis912is coupled to the distal end of main tube910, and a distal clevis914is coupled to the proximal clevis912by a rotational coupling. The forceps end effector900includes jaws916and918that are coupled to the distal clevis914by a rotational coupling.

The jaws916and918are provided with several physical degrees of freedom that can be manipulated by the master controllers210and212of the master control workstation102(shown inFIGS.1and2). For example, the jaws916and918can be rotated about axis930of the link between the jaws and the distal clevis914, e.g., to open and close the jaws with respect to each other as shown by arrow932, and/or to rotate the jaws in conjunction to a different rotational position. In addition, the jaws916and918can be rotated about axis934of the link between distal clevis914and proximal clevis916, e.g., to rotate the jaws in space. In addition, the jaws916and918can be translated along linear axis936, which in some implementations can correspond to the linear axis844shown inFIG.8.

When using the example master controller portion400ofFIGS.4-6, movement of the end effector900in one or more degrees of freedom can correspond to movement in one or more degrees of freedom of the master controller handle402by a user. For example, the positions of grip members406aand406bof controller portion400in their degrees of freedom can control corresponding rotational positions of the jaws916and918about axis930. The motions of the jaws916and918in other degrees of freedom of the end effector can be controlled by particular associated degrees of freedom of a master controller210or212.

In some implementations, one or more of the degrees of freedom of the end effector900can be controlled using tendons, e.g., cables (not shown), that are mechanically coupled to one or more of the elements914,916, and918and extend through tube910to a transmission or other mechanism. For example, the tendons can be coupled to pulleys and/or other transmission elements driven by actuators and sensed by sensors provided in mechanism846coupled to arm assembly800as shown inFIG.8.

In some examples, the end effector900can be inserted through a patient's body wall (or simulated body wall) to reach a surgical site. In some implementations, main tube910may include a cavity that can provide material transfer along the tube. For example, material may be transferred between a distal end and a proximal end of tube910, or points near the proximal end and near the distal end of tube910. For example, main tube910(or other tube) can couple a surgical irrigation fluid (liquid or gas) source (not shown) to the end effector900so that irrigation fluid can be routed from a source through the main tube to exit via end effector900. Similarly, main tube910can couple a surgical suction source (not shown) to end effector900so that material from a surgical site can be drawn into end effector900and through tube910to the source. Other types of connection features can be provided in other implementations.

Other types of arm assemblies and types of end effectors can be used in other implementations. For example, end effector mechanisms and instruments can include flexible elements, articulated “snake” arms, steerable guide tubes, catheters, scalpels or cutting blades, electro-surgical elements (e.g., monopolar or bipolar electrical instruments), harmonic cutters, scissors, forceps, retractors, dilators, clamps, cauterizing tools, needles, needle drivers, staplers, drills, probes, scopes, light sources, guides, measurement devices, vessel sealers, laparoscopic tools, or other tip, mechanism or device.

FIG.10is a diagrammatic illustration of a graph1000of example output force profiles that can be used with one or more features described herein. Graph1000has a vertical dimension indicating a scale of an output force that is provided on a handle, e.g., the master controller handle402as described above with reference toFIGS.4-7B. For example, the output force can be output on each of the grip members406aand406bin the rotary degrees of freedom408aand408b, respectively, using the active actuator411and a transmission mechanism including main shaft602and intermediate links714aand714b. In some examples, the output force (grip force) can be output in directions on the grip members406aand406bto bias the grip members towards or away from each other, as provided by the link structure shown inFIGS.7A and7B.

Graph1000has a horizontal dimension indicating a range of grip positions, e.g., the angular position of a grip member406aor406bin its rotary degree of freedom. In this example, the horizontal dimension ranges from the left edge of the graph that corresponds to the closed position of the grip member (e.g., as inFIG.7B) to the right edge of the graph that corresponds to the open position of the grip member (e.g., as inFIG.7A).

A number of different force profiles are shown which, in some implementations, can be used in association with the grip members406aand406bto provide different forces for different grip member positions. For example, different force profiles can be used to apply different forces when controlling different types of end effectors of a slave device. A force profile indicates the particular force output on a grip member406aor406bat a particular position of the grip member in its rotary degree of freedom. In some implementations, a force profile can indicate the force output on both grip members406aand406bat corresponding positions in their degrees of freedom, e.g., using the mechanisms ofFIGS.4-7B. In some implementations, the force profile can be the result of an actuator force output from an active actuator (e.g., actuator411) in combination with a passive force provided by a passive actuator such as spring708. In some examples, to achieve a particular force profile output, the actuator force output may vary based on the linearity of the mechanical system between the actuator and the grip members and based on the involved forces from spring708, e.g., as indicated below in examples ofFIGS.11A-11B.

In some examples, a force profile can be defined by a force output function that indicates the output force on the grip member based on the position of the grip member, e.g., a linear function, an exponential function, etc. In further examples, one or more of the force profiles can describe a multiple-stage force output, where different force output functions can be used at different ranges of positions of a grip member to provide different force sensations on the grip member at the different position ranges. For example, a multiple-stage force profile can include multiple different linear force output functions that indicate the amount of force output on a grip member at the positions of the grip member in an associated range of positions in the degree of freedom. In some examples, multiple linear functions of a force profile can have different slopes, or other characteristics or shapes so as to provide different sensations to the user at different position ranges of the grip member. For example, a more resistant spring force can be output in one position range, a force bump can be output in a different position range, etc.

In some examples, a force profile1002can describe a multiple-stage force output on the grip members406aand406bwhich can be enabled by implementations of actuators and mechanisms described herein. Three different example stages are shown, each using a linear force output function. For example, at an open position of the grip members, represented at the right edge of the graph1000, force profile1002indicates that a smaller output force applied in the directions of the degrees of freedom that force the grip members apart from each other, thus allowing a user to move the grip members together more easily. As the grip members are moved closer toward each other, corresponding to a direction from right to left on the graph1000, the force opposing this motion is increased linearly as shown on the right linear section of profile1002. At a point1004of the profile1002, the opposing output force is ramped up with a higher slope (e.g., higher increase in opposing force from left to right on graph1000) for positions closer to the closed position of the grip members. This increase in opposing force can be a “bumper” that notifies the user of a particular position in the range of motion of the grip members406aand406b. In some examples, this increased force is provided for a short range of positions between points1004and1006, and then at point1006the opposing force is increased at a more gradual (lower) rate for positions of the grip members closer to each other (to the left of point1006on profile1002), e.g., the increase in opposing force is less to the left of point1006than the increase in opposing force between points1004and1006, in a direction from left to right. In some examples, the change in force to the left of point1006to a more gradually-increasing force can cause the higher-sloped increase in force output from point1004to point1006to be more noticeable to the user. For example, in some implementations, a user may be more sensitive in feeling changes or transitions in the rate of force output (e.g., at points1004and1006) than in changes of force provided at a constant rate (e.g., between points1004and1006).

In some examples, the force bumper provided by the changes in force output to the left of point1004can describe resistance to closing the grip members406aand406bwhich can to notify the user. For example, a particular instrument used for a controlled end effector may cause a particular action or effect if commanded by the grip members to close past the point1004from left to right in the graph1000. In one example, a forceps-like instrument (e.g., forceps end effector900) may hold a particular item between its jaws. If the grip members406aand406bare moved closer to each other than (to the left of) the position at point1006, the jaws will be fully closed. The increase in force output at point1004can thus reduce the likelihood that the user will inadvertently move the grip members406aand406bcloser together than the position at point1006. In addition, the increase in force output can notify the user that the grip members406aand406bhave reached the position beyond which the forceps will be closed. In this and other examples, a sudden change in stiffness followed by an increase in stiffness as the grip members are moved in particular directions (e.g., toward the closed position) can signify to the user a controller movement zone to be entered with user intent.

In some implementations, the force profile1002can simulate the use of two physical springs concentrically positioned, one inside the other, that provide resistance to the grip members406aand406b. For example, the force output to the right of point1002of force profile1002can simulate the simulated compression of a first spring before a second spring has been contacted. The force output between points1004and1006can be the simulated compression of both the first spring and a second spring that has a partial preloaded compression, which causes a greater rate of resistance to closing the grip members between points1004and1006. The force output to the left of point1006can be the simulated compression of the first spring and the second spring, where the second spring is compressing after having moved through its preload in the region between points1004and1006.

A force profile1010can describe a multiple-stage force to be output on the grip members406aand406b. Force profile1010can be similar to force profile1002by having three stages with linear force output functions. For example, the force output for positions to the right of point1012on the force profile1010provide resistance to closing the grip members406aand406bfrom a fully open position (represented at the right limit to profile1010). For grip positions between point1012and point1014on the profile1010, an increased force is output, resisting closure of the grip members406aand406b. For grip positions closer to the closed position than (to the left of) point1014on profile1010, the rate of increase in output force resisting closure is reduced relative to the stage between points1012and1014.

In some examples, the force profile1010can be provided when controlling an end effector that is different than an end effector controlled using the force profile1002. For example, the point1012on profile1010occurs closer to the fully open position of the grip members406aand406bthan the corresponding point1004of force profile1002. This causes a force “bumper” at a different position of the grip members. For example, this can be useful for particular types of end effectors. In one example, a bipolar cautery instrument may require that the jaws of the controlled instrument be apart by a particular distance or less in order for cauterizing energy to pass between the jaws of the instrument. The point1012, and the change in force output at that point, can indicate that particular distance, e.g., can indicate the position of the grip members406aand406bthat will cause the jaws of the instrument to be positioned at that particular distance.

A force profile1020can describe a multiple-stage force to be output on the grip members406aand406b. For example, force profile1020can provide three different force output stages similarly to force profiles1002and1010, where point1022on force profile1020is similar to points1004and1012of force profiles1002and1010, and point1024on force profile1020is similar to points1006and1014of force profiles1002and1010. In some examples, force profile1020can be used in the control of a different type of instrument at the end effector. For example, the force output at grip positions between points1022and1024has a lower slope and is closer to the rightmost stage of force profile1020than corresponding output forces used for profiles1002and1010. This causes the force bumper between points1020and1022on profile1020to be less noticeable to the user operating the grip members406aand406b. For example, some instruments may not need a strong force bumper to indicate a particular position of the grip members or particular instrument state. In some implementations, the less-noticeable force bumper of the force profile1020can be used to more subtly indicate a particular grip position to the user.

A force profile1030can describe a two-stage force applied to the grip members406aand406b. The force output for grip positions to the right of point1032on the force profile1030provide resistance to closing the grip members406aand406bfrom a fully open position. For grip positions to the right of point1032on the profile1030, a force is output on the grip members406aand406bthat increases at a greater rate (e.g., greater slope) from right to left than in the rightmost stages of the force profiles1002,1010, and1020. For grip positions to the left of point1032on profile1030, the change in output force is reduced so that it is almost flat, e.g., an almost constant force output at the grip positions from point1032to the closed position at the left.

In some examples, the force profile1030can be used in the control of a particular type of end effector. For example, a particular end effector instrument may provide a particular action or effect if commanded with grip positions to the left of the point1032. In one example, the end effector instrument can be a clip applier that has jaws similar to a forceps instrument, and which are specialized to hold an open clip between its jaws. The clip applier jaws can be closed to permanently close the clip, where a closed clip can be used to join or attach portions of surgical tissue, for example. In the force profile1030, the point1032can indicate a closing grip position, where grip positions to the left of the point1032will cause the controlled clip applier to close, which in turn causes the held clip to close. The grip positions to the right of point1032can thus receive increased resistance as shown for profile1030. The user has to overcome a stronger resistance to move the grips past the point1032, thus reducing the likelihood that the user operator will inadvertently close the grips past the point1032and thus inadvertently close a held clip. Some implementations can position the point1032further to the left on the force profile1030, e.g., to provide more movement range for the grip members when moving from the open position before the grip members encounter a bumper indicating the position to close the clip.

A force profile1040can describe a single-stage force applied to the grip members406aand406b. For example, a linearly-increasing spring force can be applied to the degrees of freedom of the grip members406aand406b, or to a portion of the degrees of freedom. In the example of force profile1040, a linearly-increasing force is applied to the grip members within a small range of grip positions adjacent to the closed position of the grip members at the left side of the graph1000(e.g., about 3 degrees of the 30 degree range of grip positions). This force output allows the grip members to feel a steeply-increasing force resistance the closer they are moved to the closed position.

In some implementations, the grip members can be held, or can be biased to be held, to maintain a position within a controlled and/or predefined range portion of their degrees of freedom, e.g., a limited range of positions, due to actuator forces provided by the active actuator. The predefined range portion can be a subset of positions of the full range of positions allowed in the degree of freedom of the grip member. For example, in some implementations using force profile1042, the grip members406aand406bcan be held to an approximate position1042(or within a small range of positions approximately centered on position1042), e.g., at about 3 degrees from the closed position. In the example ofFIGS.4-7B, to hold the grip positions, the actuator411outputs force on the main shaft602in the opposite direction to the force output shown in graph1000so that the grip members406aand406bresist being opened further by the force provided from the physical spring708. The actuator411can hold this position of the grip members whether or not the user is touching or holding the grip members. If a user moves the grip members closer to a closed position from the held position1042, the grip members are allowed to close, with a force resisting the closing motion as indicated by force profile1040. In some examples, this small range of motion of the grip members allowed near the closed position can be used in some implementations to relieve stress or tension on a user's fingers or hands during use of the grip members by the user, e.g., caused by the user holding the grips tightly.

Some implementations can hold the grip members to a single position or a small sub-range of the grip member's movement when controlling particular types of instruments as the end effector. For example, a hook (e.g., cautery hook), probe, spatula, or other type of instrument that has a single tip or monopole can be controlled by the handle402having the grip members406aand406bheld in a closed position, or held close to a closed position, e.g., using a force profile similar to profile1040. The degrees of freedom of the controller portion400other than the grip member degrees of freedom can be used to control and move the end effector in corresponding degrees of freedom in space.

In other implementations, the grip members406aand406bcan be held at other positions or sub-ranges of positions in the degrees of freedom of the grip members, e.g., in the middle of the degree of freedom, near the open position of the grip members406aand406b, etc. In some implementations, if a force profile indicates that an amount of force should be quickly changed by a large magnitude (e.g., more than a particular threshold amount of force), then the force can be gradually changed from its current output to the indicated level of output, e.g., ramping the forces. For example, a user may suddenly be detected using the controller, causing a sudden change of output from zero force to a high magnitude force indicated by the force profile. Such a force can be gradually ramped up to the indicated high magnitude. In another example, the grip members may be being held at a closed position by the actuator force during use, and then a condition occurs to cause the force to be removed from the grips. If the force is quickly removed, this may allow the spring to force open the grips quickly, which may be alarming to the user. Thus, the actuator force can be ramped down by the actuator and gradually removed from the grip members to reduce this effect.

Some implementations can allow user or operator customization of force profiles. For example, if a particular user prefers that a large range of motion be provided to the grip members to control a particular instrument, then a force profile providing a normal (smaller) range of motion of the grip members can be changed to a different force profile that allows a larger range of motion to be more easily used (e.g., by moving a force change point such as point1004further to the left in the graph ofFIG.10). In some implementations, a different set of customized force profiles can be associated with different users. For example, the identity of the user using the master controller can be determined, e.g., using any of known techniques such as user login with password, user biometrics recognition (voice, fingerprint, retina, etc.), and other techniques. A stored set of customized force profiles associated with that user can then be selected and used during controller operation. These force profiles can also be selected based on the type of end effector being used, as described herein.

In some implementations, a single grip member406aor406bcan be moved and actuated independently of the other grip members. For example, each a grip member406aand406bcan receive force derived from output of a respective associated actuator. One of such independent grip members406aand406bcan receive forces to achieve a different force profile than the other grip member406. In some examples, one grip member406aor406bcan be held at or close to a closed position, e.g., based on force profile1040, and the other grip member406bor406acan be provided with forces based on a different force profile, e.g., force profile1002,1010, etc. In some implementations, an end effector can be controlled based on movement of one grip member406aor406bin its degree of freedom, e.g., the grip member controlled by force profile1002or1010.

In some implementations, other types of force profiles can be used. For example, force profile curves can include bumps or spikes, e.g., where the profile goes higher to a point and then lower than the point in the direction from left to right. Some implementations can use force profiles that are discontinuous or otherwise have large jumps in force output. For example, in the direction from left to right on graph1000, the profile can stop at a first grip position and resume for further positions at a different output force that is higher or lower than the output force at the first grip position.

FIG.11Ais a diagrammatic illustration of a graph1100of example actuator force profiles that show example forces output by an actuator over a position range of a controller handle. In this example, graph1100has a vertical axis indicating an example scale of an output force that is provided by the actuator411onto the main shaft602as shown inFIGS.6and7A-7B. A positive force on this axis indicates a force in the linear direction on the main shaft that biases the grip members406aand406btoward their open position, and a negative force on this axis indicates a force that biases the grip members406aand406btoward their closed position. Graph1100has a horizontal dimension indicating an example range of positions of a component of the controller receiving force from the actuator, e.g., the positions occupied by the main shaft602or a moving portion of an actuator. In this example, the horizontal dimension ranges from the left edge of the graph that corresponds to the shaft position at the closed position of the grip members (e.g., as inFIG.7B) to the right edge that corresponds to the shaft position at the open position of the grip members (e.g., as inFIG.7A). The force curves shown in graph1100can be similar to force curves for mechanisms other than the implementations ofFIGS.4-7B.

A force curve1102indicates forces required to be output by the actuator411to causing a resulting force that maintains the grip members406aand406bat approximately a static position in their degrees of freedom. The actuator411outputs a force that maintains the grip members406aand406bin opposition to the force provided by the spring708. The spring biases the grip members406aand406btoward the open position, so the forces of curve1102are in the negative direction that bias toward the closed position. The curve1102shows a linear output required over the range of positions of the main shaft602, in opposition to the linear force provided by the spring over that range of positions.

A force curve1110indicates forces required to be output by the actuator411to cause a consistent resulting nominal force on the grip members406aand406b. In one example, the particular nominal force can be a force magnitude at a position of a force profile such as any of the force profiles shown inFIG.10. Curve1110shows positive forces so that the actuator output force works in conjunction with the force provided by the spring708to resist closing of the grip members toward the closed position.

The force curve1110shows a particular actuator force at the right end of the curve1110(e.g., about 0.4 pounds in one example) which is output by the actuator at the open position of the grip members406aand406b. The curve1110dips slightly over the curve toward the left of the open position (as the grip members406aand406bare at positions closer to the closed position), and rises again at the left end of the curve1110at the closed position of the grip members, corresponding to a force value slightly above the value at the right end of the curve1110. Thus, force curve1110indicates an approximately linear output required by the actuator over the range of positions of the grip members to provide the desired nominal force on the grip members. The actuator output need not be compensated significantly to provide a consistent output force on the grip members406aand406bover the range of motion of the grip members.

A force curve1120indicates forces required to be output by the actuator411to cause a consistent resulting higher force on the grip members406aand406bthan indicated by force curve1110. For example, the force indicated by curve1120can be output by the actuator to provide a particular maximum force level to the grip members (e.g., 1.5 Newtons in some examples). Force curve1120can be similarly shaped to force curve1110, except at higher values of force output by the actuator. A particular actuator force at the right end of the curve1120(e.g., about 0.5 pounds in one example) is output at the open position of the grip members406aand406b. The curve1120is approximately flat over the curve toward the left of the open position, and then rises near the closed position of the grip members, ending at a force value above the value at the right end of the curve1110(e.g., about 0.8 pounds in the example). Thus, force curve1120indicates a rising output force required to be output by the actuator as the grip members are moved toward the closed position, in order to provide the desired maximum force on the grip members.

The force curves1102,1110and1120also indicate that the range of output of the actuator411is being utilized efficiently. For example, force curve1110requires a maximum of about-0.8 pounds output force, and force curve1120requires a maximum of about 0.8 pounds output force, which are approximately the same in opposite directions.

The shape and range of the curves1102,1110, and1120can be determined based on the particular mechanisms used in the master controller to provide forces on the grip members. For example, the lengths of the main shaft602and other links used in the transmission mechanism, the locations of the couplings between the shaft and links, the properties of the spring708(e.g., spring constant, preload), and/or the force output capability of the actuator411over its output range can all be tuned to provide curves similar to the force curves shown inFIG.11A, or other desired force curves. For example, nonlinearity of actuator forces and/or of forces provided by the spring708can be compensated in other characteristics of the system, including link lengths and coupling locations between links. Non-linearity of the components can be leveraged to provide a realistic experience of spring forces and other forces on the grip members, and, for example, can allow the user to feel as if he or she is manipulating a controlled object realistically.

FIG.11Bis a diagrammatic illustration of a graph1150of additional example actuator force profiles that show another example effect of force output on grip members406aand406bby an actuator, similarly toFIG.11A. In some examples, the force curves ofFIG.11Bresult from a more poorly-matched controller system than the controller system providing the force curves ofFIG.11A.

A force curve1152indicates forces required to be output by the actuator411to causing a resulting force that maintains the grip members406aand406bat a static position in their degrees of freedom, similarly to force curve1102ofFIG.11A. The curve1152is different than the curve1102in that it requires a different range of output forces from the actuator, e.g., a smaller range, and a more extreme force output at the closed position of the grip members (e.g., about-1 pounds instead of about-0.8 pounds inFIG.11A). Thus, the mechanism transmitting the forces to the grip members does not spread out the required forces into a greater portion of the range of actuator force output as much as the mechanism used for the force curves ofFIG.11A.

A force curve1160indicates forces required to be output by the actuator411to cause a consistent resulting nominal force on the grip members406aand406b, similarly to force curve1110ofFIG.11A. Curve1160is less consistent than curve1110. For example, the force for the open position is about 0.2 pounds, and the required force for the closed position is about 0.4 pounds, which is a more extreme difference than in curve1110. The mechanism used for curves1160and1170does not perform consistently over the position range of the grip members, and requires significant compensation from the actuator.

A force curve1170indicates forces required to be output by the actuator411to cause a consistent maximum force level force on the grip members406aand406bthat is higher than the forces for force curve1160. This is similar to force curve1120ofFIG.11A. Curve1170is less consistent than curve1120or curve1160. For example, the required force for the open position is less than 0.4 pounds, and the required force for the closed position is about 0.7 pounds, which is a more extreme difference than in curve1120. The mechanism used for curve1170does not perform as consistently over the position range of the grip members as the mechanism used for curve1120, requiring more compensation from the actuator.

In some implementations, the mechanism used to determine the force curves1152,1160, and1170can be considered more poorly executed than the mechanism used to determine the force curves ofFIG.11A. In a poorly-executed system, for example, the force on the grips may drop off as the user closes the grips or the force output may otherwise act erratically, resulting in a controller that may feel uncontrollable and unnatural. In some examples, the controller used forFIG.11Bcan differ from the controller used forFIG.11Aby having one or more different lengths for grip members406aor406b, one or more different lengths for intermediate link members714, different spring properties for spring708, different location of the rotational couplings between the grip members406and the link members714, etc.

FIG.12is a perspective view of an example implementation of a controller portion1200including a crank arm transmission providing a linear force output from a rotary actuator. For example, a master controller handle1201can be similar to the master controller handle402described above with respect toFIGS.4-7B, or a different handle can be used. Handle1201and controller portion1200can include one or more of the features described for handle402and other implementations described herein.

Controller portion1200can include a main shaft1202connected to and driving grip members1204aand1204bsimilarly to corresponding components in the implementations described above forFIGS.4-7B. In some implementations, an actuator1206can be coupled to a pulley1208by a belt1209to provide rotation of the controller handle1201similarly as described above with reference toFIG.6.

Main shaft1202can be connected to the crank arm transmission that includes a rail piece1210, linkage1216, and crank1224. The main shaft1202is decoupled in rotation to the carriage piece1210so that the shaft can rotate about axis1212(e.g., longitudinal axis of main shaft1202) independently of the carriage piece1210and is linearly coupled to the carriage piece1210along the lengthwise axis1212. In some implementations, the carriage piece1210can be constrained linearly along or parallel to the axis1212of the main shaft1202by a slot on the bottom of the carriage piece1210that engages a guide rail1214that is coupled to the link1215, similarly as described above for the moving portion of the voice coil actuator411ofFIGS.4-6. A linkage1216includes a first link1218that is rigidly coupled to the carriage piece1210at a first end of the first link1218, and a second link1220that is coupled at a first end to a second end of the first link1218at a coupling1222. A second end of second link1220is coupled to a first end of a crank1224by a rotational coupling1225. A second end of crank1224is rigidly coupled to the rotating shaft of a rotary actuator1226, where the rotating shaft of actuator1226can be perpendicular to the shaft1212of the main shaft1202. For example, actuator1226can be an active actuator in some implementations, e.g., a motor.

The actuator1226can output a force on its rotatable shaft to provide rotary force on and motion of the crank1224. The force on and motion of the crank1224causes motion in second link1220, which causes first link1218to move linearly due to the rail1214engaged by the carriage piece1210. The linear motion of first link1218and carriage piece1210provides linear force on and motion of the main shaft1202, causing force on and/or motion of the grip members1204aand1204b.

The described mechanism can convert the rotary force output of the actuator1226to linear force by mirroring the mechanism used for the grip members1204aand1204b. For example, the first link1218, second link1220, and crank1224can have rotary couplings that are spaced relative to each other proportionally the same distance as the distance between rotational couplings between the main shaft1202, an intermediate link (similar to the intermediate links714aor714bshown inFIGS.7A-7B), and a grip member1204aor1204b, respectively. For example, the crank1224can mirror a grip member1204aor1204band can rotate about the actuator shaft similarly to a grip member1204aor1204brotating at the coupling1205aor1205b, respectively. The first link1218can mirror the main shaft1202and the second link1220can mirror an intermediate link connecting the main shaft1202and a grip member1204aor1204b(e.g., similar to intermediate link714aor714b). The mirrored mechanism allows the actuator1226to provide force at the grip members1204aand1204bin a linear relationship. This removes or reduces a need for compensation of actuator output with different output forces at different grip member positions to maintain a consistent force at the grip members1204aand1204bas described above forFIGS.11A and11B.

Actuators1206and/or1226can be any of a variety of types of actuators similarly as described herein for other implementations. For example, active actuators can be used, e.g., motors (e.g., DC motors), voice coils, or other types of active actuators. Passive actuators (e.g., springs, brakes, etc.) can be used in some implementations to provide resistance in particular directions of the grip members, in rotation of the handle1201about axis1212, etc.

Similarly as described in the other implementations herein, one or more sensors can be coupled to the handle1201and/or other components of the controller portion1200and can detect the positions of the grip members1204aand1204b. For example, in some implementations, a rotary encoder can be included in the housing of actuator1226to detect rotation of the shaft of actuator1226. In some implementations, a linear sensor can be coupled to the link1215to sense linear motion of the carriage piece1210or link1218. Similarly, one or more sensors can be coupled to one or more components of the controller portion1200and can detect the roll position of the handle1201about axis1212. For example, in some implementations, a rotary encoder can be included in the housing for actuator1206to detect rotation of the shaft of actuator1206. The sensors can send signals describing sensed positions or motion to one or more control circuits of the teleoperated system100. In some modes or implementations, the control circuits can provide control signals indicating sensed positions or motion to the slave manipulator device104. The sensors can be any of a variety of types of sensors, e.g., a magnetic sensor (e.g., magnetic incremental linear position sensor, Hall Effect sensor, etc.), optical sensor, encoder, resistance sensor, etc.

FIG.13is a perspective view of an example implementation of a controller portion1300including a ballscrew transmission coupled to a rotary actuator. In this example, a master controller handle1301can be similar to the master controller handle402described above with respect toFIGS.4-7B, or a different handle can be used. Handle1301and controller portion1300can include one or more of the features described for handle402and other implementations described herein.

Controller portion1300can include a main shaft1302connected to and driving grip members1304aand1304bsimilarly to corresponding components in the implementations described above forFIGS.4-7B. In some implementations, an actuator1306can be coupled to a pulley1308by a belt1309to provide rotation of the controller handle1301similarly as described above with reference toFIG.6.

Main shaft1302can be connected to a ballscrew transmission, including a ballscrew nut1310and a ballscrew1316. The main shaft1302is linearly coupled to the ballscrew nut1310along the lengthwise axis1312and is decoupled in rotation to the ballscrew nut1310so that the shaft1302can rotate about axis1312independently of the ballscrew nut1310. In some implementations, the ballscrew nut1310can be constrained in its movement linearly along or parallel to the axis1312of the main shaft1302by a slot on the bottom of the ballscrew nut1310that engages a guide rail1314that is coupled to the link1315, similarly as described above for the moving portion of the voice coil actuator411ofFIGS.4-6and guide rail1214ofFIG.12. Ballscrew1316is a threaded member that engages a threaded aperture of ballscrew nut1310. The ballscrew1316is rigidly coupled at its other end to the rotating shaft of a rotary actuator1320.

The actuator1320can output a force on its rotatable shaft to cause the ballscrew1316to rotate. The rotation of the ballscrew1316causes the ballscrew nut1310to move linearly along the linear axis1312(e.g., longitudinal axis of the ballscrew1316and main shaft1302) as constrained by rail1314. The linear motion of the ballscrew nut1310moves the main shaft1302linearly, causing rotational force on the grip members1304aand1304b. The ballscrew transmission thus converts rotary force from rotary actuator1320to linear force applied to the main shaft1302. In some implementations, actuator1320can be coupled to a sensor such as a rotary encoder1330that determines a rotational position of the actuator shaft and main shaft1302.

Actuators1306and/or1320can be any of a variety of types of actuators similarly as described herein for other implementations. For example, active actuators can be used, e.g., motors (e.g., DC motors), voice coils, or other types of active actuators. Passive actuators (e.g., springs, brakes, etc.) can be used in some implementations to provide resistance in particular directions of the grip members, in rotation of the handle1301about axis1312, etc.

Similarly as described in the other implementations herein, one or more sensors can be coupled to the handle1301and/or other components of the controller portion1300and can detect the positions of the grip members1304aand1304b. For example, in some implementations, in addition to or instead of rotary encoder1330, a linear sensor can be coupled to the link1315to sense linear motion of the ballscrew nut1310. Similarly, one or more sensors can be coupled to one or more components of the controller portion1300and can detect the roll position of the handle1301about axis1312. For example, in some implementations, a rotary encoder can be included in the housing for actuator1306to detect rotation of the shaft of actuator1306. The sensors can send signals describing sensed positions or motion to one or more control circuits of the teleoperated system100. In some modes or implementations, the control circuits can provide control signals indicating sensed positions or motion to the slave manipulator device104. The sensors can be any of a variety of types of sensors, e.g., a magnetic sensor (e.g., magnetic incremental linear position sensor, Hall Effect sensor, etc.), optical sensor, encoder, resistance sensor, etc.

FIG.14Ais a side elevational view of an example implementation of a controller portion1400including a cam mechanism to provide forces on controller grips. In some implementations, the main shaft of the controller portion1400need not be moved or biased linearly along its lengthwise axis to provide force on the grip members of the handle, and is instead rotated to provide forces on the grip members. Controller portion1400can include one or more of the features and components described for handle402and other implementations described herein.

A main shaft1402can be coupled to an actuator (not shown) that rotates the shaft1402about its longitudinal axis1404. For example, the main shaft1402can be rotated by a rotary actuator similarly to actuator1320ofFIG.13. In some implementations, an actuator that rotates the main shaft1402can be positioned directly in-line with the main shaft1402, e.g., such that the rotated shaft of the actuator is aligned with axis1404. The main shaft1402can be rotated by actuators in other configurations in other implementations, e.g., as described in various implementations herein.

Main shaft1402extends through a central portion1406of the handle1400and can be coupled to the front of the handle such that it is rotatable about axis1404. The shaft1402is rigidly coupled to a cam1408. Grip members1410aand1410bare coupled to the central portion1406at rotary couplings1412aand1412b, respectively. In some implementations, finger loops1414aand1414bcan be attached to the grip members, e.g., to assist securing a user's fingers to the grip members when in use. Any of the implementations described herein can use similar finger loops for their grip members.

Grip members1410aand1410bare coupled to rollers1418aand1418b, respectively, which each rotate about their own lengthwise axes independently of the grip members. The rollers1418aand1418bcontact the surface of the cam1408, and a spring1416can be included to bias the rollers1418aand1418bagainst the cam surface as the cam is rotated. In the example ofFIG.14A, spring1416is shown as a helical spring that is coupled between the grip member1410aand the grip member1410b, and the spring is in tension in the position shown inFIG.14Ato bias the group members1410aand1410bagainst the surface of the cam1408. In other implementations, other types of springs can be used for spring1416, and/or the spring1416can be placed in different locations of the controller portion1400. For example, one or more flat springs, leaf springs, or other types of springs can be used, e.g., coupled between the group members1410aand1410bor between each group member and the central portion1406.

FIG.14Bis a perspective view of an example cam mechanism including cam1408and rollers1418aand1418bthat can be used in an implementation described inFIG.14A. Cam1408includes an outer surface including portions1420aand1420bhaving continuously different radii centered on the axis1404of rotation. As the cam1408rotates, the spring force from spring1416biases the rollers1418against the surface of the cam1408, and the rollers1418aand1418bare moved further or closer to the axis1404depending on the particular portions of the cam surface that are contacting rollers1418aand1418b(e.g., depending on the angular position of the cam about axis1404) and depending on the direction of rotation of the cam1408. For example, if the cam1408is rotated in direction1422from the position shown, the roller1418aand grip member1410aare allowed to rotate about coupling1412acloser to the axis1404, since the cam surface1420acurves closer to the axis1404as the cam is rotated. Similarly, the roller1418band grip member1418bare allowed to rotate about coupling1412bcloser to the axis1404as cam surface1420bcurves closer to the axis1404.

Referring toFIG.14A, forces can be output in the rotary degrees of freedom of the grip members1410aand1410bby rotating the cam1408. Handle implementation1400therefore does not translate the main shaft1402to provide forces to the grip members. In some implementations, a linear actuator and a transmission providing linear forces need not be used, and a rotary actuator can directly drive the main shaft1402or can drive the main shaft1402via a transmission mechanism.

Some implementations can also provide a second actuator to rotate the handle1400about the axis1404, e.g., rotate the grip members1410aand1410band the handle body1406, similarly to actuators414,1206, or1306in the above implementations. In such examples, cam1408can also be rotated with the other portions of the handle1400. In some implementations, such a second actuator can be positioned on the same axis as the first actuator providing rotation of cam1408that provides forces on grip members1410aand1410b. For example, the main shaft1402can be driven by the first actuator and positioned within a hollow shaft1426that is coupled to the handle body1406and to the rotating shaft of the second actuator.

FIG.15is a perspective view of an example implementation of a controller portion1500including a capstan mechanism to transmit force from an actuator. In some implementations, a master controller handle1502of the controller portion1500can be similar to the master controller handle402described above with respect toFIGS.4-7B, or a different handle can be used. Handle1502and controller portion1500can include one or more of the features described for controller portion400and other controller implementations described herein.

Controller portion1500can include a main shaft1504connected to and driving grip members1506aand1506b, similarly to corresponding components in the implementations described above forFIGS.4-7B. In some implementations, an actuator1508(e.g., motor) can be rigidly mounted to the link1511and can be used to drive rotation of the handle1502similarly to actuator414ofFIG.4. As shown inFIG.15, actuator1508can be oriented such that its rotating shaft rotates about an axis that is oriented perpendicular (90 degrees) to the longitudinal axis1510of the main shaft1504, as described in greater detail below. In other implementations, actuator1508can be implemented and oriented similarly to actuator414, e.g., such that its rotating shaft rotates about an axis that is parallel to the axis1510and, for example, its shaft is connected to a pulley by a belt to provide rotation of the controller handle1502similarly as described above with reference toFIG.6.

An actuator1512can be provided to drive linear motion of the main shaft1502along axis1510. In some implementations, actuator1512can be a rotary DC gear motor or other type of rotary actuator. In the implementation ofFIG.15, similarly to actuator1508, actuator1512can be rigidly mounted to the link1511and oriented such that its rotating shaft rotates about an axis that is oriented perpendicular (90 degrees) to the axis1510. Actuator1512can be oriented in other ways in other implementations.

Main shaft1504can be connected to a capstan mechanism1516provided between the main shaft1504and the actuator1512. The capstan mechanism1516includes a linear carriage1518that is coupled to the main shaft1504. The main shaft1504is decoupled in rotation from the linear carriage1518such that the main shaft can be rotated independently of the linear carriage1518. The linear carriage1518can move linearly, e.g., slide, upon a linear rail1520that is rigidly coupled to the link1511. The linear rail1520is aligned parallel to the main shaft to allow linear motion of the linear carriage1518.

The capstan mechanism1516also includes a capstan drum1522having helical grooves, and which is rigidly coupled to the rotating shaft of actuator1512. The capstan drum1522is coupled to the linear carriage1518by a cable1524. For example, cable1524can be a high-stiffness metal cable in some implementations. A first end of cable1524can be attached to a groove (or via some other fastening mechanism) at a first portion of the linear carriage1518, e.g., the end or a portion of the carriage1518that is closest to the handle1502. The cable1524is wrapped a number of times around the capstan drum1522, e.g., within the grooves of the capstan drum. The second end of the cable1524can be attached at a second portion of the linear carriage1518, e.g., the end or a portion of the carriage1518that is further from the handle1502than the first portion of the carriage. In the example shown, the second end of the cable1524is attached to a nut1526on a threaded screw coupled at the second portion of the linear carriage1518, where the nut1526and second end of the cable1524can be moved closer or further from the carriage1518along the screw to adjust the tension in the cable. Other mechanisms can be used to tension the cable in other implementations.

The driven rotation of the shaft of the actuator1512directly drives the constrained linear motion of the linear carriage1518and the main shaft1504via the cable1524, thus causing forces on the grip members1506aand1506bto bias them toward open and closed positions in accordance with the linear motion of the main shaft1504, similarly as described in other implementations herein. In some implementations, a benefit of the capstan mechanism is that it can provide a high-stiffness, low-backlash transmission for active forces on the grip members1506aand1506band handle1502while allowing the actuator(s) to be mounted 90-degrees to the main shaft1504. This can reduce the packaging size, mass, and inertia of the controller portion1500.

Actuators1508and/or1512can be any of a variety of types of actuators similarly as described herein for the other implementations. For example, these actuators can be active actuators, e.g., motors (e.g., DC motors), voice coils, or other types of active actuators. Passive actuators (e.g., springs, brakes, etc.) can be used in some implementations to provide resistance in particular directions of the grip members, in rotation of the handle1500about axis1510, etc.

Similarly as described in the other implementations herein, one or more sensors can be coupled to the handle1502and/or other components of the controller portion1500and can detect the positions of the grip members1506aand1506b. For example, in some implementations, a rotary encoder can be included in the housing of actuator1512to detect rotation of the shaft of actuator1512. In some implementations, a linear sensor can be coupled to the link1511to sense linear motion of the linear carriage1518(e.g., secondary sensor1610ofFIG.16B). Similarly, one or more sensors can be coupled to one or more components of the controller portion1500and can detect the roll position of the handle1502about axis1510. For example, in some implementations, a rotary encoder can be included in the housing for actuator1508to detect rotation of the shaft of actuator1508. The sensors can send signals describing sensed positions or motion to one or more control circuits of the teleoperated system100. In some modes or implementations, the control circuits can provide control signals to the slave manipulator device104. The sensors can be any of a variety of types of sensors, e.g., a magnetic sensor (e.g., magnetic incremental linear position sensor, Hall Effect sensor, etc.), optical sensor, encoder, resistance sensor, etc.

In some implementations, transmission mechanisms other than the capstan mechanism1516can be used. For example, a rack and pinion mechanism can be used, where a pinion gear can be used instead of the capstan drum1522and a rack gear can be provided on the linear carriage1518to engage the pinion gear. In another example, a drive wheel can be used instead of the capstan drum1522, e.g., using friction to couple or engage the drive wheel to a linear surface of the linear carriage1518and move or force the carriage linearly when the drive wheel is rotated.

FIGS.16A and16Bare side elevational views of the controller portion1500ofFIG.15, whereFIG.16Ashows one side of the controller portion1500andFIG.16Bshows the opposite side.

InFIG.16A, the grip members1506aand1506bare in an open position, e.g., the grip members are at a position in which their disconnected ends are furthest away from each other as allowed by the coupled mechanism. To cause this position from a position in which the links are closer to each other, the actuator1512causes a force in the direction1602away from the grip members1506aand1506b. For example, the capstan drum1522coupled to the actuator1512can be rotated in a rotational direction to move cable1524and cause forces on linear carriage1518that cause it to move in the direction1602. Linear carriage1518is coupled to the main shaft1504and causes the main shaft1504to move in the same direction, thus transmitting force to the grip members1506aand1506bin directions toward their open positions. Similarly, the capstan drum1522can be rotated in the opposite rotational direction to move cable1524in the opposite direction and cause forces on linear carriage1518in the direction opposite to direction1602, causing the main shaft1504to move and transmit force to the grip members1506aand1506bin directions toward their closed positions.

InFIG.16B, actuator1508has a rotary shaft that is rigidly coupled to a roll bevel pinion1606. The roll bevel pinion1606includes a number of teeth that engage a number of grooves/teeth of a roll gear (ring)1530(shown inFIG.15). Rotation of the roll bevel pinion1606about the axis of rotation of the actuator shaft causes rotation of the roll bevel pinion1606about axis1510of the controller portion1500. This causes rotational forces to the handle1502, e.g., can cause the handle1502to rotate about axis1510. The roll bevel pinion1606and roll gear1530thus can provide rotational forces to the handle1502similarly to the actuator414, belt620, and pulley622described with reference toFIGS.4-6.

A sensor1610can be provided to sense linear motion of the linear carriage1518and main shaft1504along the axis1510. In some implementations, sensor1610can be included in addition to a sensor (e.g., rotary encoder)1612that can sense the rotation of the shaft of actuator1512and capstan drum1522to thereby sense linear motion of the main shaft1504.

FIGS.17A and17Bare side elevational, cross-sectional views of the interior of controller portion1500ofFIG.15and show different positions of the grip members1506aand1506b.

FIG.17Ashows an open position of the grip members1506aand1506b. Main shaft1504is coupled to linear carriage1518such that when linear carriage1518moves linearly parallel to axis1510, the main shaft1504is moved correspondingly along axis1510. The main shaft1504is decoupled in rotation from the linear carriage1518such that the main shaft can be rotated independently of the linear carriage1518. For example, one or more couplings1702(e.g., bearings) can couple the main shaft1504to the linear carriage1518along the linear directions of axis1510, and can decouple the main shaft1504and linear carriage1518in the rotational directions around axis1510so that the main shaft1504can continuously rotate independently of the linear carriage1518.

Roll bevel pinion1606is coupled to the rotating shaft of actuator1508. The roll bevel pinion1606is engaged with roll gear1530to cause rotary forces about axis1510to the handle1502. For example, the roll gear1530can be rigidly coupled to a member including a plate1704(similar to plate430shown inFIG.4), where the member and plate are rigidly coupled to the central portion1706of the handle1502and rotate with the handle1502. Thus, the roll gear1530can transmit rotational forces to the handle1502around axis1510.

FIG.17Bshows a closed position of the grip members1506aand1506b. Linear carriage1518has been moved by actuator1512in a direction1710to linearly move the main shaft1504along the axis1510toward the handle1502. In accordance with the movement of the main shaft1504, the grip members1506aand1506bhave been moved to a closed position by the linkages in handle1502, which can be similar and operate similarly to the linkages shown in handle402inFIGS.7A and7B.

Other components and alternative implementations described herein for other implementations can also be used in the controller portion1500.

FIG.18is a perspective view of an example implementation of a controller portion1800including a capstan mechanism to transmit force from an actuator. Controller portion1800can include several components which operate similarly to corresponding components of the controller portion1500, some of which are labelled inFIGS.18-20Bwith the same reference numbers as shown inFIGS.15-17B. Some differently-numbered components can also operate similarly to corresponding components of the controller portion1500described above.

Controller portion1800can include a main shaft1804connected to and driving grip members1506aand1506b, similarly to corresponding components in the implementations described above forFIGS.4-7BandFIG.15. In some implementations, actuator1508(e.g., motor) can be rigidly mounted to the link1511and can be used to drive rotation of the handle1502. As shown in the implementation ofFIG.18, actuator1508can be oriented such that its rotating shaft rotates gear1806about an axis that is oriented perpendicular (90 degrees) to the longitudinal axis1510of the main shaft1504, to engage roll gear (ring)1530and cause roll gear1530to rotate.

In the implementation ofFIG.18, actuator1512can be rigidly mounted to the link1511and is positioned such that the axis of rotation of the shaft of actuator1512is approximately perpendicular to longitudinal axis1510. In some implementations, the axis of actuator1512can be positioned close to the linear rail1904(seeFIG.19B) to reduce force and friction on the rail1904. In some implementations, the axis of rotation of the shaft of actuator1512extends such that it is positioned closer to axis1510(and/or closer to the center of the controller portion1800) than the rotary axis of actuator1512shown inFIG.15. Actuator1512in the configuration ofFIG.18may thus provide less inertia to the rotation of the handle1502than in the configuration ofFIG.15.

Main shaft1804is connected to a capstan mechanism1816provided between the main shaft1804and the actuator1512. The capstan mechanism1816includes a linear carriage1818that is coupled to the main shaft1804and which can move linearly, e.g., slide, upon a linear rail (seeFIG.19B) that is rigidly coupled to the link1511, similarly to linear carriage1518.

The capstan drum1522is coupled to the linear carriage1818by cable1524that is wrapped around capstan drum1522as described above. The first end of cable1524can be attached to a first portion of the linear carriage1818, e.g., the end or portion of the carriage1818that is closest to the handle1502. The second end of the cable1524can be attached at a second portion of the linear carriage1818, e.g., the end or portion of the carriage1818that is further from the handle1502than the first portion of the carriage1818. In the example shown, the second end of the cable1524is attached to a disc1826coupled at the second portion of the linear carriage1518, where the disc1826can be rotated to move the second end of the cable1524closer or further from the capstan drum1522to adjust the tension in the cable.

The driven rotation of the shaft of the actuator1512directly drives the constrained linear motion of the linear carriage1818and the main shaft1804via the cable1524, thus causing forces on the grip members1506aand1506bto bias them toward open and closed positions in accordance with the linear motion of the main shaft1504, similarly as described in other implementations herein.

Actuator1508has a rotary shaft that is rigidly coupled to a roll bevel pinion1806. The roll bevel pinion1806includes a number of teeth that engage a number of grooves/teeth of a roll gear1530. Rotation of the roll bevel pinion1806about the axis of rotation of the shaft of actuator1508causes rotation of the roll bevel pinion1806about axis1510. This causes rotational forces to the handle1502similarly as described above.

FIGS.19A and19Bare side elevational views of the controller portion1800ofFIG.18, whereFIG.19Ashows one side of the controller portion1800andFIG.19Bshows the opposite side.FIGS.19A and19Bare similar toFIGS.16A and16Band some corresponding components are numbered the same.

InFIG.19A, the grip members1506aand1506bare in an open position, e.g., the grip members are at a position in which their disconnected ends are furthest away from each other as allowed by the coupled mechanism. To cause this position from a position in which the links are closer to each other, the actuator1512causes a force on shaft1804in the direction1902away from the grip members1506aand1506b. For example, the capstan drum1522coupled to the actuator1512can be rotated in a rotational direction to move cable1524and cause forces on linear carriage1818that cause it to move in the direction1902. Linear carriage1818is coupled to the main shaft1804and causes the main shaft1804to move in the same direction, thus transmitting force to the grip members1506aand1506bin directions toward their open positions. Similarly, the capstan drum1522can be rotated in the opposite rotational direction to move cable1524in the opposite direction and cause forces on linear carriage1818in the direction opposite to direction1902, causing the main shaft1804to move and transmit force to the grip members1506aand1506bin directions toward their closed positions.

InFIG.19B, the grip members1506aand1506bare in an open position. Guide rail1904is coupled to the link1511, and the linear carriage1818includes a groove piece1906that slides along the guide rail1904. The components operate similarly as corresponding components in implementations described above.

A linear sensor1910can be provided to sense linear motion of the linear carriage1818and main shaft1804along the axis1510. In some implementations, sensor1910can be included in addition to a sensor (e.g., rotary encoder)1912that can sense the rotation of the shaft of actuator1512and capstan drum1522to thereby sense the linear motion of the main shaft1804.

FIGS.20A and20Bare side elevational, cross-sectional views of the interior of controller portion1800ofFIG.18and show different positions of the grip members1506aand1506b.

FIG.20Ashows an open position of the grip members1506aand1506b. Main shaft1804is coupled to linear carriage1818such that when linear carriage1818moves linearly parallel to axis1510, the main shaft1804is moved correspondingly along axis1510. The main shaft1804is decoupled in rotation from the linear carriage1818such that the main shaft can be rotated independently of the linear carriage1818. For example, one or more couplings2002(e.g., bearings) can couple the main shaft1804to the linear carriage1818along the linear directions of axis1510, and can decouple the main shaft1804and linear carriage1818in the rotational directions around axis1510.

Roll bevel pinion1806is engaged with roll gear1530to cause rotary forces about axis1510to the handle1502. For example, the roll gear1530can be rigidly coupled to a member/plate2008(similar to plate1704and430), where the member/plate is rigidly coupled to, or is an extension of, the central portion1706of the handle1502and rotates with the handle1502. Thus, the roll gear1530can transmit rotational forces to the handle1502around axis1510.

In this example implementation, a linear bushing2004is positioned around the main shaft1804and extends from the member2008toward the handle1502. Linear bushing2004guides the main shaft1804such that it maintains its position along axis1510. The bushing2004can provide a clearance or gap2010at a portion of the length of the bushing, e.g., a portion between the member/plate2008and a front portion of the bushing2004that contacts the main shaft1804. The gap2010allows some lateral movement, e.g., angular tilt or play, of the rear portion of the shaft1804at linear carriage1818, e.g., movement having component directions perpendicular to the axis1510. This allowance for play can reduce binding of the main shaft1804against the bushing2004, e.g., if there is misalignment of the shaft1804relative to axis1510. In some examples, the bushing2004can be made of slippery material, e.g., plastic.

In some examples, such as the implementation shown, the main shaft1804can be made hollow to allow one or more components to be routed through the main shaft. For example, one or more cables can be routed through main shaft1804. In some examples, cables that connect a button440(seeFIG.18) to a controller positioned at the rear of the controller portion1800can be routed through the main shaft1804. In some implementations, the main shaft1804can include one or more notches or apertures that allow components such as cables to be routed within the housing of the controller portion1800. For example, inFIG.20A, dashed line2012represents a cable that can be routed from the rear portion of the shaft near carriage1818, through the hollow main shaft1804, and out of an aperture2014of the shaft1804into an interior space, in which the cable is coupled to an electrical contact that in turn electrically connects to the buttons440. For example, the cable2012can be routed with slack or a loop as shown to allow the main shaft1804to be moved forward and back without over-stretching the cable.

FIG.20Bshows a closed position of the grip members1506aand1506b. Linear carriage1818has been moved by actuator1512in a direction2020to linearly move the main shaft1504along the axis1510toward the handle1502. In accordance with the movement of the main shaft1804, the grip members1506aand1506bhave been moved to a closed position by the linkages in handle1502, which can be similar to and operate similarly as described above.

Other components and alternative implementations described herein for other implementations can also be used in the controller portion1800.

FIG.21is a diagrammatic illustration of an example implementation of a controller system2100including multiple independently-actuated grips.FIG.21is provided in an abstracted or schematic illustration. The mechanisms can be similar in implementation as in other example controller portions described herein.

Controller system2100includes a first shaft2102coupled to a first grip member2104, and a second shaft2106coupled to a second grip member2108. Grip members2104and2108can be similar to the grip members406a,406b, etc. described for other implementations herein. For example, the grip members2104and2108can each be rotated in a rotary degree of freedom and can be coupled to their respective shafts by an intermediate linkage. In some implementations, each grip member2104and2108can be rotated in its degree of freedom independently of the other grip member2108and2104, respectively. In this example, the first shaft2102is coupled to an intermediate link2110by a rotational coupling2112, and the intermediate link2110is coupled to grip member2104at its other end by a rotational coupling2114. Similarly, the second shaft2106is coupled to an intermediate link2118by a rotational coupling2120, and the intermediate link2118is coupled to grip member2108at its other end by a rotational coupling2122.

First shaft2102is coupled to a first actuator2126at the shaft's end opposite to the intermediate link2110. For example, the first shaft2102can be coupled to the first actuator2126by a rotary coupling2128that couples the first shaft2102to the first actuator2126in the linear degree of freedom along the shaft, and decouples in rotation the first shaft2102from the first actuator2126such that the first shaft can be rotated independently of the first actuator2126. In some examples, first actuator2126can be a linear actuator outputting a linear force along the axis of first shaft2102, e.g., a voice coil or other type of actuator. For example, a voice coil actuator2126can include a magnet2130and a coil2132. In this example, the magnet2128is grounded to a linear rail2134that constrains the motion of the magnet2128along a linear axis of the first shaft2102. Actuator2126can be similar to actuator411ofFIGS.4-7B. In some implementations, actuator2126can be a different type of actuator, e.g., a motor (e.g., rotary actuator).

In this example, second shaft2106extends through a hollow interior portion of the first shaft2102and extends through the first actuator2126. Second shaft2106is coupled to a second actuator2140at the shaft's end opposite to the intermediate link2118. For example, the second shaft2106can be coupled to the second actuator2140by a rotational coupling2142that couples the second shaft2106to the second actuator2140in the linear degree of freedom along the shaft, where the second shaft2106is decoupled in rotation from the second actuator2140such that the second shaft can be rotated independently of the second actuator2140. In some examples, second actuator2140can be a linear actuator outputting a linear force along the axis of second shaft2106, e.g., similarly to first actuator2126and first shaft2102. Second actuator2140can be a voice coil or other type of actuator similarly to the first actuator2126. For example, a voice coil actuator2140can include a magnet2144and a coil2146. In this example, the magnet2144is grounded to a linear rail2148that constrains the motion of the magnet2144along a linear (e.g., longitudinal) axis of the second shaft2106.

A first spring2150is coupled between the first shaft2102and a handle body or central portion of the controller, e.g., similarly to spring708shown inFIGS.7A-7B. A second spring2152is coupled between the second shaft2106and the handle body of the controller. In some implementations, the second spring2152can extend within the helical diameter of the first spring2150, such that the first and second springs are concentric along most of their lengths, e.g., centered along the same axis that is about parallel to the first and second shafts2102and2106. In some other examples, the first spring2150can extend within the helical diameter of the second spring2152, or the first and second springs can be approximately parallel to each other and not concentric.

In operation, the controller system2100can provide forces independently in the degrees of freedom of the grip members2104and2108. For example, first actuator2126can output a linear force on first shaft2102, and the force can be output on intermediate link2110, which provides the force as a rotational force in the degree of freedom of grip member2104. Second actuator2140can output a linear force on second shaft2106independently of the force output of the first actuator on first shaft2102. The force on the second shaft can be output on intermediate link2118, which provides the force as a rotational force in the degree of freedom of grip member2108. In some implementations, a handle portion including the grip members2104and2108, first and second shafts2102and2106, and springs2150and2152can be rotated in unison in a rotary degree of freedom about the axis defined by the first and second shafts2102and2106. For example, a third actuator (not shown) can transmit forces to this handle portion in this rotary degree of freedom, similarly to actuator414ofFIGS.4-6.

In some implementations, other components can be used in system2100. For example, one or both of first actuator2126and second actuator2140can be replaced by a rotary actuator and a transmission mechanism for converting the rotary force output by the actuator to a linear force output on the first shaft2102and/or second shaft2106(e.g., using an implementation as shown inFIG.12and/orFIG.13).

One or more features described herein can be used with other types of master controllers. For example, ungrounded master controllers can be used, which are free to move in space and disconnected from ground. In some examples, one or more handles similar to handle402and/or grip members406aand406bcan be coupled to a mechanism worn on a user's hand and which is ungrounded, allowing the user to move grips freely in space. In some examples, the positions of the grips can be sensed by a mechanism coupling the grips together and constraining their motion relative to each other. Some implementations can use glove structures worn by a user's hand. Furthermore, some implementations can use sensors coupled to other structures to sense the grips within space, e.g., using video cameras or other sensors that can detect motion in 3D space. Some examples of ungrounded master controllers are described in U.S. Pat. Nos. 8,543,240 and 8,521,331, both incorporated herein by reference. The detection of user touch described herein can be used with ungrounded master controllers. For example, vibration can be applied to a handle (e.g., grip) by one or more actuators coupled to the handle, and this vibration can be sensed similarly as described herein to determine if the handle is contacted or grasped by the user.

FIG.22is a flow diagram illustrating an example method2200to provide forces on a controller. Method2200can, for example, be used with an example teleoperated system or other control system in which the controller is a master controller that controls a slave device. For example, in some implementations, the controller is a component of a workstation, e.g., master control workstation102ofFIG.1, and method2200can be performed by a control circuit component of the master control workstation102. In some examples, the control circuit can include one or more processors, e.g., microprocessors or other control circuits, some examples of which are described below with reference toFIG.23. A single master controller is referred to in method2200for explanatory purposes. The master controller can be, for example, any of the controller implementations described herein, and/or one of master controller210or212ofFIG.2. Multiple master controllers can be similarly processed as described in method2200, e.g., each master controller210and212ofFIG.2. Other implementations can use a controller having one or more features described herein with other types of systems, e.g., non-teleoperated systems, a virtual environment (e.g., medical simulation) having no physical slave device and/or no physical subject interacting with a physical slave device, etc.

In block2201, some implementations can select one or more particular force profiles for use by the control system, which designate particular forces to be output on the controller (e.g., to grip members of the controller based on grip member position). For example, the selection of force profiles can be based on the type of end effector currently controlled by the master controller. In some examples, multiple different force profiles can be available as described with reference to the examples ofFIG.10, and each of these force profiles can be associated with one or more types of end effectors usable with the slave device. The type of end effector currently connected to the slave device can be determined (e.g., via identifying information automatically read by the slave device or input manually by an operator) and a force profile associated with that type can be selected for use from storage (e.g., connected memory or storage device). The types of end effectors can be organized in various ways in different implementations. For example, types can be based on specific models of end effectors, where each difference in physical dimensions and/or operation of an end effector is defined as a different type (e.g., each particular model of forceps can be considered a different type). Alternatively, multiple end effector models can be grouped into one or more types, e.g., multiple models of forceps can be grouped into a single type despite the models having some physical differences. In some implementations, end effector types can be defined more broadly and a single type can include end effectors having similar operation. For example, grasping end effectors having jaws can be defined as an end effector type.

In some implementations, the selected force profile can be a set of force profiles, where one of the force profiles of the set is selected for use based on a current mode or other condition of operation of the teleoperated system or controller. For example, one profile in the set can be designated for use during non-controlling mode of the teleoperated system, and a different profile of the set can be designated for use during controlling mode of the teleoperated system. In some implementations in which multiple master controllers are provided for controlling multiple slave device end effectors (e.g., as in master control workstation102ofFIG.2), each master controller can be assigned its own force profile (or set of force profiles) based on the end effector it controls.

In block2202, a non-controlling mode of the control system (e.g., teleoperated system100) is activated. The non-controlling mode can also be considered a “safe mode” in which the master controller is not enabled to provide control signals to move a controlled device such as slave device104, even if the master controller is manipulated by the user. Thus, for example, the controlled device is disconnected from the master controller for non-controlling mode, e.g., the controlled device is not being controlled by the master controller. For example, the master controllers210and212can be manipulated by a user in non-controlling mode but will not cause any controlled motion of the components of the manipulator slave device104.

In block2204, information is received describing a current position of one or more end effectors that can be controlled by the control system. In some implementations, this information can be received by a master controller system (e.g., master control workstation) from the slave device. The information can be derived from sensor information sensed by position sensors of the slave device and relayed to the master controller.

In some examples, the end effector can include one or more instruments, such as the surgical instrument900shown inFIG.9which is coupled to one or more arms of a slave arm device. For example, the slave arm joints and position can be controlled by the master controller. In addition, one or more components or portions of the instruments can be controlled by the master controller. For example, the jaws or pincher elements of forceps, scissors, graspers, dissectors, clamps, sealers, shears, staplers, clip appliers, needle drivers, and other instruments can be controlled by the master controller as described above with respect toFIG.9. The information received in block2204can describe, for example, the positions of such jaws in their degrees of freedom (e.g., as angular information), and/or can describe the positions of the jaws relative to each other. In some implementations, the information can describe the current position of other types end effector (e.g., one or more components or portions of an instrument), e.g., such as portions of a retractor, cautery hook, spatula, or other component in a rotary degree of freedom of the end effector, a position in a linear degree of freedom, etc.

In block2206, one or more of the grip members of the master controller are matched to the position of the end effector (or the grip members are otherwise positioned based on the end effector). For example, a control circuit coupled to the master controller can control one or more actuators to move the grip members to a position that corresponds to the position of one or more components of the end effector. In some implementations, the grip members can be moved to positions in their degrees of freedom that correspond to the current positions of associated components of the end effector in their degree of freedom. For example, each of two jaws of a forceps or similar surgical instrument can be associated with one of the grip members, and a grip member can be positioned within its degree of freedom to a position that is proportional to a position that the associated jaw is positioned within its degree of freedom. In one example, a jaw may have a position that is spaced away from one limit of the movement range by an angular amount that is 20% of the entire angular movement range of the jaw, and the associated grip member can be similarly positioned at a position that is 20% of its movement range away from a corresponding limit of its movement range. In another example, a clip applier instrument may be spaced so that the jaws of the clip applier are open and holding a clip, and the associated grip members can be similarly positioned to an open position in the degrees of freedom of the grip members.

In another example of positioning one or more of the grip members based on the end effector, one or more of the grip members of the master controller can be constrained or held to a single position if the end effector has a single moveable components or no moveable components. For example, each inactive grip member can be moved to a closed position of the grip member and maintained at that closed position while the associated end effector is being controlled. In some implementations, one grip member can be operated to control a component of the end effector such that it can be moved within its rotary degree of freedom within a designated movement range, receive forces from the actuator for a force profile, etc., while the other grip member is constrained, e.g., held to a closed position. For example, some end effectors such as a stapler instrument may have components that can be controlled with one moveable grip member. In some implementations, the positioning of the grip members in non-controlling mode can be determined based on a force profile selected in block2201.

The alignment of position of the end effector and the controller grip members can allow the grip members to be controlled more accurately when a user first contacts the grip members. For example, if an instrument's jaws are in a fully open position but the grip members are in a half-closed position, then the full movement range of the grip members is not available. The grip members can therefore be fully opened to match the jaws of the end effector.

In block2208, it is determined whether a controlling mode (e.g., “following mode”) has been entered or activated by the control system. The controlling mode allows the master controller to control the movements of the slave device. For example, motion of grip members of the master controller can control corresponding motion of jaws of a surgical instrument of the slave device, and/or motion of the master controller in other degrees of freedom can control corresponding motions of the surgical instrument in space.

The activation of controlling mode can be initiated based on any of a variety of conditions. For example, some implementations can initiate the controlling mode in response to detecting the presence of a user at or near the master controller. For example, presence sensor214on the master control workstation102, as described with respect toFIG.2, can sense whether the head of a user has been detected in an operating position for the master controllers210and212, such as in a viewing recess211of master control workstation102. In some implementations, other sensors can be used to sense user presence. Some implementations can detect whether the user has grasped or otherwise contacted the master controller grip members, e.g., via contact sensors, optical sensors, motion sensors, sensing a change in an output force or vibration applied on the master controller, etc.

If controlling mode has not been entered, then the method returns to block2206to continue to match the position of the grip members to the end effector position. If controlling mode has been entered as determined in block2208, then the method continues to block2210, in which forces are applied to the grip members of the master controller during operation of the master controller to control the slave device. The forces can be output in the degrees of freedom of the grip members from one or more actuators as described in various implementations herein.

For example, the forces applied to the grip members can be based on one or more force profiles that indicate the force applied for each position of the grip members. For example, a force profiles selected in block2201can be used to provide particular forces for a particular type of end effector being controlled on the slave device, as described above.

In some implementations, the forces applied to the grip members can be based on one or more of a variety of states or conditions during the system operation. In some examples, if a controlled end effector encounters a physical object or surface, sensors of the slave device can relay this condition to the master control system, which then controls forces on the grip members to simulate or alert the user of the physical object or surface. For example, if jaws of the end effector pick or hold an object, forces can be output on the grip members at positions corresponding to the size of the held object in the movement range of the instrument jaws, making the grip members harder to close past corresponding positions. In another example, a vibration can be output on the grip members for a particular amount of time and with sufficiently high frequency to simulate the feel of a hard surface at particular positions of the grip members. Such a vibration can impart a sensation of initially impacting a hard surface.

The forces output on the grip members can be coordinated with conditions occurring within a simulation, e.g., a virtual environment created and responding to input provided by the master controller. For example, a simulation of a medical procedure (or other procedure) may allow the master controller to provide control signals to control a physical end effector within a virtual environment, e.g., a virtual operating site or virtual patient. The virtual environment can be displayed to the user on one or more display screens or other display devices, for example, which may also show the physical slave device in that environment (e.g., based on a camera capturing images of the physical slave device). If the physical end effector is determined by the simulation to interact with a virtual obstruction (e.g., a portion of a virtual patient), then forces can be controlled by the master control system to be output on the grip members to simulate interaction with a real environment. For example, a virtual object held by the physical instrument can provide forces on the grip members similarly as a physical object would, as described above. In another example, a simulation of a procedure may allow the master controller to provide control signals to control a virtual slave device within a virtual environment, e.g., a virtual slave device having a virtual end effector that interacts with a virtual operating site or virtual patient. The virtual slave device and environment can be displayed to the user on one or more display screens or other display devices, for example. Forces can be output on the grip members similarly as described above based on interaction of the virtual end effector with virtual objects or surfaces.

Some implementations can indicate to the user via forces output on the grip members of the master controller that the control system has activated a different mode of operation, e.g., the forces can indicate a change in the operating mode. Modes of operation can include the controlling mode and non-controlling mode described herein. In some implementations, additional modes of operation can be provided for a teleoperated system or other control system, and these modes can be indicated by different characteristics of force sensations such as vibrations, bumps (single force pulses), springs (increasing force in a direction of movement), etc. For example, a particular mode can provide control of a camera of the slave device104to the master controller instead of control of an end effector, or provide control over other slave device functions or master controller functions.

Various other conditions can cause force output on the grip members (and in other degrees of freedom of the master controller), e.g., alert forces to alert the user of a particular event or condition, forces to cause the user to provide a particular control signal (e.g., a resistance, when overcome, causes a selection of a user interface element or option in a displayed user interface), etc.

In block2212, it is checked whether the control system exits the controlling mode. For example, controlling mode can be exited in response to the user physically leaving a position to use the master controller (e.g., the user's head or hand no longer sensed in proximity to the controller or controller workstation). In other examples, controlling mode can be exited in response to the procedure being completed, user input (e.g., the user selecting an input device or displayed element in a user interface, voice command, etc.), a condition in the procedure (e.g., an unsafe movement or position of the slave device occurs), etc.

If the control system has not exited the controlling mode, the method returns to block2210to continue outputting forces during the operation of the teleoperated system. If the control system has exited the controlling mode, the method returns to block2202to activate non-controlling mode.

Some implementations of method2200can output forces on the master controller, such as on the grip members of the controller, in non-controlling mode. For example, the master controller may be able to be used in a graphical interface control mode that is a non-controlling mode, where the master controller movement in one or more degrees of freedom can interface with elements of a user interface displayed on a display. In some implementations, forces can be output on the grip members to assist a user in interacting with displayed interface elements. For example, a pinching motion of the grip members can be used to control a zoom level of a view displayed by a display screen, and forces can be output on the grip members to indicate particular zoom levels (e.g., a force “bump” output at each different zoom level). Forces can also be used to indicate a limit to adjustment of a graphical element or parameter. For example, a ramping force similar to force profile1040ofFIG.10can be output at or near a position in the grip members' degrees of freedom that correspond to a maximum parameter value in a graphical interface that can be set by controlling the grip members (e.g., a maximum or minimum zoom level, maximum scroll position of a menu, etc.).

In various implementations, forces output on the grip members can be disabled during a procedure, e.g., for safety reasons, to allow a particular form of control to the user, or for other reasons. All forces can be disabled to the grip members, and/or particular forces can be disabled, such as forces based on designated interactions of the slave device in the procedure (e.g., forces from objects held by controlled instruments, etc.).

The blocks described in the methods disclosed herein can be performed in a different order than shown and/or simultaneously (partially or completely) with other blocks, where appropriate. Some blocks can be performed for one portion of data and later performed again, e.g., for another portion of data. Not all of the described blocks need be performed in various implementations. In some implementations, blocks can be performed multiple times, in a different order, and/or at different times in the methods.

FIG.23is a block diagram of an example master-slave system2300which can be used with one or more features described herein. System2300includes a master device2302that a user may manipulate in order to control a slave device2304in communication with the master device2302. In some implementations, master device2302can be, or can be included in, master control workstation102ofFIG.1. More generally, master device2302can be any type of device providing a master controller that can be physically manipulated by a user. Master device2302generates control signals C1 to Cx indicating positions, states, and/or changes of one or more master controllers in their degrees of freedom. The master device2302can also generate control signals (not shown) indicating selection of physical buttons and other manipulations by the user.

A control system2310can be included in the master device2302, in the slave device2304, or in a separate device, e.g., an intermediary device between master device2302and slave device2304. In some implementations, the control system2310can be distributed among multiple of these devices. Control system2310receives control signals C1 to Cx and generates actuation signals A1 to Ay, which are sent to slave device2304. Control system2310can also receive sensor signals B1 to By from the slave device2304that indicate positions, states, and/or changes of various slave components (e.g., manipulator arm elements). Control system2310can include general components such as a processor2312, memory2314, and interface hardware2316and2318for communication with master device2302and slave device2304, respectively. Processor2312can execute program code and control basic operations of the system2300, and can include one or more processors of various types, including microprocessors, application specific integrated circuits (ASICs), and other electronic circuits. Memory2314can store instructions for execution by the processor and can include any suitable processor-readable storage medium, e.g., random access memory (RAM), read-only memory (ROM), Electrical Erasable Read-only Memory (EEPROM), Flash memory, etc. Various other input and output devices can also be coupled to the control system2310, e.g., display(s)2320such as the viewer213of the master control workstation102and/or display124ofFIG.2.

In this example, control system2310includes a mode control module2340, a controlling mode module2350, and a non-controlling mode module2360. Other implementations can use other modules, e.g., a force output control module, sensor input signal module, etc. As used herein, the term “module” can refer to a combination of hardware (e.g., a processor such as an integrated circuit or other circuitry) and software (e.g., machine or processor executable instructions, commands, or code such as firmware, programming, or object code). A combination of hardware and software can include hardware only (i.e., a hardware element with no software elements), software hosted by hardware (e.g., software that is stored at a memory and executed or interpreted by or at a processor), or a combination of hardware and software hosted at hardware. In some implementations, the modules2340,2350, and2360can be implemented using the processor2312and memory2314, e.g., program instructions stored in memory2314and/or other memory or storage devices connected to control system2310.

Mode control module2340can detect when a user initiates a controlling mode and a non-controlling mode of the system, e.g., by user selection of controls, sensing a presence of a user at a master control workstation or master controller, sensing required manipulation of a master controller, etc. The mode control module can set the controlling mode or a non-controlling mode of the control system2310based on one or more control signals C1 to Cx. For example, mode control module2340may activate controlling mode operation if user detection module2330detects that a user is in proper position for use of the master control and that signals (e.g., one or more signals C1 to Cx) indicate the user has contacted the master controller. The mode control module2340may disable controlling mode if no user touch is detected on the master controller and/or if a user is not in proper position for use of the master controller. For example, the mode control module2340can inform control system2310or send information directly to controlling mode module2350to prevent the controlling mode module2350from generating actuation signals A1 to An that move slave device2304.

In some implementations, controlling mode module2350may be used to control a controlling mode of control system2310. Controlling mode module2350can receive control signals C1 to Cx and can generate actuation signals A1 to Ay that control actuators of the slave device2304and cause it to follow the movement of master device2302, e.g., so that the movements of slave device2304correspond to a mapping of the movements of master device2302. Controlling mode module2350can be implemented using conventional techniques.

Controlling mode module2350can also be used to control forces on the master controller of the master device2302as described herein, e.g., forces output on one or more components of the master controller, e.g., grip members, using one or more control signals D1 to Dx output to actuator(s) used to apply forces to the components. For example, one or more of control signals D1 to Dx can be output to one or more actuators configured to output forces to the grip members of the master controller as described herein, and output to one or more other actuators of the master controller, e.g., actuators configured to output forces in a rotary degree of freedom of the controller, actuators configured to output forces on arm links coupled to the master controller, etc. In some examples, control signals D1 to Dx can be used to provide force feedback, gravity compensation, etc.

In some implementations, a non-controlling mode module2360may be used to control a non-controlling mode of system2300. In the non-controlling mode, movement in one or more degrees of freedom of master device2302, or other manipulations of master device2302, has no effect on the movement of one or more components of slave2304. In some examples, non-controlling mode may be used when a portion of slave2304, e.g., a slave arm assembly, is not being controlled by master2302, but rather is floating in space and may be manually moved. For non-controlling mode, non-controlling mode module2360may allow actuator systems in the slave2304to be freewheeling or may generate actuation signals A1 to An, for example, to allow motors in an arm to support the expected weight of the arm against gravity, where brakes in the arm are not engaged and permit manual movement of the arm. For example, in a medical procedure, non-controlling mode may allow a surgical side assistant to easily manipulate and reposition an arm or other slave component relative to a patient or directly make some other clinically appropriate adjustment of the arm or slave component.

In some implementations, non-controlling mode can include one or more other operating modes of the control system2310. For example, a non-controlling mode can be a selection mode in which movement of the master controller in one or more of its degrees of freedom and/or selection of controls of the master controller (e.g., buttons440ofFIG.4) can control selection of displayed options, e.g., in a graphical user interface displayed by display2320and/or other display device. A viewing mode can allow movement of the master controller to control a display provided from cameras, or movement of cameras, that may not be included in the slave device2304. Control signals C1 to Cx can be used by the non-controlling mode module2360to control such elements (e.g., cursor, views, etc.) and control signals D1 to Dx can be determined by the non-controlling mode module to cause output of forces on the master controller during such non-controlling modes, e.g., to indicate to the user interactions or events occurring during such modes.

Some implementations described herein, e.g., method600, can be implemented, at least in part, by computer program instructions or code which can be executed on a computer. For example, the code can be implemented by one or more digital processors (e.g., microprocessors or other processing circuitry). Instructions can be stored on a computer program product including a non-transitory computer readable medium (e.g., storage medium), where the computer readable medium can include a magnetic, optical, electromagnetic, or semiconductor storage medium including semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), flash memory, a rigid magnetic disk, an optical disk, a memory card, a solid-state memory drive, etc. The media may be or be included in a server or other device connected to a network such as the Internet that provides for the downloading of data and executable instructions. Alternatively, implementations can be in hardware (logic gates, etc.), or in a combination of hardware and software. Example hardware can be programmable processors (e.g. Field-Programmable Gate Array (FPGA), Complex Programmable Logic Device), general purpose processors, graphics processors, Application Specific Integrated Circuits (ASICs), and the like.

Note that the functional blocks, operations, features, methods, devices, and systems described in the present disclosure may be integrated or divided into different combinations of systems, devices, and functional blocks as would be known to those skilled in the art.

Although the present implementations have been described in accordance with the examples shown, one of ordinary skill in the art will readily recognize that there can be variations to the implementations and those variations would be within the spirit and scope of the present disclosure. Accordingly, many modifications may be made by one of ordinary skill in the art without departing from the spirit and scope of the appended claims.