Patent Description:
Systems and methods using mechanical compliance to improve robot performance during grasping and manipulation are known. Purpose-built compliant elements exist commercially that function as safety guards, such as, for example, position sensors sold by ABB Automation Technology Products AB of Sweden. These devices may include magnetic breakaway or spring elements that deflect when contact between the robot and the environment is made. Additionally, these designs can include rudimentary on/off sensing of a breakaway state, which is often used as a stop signal to the robot controller.

More modern robotic systems in industry and academia have incorporated flexible elements and deformation sensors in the joints of a robot arm (see for example, the Baxter Robot sold by Rethink Robotics, Inc. of Boston, Massachusetts and the DLR Lightweight Robot III developed by the Institute of Robotics and Mechanics at German Aerospace Center in Germany). Through the combined sensing of deformation at each j oint, an approximation of the force at the end-effector may be deduced. Such an implementation is undesirable in certain applications however (for example, due to unnecessary added compliance that may degrade the positional accuracy of the end-effector, added mechanical complexity and cost, and decreased payload capabilities of the robotic system), with the added complication that any highly flexible end-effector on the robot arm causes the loads transmitted through to the joints to be fairly small and difficult to reliably measure.

Force sensors are also known to be used in robotic manipulation systems. A typical force sensor consists of a rigid plate instrumented with several micro-scale deformation sensors such as strain gauges. This plate is commonly placed between the robot end-effector and the robot arm, and used to sense forces and torques acting on the end-effector. These sensors tend to be expensive and difficult to calibrate accurately since they measure deflections or strain on very small scales. Furthermore, a force sensor mounted between the end-effector and robot arm suffers from the issue mentioned above for joint-sensors, namely that highly flexible elements on the end-effector will not create significant forces for detection at the force sensor.

There remains a need therefore for an improved sensing system for robotic and other sortation systems.

<CIT> appears to disclose a supple grip which provides compensation for minor misalignment created by positioning errors of a workpiece and a robot arm. The grip has pincers having a jack of parallel displacement between an open and closed position, and having also a sensitive block which is fixed to the grip and is faced with an elastically deformable element and carries a device of measuring the elastic deformation. The deformation is detected by a magnetoresistive transducer placed between two facing ferrite permanent magnets. The transducer provides feedback data to a controller which regulates supply to hydraulic cylinders operating on the pincers.

<CIT> appears to disclose a gripping device comprising a displacement-type force sensor provided on a side of a driving mechanism opposite fingers to which the driving mechanism is connected to form a grip section. The driving mechanism is supported on a housing with an elastic member being disposed therebetween at a position closer to the fingers than the center of gravity of the grip section.

<CIT> appears to disclose a device, particularly used for lifting the windscreen or other window panes of a car during assembly, the device being equipped with several gripping units working with a suction mechanism. Each of the gripping elements is fitted with a stop designed as an axially moving central shaft with a rounded tip surrounded by a helical spring and an elastic rubber case, guided through the main housing accommodating a pneumatically operated clamping mechanism. The lifting level of the shaft is controlled by a sensor attached to the top of the housing in order to adjust the position of the gripping elements to the shape of the pane to be lifted.

<CIT> appears to disclose a handling apparatus for automated or robot-supported contact tasks. The handling apparatus has the following components: a mechanical interface for releasably or fixedly connecting the handling apparatus to a manipulator; a holder, which is movable in relation to the interface, for holding a tool; at least one static-frictionless adjusting element for positioning the holder in relation to the interface to the manipulator; a sensor device for directly or indirectly measuring the force acting on the at least one adjusting element; and a closed-loop controller which is configured to regulate the contact force depending on a predefinable force profile when there is contact between the handling apparatus and a surface.

In accordance with an embodiment, the invention provides a robotic system for grasping and moving objects according to the appended claims <NUM>-<NUM>.

In accordance with a further embodiment, the invention provides a method of grasping and moving objects according to the appended claims <NUM>-<NUM>.

The drawings are shown for illustrative purposed only.

The invention provides in accordance with an embodiment, a novel sensing manipulator that tracks the physical deformation of a robot end-effector as it makes contact with an environment, including an object within the environment. Many robot end-effector designs rely on flexible passively-compliant elements that deform to accommodate the environment. This compliance is used to improve the quality and reliability of contact during grasping and manipulation, and to reduce the impact loads applied to both the robot and objects during contact.

The novel sensing manipulator discussed herein in accordance with certain embodiments tracks these various modes of deformation, and provides this information for use in higher-level automation software to determine significant details about the state of end-effector contact with the environment. This mode of sensing eliminates the need for an additional complex mechanical element traditionally used to sense forces or add compliance to a robot system, while minimally altering the stiffness and inertia of the pre-existing hardware. Placing the sensor as close as possible to the contact site, in accordance with an embodiment, ensures it is able to obtain signals relevant to the manipulation task unaltered by the dynamics of transmission through the robot structure.

In accordance with certain embodiments, sensing manipulators of the present invention may have several primary features with many ancillary benefits, summarized here and discussed in more detail below.

The position deformation sensor design methodology provides A) a sensing strategy that can sense the deformation of a compliant element along multiple axes simultaneously, B) a sensing system that can be applied to a variety of pre-existing compliant elements and eliminates the need for new mechanical complexity along the serial chain of a robot arm, C) a sensor solution that minimally affects the stiffness or inertia of existing compliant elements, and D) a sensor that is placed near the end-effector contact surface to obtain data that is both highly sensitive and is unaltered by the dynamics of force transmission through the robot.

The novel software and algorithms of certain embodiments of the invention further provide A) software strategies that use the sensor information to detect the presence or absence of contact with the world, and B) software strategies that detect the amount of force and torque imparted on the end-effector due to the external load of the object and grasping configuration.

This general approach of deflection sensing and algorithms applied to process the resultant data, is illustrated via several examples as follows. The design and methodology may be understood initially by considering a simplified illustration of the deflection sensor design as shown in <FIG> shows a deformation sensor application diagram in accordance with an embodiment of the present invention, where the deformation sensor is positioned adjacent the environment such that the sensing of the deflection sensor of <FIG> occurs at the point of contact with the environment.

In particular, the robotic system <NUM> includes a movement detection system <NUM> such as a deflection sensor that is provided with a compliant interface <NUM> such as a vacuum cup, for engaging an environment <NUM>. The movement detection system <NUM> and the compliant interface <NUM> are coupled to an end effector <NUM> attached to a robotic mass <NUM> of the robotic system. The compliant interface is formed in a shape of a tubular or conical bellows using a flexible material as shown at <NUM> and 14a in <FIG> respectively. Note that the compliant interface may move in not only a direction as shown at A, but may also move in second directions shown at B (as shown) and D (into and out of the page) that are transverse to the first direction, as well as directions as shown at C that are partially transverse to the first direction. Also note the compliant interface is not necessarily a part of the deflection sensor itself, but may, in certain embodiments, be a natural part of the manipulation system.

The deformation sensor may be applied to systems where the deformation is not tightly constrained but rather provides multi-axis sensing, meaning that deformation may occur linearly, rotationally, or along complex paths. The ability to allow for and sense this complex deformation is a key differentiator from prior art systems. Several technologies can be applied to provide sensors to the compliant interface. It is important that this sensing not restrict or impede the compliant motion, or add significant inertia or mass. Several sensors could be applied to measure the deformation including but not limited to; flex sensors (such as flex-sensitive resistors or capacitive sensors), magnetic field sensors (such as a compass or hall-effect sensors), or potentiometers.

<FIG> shows a sensing manipulator <NUM> not in accordance with the invention wherein the sensing manipulator includes a movement detection system <NUM>. The movement detection system <NUM> includes a static <NUM>-axis magnetic field sensor <NUM> that is aligned against a magnet <NUM> attached to the central part of the compliant cup <NUM> by a ring <NUM>. A vacuum is provided at an open end <NUM> of the complaint cup <NUM>. As the compliant cup <NUM> moves, so too does the ring <NUM>. As the ring <NUM> around the cup moves, so too does a bracket <NUM> as well as a magnet <NUM>, which movement is detected with respect to the magnet sensor <NUM> attached to the articulated arm <NUM> for sensing the axial flexure of the vacuum cup from which translations/roll/pitch/of the cup. When the magnetic field sensor is employed, the system may determine not only movements in the elongated direction (x) of the deflection sensor with respect to the articulated arm, but also movements in directions (y and z) that are transverse to the elongated direction of the deflection sensor as well as directions that are partially transverse to the elongated direction of the deflection sensor.

With reference to <FIG>, in accordance with the present invention, the system includes an articulated arm <NUM> to which is attached an end effector <NUM>, again, which is a tubular or conical shaped bellows. The end effector <NUM> also includes a sensor <NUM> that includes an attachment band <NUM> on the bellows, as well as a bracket <NUM> attached to magnetic field sensor <NUM>, and a magnet <NUM> is mounted on the articulated arm <NUM>. As the bellows moves in any of three directions (e.g., toward and away from the articulated arm as shown diagrammatically at A, in directions transverse to the direction A as shown at B, and directions partially transverse to the direction A as shown at C. The magnetic field sensor <NUM> communicates (e.g., wirelessly) with a controller <NUM>, which also communicates with a flow monitor <NUM> to determine whether a high flow grasp of an object is sufficient for continued grasp and transport as discussed further below. In certain embodiment, for example, the system may return the object if the air flow is insufficient to carry the load, or may increase the air flow to safely maintain the load.

<FIG> show an object <NUM> being lifted from a surface <NUM> by the end effector <NUM> that includes the load detection device of <FIG>. Upon engaging the object <NUM>, the system notes the position of the detection device. Once the object <NUM> is lifted (<FIG>), the system notes the change in the sensor output. In this example, the load provided by the object <NUM> is relatively light. <FIG>, however, show the end effector lifting a heavy object.

<FIG> show an object <NUM> being lifted from a surface <NUM> by the end effector <NUM> that includes the load detection device of <FIG>. Upon engaging the object <NUM>, the system notes the position of the detection device. Once the object <NUM> is lifted (<FIG>), the system notes the change in the position of the detection device. As noted above, in this example, the object <NUM> is heavy, presenting a higher load.

The system may also detect whether a load is not sufficiently balanced. <FIG> show an object <NUM> being lifted from a surface <NUM> by the end effector <NUM> that includes the load detection device of <FIG>. Upon engaging the object <NUM>, the system notes the position of the detection device. Once the object <NUM> is lifted (<FIG>), the system notes the change in the position of the detection device. In this example, the object <NUM> presents a non-balanced load. The compliant element may therefore, undergo substantial translational and angular deformation.

Various further platform applications include the following. The deformation sensor concept is designed to integrate with existing passive and active compliant components of a robot end-effector. Not in accordance with the invention, the movement detection system includes force-sensitive resistors. <FIG> and <FIG>, for example, show a sensing manipulator <NUM> together with a vacuum cup <NUM> wherein the movement detection system includes an array (e.g., three) of detectors <NUM> for sensing the axial flexure of the vacuum cup from which translations/roll/pitch/of the cup can be deduced. In particular, the force-sensitive resistors may include a conductive polymer that is printed on a surface, wherein the conductive polymer changes it resistance in a predictable manner when a force is applied to the surface. The sensing manipulator <NUM> may be attached to a robotic arm via a mounting element <NUM> (which couples to a robotic arm mount that passes between two of the detectors <NUM>). A vacuum may be provided at an open end <NUM> of the vacuum cup <NUM> for engaging an object <NUM> (as shown in <FIG>).

Another such alternative compliant element example not in accordance with the invention is the use of a two-fingered robot gripper either at the wrist (as shown in <FIG>) or on the finger tips (as shown in Figures 11A and 11B). Normally compliance is built in at the fingertips or directly behind the wrist of the gripper. A deflection sensor could easily be adapted to accommodate similar alternative designs. In particular, <FIG> shows a sensing manipulator <NUM> that is attached to a robotic arm <NUM>. The sensing manipulator <NUM> includes a compliant section <NUM> and a sensing section <NUM> that includes a two finger gripper end effector <NUM>. As shown at D and E, the sensing section <NUM> may provide sensing of the position and orientation of the end effector <NUM> with respect to the robotic arm <NUM>, e.g., by magnetic or capacitive sensing.

<FIG> shows a sensing manipulator <NUM> that is attached to a robotic arm <NUM>. The sensing manipulator <NUM> includes a gripper <NUM> that includes two jaws <NUM>. On one or both jaws is provided a compliant element <NUM>, and on the compliant element <NUM> is provided a magnet <NUM>. With further reference to <FIG> (which shows an enlarged view of a portion of one jaw <NUM>) a corresponding magnetic sensor <NUM> is provided on the jaw. When the compliant element <NUM> is under a load (as shown by a force as shown at F), the sensor <NUM> will move respect to the sensor <NUM>, providing position and orientation sensing data.

The stiffness and sensitivity of the compliant material are also important considerations. Note from <FIG> that the location of sensing is along the preexisting compliant structure of the robot system. This allows a system using the deformation sensor to maintain it's original stiffness and compliance properties, unlike prior art solutions. Also important to note is the target location for the deformation sensor in the system. The more distal the sensor is the closer it is to the interaction point, where non-linear complicating effects from the robot are less significant.

The software may involve high-level automation software that uses the data output from the deformation to make a series of important decisions as follows.

The most straightforward application of the sensor is thresholding the deformation values from the sensor to detect when contact with the world has occurred. If any axis of deformation moves outside nominal levels, then robot motion can be stopped and appropriate gripping strategy motions may be executed (such as pushing more or less on the environment as needed).

When approaching an object for grasping, a robot arm will often first make contact with the object by pushing into it (either intentionally or unintentionally). Compliance is often used in robotic systems by allowing the end-effector to passively re-adjust to the environment by bending against the contact point. By using the deformation sensor to sense this angle of deflection, and then actively controlling the robot to re-adjust and compensate for the deflection by re-positioning itself, grasps can be made more reliable and centered on the object.

Given a model of how the compliant element deflects under load, the deformation changes may be mapped to forces and torques on the end-effector. This may allow for a number of force-sensing strategies, such as force-guided insertions and grasps, and force-guided placement of objects on surfaces.

Similar to the above two points, after an object is grasped and lifted, gravitational effects will cause the robot end-effector to deflect under the load. Depending on the location of the grasp point with respect to the center-of-mass of the object, this may cause various deformations in the compliant element of the end-effector. Also, a poorly chosen grasp location on a heavy object can induce oscillations between the compliant components and object. The deformation sensor would be capable of sensing both these effects, and may be used to guide the robot to accept or reject grasps and give important information about the direction of the misalignment.

Due to centripetal effects the end-effector is often the most dangerous point on a moving robot arm. During motions where no environmental interaction is expected the deformation sensor can be monitored for changes and the robot stopped when unexpected events occur. The deformation has advantages over the more traditional joint-level or wrist-level safety guards on a robot, since it is designed into the low-inertia low-mass endpoint of the robot, and has the potential to respond before any damage has been done to the robot, environment, or human obstacles.

The deformation sensing strategy presented here provides a framework that allows sensitive high-resolution sensing of contact between a robot and it's environment, while minimally altering the physical attributes of the robot's compliance. Given a model or properly tuned heuristics the sensor may be used to resolve important information for robot decision making to improve manipulation task performance.

Claim 1:
A robotic system for grasping and moving objects, said robotic system comprising:
an articulated arm (<NUM>);
a vacuum end-effector (<NUM>) attached to the articulated arm (<NUM>), the vacuum end effector (<NUM>) including a compliant section coupled to a vacuum source, the compliant section comprising a tubular or conical bellows;
the vacuum end-effector (<NUM>) further including a sensing manipulator that comprises a flow monitor (<NUM>), a movement detection system, and a controller (<NUM>),
the flow monitor (<NUM>) providing information regarding an air flow provided at the compliant section of the vacuum end-effector (<NUM>),
the movement detection system providing information regarding movement of the compliant section of the vacuum end-effector (<NUM>) with respect to the articulated arm (<NUM>) in at least two degrees of freedom, the movement detection system comprising a magnetic field sensor (<NUM>) aligned against a magnet (<NUM>) mounted to the articulated arm (<NUM>), the magnetic field sensor (<NUM>) being attached to a bracket (<NUM>) that is further attached to an attachment band (<NUM>) surrounding a portion of the tubular or conical bellows; and
the controller (<NUM>) communicating with the flow monitor (<NUM>) and the magnetic field sensor (<NUM>) for determining, responsive to both the information regarding the air flow provided at the compliant section of the vacuum end-effector (<NUM>) from the flow monitor (<NUM>) and the information regarding movement of the compliant section of the vacuum end-effector (<NUM>) with respect to the articulated arm (<NUM>) from the movement detection system , whether to increase the air flow provided at the compliant section of the vacuum end-effector (<NUM>) to maintain a grasp of an object by the vacuum end-effector (<NUM>) for transport.