Patent Description:
In petrochemical exploration and production, many offshore wells are at depths well beyond the reach of divers. In these instances, a submersible remote operated vehicle (ROV) is controlled from above the water's surface to perform some operations in the construction and control of the wells. The ROV has a manipulator arm that can mount tools for use in performing these operations. Some manipulator arms have the capability to remotely release from and attach to tools, so that different tools can be interchanged while the ROV is subsea.

<CIT> describes a subsea manipulator tool changer that comprises an ROV attachable standardized tool receiver and a tool, the subsea manipulator tool changer comprising a standardized tool latch and a standardized tool latch receiver whereby tools can be attached and detached subsea.

<CIT> describes a multi-joint robot using substantially one sensor, capable of performing a proper repositioning motion of an arm of the robot. The controller has a disturbance torque estimating part which estimates a first disturbance torque and a second disturbance torque, by calculating a torque generated by a mass and motion of the robot and subtracting the calculated torque from the first torque and the second torque detected by a torque detecting part. The controller has a repositioning commanding part which generates a motion command for rotating each axis so that the disturbance torque is reduced, when the disturbance torque exceeds a torque threshold. Since the axis is repositioned based on the motion command, a portion of the robot pushed by the operator is repositioned.

<CIT> describes an underwater vehicle and sea floor station for servicing wells.

<CIT> describes a single-module deployable bolted flange connection apparatus that makes up standard flange joints for various pipeline tie-in situations, such as spool piece connection and flowline-tree connections, without the use of divers and auxiliary multiple pieces of equipment.

Throughout the figures, like reference numbers are used to indicate the like parts.

<FIG> shows an example submersible remote operated vehicle (ROV) <NUM> operating subsea. The ROV <NUM> can be controlled by a human operator from a control interface <NUM>, typically on a vessel <NUM> (e.g., a platform, ship or other vessel) above a surface <NUM> of a body of water, to fly through the water and perform certain operations. The ROV <NUM> of <FIG> includes a manipulator arm <NUM> with a tool <NUM> attached to its end. In certain instances, the ROV <NUM> can include one or more additional arms, such as a grabber or other type of arm, but the manipulator arm <NUM> is the most dexterous, having multiple pivot and rotational j oints <NUM> that enable movement of the arm in multiple degrees of freedom. In certain instances, the joints <NUM> in the manipulator arm <NUM> collectively provide <NUM> degrees of freedom (i.e., movement along the X-axis, Y-axis, Z-axis, roll, pitch, and yaw). Each joint <NUM> includes a mechanical joint that enables movement between the connected segments of the arm <NUM>, one or more actuators to drive movement of the joint and, in certain instances, one or more sensors, such as position and force (linear and/or torque) sensors.

The control interface <NUM> is communicably coupled to the ROV <NUM> submerged in the water. In some cases, the ROV <NUM> is connected to the control interface <NUM> through a tether management system (TMS) <NUM>, also submerged in the water, and supported from the vessel <NUM>. The operator controls the ROV <NUM> to fly around and perform operations and the TMS <NUM>, in performing those operations, via the control interface <NUM>. An umbilical <NUM> extends between the control interface <NUM> at the vessel <NUM> to the TMS <NUM>. The TMS <NUM> pays out and takes up a tether <NUM> that extends between the TMS <NUM> and the ROV <NUM>. The umbilical <NUM> and tether <NUM> communicate power, e.g., electrical power, and data between the control interface <NUM> and the TMS <NUM> and ROV <NUM>. The data communicated on the umbilical <NUM> and tether <NUM> includes control signals to actuators of the TMS <NUM> and ROV <NUM> and other control communications, output from sensors at the TMS <NUM> and ROV <NUM>, and other data.

The ROV <NUM>, in turn, supplies power, e.g., electrical and/or hydraulic power, and exchanges data with the tool <NUM> through the manipulator arm <NUM>, enabling the operator to actuate and operate the tool <NUM> via the control interface <NUM>. The tool <NUM> and ROV <NUM> communicate data including control signals to actuators in the tool <NUM>, output from sensors in the tool <NUM>, and other data, via the manipulator arm <NUM>, which, in turn, can be communicated with the control interface <NUM>.

<FIG> shows the tool <NUM> as a jaws with two parts that can be operated to open and close to grasp and hold objects. But, there are a multitude of different tools that can be used with an ROV, including torque tools, cutters and other tools. A tool interchange <NUM> mounts at the end of the manipulator arm <NUM>, between the manipulator arm <NUM> and the tool <NUM>, becoming the interface between the arm <NUM> and the tool <NUM>. The tool interchange <NUM> enables the ROV <NUM> to change out tools <NUM> while subsea with no outside assistance. While there are many examples of tool interchange <NUM> that could be used herein, co-pending <CIT> and entitled "Submersible Remote Operated Vehicle Tool Interchange," shows an example tool interchange <NUM> that can be used herein.

As discussed in more detail below, the operator can operate the manipulator arm <NUM> to dock into a tool holder of a tool storage unit <NUM>. The operator can then actuate the tool interchange <NUM> to release the tool from the manipulator arm <NUM>, withdraw the manipulator arm <NUM> from the tool holder of the tool storage unit <NUM> and leave the tool <NUM> in the tool storage unit - in other words, stow the tool <NUM>. The operator can then operate the manipulator arm <NUM> to dock in a different tool holder storing a different tool <NUM>, and actuate the tool interchange <NUM> to lock to and establish data and power communications with the different tool <NUM> - in other words, connect to a tool <NUM>. Thereafter, the operator can withdraw the manipulator arm <NUM> from the tool holder and use the different tool <NUM> in performing operations. The tool storage unit <NUM> may be on the ROV <NUM>, on the TMS <NUM>, in both locations and/or elsewhere. The tool storage unit <NUM> may be a fixed tool storage unit (i.e., with one or more tool holders fixed in position) or a changeable type (i.e., with multiple tool holders, each moveable to be selectively presented for connecting to or stowing a tool). In certain instances of the tool storage unit <NUM> being a changeable type, the operator can use the control interface <NUM> to select a particular tool or tool holder from a menu, and the tool storage unit <NUM> will operate to move the tool holders to present the tool holder to allow the manipulator arm <NUM> to connect to or stow a tool. In certain instances, the tool storage unit <NUM> is a carousel type, where the tool holders are arranged on a disk that rotates on its central axis to selectively align the tool holders to be presented.

<FIG> is a perspective view of an example tool holder <NUM> that can be used in the tool storage unit <NUM> described above. The example tool holder <NUM> is shown with a docked manipulator arm, and more specifically, shown receiving a tool <NUM> locked to a tool interchange <NUM>. The remainder of the manipulator arm <NUM> has been omitted for clarity of illustration, but would extend outward from the back of the tool interchange <NUM> (similar to that shown in <FIG>). The tool holder <NUM> includes a housing <NUM>, shown here as a frame, with a face plate <NUM>. The housing <NUM> defines a receptacle <NUM> that receives and holds the tool <NUM>, so that the tool can be stored when not in use.

The face plate <NUM>, better shown in the perspective view of <FIG>, has an opening <NUM> sized to pass the tool <NUM>. A plurality of lead-in ramps <NUM> are positioned surrounding the opening <NUM>. In <FIG>, three lead-in ramps <NUM> are shown, equally distributed around the opening <NUM>, but fewer or additional lead-in ramps <NUM> could be provided. Also, <FIG> shows the lead-in ramps <NUM> formed on a common ring affixed to the front surface of the face plate <NUM>, while <FIG> shows discrete lead-in ramps <NUM>, separately affixed to the front surface of the plate <NUM>. The lead-in ramps <NUM> each have a ramped inward facing surface <NUM>, and the surfaces <NUM> of the lead-in ramps <NUM> cooperate with one another to define a generally conical guide, decreasing in diameter toward the opening <NUM>. Accordingly, the lead-in ramps <NUM> are able to contact the outer periphery of a tool <NUM> received at the mouth of the lead-in ramps <NUM> and guide the tool <NUM> in position and orientation toward and through the opening <NUM> and into the receptacle <NUM> as the tool <NUM> is moved toward and into the receptacle <NUM>. The face plate <NUM> also includes features to lock the tool in place. In certain instances, the features can be provided on the lead-in ramps <NUM> (or the ring at the base of the lead-in ramps <NUM> as in <FIG>) in the form of a twist-lock keyway <NUM> that engages corresponding key <NUM> on the periphery of the tool <NUM>. When the keys <NUM> of the tool <NUM> are received in the keyways <NUM>, and the tool <NUM> rotated (clockwise in <FIG>), the keys <NUM> lock into the keyways <NUM>, eventually bottom out at the end of the keyways <NUM>, and signal to the ROV or operator that the tool <NUM> is fully received, locked and rotationally aligned in the tool holder <NUM>. The key/keyways also axially align and support the tool <NUM> in the tool holder <NUM>. In certain instances, the key/keyways can be reversed, having keys carried on the face plate <NUM> and a keyway on the tool <NUM>.

The face plate <NUM> also has a visual tag <NUM> that includes an alignment fiducial <NUM> and a tool location identification marking <NUM>. In certain instances, the alignment fiducial <NUM> can be embedded within the tool location marking <NUM> as shown, or it can be separate from the tool location identification marking <NUM>. The alignment fiducial <NUM> is in a specified position and orientation relative to the opening <NUM> and receptacle <NUM> of the tool holder <NUM>, so that it can be used (as discussed below) in guiding the manipulator arm to dock. In certain instances, the alignment fiducial <NUM> is of a type for 2D position location. In certain instances, the tool location marking <NUM> can include a human readable marking identifying the tool <NUM> specified to be stored in the associated receptacle <NUM> and/or a machine readable marking, like a barcode, quick response "QR" code, or another type of machine readable marking, identifying the tool location.

A camera <NUM> is carried on the manipulator arm, shown in <FIG> as being mounted directly to the tool interchange <NUM> (that, in turn, is mounted directly to the manipulator arm). In other instances, the camera <NUM> could be mounted directly to the arm itself or to another component carried by the arm to enable the camera <NUM> clear sight of the visual tag <NUM> as the tool <NUM> is being docked. In certain instances, the camera <NUM> can be carried by the tool storage unit or something else (e.g., the TMS or other device carrying the tool storage unit). For example, in instances where the tool storage unit is remote from the ROV, the camera <NUM> might be mounted remote from the ROV.

<FIG> shows the tool <NUM> as it would be positioned when stowed in the tool holder <NUM>, but with the remainder of the tool holder <NUM> omitted for clarity of illustration. When stowed, a male mount <NUM> of the tool <NUM> protrudes through the opening <NUM> of the face plate <NUM>. The male mount <NUM> is received in a corresponding receptacle of the tool interchange <NUM> on the manipulator arm <NUM>, and the tool interchange <NUM> grips the male mount <NUM> to lock the tool <NUM> to the manipulator arm <NUM>. In certain instances, the male mount <NUM> includes alignment features to center the tool interchange <NUM> to the tool <NUM> and rotationally align the tool interchange <NUM> to the tool <NUM>. For example, in certain instances, the male mount <NUM> is generally frusta-conical which centers in a corresponding frusta-conical shape of the tool interchange <NUM> to center the tool <NUM> to the tool interchange <NUM>, and position the tool <NUM> in a specified location relative to the manipulator arm <NUM>. Rotational alignment, for example, can be achieved by a key and keyway on the male mount <NUM> and tool interchange, in certain instances.

The concepts herein encompass a system for control of the TMS <NUM> and ROV <NUM> (including the manipulator arm <NUM> and tool <NUM>) by computer, having a processor <NUM> with memory <NUM>, that receives input from sensors <NUM> of the TMS and/or the ROV and signals actuators <NUM> that operate the TMS <NUM> and/or the ROV <NUM>. In particular, the memory <NUM> stores instructions that cause the processor <NUM> to perform the operations described herein. The actuators <NUM> include actuators at each of the joints (e.g., joints <NUM>, <FIG>) that move the joints in moving the manipulator arm, as well as other actuators. The sensors <NUM> include one or more position sensors, force sensors (linear and/or torque), the camera (e.g., camera <NUM>), as well as other sensors. In certain instances, the sensors include a sensor in the manipulator arm, near the tool interface, capable of sensing force and/or torques in <NUM> degrees of freedom. In certain instances, the sensors include a force (liner and/or torque), pressure and/or position sensor at each of the joints (e.g., joints <NUM>) of the manipulator arm. The processor <NUM> could be a single processor or multiple processors in communication with each other, and the memory <NUM> could be a single memory or multiple memories in communication with each other. <FIG> schematically shows an example of the control interface <NUM>, and for convenience of reference, processor <NUM> and memory <NUM> are depicted within a housing <NUM> of the interface <NUM>. Although shown as all within the housing <NUM>, processor <NUM> could be distributed, with aspects remote from the housing <NUM>. In certain instances, all or some of the memory <NUM> could be embedded in a processor. Likewise, although the memory <NUM> is depicted as one memory, it could be multiple memories all within the housing <NUM> or with one or more of the memories distributed, remote from the housing <NUM>. In certain instances, the processor <NUM> is a system of processors distributed in the manipulator arm, ROV and/or housing <NUM> each with associated (embedded or separate) memory. The processor <NUM> and memory <NUM> are in communication with the sensors of the system.

The control interface <NUM> includes a display <NUM> and a user input <NUM> through which the human operator interfaces with the control interface <NUM>. The display <NUM> can include one or multiple screens, goggles and/or other types of displays. The user input <NUM> can include one or multiple types of user input, such as keyboards, hand controllers, physical buttons and switches and/or other types of user inputs. The control interface <NUM> can present the human operator with information about the operation of the ROV, TMS and the environment, as well as menus of options for controlling the system, such as soft menus <NUM> as depicted in <FIG> displayed via the display <NUM>.

In operation, the human operator commands operation of the ROV <NUM>, including the manipulator arm <NUM> and any connected tool <NUM>, and the TMS <NUM> via the control interface <NUM>. Beyond controlling the ROV <NUM> to fly around and controlling the manipulator arm <NUM> and tool <NUM> in performing operations, the operator can effectuate docking the arm <NUM> to a tool holder, such as tool holder <NUM> (<FIG>), of a tool storage unit <NUM>. The system described herein automates the docking, which can be activated via the control interface <NUM> (e.g., via a menu <NUM>, the input <NUM> and/or other manner).

<FIG> is a flow chart of example method steps in automated docking of the manipulator arm (with or without an attached tool) to a tool holder. In certain instances, before the automation begins, the operator operates the arm to an initial or ready position in proximity to the tool holder. For example, when the tool holder is apart from the ROV, the operator flys the ROV to the tool holder and operates the arm to a position near the tool holder. When the tool holder is on the ROV, the operator positions the arm so it is free of external obstructions and able to move to the tool holder. In response to a command to activate the automated docking, the automated docking sequence is initiated, at operation <NUM>, by determining a nominal path between the current or ready position and orientation of the arm and a docked position and orientation with the arm docked to the tool holder. Throughout operation of the arm, both before and after activation of the automated docking, the system receives and logs input from sensors in the arm and thus knows the current position and orientation of the tool, the joints, and the arm, overall, in <NUM> degrees of freedom. When the tool storage unit is carried on the ROV, the system likewise knows the position and orientation of the tool holder in <NUM> degrees of freedom and can, in turn, automatically (i.e., without human assistance) calculate the nominal path of the arm to dock to the tool holder. In calculating the nominal path, the system operates a kinematic model of the manipulator arm and uses this model to calculate the movement of the joints necessary to move the tool into proximity of the opening in the tool holder and to dock with the tool holder, i.e., position and orient the arm so the arm can be connected to a tool protruding from the opening or a tool on the arm inserted into the receptacle of the tool holder. When the tool storage unit is carried apart from the ROV, such as on the TMS, the operator flies the ROV to the tool holder and operates the arm into proximity to the tool holder so the camera can see the tool holder. Once the camera is able to see the tool holder, and the system recognizes the position and orientation of the tool holder, the system can automatically calculate and adjusts the nominal path of the manipulator arm from its current position and orientation to the position and orientation corresponding to the arm docked with the tool holder. In certain instances, the system identifies the alignment fiducial on the tool holder and uses the alignment fiducial in recognizing the tool holder position and orientation.

At operation <NUM>, the system implements movement of the manipulator arm along the nominal path by signaling the actuators at the joints of the manipulator arm to move according to the determined nominal path. In certain instances, the movement can be fully automatic, with no input from the human operator. In other instances, the movement can be controlled to some degree by the human operator. For example, in certain instances, the human operator, using the control interface, can control the speed and start/stop the movement of the arm while the arm is automatically guided along the nominal path.

At operation <NUM>, the system can implement one or more control loops taking feedback from sensors in iteratively controlling the path of the arm (with or without tool) from the nominal path to account for discrepancies between the nominal path and the actual path needed to dock the arm to the tool holder (operation <NUM>). In certain instances, operation <NUM> can be implemented using images (still or video) from the camera, and in certain instances other sensors (e.g. position sensors at the joints and/or other sensors), as feedback to iteratively make corrections to the manipulator arm path. This control loop is discussed in more detail with respect to <FIG>. In certain instances, operation <NUM> can be implemented using output from the one or more force sensors of the manipulator arm that register forces exerted by the arm on the environment, and in certain instances other sensors (e.g., position sensors at the joints and/or other sensors), as feedback to iteratively make corrections to the manipulator arm path. This force accommodation control loop is discussed in more detail with respect to <FIG>. In certain instances, both the image feedback and force accommodation can be used in nested control loops and/or the control corrections of each relatively weighted (e.g., with the image feedback having greater or lesser influence on the determined corrections than the force sensor feedback) in iteratively correcting the manipulator arm path.

Referring to <FIG>, in operation of the image feedback loop, the camera takes an image of the tool holder at operation <NUM>. At operation <NUM>, the image is analyzed to identify the position and orientation of the tool holder. In certain instances, the image is analyzed to identify the position and orientation of the tool holder using alignment fiducial. Thereafter, a pose estimation calculation is performed. The image of the tool holder, and in certain instances the alignment fiducial, is compared to a specified location relative to the camera to calculate the position and orientation of the tool holder to the camera. At operation <NUM>, the system, in turn, determines the position and orientation of the tool holder relative to the manipulator arm, based on a specified relationship between characteristics in tool holder image, e.g., the alignment fiducial and/or other characteristics, and a specified relationship between the camera and the manipulator arm. At operation <NUM>, the system calculates whether the manipulator arm is positioned and oriented to successfully dock based on the current path. If not, it calculates corrections from the current path to a new specified path that (at least based on the current data) positions and orients the arm to successfully dock to the tool holder, and sends new signals to the actuators at the joints to effectuate the new specified positions and orientations. In certain instances, the system can additionally or alternately operate other aspects of the ROV, such as the thrusters used to navigate the ROV in the water, to adjust the position and orientation of the arm to successfully dock to the tool holder. Operating the thrusters to reposition the ROV, for example, may be helpful in instances where the tool storage unit is separate from the ROV (e.g., on the TMS or elsewhere). Finally, at operation <NUM>, the manipulator arm moves to the new specified positions and orientations. The feedback loop begins again at operation <NUM> and repeats, updating the manipulator path position and orientation, until the manipulator arm is successfully docked on the tool holder (e.g., as shown in <FIG>).

Referring to <FIG>, in operation of the force accommodation loop, the one or more force sensors measure forces (linear and/or torque) exerted by the manipulator arm on the environment at operation <NUM>. For example, when the manipulator arm or tool contacts one or more of the lead-in ramps of the tool holder, the force sensors measure the resultant forces in the <NUM> degrees of freedom.

Based on this sensor input, the system determines whether the force in any of the <NUM> degrees exceeds a specified threshold. If it is determined that the measured force in any of the <NUM> degrees exceeds the corresponding specified threshold, at operation <NUM>, the system calculates movements of the manipulator arm (in <NUM> degrees of freedom) to reduce the force below the exceeded specified threshold or thresholds and signals the actuators at the joints to effectuate the movement. In certain instances, there can be different specified thresholds for some or all of the <NUM> degrees of freedom. For example, as the manipulator arm is extended forward, toward the tool holder and tool holder receptacle, the ramped surfaces of the lead-in ramps drive the arm laterally to orient and center the arm on the opening in the tool holder and the tool receptacle. Thus, by having a lower specified threshold in the degree or degrees of movement that correspond to these imposed lateral forces than the specified threshold in the degree of movement corresponding to the forward extension direction, the system accommodates the mechanical alignment imposed by the lead-in ramps. In certain instances, if the forward extension corresponds to the Y-axis, the specified threshold for Y-axis is greater than the specified threshold for the X-axis, Z-axis, pitch and yaw. In certain instances, the force accommodation can be implemented as a stiffness control, an impedance control, an admittance control or another type of control. For example, in certain instances the force control can be a hybrid position/force control that prioritizes force control for specified degrees of freedom over position control for those degrees of freedom and vice versa. Moreover, the position control, including control to a nominal path and/or image feedback discussed above, can be operated as either an inner control loop to the force accommodation control loop or operated as an outer control loop to the force accommodation control loop, where the inner control loop would have a faster loop rate and have priority over the outer control loop. For example, in certain instances, an admittance control force accommodation loop has position feedback loop as an inner loop. In another example, in certain instances, an impedance control force accommodation is the inner loop with position feedback as the outer loop.

At operation <NUM>, the manipulator arm moves per the movements signaled in operation <NUM>. The feedback loop begins again at operation <NUM> and repeats, updating the manipulator path position and orientation, until the manipulator arm is successfully docked on the tool holder (e.g., as shown in <FIG>).

Referring back to <FIG>, once the manipulator arm is docked, at operation <NUM>, in certain instances, if the manipulator arm is without a tool, the tool interface can be manually (e.g., by the human operator) or automatically actuated to connect to a tool in the tool holder. In certain instances, if the manipulator arm is connected to a tool, the tool interface can be manually or automatically actuated to release and stow the tool in the tool holder.

Thereafter, the manipulator arm can be withdrawn and moved back to the (same or different) ready position, for example, by automatically determining a nominal path to the ready position at operation <NUM>. At operation <NUM>, the system implements movement of the manipulator arm along the nominal path by signaling the actuators at the joints of the manipulator arm to move according to the determined nominal path. As discussed above, in certain instances, the movement can be fully automatic, with no input from the human operator. In other instances, the movement can be controlled to some degree by the human operator. At operation <NUM>, as discussed above (operation <NUM>), the system can implement one or more control loops taking feedback from sensors in iteratively controlling the path of the arm (with or without tool) from the nominal path to account for discrepancies between the nominal path and the actual path needed to position the arm in the ready position. Finally, the process is complete at operation <NUM>, when the arm has been moved to the ready position.

The process of <FIG> can be performed again to dock to a different tool holder and acquire/stow a different tool.

In certain instance, automating docking of the manipulator arm to the tool holder can speed up the process of changing or stowing tools. For example, wholly manually controlling the manipulator arm to dock to the tool holder is a difficult process, requiring a high level of operator skill and takes even a highly skilled ROV operator tens of minutes to complete. In certain instances, automated docking can be completed within a minute or a few minutes. The time saved translates directly to costs saved as, not only is the ROV's work performed more quickly, but other work need not wait as long on the ROV's work. Automated docking of the arm to the tool in the holder can also reduce the required operator skill level.

Claim 1:
A method, comprising:
receiving data from a submersible remote operated vehicle, i.e., "ROV" (<NUM>) about the operation of an arm (<NUM>) of the ROV; and
automatically controlling, based on the data, movement of the arm relative to a target;
where receiving data comprises receiving data from a camera (<NUM>), the data comprising an image of an alignment fiducial (<NUM>) associated with the target; and
where automatically controlling movement of the arm comprises automatically controlling the movement of the arm to align the arm relative to the target.