# Tutorial Review on Space Manipulators for Space Debris Mitigation
## 5 Space Manipulator Operations--Evolution from Teleoperation to Autonomy  
All robotic systems employ cameras for visual feedback, and there are several common camera configurations. The camera configuration for a space robot typically comprises at least one mast-mounted binocular camera mounted onto the spacecraft for a global view of the workspace and another camera mounted onto the elbow and/or wrist (eye-in-hand) for close-up observation of manipulation tasks. For example, SRMS mounted one pan-tilt camera at the elbow and another at the end-effector; similarly, SSRMS mounted one pan-tilt camera at each side of the elbow and one pan-tilt camera at each end-effector. Vision-based relative navigation for inspecting uncooperative satellites have been addressed using a stereocamera and three-axis gyroscopes [68]. On-orbit servicing manipulation requires more complex vision processing under variable illumination conditions, however. One plausible assumption on servicing satellite targets is that partial cooperation with attitude control may be available but no visual markers are likely to be discernible [69]. The vision system will have to employ template recognition methods derived from CAD files. However, an unscented Kalman filter can estimate shape and relative attitude, position and velocity of a satellite target by building a 3D map of points from features detected by a monocular camera and LIDAR on the chaser [70]. This suggests that template-matching with a priori data may be dispensed with, but a priori model data does yield superior accuracy [71]. There are multiple transforms between coordinate frames to convert between cameras, pan-tilt units and the end effector [72]. Pose-based visual servoing provides closed-loop feedback control from the error between desired and actual poses in images of the 3D workspace. Pose-based visual servoing is based on the image Jacobian that relates the 3D object pose to the 2D image. The target coordinates (\(x_{t}\),\(y_{t}\),\(z_{t}\)) are related to equivalent camera coordinates (\(x_{c}\),\(y_{c}\),\(z_{c}\)  
Figure 1: Dual manipulator servicer ATLAS (advanced telerobotic actuation system) concept. by a \(3\times 3\) rotation (direction cosine) matrix. A pinhole camera model relates 3D camera coordinates (\(x_{c}y_{c}z_{c}\)) to image projection coordinates (\(x\),\(y\)):  
\[x=\frac{fx_{c}}{f-z_{c}}\text{ and }y=\frac{fy_{c}}{f-z_{c}} \tag{6}\]  
where \(f\) = camera focal length. To compensate for jittering and other perturbations, an extended Kalman filter may be employed using the pin-hole camera model [73]. In image-based visual servoing, only image coordinates are computed. Feature-based servoing is the most common form of visual servoing [74]. Features begin with object image definition--the \"snake\" (active contour model) is a deformable curve that extracts object shape by shrink wrapping around object contours on an image based on energy minimisation [75]. The energy minimum lies on the edges of the object image and requires the computation of gradient masks. Feature matching may be achieved through the principle of maximum entropy. To robustify image-based visual servoing, the extended Kalman filter may be employed to estimate feature motion in images [76]. Kalman filters are suited to visual object tracking [77] and an adaptive extended Kalman filter has been applied visual servoing of non-cooperative satellite targets for manipulator capture [78]. Intensity-based servoing offers robustness without the need for feature extraction or pose estimation [79]. Generalised angle representation which is translation, rotation and scale invariant offers a rapid means of computing shape [80]. Laser scanning offers a means to supplement vision with range-finding uncorrupted by scene illumination [81]. The photonic mixer device (PMD) camera exploits phase shift measurements (rather than intensity) to extract range and generate 3D images making them ideal for spacecraft pose estimation at short range during capture [82].