## 3 Offline Optimization and Adaptation Under Nonlinear Task Constraints
### Optimization and Adaptation Procedure
#### 3.3.2 Adaptation Procedure
It is further converted to the table frame at the tabletop surface. Utilizing our trajectory generalization method (see Section 2.1), the trajectory is generalized with the same start and end points, without via-points, to generate the template trajectory for each skill (see Figure 5b). To account for the inherent noise in the Kinect FORTH system, the trajectories are aligned to the same start and end point, ensuring consistency for subsequent offline task \begin{table}
\begin{tabular}{c l} \hline \hline
**Real-Time Adaptation** & **Description** \\ \hline \(n\left(\xi_{k}\right)\) & The normal vector of the deflection hyperplane at point \(\xi\) \\ \(\mathbf{E}(\xi_{k}),\mathbf{D}(\xi_{k})\) & The basis and the diagonal eigenvalue matrix \\ \(\eta_{k},\rho_{k}\) & The safety magnitude and reactivity parameter for \(\Gamma(\xi_{k})\) \\ \(n\left(\xi_{k}/\eta_{k}\right)^{\gamma}\xi_{k}\) & Tail-effect definition for \(\Gamma(\xi_{k})\) \\ \(M(\xi_{k})\in\mathbb{R}^{3\times 3},w_{k}(\xi_{k})\) & The modulation matrix and weighted coefficient for each dynamic obstacle \\ \(\|Q\left(\xi\right)\|_{\text{all}},\mathbf{q}(\xi)\|_{\text{all}}\) & The weighted magnitude and directions for all dynamic obstacles \\ \(\xi_{\text{init}},\xi_{\text{mod}}\) & The initial and modulated velocity in Cartesian space \\ \(C_{0}\) & The closest point between the robot links and the obstacle \\ \(x_{\text{pr}}^{0}J_{0}\) & The retreat velocity and the Jacobian matrix at point \(C_{0}\) \\ \(\|d_{0}\|\) & The minimum detected distance between robot links 1 to 6 and the obstacle \\ \(d_{m}\) & The safety margin for dynamic obstacles \\ \(w_{1}^{\text{rel}},\xi_{\text{all}}^{\text{rel}}\) & The weight term and weighted relative velocity \\ \(\{\xi_{\text{next}}\},\xi_{\text{next}}\) & The next generalized trajectory sets and the next velocity computed \\ \(d_{n}\) & The distance at which the obstacle begins to affect the manipulator \\ \(v_{n}\) & The scalar velocity \\ \(q_{\text{mod}}\) & The modulated joint configuration \\ \hline \hline \end{tabular}
\end{table} 
Table 1: _Cont._

Figure 5: Template trajectory generalization from only one human demonstration. (**a**) Snapshots of human hand centroid tracking with the Kinect FORTH system. The first, middle, and last rows denote move above, left, and right around the obstacle, respectively. (**b**) The generalized template trajectory for three modes at the minimum Z-position of the robot gripper center with the same start point (symbol ‘o’) and end point (symbol ‘x’). start point is set, and the end point is determined based on the emergency stop 3D positions detected by YOLO [46]. The template trajectory corresponds to the minimum Z-position of the robot gripper center. ### Offline Movement Adaptation

Given the template trajectory, we validate the offline task-constrained adaptation algorithm (Algorithm 1) under a static scene with the presence of a new big bottle, as shown in Figure 6a. We randomly place the emergency stop and the bottle with 3D positions detected by YOLO. For each skill, the robot first generates collision-free end-effector trajectory sets with via-point trajectory generalization (see Section 2.1). We select the feature via-points corresponding to the max norm of shape modulation \(f(\mathbf{p})\) and set discrete via-points with equal spacing to construct the initial trajectory sets. To be specific, for the above skill, the X- and Y-positions are within the radius \(R_{\text{obs}}=39\) mm around the obstacle with Z within \(\pm 100\) mm around the template trajectory via-points; for the left and right skill, we fix X and set the Y-, Z-positions distributed within \(\pm 100\) mm around. Meanwhile, the X-, Y-, and Z-positions are unioned based on the limited Y- and Z-ranges for task execution. We check the collisions and fine-tune the via-points with the threshold \(\gamma=30\) mm; the resultant collision-free trajectory sets are shown in Figure 6b.