# Hands-On Deformation of Volumetric Anatomical Images on a Touchscreen
## 5 Methods: Direct Handle Manipulation
### Optimal Linearized Manipulation  
For smooth and intuitive manipulation, we visualize the transformed handle continuously as the user moves fingers on the screen. After each finger motion, we compute the optimal transformation \(\{\mathbf{t},\mathbf{R}\}\), and we update the 3D handle points as \(\mathbf{q}_{j}\leftarrow\mathbf{R}\,\mathbf{q}_{j}+\mathbf{t}\). In this way, each incremental finger motion results in a small rigid transformation.  
The optimization (3) constitutes a nonlinear least-squares problem. In previous work [9], this problem was solved using an iterative Levenberg-Marquardt solver. However, under the assumption of small transformations, we can safely linearize the error terms (2), and solve a linear least-squares problem instead. This amounts to performing one Gauss-Newton iteration of the original nonlinear problem.  
Two nonlinear functions must be linearized: the rotation and the perspective projection. The gradient of a projected vector can be derived from (4) as:  
\[\frac{\partial(\text{Proj}(\mathbf{v}))}{\partial\mathbf{v}}=\frac{1}{\mathbf{ v}^{T}\,\mathbf{k}}\,\mathbf{P}\,\bigg{(}\mathbf{I}-\frac{\mathbf{v}\,\mathbf{k}^{T} }{\mathbf{v}^{T}\,\mathbf{k}}\bigg{)}. \tag{5}\]  
To linearize the rotation, it is convenient to express it using an axis-angle representation \(\mathbf{r}\), where the magnitude \(\|\mathbf{r}\|\) represents a rotation angle, and the direction \(\frac{\mathbf{r}}{\|\mathbf{r}\|}\) represents the axis of rotation. Then, under small rotations, a rotation operation can be linearly approximated as  
\[\mathbf{R}\,\mathbf{q}\approx(\mathbf{I}+\text{Skew}(\mathbf{r}))\,\mathbf{q }=\mathbf{q}-\text{Skew}(\mathbf{q})\,\mathbf{r}, \tag{6}\]  
where Skew\((\cdot)\) denotes a skew-symmetric matrix representing the cross-product linear transformation.  
The gradient of a rotated vector can easily be derived from this linear approximation, resulting in:  
\[\frac{\partial(\mathbf{R}\,\mathbf{q})}{\partial\mathbf{r}}=-\text{Skew}( \mathbf{q}). \tag{7}\]  
After linearizing the projection and the rotation, the gradients of error terms (2) w.r.t. the translation and the rotation can be easily derived, resulting in:  
\[\frac{\partial\mathbf{f}_{j}}{\partial\mathbf{t}} =\frac{1}{\mathbf{q}_{j}^{T}\,\mathbf{k}}\,\mathbf{P}\,\bigg{(} \mathbf{I}-\frac{\mathbf{q}_{j}\,\mathbf{k}^{T}}{\mathbf{q}_{j}^{T}\,\mathbf{ k}}\bigg{)}, \tag{8}\] \[\frac{\partial\mathbf{f}_{j}}{\partial\mathbf{r}} =\frac{1}{\mathbf{q}_{j}^{T}\,\mathbf{k}}\,\mathbf{P}\,\bigg{(} \frac{\mathbf{q}_{j}\,\mathbf{k}^{T}}{\mathbf{q}_{j}^{T}\,\mathbf{k}}- \mathbf{I}\bigg{)}\,\text{Skew}\big{(}\mathbf{q}_{j}\big{)}. \tag{9}\]  
With these ingredients, we solve the linearized least-squares problem (3) as a small linear system. Grouping the unknown translation and rotation as \(\mathbf{x}=\left(\begin{array}{c}\mathbf{t}\\ \mathbf{r}\end{array}\right)\), the resulting linear system can be written as:  
\[\mathbf{A}\,\mathbf{x}=\mathbf{b}, \tag{10}\] \[\text{with }\mathbf{A}=\sum_{i}\left(\frac{\partial\mathbf{f}_{i}}{ \partial\mathbf{t}}\,\,\,\frac{\partial\mathbf{r}_{i}}{\partial\mathbf{t}} \right)^{T}\left(\frac{\partial\mathbf{f}_{i}}{\partial\mathbf{t}}\,\,\,\frac{ \partial\mathbf{r}_{i}}{\partial\mathbf{t}}\right)\] (11) \[\text{and }\mathbf{b}=-\sum_{i}\left(\frac{\partial\mathbf{f}_{i}}{ \partial\mathbf{t}}\,\,\,\frac{\partial\mathbf{r}_{i}}{\partial\mathbf{t}} \right)^{T}(\text{Proj}(\mathbf{q}_{j})-\mathbf{p}_{i}). \tag{12}\]