Repository: napari/napari

Issue:
Affine matrix composition order 
## 🐛 Bug

As per my understanding affine matrix composition order [here](https://github.com/napari/napari/blob/master/napari/utils/transforms/transform_utils.py#L95) should be `full_rotate @ full_shear @full_scale`
here are few links to confirming my theory 
 https://people.cs.clemson.edu/~dhouse/courses/401/notes/affines-matrices.pdf and this https://gamedev.stackexchange.com/questions/29260/transform-matrix-multiplication-order

## To Reproduce
download two LLSM sample files from [here](https://drive.google.com/drive/folders/1MwpgwzLBYI7L2Q4880CKUETcjgjBE0Pb?usp=sharing) `test2.tif` is non-deskewed file and `test2_deskew.tif` is a deskewed version of the `test2.tif`, Add these two images to napari and apply affine transform on a non-deskewed image so that both overlaps.
something like 
```
import numpy as np
import napari
from napari.utils.transforms import Affine, shear_matrix_from_angle
import tifffile

angle = 31.8
dx = 0.104 # um
dz = 0.4 # step size um


shear = shear_matrix_from_angle(31.8)
scale = [np.sin(np.deg2rad(angle))* dz, dx, dx]


with napari.gui_qt():
    viewer = napari.Viewer(ndisplay=3)

    image2_deskew = tifffile.imread("./test2_deskew.tif")
    image2 = tifffile.imread("./test2.tif")
    viewer.add_image(image2, name='shear 2', colormap='bop orange', 
                         scale=scale, shear=shear, blending="additive") 
    viewer.add_image(image2_deskew, name='deskewed 2', colormap='bop purple', 
                    scale=scale, blending="additive")
```

we can also verify the above bug by constructing the affine matrix from scratch something like
```
angle = 31.8
dx = 0.104 # um
dz = 0.4 # step size um

shear = np.array([[ 1.,  0., 0.,  0.],
                [ 0.,  1.,  0.,  0.],
                [ np.tan(np.deg2rad(90-angle)),  0,  1.,  0.],
                [ 0.,  0.,  0.,  1.]])

scale = np.array([[ np.sin(np.deg2rad(angle))* dz,  0., 0.,  0.],
                [ 0.,  dx,  0.,  0.],
                [ 0.,  0,  dx,  0.],
                [ 0.,  0.,  0.,  1.]])


with napari.gui_qt():
    viewer = napari.Viewer(ndisplay=3)

    image2_deskew = tifffile.imread("./test2_deskew.tif")
    image2 = tifffile.imread("./test2.tif")

    affine = shear @ scale
    viewer.add_image(image2, name='shear 2', colormap='bop orange', 
                         affine=affine, blending="additive") 
    viewer.add_image(image2_deskew, name='deskewed 2', colormap='bop purple', 
                    scale=[np.sin(np.deg2rad(angle))* dz, dx, dx], blending="additive")
```

output looks like
![failed](https://user-images.githubusercontent.com/17413/98301837-88ffdd00-200f-11eb-86be-83ce3f9995a0.png)


## Expected behavior
I am expecting 
![success](https://user-images.githubusercontent.com/17413/98301864-95843580-200f-11eb-9ed1-1355645a4c59.png)


## Environment

napari: 0.4.1.dev15+gbc3298b
Platform: Windows-10-10.0.19041-SP0
Python: 3.8.1 (default, Mar 2 2020, 13:06:26) [MSC v.1916 64 bit (AMD64)]
Qt: 5.12.9
PySide2: 5.13.2
NumPy: 1.18.1
SciPy: 1.5.3
Dask: 2.30.0
VisPy: 0.6.5

GL version: 4.6.0 NVIDIA 457.09
MAX_TEXTURE_SIZE: 32768

Plugins:
- napari_plugin_engine: 0.1.8
- svg: 0.1.4


Affine matrix composition order 
## 🐛 Bug

As per my understanding affine matrix composition order [here](https://github.com/napari/napari/blob/master/napari/utils/transforms/transform_utils.py#L95) should be `full_rotate @ full_shear @full_scale`
here are few links to confirming my theory 
 https://people.cs.clemson.edu/~dhouse/courses/401/notes/affines-matrices.pdf and this https://gamedev.stackexchange.com/questions/29260/transform-matrix-multiplication-order

## To Reproduce
download two LLSM sample files from [here](https://drive.google.com/drive/folders/1MwpgwzLBYI7L2Q4880CKUETcjgjBE0Pb?usp=sharing) `test2.tif` is non-deskewed file and `test2_deskew.tif` is a deskewed version of the `test2.tif`, Add these two images to napari and apply affine transform on a non-deskewed image so that both overlaps.
something like 
```
import numpy as np
import napari
from napari.utils.transforms import Affine, shear_matrix_from_angle
import tifffile

angle = 31.8
dx = 0.104 # um
dz = 0.4 # step size um


shear = shear_matrix_from_angle(31.8)
scale = [np.sin(np.deg2rad(angle))* dz, dx, dx]


with napari.gui_qt():
    viewer = napari.Viewer(ndisplay=3)

    image2_deskew = tifffile.imread("./test2_deskew.tif")
    image2 = tifffile.imread("./test2.tif")
    viewer.add_image(image2, name='shear 2', colormap='bop orange', 
                         scale=scale, shear=shear, blending="additive") 
    viewer.add_image(image2_deskew, name='deskewed 2', colormap='bop purple', 
                    scale=scale, blending="additive")
```

we can also verify the above bug by constructing the affine matrix from scratch something like
```
angle = 31.8
dx = 0.104 # um
dz = 0.4 # step size um

shear = np.array([[ 1.,  0., 0.,  0.],
                [ 0.,  1.,  0.,  0.],
                [ np.tan(np.deg2rad(90-angle)),  0,  1.,  0.],
                [ 0.,  0.,  0.,  1.]])

scale = np.array([[ np.sin(np.deg2rad(angle))* dz,  0., 0.,  0.],
                [ 0.,  dx,  0.,  0.],
                [ 0.,  0,  dx,  0.],
                [ 0.,  0.,  0.,  1.]])


with napari.gui_qt():
    viewer = napari.Viewer(ndisplay=3)

    image2_deskew = tifffile.imread("./test2_deskew.tif")
    image2 = tifffile.imread("./test2.tif")

    affine = shear @ scale
    viewer.add_image(image2, name='shear 2', colormap='bop orange', 
                         affine=affine, blending="additive") 
    viewer.add_image(image2_deskew, name='deskewed 2', colormap='bop purple', 
                    scale=[np.sin(np.deg2rad(angle))* dz, dx, dx], blending="additive")
```

output looks like
![failed](https://user-images.githubusercontent.com/17413/98301837-88ffdd00-200f-11eb-86be-83ce3f9995a0.png)


## Expected behavior
I am expecting 
![success](https://user-images.githubusercontent.com/17413/98301864-95843580-200f-11eb-9ed1-1355645a4c59.png)


## Environment

napari: 0.4.1.dev15+gbc3298b
Platform: Windows-10-10.0.19041-SP0
Python: 3.8.1 (default, Mar 2 2020, 13:06:26) [MSC v.1916 64 bit (AMD64)]
Qt: 5.12.9
PySide2: 5.13.2
NumPy: 1.18.1
SciPy: 1.5.3
Dask: 2.30.0
VisPy: 0.6.5

GL version: 4.6.0 NVIDIA 457.09
MAX_TEXTURE_SIZE: 32768

Plugins:
- napari_plugin_engine: 0.1.8
- svg: 0.1.4




Discussion:
> As per my understanding affine matrix composition order here should be `full_rotate @ full_shear @full_scale`

Hmm, I see the recommendation is to scale first and then shear/ rotate whereas we're doing `full_rotate @ full_scale @full_shear` right now, which is a little odd.

I guess scaling first does make sense as otherwise you'd have non-uniform scaling. Curious if @tlambert03 has thoughts or a recommendation here?

I'd be happy to change the composition order, but then we also need to change something in our decomposition code as those always need to match each other. I'll take a look now. Thanks for raising.
> As per my understanding affine matrix composition order here should be `full_rotate @ full_shear @full_scale`

Hmm, I see the recommendation is to scale first and then shear/ rotate whereas we're doing `full_rotate @ full_scale @full_shear` right now, which is a little odd.

I guess scaling first does make sense as otherwise you'd have non-uniform scaling. Curious if @tlambert03 has thoughts or a recommendation here?

I'd be happy to change the composition order, but then we also need to change something in our decomposition code as those always need to match each other. I'll take a look now. Thanks for raising.

Resolving patch:
diff --git a/napari/utils/transforms/transform_utils.py b/napari/utils/transforms/transform_utils.py
--- a/napari/utils/transforms/transform_utils.py
+++ b/napari/utils/transforms/transform_utils.py
@@ -93,7 +93,7 @@ def compose_linear_matrix(rotate, scale, shear) -> np.array:
     full_scale = embed_in_identity_matrix(scale_mat, ndim)
     full_rotate = embed_in_identity_matrix(rotate_mat, ndim)
     full_shear = embed_in_identity_matrix(shear_mat, ndim)
-    return full_rotate @ full_scale @ full_shear
+    return full_rotate @ full_shear @ full_scale
 
 
 def expand_upper_triangular(vector):
@@ -194,13 +194,15 @@ def decompose_linear_matrix(
         tri = upper_tri.T
 
     scale = np.diag(tri).copy()
-    tri_normalized = tri / scale[:, np.newaxis]
 
+    # Take any reflection into account
     if np.linalg.det(rotate) < 0:
         scale[0] *= -1
         tri[0] *= -1
         rotate = matrix @ np.linalg.inv(tri)
 
+    tri_normalized = tri @ np.linalg.inv(np.diag(scale))
+
     if upper_triangular:
         shear = tri_normalized[np.triu(np.ones((n, n)), 1).astype(bool)]
     else:


Test patch:
diff --git a/napari/utils/transforms/_tests/test_transform_utils.py b/napari/utils/transforms/_tests/test_transform_utils.py
--- a/napari/utils/transforms/_tests/test_transform_utils.py
+++ b/napari/utils/transforms/_tests/test_transform_utils.py
@@ -39,6 +39,17 @@ def test_decompose_linear_matrix(upper_triangular):
     np.testing.assert_almost_equal(B, C)
 
 
+def test_composition_order():
+    """Test composition order."""
+    # Order is rotate, shear, scale
+    rotate = np.array([[0, -1], [1, 0]])
+    shear = np.array([[1, 3], [0, 1]])
+    scale = [2, 5]
+
+    matrix = compose_linear_matrix(rotate, scale, shear)
+    np.testing.assert_almost_equal(matrix, rotate @ shear @ np.diag(scale))
+
+
 def test_shear_matrix_from_angle():
     """Test creating a shear matrix from an angle."""
     matrix = shear_matrix_from_angle(35)