Repository: scipy/scipy

Issue:
add procrustes?
The problem is to find a transformation of n p-dimensional points `X_i`, maybe allowing translation/rotation/scaling, so that the transformed points closely match up with n corresponding target points `Y_i` with mismatch penalized by maybe sum of squares of errors.  I've written code for this at one time, downstream projects e.g. `skbio.math.stats.spatial.procrustes` also have implementations, matlab has it, web forums have questions about how to do this in numpy/scipy, and it would seem to be in scope to be added to scipy.
http://en.wikipedia.org/wiki/Orthogonal_Procrustes_problem



Discussion:
I think this would be nice to add in to scipy. I haven't contributed to any scipy enhancements before, but I'd like to try this one/help out with it, if the decision is made to add in this feature.

If there's interest to add this to scipy, we would be willing to contribute the implementation in [`skbio.math.stats.spatia.procrustes`](https://github.com/biocore/scikit-bio/blob/master/skbio/math/stats/spatial.py).

cc @gregcaporaso, @jairideout, @ebolyen, @antgonza, @rob-knight, @wdwvt1 

Sounds good to me, if there is nothing there to do that. :+1:

Looks good to me. I see `skbio` puts it in `spatial`, but it looks like in scipy it would belong in `linalg` right?

Could you bring it up on the mailing list? That's where decisions on including new functionality are made.

~~Sounds good I am waiting for the e-mail confirmation from the scipy-dev list, once I get that I'll reference this issue there.~~

I think it would be better to return the orthogonal transformation matrix omega. In case of a solver for the orthogonal Procrustes problem in scipy.linalg it should be the only result that is returned.

Procrustes analysis is regarded a spatial statistics problem. But solving the orthogonal Procrustes problem is a linear algebra problem. I think a low level solver for the orthogonal Procrustes problem would fit best in scipy.linalg, but a high level Procrustes analysis (with full statistical tests) could go in scipy.spatial.

Also, don't use np.transpose(a) in the code. Using a.T is preferable, and np.dot is smart enough to handle C and Fortran arrays correctly with transpose flags in BLAS. 

In SciPy, code that uses scipy.linalg.svd should preferably use Fortran arrays consistently to avoid f2py overhead. So the procrustes solver would typically start with calling np.asfortranarray on its inputs. This also allows the input to be "array-like". 

@sturlamolden Maybe a `scipy.linalg.orthogonal_procrustes(A, B)` function which uses the Wikipedia notation A, B, and R, and which is implemented similarly to the two lines

``` python
u, s, vh = np.linalg.svd(np.dot(np.transpose(mtx1), mtx2))
q = np.dot(np.transpose(vh), np.transpose(u))
```

in the scikit-bio helper function?

scipy.linalg.procrustes would be a shorter name.

I would also suggest thet we use scipy.linalg.svd instead of numpy.linalg.svd in SciPy. 

The main thing would be that a function in scipy.linalg.\* should solve the linear algebra problem, and nothing else. Cf. that scipy.linalg.lstsq solves the OLS problem, but does not actually do linear regression analysis. 

scipy.spatial.procrustes could do full Procrustes analysis and call scipy.linalg.procrustes as a utility function.

Added a PR https://github.com/scipy/scipy/pull/3809 for the orthogonal part that is purely linear algebra, without translation or scaling.  This follows the API suggested by @sturlamolden, but with the longer function name.

As I was adding that PR, I noticed small inefficiencies in the scikit-bio procrustes code which I thought I'd write down while I still remember.  No extra dimension is needed for reflection.  If you remove the extra-dimension-related code you will find that the 2d example in the docstring, which requires reflection, will still work.  Also, code like `np.sqrt(np.trace(np.dot(result, np.transpose(result))))` can be replaced by `norm(result)`.

@sturlamolden, @argriffing should I wait for #3809 to be merged and then work on porting over the procrustes code with the improvements that @argriffing has pointed out thus far? Otherwise what would be the best way to do this?

I tried setting up scikit-bio for development but its tests keep failing and I'm not patient enough to finish fixing it, so I probably won't make a scikit-bio PR for those improvements soon.  Currently I think I have an incompatible version of "mpl_toolkits."

@argriffing, what issues were you having with scikit-bio install? pip install works for us on linux and mac (though we don't currently support windows, so we expect issues there). 

> what issues were you having with scikit-bio install?

When I run `nosetests --with-doctest skbio` from my home directory as suggested on http://scikit-bio.org/ I get the error

```
[...]/site-packages/mpl_toolkits/mplot3d/axes3d.py", line 1040, in cla
        self.zaxis._set_scale('linear')
    AttributeError: 'ZAxis' object has no attribute '_set_scale'
```

I assume this is because I screwed up the installation or in trying to set up a development environment or because I have an obsolete version of some package.  This is on Ubuntu with Python 2.7.  If I find myself motivated to try to fix it then if I have any questions I'll be sure to ask on a scikit-bio forum!

Two skbio PRs are submitted!

Thanks!

On (Jul-22-14|18:15), argriffing wrote:

> Two skbio PRs are submitted!
> 
> ---
> 
> Reply to this email directly or view it on GitHub:
> https://github.com/scipy/scipy/issues/3786#issuecomment-49822123

Just to give an update for this issue: the two skbio PRs are now merged, I think the scipy linalg procrustes PR should be in a reviewable state, and the hypothetical scipy spatial procrustes PR has not yet been created.

@argriffing, I just added one minor documentation comment to #3809! Thanks, I'll try to get a PR ready in the next few days.

Linalg procrustes PR merged in e346aa5, can this be closed?

@ev-br this isn't quite finished, because the merged `orthogonal_procrustes` just implements the core linear algebra common to all procrustes-related applications.  It is not really useful by itself.  I think there is interest in adding a spatial data analysis layer on top of it, so that if you pass two matched sets of points to a function, then the function will return enough information to let the caller easily transform a new point from the first space towards the second space, subject to some constraint.  For example you could allow only transformations consisting of compositions of translation, rotation, reflection, and scaling, but not shear.  On the other hand if it's bad form to have multiple github PRs associated with a single github issue then a new issue could be opened and this one could be closed.

Ok, let's keep this open then.


Resolving patch:
diff --git a/scipy/spatial/__init__.py b/scipy/spatial/__init__.py
--- a/scipy/spatial/__init__.py
+++ b/scipy/spatial/__init__.py
@@ -81,6 +81,7 @@
    distance_matrix
    minkowski_distance
    minkowski_distance_p
+   procrustes
 
 """
 
@@ -90,6 +91,7 @@
 from .ckdtree import *
 from .qhull import *
 from ._plotutils import *
+from .procrustes import procrustes
 
 __all__ = [s for s in dir() if not s.startswith('_')]
 __all__ += ['distance']
diff --git a/scipy/spatial/procrustes.py b/scipy/spatial/procrustes.py
new file mode 100644
--- /dev/null
+++ b/scipy/spatial/procrustes.py
@@ -0,0 +1,132 @@
+"""
+This module provides functions to perform full Procrustes analysis.
+
+This code was originally written by Justin Kucynski and ported over from
+scikit-bio by Yoshiki Vazquez-Baeza.
+"""
+
+from __future__ import absolute_import, division, print_function
+
+import numpy as np
+from scipy.linalg import orthogonal_procrustes
+
+
+__all__ = ['procrustes']
+
+
+def procrustes(data1, data2):
+    r"""Procrustes analysis, a similarity test for two data sets
+
+    Each input matrix is a set of points or vectors (the rows of the matrix).
+    The dimension of the space is the number of columns of each matrix. Given
+    two identically sized matrices, procrustes standardizes both such that:
+
+    - :math:`tr(AA^{T}) = 1`.
+
+    - Both sets of points are centered around the origin.
+
+    Procrustes ([1]_, [2]_) then applies the optimal transform to the second
+    matrix (including scaling/dilation, rotations, and reflections) to minimize
+    :math:`M^{2}=\sum(data1-data2)^{2}`, or the sum of the squares of the
+    pointwise differences between the two input datasets.
+
+    This function was not designed to handle datasets with different numbers of
+    datapoints (rows).  If two data sets have different dimensionality
+    (different number of columns), simply add columns of zeros the smaller of
+    the two.
+
+    Parameters
+    ----------
+    data1 : array_like
+        matrix, n rows represent points in k (columns) space data1 is the
+        reference data, after it is standardised, the data from data2 will
+        be transformed to fit the pattern in data1 (must have >1 unique
+        points).
+
+    data2 : array_like
+        n rows of data in k space to be fit to data1.  Must be the  same
+        shape (numrows, numcols) as data1 (must have >1 unique points).
+
+    Returns
+    -------
+    mtx1 : array_like
+        a standardized version of data1
+    mtx2 : array_like
+        the orientation of data2 that best fits data1. Centered, but not
+        necessarily :math:`tr(AA^{T}) = 1`.
+    disparity : float
+        :math:`M^{2}` as defined above.
+
+    Raises
+    ------
+    ValueError
+        If the input arrays are not two-dimensional.
+        If the shape of the input arrays is different.
+        If the input arrays have zero columns or zero rows.
+
+    See Also
+    --------
+    orthogonal_procrustes
+
+    Notes
+    -----
+    - The disparity should not depend on the order of the input matrices, but
+      the output matrices will, as only the first output matrix is guaranteed
+      to be scaled such that :math:`tr(AA^{T}) = 1`.
+
+    - Duplicate data points are generally ok, duplicating a data point will
+      increase its effect on the procrustes fit.
+
+    - The disparity scales as the number of points per input matrix.
+
+    References
+    ----------
+    .. [1] Krzanowski, W. J. (2000). "Principles of Multivariate analysis".
+    .. [2] Gower, J. C. (1975). "Generalized procrustes analysis".
+
+    Examples
+    --------
+    >>> import numpy as np
+    >>> from scipy.spatial import procrustes
+    # the b matrix is a rotated, shifted, scaled and mirrored version of a
+    >>> a = np.array([[1, 3], [1, 2], [1, 1], [2, 1]], 'd')
+    >>> b = np.array([[4, -2], [4, -4], [4, -6], [2, -6]], 'd')
+    >>> mtx1, mtx2, disparity = procrustes(a, b)
+    >>> round(disparity)
+    0.0
+
+    """
+
+    mtx1 = np.array(data1, dtype=np.double, copy=True)
+    mtx2 = np.array(data2, dtype=np.double, copy=True)
+
+    if mtx1.ndim != 2 or mtx2.ndim != 2:
+        raise ValueError("Input matrices must be two-dimensional")
+    if mtx1.shape != mtx2.shape:
+        raise ValueError("Input matrices must be of same shape")
+    if mtx1.size == 0:
+        raise ValueError("Input matrices must be >0 rows and >0 cols")
+
+    # translate all the data to the origin
+    mtx1 -= np.mean(mtx1, 0)
+    mtx2 -= np.mean(mtx2, 0)
+
+    norm1 = np.linalg.norm(mtx1)
+    norm2 = np.linalg.norm(mtx2)
+
+    if norm1 == 0 or norm2 == 0:
+        raise ValueError("Input matrices must contain >1 unique points")
+
+    # change scaling of data (in rows) such that trace(mtx*mtx') = 1
+    mtx1 /= norm1
+    mtx2 /= norm2
+
+    # transform mtx2 to minimize disparity
+    R, s = orthogonal_procrustes(mtx1, mtx2)
+    mtx2 = np.dot(mtx2, R.T) * s
+
+    # measure the dissimilarity between the two datasets
+    disparity = np.sum(np.square(mtx1 - mtx2))
+
+    return mtx1, mtx2, disparity
+


Test patch:
diff --git a/scipy/spatial/tests/test_procrustes.py b/scipy/spatial/tests/test_procrustes.py
new file mode 100644
--- /dev/null
+++ b/scipy/spatial/tests/test_procrustes.py
@@ -0,0 +1,120 @@
+from __future__ import absolute_import, division, print_function
+
+from unittest import TestCase, main
+
+import numpy as np
+from scipy.spatial.procrustes import procrustes
+
+
+class ProcrustesTests(TestCase):
+    def setUp(self):
+        """creates inputs"""
+        # an L
+        self.data1 = np.array([[1, 3], [1, 2], [1, 1], [2, 1]], 'd')
+
+        # a larger, shifted, mirrored L
+        self.data2 = np.array([[4, -2], [4, -4], [4, -6], [2, -6]], 'd')
+
+        # an L shifted up 1, right 1, and with point 4 shifted an extra .5
+        # to the right
+        # pointwise distance disparity with data1: 3*(2) + (1 + 1.5^2)
+        self.data3 = np.array([[2, 4], [2, 3], [2, 2], [3, 2.5]], 'd')
+
+        # data4, data5 are standardized (trace(A*A') = 1).
+        # procrustes should return an identical copy if they are used
+        # as the first matrix argument.
+        shiftangle = np.pi / 8
+        self.data4 = np.array([[1, 0], [0, 1], [-1, 0],
+                              [0, -1]], 'd') / np.sqrt(4)
+        self.data5 = np.array([[np.cos(shiftangle), np.sin(shiftangle)],
+                              [np.cos(np.pi / 2 - shiftangle),
+                               np.sin(np.pi / 2 - shiftangle)],
+                              [-np.cos(shiftangle),
+                               -np.sin(shiftangle)],
+                              [-np.cos(np.pi / 2 - shiftangle),
+                               -np.sin(np.pi / 2 - shiftangle)]],
+                              'd') / np.sqrt(4)
+
+    def test_procrustes(self):
+        # tests procrustes' ability to match two matrices.
+        #
+        # the second matrix is a rotated, shifted, scaled, and mirrored version
+        # of the first, in two dimensions only
+        #
+        # can shift, mirror, and scale an 'L'?
+        a, b, disparity = procrustes(self.data1, self.data2)
+        np.testing.assert_allclose(b, a)
+        np.testing.assert_almost_equal(disparity, 0.)
+
+        # if first mtx is standardized, leaves first mtx unchanged?
+        m4, m5, disp45 = procrustes(self.data4, self.data5)
+        np.testing.assert_equal(m4, self.data4)
+
+        # at worst, data3 is an 'L' with one point off by .5
+        m1, m3, disp13 = procrustes(self.data1, self.data3)
+        #self.assertTrue(disp13 < 0.5 ** 2)
+
+    def test_procrustes2(self):
+        # procrustes disparity should not depend on order of matrices
+        m1, m3, disp13 = procrustes(self.data1, self.data3)
+        m3_2, m1_2, disp31 = procrustes(self.data3, self.data1)
+        np.testing.assert_almost_equal(disp13, disp31)
+
+        # try with 3d, 8 pts per
+        rand1 = np.array([[2.61955202, 0.30522265, 0.55515826],
+                         [0.41124708, -0.03966978, -0.31854548],
+                         [0.91910318, 1.39451809, -0.15295084],
+                         [2.00452023, 0.50150048, 0.29485268],
+                         [0.09453595, 0.67528885, 0.03283872],
+                         [0.07015232, 2.18892599, -1.67266852],
+                         [0.65029688, 1.60551637, 0.80013549],
+                         [-0.6607528, 0.53644208, 0.17033891]])
+
+        rand3 = np.array([[0.0809969, 0.09731461, -0.173442],
+                         [-1.84888465, -0.92589646, -1.29335743],
+                         [0.67031855, -1.35957463, 0.41938621],
+                         [0.73967209, -0.20230757, 0.52418027],
+                         [0.17752796, 0.09065607, 0.29827466],
+                         [0.47999368, -0.88455717, -0.57547934],
+                         [-0.11486344, -0.12608506, -0.3395779],
+                         [-0.86106154, -0.28687488, 0.9644429]])
+        res1, res3, disp13 = procrustes(rand1, rand3)
+        res3_2, res1_2, disp31 = procrustes(rand3, rand1)
+        np.testing.assert_almost_equal(disp13, disp31)
+
+    def test_procrustes_shape_mismatch(self):
+        np.testing.assert_raises(ValueError, procrustes,
+                                 np.array([[1, 2], [3, 4]]),
+                                 np.array([[5, 6, 7], [8, 9, 10]]))
+
+    def test_procrustes_empty_rows_or_cols(self):
+        empty = np.array([[]])
+        np.testing.assert_raises(ValueError, procrustes, empty, empty)
+
+    def test_procrustes_no_variation(self):
+        np.testing.assert_raises(ValueError, procrustes,
+                                 np.array([[42, 42], [42, 42]]),
+                                 np.array([[45, 45], [45, 45]]))
+
+    def test_procrustes_bad_number_of_dimensions(self):
+        # fewer dimensions in one dataset
+        np.testing.assert_raises(ValueError, procrustes,
+                                 np.array([1, 1, 2, 3, 5, 8]),
+                                 np.array([[1, 2], [3, 4]]))
+
+        # fewer dimensions in both datasets
+        np.testing.assert_raises(ValueError, procrustes,
+                                 np.array([1, 1, 2, 3, 5, 8]),
+                                 np.array([1, 1, 2, 3, 5, 8]))
+
+        # zero dimensions
+        np.testing.assert_raises(ValueError, procrustes, np.array(7),
+                                 np.array(11))
+
+        # extra dimensions
+        np.testing.assert_raises(ValueError, procrustes,
+                                 np.array([[[11], [7]]]),
+                                 np.array([[[5, 13]]]))
+
+if __name__ == '__main__':
+    main()