Repository: voxel51/fiftyone

Issue:
[BUG] Issue with Orthographic Projection when supplied projection_normal=(0, 0, -1)
### Describe the problem

The method [`_rotation_matrix_from_vectors`](https://github.com/voxel51/fiftyone/blob/7bea0c2b4aae5cad19d84c1ac49e0baa65fbe817/fiftyone/utils/utils3d.py#L777-L786) is ill-defined and returns `NaN` when the angle between `vec1` and `vec2` is 180º. 

From a geometrical standpoint the issue is being cause by the fact that there is no longer a single "shortest" rotation that transforms `vec1` to `vec2` There are now infinite perpendicular axes to `vec1` and `vec2` that can be used to rotate `vec1` onto `vec2`.

From a numerical perspective, what is happening is that `s` in this [line](https://github.com/voxel51/fiftyone/blob/7bea0c2b4aae5cad19d84c1ac49e0baa65fbe817/fiftyone/utils/utils3d.py#L786) is 0. And the rotation matrix returned becomes `NaN`.

A solution would would be to pick a random axis perpendicular to `vec1` and `vec2` and apply rotation 180º around it. 

### Code to reproduce issue

I triggered the issue when calling  `fiftyone.utils.utils3d.compute_orthographic_projection_images`, with 

```python
compute_orthographic_projection_images(
    ...
    projection_normal=(0, 0, -1),
)
```

But you can shortcut all the stack call and simply call 

```python
fiftyone.utils.utils3d. _rotation_matrix_from_vectors((0, 0, -1), (0, 0, 1))
```

In general, it would be ideal if one could specify one not only the projection normal, but an actual full rotation matrix to fully define the camera orientation. As things are, the pose still has some degrees of freedom.


### System information

-   **OS Platform and Distribution** (e.g., Linux Ubuntu 22.04): macOS Sonoma 14.4 (23E214)
-   **Python version** (`python --version`): 3.10.14
-   **FiftyOne version** (`fiftyone --version`): v0.23.7
-   **FiftyOne installed from** (pip or source): pip

### Other info/logs

n.a

### Willingness to contribute

The FiftyOne Community encourages bug fix contributions. Would you or another
member of your organization be willing to contribute a fix for this bug to the
FiftyOne codebase?

-   [ ] Yes. I can contribute a fix for this bug independently
-   [x] Yes. I would be willing to contribute a fix for this bug with guidance
        from the FiftyOne community
-   [ ] No. I cannot contribute a bug fix at this time



Discussion:
Thanks @SergioRAgostinho !
I'll admit I did not know much about this but after doing some background research I understand, and what you're saying makes sense!

You seem very knowledgeable about this subject, would you be willing to submit a PR?

> but an actual full rotation matrix to fully define the camera orientation. As things are, the pose still has some degrees of freedom.

We'll take a PR for this too if you can!!
Hi @swheaton. I think I might need a bit more guidance. I'm fighting a bit the import structure of fiftyone and maybe you have good suggestions.

I started working on this by adding a unit test test under `tests/unittests/utils/utils3d_tests.py`, but ran into problems. To give a sample of what happened, here are the steps I followed.


1. Clone the latest `develop`
2. `cd` into it
3. Create a conda env for Python 3.8 (no 3.7 for macOS arm apparently) and activate it.
4. `bash install.bash -d -p`
5. Added a unit test under  `tests/unittests/utils/utils3d_tests.py`
6. Added the code below

    ```python
    from fiftyone.utils.utils3d import _rotation_matrix_from_vectors
    ```

7. Ran `pytest tests/unittests/utils/utils3d_tests.py`
8. Was met with

    ```
    ========================================================================= ERRORS =========================================================================
    ________________________________________________ ERROR collecting tests/unittests/utils/utils3d_tests.py _________________________________________________
    tests/unittests/utils/utils3d_tests.py:10: in <module>
        from fiftyone.utils.utils3d import compute_cuboid_iou
    fiftyone/utils/utils3d.py:17: in <module>
        import fiftyone.core.fields as fof
    fiftyone/core/fields.py:22: in <module>
        import fiftyone.core.odm as foo
    fiftyone/core/odm/__init__.py:10: in <module>
        from .database import (
    fiftyone/core/odm/database.py:69: in <module>
        class DatabaseConfigDocument(Document):
    fiftyone/core/odm/database.py:75: in DatabaseConfigDocument
        version = fof.StringField()
    E   AttributeError: partially initialized module 'fiftyone.core.fields' has no attribute 'StringField' (most likely due to a circular import)
    ================================================================ short test summary info =================================================================
    ERROR tests/unittests/utils/utils3d_tests.py - AttributeError: partially initialized module 'fiftyone.core.fields' has no attribute 'StringField' (most likely due to a circular import)
    ```

I assume this is something very specific of the repo structure and import strategy.

1. Should the test file be added somewhere else?
2. Am I invoking pytest incorrectly?
3. Is there some special way to import things that I'm missing?
@SergioRAgostinho Thanks for trying it!
That should work, however I have 2 suggestions:
1. Just use `fou3d._rotation_matrix_from_vectors()` in your test
2. I think if you set `PYTHONPATH` it will work. `PYTHONPATH=/path/to/fiftyone pytest tests/unittests/utils/utils3d_tests.py`

Resolving patch:
diff --git a/fiftyone/utils/utils3d.py b/fiftyone/utils/utils3d.py
--- a/fiftyone/utils/utils3d.py
+++ b/fiftyone/utils/utils3d.py
@@ -704,7 +704,18 @@ def _parse_point_cloud(
     ):
         # rotate points so that they are perpendicular to the projection plane
         # as opposed to the default XY plane
-        R = _rotation_matrix_from_vectors(projection_normal, [0, 0, 1])
+        normal = np.asarray(projection_normal).reshape((1, 3))
+        with warnings.catch_warnings():
+            # There are multiple rotations that can align two vectors. This is known
+            # and accepted, so we suppress the warning.
+            warnings.filterwarnings(
+                "ignore",
+                message="Optimal rotation is not uniquely or poorly defined for the given sets of vectors\.",
+                category=UserWarning,
+            )
+            R = sp.transform.Rotation.align_vectors([[0, 0, 1]], normal)[
+                0
+            ].as_matrix()
         pc = pc.rotate(R, center=[0, 0, 0])
 
     if bounds is None:
@@ -774,18 +785,6 @@ def _clamp_to_discrete(arr, discrete):
     return clamp_list[np.clip(idx, 0, len(clamp_list) - 1)]
 
 
-# Reference: https://math.stackexchange.com/q/180418
-def _rotation_matrix_from_vectors(vec1, vec2):
-    """Returns the rotation matrix that aligns vec1 to vec2."""
-    a = (np.asarray(vec1) / np.linalg.norm(vec1)).reshape(3)
-    b = (np.asarray(vec2) / np.linalg.norm(vec2)).reshape(3)
-    v = np.cross(a, b)
-    c = np.dot(a, b)
-    s = np.linalg.norm(v)
-    K = np.array([[0, -v[2], v[1]], [v[2], 0, -v[0]], [-v[1], v[0], 0]])
-    return np.eye(3) + K + K.dot(K) * ((1 - c) / (s**2))
-
-
 def _parse_size(size, bounds):
     width, height = size
 


Test patch:
diff --git a/tests/unittests/utils3d_tests.py b/tests/unittests/utils3d_tests.py
--- a/tests/unittests/utils3d_tests.py
+++ b/tests/unittests/utils3d_tests.py
@@ -239,6 +239,46 @@ def test_mono_projection(self):
         )
 
 
+class ParsePointCloudTests(BaseOrthographicProjectionTests):
+    def test_rotation_matrix_from_projection_normal(self):
+
+        # By default, the projection normal is set to (0, 0, 1)
+        test_cases = [
+            (0, 1, 0),  # 90º rotation
+            (0, 0, 1),  # 0º rotation
+            (0, 0, 1),  # 180º rotation
+            (1, 2, 3),  # arbitrary rotation
+        ]
+
+        for projection_normal in test_cases:
+            with self.subTest(projection_normal=projection_normal):
+
+                # because the rotation is ambiguous, we can't compare original and rotated arbitrary points
+                # we can only ensure that points along the user specified projection normal vector
+                # will invariably be aligned with the z-axis after rotation
+                points = np.array([projection_normal])
+
+                pc = o3d.geometry.PointCloud()
+                pc.points = o3d.utility.Vector3dVector(points)
+                o3d.io.write_point_cloud(
+                    self.test_pcd_path, pc, write_ascii=True
+                )
+
+                points_rotated = fou3d._parse_point_cloud(
+                    self.test_pcd_path,
+                    bounds=[(-1, -1, 0), (1, 1, 10)],
+                    projection_normal=projection_normal,
+                )[0]
+
+                # After the point is rotated, it's translated so that the minimum bound lies at the origin
+                # this is why we add 1 to the x and y coordinates
+                np.testing.assert_allclose(
+                    (1, 1, np.linalg.norm(projection_normal)),
+                    points_rotated[0],
+                    atol=1e-15,
+                )
+
+
 class DataModelTests(unittest.TestCase):
     @drop_datasets
     def test_orthographic_projection_metadata_field(self):