Repository: BAMWelDX/weldx

Issue:
CSM SpatialData serialization
I didn't debug in detail but it appears the CSM serialization does not serialize `SpatialData` that is attached?

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/BAMWelDX/weldx/pull/364?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX) Report
> Merging [#364](https://codecov.io/gh/BAMWelDX/weldx/pull/364?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX) (2c0f7ea) into [master](https://codecov.io/gh/BAMWelDX/weldx/commit/eb6ef561128aa71a00d0376cf7e72584f329e92c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX) (eb6ef56) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/BAMWelDX/weldx/pull/364/graphs/tree.svg?width=650&height=150&src=pr&token=wdof1qQTsn&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX)](https://codecov.io/gh/BAMWelDX/weldx/pull/364?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX)

```diff
@@           Coverage Diff           @@
##           master     #364   +/-   ##
=======================================
  Coverage   96.91%   96.91%           
=======================================
  Files          87       87           
  Lines        5214     5221    +7     
=======================================
+ Hits         5053     5060    +7     
  Misses        161      161           
```


| [Impacted Files](https://codecov.io/gh/BAMWelDX/weldx/pull/364?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX) | Coverage Δ | |
|---|---|---|
| [...ore/transformations/coordinate\_system\_hierarchy.py](https://codecov.io/gh/BAMWelDX/weldx/pull/364/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX#diff-d2VsZHgvYXNkZi90YWdzL3dlbGR4L2NvcmUvdHJhbnNmb3JtYXRpb25zL2Nvb3JkaW5hdGVfc3lzdGVtX2hpZXJhcmNoeS5weQ==) | `100.00% <100.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/BAMWelDX/weldx/pull/364?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/BAMWelDX/weldx/pull/364?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX). Last update [eb6ef56...2c0f7ea](https://codecov.io/gh/BAMWelDX/weldx/pull/364?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=BAMWelDX).

pep8speaks: Hello @vhirtham! Thanks for updating this PR. 

* In the file [`weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py`](https://github.com/BAMWelDX/weldx/blob/6a3bb9a5fa232b30eca7d2f3fad9cb8985cdba6b/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py):

> [Line 469:26](https://github.com/BAMWelDX/weldx/blob/6a3bb9a5fa232b30eca7d2f3fad9cb8985cdba6b/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py#L469): [E231](https://duckduckgo.com/?q=pep8%20E231) missing whitespace after ':'
> [Line 469:26](https://github.com/BAMWelDX/weldx/blob/6a3bb9a5fa232b30eca7d2f3fad9cb8985cdba6b/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py#L469): [E999](https://duckduckgo.com/?q=pep8%20E999) SyntaxError: invalid syntax
> [Line 469:28](https://github.com/BAMWelDX/weldx/blob/6a3bb9a5fa232b30eca7d2f3fad9cb8985cdba6b/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py#L469): [E251](https://duckduckgo.com/?q=pep8%20E251) unexpected spaces around keyword / parameter equals











Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b143ff8..2297253 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -45,6 +45,8 @@
 - add `core/graph/di_node`, `core/graph/di_edge` & `core/graph/di_graph` for implementing a
   generic `networkx.DiGraph` [[#330]](https://github.com/BAMWelDX/weldx/pull/330)
 - compatibility with ASDF-2.8 [[#355]](https://github.com/BAMWelDX/weldx/pull/355)
+- data attached to an instance of the `CoordinateSystemManger` is now also stored in a WelDX file 
+  [[#364]](https://github.com/BAMWelDX/weldx/pull/339)
 - replace references to base asdf tags with `-1.*` version wildcard [[#373]](https://github.com/BAMWelDX/weldx/pull/373)
 - update `single-pass-weldx.1.0.0.schema` to allow groove types by
   wildcard [[#373]](https://github.com/BAMWelDX/weldx/pull/373)
diff --git a/README.md b/README.md
index fc8dbbc..667a84c 100644
--- a/README.md
+++ b/README.md
@@ -102,7 +102,6 @@ This research is funded by the Federal Ministry of Education and Research of Ger
 
 [![pytest](https://github.com/BAMWelDX/weldx/workflows/pytest/badge.svg?branch=master)](https://github.com/BAMWelDX/weldx/actions?query=workflow%3Apytest+branch%3Amaster)
 [![conda build](https://github.com/BAMWelDX/weldx/workflows/conda%20build/badge.svg?branch=master)](https://github.com/BAMWelDX/weldx/actions?query=workflow%3A%22conda+build%22+branch%3Amaster)
-[![](https://travis-ci.com/BAMWelDX/weldx.svg?branch=master)](https://travis-ci.com/BAMWelDX/weldx)
 [![Build status](https://ci.appveyor.com/api/projects/status/6yvswkpj7mmdbrk1/branch/master?svg=true)](https://ci.appveyor.com/project/BAMWelDX/weldx/branch/master)
 
 ### Code Status
diff --git a/weldx/asdf/schemas/weldx.bam.de/weldx/core/transformations/coordinate_system_hierarchy-1.0.0.yaml b/weldx/asdf/schemas/weldx.bam.de/weldx/core/transformations/coordinate_system_hierarchy-1.0.0.yaml
index 4bda092..5413e6f 100644
--- a/weldx/asdf/schemas/weldx.bam.de/weldx/core/transformations/coordinate_system_hierarchy-1.0.0.yaml
+++ b/weldx/asdf/schemas/weldx.bam.de/weldx/core/transformations/coordinate_system_hierarchy-1.0.0.yaml
@@ -97,7 +97,20 @@ properties:
     items:
       tag: "tag:weldx.bam.de:weldx/core/transformations/coordinate_transformation-1.0.0"
 
-propertyOrder: [name, root_system_name, reference_time, subsystems, subsystem_data,  coordinate_systems]
+  spatial_data:
+    type: array
+    items:
+      type: object
+      properties:
+        coordinate_system:
+          type: string
+        name:
+          type: string
+        data:
+          tag: "tag:weldx.bam.de:weldx/core/geometry/spatial_data-1.*"
+      required: [coordinate_system, name, data]
+
+propertyOrder: [name, root_system_name, reference_time, subsystems, subsystem_data, coordinate_systems, spatial_data]
 required: [name, root_system_name, coordinate_systems]
 flowStyle: block
 ...
diff --git a/weldx/asdf/tags/weldx/core/geometry/spatial_data.py b/weldx/asdf/tags/weldx/core/geometry/spatial_data.py
index 4b0f44b..ab62360 100644
--- a/weldx/asdf/tags/weldx/core/geometry/spatial_data.py
+++ b/weldx/asdf/tags/weldx/core/geometry/spatial_data.py
@@ -1,5 +1,7 @@
 from copy import deepcopy
 
+import numpy as np
+
 from weldx.asdf.types import WeldxType
 from weldx.geometry import SpatialData
 
@@ -57,4 +59,6 @@ class SpatialDataTypeASDF(WeldxType):
             An instance of the 'weldx.geometry.point_cloud' type.
 
         """
+        if "coordinates" in tree:
+            tree["coordinates"] = np.asarray(tree["coordinates"])
         return SpatialData(**tree)
diff --git a/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py b/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py
index 704ae95..8e6a87c 100644
--- a/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py
+++ b/weldx/asdf/tags/weldx/core/transformations/coordinate_system_hierarchy.py
@@ -406,6 +406,13 @@ class CoordinateSystemManagerASDF(WeldxType):
             if subsystem.parent_system == node.name
         ]
 
+        spatial_data = None
+        if len(node._data) > 0:
+            spatial_data = [
+                dict(name=k, coordinate_system=v.coordinate_system_name, data=v.data)
+                for k, v in node._data.items()
+            ]
+
         tree = {
             "name": node.name,
             "reference_time": node.reference_time,
@@ -413,6 +420,7 @@ class CoordinateSystemManagerASDF(WeldxType):
             "subsystems": subsystem_data,
             "root_system_name": node.root_system_name,
             "coordinate_systems": coordinate_system_data,
+            "spatial_data": spatial_data,
         }
         return tree
 
@@ -458,4 +466,8 @@ class CoordinateSystemManagerASDF(WeldxType):
         cls._add_coordinate_systems_to_subsystems(tree, csm, subsystem_data_list)
         cls._merge_subsystems(tree, csm, subsystem_data_list)
 
+        if (spatial_data := tree.get("spatial_data")) is not None:
+            for item in spatial_data:
+                csm.assign_data(item["data"], item["name"], item["coordinate_system"])
+
         return csm
diff --git a/weldx/transformations/cs_manager.py b/weldx/transformations/cs_manager.py
index 89d630a..16bcc12 100644
--- a/weldx/transformations/cs_manager.py
+++ b/weldx/transformations/cs_manager.py
@@ -1149,7 +1149,7 @@ class CoordinateSystemManager:
 
     def get_data(
         self, data_name, target_coordinate_system_name=None
-    ) -> Union[np.ndarray, xr.DataArray]:
+    ) -> Union[np.ndarray, SpatialData]:
         """Get the specified data, optionally transformed into any coordinate system.
 
         Parameters


Test patch:
diff --git a/weldx/tests/asdf_tests/test_asdf_core.py b/weldx/tests/asdf_tests/test_asdf_core.py
index 24a7645..a66a437 100644
--- a/weldx/tests/asdf_tests/test_asdf_core.py
+++ b/weldx/tests/asdf_tests/test_asdf_core.py
@@ -14,7 +14,7 @@ from scipy.spatial.transform import Rotation
 
 import weldx.transformations as tf
 from weldx.asdf.tags.weldx.core.file import ExternalFile
-from weldx.asdf.util import _write_buffer, _write_read_buffer
+from weldx.asdf.util import _write_buffer, write_read_buffer
 from weldx.constants import WELDX_QUANTITY as Q_
 from weldx.core import MathematicalExpression as ME  # nopep8
 from weldx.core import TimeSeries
@@ -52,7 +52,7 @@ _base_rotation = Rotation.from_euler(
     ],
 )
 def test_rotation(inputs):
-    data = _write_read_buffer({"rot": inputs})
+    data = write_read_buffer({"rot": inputs})
     r = data["rot"]
     assert np.allclose(r.as_quat(), inputs.as_quat())
     if hasattr(inputs, "wx_meta"):
@@ -79,7 +79,7 @@ def test_rotation_euler_prefix(inputs):
     """Test unit prefix handling."""
     degrees = "degree" in str(inputs.u)
     rot = WXRotation.from_euler(seq="x", angles=inputs)
-    data = _write_read_buffer({"rot": rot})
+    data = write_read_buffer({"rot": rot})
     r = data["rot"].as_euler("xyz", degrees=degrees)[0]
     r = Q_(r, "degree") if degrees else Q_(r, "rad")
     assert np.allclose(inputs, r)
@@ -118,7 +118,7 @@ def test_xarray_data_array(copy_arrays, lazy_load):
     """Test ASDF read/write of xarray.DataArray."""
     dax = get_xarray_example_data_array()
     tree = {"dax": dax}
-    dax_file = _write_read_buffer(
+    dax_file = write_read_buffer(
         tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
     )["dax"]
     assert dax.identical(dax_file)
@@ -168,7 +168,7 @@ def get_xarray_example_dataset():
 def test_xarray_dataset(copy_arrays, lazy_load):
     dsx = get_xarray_example_dataset()
     tree = {"dsx": dsx}
-    dsx_file = _write_read_buffer(
+    dsx_file = write_read_buffer(
         tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
     )["dsx"]
     assert dsx.identical(dsx_file)
@@ -227,7 +227,7 @@ def test_local_coordinate_system(
 ):
     """Test (de)serialization of LocalCoordinateSystem in ASDF."""
     lcs = get_local_coordinate_system(time_dep_orientation, time_dep_coordinates)
-    data = _write_read_buffer(
+    data = write_read_buffer(
         {"lcs": lcs}, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
     )
     assert data["lcs"] == lcs
@@ -299,7 +299,7 @@ def get_example_coordinate_system_manager():
 def test_coordinate_system_manager(copy_arrays, lazy_load):
     csm = get_example_coordinate_system_manager()
     tree = {"cs_hierarchy": csm}
-    data = _write_read_buffer(
+    data = write_read_buffer(
         tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
     )
     csm_file = data["cs_hierarchy"]
@@ -361,7 +361,7 @@ def get_coordinate_system_manager_with_subsystems(nested: bool):
 def test_coordinate_system_manager_with_subsystems(copy_arrays, lazy_load, nested):
     csm = get_coordinate_system_manager_with_subsystems(nested)
     tree = {"cs_hierarchy": csm}
-    data = _write_read_buffer(
+    data = write_read_buffer(
         tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
     )
     csm_file = data["cs_hierarchy"]
@@ -403,13 +403,50 @@ def test_coordinate_system_manager_time_dependencies(
     csm_root.merge(csm_sub_2)
 
     tree = {"cs_hierarchy": csm_root}
-    data = _write_read_buffer(
+    data = write_read_buffer(
         tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
     )
     csm_file = data["cs_hierarchy"]
     assert csm_root == csm_file
 
 
+@pytest.mark.parametrize("copy_arrays", [True, False])
+@pytest.mark.parametrize("lazy_load", [True, False])
+def test_coordinate_system_manager_with_data(copy_arrays, lazy_load):
+    """Test if data attached to a CSM is stored and read correctly."""
+    csm = tf.CoordinateSystemManager("root", "csm")
+    csm.create_cs("cs_1", "root", coordinates=[1, 1, 1])
+    csm.create_cs("cs_2", "root", coordinates=[-1, -1, -1])
+    csm.create_cs("cs_11", "cs_1", coordinates=[1, 1, 1])
+
+    data_11 = SpatialData(coordinates=np.array([[1.0, 2.0, 3.0], [3.0, 2.0, 1.0]]))
+    data_2 = SpatialData(
+        coordinates=np.array(
+            [
+                [0.0, 0.0, 0.0],
+                [1.0, 0.0, 0.0],
+                [1.0, 1.0, 0.0],
+                [0.0, 1.0, 0.0],
+            ]
+        ),
+        triangles=np.array([[0, 1, 2], [0, 2, 3]], dtype="uint32"),
+    )
+
+    csm.assign_data(data_11, "data_11", "cs_11")
+    csm.assign_data(data_2, "data_2", "cs_2")
+
+    tree = {"csm": csm}
+    buffer = write_read_buffer(
+        tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
+    )
+    csm_buffer = buffer["csm"]
+
+    for data_name in csm.data_names:
+        sd = csm.get_data(data_name)
+        sd_buffer = csm_buffer.get_data(data_name)
+        assert sd == sd_buffer
+
+
 # --------------------------------------------------------------------------------------
 # TimeSeries
 # --------------------------------------------------------------------------------------
@@ -428,7 +465,7 @@ def test_coordinate_system_manager_time_dependencies(
     ],
 )
 def test_time_series_discrete(ts, copy_arrays, lazy_load):
-    ts_file = _write_read_buffer(
+    ts_file = write_read_buffer(
         {"ts": ts}, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
     )["ts"]
     if isinstance(ts.data, ME):
@@ -616,7 +653,7 @@ class TestExternalFile:
             asdf_save_content=store_content,
         )
         tree = {"file": ef}
-        ef_file = _write_read_buffer(
+        ef_file = write_read_buffer(
             tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
         )["file"]
 
@@ -661,7 +698,7 @@ class TestPointCloud:
 
         pc = SpatialData(coordinates=coordinates, triangles=triangles)
         tree = {"point_cloud": pc}
-        pc_file = _write_read_buffer(
+        pc_file = write_read_buffer(
             tree, open_kwargs={"copy_arrays": copy_arrays, "lazy_load": lazy_load}
         )["point_cloud"]
 
@@ -678,7 +715,7 @@ class TestGraph:
         g.add_edges_from(
             [("A", "B"), ("A", "C"), ("A", "F"), ("D", "C"), ("B", "H"), ("X", "A")]
         )
-        g2 = _write_read_buffer({"graph": g})["graph"]
+        g2 = write_read_buffer({"graph": g})["graph"]
 
         assert all(e in g.edges for e in g2.edges)
         assert all(n in g.nodes for n in g2.nodes)
diff --git a/weldx/tests/test_geometry.py b/weldx/tests/test_geometry.py
index 4a1caf5..ab3d4c7 100644
--- a/weldx/tests/test_geometry.py
+++ b/weldx/tests/test_geometry.py
@@ -4,7 +4,7 @@ import copy
 import math
 from pathlib import Path
 from tempfile import TemporaryDirectory
-from typing import List, Union
+from typing import Dict, List, Union
 
 import numpy as np
 import pint
@@ -2906,6 +2906,8 @@ class TestSpatialData:
         if len(arguments) > 1 and arguments[1] is not None:
             np.all(arguments[1] == pc.triangles)
 
+    # test_class_creation_exceptions ---------------------------------------------------
+
     @staticmethod
     @pytest.mark.parametrize(
         "arguments, exception_type, test_name",
@@ -2931,6 +2933,57 @@ class TestSpatialData:
         with pytest.raises(exception_type):
             SpatialData(*arguments)
 
+    # test_comparison ------------------------------------------------------------------
+
+    @staticmethod
+    @pytest.mark.parametrize(
+        "kwargs_mod, expected_result",
+        [
+            ({}, True),
+            (dict(coordinates=[[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 1]]), False),
+            (dict(coordinates=[[0, 0, 0], [1, 0, 0], [1, 1, 0]]), False),
+            (dict(triangles=[[0, 1, 2], [2, 3, 1]]), False),
+            (dict(triangles=[[0, 1, 2], [2, 3, 1], [2, 3, 1]]), False),
+            (dict(triangles=[[0, 1, 2]]), False),
+            (dict(triangles=None), False),
+            (dict(attributes=dict(data=[2, 2, 3])), False),
+            (dict(attributes=dict(dat=[1, 2, 3])), False),
+            # uncomment once issue #376 is resolved
+            # (dict(attributes=dict(data=[1, 2, 3], more=[1, 2, 5])), False),
+            (dict(attributes={}), False),
+            (dict(attributes=None), False),
+        ],
+    )
+    def test_comparison(kwargs_mod: Dict, expected_result: bool):
+        """Test the comparison operator by comparing two instances.
+
+        Parameters
+        ----------
+        kwargs_mod :
+            A dictionary of key word arguments that is used to overwrite the default
+            values in the RHS `SpatialData`. If an empty dict is passed, LHS and RHS
+            are constructed with the same values.
+        expected_result :
+            Expected result of the comparison
+
+        """
+        from copy import deepcopy
+
+        default_kwargs = dict(
+            coordinates=[[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0]],
+            triangles=[[0, 1, 2], [2, 3, 0]],
+            attributes=dict(data=[1, 2, 3]),
+        )
+        reference = SpatialData(**default_kwargs)
+
+        kwargs_other = deepcopy(default_kwargs)
+        kwargs_other.update(kwargs_mod)
+        other = SpatialData(**kwargs_other)
+
+        assert (reference == other) == expected_result
+
+    # test_read_write_file -------------------------------------------------------------
+
     @staticmethod
     @pytest.mark.parametrize(
         "filename",