Repository: aas-core-works/aas-core3.0-python (language: python)

Issue:
Failed XML deserialization on special XML attributes
The python XML deserialization ensures, that elements won't contain any attributes when parsing the different element types (`_raise_if_has_tail_or_attrib`), this is in accordance with the [spec](https://github.com/admin-shell-io/aas-specs/tree/master/schemas/xml#why-no-xml-attributes) that mandates that the XML representation must not use attributes for modelling purposes.

However, this leads to issues with special XML attributes, currently, aas-core3.0-python, for example, fails to deserialize the following example (Note: the C# SDK has no such check in place and will happily deserialize the document):

```xml
<?xml version="1.0" encoding="UTF-8"?>
<environment
    xmlns="https://admin-shell.io/aas/3/0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="https://admin-shell.io/aas/3/0/AAS.xsd">
</environment>
```

Interestingly, `xml.etree` does not consider xml namespace attributes as attributes, i.e., if we drop the `xsi:schemaLocation` line, the xmlization can also process the example.


Discussion / hints:
Hi @empwilli ! Thanks for reporting! 

Actually, there were related problems with C# and Golang SDKs:
https://github.com/aas-core-works/aas-core3.0-csharp/issues/24
https://github.com/aas-core-works/aas-core3.0-golang/issues/25

Please give me some time -- I hope to be able to fix this till the end of the week.
@empwilli : in the meanwhile, could you please double-check -- what should be the correct behavior here -- should attributes from the namespace `xsi` simply be ignored?

Please see this comment and following: https://github.com/aas-core-works/aas-core3.0-golang/issues/25#issuecomment-2112039644

We probably need a similar solution to: https://github.com/aas-core-works/aas-core3.0-golang/issues/25#issuecomment-2114208850


> @empwilli : in the meanwhile, could you please double-check -- what should be the correct behavior here -- should attributes from the namespace xsi simply be ignored?

To my understanding we should at least allow, i.e., ignore, the `xsi:schemaLocation` and `xsi:noNamespaceSchemaLocation`, as they provide means to refer to the AAS XML schema. I'm a little bit conflicted wrt. to `xsi:nil` and `xsi:type`, though, as they allow to change the semantics of the model.

I understand that one has to be careful not to introduce an ever growing list of ignored attributes, though. In a quick google search I could not find further standard attributes that affect schema validation/interpretation of XML documents. So I'd argue to either completely ignore attributes with `xsi` prefix, or, at least to ignore `xsi:schemaLocation` and `xsi:noNamespaceSchemaLocation`.  

Edit: formatting

Fix patch:
diff --git a/docs/how-to-build-docs.md b/docs/how-to-build-docs.md
index c5c27ec0..c85a017c 100644
--- a/docs/how-to-build-docs.md
+++ b/docs/how-to-build-docs.md
@@ -28,5 +28,5 @@ The documentation is in the `docs/build` directory.
 Test the examples in the documentation with the ``doctest`` builder:
 
 ```
-sphinx-build source -b doctest
+sphinx-build source build -b doctest
 ```
diff --git a/docs/source/getting_started/xmlize.rst b/docs/source/getting_started/xmlize.rst
index b533613c..a8029a33 100644
--- a/docs/source/getting_started/xmlize.rst
+++ b/docs/source/getting_started/xmlize.rst
@@ -88,3 +88,62 @@ Errors
 
 If the XML document comes in an unexpected form, our SDK throws a :py:class:`aas_core3.xmlization.DeserializationException`.
 This can happen, for example, if unexpected XML elements or XML attributes are encountered, or an expected XML element is missing.
+
+Disregarding XML Attributes
+===========================
+The specification mandates to use no XML attributes, but some libraries and tools still add their own XML attributes in the XML serialization of an AAS.
+You need to remove them to avoid de-serialization errors.
+
+To that end, you need to operate directly on an iterator of XML events and elements coming from :py:func:`xml.etree.ElementTree.iterparse`.
+The attributes need to be cleared as you iterate and re-yield over the iterator.
+
+Here is an example snippet:
+
+.. testcode::
+
+    import io
+    import xml.etree.ElementTree
+    from typing import Tuple, Iterator
+
+    import aas_core3.xmlization as aas_xmlization
+
+    text = (
+            '<?xml version="1.0" encoding="UTF-8"?>\n'
+        '<environment\n'
+        '    xmlns="https://admin-shell.io/aas/3/0"\n'
+        '    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"\n'
+        '    xsi:schemaLocation="https://admin-shell.io/aas/3/0/AAS.xsd">\n'
+        '</environment>\n'
+    )
+
+    iterator = xml.etree.ElementTree.iterparse(
+        io.StringIO(text),
+        # The XML de-serializer needs to operate on 'start' and 'end' events.
+        events=("start", "end")
+    )
+
+    def with_attributes_removed(
+            an_iterator: Iterator[Tuple[str, xml.etree.ElementTree.Element]]
+    ) -> Iterator[Tuple[str, xml.etree.ElementTree.Element]]:
+        """
+        Map the :paramref:`iterator` such that all attributes are removed.
+
+        :param an_iterator: to be mapped
+        :yield: event and element without attributes from :paramref:`iterator`
+        """
+        for event, element in an_iterator:
+            element.attrib.clear()
+
+            yield event, element
+
+    environment = aas_xmlization.environment_from_iterparse(
+        iterator=with_attributes_removed(iterator)
+    )
+
+    # The attributes are lost in the subsequent serialization.
+    serialization = aas_xmlization.to_str(environment)
+
+    assert (
+        '<environment xmlns="https://admin-shell.io/aas/3/0"/>'
+        == serialization
+    )


Test patch:
diff --git a/tests/test_xmlization_when_text_attached_to_end_element.py b/tests/test_xmlization_when_text_attached_to_end_element.py
index 18cbe1eb..420f8108 100644
--- a/tests/test_xmlization_when_text_attached_to_end_element.py
+++ b/tests/test_xmlization_when_text_attached_to_end_element.py
@@ -1,22 +1,26 @@
-"""
-Test the edge case when the ElementTree parser attaches the text to the end element.
-
-The quirky part is that the text is *sometimes* attached to the start element
-and *other times* to the end element. We could not figure out which situation
-occurs when.
-
-Please see also the note in the Python documentation:
-https://docs.python.org/3/library/xml.etree.elementtree.html#xml.etree.ElementTree.XMLPullParser.read_events
-"""
+"""Test the edge cases with the ElementTree parser."""
 
 # pylint: disable=missing-docstring
-
+import io
 import unittest
+import xml.etree.ElementTree
+from typing import Tuple, Iterator
 
 import aas_core3.xmlization as aas_xmlization
 
 
-class TestOnXMLsFromTheWild(unittest.TestCase):
+class TestTextAttachedToAnEndElement(unittest.TestCase):
+    """
+    Test the edge case when the ElementTree attaches the text to the end element.
+
+    The quirky part is that the text is *sometimes* attached to the start element
+    and *other times* to the end element. We could not figure out which situation
+    occurs when.
+
+    Please see also the note in the Python documentation:
+    https://docs.python.org/3/library/xml.etree.elementtree.html#xml.etree.ElementTree.XMLPullParser.read_events
+    """
+
     def test_on_a_large_file(self) -> None:
         # Please see:
         # https://github.com/aas-core-works/aas-core3.0-python/issues/17
@@ -642,5 +646,58 @@ def test_on_a_large_file(self) -> None:
         _ = aas_xmlization.environment_from_str(text)
 
 
+class TestOnXSIAttributes(unittest.TestCase):
+    """
+    Test how we can deal gracefully with common ``xsi:`` attributes.
+
+    The specification does not allow any XML attributes, but many applications simply
+    add ``xsi`` attributes by default.
+
+    See:
+    https://github.com/aas-core-works/aas-core3.0-python/issues/44
+    """
+
+    def test_pipeline_approach(self) -> None:
+        text = """\
+<?xml version="1.0" encoding="UTF-8"?>
+<environment
+    xmlns="https://admin-shell.io/aas/3/0"
+    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+    xsi:schemaLocation="https://admin-shell.io/aas/3/0/AAS.xsd">
+</environment>
+"""
+
+        iterator = xml.etree.ElementTree.iterparse(
+            io.StringIO(text), events=("start", "end")
+        )
+
+        def with_attributes_removed(
+            an_iterator: Iterator[Tuple[str, xml.etree.ElementTree.Element]]
+        ) -> Iterator[Tuple[str, xml.etree.ElementTree.Element]]:
+            """
+            Map the :paramref:`iterator` such that all attributes are removed.
+
+            :param an_iterator: to be mapped
+            :yield: event and element without attributes from :paramref:`iterator`
+            """
+            for event, element in an_iterator:
+                element.attrib.clear()
+
+                yield event, element
+
+        environment = aas_xmlization.environment_from_iterparse(
+            iterator=with_attributes_removed(iterator)
+        )
+
+        # NOTE (mristin):
+        # The attributes are lost in the subsequent serialization.
+        serialization = aas_xmlization.to_str(environment)
+
+        self.assertEqual(
+            '<environment xmlns="https://admin-shell.io/aas/3/0"/>',
+            serialization,
+        )
+
+
 if __name__ == "__main__":
     unittest.main()