Repository: tefra/xsdata
Pull request #1037 (2024-05-11 14:14:56)

feat: Remove whitespace from bytes encoded xml strings
## 📒 Description

At some point we enabled strict bytes parsing and in a lot of samples people/libraries love to split the values into multiple lines which now fails in xsdata. Let's ensure no whitespace exists before we parse the actual value

Resolves #xxxx

## 🔗 What I've Done

> Write a description of the steps taken to resolve the issue

## 💬 Comments

> A place to write any comments to the reviewer.

## 🛫 Checklist

- [ ] Updated docs
- [ ] Added unit-tests
- [ ] [Sample tests](https://github.com/tefra/xsdata-samples) pass
- [ ] [W3C tests](https://github.com/tefra/xsdata-w3c-tests) pass


sonarcloud[bot]: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=tefra_xsdata&pullRequest=1037) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 New issues](https://sonarcloud.io/project/issues?id=tefra_xsdata&pullRequest=1037&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/component_measures?id=tefra_xsdata&pullRequest=1037&metric=new_accepted_issues&view=list)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=tefra_xsdata&pullRequest=1037&resolved=false&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/no-data-16px.png '') No data about Coverage  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=tefra_xsdata&pullRequest=1037&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=tefra_xsdata&pullRequest=1037)


codecov[bot]: ## [Codecov](https://app.codecov.io/gh/tefra/xsdata/pull/1037?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Chris+Tsou) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 99.74%. Comparing base [(`d561ff5`)](https://app.codecov.io/gh/tefra/xsdata/commit/d561ff558ede1eb066823858f3addc5b3a392f64?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Chris+Tsou) to head [(`a5f36fa`)](https://app.codecov.io/gh/tefra/xsdata/pull/1037?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Chris+Tsou).


<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##              main    #1037      +/-   ##
===========================================
- Coverage   100.00%   99.74%   -0.26%     
===========================================
  Files          115      115              
  Lines         9282     9284       +2     
  Branches      2190     2190              
===========================================
- Hits          9282     9260      -22     
- Misses           0       20      +20     
- Partials         0        4       +4     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/tefra/xsdata/pull/1037?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Chris+Tsou).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Chris+Tsou).


Patch:
diff --git a/xsdata/formats/converter.py b/xsdata/formats/converter.py
index 9da4a854..cbdd9c66 100644
--- a/xsdata/formats/converter.py
+++ b/xsdata/formats/converter.py
@@ -2,6 +2,7 @@ import abc
 import base64
 import binascii
 import math
+import re
 from datetime import date, datetime, time
 from decimal import Decimal, InvalidOperation
 from enum import Enum, EnumMeta
@@ -424,6 +425,7 @@ class BytesConverter(Converter):
 
         try:
             fmt = kwargs.get("format")
+            value = re.sub(r"\s+", "", value)
 
             if fmt == "base16":
                 return binascii.unhexlify(value)


Test patch:
diff --git a/tests/formats/test_converter.py b/tests/formats/test_converter.py
index a1d0d6fe..6cf5f11a 100644
--- a/tests/formats/test_converter.py
+++ b/tests/formats/test_converter.py
@@ -1,9 +1,9 @@
 import sys
+import unittest
 from datetime import date, datetime, time
 from decimal import Decimal
 from enum import Enum
 from typing import Any
-from unittest import TestCase
 from xml.etree.ElementTree import QName
 
 import pytest
@@ -15,7 +15,7 @@ from xsdata.models.datatype import XmlDuration, XmlPeriod
 from xsdata.models.enums import UseType
 
 
-class ConverterFactoryTests(TestCase):
+class ConverterFactoryTests(unittest.TestCase):
     def test_deserialize(self):
         with self.assertRaises(ConverterError) as cm:
             converter.deserialize("a", [int])
@@ -106,7 +106,7 @@ class ConverterFactoryTests(TestCase):
         converter.unregister_converter(MinusOneInt)
 
 
-class StrConverterTests(TestCase):
+class StrConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(str)
 
@@ -119,7 +119,7 @@ class StrConverterTests(TestCase):
         self.assertEqual("1", self.converter.serialize(1))
 
 
-class StringConverterTests(TestCase):
+class StringConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(str)
 
@@ -134,7 +134,7 @@ class StringConverterTests(TestCase):
         self.assertEqual("1", self.converter.serialize(1))
 
 
-class BoolConverterTests(TestCase):
+class BoolConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(bool)
 
@@ -155,7 +155,7 @@ class BoolConverterTests(TestCase):
         self.assertEqual("false", self.converter.serialize(False))
 
 
-class IntConverterTests(TestCase):
+class IntConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(int)
 
@@ -171,7 +171,7 @@ class IntConverterTests(TestCase):
         self.assertEqual("2", self.converter.serialize(2))
 
 
-class FloatConverterTests(TestCase):
+class FloatConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(float)
 
@@ -191,14 +191,14 @@ class FloatConverterTests(TestCase):
         self.assertEqual("8.77683E-08", self.converter.serialize(float("8.77683E-8")))
 
 
-class BytesConverterTests(TestCase):
+class BytesConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(bytes)
 
     def test_serialize_with_base16_format(self):
         inputs = [
-            ("312d322d33", "312D322D33"),
-            ("312D322D33", "312D322D33"),
+            ("312d\n322d33", "312D322D33"),
+            ("312D \t322D33", "312D322D33"),
         ]
 
         for actual, expected in inputs:
@@ -245,7 +245,7 @@ class BytesConverterTests(TestCase):
             self.converter.deserialize("foo", format="foo")
 
 
-class DecimalConverterTests(TestCase):
+class DecimalConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(Decimal)
 
@@ -265,7 +265,7 @@ class DecimalConverterTests(TestCase):
         )
 
 
-class DateTimeConverterTests(TestCase):
+class DateTimeConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(datetime)
 
@@ -298,7 +298,7 @@ class DateTimeConverterTests(TestCase):
             self.converter.deserialize("21 June 2018 15:40", format="")
 
 
-class DateConverterTests(TestCase):
+class DateConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(date)
 
@@ -311,7 +311,7 @@ class DateConverterTests(TestCase):
         self.assertEqual(original, self.converter.serialize(obj, format=fmt))
 
 
-class TimeConverterTests(TestCase):
+class TimeConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(time)
 
@@ -324,7 +324,7 @@ class TimeConverterTests(TestCase):
         self.assertEqual(original, self.converter.serialize(obj, format=fmt))
 
 
-class QNameConverterTests(TestCase):
+class QNameConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(QName)
 
@@ -365,7 +365,7 @@ class EnumA(Enum):
     J = (Telephone(1, 2, 3), Telephone(4, 5, 6))
 
 
-class EnumConverterTests(TestCase):
+class EnumConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(Enum)
 
@@ -399,7 +399,7 @@ class EnumConverterTests(TestCase):
         self.assertEqual("2.1", self.converter.serialize(EnumA.C))
 
 
-class ProxyConverterTests(TestCase):
+class ProxyConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = ProxyConverter(lambda x: int(x))
 
@@ -413,7 +413,7 @@ class ProxyConverterTests(TestCase):
         self.assertEqual("1", self.converter.serialize(1))
 
 
-class XmlDurationConverterTests(TestCase):
+class XmlDurationConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(XmlDuration)
 
@@ -429,7 +429,7 @@ class XmlDurationConverterTests(TestCase):
         self.assertNotIsInstance(actual, XmlDuration)
 
 
-class XmlPeriodConverterTests(TestCase):
+class XmlPeriodConverterTests(unittest.TestCase):
     def setUp(self):
         self.converter = converter.type_converter(XmlPeriod)