Repository: SAP/python-pyodata

Issue:
Malformed value for primitive Edm.DateTimeOffset type
We are using pyodata v1.7.1 the whole time and I'm tried to update to the latest v1.10.0 version. But now I get this error when querying data from the odata service:

```
Traceback (most recent call last):
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/pyodata/v2/model.py", line 534, in from_json
    milliseconds_since_epoch = matches.group('milliseconds_since_epoch')
AttributeError: 'NoneType' object has no attribute 'group'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/prefect/engine.py", line 1192, in orchestrate_task_run
    result = await run_sync(task.fn, *args, **kwargs)
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/prefect/utilities/asyncutils.py", line 57, in run_sync_in_worker_thread
    return await anyio.to_thread.run_sync(call, cancellable=True)
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/workspace/odatavenv/flow_utilities/prefect/tasks/odata.py", line 60, in fetch_odata
    for cnt, batch in enumerate(batches, start=1):
  File "/workspace/odatavenv/flow_utilities/api/odata.py", line 114, in query_all
    yield list(self.query(skip, top))
  File "/workspace/odatavenv/flow_utilities/api/odata.py", line 98, in query
    response = self._entity_service().skip(skip).top(top).execute()
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/pyodata/v2/service.py", line 349, in execute
    return self._call_handler(response)
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/pyodata/v2/service.py", line 362, in _call_handler
    return self._handler(response)
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/pyodata/v2/service.py", line 1509, in get_entities_handler
    entity = EntityProxy(self._service, self._entity_set, self._entity_set.entity_type, props)
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/pyodata/v2/service.py", line 814, in __init__
    self._cache[type_proprty.name] = type_proprty.from_json(proprties[type_proprty.name])
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/pyodata/v2/model.py", line 872, in from_json
    return self.typ.traits.from_json(value)
  File "/workspace/tmp/venv/odatavenv/lib/python3.10/site-packages/pyodata/v2/model.py", line 537, in from_json
    raise PyODataModelError(
pyodata.exceptions.PyODataModelError: Malformed value /Date(1599659339088)/ for primitive Edm.DateTimeOffset type. Expected format is /Date(<ticks>±<offset>)/
```

Maybe this is related to our instance because they are uncompatible dates (see https://github.com/SAP/python-pyodata/issues/143). 

Discussion:
codecov-commenter: # [Codecov](https://codecov.io/gh/SAP/python-pyodata/pull/232?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) Report
Base: **92.34**% // Head: **92.31**% // Decreases project coverage by **`-0.02%`** :warning:
> Coverage data is based on head [(`2865f22`)](https://codecov.io/gh/SAP/python-pyodata/pull/232?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) compared to base [(`5d14908`)](https://codecov.io/gh/SAP/python-pyodata/commit/5d1490871f4b824a82dd6eaa148444a99ea4f47d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).
> Patch coverage: 100.00% of modified lines in pull request are covered.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master     #232      +/-   ##
==========================================
- Coverage   92.34%   92.31%   -0.03%     
==========================================
  Files           6        6              
  Lines        2847     2849       +2     
==========================================
+ Hits         2629     2630       +1     
- Misses        218      219       +1     
```


| [Impacted Files](https://codecov.io/gh/SAP/python-pyodata/pull/232?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | |
|---|---|---|
| [pyodata/v2/model.py](https://codecov.io/gh/SAP/python-pyodata/pull/232/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cHlvZGF0YS92Mi9tb2RlbC5weQ==) | `93.29% <100.00%> (-0.05%)` | :arrow_down: |

Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)

</details>

[:umbrella: View full report at Codecov](https://codecov.io/gh/SAP/python-pyodata/pull/232?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).   
:loudspeaker: Do you have feedback about the report comment? [Let us know in this issue](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None).

jfilak: LGTM

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 0f876b9..7175a5d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [Unreleased]
 
+- model: fix edge case for Edm.DateTimeOffset.from_json() without offset - Petr Hanak
+
 ## [1.10.0]
 
 ### Added
diff --git a/pyodata/v2/model.py b/pyodata/v2/model.py
index 065f93e..4f15c84 100644
--- a/pyodata/v2/model.py
+++ b/pyodata/v2/model.py
@@ -529,6 +529,12 @@ class EdmDateTimeOffsetTypTraits(EdmPrefixedTypTraits):
         return f'/Date({ticks}{offset_in_minutes:+05})/'
 
     def from_json(self, value):
+        # special edge case:
+        # datetimeoffset'yyyy-mm-ddThh:mm[:ss]' = defaults to UTC, when offset value is not provided in responde data by service but the metadata is EdmDateTimeOffset
+        # intentionally just for from_json, generation of to_json should always provide timezone info
+        if re.match(r"^/Date\((?P<milliseconds_since_epoch>-?\d+)\)/$", value):
+            value = value.replace(')', '+0000)')
+
         matches = re.match(r"^/Date\((?P<milliseconds_since_epoch>-?\d+)(?P<offset_in_minutes>[+-]\d+)\)/$", value)
         try:
             milliseconds_since_epoch = matches.group('milliseconds_since_epoch')


Test patch:
diff --git a/tests/test_model_v2.py b/tests/test_model_v2.py
index 5bdef48..258dde7 100644
--- a/tests/test_model_v2.py
+++ b/tests/test_model_v2.py
@@ -690,6 +690,9 @@ def test_traits_datetimeoffset(type_date_time_offset):
 def test_traits_datetimeoffset_to_literal(type_date_time_offset):
     """Test Edm.DateTimeOffset trait: Python -> literal"""
 
+    testdate = datetime(1, 1, 1, 0, 0, 0, 0, tzinfo=timezone.utc)
+    assert type_date_time_offset.traits.to_literal(testdate) == "datetimeoffset'0001-01-01T00:00:00+00:00'"
+
     testdate = datetime(2005, 1, 28, 18, 30, 44, 123456, tzinfo=timezone(timedelta(hours=3, minutes=40)))
     assert type_date_time_offset.traits.to_literal(testdate) == "datetimeoffset'2005-01-28T18:30:44.123456+03:40'"
 
@@ -746,7 +749,7 @@ def test_traits_datetimeoffset_from_invalid_literal(type_date_time_offset):
     assert str(e_info.value).startswith('Cannot decode datetimeoffset from value xyz')
 
 
-def test_traits_datetimeoffset_from_odata(type_date_time_offset):
+def test_traits_datetimeoffset_from_json(type_date_time_offset):
     """Test Edm.DateTimeOffset trait: OData -> Python"""
 
     # parsing full representation
@@ -768,6 +771,14 @@ def test_traits_datetimeoffset_from_odata(type_date_time_offset):
     assert testdate.microsecond == 0
     assert testdate.tzinfo == timezone(-timedelta(minutes=5))
 
+    # parsing special edge case with no offset provided, defaults to UTC
+    testdate = type_date_time_offset.traits.from_json("/Date(217567986000)/")
+    assert testdate.year == 1976
+    assert testdate.minute == 33
+    assert testdate.second == 6
+    assert testdate.microsecond == 0
+    assert testdate.tzinfo == timezone.utc
+
     # parsing below lowest value with workaround
     pyodata.v2.model.FIX_SCREWED_UP_MINIMAL_DATETIME_VALUE = True
     testdate = type_date_time_offset.traits.from_json("/Date(-62135596800001+0001)/")