Repository: FreeOpcUa/opcua-asyncio

Issue:
asyncua.common.utils.NotEnoughData parsing an ExtensionObject with length -1
**Describe the bug** <br />
When I subscribe using this library, to my equipment's OPC/UA server (which is not based on FreeOpcUa), it sends a `PublishResponse` with a `NotificationData` containing an `ExtensionObject` of length -1 (0xffff).  As a result, I get an exception: `asyncua.common.utils.NotEnoughData: Not enough data left in buffer, request for 4, we have 0`

I believe the equipment's OPC/UA server is non-compliant - a length of -1 is not valid.  However, it appears to be a common enough bug with OPC/UA servers that the [OPC Foundation's reference implementation](https://github.com/OPCFoundation/UA-.NETStandard) have implemented a [interop fix](https://github.com/OPCFoundation/UA-.NETStandard/commit/a887f909f1d314cfa7c32989628afa754984c4f1).

Can we apply the same interop fix here?

I believe the change is simply to change [ua_binary.py;546-547](https://github.com/FreeOpcUa/opcua-asyncio/blob/e6d646d12f1db1e59229034439f9b0baf9d2da6e/asyncua/ua/ua_binary.py#L546-L547) from:

```python
        if length < 1:
            body = Buffer(b"")
```

...to...

```python
        if length == -1:
            # Interop fix for old OPC/UA implementations that omit to fill in the length
            body = data
        elif length < 1:
            body = Buffer(b"")
```

**To Reproduce**<br />
I can reproduce this using the built-in `uasubscribe` tool.  The OPC/UA server sends back an `ExtensionObject` of the following form:

```
0000   01 00 2b 03 01 ff ff ff ff 05 00 00 00 c9 00 00
0010   00 0d 0a 54 37 8e 3c 80 29 12 ab 53 95 db 01 80
0020   29 12 ab 53 95 db 01 c9 00 00 00 0d 0b 00 00 00
0030   00 98 40 8b 3f 40 15 25 ab 53 95 db 01 40 15 25
0040   ab 53 95 db 01 c9 00 00 00 0d 0b 00 00 00 a0 b7
0050   b3 91 3f d0 13 4b ab 53 95 db 01 d0 13 4b ab 53
0060   95 db 01 c9 00 00 00 0d 0b 00 00 00 a0 09 15 8c
0070   3f 60 9a 84 ab 53 95 db 01 60 9a 84 ab 53 95 db
0080   01 c9 00 00 00 0d 0b 00 00 00 e0 43 a6 91 3f d0
0090   86 a0 ab 53 95 db 01 d0 86 a0 ab 53 95 db 01 00
00a0   00 00 00
```

Octets 0-3 are the `TypeId`, octet 4 is the `EncodingMask` and octet 5-8 are the length (-1).

I see the following exception:

```
WARNING:asyncua.client.client:Requested session timeout to be 3600000ms, got 100000ms instead
WARNING:asyncua.client.client:Revised values returned differ from subscription values: CreateSubscriptionResult(SubscriptionId=4, RevisedPublishingInterval=500.0, RevisedLifetimeCount=9900, RevisedMaxKeepAliveCount=150)
Type Ctr-C to exit
ERROR:asyncua.client.ua_client.UaClient:Error parsing notification from server
Traceback (most recent call last):
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/client/ua_client.py", line 579, in publish
    response = struct_from_binary(ua.PublishResponse, data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 696, in struct_from_binary
    return _create_dataclass_deserializer(objtype)(data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 687, in decode
    kwargs[field.name] = deserialize_field(data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 687, in decode
    kwargs[field.name] = deserialize_field(data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 687, in decode
    kwargs[field.name] = deserialize_field(data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 254, in deserialize
    return list(unpack_array(data, length))
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 247, in <genexpr>
    return (deserialize_element(data) for _ in range(length))
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 542, in extensionobject_from_binary
    return from_binary(cls, body)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 629, in from_binary
    return _create_type_deserializer(uatype, type(None))(data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 687, in decode
    kwargs[field.name] = deserialize_field(data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 589, in _deserialize
    size = Primitives.Int32.unpack(data)
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/ua/ua_binary.py", line 131, in unpack
    return struct.unpack(self.format, data.read(self.size))[0]
  File "/home/mattwilliams/.local/lib/python3.8/site-packages/asyncua/common/utils.py", line 62, in read
    raise NotEnoughData(f"Not enough data left in buffer, request for {size}, we have {self._size}")
asyncua.common.utils.NotEnoughData: Not enough data left in buffer, request for 4, we have 0
```

**Expected behavior**<br />
I expect the `ExtensionObject` to be parsed correctly, by assuming that -1 means the rest of the data buffer.

**Version**<br />
Python-Version: 3.8<br />
opcua-asyncio Version (e.g. master branch, 0.9): 1.1.5

Hints from issue comments:
@oroulet: Fine enough. Send a MR

Solution patch:
diff --git a/asyncua/ua/ua_binary.py b/asyncua/ua/ua_binary.py
index adc9beec..fe3939d1 100644
--- a/asyncua/ua/ua_binary.py
+++ b/asyncua/ua/ua_binary.py
@@ -543,7 +543,10 @@ def extensionobject_from_binary(data: Buffer) -> ua.ExtensionObject:
     body = None
     if encoding & (1 << 0):
         length = Primitives.Int32.unpack(data)
-        if length < 1:
+        if length == -1:
+            # Interop fix for old OPC/UA implementations that fail to fill in the length
+            body = data
+        elif length < 1:
             body = Buffer(b"")
         else:
             body = data.copy(length)


Test patch:
diff --git a/tests/test_unit.py b/tests/test_unit.py
index e339dc56..c071fe13 100644
--- a/tests/test_unit.py
+++ b/tests/test_unit.py
@@ -558,6 +558,22 @@ def test_unknown_extension_object():
     assert obj2.Body == b"example of data in custom format"
 
 
+def test_extension_object_missing_length():
+    obj = ua.DataChangeNotification()
+    binary = bytearray(extensionobject_to_binary(obj))
+
+    # Patch the binary to replace the correct length with -1 (i.e. missing),
+    # which some old OPC/UA implementations mistakenly do.
+    # Bytes 0-4 are the node ID, byte 4 is the encoding mask, so the length
+    # is bytes 5-8.
+    binary[5:9] = b"\xff\xff\xff\xff"
+
+    obj2 = extensionobject_from_binary(ua.utils.Buffer(bytes(binary)))
+    assert type(obj) is type(obj2)
+    assert obj.MonitoredItems == obj2.MonitoredItems
+    assert obj.DiagnosticInfos == obj2.DiagnosticInfos
+
+
 def test_datetime():
     now = datetime.now(timezone.utc)
     epch = ua.datetime_to_win_epoch(now)