Repository: aviramha/ormsgpack (language: python)

Issue:
BIGINT passthrough semantics
The new BIGINT_PASSTHROUGH in 1.1.0 seems to be a bit confusing.
Originally I thought it uses passthrough for the ints ormsgpack simply rejected before but it turns out that it also passes through u64 which previously was efficiently converted to the uint64 msgpack format.
I think the library should use msgpack formats if possible and the passthrough should extend that functionality instead of changing it.
So I propose that passthrough would happen only if the int can't fit into either int64 or uint64, which means it can't be converted to a native msgpack type and needs to be encoded.
BIGINT passthrough semantics
The new BIGINT_PASSTHROUGH in 1.1.0 seems to be a bit confusing.
Originally I thought it uses passthrough for the ints ormsgpack simply rejected before but it turns out that it also passes through u64 which previously was efficiently converted to the uint64 msgpack format.
I think the library should use msgpack formats if possible and the passthrough should extend that functionality instead of changing it.
So I propose that passthrough would happen only if the int can't fit into either int64 or uint64, which means it can't be converted to a native msgpack type and needs to be encoded.
BIGINT passthrough semantics
The new BIGINT_PASSTHROUGH in 1.1.0 seems to be a bit confusing.
Originally I thought it uses passthrough for the ints ormsgpack simply rejected before but it turns out that it also passes through u64 which previously was efficiently converted to the uint64 msgpack format.
I think the library should use msgpack formats if possible and the passthrough should extend that functionality instead of changing it.
So I propose that passthrough would happen only if the int can't fit into either int64 or uint64, which means it can't be converted to a native msgpack type and needs to be encoded.


Discussion / hints:
I understand your opinion. @TariqTNO any objection?
I have no objections on this. I didn't do this previously it was slightly more efficient to do the passthrough like this as it can be checked in the type determination already and kept the int conversion a bit more simple. But then again, I've got no objections to the suggested changes.
Great. @pejter do you mind making a PR?
I understand your opinion. @TariqTNO any objection?
I have no objections on this. I didn't do this previously it was slightly more efficient to do the passthrough like this as it can be checked in the type determination already and kept the int conversion a bit more simple. But then again, I've got no objections to the suggested changes.
Great. @pejter do you mind making a PR?
I understand your opinion. @TariqTNO any objection?
I have no objections on this. I didn't do this previously it was slightly more efficient to do the passthrough like this as it can be checked in the type determination already and kept the int conversion a bit more simple. But then again, I've got no objections to the suggested changes.
Great. @pejter do you mind making a PR?

Fix patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 788b3ac3..7459a265 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,7 @@
 - Updated README to include new options. by [@ThomasTNO](https://github.com/aviramha/ormsgpack/pull/70)
 - Updated dependencies
 - Renamed in `numpy.rs` `from_parent` to `to_children` to fix new lint rules
+- Extended int passthrough to support u64. by [@pejter](https://github.com/aviramha/ormsgpack/pull/77)
 ## 1.1.0 - 8/1/2022
 ### Added
 - Add optional passthrough for tuples. by [@TariqTNO](https://github.com/aviramha/ormsgpack/pull/64)
diff --git a/pyproject.toml b/pyproject.toml
index e75f3feb..8b4fd8a9 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -16,6 +16,7 @@ classifiers = [
     "Programming Language :: Python :: 3.7",
     "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: Implementation :: CPython",
     "Programming Language :: Python",
     "Programming Language :: Rust",
@@ -32,5 +33,5 @@ strip = true
 
 [tool.black]
 line-length = 88
-target-version = ['py36', 'py37', 'py38']
+target-version = ['py36', 'py37', 'py38', 'py39', 'py310']
 include = '\.pyi?$'
diff --git a/src/serialize/serializer.rs b/src/serialize/serializer.rs
index 61415dc5..12ae9012 100644
--- a/src/serialize/serializer.rs
+++ b/src/serialize/serializer.rs
@@ -71,7 +71,14 @@ pub fn pyobject_to_obtype(obj: *mut pyo3::ffi::PyObject, opts: Opt) -> ObType {
         } else if ob_type == BYTES_TYPE {
             ObType::Bytes
         } else if ob_type == INT_TYPE
-            && (opts & PASSTHROUGH_BIG_INT == 0 || ffi!(_PyLong_NumBits(obj)) <= 63)
+            && (opts & PASSTHROUGH_BIG_INT == 0
+                || ffi!(_PyLong_NumBits(obj)) <= {
+                    if ffi!(Py_SIZE(obj)) > 0 {
+                        64
+                    } else {
+                        63
+                    }
+                })
         {
             ObType::Int
         } else if ob_type == BOOL_TYPE {
@@ -118,7 +125,14 @@ pub fn pyobject_to_obtype_unlikely(obj: *mut pyo3::ffi::PyObject, opts: Opt) ->
             ObType::StrSubclass
         } else if opts & PASSTHROUGH_SUBCLASS == 0
             && is_subclass!(ob_type, Py_TPFLAGS_LONG_SUBCLASS)
-            && (opts & PASSTHROUGH_BIG_INT == 0 || ffi!(_PyLong_NumBits(obj)) <= 63)
+            && (opts & PASSTHROUGH_BIG_INT == 0
+                || ffi!(_PyLong_NumBits(obj)) <= {
+                    if ffi!(Py_SIZE(obj)) > 0 {
+                        64
+                    } else {
+                        63
+                    }
+                })
         {
             ObType::Int
         } else if opts & PASSTHROUGH_SUBCLASS == 0


Test patch:
diff --git a/tests/test_type.py b/tests/test_type.py
index 590bcf02..2653ba8d 100644
--- a/tests/test_type.py
+++ b/tests/test_type.py
@@ -203,9 +203,7 @@ def test_uint_64_passthrough(value):
     """
     uint 64-bit with passthrough
     """
-    result = ormsgpack.unpackb(ormsgpack.packb(value, option=ormsgpack.OPT_PASSTHROUGH_BIG_INT, default=lambda x: {"int": x.to_bytes(16, "little", signed=True)}))
-    assert list(result.keys()) == ["int"]
-    assert int.from_bytes(result["int"], "little", signed=True) == value
+    assert ormsgpack.unpackb(ormsgpack.packb(value, option=ormsgpack.OPT_PASSTHROUGH_BIG_INT)) == value
 
 
 @pytest.mark.parametrize("value", (18446744073709551616, -9223372036854775809))