Repository: meltano/sdk
Pull request #2755 (2024-11-11 19:42:25)

fix: Safely compare UUID replication keys with state bookmarks
This change will return a stringified value for an instance of uuid.UUID.

<!-- readthedocs-preview meltano-sdk start -->
----
📚 Documentation preview 📚: https://meltano-sdk--2755.org.readthedocs.build/en/2755/

<!-- readthedocs-preview meltano-sdk end -->

codspeed-hq[bot]: ## [CodSpeed Performance Report](https://codspeed.io/meltano/sdk/branches/nikzavada:patch-1)

### Merging #2755 will **not alter performance**

<sub>Comparing <code>nikzavada:patch-1</code> (4e4a613) with <code>main</code> (ae7516b)</sub>



### Summary

`✅ 6` untouched benchmarks





codecov[bot]: ## [Codecov](https://app.codecov.io/gh/meltano/sdk/pull/2755?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) Report
Attention: Patch coverage is `33.33333%` with `2 lines` in your changes missing coverage. Please review.
> Project coverage is 90.70%. Comparing base [(`ae7516b`)](https://app.codecov.io/gh/meltano/sdk/commit/ae7516b8a16d625c05258570c74e242f1cf64e88?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) to head [(`6a823a7`)](https://app.codecov.io/gh/meltano/sdk/commit/6a823a7753eee26c8cfd4b47e0f18cba1e8d7025?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).

| [Files with missing lines](https://app.codecov.io/gh/meltano/sdk/pull/2755?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) | Patch % | Lines |
|---|---|---|
| [singer\_sdk/helpers/\_typing.py](https://app.codecov.io/gh/meltano/sdk/pull/2755?src=pr&el=tree&filepath=singer_sdk%2Fhelpers%2F_typing.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano#diff-c2luZ2VyX3Nkay9oZWxwZXJzL190eXBpbmcucHk=) | 33.33% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/meltano/sdk/pull/2755?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #2755      +/-   ##
==========================================
- Coverage   90.74%   90.70%   -0.04%     
==========================================
  Files          62       62              
  Lines        5099     5102       +3     
  Branches      658      659       +1     
==========================================
+ Hits         4627     4628       +1     
- Misses        330      331       +1     
- Partials      142      143       +1     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/meltano/sdk/pull/2755?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).


Patch:
diff --git a/singer_sdk/helpers/_state.py b/singer_sdk/helpers/_state.py
index fd7dee37..565dbd51 100644
--- a/singer_sdk/helpers/_state.py
+++ b/singer_sdk/helpers/_state.py
@@ -218,6 +218,10 @@ def increment_state(
                 extra={"replication_key": replication_key},
             )
         progress_dict = stream_or_partition_state[PROGRESS_MARKERS]
+    # TODO: Instead of forcing all values to be JSON-compatible strings and hope
+    # we catch all cases, we should allow the stream to define how to
+    # the values from the state and the record should be pre-processed.
+    # https://github.com/meltano/sdk/issues/2753
     old_rk_value = to_json_compatible(progress_dict.get("replication_key_value"))
     new_rk_value = to_json_compatible(latest_record[replication_key])
 
diff --git a/singer_sdk/helpers/_typing.py b/singer_sdk/helpers/_typing.py
index b02ecb51..d3b629c1 100644
--- a/singer_sdk/helpers/_typing.py
+++ b/singer_sdk/helpers/_typing.py
@@ -6,6 +6,7 @@ import copy
 import datetime
 import logging
 import typing as t
+import uuid
 from enum import Enum
 from functools import lru_cache
 
@@ -42,6 +43,8 @@ def to_json_compatible(val: t.Any) -> t.Any:  # noqa: ANN401
     if isinstance(val, (datetime.datetime,)):
         # Make naive datetimes UTC
         return (val.replace(tzinfo=UTC) if val.tzinfo is None else val).isoformat("T")
+    if isinstance(val, (uuid.UUID,)):
+        return str(val)
     return val
 
 


Test patch:
diff --git a/tests/core/test_state_handling.py b/tests/core/test_state_handling.py
index f58a0128..85fd4812 100644
--- a/tests/core/test_state_handling.py
+++ b/tests/core/test_state_handling.py
@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import logging
+import uuid
 
 import pytest
 
@@ -155,3 +156,21 @@ def test_null_replication_value(caplog):
     ), "State should not be updated."
     assert caplog.records[0].levelname == "WARNING"
     assert "is null" in caplog.records[0].message
+
+
+def test_uuidv7_replication_value():
+    stream_state = {
+        "replication_key": "id",
+        "replication_key_value": "01931c63-b14e-7ff3-8621-e577ed392dc8",
+    }
+    new_string_val = "01931c63-b14e-7ff3-8621-e578edbca9a3"
+
+    _state.increment_state(
+        stream_state,
+        latest_record={"id": uuid.UUID(new_string_val)},
+        replication_key="id",
+        is_sorted=True,
+        check_sorted=True,
+    )
+
+    assert stream_state["replication_key_value"] == new_string_val