Repository: meltano/sdk
Pull request #2756 (2024-11-11 20:00:00)

fix: Safely compare UUID replication keys with state bookmarks
A backport of https://github.com/meltano/sdk/pull/2755 for v0.42.


<!-- readthedocs-preview meltano-sdk start -->
----
📚 Documentation preview 📚: https://meltano-sdk--2756.org.readthedocs.build/en/2756/

<!-- readthedocs-preview meltano-sdk end -->

codspeed-hq[bot]: ## [CodSpeed Performance Report](https://codspeed.io/meltano/sdk/branches/2755-backport)

### Merging #2756 will **not alter performance**

<sub>Comparing <code>2755-backport</code> (9c80131) with <code>v0.42</code> (3ebd4dd)</sub>



### Summary

`✅ 6` untouched benchmarks





codecov[bot]: ## [Codecov](https://app.codecov.io/gh/meltano/sdk/pull/2756?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 90.74%. Comparing base [(`3ebd4dd`)](https://app.codecov.io/gh/meltano/sdk/commit/3ebd4dd16ccdc7a150de2e71f0c6863a3408f55c?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) to head [(`9c80131`)](https://app.codecov.io/gh/meltano/sdk/commit/9c8013118a62d9a86e2348815f816e814d59a31b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##            v0.42    #2756   +/-   ##
=======================================
  Coverage   90.74%   90.74%           
=======================================
  Files          62       62           
  Lines        5099     5102    +3     
  Branches      658      659    +1     
=======================================
+ Hits         4627     4630    +3     
  Misses        330      330           
  Partials      142      142           
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/meltano/sdk/pull/2756?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).


Patch:
diff --git a/singer_sdk/helpers/_state.py b/singer_sdk/helpers/_state.py
index fd7dee37..565dbd51 100644
--- a/singer_sdk/helpers/_state.py
+++ b/singer_sdk/helpers/_state.py
@@ -218,6 +218,10 @@ def increment_state(
                 extra={"replication_key": replication_key},
             )
         progress_dict = stream_or_partition_state[PROGRESS_MARKERS]
+    # TODO: Instead of forcing all values to be JSON-compatible strings and hope
+    # we catch all cases, we should allow the stream to define how to
+    # the values from the state and the record should be pre-processed.
+    # https://github.com/meltano/sdk/issues/2753
     old_rk_value = to_json_compatible(progress_dict.get("replication_key_value"))
     new_rk_value = to_json_compatible(latest_record[replication_key])
 
diff --git a/singer_sdk/helpers/_typing.py b/singer_sdk/helpers/_typing.py
index b02ecb51..d3b629c1 100644
--- a/singer_sdk/helpers/_typing.py
+++ b/singer_sdk/helpers/_typing.py
@@ -6,6 +6,7 @@ import copy
 import datetime
 import logging
 import typing as t
+import uuid
 from enum import Enum
 from functools import lru_cache
 
@@ -42,6 +43,8 @@ def to_json_compatible(val: t.Any) -> t.Any:  # noqa: ANN401
     if isinstance(val, (datetime.datetime,)):
         # Make naive datetimes UTC
         return (val.replace(tzinfo=UTC) if val.tzinfo is None else val).isoformat("T")
+    if isinstance(val, (uuid.UUID,)):
+        return str(val)
     return val
 
 


Test patch:
diff --git a/tests/core/test_state_handling.py b/tests/core/test_state_handling.py
index f58a0128..85fd4812 100644
--- a/tests/core/test_state_handling.py
+++ b/tests/core/test_state_handling.py
@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import logging
+import uuid
 
 import pytest
 
@@ -155,3 +156,21 @@ def test_null_replication_value(caplog):
     ), "State should not be updated."
     assert caplog.records[0].levelname == "WARNING"
     assert "is null" in caplog.records[0].message
+
+
+def test_uuidv7_replication_value():
+    stream_state = {
+        "replication_key": "id",
+        "replication_key_value": "01931c63-b14e-7ff3-8621-e577ed392dc8",
+    }
+    new_string_val = "01931c63-b14e-7ff3-8621-e578edbca9a3"
+
+    _state.increment_state(
+        stream_state,
+        latest_record={"id": uuid.UUID(new_string_val)},
+        replication_key="id",
+        is_sorted=True,
+        check_sorted=True,
+    )
+
+    assert stream_state["replication_key_value"] == new_string_val