Repository: meltano/sdk
Pull request #2613 (2024-08-20 22:55:00)

fix: Mismatch between timezone-aware and naive datetimes in start date and bookmarks is now correctly handled
Related:

- Closes https://github.com/meltano/sdk/issues/2612
- https://github.com/meltano/sdk/issues/2612

<!-- readthedocs-preview meltano-sdk start -->
----
📚 Documentation preview 📚: https://meltano-sdk--2613.org.readthedocs.build/en/2613/

<!-- readthedocs-preview meltano-sdk end -->

codspeed-hq[bot]: ## [CodSpeed Performance Report](https://codspeed.io/meltano/sdk/branches/2612-bug-tap-crashes-when-start_date-and-replication-value-are-not-both-tz-aware-or-viceversa)

### Merging #2613 will **not alter performance**

<sub>Comparing <code>2612-bug-tap-crashes-when-start_date-and-replication-value-are-not-both-tz-aware-or-viceversa</code> (148dec7) with <code>main</code> (162b910)</sub>



### Summary

`✅ 6` untouched benchmarks





codecov[bot]: ## [Codecov](https://app.codecov.io/gh/meltano/sdk/pull/2613?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 89.71%. Comparing base [(`162b910`)](https://app.codecov.io/gh/meltano/sdk/commit/162b9101511d9e2649001556801b5e181377e1dc?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) to head [(`cf69c20`)](https://app.codecov.io/gh/meltano/sdk/commit/cf69c205790541043f277902d89b5e4215f115ab?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #2613      +/-   ##
==========================================
+ Coverage   89.70%   89.71%   +0.01%     
==========================================
  Files          58       58              
  Lines        4848     4853       +5     
  Branches      947      948       +1     
==========================================
+ Hits         4349     4354       +5     
  Misses        349      349              
  Partials      150      150              
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/meltano/sdk/pull/2613?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).


Patch:
diff --git a/singer_sdk/streams/core.py b/singer_sdk/streams/core.py
index 3f8ca6a7..739c254d 100644
--- a/singer_sdk/streams/core.py
+++ b/singer_sdk/streams/core.py
@@ -365,6 +365,23 @@ class Stream(metaclass=abc.ABCMeta):  # noqa: PLR0904
         state = self.get_context_state(context)
         write_replication_key_signpost(state, value)
 
+    def _parse_datetime(self, value: str) -> datetime.datetime:  # noqa: PLR6301
+        """Parse a datetime string.
+
+        Args:
+            value: The datetime string.
+
+        Returns:
+            The parsed datetime, timezone-aware preferred.
+        """
+        result = datetime_fromisoformat(value)
+
+        # Ensure datetime is timezone-aware
+        if not result.tzinfo:
+            result = result.astimezone()
+
+        return result
+
     def compare_start_date(self, value: str, start_date_value: str) -> str:
         """Compare a bookmark value to a start date and return the most recent value.
 
@@ -384,7 +401,7 @@ class Stream(metaclass=abc.ABCMeta):  # noqa: PLR0904
             The most recent value between the bookmark and start date.
         """
         if self.is_timestamp_replication_key:
-            return max(value, start_date_value, key=datetime_fromisoformat)
+            return max(value, start_date_value, key=self._parse_datetime)
 
         return value
 


Test patch:
diff --git a/tests/core/test_streams.py b/tests/core/test_streams.py
index 592f921e..1a25defa 100644
--- a/tests/core/test_streams.py
+++ b/tests/core/test_streams.py
@@ -141,6 +141,18 @@ def test_stream_apply_catalog(stream: Stream):
             parse(CONFIG_START_DATE).replace(tzinfo=datetime.timezone.utc),
             id="datetime-repl-key-old-bookmark",
         ),
+        pytest.param(
+            "test",
+            None,
+            "2021-01-02T00:00:00-08:00",
+            datetime.datetime(
+                2021,
+                1,
+                2,
+                tzinfo=datetime.timezone(datetime.timedelta(hours=-8)),
+            ),
+            id="datetime-repl-key-recent-bookmark-tz-aware",
+        ),
         pytest.param(
             "unix_ts",
             None,