Repository: meltano/sdk
Pull request #2780 (2024-11-28 18:46:08)

feat(taps): Numeric values are now parsed as `decimal.Decimal` in REST and GraphQL stream responses
This means under normal circumstances, `decimal.Decimal` would be used across the board:

1. A float value of `"value": 3.14` in the API response is parsed as `Decimal("3.14")`
2. The `Decimal("3.14")` value is serialized to JSON as `value: 3.14` in the tap output
3. The float value of `"value": 3.14` in the `RECORD` Singer message will be parsed as `Decimal("3.14")` by the target.

That should ensure we preserve precision through the pipeline.

> [!CAUTION]
> Now, this _could_ be a breaking change if folks are doing floating-point math with these values, but I think it's rare enough that we can make do with a call-out in the release notes.

<!-- readthedocs-preview meltano-sdk start -->
----
📚 Documentation preview 📚: https://meltano-sdk--2780.org.readthedocs.build/en/2780/

<!-- readthedocs-preview meltano-sdk end -->

codspeed-hq[bot]: ## [CodSpeed Performance Report](https://codspeed.io/meltano/sdk/branches/edgarrmondragon/feat/rest-number-decimal)

### Merging #2780 will **not alter performance**

<sub>Comparing <code>edgarrmondragon/feat/rest-number-decimal</code> (a92b8a7) with <code>main</code> (8474a24)</sub>



### Summary

`✅ 6` untouched benchmarks  




codecov[bot]: ## [Codecov](https://app.codecov.io/gh/meltano/sdk/pull/2780?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 90.88%. Comparing base [(`8474a24`)](https://app.codecov.io/gh/meltano/sdk/commit/8474a24a2c0a1855619581b9b4184cf9ed3c222e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano) to head [(`a92b8a7`)](https://app.codecov.io/gh/meltano/sdk/commit/a92b8a7f7ece7358448d2f87954b5219a15e5026?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).
> Report is 1 commits behind head on main.

<details><summary>Additional details and impacted files</summary>


```diff
@@           Coverage Diff           @@
##             main    #2780   +/-   ##
=======================================
  Coverage   90.88%   90.88%           
=======================================
  Files          62       62           
  Lines        5165     5166    +1     
  Branches      667      667           
=======================================
+ Hits         4694     4695    +1     
  Misses        330      330           
  Partials      141      141           
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/meltano/sdk/pull/2780?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=meltano).


Patch:
diff --git a/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/graphql-client.py b/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/graphql-client.py
index 289199b1..efdaa0c7 100644
--- a/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/graphql-client.py
+++ b/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/graphql-client.py
@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import decimal
 import typing as t
 
 import requests  # noqa: TCH002
@@ -61,7 +62,7 @@ class {{ cookiecutter.source_name }}Stream({{ cookiecutter.stream_type }}Stream)
             Each record from the source.
         """
         # TODO: Parse response body and return a set of records.
-        resp_json = response.json()
+        resp_json = response.json(parse_float=decimal.Decimal)
         yield from resp_json.get("<TODO>")
 
     def post_process(
diff --git a/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/rest-client.py b/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/rest-client.py
index 35a53303..5eb38aaf 100644
--- a/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/rest-client.py
+++ b/cookiecutter/tap-template/{{cookiecutter.tap_id}}/{{cookiecutter.library_name}}/rest-client.py
@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import decimal
 import typing as t
 {% if cookiecutter.auth_method in ("OAuth2", "JWT") -%}
 from functools import cached_property
@@ -204,7 +205,10 @@ class {{ cookiecutter.source_name }}Stream({{ cookiecutter.stream_type }}Stream)
             Each record from the source.
         """
         # TODO: Parse response body and return a set of records.
-        yield from extract_jsonpath(self.records_jsonpath, input=response.json())
+        yield from extract_jsonpath(
+            self.records_jsonpath,
+            input=response.json(parse_float=decimal.Decimal),
+        )
 
     def post_process(
         self,
diff --git a/singer_sdk/streams/rest.py b/singer_sdk/streams/rest.py
index dcfdf710..f4f3e7fc 100644
--- a/singer_sdk/streams/rest.py
+++ b/singer_sdk/streams/rest.py
@@ -4,6 +4,7 @@ from __future__ import annotations
 
 import abc
 import copy
+import decimal
 import logging
 import sys
 import typing as t
@@ -779,7 +780,10 @@ class RESTStream(_HTTPStream, t.Generic[_TToken], metaclass=abc.ABCMeta):
         Yields:
             One item for every item found in the response.
         """
-        yield from extract_jsonpath(self.records_jsonpath, input=response.json())
+        yield from extract_jsonpath(
+            self.records_jsonpath,
+            input=response.json(parse_float=decimal.Decimal),
+        )
 
     def get_new_paginator(self) -> BaseAPIPaginator:
         """Get a fresh paginator for this API endpoint.


Test patch:
diff --git a/tests/core/test_streams.py b/tests/core/test_streams.py
index 6dabf1f2..562621a5 100644
--- a/tests/core/test_streams.py
+++ b/tests/core/test_streams.py
@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import datetime
+import decimal
 import logging
 import typing as t
 import urllib.parse
@@ -568,6 +569,37 @@ def test_non_json_payload(tap: Tap, requests_mock: requests_mock.Mocker):
     ]
 
 
+def test_parse_response(tap: Tap):
+    content = """[
+        {"id": 1, "value": 3.14159},
+        {"id": 2, "value": 2.71828}
+    ]
+    """
+
+    class MyRESTStream(RESTStream):
+        url_base = "https://example.com"
+        path = "/dummy"
+        name = "dummy"
+        schema = {  # noqa: RUF012
+            "type": "object",
+            "properties": {
+                "id": {"type": "integer"},
+                "value": {"type": "number"},
+            },
+        }
+
+    stream = MyRESTStream(tap=tap)
+
+    response = requests.Response()
+    response._content = content.encode("utf-8")
+
+    records = list(stream.parse_response(response))
+    assert records == [
+        {"id": 1, "value": decimal.Decimal("3.14159")},
+        {"id": 2, "value": decimal.Decimal("2.71828")},
+    ]
+
+
 @pytest.mark.parametrize(
     "input_catalog,selection",
     [