Repository: meltano/sdk

Issue:
bug: ValueError: Out of range float values are not JSON compliant
### Singer SDK Version

0.34.1

### Is this a regression?

- [ ] Yes

### Python Version

3.9

### Bug scope

Taps (catalog, state, etc.)

### Operating System

MacOS

### Description

I get `ValueError: Out of range float values are not JSON compliant`  when trying to sync Amazon Document DB collection. 

The probably reason is that `simple_json.dumps()` has `allow_nan=False` default, while standard `dumps()` function from standard `json` library has `allow_nan=True`. 

As there is no way to pass configuration from outside could we add this functionality? I can introduce relevant fix. 

### Code

_No response_

Discussion:
@edgarrmondragon: Hi @dgawlowsky, thanks for logging!

I think having `allow_nan` by default would violate the interoperability principle in the Singer ecosystem since it's specific to Python's JSON implementation: while most taps and targets are written in Python, they could be programmed in any language.

There's also the question of how the `jsonchema` library that we use for validation on the target side would treat the NaN values, although it's now possible to disable validation on the user end (631d5df03bd3afef40763ac01b9f169169ec4135), if they're fine with moving NaNs around and skipping schema validation.

Do start a PR and we can discuss the details there!

Related: https://github.com/meltano/sdk/issues/2211

@dgawlowsky: Hi @edgarrmondragon - thanks for prompt response again. Opened PR with this one line change. Let's discuss there. @menzenski: Also running into this issue, and also in an AWS DocumentDB context.@edgarrmondragon: @menzenski the situation should've improved somewhat with [v0.45.6](https://github.com/meltano/sdk/releases/tag/v0.45.6), which shipped

- https://github.com/meltano/sdk/issues/2961

Can you confirm if that's enough for your use case or do you need to actually communicate the NaN values to the target somehow?@menzenski: @edgarrmondragon I updated my tap-mongodb fork from 0.41.0 to 0.45.11 today. The changes you mention had no effect for me, but looking at the code I think that's expected - the issues I am experiencing are in the values of a field that I declare as just `dict[str, Any]` in the schema. I think the changes in 0.45.6 only apply automatically to fields that are typed as `float` within the tap schema. (So, I think I need to basically make that same change within my tap's code. I'm working on that today).@edgarrmondragon: Ok, I'm now convinced that we should just force those values to be encoded as `null` in the Singer message across the board.

Does that make sense for your use case @menzenski?@menzenski: > Does that make sense for your use case @menzenski?

Yes - for my use case it would be awesome to have it handled globally like that@menzenski: @edgarrmondragon thanks for making that change so fast. I've tested meltano/sdk commit b3a7cb303ba0c6bc988e12fa3d6998e438efcf26 in my tap and it solves this for me. very very appreciated!

Solution patch:
diff --git a/singer_sdk/singerlib/json.py b/singer_sdk/singerlib/json.py
index 1c9855e3..2a4e60f5 100644
--- a/singer_sdk/singerlib/json.py
+++ b/singer_sdk/singerlib/json.py
@@ -59,5 +59,6 @@ def serialize_json(obj: object, **kwargs: t.Any) -> str:
         use_decimal=True,
         default=_default_encoding,
         separators=(",", ":"),
+        ignore_nan=True,
         **kwargs,
     )


Test patch:
diff --git a/tests/singerlib/encoding/test_simple.py b/tests/singerlib/encoding/test_simple.py
index 7ce7f5bd..098c7276 100644
--- a/tests/singerlib/encoding/test_simple.py
+++ b/tests/singerlib/encoding/test_simple.py
@@ -104,3 +104,18 @@ def test_write_message():
     assert out.getvalue() == (
         '{"type":"RECORD","stream":"test","record":{"id":1,"name":"test"}}\n'
     )
+
+
+@pytest.mark.parametrize("float_value", [float("nan"), float("inf"), float("-inf")])
+def test_encode_nan_values(float_value):
+    writer = SimpleSingerWriter()
+    message = RecordMessage(
+        stream="test",
+        record={"id": 1, "name": float_value},
+    )
+    with redirect_stdout(io.StringIO()) as out:
+        writer.write_message(message)
+
+    assert out.getvalue() == (
+        '{"type":"RECORD","stream":"test","record":{"id":1,"name":null}}\n'
+    )