Repository: apache/airflow

Issue:
Airflow Stable REST API [GET api/v1/pools] issue
**Apache Airflow version**: v2.0.2

**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): N/A

**Environment**: AWS

- **Cloud provider or hardware configuration**: AWS EC2 Instance
- **OS** (e.g. from /etc/os-release): Ubuntu Server 20.04 LTS
- **Kernel** (e.g. `uname -a`): Linux ip-172-31-23-31 5.4.0-1048-aws #50-Ubuntu SMP Mon May 3 21:44:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
- **Install tools**: 
- **Others**: Python version: 3.8.5

**What happened**: Using Airflow Stable REST API [GET api/v1/pools] results in Ooops!  This only occurs when the pools have "Running Slots".  If no tasks are running and the slots are zero, then it works just fine.

<!-- (please include exact error messages if you can) -->

Something bad has happened.
Please consider letting us know by creating a bug report using GitHub.

Python version: 3.8.5
Airflow version: 2.0.2
Node: ip-172-31-23-31.ec2.internal
-------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tool/gto_env/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/tool/gto_env/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/tool/gto_env/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/tool/gto_env/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/tool/gto_env/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/tool/gto_env/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/decorators/decorator.py", line 48, in wrapper
    response = function(request)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/decorators/uri_parsing.py", line 144, in wrapper
    response = function(request)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/decorators/validation.py", line 384, in wrapper
    return function(request)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/decorators/response.py", line 104, in wrapper
    return _wrapper(request, response)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/decorators/response.py", line 89, in _wrapper
    self.operation.api.get_connexion_response(response, self.mimetype)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/apis/abstract.py", line 351, in get_connexion_response
    response = cls._response_from_handler(response, mimetype)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/apis/abstract.py", line 331, in _response_from_handler
    return cls._build_response(mimetype=mimetype, data=response, extra_context=extra_context)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/apis/flask_api.py", line 173, in _build_response
    data, status_code, serialized_mimetype = cls._prepare_body_and_status_code(data=data, mimetype=mimetype, status_code=status_code, extra_context=extra_context)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/apis/abstract.py", line 403, in _prepare_body_and_status_code
    body, mimetype = cls._serialize_data(data, mimetype)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/apis/flask_api.py", line 190, in _serialize_data
    body = cls.jsonifier.dumps(data)
  File "/home/tool/gto_env/lib/python3.8/site-packages/connexion/jsonifier.py", line 44, in dumps
    return self.json.dumps(data, **kwargs) + '\n'
  File "/home/tool/gto_env/lib/python3.8/site-packages/flask/json/__init__.py", line 211, in dumps
    rv = _json.dumps(obj, **kwargs)
  File "/usr/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/usr/lib/python3.8/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/usr/lib/python3.8/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/home/tool/gto_env/lib/python3.8/site-packages/airflow/utils/json.py", line 74, in _default
    raise TypeError(f"Object of type '{obj.__class__.__name__}' is not JSON serializable")
TypeError: Object of type 'Decimal' is not JSON serializable

**What you expected to happen**:  I expect the appropriate JSON response

<!-- What do you think went wrong? -->

**How to reproduce it**:
On an Airflow instance, run some tasks and while the tasks are running query the pools via the API.  NOTE: That you have to query the specific pool that has tasks running, if you avoid the pool using limit and/or offset then the issue will not occur.  You must try to return a pool with running_slots > 0

**Anything else we need to know**:

Not really



Discussion:
Thanks for opening your first issue here! Be sure to follow the issue template!


Resolving patch:
diff --git a/airflow/utils/json.py b/airflow/utils/json.py
--- a/airflow/utils/json.py
+++ b/airflow/utils/json.py
@@ -17,6 +17,7 @@
 # under the License.
 
 from datetime import date, datetime
+from decimal import Decimal
 
 import numpy as np
 from flask.json import JSONEncoder
@@ -37,12 +38,19 @@ def __init__(self, *args, **kwargs):
         self.default = self._default
 
     @staticmethod
-    def _default(obj):
+    def _default(obj):  # pylint: disable=too-many-return-statements
         """Convert dates and numpy objects in a json serializable format."""
         if isinstance(obj, datetime):
             return obj.strftime('%Y-%m-%dT%H:%M:%SZ')
         elif isinstance(obj, date):
             return obj.strftime('%Y-%m-%d')
+        elif isinstance(obj, Decimal):
+            _, _, exponent = obj.as_tuple()
+            if exponent >= 0:  # No digits after the decimal point.
+                return int(obj)
+            # Technically lossy due to floating point errors, but the best we
+            # can do without implementing a custom encode function.
+            return float(obj)
         elif isinstance(
             obj,
             (


Test patch:
diff --git a/tests/utils/test_json.py b/tests/utils/test_json.py
--- a/tests/utils/test_json.py
+++ b/tests/utils/test_json.py
@@ -16,11 +16,13 @@
 # specific language governing permissions and limitations
 # under the License.
 
+import decimal
 import json
 import unittest
 from datetime import date, datetime
 
 import numpy as np
+import parameterized
 import pytest
 
 from airflow.utils import json as utils_json
@@ -34,6 +36,12 @@ def test_encode_datetime(self):
     def test_encode_date(self):
         assert json.dumps(date(2017, 5, 21), cls=utils_json.AirflowJsonEncoder) == '"2017-05-21"'
 
+    @parameterized.parameterized.expand(
+        [("1", "1"), ("52e4", "520000"), ("2e0", "2"), ("12e-2", "0.12"), ("12.34", "12.34")],
+    )
+    def test_encode_decimal(self, expr, expected):
+        assert json.dumps(decimal.Decimal(expr), cls=utils_json.AirflowJsonEncoder) == expected
+
     def test_encode_numpy_int(self):
         assert json.dumps(np.int32(5), cls=utils_json.AirflowJsonEncoder) == '5'