Repository: mathesar-foundation/mathesar

Issue:
Number scale not persisting
## Reproduce

1. In the front end, edit the DB options for a Number column.
1. Set "Number Type" to "Decimal". Set "Decimal Places" to "3". Click Save.
1. Observe a PATCH request to the columns API which specifies `type_options.scale` to `2`. Good.
1. In the response of this request (and also in the response of the subsequent GET request to the columns endpoint), expect the column to show `type_options` with an object containing a `scale` of `2`.
1. Instead, observe `type_options` to be `null`.



Discussion:
`Numeric` [column type signature](https://www.postgresql.org/docs/current/datatype-numeric.html#DATATYPE-INT) accepts `scale` along with `precision`. I don't think it is possible to specify `scale` without `precision`.

Also, it is not possible for the backend to set a default value as the backend is not aware of the possible `type_options` for a particular type. So it would end up setting a default value for all the types. We could make it work like `display_options` but that would take a bit of time, so it would be better for the frontend to set a default value for precision.
I am going to re-open this issue and mark as draft, because `scale` is also required for money type. However, there is no point in showing precision to the user, and the backend must assume a default value in that case (preferably the max value allowed).

It is much better to set a default value on the backend rather than on the frontend.
@pavish, Mukesh and I have been discussing this privately which has seemed appropriate to me since we're trying to encourage faster work and the responsibility for Numbers rests squarely within our two-person team for this sprint. We're still trying to sort out how we want to handle some of the challenges here, and at this point I'm planning to raise some discussion points with the whole core team during the cool-down period. 
@seancolsen That's cool. I just wanted this issue to be open and marked as draft instead of closing so that the issue can be referenced for later discussions, since I might need the scale and precision options for grouping (I haven't checked if such a grouping option is needed yet though).
> I might need the scale and precision options for filtering and grouping.

Can you elaborate?
I've been building/configuring the input components for all types to display in filters and groups, and they need to make use of the type and display options. I've already configured it for number in the last PR that was raised.

I assume we can leave the configuration of these input components to the owner of the data type, instead of me working on all of them.
I see. Thanks
@silentninja and I discussed this on a call today and agreed to move forward with setting the default precision on the back end. As such, I'm marking this ticket as ready.

Resolving patch:
diff --git a/mathesar/api/db/viewsets/columns.py b/mathesar/api/db/viewsets/columns.py
--- a/mathesar/api/db/viewsets/columns.py
+++ b/mathesar/api/db/viewsets/columns.py
@@ -34,7 +34,12 @@ def create(self, request, table_pk=None):
         # We only support adding a single column through the API.
         serializer = ColumnSerializer(data=request.data, context={'request': request})
         serializer.is_valid(raise_exception=True)
-
+        type_options = request.data.get('type_options', None)
+        if type_options is not None:
+            scale = type_options.get('scale', None)
+            precision = type_options.get('precision', None)
+            if scale is not None and precision is None:
+                request.data['type_options']['precision'] = 1000
         if 'source_column' in serializer.validated_data:
             column = table.duplicate_column(
                 serializer.validated_data['source_column'],
@@ -44,6 +49,7 @@ def create(self, request, table_pk=None):
             )
         else:
             try:
+                # TODO Refactor add_column to user serializer validated date instead of request data
                 column = table.add_column(request.data)
             except ProgrammingError as e:
                 if type(e.orig) == DuplicateColumn:
diff --git a/mathesar/api/serializers/columns.py b/mathesar/api/serializers/columns.py
--- a/mathesar/api/serializers/columns.py
+++ b/mathesar/api/serializers/columns.py
@@ -31,6 +31,11 @@ class TypeOptionSerializer(MathesarErrorMessageMixin, serializers.Serializer):
     scale = serializers.IntegerField(required=False)
     fields = serializers.CharField(required=False)
 
+    def validate(self, attrs):
+        if attrs.get('scale', None) is not None and attrs.get('precision', None) is None:
+            attrs['precision'] = 1000
+        return super().validate(attrs)
+
     def run_validation(self, data=empty):
         # Ensure that there are no unknown type options passed in.
         if data is not empty and data is not None:


Test patch:
diff --git a/mathesar/tests/api/test_column_api.py b/mathesar/tests/api/test_column_api.py
--- a/mathesar/tests/api/test_column_api.py
+++ b/mathesar/tests/api/test_column_api.py
@@ -338,17 +338,18 @@ def test_column_create_wrong_display_options(
 
 
 @pytest.mark.parametrize(
-    "type_,type_options",
+    "type_,type_options,expected_type_options",
     [
-        ("NUMERIC", {"precision": 5, "scale": 3}),
-        ("VARCHAR", {"length": 5}),
-        ("CHAR", {"length": 5}),
-        ("INTERVAL", {"precision": 5}),
-        ("INTERVAL", {"precision": 5, "fields": "second"}),
-        ("INTERVAL", {"fields": "day"}),
+        ("NUMERIC", {"precision": 5, "scale": 3}, {"precision": 5, "scale": 3}),
+        ("NUMERIC", {"scale": 3}, {"precision": 1000, "scale": 3}),
+        ("VARCHAR", {"length": 5}, {"length": 5}),
+        ("CHAR", {"length": 5}, {"length": 5}),
+        ("INTERVAL", {"precision": 5}, {"precision": 5}),
+        ("INTERVAL", {"precision": 5, "fields": "second"}, {"precision": 5, "fields": "second"}),
+        ("INTERVAL", {"fields": "day"}, {"fields": "day"}),
     ]
 )
-def test_column_create_retrieve_options(column_test_table, client, type_, type_options):
+def test_column_create_retrieve_options(column_test_table, client, type_, type_options, expected_type_options):
     name = "anewcolumn"
     cache.clear()
     num_columns = len(column_test_table.sa_columns)
@@ -367,7 +368,7 @@ def test_column_create_retrieve_options(column_test_table, client, type_, type_o
     actual_new_col = new_columns_response.json()["results"][-1]
     assert actual_new_col["name"] == name
     assert actual_new_col["type"] == type_
-    assert actual_new_col["type_options"] == type_options
+    assert actual_new_col["type_options"] == expected_type_options
 
 
 invalid_type_options = [
@@ -704,7 +705,8 @@ def test_column_update_name_type_nullable_default(column_test_table, client):
 def test_column_update_type_options(column_test_table, client):
     cache.clear()
     type_ = "NUMERIC"
-    type_options = {"precision": 3, "scale": 1}
+    type_options = {"scale": 1}
+    expected_type_options = {'precision': 1000, 'scale': 1}
     data = {"type": type_, "type_options": type_options}
     column = _get_columns_by_name(column_test_table, ['mycolumn3'])[0]
     response = client.patch(
@@ -712,7 +714,7 @@ def test_column_update_type_options(column_test_table, client):
         data,
     )
     assert response.json()["type"] == type_
-    assert response.json()["type_options"] == type_options
+    assert response.json()["type_options"] == expected_type_options
 
 
 def test_column_update_type_options_no_type(column_test_table, client):