Repository: mathesar-foundation/mathesar

Issue:
Fix API handling of multi-column UNIQUE and PK columns
See https://github.com/centerofci/mathesar/issues/2231#issuecomment-1380309139 for some insight.
Fix API handling of multi-column UNIQUE and PK columns
See https://github.com/centerofci/mathesar/issues/2231#issuecomment-1380309139 for some insight.


Discussion:
Isn't this the same issue as #2231? Why do we need two separate issues?
#2231 is about the solving the frontend bug and this is about solving the backend bug. The underlying UX problem is singular, but its solution involves solving two different problems by two teams. That's why I created a second issue. Basically I've been been making issues for work items, not "UX problems".

If you see problems with this, let's dicuss it. I like this, because my tasks then map to issues conveniently and for example I can just have "Fixes #XXXX" on top of each PR and GH will autoclose the parent issue. Otherwise, I have to do "Related #XXXX", because I'm only solving part of the problem, and then I have to remember/track remaining work in #XXXX in some other way, which is a drain on my capacity to task manage.
@dmos62 I think it's good to have separate issues from backend and frontend. It wasn't apparent to me from the issue descriptions that that's how they were organized. I think it would be helpful to update the description of #2231 to focus it on the frontend and also mark it as blocked by this issue.
@kgodey, done.
Hi @dmos62 , can i work on this?
@Hatilar420 sure. This is a less clear cut task though. Be proactive about making sure your understanding of the task is in sync with ours.
Hi @dmos62, are there any pre-existing tables in the codebase or a URL? That might help me recreate this issue quickly.
@Hatilar420 I don't think so.
Hi @dmos62, I am able to render tables with multi-column PK constrained. Which is opposed to the observed behaviour in #2231 . However, when I try to patch or delete any cell, the server throws a 500 internal error.

- Attached is SS for multiple constrained PK rendering.
![image](https://user-images.githubusercontent.com/29446639/224211960-44475b97-ba9d-448e-b75c-90f8a928daf0.png)



Is this the correct behaviour I am observing?

- The SQL I used to generate this table.
`CREATE TABLE anotherTable (
    id INTEGER NOT NULL,
    name VARCHAR(50) NOT NULL,
    data TEXT,
    PRIMARY KEY (id, name)
);
`



As I see it there are at least two tasks here:
- 500s should not happen, 
- Backend should accurately describe the primary and unique constraints (including when they are multi-column) associated with a table to the frontend.
Hi @dmos62, It might take some time, which can be more than a week, to open a PR for this issue as I am busy with some college work and some work from my internship. Is it okay with the contributor rules and with you?  
@Hatilar420 that's ok. We have a policy of unassigning contributors that are inactive for 7 days, but we can make an exception. Let's say if you update us within the next 14 days, we won't unassign you. Does that sound good?
Yes, sounds good. Thank you @dmos62  :) . 
Hi @dmos62, I am sorry for this late reply, I was busy with the GSOC proposal and college Assignments.
An update so far is that. I was able to fix this issue where the first column. (ie the id column is unique). However, in the cases where the `Id` column is not Unique, since we are working with composite Pkeys and the `Id` column can have the same value in some records, I was able to delete and patch (with the data which is meant for another record with same `Id`) multiple records having the same `id` values.
 
Do we want this behaviour?

EDIT
The Frontend also doesn't seem to render this behaviour with the records having the same `id` and gives an error for multiple Keys.  The same Error as issue #2231




@Hatilar420 I'm not sure I understand. Please open a draft PR so that I can see what you're attempting to do. Maybe create a test to demonstrate the behaviour you're describing.
Sure @dmos62, will open it Soon. Thanks :)
Isn't this the same issue as #2231? Why do we need two separate issues?
#2231 is about the solving the frontend bug and this is about solving the backend bug. The underlying UX problem is singular, but its solution involves solving two different problems by two teams. That's why I created a second issue. Basically I've been been making issues for work items, not "UX problems".

If you see problems with this, let's dicuss it. I like this, because my tasks then map to issues conveniently and for example I can just have "Fixes #XXXX" on top of each PR and GH will autoclose the parent issue. Otherwise, I have to do "Related #XXXX", because I'm only solving part of the problem, and then I have to remember/track remaining work in #XXXX in some other way, which is a drain on my capacity to task manage.
@dmos62 I think it's good to have separate issues from backend and frontend. It wasn't apparent to me from the issue descriptions that that's how they were organized. I think it would be helpful to update the description of #2231 to focus it on the frontend and also mark it as blocked by this issue.
@kgodey, done.
Hi @dmos62 , can i work on this?
@Hatilar420 sure. This is a less clear cut task though. Be proactive about making sure your understanding of the task is in sync with ours.
Hi @dmos62, are there any pre-existing tables in the codebase or a URL? That might help me recreate this issue quickly.
@Hatilar420 I don't think so.
Hi @dmos62, I am able to render tables with multi-column PK constrained. Which is opposed to the observed behaviour in #2231 . However, when I try to patch or delete any cell, the server throws a 500 internal error.

- Attached is SS for multiple constrained PK rendering.
![image](https://user-images.githubusercontent.com/29446639/224211960-44475b97-ba9d-448e-b75c-90f8a928daf0.png)



Is this the correct behaviour I am observing?

- The SQL I used to generate this table.
`CREATE TABLE anotherTable (
    id INTEGER NOT NULL,
    name VARCHAR(50) NOT NULL,
    data TEXT,
    PRIMARY KEY (id, name)
);
`



As I see it there are at least two tasks here:
- 500s should not happen, 
- Backend should accurately describe the primary and unique constraints (including when they are multi-column) associated with a table to the frontend.
Hi @dmos62, It might take some time, which can be more than a week, to open a PR for this issue as I am busy with some college work and some work from my internship. Is it okay with the contributor rules and with you?  
@Hatilar420 that's ok. We have a policy of unassigning contributors that are inactive for 7 days, but we can make an exception. Let's say if you update us within the next 14 days, we won't unassign you. Does that sound good?
Yes, sounds good. Thank you @dmos62  :) . 
Hi @dmos62, I am sorry for this late reply, I was busy with the GSOC proposal and college Assignments.
An update so far is that. I was able to fix this issue where the first column. (ie the id column is unique). However, in the cases where the `Id` column is not Unique, since we are working with composite Pkeys and the `Id` column can have the same value in some records, I was able to delete and patch (with the data which is meant for another record with same `Id`) multiple records having the same `id` values.
 
Do we want this behaviour?

EDIT
The Frontend also doesn't seem to render this behaviour with the records having the same `id` and gives an error for multiple Keys.  The same Error as issue #2231




@Hatilar420 I'm not sure I understand. Please open a draft PR so that I can see what you're attempting to do. Maybe create a test to demonstrate the behaviour you're describing.
Sure @dmos62, will open it Soon. Thanks :)

Resolving patch:
diff --git a/mathesar/api/serializers/constraints.py b/mathesar/api/serializers/constraints.py
--- a/mathesar/api/serializers/constraints.py
+++ b/mathesar/api/serializers/constraints.py
@@ -17,10 +17,17 @@
 from mathesar.models.base import Column, Constraint, Table
 
 
-class Table_Filtered_Column_queryset(serializers.PrimaryKeyRelatedField):
+class TableFilteredPrimaryKeyRelatedField(serializers.PrimaryKeyRelatedField):
+    """
+    Limits the accepted related primary key values to a specific table.
+    For example, if the PrimaryKeyRelatedField is instantiated with a
+    Column queryset, only columns in the "associated table" are
+    accepted. The "associated table" is defined by the context dict's
+    `table_id` value.
+    """
     def get_queryset(self):
         table_id = self.context.get('table_id', None)
-        queryset = super(Table_Filtered_Column_queryset, self).get_queryset()
+        queryset = super(TableFilteredPrimaryKeyRelatedField, self).get_queryset()
         if table_id is None or not queryset:
             return None
         return queryset.filter(table__id=table_id)
@@ -29,7 +36,7 @@ def get_queryset(self):
 class BaseConstraintSerializer(serializers.ModelSerializer):
     name = serializers.CharField(required=False)
     type = serializers.CharField()
-    columns = Table_Filtered_Column_queryset(queryset=Column.current_objects, many=True)
+    columns = TableFilteredPrimaryKeyRelatedField(queryset=Column.current_objects, many=True)
 
     class Meta:
         model = Constraint


Test patch:
diff --git a/mathesar/tests/api/test_constraint_api.py b/mathesar/tests/api/test_constraint_api.py
--- a/mathesar/tests/api/test_constraint_api.py
+++ b/mathesar/tests/api/test_constraint_api.py
@@ -2,6 +2,7 @@
 
 import pytest
 from sqlalchemy import Column as SAColumn, ForeignKey, Integer, MetaData, Table as SATable, select
+from sqlalchemy.sql import text
 
 from db.columns.operations.select import get_column_attnum_from_name
 from db.constraints.base import UniqueConstraint
@@ -9,6 +10,34 @@
 from mathesar.models.base import Constraint, Table, Column
 from mathesar.api.exceptions.error_codes import ErrorCodes
 from db.metadata import get_empty_metadata
+from mathesar.state import reset_reflection
+
+
+@pytest.fixture
+def multi_column_primary_key_table(create_schema, get_uid, engine):
+    prefix = "multi_col_pk"
+    schema_name = f"schema_{prefix}_{get_uid()}"
+    schema = create_schema(schema_name)
+    db_name = schema.database.name
+    table_name = f"table_{prefix}_{get_uid()}"
+    query = f"""
+        CREATE TABLE "{schema_name}"."{table_name}" (
+            column1 INT,
+            column2 INT,
+            column3 INT,
+            PRIMARY KEY (column1, column2)
+        );
+    """
+    with engine.connect() as conn:
+        conn.execute(text(query))
+        conn.commit()
+    reset_reflection(db_name=db_name)
+    # NOTE filtering by name is impossible here, because db object names are a dynamic properties, not model fields
+    all_tables = Table.current_objects.all()
+    for table in all_tables:
+        if table.name == table_name:
+            return table
+    raise Exception("Should never happen.")
 
 
 def _verify_primary_and_unique_constraints(response):
@@ -582,3 +611,21 @@ def test_invalid_constraint_type(create_patents_table, client):
     assert response.status_code == 400
     assert response_data['code'] == ErrorCodes.UnsupportedConstraint.value
     assert f'Operations related to {invalid_constraint} constraint are currently not supported' in response_data['message']
+
+
+def test_multi_column_primary_key_constraint_list(multi_column_primary_key_table, client):
+    table = multi_column_primary_key_table
+    response = client.get(f'/api/db/v0/tables/{table.id}/constraints/')
+    response_data = response.json()
+    constraints_data = response_data['results']
+    assert len(constraints_data) == 1
+    constraint_data = constraints_data[0]
+    assert constraint_data['type'] == 'primary'
+    expected_pk_col_names = set(['column1', 'column2'])
+    expected_pk_col_ids = set(
+        col.id
+        for col
+        in table.columns.all()
+        if col.name in expected_pk_col_names
+    )
+    assert set(constraint_data['columns']) == expected_pk_col_ids