internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: mathesar-foundation/mathesar

Issue:
Multiple tables with the same oid
## Description

Certain bugs or improper usage of django queryset reflection managers leads to the creation of multiple of Table objects for the same underlying table in the users table.

## To Reproduce
- Create a table directly on the user's database using `db.tables.operations.create.create_mathesar_table`, this won't create a Table object on mathesar's django database.
- Now create a Django Table object using `Table.objects.get_or_create(oid=db_table_oid, schema=schema)`
- There will be a race condition and two Table objects will be created
- Then Calling `models.Table.objects.get(oid=oid, schema=schema)` will throw the exception `mathesar.models.Table.MultipleObjectsReturned: get() returned more than one Table -- it returned 2!`

## Environment
Our docker container.



Discussion:
~OIDs should be set to `unique=True` in the `DatabaseObject` model.  In fact, they should be globally unique on a per-DB basis.~

Scratch that:  See the setup in the schema model.  The pair `database, oid` is needed (since we can have more than one DB).  Like this:
```
    class Meta:
        constraints = [
            models.UniqueConstraint(fields=["oid", "database"], name="unique_schema")
        ]
```

Ideally, we'd actually enforce that uniqueness over the union of all models that represent objects living in a database (schemas, tables, views, etc.)
> OIDs should be set to `unique=True` in the `DatabaseObject` model. In fact, they should be globally unique on a per-DB basis.

I agree with @mathemancer, we would still be getting an error but it should help us find the actual cause. Since the `oid` field is part of an abstract model, I would want the unique constraint to be in a meta object, so that it is easier to extend.
Can I work on this ?
Thanks!, @Anish9901 I have assigned the issue to you.
Is there a way i can test this issue locally?


Resolving patch:
diff --git a/mathesar/migrations/0027_auto_20220329_1855.py b/mathesar/migrations/0027_auto_20220329_1855.py
new file mode 100644
--- /dev/null
+++ b/mathesar/migrations/0027_auto_20220329_1855.py
@@ -0,0 +1,17 @@
+# Generated by Django 3.1.14 on 2022-03-29 18:55
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('mathesar', '0026_auto_20220126_1347'),
+    ]
+
+    operations = [
+        migrations.AddConstraint(
+            model_name='table',
+            constraint=models.UniqueConstraint(fields=('oid', 'schema'), name='unique_table'),
+        ),
+    ]
diff --git a/mathesar/models.py b/mathesar/models.py
--- a/mathesar/models.py
+++ b/mathesar/models.py
@@ -167,6 +167,11 @@ class Table(DatabaseObject):
                                related_name='tables')
     import_verified = models.BooleanField(blank=True, null=True)
 
+    class Meta:
+        constraints = [
+            models.UniqueConstraint(fields=["oid", "schema"], name="unique_table")
+        ]
+
     def validate_unique(self, exclude=None):
         # Ensure oid is unique on db level
         if Table.current_objects.filter(


Test patch:
Repository: mathesar-foundation/mathesar

Issue:
Unable to set number_format to null for Money type
## Reproduce

1. Set up a Money column.

1. Submit a `PATCH` request to the columns API, e.g. `/api/db/v0/tables/15/columns/52/`

1. Send:

    ```json
    {
      "type": "MATHESAR_TYPES.MATHESAR_MONEY",
      "display_options": {
        "currency_symbol": "$",
        "currency_symbol_location": "after-minus",
        "number_format": "english"
      }
    }
    ```

    Receive success. Good.

1. Now change `display_options.number_format` to `null`, sending:

    ```json
    {
      "type": "MATHESAR_TYPES.MATHESAR_MONEY",
      "display_options": {
        "currency_symbol": "$",
        "currency_symbol_location": "after-minus",
        "number_format": null
      }
    }
    ```
    
    Expect success.

    Receive:

    ```json
    [
      {
        "code": 2024,
        "field": "number_format",
        "message": "This field may not be null.",
        "detail": {}
      }
    ]
    ```



Discussion:

Resolving patch:
diff --git a/mathesar/api/serializers/shared_serializers.py b/mathesar/api/serializers/shared_serializers.py
--- a/mathesar/api/serializers/shared_serializers.py
+++ b/mathesar/api/serializers/shared_serializers.py
@@ -112,7 +112,7 @@ class BooleanDisplayOptionSerializer(MathesarErrorMessageMixin, OverrideRootPart
 
 
 class AbstractNumberDisplayOptionSerializer(serializers.Serializer):
-    number_format = serializers.ChoiceField(required=False, choices=['english', 'german', 'french', 'hindi', 'swiss'])
+    number_format = serializers.ChoiceField(required=False, allow_null=True, choices=['english', 'german', 'french', 'hindi', 'swiss'])
 
 
 class NumberDisplayOptionSerializer(


Test patch:
Repository: mathesar-foundation/mathesar

Issue:
Unexpected results when grouping by multiple columns
## Steps to reproduce

1. Use the Library Management schema

1. Go to "Items"

1. Group by "Acquisition Date" (year), then by "Acquisition Price".

1. Observe a group for `1903`, `$6.63` containing two records, with ids 2296 and 1393.

1. Expect both records in the group to have "Acquisition Price" equal to `$6.63`.

1. Observe record 1393 is in the group but has "Acquisition Price" equal to `$5.75`.

    ![image](https://user-images.githubusercontent.com/42411/186416357-ca4f8b46-bc0b-4d44-9b4f-68b62cea67af.png)




Discussion:

Resolving patch:
diff --git a/db/records/operations/group.py b/db/records/operations/group.py
--- a/db/records/operations/group.py
+++ b/db/records/operations/group.py
@@ -251,11 +251,17 @@ def get_group_augmented_records_pg_query(table, group_by):
 
 
 def _get_distinct_group_select(table, grouping_columns, preproc):
+    def _get_processed_column(proc, col):
+        if proc is not None:
+            pcol = get_db_function_subclass_by_id(proc).to_sa_expression(col)
+        else:
+            pcol = col
+        return pcol
+
     if preproc is not None:
         processed_columns = [
-            get_db_function_subclass_by_id(proc).to_sa_expression(col)
+            _get_processed_column(proc, col)
             for proc, col in zip(preproc, grouping_columns)
-            if proc is not None
         ]
     else:
         processed_columns = grouping_columns


Test patch:
diff --git a/db/tests/records/operations/test_group.py b/db/tests/records/operations/test_group.py
--- a/db/tests/records/operations/test_group.py
+++ b/db/tests/records/operations/test_group.py
@@ -531,6 +531,14 @@ def test_smoke_get_group_augmented_records_pg_query_magnitude(magnitude_table_ob
             mode=group.GroupMode.DISTINCT.value,
             preproc=[None, 'extract_email_domain']
         ),
+        259
+    ),
+    (
+        group.GroupBy(
+            ['Student Email'],
+            mode=group.GroupMode.DISTINCT.value,
+            preproc=['extract_email_domain']
+        ),
         3
     ),
     (
Repository: mathesar-foundation/mathesar

Issue:
API should respond with HTTP 404 status when attempting to delete a record that does not exist
## Reproduce

1. `DELETE` `http://localhost:8000/api/db/v0/tables/1/records/0/`
1. Expect response to have HTTP status code 404 and a body with an error message indicating the record could not be found.
1. Observe response to have HTTP status code 204



Discussion:
Assigning myself, should be an easy fix.

Resolving patch:
diff --git a/mathesar/api/db/viewsets/records.py b/mathesar/api/db/viewsets/records.py
--- a/mathesar/api/db/viewsets/records.py
+++ b/mathesar/api/db/viewsets/records.py
@@ -4,7 +4,9 @@
 from rest_framework.response import Response
 from sqlalchemy_filters.exceptions import BadSortFormat, SortFieldNotFound
 
+from mathesar.api.exceptions.error_codes import ErrorCodes
 import mathesar.api.exceptions.database_exceptions.exceptions as database_api_exceptions
+import mathesar.api.exceptions.generic_exceptions.base_exceptions as generic_api_exceptions
 from db.functions.exceptions import (
     BadDBFunctionFormat, ReferencedColumnsDontExist, UnknownDBFunctionID,
 )
@@ -199,6 +201,12 @@ def partial_update(self, request, pk=None, table_pk=None):
 
     def destroy(self, request, pk=None, table_pk=None):
         table = get_table_or_404(table_pk)
+        if table.get_record(pk) is None:
+            raise generic_api_exceptions.NotFoundAPIException(
+                NotFound,
+                error_code=ErrorCodes.RecordNotFound.value,
+                message="Record doesn't exist"
+            )
         table.delete_record(pk)
         return Response(status=status.HTTP_204_NO_CONTENT)
 
diff --git a/mathesar/api/exceptions/error_codes.py b/mathesar/api/exceptions/error_codes.py
--- a/mathesar/api/exceptions/error_codes.py
+++ b/mathesar/api/exceptions/error_codes.py
@@ -7,6 +7,8 @@ class ErrorCodes(Enum):
     # API Error
     MethodNotAllowed = 4006
     NotFound = 4005
+    TableNotFound = 4041
+    RecordNotFound = 4042
     UnknownError = 4999
     # Generic Errors
     ProgrammingError = 4101
diff --git a/mathesar/api/utils.py b/mathesar/api/utils.py
--- a/mathesar/api/utils.py
+++ b/mathesar/api/utils.py
@@ -1,7 +1,9 @@
 from rest_framework.exceptions import NotFound
+import mathesar.api.exceptions.generic_exceptions.base_exceptions as generic_api_exceptions
 import re
 
 from db.records.operations import group
+from mathesar.api.exceptions.error_codes import ErrorCodes
 from mathesar.models.base import Table
 from mathesar.utils.preview import column_alias_from_preview_template
 
@@ -20,7 +22,11 @@ def get_table_or_404(pk):
     try:
         table = Table.objects.get(id=pk)
     except Table.DoesNotExist:
-        raise NotFound
+        raise generic_api_exceptions.NotFoundAPIException(
+            NotFound,
+            error_code=ErrorCodes.TableNotFound.value,
+            message="Table doesn't exist"
+        )
     return table
 
 


Test patch:
Repository: mathesar-foundation/mathesar

Issue:
Reflection no longer runs on a schedule
## Description
<!-- A clear and concise description of what the bug is. -->

With some recent performance improvements, we introduced a known regression in that we no longer reflect DB objects on some schedule under all circumstances.

In particular, if you add a table (or tables) via some other client, it will never appear in the UI unless you've also done some other DDL operation _through_ the UI.

## Expected behavior
<!-- A clear and concise description of what you expected to happen. -->

It should be possible to reflect DB objects when needed.


Discussion:

Resolving patch:
diff --git a/mathesar/urls.py b/mathesar/urls.py
--- a/mathesar/urls.py
+++ b/mathesar/urls.py
@@ -29,6 +29,7 @@
     path('api/db/v0/', include(db_router.urls)),
     path('api/db/v0/', include(db_table_router.urls)),
     path('api/ui/v0/', include(ui_router.urls)),
+    path('api/ui/v0/reflect/', views.reflect_all, name='reflect_all'),
     path('auth/', include('django.contrib.auth.urls')),
 
     path('', views.home, name='home'),
diff --git a/mathesar/views.py b/mathesar/views.py
--- a/mathesar/views.py
+++ b/mathesar/views.py
@@ -1,6 +1,10 @@
 from django.shortcuts import render, redirect, get_object_or_404
 from django.contrib.auth.decorators import login_required
+from rest_framework import status
+from rest_framework.decorators import api_view
+from rest_framework.response import Response
 
+from mathesar.state import reset_reflection
 from mathesar.models.base import Database, Schema, Table
 from mathesar.api.serializers.databases import DatabaseSerializer, TypeSerializer
 from mathesar.api.serializers.schemas import SchemaSerializer
@@ -106,6 +110,13 @@ def render_schema(request, database, schema):
         return redirect('schema_home', db_name=database.name, schema_id=schema.id)
 
 
+@login_required
+@api_view(['POST'])
+def reflect_all(_):
+    reset_reflection()
+    return Response(status=status.HTTP_200_OK)
+
+
 @login_required
 def home(request):
     database = get_current_database(request, None)


Test patch:
diff --git a/mathesar/tests/api/test_reflection_api.py b/mathesar/tests/api/test_reflection_api.py
new file mode 100644
--- /dev/null
+++ b/mathesar/tests/api/test_reflection_api.py
@@ -0,0 +1,3 @@
+def test_reflect_endpoint(client):
+    response = client.post('/api/ui/v0/reflect/')
+    assert response.status_code == 200
Repository: mathesar-foundation/mathesar

Issue:
Implement _array mathesar type
- [ ] Cell component for array type
- [ ] Type specific handling for items within array
- [ ] Handle filtering, grouping


Discussion:

Resolving patch:
diff --git a/mathesar/database/types.py b/mathesar/database/types.py
--- a/mathesar/database/types.py
+++ b/mathesar/database/types.py
@@ -112,12 +112,18 @@ class UIType(Enum):
             MathesarCustomType.MATHESAR_JSON_OBJECT,
         }
     )
+    ARRAY = (
+        'array',
+        'Array',
+        {
+            PostgresType._ARRAY,
+        }
+    )
     # These are default Postgres types that we don't have specific behavior for yet in the UI.
     OTHER = (
         'other',
         'Other',
         {
-            PostgresType._ARRAY,
             PostgresType.BIT_VARYING,
             PostgresType.BIT,
             PostgresType.BYTEA,


Test patch:
Repository: mathesar-foundation/mathesar

Issue:
Show banner throughout application when "live demo mode" is turned on.
We should show a banner at the top of the screen on all pages that explains that Mathesar is in live demo mode and that each session has its own copy of demo data and that data will be deleted regularly.

Assigning this to @mathemancer to make sure it gets implemented at some point, @ghislaineguerin for the design, and @pavish for the frontend.


Discussion:
Link to design in figma: https://www.figma.com/file/xHb5oIqye3fnXtb2heRH34/Styling?node-id=3066%3A21916

Resolving patch:
diff --git a/config/context_processors.py b/config/context_processors.py
--- a/config/context_processors.py
+++ b/config/context_processors.py
@@ -6,7 +6,8 @@
 def frontend_settings(request):
     frontend_settings = {
         'development_mode': settings.MATHESAR_MODE == 'DEVELOPMENT',
-        'manifest_data': get_manifest_data()
+        'manifest_data': get_manifest_data(),
+        'live_demo_mode': getattr(settings, 'MATHESAR_LIVE_DEMO', False)
     }
     # Only include development URL if we're in development mode.
     if frontend_settings['development_mode'] is True:


Test patch:
Repository: mathesar-foundation/mathesar

Issue:
Per-request metadata cache
## Problem
<!-- Please provide a clear and concise description of the problem that this feature request is designed to solve.-->

Currently, our `metadata` caching is global over the entire app. This may create problems if we have users simultaneously accessing different DBs. 

## Proposed solution
<!-- A clear and concise description of your proposed solution or feature. -->

We should use https://github.com/anexia/django-request-cache to cache metadata per-request only.

## Additional context
<!-- Add any other context or screenshots about the feature request here.-->

We occasionally run into state-based bugs that this would probably solve.


Discussion:

Resolving patch:
diff --git a/config/settings.py b/config/settings.py
--- a/config/settings.py
+++ b/config/settings.py
@@ -53,6 +53,8 @@ def pipe_delim(pipe_string):
     "django.contrib.messages.middleware.MessageMiddleware",
     "django.middleware.clickjacking.XFrameOptionsMiddleware",
     "mathesar.middleware.CursorClosedHandlerMiddleware",
+    'django_userforeignkey.middleware.UserForeignKeyMiddleware',
+    'django_request_cache.middleware.RequestCacheMiddleware',
 ]
 
 ROOT_URLCONF = "config.urls"
diff --git a/mathesar/state/metadata.py b/mathesar/state/metadata.py
--- a/mathesar/state/metadata.py
+++ b/mathesar/state/metadata.py
@@ -1,11 +1,12 @@
 from db.metadata import get_empty_metadata
+from django_request_cache import cache_for_request
 
 
+@cache_for_request
 def get_cached_metadata():
     """
     Cached to minimize reflection queries to Postgres.
     """
-    global _metadata_cache
     return _metadata_cache
 
 


Test patch:
Repository: mathesar-foundation/mathesar

Issue:
Money formatting should start off with correct decimal places
By default, money columns should have 2 decimal places in the display options.

Currently, when you import the money in the demo video, there are values like `$15.6` – by default, that should be `$15.60`.


Discussion:

Resolving patch:
diff --git a/mathesar/api/serializers/columns.py b/mathesar/api/serializers/columns.py
--- a/mathesar/api/serializers/columns.py
+++ b/mathesar/api/serializers/columns.py
@@ -5,7 +5,7 @@
 
 from db.columns.exceptions import InvalidTypeError
 from db.columns.exceptions import InvalidTypeOptionError
-from db.types.base import PostgresType
+from db.types.base import PostgresType, MathesarCustomType
 from db.types.operations.convert import get_db_type_enum_from_id
 from mathesar.api.exceptions.database_exceptions import (
     exceptions as database_api_exceptions
@@ -217,6 +217,21 @@ def validate(self, data):
                 })
         return data
 
+    def to_representation(self, instance):
+        # Set default display_options for mathesar_money type if none are provided.
+        if (
+            instance.db_type == MathesarCustomType.MATHESAR_MONEY
+            and instance.display_options is None
+        ):
+            instance.display_options = {
+                'use_grouping': 'true',
+                'number_format': None,
+                'currency_symbol': None,
+                'maximum_fraction_digits': 2,
+                'minimum_fraction_digits': 2,
+                'currency_symbol_location': 'after-minus'}
+        return super().to_representation(instance)
+
     @property
     def validated_model_fields(self):
         return {key: self.validated_data[key] for key in self.validated_data if key in self.Meta.model_fields}


Test patch:
diff --git a/mathesar/tests/api/test_column_api_display_options.py b/mathesar/tests/api/test_column_api_display_options.py
--- a/mathesar/tests/api/test_column_api_display_options.py
+++ b/mathesar/tests/api/test_column_api_display_options.py
@@ -301,6 +301,26 @@ def test_column_create_wrong_display_options(
     assert response.status_code == 400
 
 
+def test_default_display_options_for_mathesar_money(
+    column_test_table, client
+):
+    name = "moneycolumn"
+    expected_display_options = {
+        'use_grouping': 'true',
+        'number_format': None,
+        'currency_symbol': None,
+        'maximum_fraction_digits': 2,
+        'minimum_fraction_digits': 2,
+        'currency_symbol_location': 'after-minus'}
+
+    data = {"name": name, "type": MathesarCustomType.MATHESAR_MONEY.id, "display_options": None}
+
+    response = client.post(f"/api/db/v0/tables/{column_test_table.id}/columns/", data)
+    response_data = response.json()
+    assert response.status_code == 201
+    assert response_data["display_options"] == expected_display_options
+
+
 def test_column_update_display_options(column_test_table_with_service_layer_options, client):
     table, _ = column_test_table_with_service_layer_options
     column_indexes = [2, 3, 4]
Repository: mathesar-foundation/mathesar

Issue:
Don't allow public schema to be edited
We currently allow editing the public schema's name and description from the schema page. We should disable this.


<img width="968" alt="Screen Shot 2022-11-18 at 12 03 58 PM" src="https://user-images.githubusercontent.com/287034/202761131-265da99a-6f76-4f02-b680-7a98851427b3.png">



Discussion:
Marking this as both backend and frontend so we can make sure to disable this at an API level as well.
@kgodey Can I tackle this issue!
Sure, go ahead @ayao451!
I'm interested to work upon. May I?
Sure, go ahead @harikishantk 
Hi @harikishantk. Since there has been no response here for the last 2 weeks. I am gonna take up the frontend work here. 

Resolving patch:
diff --git a/mathesar/api/db/viewsets/schemas.py b/mathesar/api/db/viewsets/schemas.py
--- a/mathesar/api/db/viewsets/schemas.py
+++ b/mathesar/api/db/viewsets/schemas.py
@@ -12,6 +12,7 @@
 from mathesar.api.serializers.schemas import SchemaSerializer
 from mathesar.models.base import Schema
 from mathesar.utils.schemas import create_schema_and_object
+from mathesar.api.exceptions.validation_exceptions.exceptions import EditingPublicSchemaIsDisallowed
 
 
 class SchemaViewSet(AccessViewSetMixin, viewsets.GenericViewSet, ListModelMixin, RetrieveModelMixin):
@@ -44,6 +45,11 @@ def partial_update(self, request, pk=None):
         serializer.is_valid(raise_exception=True)
 
         schema = self.get_object()
+
+        # We forbid editing the public schema
+        if schema.name == "public":
+            raise EditingPublicSchemaIsDisallowed()
+
         schema.update_sa_schema(serializer.validated_data)
 
         # Reload the schema to avoid cached properties
diff --git a/mathesar/api/exceptions/error_codes.py b/mathesar/api/exceptions/error_codes.py
--- a/mathesar/api/exceptions/error_codes.py
+++ b/mathesar/api/exceptions/error_codes.py
@@ -57,3 +57,4 @@ class ErrorCodes(Enum):
     DictHasBadKeys = 4416
     DeletedColumnAccess = 4418
     IncorrectOldPassword = 4419
+    EditingPublicSchema = 4421
diff --git a/mathesar/api/exceptions/validation_exceptions/exceptions.py b/mathesar/api/exceptions/validation_exceptions/exceptions.py
--- a/mathesar/api/exceptions/validation_exceptions/exceptions.py
+++ b/mathesar/api/exceptions/validation_exceptions/exceptions.py
@@ -154,8 +154,18 @@ class IncorrectOldPassword(MathesarValidationException):
 
     def __init__(
             self,
-
             field=None,
     ):
-        message = "Old password is not correct"
+        message = "Old password is not correct."
         super().__init__(None, self.error_code, message, field, None)
+
+
+class EditingPublicSchemaIsDisallowed(MathesarValidationException):
+    error_code = ErrorCodes.EditingPublicSchema.value
+
+    def __init__(
+            self,
+            message="Editing the public schema is disallowed.",
+            field=None,
+    ):
+        super().__init__(None, self.error_code, message, field)


Test patch: