Repository: mlflow/mlflow

Issue:
[BUG] re-creating a previously deleted model name will result in SQL error
### Issues Policy acknowledgement

- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)

### Where did you encounter this bug?

Local machine

### Willingness to contribute

No. I cannot contribute a bug fix at this time.

### MLflow version

- Client: 2.11.1
- Tracking server: 2.11.1


### System information

- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: MacOS
- **Python version**: 3.10
- **yarn version, if running the dev UI**:


### Describe the problem

When attempting to create a new model using `client.create_registered_model()`, with a name that has been used previously (but subsequently deleted), you will get a Python error because the SQL table with permissions already has that model.

Note, the model will still be created, but I am not sure if there are any other effects, like permissions being messed up.

### Tracking information

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```shell
REPLACE_ME
```


### Code to reproduce issue

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```python
import os
import mlflow

# create a client
os.environ["MLFLOW_TRACKING_URI"] = "https://mlflow.example.com"
os.environ["MLFLOW_TRACKING_INSECURE_TLS"] = "true"
os.environ["MLFLOW_TRACKING_USERNAME"] = "admin"
os.environ["MLFLOW_TRACKING_PASSWORD"] = "admin"
client = mlflow.tracking.MlflowClient()

# configs
MODEL_NAME = "my-model-3"

# create a registered model
print("===== Creating registered model...")
client.create_registered_model(name=MODEL_NAME)
print("===== Success.")

# delete the registered model
print("===== Deleting registered model...")
client.delete_registered_model(name=MODEL_NAME)
print("===== Success.")

# try to re-create the registered model
print("===== Re-creating registered model...")
client.create_registered_model(name=MODEL_NAME)
print("===== Success.")
```


### Stack trace

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```
===== Creating registered model...
===== Success.
===== Deleting registered model...
===== Success.
===== Re-Creating registered model...
Traceback (most recent call last):
  File "/Users/mathew/PycharmProjects/test-mlflow/python/bad.py", line 36, in <module>
    client.create_registered_model(name=model_name)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/mlflow_torch/lib/python3.10/site-packages/mlflow/tracking/client.py", line 2192, in create_registered_model
    return self._get_registry_client().create_registered_model(name, tags, description)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/mlflow_torch/lib/python3.10/site-packages/mlflow/tracking/_model_registry/client.py", line 63, in create_registered_model
    return self.store.create_registered_model(name, tags, description)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/mlflow_torch/lib/python3.10/site-packages/mlflow/store/model_registry/rest_store.py", line 82, in create_registered_model
    response_proto = self._call_endpoint(CreateRegisteredModel, req_body)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/mlflow_torch/lib/python3.10/site-packages/mlflow/store/model_registry/base_rest_store.py", line 44, in _call_endpoint
    return call_endpoint(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/mlflow_torch/lib/python3.10/site-packages/mlflow/utils/rest_utils.py", line 220, in call_endpoint
    response = verify_rest_response(response, endpoint)
  File "/opt/homebrew/Caskroom/miniforge/base/envs/mlflow_torch/lib/python3.10/site-packages/mlflow/utils/rest_utils.py", line 152, in verify_rest_response
    raise RestException(json.loads(response.text))
mlflow.exceptions.RestException: RESOURCE_ALREADY_EXISTS: Registered model permission (name=my-model-3, username=admin) already exists. Error: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "unique_name_user"
DETAIL:  Key (name, user_id)=(my-model-3, 1) already exists.

[SQL: INSERT INTO registered_model_permissions (name, user_id, permission) VALUES (%(name)s, %(user_id)s, %(permission)s) RETURNING registered_model_permissions.id]
[parameters: {'name': 'my-model-3', 'user_id': 1, 'permission': 'MANAGE'}]
(Background on this error at: https://sqlalche.me/e/20/gkpj)
```


### Other info / logs

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```
REPLACE_ME
```


### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/deployments`: MLflow Deployments client APIs, server, and third-party Deployments integrations
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [X] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations


Discussion:
@thesuperzapper Could you share more about the database, and also check if there is any configuration that prevent the hard delete of a row? I could delete and re-register the model on my end with MLflow 2.11.1 with SQLite.

@B-Step62 I am using PostgreSQL `16.1.0`, but I think the issue is that `registered_model_permissions` is not getting cleaned up when you delete a model. 

So, databases that actually enforce "unique constraints" will fail when recreating a model and will be unable to INSERT a new row to `registered_model_permissions` (because one with that key already exists). Clearly the semantics should be UPSERT (or we should remove the permission row when deleting the model).

I am not sure if SQLite is used the same as more traditional DBs in MLFlow.
@thesuperzapper Understood, thank you for the description! Yes that seems to be a bug, I will investigate and file a patch.

Resolving patch:
diff --git a/mlflow/server/auth/__init__.py b/mlflow/server/auth/__init__.py
--- a/mlflow/server/auth/__init__.py
+++ b/mlflow/server/auth/__init__.py
@@ -502,6 +502,21 @@ def set_can_manage_registered_model_permission(resp: Response):
     store.create_registered_model_permission(name, username, MANAGE.name)
 
 
+def delete_can_manage_registered_model_permission(resp: Response):
+    """
+    Delete registered model permission when the model is deleted.
+
+    We need to do this because the primary key of the registered model is the name,
+    unlike the experiment where the primary key is experiment_id (UUID). Therefore,
+    we have to delete the permission record when the model is deleted otherwise it
+    conflicts with the new model registered with the same name.
+    """
+    # Get model name from request context because it's not available in the response
+    name = request.get_json(force=True, silent=True)["name"]
+    username = authenticate_request().username
+    store.delete_registered_model_permission(name, username)
+
+
 def filter_search_experiments(resp: Response):
     if sender_is_admin():
         return
@@ -610,6 +625,7 @@ def filter_search_registered_models(resp: Response):
 AFTER_REQUEST_PATH_HANDLERS = {
     CreateExperiment: set_can_manage_experiment_permission,
     CreateRegisteredModel: set_can_manage_registered_model_permission,
+    DeleteRegisteredModel: delete_can_manage_registered_model_permission,
     SearchExperiments: filter_search_experiments,
     SearchRegisteredModels: filter_search_registered_models,
 }


Test patch:
diff --git a/tests/server/auth/test_auth.py b/tests/server/auth/test_auth.py
--- a/tests/server/auth/test_auth.py
+++ b/tests/server/auth/test_auth.py
@@ -17,9 +17,11 @@
 from mlflow.environment_variables import MLFLOW_TRACKING_PASSWORD, MLFLOW_TRACKING_USERNAME
 from mlflow.exceptions import MlflowException
 from mlflow.protos.databricks_pb2 import (
+    RESOURCE_DOES_NOT_EXIST,
     UNAUTHENTICATED,
     ErrorCode,
 )
+from mlflow.server.auth.routes import GET_REGISTERED_MODEL_PERMISSION
 from mlflow.utils.os import is_windows
 
 from tests.helper_functions import random_str
@@ -289,6 +291,55 @@ def test_search_registered_models(client, monkeypatch):
         assert names == [f"rm{i}" for i in readable]
 
 
+def test_create_and_delete_registered_model(client, monkeypatch):
+    username1, password1 = create_user(client.tracking_uri)
+
+    # create a registered model
+    with User(username1, password1, monkeypatch):
+        rm = client.create_registered_model("test_model")
+
+    # confirm the permission is set correctly
+    with User(username1, password1, monkeypatch):
+        response = requests.get(
+            url=client.tracking_uri + GET_REGISTERED_MODEL_PERMISSION,
+            params={"name": rm.name, "username": username1},
+            auth=(username1, password1),
+        )
+
+    permission = response.json()["registered_model_permission"]
+    assert permission["name"] == rm.name
+    assert permission["permission"] == "MANAGE"
+
+    # trying to create a model with the same name should fail
+    with User(username1, password1, monkeypatch):
+        with pytest.raises(MlflowException, match=r"RESOURCE_ALREADY_EXISTS"):
+            client.create_registered_model("test_model")
+
+    # delete the registered model
+    with User(username1, password1, monkeypatch):
+        client.delete_registered_model(rm.name)
+
+    # confirm the registered model permission is also deleted
+    with User(username1, password1, monkeypatch):
+        response = requests.get(
+            url=client.tracking_uri + GET_REGISTERED_MODEL_PERMISSION,
+            params={"name": rm.name, "username": username1},
+            auth=("admin", "password"),  # Check with admin because the user permission is deleted
+        )
+
+    assert response.status_code == 404
+    assert response.json()["error_code"] == ErrorCode.Name(RESOURCE_DOES_NOT_EXIST)
+    assert (
+        response.json()["message"]
+        == f"Registered model permission with name={rm.name} and username={username1} not found"
+    )
+
+    # now we should be able to create a model with the same name
+    with User(username1, password1, monkeypatch):
+        rm = client.create_registered_model("test_model")
+    assert rm.name == "test_model"
+
+
 def _wait(url: str):
     t = time.time()
     while time.time() - t < 5: