Repository: mlflow/mlflow
Pull request: Fix two models one code name #5923
Signed-off-by: Anas BELFADIL <56280198+BFAnas@users.noreply.github.com>

## Related Issues/PRs

<!--
Please reference any related feature requests, issues, or PRs here. For example, `#123`. To automatically close the referenced items when this PR is merged, please use a closing keyword (close, fix, or resolve). For example, `Close #123`. See https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue for more information.
-->
Close #5923

## What changes are proposed in this pull request?

Prevent a model with `code_path` from using cached modules.

## How is this patch tested?

Unit test

## Does this PR change the documentation?

- [ ] No. You can skip the rest of this section.
- [ ] Yes. Make sure the changed pages / sections render correctly by following the steps below.

1. Check the status of the `ci/circleci: build_doc` check. If it's successful, proceed to the
   next step, otherwise fix it.
2. Click `Details` on the right to open the job page of CircleCI.
3. Click the `Artifacts` tab.
4. Click `docs/build/html/index.html`.
5. Find the changed pages / sections and make sure they render correctly.

## Release Notes

### Is this a user-facing change?

- [ ] No. You can skip the rest of this section.
- [ ] Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

### What component(s), interfaces, languages, and integrations does this PR affect?
Components
- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

Interface
- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

Language
- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

Integrations
- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations

<!--
Insert an empty named anchor here to allow jumping to this section with a fragment URL
(e.g. https://github.com/mlflow/mlflow/pull/123#user-content-release-note-category).
Note that GitHub prefixes anchor names in markdown with "user-content-".
-->
<a name="release-note-category"></a>
### How should the PR be classified in the release notes? Choose one:

- [ ] `rn/breaking-change` - The PR will be mentioned in the "Breaking Changes" section
- [ ] `rn/none` - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
- [ ] `rn/feature` - A new user-facing feature worth mentioning in the release notes
- [ ] `rn/bug-fix` - A user-facing bug fix worth mentioning in the release notes
- [ ] `rn/documentation` - A user-facing documentation change worth mentioning in the release notes


Problem statement:
Two models one code name 
### Willingness to contribute

Yes. I can contribute a fix for this bug independently.

### System information

- **Have I written custom code (as opposed to using a stock example script provided in MLflow)**: custom code
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: 20.04
- **MLflow installed from (source or binary)**: source
- **MLflow version (run ``mlflow --version``)**: 1.23.1
- **Python version**: 3.9
- **yarn version, if running the dev UI**: N/A


### Describe the problem

When two mlflow models have the same name for the code packaged with them, for example: a preprocessing code name `prep_features` was packaged with two different mlflow models. If the first mlflow model is loaded, then the second one is also loaded, python will only load the `prep_features` module once. This leads to problems if `prep_features` for the second model is different from the first model.



### Tracking information

_No response_

### Code to reproduce issue

prep_features. py change 1 to 2 when creating custom_model2 with the 2m1cn.py script
```python
def preprocess_features():
    return 1 # 2 for custom_model2
```
2m1cn.py use this file to create custom_model1 and custom_model2 
```python
import mlflow.pyfunc

class CM(mlflow.pyfunc.PythonModel):
    def __init__(self):
        pass
    def predict(self, context, model_input):
        from prep_features import preprocess_features
        return preprocess_features()

# Construct and save the model
model_path = "custom_model1" # "custom_model2"
code_path = ["prep_features.py"]
custom_model = CM()
mlflow.pyfunc.save_model(path=model_path, python_model=custom_model, code_path=code_path)
```

test.py now use this file to test your created models
```python
# Evaluate the models
import mlflow.pyfunc
import pandas as pd
model_input = pd.DataFrame([range(10)])
loaded_model1 = mlflow.pyfunc.load_model("custom_model1")
print(f"Model1: {loaded_model1.predict(model_input)}")
loaded_model2 = mlflow.pyfunc.load_model("custom_model2")
print(f"Model2: {loaded_model2.predict(model_input)}")
```

The output should be 1 for custom_model1 and 2 for custom_model 2. But because of the bug the output is 1 for both.

### Other info / logs

_No response_

### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [X] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations

Hints from issue discussion:

Commit message: Fix two models one code name #5923

Signed-off-by: Anas BELFADIL <56280198+BFAnas@users.noreply.github.com>

Patch to review:
diff --git a/mlflow/utils/model_utils.py b/mlflow/utils/model_utils.py
index 6fd202bc47f30..beb3d04703636 100644
--- a/mlflow/utils/model_utils.py
+++ b/mlflow/utils/model_utils.py
@@ -118,6 +118,16 @@ def _validate_and_copy_code_paths(code_paths, path, default_subpath="code"):
 
 def _add_code_to_system_path(code_path):
     sys.path = [code_path] + _get_code_dirs(code_path) + sys.path
+    # delete code modules that are already in sys.modules 
+    # so they will get reloaded anew from the correct code path
+    # othterwise python will use the alredy loaded modules 
+    modules = []
+    for path in [code_path] + _get_code_dirs(code_path):
+        modules_py = glob.glob(join(path, "*.py"))
+        modules += [ basename(f)[:-3] for f in modules_py if isfile(f) and not f.endswith('__init__.py')]
+    for module in modules:
+        if module in sys.modules:
+            sys.modules.pop(module)
 
 
 def _validate_and_prepare_target_save_path(path):


Code review comments:
diff_hunk: @@ -118,6 +118,16 @@ def _validate_and_copy_code_paths(code_paths, path, default_subpath="code"):
 
 def _add_code_to_system_path(code_path):
     sys.path = [code_path] + _get_code_dirs(code_path) + sys.path
+    # delete code modules that are already in sys.modules 
+    # so they will get reloaded anew from the correct code path
+    # othterwise python will use the alredy loaded modules 
+    modules = []
+    for path in [code_path] + _get_code_dirs(code_path):
+        modules_py = glob.glob(join(path, "*.py"))
path: mlflow/utils/model_utils.py
text: @user1:
you'll need to import this (local is fine for `glob`) and join is... `os.path.join`, right?

@author:
Yes, thanks for the suggestion.

diff_hunk: @@ -118,6 +118,16 @@ def _validate_and_copy_code_paths(code_paths, path, default_subpath="code"):
 
 def _add_code_to_system_path(code_path):
     sys.path = [code_path] + _get_code_dirs(code_path) + sys.path
+    # delete code modules that are already in sys.modules 
+    # so they will get reloaded anew from the correct code path
+    # othterwise python will use the alredy loaded modules 
+    modules = []
+    for path in [code_path] + _get_code_dirs(code_path):
+        modules_py = glob.glob(join(path, "*.py"))
+        modules += [ basename(f)[:-3] for f in modules_py if isfile(f) and not f.endswith('__init__.py')]
path: mlflow/utils/model_utils.py
text: @user1:
`os.path.basename()` ?
`os.path.isfile()` ?

@author:
yes!

diff_hunk: @@ -118,6 +118,16 @@ def _validate_and_copy_code_paths(code_paths, path, default_subpath="code"):
 
 def _add_code_to_system_path(code_path):
     sys.path = [code_path] + _get_code_dirs(code_path) + sys.path
+    # delete code modules that are already in sys.modules 
+    # so they will get reloaded anew from the correct code path
+    # othterwise python will use the alredy loaded modules 
path: mlflow/utils/model_utils.py
text: @user1:
```suggestion
    # otherwise python will use the already loaded modules 
```