Repository: mlflow/mlflow

Issue:
[BUG] mlflow.projects.run can't recognize local URI
### Willingness to contribute

Yes. I can contribute a fix for this bug independently.

### MLflow version

1.26.1

### System information

- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Windows 10
- **Python version**: 3.10.4
- **yarn version, if running the dev UI**: N/A


### Describe the problem

I'm recreating the hyperparameter optimization project from https://github.com/mlflow/mlflow/tree/master/examples/hyperparam , i.e. I have coded all the files from scratch on my local machine, and I am running into the following problem while invoking the `search_random.py` script using the command `mlflow -e random --experiment-id 2 . --env-manager local` from the root directory of the project.

The error:
```python-traceback
File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\git\cmd.py", line 502, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git fetch -v --depth=1 origin
  stderr: 'fatal: 'D:/MyStuff/Learn/MachineLearning/MLFlow/LearnMLFlow/tut2' does not appear to be a git repository
fatal: Could not read from remote repository.'
```

Through some debugging, I found that the exception is thrown at the point in the code where run an instance of the `train.py` script on the new thread:

```python
with mlflow.start_run(nested=True) as child_run:
    p = mlflow.projects.run(
        run_id=child_run.info.run_id,
        uri=".",
        entry_point="train",
        parameters={
            "training_data": training_data,
            "epochs": str(nepochs),
            "learning_rate": str(lr),
            "momentum": str(momentum),
            "seed": str(seed),
        },
        experiment_id=experiment_id,
        synchronous=False,
        env_manager='local'
    )
    succeeded = p.wait()
```

Specifically, the exception is thrown on the `p.wait()` call. I can't seem to understand why MLFlow thinks the project is at a remote repository even though I've specified the `uri` for the project as the current directory in the constructor. I've also tried replacing the `.` with the absolute path of the project directory, but the error persists.

### Tracking information

_No response_

### Code to reproduce issue

```python
with mlflow.start_run(nested=True) as child_run:
    p = mlflow.projects.run(
        run_id=child_run.info.run_id,
        uri=".",
        entry_point="train",
        parameters={
            "training_data": training_data,
            "epochs": str(nepochs),
            "learning_rate": str(lr),
            "momentum": str(momentum),
            "seed": str(seed),
        },
        experiment_id=experiment_id,
        synchronous=False,
        env_manager='local'
    )
    succeeded = p.wait()
```

### Other info / logs

```python-traceback
2022/05/31 03:07:45 INFO mlflow.projects.utils: === Created directory C:\Users\dhruv\AppData\Local\Temp\tmphzol9g4d for downloading remote URIs passed to arguments of type 'path' ===
2022/05/31 03:07:45 INFO mlflow.projects.backend.local: === Running command 'python search_random.py  http://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv --max-runs 8 --max-p 2 --epochs 32 --metric rmse --seed 97531' in run with ID '6e2312ed5b914f32b933f160035dc3cf' ===
2022/05/31 03:07:48 INFO mlflow.projects.utils: === Fetching project from D:\MyStuff\Learn\MachineLearning\MLFlow\LearnMLFlow\tut2 into C:\Users\dhruv\AppData\Local\Temp\tmpg0sm6lo5 ===
Traceback (most recent call last):
  File "D:\MyStuff\Learn\MachineLearning\MLFlow\LearnMLFlow\tut2\search_random.py", line 123, in <module>
    run()
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "D:\MyStuff\Learn\MachineLearning\MLFlow\LearnMLFlow\tut2\search_random.py", line 90, in run
    _, null_train_loss, null_val_loss, null_test_loss = new_eval(0, experiment_id)((0, 0))
  File "D:\MyStuff\Learn\MachineLearning\MLFlow\LearnMLFlow\tut2\search_random.py", line 47, in eval
    p = mlflow.projects.run(
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\mlflow\projects\__init__.py", line 331, in run
    submitted_run_obj = _run(
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\mlflow\projects\__init__.py", line 95, in _run
    submitted_run = backend.run(
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\mlflow\projects\backend\local.py", line 64, in run
    work_dir = fetch_and_validate_project(project_uri, version, entry_point, params)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\mlflow\projects\utils.py", line 130, in fetch_and_validate_project
    work_dir = _fetch_project(uri=uri, version=version)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\mlflow\projects\utils.py", line 164, in _fetch_project
    _fetch_git_repo(parsed_uri, version, dst_dir)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\mlflow\projects\utils.py", line 202, in _fetch_git_repo
    origin.fetch(depth=GIT_FETCH_DEPTH)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\git\remote.py", line 886, in fetch
    res = self._get_fetch_info_from_stderr(proc, progress,
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\git\remote.py", line 750, in _get_fetch_info_from_stderr
    proc.wait(stderr=stderr_text)
  File "D:\Softwares\Anaconda\envs\mlflow\lib\site-packages\git\cmd.py", line 502, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git fetch -v --depth=1 origin
  stderr: 'fatal: 'D:/MyStuff/Learn/MachineLearning/MLFlow/LearnMLFlow/tut2' does not appear to be a git repository
fatal: Could not read from remote repository.'
2022/05/31 03:07:49 ERROR mlflow.cli: === Run (ID '6e2312ed5b914f32b933f160035dc3cf') failed ===
```

### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [X] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [X] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [X] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations
More specific git uri regex
Signed-off-by: Stefan Kokov <stefan.kokov@fadata.eu>

## What changes are proposed in this pull request?
Fix #2824 

(Please fill in changes proposed in this fix)

## How is this patch tested?

(Details)

## Release Notes

### Is this a user-facing change?

- [x] No. You can skip the rest of this section.
- [ ] Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

### What component(s), interfaces, languages, and integrations does this PR affect?
Components 
- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: Local serving, model deployment tools, spark UDFs
- [ ] `area/server-infra`: MLflow server, JavaScript dev server
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

Interface 
- [ ] `area/uiux`: Front-end, user experience, JavaScript, plotting
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

Language 
- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

Integrations
- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations

### How should the PR be classified in the release notes? Choose one:

- [ ] `rn/breaking-change` - The PR will be mentioned in the "Breaking Changes" section
- [ ] `rn/none` - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
- [ ] `rn/feature` - A new user-facing feature worth mentioning in the release notes
- [ ] `rn/bug-fix` - A user-facing bug fix worth mentioning in the release notes
- [ ] `rn/documentation` - A user-facing documentation change worth mentioning in the release notes



Discussion:
The problem seems to be the regular expression matching for Git URIs on line 34 of `mlflow/projects/utils.py`. 

```python
_GIT_URI_REGEX = re.compile(r"^[^/]*:")
```

Notice that this matches absolute file paths in Windows too:

```
D:\SomeRandomDirectory\
```

The solution would be to use a more robust Regex Pattern, say like the one suggested [here](https://stackoverflow.com/questions/2514859/regular-expression-for-git-repository):

```python
_GIT_URI_REGEX = re.compile(r"((git|ssh|http(s)?)|(git@[\w\.]+))(:(//)?)([\w\.@\:/\-~]+)(\.git)(/)?")
```

This needs a pull request to be fixed unless I'm missing something obvious here. I would love maintainer clarification on this. Thanks.
@dkapur17 Using the more robust regex sounds good to me, looking forward to your PR :)
Sweet! Will make the PR asap.
any fix for this..?

Resolving patch:
diff --git a/mlflow/projects/utils.py b/mlflow/projects/utils.py
--- a/mlflow/projects/utils.py
+++ b/mlflow/projects/utils.py
@@ -3,6 +3,7 @@
 import re
 import tempfile
 import urllib.parse
+import pathlib
 
 from distutils import dir_util
 
@@ -29,8 +30,9 @@
 )
 from mlflow.utils.rest_utils import augmented_raise_for_status
 
-# TODO: this should be restricted to just Git repos and not S3 and stuff like that
-_GIT_URI_REGEX = re.compile(r"^[^/]*:")
+_GIT_URI_REGEX = re.compile(
+    r"^((git|ssh|http(s)?)|(git@[\w\.]+))(:(//)?)([\w\.@\:/\-~]+)(\.git)(/)?"
+)
 _FILE_URI_REGEX = re.compile(r"^file://.+")
 _ZIP_URI_REGEX = re.compile(r".+\.zip$")
 MLFLOW_LOCAL_BACKEND_RUN_ID_CONFIG = "_mlflow_local_backend_run_id"
@@ -97,9 +99,31 @@ def _is_file_uri(uri):
     return _FILE_URI_REGEX.match(uri)
 
 
+def _is_git_repo(path):
+    """Returns True if passed-in path is a valid git repository"""
+    import git
+
+    try:
+        git.Repo(path)
+        return True
+    except git.exc.InvalidGitRepositoryError:
+        return False
+
+
 def _is_local_uri(uri):
-    """Returns True if the passed-in URI should be interpreted as a path on the local filesystem."""
-    return not _GIT_URI_REGEX.match(uri)
+    """Returns True if passed-in URI should be interpreted as a path on the local filesystem."""
+    if _GIT_URI_REGEX.match(uri):
+        return False
+
+    parsed_uri = urllib.parse.urlparse(uri)
+    drive = pathlib.Path(uri).drive
+
+    if drive != "" and drive.lower()[0] == parsed_uri.scheme:
+        return not _is_git_repo(uri)
+    elif parsed_uri.scheme in ("file", ""):
+        return not _is_git_repo(parsed_uri.path)
+    else:
+        return False
 
 
 def _is_zip_uri(uri):
@@ -159,7 +183,6 @@ def _fetch_project(uri, version=None):
         if use_temp_dst_dir:
             dir_util.copy_tree(src=parsed_uri, dst=dst_dir)
     else:
-        assert _GIT_URI_REGEX.match(parsed_uri), "Non-local URI %s should be a Git URI" % parsed_uri
         _fetch_git_repo(parsed_uri, version, dst_dir)
     res = os.path.abspath(os.path.join(dst_dir, subdirectory))
     if not os.path.exists(res):


Test patch:
diff --git a/tests/projects/test_utils.py b/tests/projects/test_utils.py
--- a/tests/projects/test_utils.py
+++ b/tests/projects/test_utils.py
@@ -19,6 +19,7 @@
     get_or_create_run,
     fetch_and_validate_project,
     load_project,
+    _GIT_URI_REGEX,
 )
 from mlflow.utils.mlflow_tags import MLFLOW_PROJECT_ENTRY_POINT, MLFLOW_SOURCE_NAME
 from tests.projects.utils import (
@@ -64,6 +65,12 @@ def test_is_zip_uri():
     assert not _is_zip_uri("C:/moo")
 
 
+def test_is_git_uri():
+    assert _GIT_URI_REGEX.match("https://github.com/mlflow/mlflow-example.git)")
+    assert _GIT_URI_REGEX.match("git@github.com:mlflow/mlflow.git")
+    assert not _GIT_URI_REGEX.match("D:\\mlflow\\mlflow-example")
+
+
 def test__fetch_project(local_git_repo, local_git_repo_uri, zipped_repo, httpserver):
     httpserver.serve_content(open(zipped_repo, "rb").read())
     # The tests are as follows: