Repository: mlflow/mlflow

Issue:
[BUG] MLflow tried to browse the wrong folder in FTP
### Issues Policy acknowledgement

- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)

### Willingness to contribute

No. I cannot contribute a bug fix at this time.

### MLflow version

`pip install mlflow==2.2.2`

### System information

- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: MasOs
- **Python version**: 3.9
- **yarn version, if running the dev UI**:


### Describe the problem

I am trying to follow scenario 5 in https://mlflow.org/docs/latest/tracking.html, with a FTP server (artifacts) and local SQLite (store), but I run issues over issues.

I run a FTP server using:
```
docker run --rm \
  --name ftpd_server -p 21:21 -p 30000-30009:30000-30009 \
  -e "PUBLICHOST=localhost" \
  -e FTP_USER_NAME=admin -e FTP_USER_PASS=123 \
  -e FTP_USER_HOME=/home/admin \
  -v/abs/path_to/artifacts:/home/admin \
  stilliard/pure-ftpd
```
I tested different Docker images with different configurations, and I **always** get a similar issue... 

I do see migrations of database when I run the server, so that is working correctly.

It feels like the MLflow server tried to browse the file system incorrectly (see stacktrace below), as a non absolute path is used. Could someone please provide some help? I have followed all the steps described in the documentation: what have I missed? I am fully stuck

### Tracking information

Result of mlflow doctor:

```shell
MLflow version: 2.2.2
Tracking URI: http://localhost:3000
Artifact URI: mlflow-artifacts:/2/18cf9305894c4dd3851149c8b5d5064c/artifacts
System information: Darwin Darwin Kernel Version 20.5.0: Sat May  8 05:10:33 PDT 2021; root:xnu-7195.121.3~9/RELEASE_X86_64
Python version: 3.9.11
MLflow version: 2.2.2
MLflow module location: /Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/__init__.py
Tracking URI: http://localhost:3000
Registry URI: http://localhost:3000
Active experiment ID: 2
Active run ID: 18cf9305894c4dd3851149c8b5d5064c
Active run artifact URI: mlflow-artifacts:/2/18cf9305894c4dd3851149c8b5d5064c/artifacts
MLflow dependencies: 
  Flask: 2.2.3
  Jinja2: 3.1.2
  alembic: 1.10.2
  click: 8.1.3
  cloudpickle: 2.2.1
  databricks-cli: 0.17.5
  docker: 6.0.1
  entrypoints: 0.4
  gitpython: 3.1.31
  gunicorn: 20.1.0
  importlib-metadata: 6.0.0
  markdown: 3.4.1
  matplotlib: 3.7.1
  numpy: 1.23.5
  packaging: 23.0
  pandas: 1.5.3
  protobuf: 4.22.1
  pyarrow: 11.0.0
  pytz: 2022.7.1
  pyyaml: 6.0
  querystring-parser: 1.2.4
  requests: 2.28.2
  scikit-learn: 1.2.2
  scipy: 1.10.1
  shap: 0.41.0
  sqlalchemy: 2.0.6
  sqlparse: 0.4.3
```

Code that runs the mlflow tracking server:
```
	mlflow server \
	--backend-store-uri sqlite:///data.db \
	--artifacts-destination ftp://admin:123@localhost:21 \
	--serve-artifacts \
	--host 127.0.0.1 \
	--port 3000
```

### Code to reproduce issue

```
import mlflow.pyfunc
import tempfile
import os
import time
from typing import Tuple
import numpy as np
import pandas as pd


class JudeLin(mlflow.pyfunc.PythonModel):
    def __init__(self, coefficients: Tuple[float, float]):
        a, b = coefficients
        self._coefficients: np.array = np.array([a,b], dtype='float64')

    def predict(self, context, model_input: pd.DataFrame):
        return self.predict_numpy(
            nparray_nx2=model_input.to_numpy()
        )

    def predict_numpy(self, nparray_nx2: np.ndarray) -> np.array:
        return np.matmul(nparray_nx2, self._coefficients)


def test_mlflow_log(experiment_id):
    model = JudeLin(coefficients=(2, -1))

    mlflow.set_tracking_uri('http://localhost:3000')
    mlflow.set_experiment("experiment_random_202")

    with tempfile.TemporaryDirectory("w+") as name:
        file_name = os.path.join(name, 'model.json')
        with open(file_name, "w") as f:
            f.write("[{}, {}]".format(*model._coefficients))

        with mlflow.start_run(run_name="my-run-{}".format(time.time())):

            print(mlflow.get_artifact_uri())

            mlflow.log_param("A", model._coefficients[0])
            mlflow.log_param("B", model._coefficients[1])
            mlflow.pyfunc.log_model(
                artifact_path="idk",
                python_model=model
            )
```

Please note that **the name of the experiment is changed each time I run the code.**

### Stack trace

The `print` gives me
```
mlflow-artifacts:/2/efb7e67ea019498abb3aee831db54172/artifacts
```
which is correct.

I see in my folder (the one controlled by the FTP server) that there is indeed a
```
mlflow-artifacts:/2/efb7e67ea019498abb3aee831db54172/artifacts/python_env.yaml
```
file that is created. However, I then run into the following error:
```
2023/03/19 21:25:27 ERROR mlflow.server: Exception on /api/2.0/mlflow-artifacts/artifacts/2/efb7e67ea019498abb3aee831db54172/artifacts/idk/requirements.txt [PUT]
Traceback (most recent call last):
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/server/handlers.py", line 473, in wrapper
    return func(*args, **kwargs)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/server/handlers.py", line 539, in wrapper
    return func(*args, **kwargs)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/server/handlers.py", line 1639, in _upload_artifact
    artifact_repo.log_artifact(tmp_path, artifact_path=head or None)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/store/artifact/ftp_artifact_repo.py", line 77, in log_artifact
    ftp.cwd(artifact_dir)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 625, in cwd
    return self.voidcmd(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 286, in voidcmd
    return self.voidresp()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 259, in voidresp
    resp = self.getresp()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 254, in getresp
    raise error_perm(resp)
ftplib.error_perm: 550 Can't change directory to 2/efb7e67ea019498abb3aee831db54172/artifacts/idk: No such file or directory
2023/03/19 21:25:43 ERROR mlflow.server: Exception on /api/2.0/mlflow-artifacts/artifacts/2/efb7e67ea019498abb3aee831db54172/artifacts/idk/requirements.txt [PUT]
Traceback (most recent call last):
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/server/handlers.py", line 473, in wrapper
    return func(*args, **kwargs)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/server/handlers.py", line 539, in wrapper
    return func(*args, **kwargs)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/server/handlers.py", line 1639, in _upload_artifact
    artifact_repo.log_artifact(tmp_path, artifact_path=head or None)
  File "/Users/justin.dekeyser/Documents/explore/virtual_assistant/poc_1/ignored/venv39/lib/python3.9/site-packages/mlflow/store/artifact/ftp_artifact_repo.py", line 77, in log_artifact
    ftp.cwd(artifact_dir)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 625, in cwd
    return self.voidcmd(cmd)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 286, in voidcmd
    return self.voidresp()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 259, in voidresp
    resp = self.getresp()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ftplib.py", line 254, in getresp
    raise error_perm(resp)
ftplib.error_perm: 550 Can't change directory to 2/efb7e67ea019498abb3aee831db54172/artifacts/idk: No such file or directory
```

### Other info / logs

_No response_

### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations


Discussion:
Hi @Judekeyser, can you reproduce the same error with `mlflow.log_artifact` or `mlflow.log_artifacts?`

```python
with mlflow.start_run():
    mlflow.log_artifact(__file__, "idk")
    # or mlflow.log_artifacts(...)
```
> mlflow-artifacts:/2/efb7e67ea019498abb3aee831db54172/artifacts/python_env.yaml

This looks incorrect. As you specify `artifact_path="idk"`, the path should be `mlflow-artifacts:/2/efb7e67ea019498abb3aee831db54172/artifacts/idk/python_env.yaml`

https://github.com/mlflow/mlflow/blob/7f75f54860c7baefcaf3fbde330e5fbbb144d696/mlflow/store/artifact/ftp_artifact_repo.py#L72-L78

- Line 77 is the failed line
- Line 75 should create `artifact_dir`, but `ftplib.error_perm: 550 Can't change directory to 2/efb7e67ea019498abb3aee831db54172/artifacts/idk: No such file or directory` sounds like it doesn't.
Hi @harupy , sorry for my mistake, the file that is created is indeed:
```
mlflow-artifacts:/2/efb7e67ea019498abb3aee831db54172/artifacts/idk/python_env.yaml
```
The file exists correctly, and then the issue occurs. If I refer to the stacktrace, it feels like there is a missing prefix / or something, and it looks like it tries to change directory to a folder that is wrongly located (because it does exist and there is a file in there)
> https://github.com/mlflow/mlflow/blob/7f75f54860c7baefcaf3fbde330e5fbbb144d696/mlflow/store/artifact/ftp_artifact_repo.py#L72-L78
> 
> * Line 77 is the failed line
> * Line 75 should create `artifact_dir`, but `ftplib.error_perm: 550 Can't change directory to 2/efb7e67ea019498abb3aee831db54172/artifacts/idk: No such file or directory` sounds like it doesn't.

I hardly doubt the bug (only) happens here, because no break point gets triggered on my side. My breakpoint in `pyfunc.log_model` triggers until there is indeed a call to a `log_artifacts` (plural) from a mlflow client). But **no breakpoint** trigger from the file you refer to, neither in `log_artifact` nor in `log_artifacts`.
> no breakpoint trigger from the file you refer to, neither in log_artifact nor in log_artifacts.

When artifact proxy is enabled, the client just sends HTTP requests to the tracking server, doesn't run `FTPArtifactRepository.log_artifact` or `log_artifacts`.
Okay but still, I think the bug location is not well posed, because when I do
```
log_artifact(
  local_path=my_json_file,
  artifact_path="idk/model.json"
)
```
I get my test successful, and the `idk/model.json` file exists at the correct place (in my case, it was `idk/model.json/model.json` because of the `artifact_path` :D but okay, `log_artifact` seems to work!

However, for a reason I don't know about, `log_model` does not work the same.

In the `pyfunc.log_model`, I went up to a call that looks like:
```
mlflow.tracking.fluent.log_artifacts(local_path, mlflow_model.artifact_path)
```
from `model.py`. In that call, `local_path` is the temporary name of the model, while `mlflow_model.artifact_path` is my parameter `"idk"` I provided. Beyond that point, as you say, I guess no other break point trigger because of the proxy thing.
> I get my test successful, and the idk/model.json file exists at the correct place (in my case, it was idk/model.json/model.json because of the artifact_path :D but okay, log_artifact seems to work!

What about `log_artifacts`?
> > I get my test successful, and the idk/model.json file exists at the correct place (in my case, it was idk/model.json/model.json because of the artifact_path :D but okay, log_artifact seems to work!
> 
> What about `log_artifacts`?

Yes, that one does not work neither. Same error, "cannot change dir":

```
log_artifacts(
  local_dir=my_directory_path_containing_stuffs,
  artifact_path="idk"
)
```
@Judekeyser Got it. Can you creaet a docker compose file that can reproduce the issue?
I am afraid no, I do not know docker-compose and I don't have it installed. I put all the commands I do in the original message, maybe doing the compose from there would be quite straightforward if you have it? 
I was able to reproduce the issue.
minimum reproducer:

```python
import tempfile
import mlflow
from pathlib import Path

mlflow.set_tracking_uri("http://localhost:3000")


with tempfile.TemporaryDirectory() as tmpdir:
    t = Path(tmpdir)

    t.joinpath("a.txt").write_text("a")
    t.joinpath("subdir").mkdir()
    t.joinpath("subdir", "b.txt").write_text("b")
    t.joinpath("subdir", "c.txt").write_text("c")

    with mlflow.start_run():
        mlflow.log_artifacts(tmpdir)

```
Inserting `/` after `ftp://admin:123@localhost:21` as shown below:

```
      mlflow server \
        --backend-store-uri sqlite:///data.db \
        --artifacts-destination ftp://admin:123@localhost:21/ \
                                                            👆 this
        --serve-artifacts \
        --host 127.0.0.1 \
        --port 3000
```

fixed the issue. Investigting why now.
> 

I confess I haven't checked the FTP specification to know whether or not the string was ill-formed.

Couldn't find a location in the FTP specification where the format is clearly stated. Online resources usually don't make the trailing `/` explicit so I thought it would be okay to drop it too here.




Resolving patch:
diff --git a/mlflow/store/artifact/ftp_artifact_repo.py b/mlflow/store/artifact/ftp_artifact_repo.py
--- a/mlflow/store/artifact/ftp_artifact_repo.py
+++ b/mlflow/store/artifact/ftp_artifact_repo.py
@@ -25,7 +25,7 @@ def __init__(self, artifact_uri):
             "username": parsed.username,
             "password": parsed.password,
         }
-        self.path = parsed.path
+        self.path = parsed.path or "/"
 
         if self.config["host"] is None:
             self.config["host"] = "localhost"
diff --git a/mlflow/store/artifact/sftp_artifact_repo.py b/mlflow/store/artifact/sftp_artifact_repo.py
--- a/mlflow/store/artifact/sftp_artifact_repo.py
+++ b/mlflow/store/artifact/sftp_artifact_repo.py
@@ -48,7 +48,7 @@ def __init__(self, artifact_uri):
             "username": parsed.username,
             "password": parsed.password,
         }
-        self.path = parsed.path
+        self.path = parsed.path or "/"
 
         import pysftp
         import paramiko


Test patch: