Repository: mlflow/mlflow

Issue:
[FR] Add functionality to log artifact from memory
### Willingness to contribute

Yes. I can contribute this feature with guidance from the MLflow community.

### Proposal Summary

Give users the ability to log artifacts from memory.

Example
```python
# pip install mlflow kaleido -U -q
import mlflow
import plotly.graph_objects as go

with mlflow.start_run():
  fig = go.Figure(data=[go.Bar(y=[2, 3, 1])])

  with tempfile.NamedTemporaryFile(suffix=".png") as tmpfile:
    fig.write_image(tmpfile.name)
    mlflow.log_artifact(tmpfile.name, "xyz.png")
```

The core challenge is supporting object-specific serialization methods. Patching the write method for every python object isn't scalable and passing the object's write method to a `log_artifact_from_memory` function is a bit "low level" for data scientists. I'd appreciate any guidance on whether this is possible. 

If it's not possible, we should at least modify the [fluent log_artfact](https://github.com/mlflow/mlflow/blob/dc51fc6b82fa78a30f8be5efed18f7867809311d/mlflow/tracking/fluent.py#L1010) docstring to use a tempfile and add some notes in the tutorials. Duplicating data is bad.

### Motivation

> #### What is the use case for this feature?
I want to log a plotly figure. The below solutions are unnecessarily complex and should be abstracted from the user
1. Use a tempfile with `mlflow.log_artifact` 
2. Convert to an object format that supports serialization with log_image/text/dict

> #### Why is this use case valuable to support for MLflow users in general?
Doing the above workarounds introduce friction into the product.

> #### Why is this use case valuable to support for your project(s) or organization?

> #### Why is it currently difficult to achieve this use case?


### Details

_No response_

### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/deployments`: MLflow Deployments client APIs, server, and third-party Deployments integrations
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [X] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [ ] `integrations/databricks`: Databricks integrations


Discussion:
@michael-berk Is the proposal to make `mlflow.log_artifact` accept objects like this?

```python
mlflow.log_artifact(fig, "xyz.png")
```



Resolving patch:
diff --git a/mlflow/tracking/client.py b/mlflow/tracking/client.py
--- a/mlflow/tracking/client.py
+++ b/mlflow/tracking/client.py
@@ -1158,11 +1158,10 @@ def log_artifact(self, run_id, local_path, artifact_path=None) -> None:
         .. code-block:: python
             :caption: Example
 
-            from mlflow import MlflowClient
+            import tempfile
+            from pathlib import Path
 
-            features = "rooms, zipcode, median_price, school_rating, transport"
-            with open("features.txt", "w") as f:
-                f.write(features)
+            from mlflow import MlflowClient
 
             # Create a run under the default experiment (whose id is '0').
             client = MlflowClient()
@@ -1170,7 +1169,11 @@ def log_artifact(self, run_id, local_path, artifact_path=None) -> None:
             run = client.create_run(experiment_id)
 
             # log and fetch the artifact
-            client.log_artifact(run.info.run_id, "features.txt")
+            with tempfile.TemporaryDirectory() as tmp_dir:
+                path = Path(tmp_dir, "features.txt")
+                path.write_text(features)
+                client.log_artifact(run.info.run_id, path)
+
             artifacts = client.list_artifacts(run.info.run_id)
             for artifact in artifacts:
                 print(f"artifact: {artifact.path}")
@@ -1198,26 +1201,27 @@ def log_artifacts(
         .. code-block:: python
             :caption: Example
 
-            import os
             import json
+            import tempfile
+            from pathlib import Path
 
             # Create some artifacts data to preserve
             features = "rooms, zipcode, median_price, school_rating, transport"
             data = {"state": "TX", "Available": 25, "Type": "Detached"}
+            with tempfile.TemporaryDirectory() as tmp_dir:
+                tmp_dir = Path(tmp_dir)
+                with (tmp_dir / "data.json").open("w") as f:
+                    json.dump(data, f, indent=2)
+                with (tmp_dir / "features.json").open("w") as f:
+                    f.write(features)
+
+                # Create a run under the default experiment (whose id is '0'), and log
+                # all files in "data" to root artifact_uri/states
+                client = MlflowClient()
+                experiment_id = "0"
+                run = client.create_run(experiment_id)
+                client.log_artifacts(run.info.run_id, tmp_dir, artifact_path="states")
 
-            # Create couple of artifact files under the local directory "data"
-            os.makedirs("data", exist_ok=True)
-            with open("data/data.json", "w", encoding="utf-8") as f:
-                json.dump(data, f, indent=2)
-            with open("data/features.txt", "w") as f:
-                f.write(features)
-
-            # Create a run under the default experiment (whose id is '0'), and log
-            # all files in "data" to root artifact_uri/states
-            client = MlflowClient()
-            experiment_id = "0"
-            run = client.create_run(experiment_id)
-            client.log_artifacts(run.info.run_id, "data", artifact_path="states")
             artifacts = client.list_artifacts(run.info.run_id)
             for artifact in artifacts:
                 print(f"artifact: {artifact.path}")
diff --git a/mlflow/tracking/fluent.py b/mlflow/tracking/fluent.py
--- a/mlflow/tracking/fluent.py
+++ b/mlflow/tracking/fluent.py
@@ -1022,16 +1022,20 @@ def log_artifact(
         :test:
         :caption: Example
 
+        import tempfile
+        from pathlib import Path
+
         import mlflow
 
         # Create a features.txt artifact file
         features = "rooms, zipcode, median_price, school_rating, transport"
-        with open("features.txt", "w") as f:
-            f.write(features)
-        # With artifact_path=None write features.txt under
-        # root artifact_uri/artifacts directory
-        with mlflow.start_run():
-            mlflow.log_artifact("features.txt")
+        with tempfile.TemporaryDirectory() as tmp_dir:
+            path = Path(tmp_dir, "features.txt")
+            path.write_text(features)
+            # With artifact_path=None write features.txt under
+            # root artifact_uri/artifacts directory
+            with mlflow.start_run():
+                mlflow.log_artifact(path)
     """
     run_id = run_id or _get_or_start_run().info.run_id
     MlflowClient().log_artifact(run_id, local_path, artifact_path)
@@ -1053,21 +1057,23 @@ def log_artifacts(
         :caption: Example
 
         import json
-        import os
+        import tempfile
+        from pathlib import Path
+
         import mlflow
 
         # Create some files to preserve as artifacts
         features = "rooms, zipcode, median_price, school_rating, transport"
         data = {"state": "TX", "Available": 25, "Type": "Detached"}
-        # Create couple of artifact files under the directory "data"
-        os.makedirs("data", exist_ok=True)
-        with open("data/data.json", "w", encoding="utf-8") as f:
-            json.dump(data, f, indent=2)
-        with open("data/features.txt", "w") as f:
-            f.write(features)
-        # Write all files in "data" to root artifact_uri/states
-        with mlflow.start_run():
-            mlflow.log_artifacts("data", artifact_path="states")
+        with tempfile.TemporaryDirectory() as tmp_dir:
+            tmp_dir = Path(tmp_dir)
+            with (tmp_dir / "data.json").open("w") as f:
+                json.dump(data, f, indent=2)
+            with (tmp_dir / "features.json").open("w") as f:
+                f.write(features)
+            # Write all files in `tmp_dir` to root artifact_uri/states
+            with mlflow.start_run():
+                mlflow.log_artifacts(tmp_dir, artifact_path="states")
     """
     run_id = run_id or _get_or_start_run().info.run_id
     MlflowClient().log_artifacts(run_id, local_dir, artifact_path)


Test patch: