Repository: mlflow/mlflow

Issue:
[FR] Allow specifying mlflow server configurations as environment variables
## MLflow Roadmap Item

This is an MLflow Roadmap item that has been prioritized by the MLflow maintainers. We’ve identified this feature as a highly requested addition to the MLflow package based on community feedback. 
We're seeking a community contribution for the implementation of this feature and will enthusiastically support the development and review of a submitted PR for this.

## Feature scope

This roadmap feature’s complexity is classified as:

- [X] `good-first-issue`: This feature is limited in complexity and effort required to implement.

## Proposal Summary

When running an MLFlow server, I would like to propose having the ability to specify flags as environment variable, such as `MLFLOW_BACKEND_STORE_URI`, `MLFLOW_DEFAULT_ARTIFACT_ROOT`,  `MLFLOW_SERVE_ARTIFACTS`, `MLFLOW_ARTIFACTS_DESTINATION`, `MLFLOW_HOST`, ...

## Motivation

The `backend-store-uri` will oftentimes contain sensitive information like database passwords. Some deployment options, like AWS ECS, won't allow you to interpolate secrets in CLI arguments at runtime.

Workarounds to this limitation of ECS include creating a custom docker image with an entrypoint that interpolates said secret, or overriding the entrypoint to use a shell.

Although this limitation is not the responsibility of MLFlow, I would argue that giving people the option to specify values that contain secret information in multiple ways (env vars, files, ...) provides a better UX for systems engineers.


### What component(s), interfaces, languages, and integrations does this feature affect?
Components 
- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for
Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: Local serving, model deployment tools, spark UDFs
- [x] `area/tracking`: Tracking Service, tracking client APIs, autologging

## Details

Leverage [Click's `envvar` argument for the `.option()` decorator](https://click.palletsprojects.com/en/7.x/options/#values-from-environment-variables).





Discussion:
A bit more specifically, this is how MLFlow on top of ECS looks like for me right now:

```hcl
entryPoint = ["sh", "-c"]
command = [
  "/bin/sh -c \"mlflow server --host=0.0.0.0 --port=80 --default-artifact-root=s3://${bucket_id} --backend-store-uri=mysql+pymysql://${db_username}:`echo -n $DB_PASSWORD`@${db_address}:${db_port}/${db_name}\""
]
secrets = [
  {
    name      = "DB_PASSWORD"
    valueFrom = secret
  },
]
```

Which breaks the encapsulation of the docker image's API and forces it to provide a shell.

And this is how this proposal would allow it to look:

```hcl
command = [
  "--host=0.0.0.0",
  "--port=80",
  "--default-artifact-root=s3://${bucket_id}",
]
secrets = [
  {
    name      = "BACKEND_STORE_URI"
    valueFrom = secret
  },
]
```

Allowing me to use any image which provides `mlflow` or `mlflow server` as an entrypoint.
I would love to see this added. Same for the `default-artifact-root` flag.
All the flags should also be env vars, flags aren't always friendly

Resolving patch:
diff --git a/mlflow/cli.py b/mlflow/cli.py
--- a/mlflow/cli.py
+++ b/mlflow/cli.py
@@ -244,6 +244,7 @@ def _validate_server_args(gunicorn_opts=None, workers=None, waitress_opts=None):
 @cli.command()
 @click.option(
     "--backend-store-uri",
+    envvar="MLFLOW_BACKEND_STORE_URI",
     metavar="PATH",
     default=DEFAULT_LOCAL_FILE_AND_ARTIFACT_PATH,
     help="URI to which to persist experiment and run data. Acceptable URIs are "
@@ -254,6 +255,7 @@ def _validate_server_args(gunicorn_opts=None, workers=None, waitress_opts=None):
 )
 @click.option(
     "--registry-store-uri",
+    envvar="MLFLOW_REGISTRY_STORE_URI",
     metavar="URI",
     default=None,
     help="URI to which to persist registered models. Acceptable URIs are "
@@ -262,6 +264,7 @@ def _validate_server_args(gunicorn_opts=None, workers=None, waitress_opts=None):
 )
 @click.option(
     "--default-artifact-root",
+    envvar="MLFLOW_DEFAULT_ARTIFACT_ROOT",
     metavar="URI",
     default=None,
     help="Directory in which to store artifacts for any new experiments created. For tracking "
@@ -352,6 +355,7 @@ def _validate_static_prefix(ctx, param, value):  # pylint: disable=unused-argume
 @cli.command()
 @click.option(
     "--backend-store-uri",
+    envvar="MLFLOW_BACKEND_STORE_URI",
     metavar="PATH",
     default=DEFAULT_LOCAL_FILE_AND_ARTIFACT_PATH,
     help="URI to which to persist experiment and run data. Acceptable URIs are "
@@ -362,6 +366,7 @@ def _validate_static_prefix(ctx, param, value):  # pylint: disable=unused-argume
 )
 @click.option(
     "--registry-store-uri",
+    envvar="MLFLOW_REGISTRY_STORE_URI",
     metavar="URI",
     default=None,
     help="URI to which to persist registered models. Acceptable URIs are "
@@ -370,6 +375,7 @@ def _validate_static_prefix(ctx, param, value):  # pylint: disable=unused-argume
 )
 @click.option(
     "--default-artifact-root",
+    envvar="MLFLOW_DEFAULT_ARTIFACT_ROOT",
     metavar="URI",
     default=None,
     help="Directory in which to store artifacts for any new experiments created. For tracking "
@@ -383,6 +389,7 @@ def _validate_static_prefix(ctx, param, value):  # pylint: disable=unused-argume
 @cli_args.SERVE_ARTIFACTS
 @click.option(
     "--artifacts-only",
+    envvar="MLFLOW_ARTIFACTS_ONLY",
     is_flag=True,
     default=False,
     help="If specified, configures the mlflow server to be used only for proxied artifact serving. "
@@ -397,12 +404,14 @@ def _validate_static_prefix(ctx, param, value):  # pylint: disable=unused-argume
 @cli_args.WORKERS
 @click.option(
     "--static-prefix",
+    envvar="MLFLOW_STATIC_PREFIX",
     default=None,
     callback=_validate_static_prefix,
     help="A prefix which will be prepended to the path of all static paths.",
 )
 @click.option(
     "--gunicorn-opts",
+    envvar="MLFLOW_GUNICORN_OPTS",
     default=None,
     help="Additional command line options forwarded to gunicorn processes.",
 )
@@ -411,6 +420,7 @@ def _validate_static_prefix(ctx, param, value):  # pylint: disable=unused-argume
 )
 @click.option(
     "--expose-prometheus",
+    envvar="MLFLOW_EXPOSE_PROMETHEUS",
     default=None,
     help="Path to the directory where metrics will be stored. If the directory "
     "doesn't exist, it will be created. "
diff --git a/mlflow/utils/cli_args.py b/mlflow/utils/cli_args.py
--- a/mlflow/utils/cli_args.py
+++ b/mlflow/utils/cli_args.py
@@ -139,6 +139,7 @@ def _resolve_env_manager(ctx, _, env_manager):
 HOST = click.option(
     "--host",
     "-h",
+    envvar="MLFLOW_HOST",
     metavar="HOST",
     default="127.0.0.1",
     help="The network address to listen on (default: 127.0.0.1). "
@@ -146,7 +147,13 @@ def _resolve_env_manager(ctx, _, env_manager):
     "server from other machines.",
 )
 
-PORT = click.option("--port", "-p", default=5000, help="The port to listen on (default: 5000).")
+PORT = click.option(
+    "--port",
+    "-p",
+    envvar="MLFLOW_PORT",
+    default=5000,
+    help="The port to listen on (default: 5000).",
+)
 
 TIMEOUT = click.option(
     "--timeout", "-t", help="Timeout in seconds to serve a request (default: 60)."
@@ -156,6 +163,7 @@ def _resolve_env_manager(ctx, _, env_manager):
 WORKERS = click.option(
     "--workers",
     "-w",
+    envvar="MLFLOW_WORKERS",
     default=None,
     help="Number of gunicorn worker processes to handle requests (default: 4).",
 )
@@ -169,6 +177,7 @@ def _resolve_env_manager(ctx, _, env_manager):
 
 ARTIFACTS_DESTINATION = click.option(
     "--artifacts-destination",
+    envvar="MLFLOW_ARTIFACTS_DESTINATION",
     metavar="URI",
     default="./mlartifacts",
     help=(
@@ -181,6 +190,7 @@ def _resolve_env_manager(ctx, _, env_manager):
 
 SERVE_ARTIFACTS = click.option(
     "--serve-artifacts",
+    envvar="MLFLOW_SERVE_ARTIFACTS",
     is_flag=True,
     default=False,
     help="If specified, enables serving of artifact uploads, downloads, and list requests "


Test patch: