Repository: mlflow/mlflow

Issue:
[BUG]  Unable to inspect recipes in mlflow-skinny on Databricks
### Issues Policy acknowledgement

- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)

### Willingness to contribute

No. I cannot contribute a bug fix at this time.

### MLflow version

- mlflow: 2.4.1

### System information

- Databricks 12.2 LTS
- Python: 3.9.5

### Describe the problem

Got `TemplateNotFound: resources/recipe_dag_template.html` when trying to inspect the recipe graph in the regression example from https://github.com/mlflow/recipes-examples on Databricks.

### Tracking information

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->


### Code to reproduce issue

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
Clone the [repo](https://github.com/mlflow/recipes-examples) but do not run `pip install -r requirements.txt`
```python
!pip install mlflow-skinny==2.4.1

from mlflow.recipes import Recipe
r = Recipe(profile="databricks")
r.clean()
r.inspect()
```

### Stack trace

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
```python
----> 1 r.inspect()

/databricks/python_shell/dbruntime/MLWorkloadsInstrumentation/_recipes.py in patched_api(self, *args, **kwargs)
    148     def patched_api(self, *args, **kwargs):
    149         try:
--> 150             result = original_api(self, *args, **kwargs)
    151             _try_record_api_call_usage_event(
    152                 usage_logger=usage_logger,

/local_disk0/.ephemeral_nfs/envs/pythonEnv-d2d5dd11-b2e9-4385-b609-d4c5248386db/lib/python3.9/site-packages/mlflow/recipes/regression/v1/recipe.py in inspect(self, step)
    402             regression_recipe.inspect(step="transform")
    403         """
--> 404         super().inspect(step=step)

/local_disk0/.ephemeral_nfs/envs/pythonEnv-d2d5dd11-b2e9-4385-b609-d4c5248386db/lib/python3.9/site-packages/mlflow/recipes/recipe.py in inspect(self, step)
    120         self._steps = self._resolve_recipe_steps()
    121         if not step:
--> 122             display_html(html_file_path=self._get_recipe_dag_file())
    123         else:
    124             output_directory = get_step_output_path(self._recipe_root_path, step, "")

/local_disk0/.ephemeral_nfs/envs/pythonEnv-d2d5dd11-b2e9-4385-b609-d4c5248386db/lib/python3.9/site-packages/mlflow/recipes/recipe.py in _get_recipe_dag_file(self)
    186 
    187         j2_env = jinja2.Environment(loader=jinja2.FileSystemLoader(os.path.dirname(__file__)))
--> 188         recipe_dag_template = j2_env.get_template("resources/recipe_dag_template.html").render(
    189             {
    190                 "recipe_yaml_help": {

/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/jinja2/environment.py in get_template(self, name, parent, globals)
   1008             name = self.join_path(name, parent)
   1009 
-> 1010         return self._load_template(name, globals)
   1011 
   1012     @internalcode

/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/jinja2/environment.py in _load_template(self, name, globals)
    967                 return template
    968 
--> 969         template = self.loader.load(self, name, self.make_globals(globals))
    970 
    971         if self.cache is not None:

/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/jinja2/loaders.py in load(self, environment, name, globals)
    124         # first we try to get the source for this template together
    125         # with the filename and the uptodate function.
--> 126         source, filename, uptodate = self.get_source(environment, name)
    127 
    128         # try to load the code from the bytecode cache if there is a

/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.9/site-packages/jinja2/loaders.py in get_source(self, environment, template)
    216             # Use normpath to convert Windows altsep to sep.
    217             return contents, os.path.normpath(filename), uptodate
--> 218         raise TemplateNotFound(template)
    219 
    220     def list_templates(self) -> t.List[str]:

TemplateNotFound: resources/recipe_dag_template.html
```


### Other info / logs

<!-- PLEASE KEEP BACKTICKS AND CHECK PREVIEW -->
N/A

### What component(s) does this bug affect?

- [ ] `area/artifacts`: Artifact stores and artifact logging
- [ ] `area/build`: Build and test infrastructure for MLflow
- [ ] `area/docs`: MLflow documentation pages
- [ ] `area/examples`: Example code
- [ ] `area/model-registry`: Model Registry service, APIs, and the fluent client calls for Model Registry
- [ ] `area/models`: MLmodel format, model serialization/deserialization, flavors
- [X] `area/recipes`: Recipes, Recipe APIs, Recipe configs, Recipe Templates
- [ ] `area/projects`: MLproject format, project running backends
- [ ] `area/scoring`: MLflow Model server, model deployment tools, Spark UDFs
- [ ] `area/server-infra`: MLflow Tracking server backend
- [ ] `area/tracking`: Tracking Service, tracking client APIs, autologging

### What interface(s) does this bug affect?

- [ ] `area/uiux`: Front-end, user experience, plotting, JavaScript, JavaScript dev server
- [ ] `area/docker`: Docker use across MLflow's components, such as MLflow Projects and MLflow Models
- [ ] `area/sqlalchemy`: Use of SQLAlchemy in the Tracking Service or Model Registry
- [ ] `area/windows`: Windows support

### What language(s) does this bug affect?

- [ ] `language/r`: R APIs and clients
- [ ] `language/java`: Java APIs and clients
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/azure`: Azure and Azure ML integrations
- [ ] `integrations/sagemaker`: SageMaker integrations
- [X] `integrations/databricks`: Databricks integrations


Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -112,24 +112,30 @@ def run(self):
         print(MINIMUM_SUPPORTED_PYTHON_VERSION)
 
 
+skinny_package_data = [
+    # include alembic files to enable usage of the skinny client with SQL databases
+    # if users install sqlalchemy and alembic independently
+    *alembic_files,
+    *extra_files,
+    *recipes_template_files,
+    *recipes_files,
+]
+
 setup(
     name="mlflow" if not _is_mlflow_skinny else "mlflow-skinny",
     version=version,
     packages=find_packages(exclude=["tests", "tests.*"]),
-    package_data={
-        "mlflow": (
-            js_files
-            + models_container_server_files
-            + alembic_files
-            + extra_files
-            + recipes_template_files
-            + recipes_files
-        ),
-    }
-    if not _is_mlflow_skinny
-    # include alembic files to enable usage of the skinny client with SQL databases
-    # if users install sqlalchemy and alembic independently
-    else {"mlflow": alembic_files + extra_files},
+    package_data=(
+        {"mlflow": skinny_package_data}
+        if _is_mlflow_skinny
+        else {
+            "mlflow": [
+                *skinny_package_data,
+                *js_files,
+                *models_container_server_files,
+            ]
+        }
+    ),
     install_requires=CORE_REQUIREMENTS if not _is_mlflow_skinny else SKINNY_REQUIREMENTS,
     extras_require={
         "extras": [


Test patch: