Repository: evidentlyai/evidently
Issue: Compatible with Metaflow-card-html
Root cause: Documentation Deficiency

I would love to integrate Evidently solution into my current ML deployment framework: Metaflow
It supports visualizing rich HTML as Metaflow Cards.

Specifically, it is possible to use a [special card (type='html')](https://github.com/outerbounds/metaflow-card-altair/blob/main/altairflow.py) that support interactive visualizations in html format.

But, Metaflow requires the final html file to be stored as a class attribute (self.html) so it can be rendered.

I identified the protected method, `_repr_html_()`, which outputs the content of HTML. But in order to make it work, it has to be using the "inline" mode, instead of the[ current hard-coded "auto"](https://github.com/evidentlyai/evidently/blob/main/src/evidently/suite/base_suite.py#L111)

Would it be possible to make this a selectable parameter, that by default = "auto"?
Or even better, expose a non-protected method to print the html output so then it becomes fully compatible with Metaflow?
---
### contents of `data_stability._repr_html_()`:
![image](https://user-images.githubusercontent.com/18636680/204552199-356f7529-8b67-4d0b-a1cd-5e8c22b5bc0a.png)

### changing to "inline" mode:
```python
        from evidently.renderers.notebook_utils import determine_template
        from evidently.utils.dashboard import TemplateParams

        dashboard_id, dashboard_info, graphs = data_stability._build_dashboard_info()
        template_params = TemplateParams(
            dashboard_id=dashboard_id,
            dashboard_info=dashboard_info,
            additional_graphs=graphs,
        )
        self.html = data_stability._render(
            determine_template("inline"), template_params
        )
```
![image](https://user-images.githubusercontent.com/18636680/204552335-49e84d85-d965-4d89-8c1e-c4f32b408c90.png)


Hi @marcellovictorino, a quick fix to try: did you try using the inline show mode when you call the report? Described here https://docs.evidentlyai.com/user-guide/tests-and-reports/supported-environments 

`data_stability.show(mode='inline')
`



> https://docs.evidentlyai.com/user-guide/tests-and-reports/supported-environments

Thanks for the quick reply!
I did try that, but it does not work in my use case, I get the error:
> Exception: Cannot import HTML from IPython.display, no way to show html

Just to clarify, I am able to use that command and load the report inline in a notebook/jupyterlab environment.
What I want to achieve is the final rendered html content in itself, so I can visualise as within metaflow-card-html plugin

It would be similar to saving the file locally and then reading its content. But it would be nice to have a more direct way to achieve this, like running `data_stability._repr_html(mode='inline')`, for example - although not ideal to abuse a protected method :)

@elenasamuylova by the way, this actually is a leaner solution from my side. I just created a custom function:
```python
def get_evidently_html(evidently_object) -> str:
    """Returns the rendered EvidentlyAI report/metric as HTML

    Should be assigned to `self.html`, installing `metaflow-card-html` to be rendered
    """
    import tempfile

    with tempfile.NamedTemporaryFile() as tmp:
        evidently_object.save_html(tmp.name)
        with open(tmp.name) as fh:
            return fh.read()
```

Would be worth adding this to the documentation, mentioning it as integration with Metaflow :)
I will definitely share it in the Metaflow Slack channel 🎉

The example Metaflow flow looks like this:
```python
import metaflow as mf


class GettingStartedEvidentlyFlow(mf.FlowSpec):
    @mf.step
    def start(self):
        import numpy as np
        from sklearn.datasets import fetch_california_housing

        data = fetch_california_housing(as_frame=True)
        housing_data = data.frame
        housing_data.rename(columns={"MedHouseVal": "target"}, inplace=True)
        housing_data["prediction"] = housing_data["target"].values + np.random.normal(
            0, 5, housing_data.shape[0]
        )
        self.reference = housing_data.sample(n=5000, replace=False)
        self.current = housing_data.sample(n=5000, replace=False)

        self.next(self.monitoring_data_quality)

    @mf.card(type="html")
    @mf.step
    def monitoring_data_quality(self):
        import os

        os.system("pip install evidently --quiet")
        from evidently.test_preset import DataStabilityTestPreset
        from evidently.test_suite import TestSuite

        from tools.helper_functions import get_evidently_html

        print("Monitoring: data quality tests")
        data_stability = TestSuite(
            tests=[
                DataStabilityTestPreset(),
            ]
        )
        data_stability.run(reference_data=self.reference, current_data=self.current)

        self.html = get_evidently_html(data_stability)

        self.next(self.end)
```

Hi Marcello @marcellovictorino, that is so awesome! Glad it is solved. Would you be interested in making the contribution to the docs here?

You can create a PR with exactly the text that you shared here and place it in a new evidently-and-metaflow.md here https://github.com/evidentlyai/evidently/tree/main/docs/book/integrations. 

Want to make sure we give the right credit! :)

Sounds great! I would be happy to raise a PR for it :)

It ties in so nicely with Metaflow!
I will aim to get it done before the end of this week. 

Cheers!
Marcello Victorino 

Sent from my iPhone

> On 29 Nov 2022, at 18:51, elenasamuylova ***@***.***> wrote:
> 
> ﻿
> Hi Marcello @marcellovictorino, that is so awesome! Glad it is solved. Would you be interested in making the contribution to the docs here?
> 
> You can create a PR with exactly the text that you shared here and place it in a new evidently-and-metaflow.md here https://github.com/evidentlyai/evidently/tree/main/docs/book/integrations.
> 
> Want to make sure we give the right credit! :)
> 
> —
> Reply to this email directly, view it on GitHub, or unsubscribe.
> You are receiving this because you were mentioned.


Awesome - looking forward to it! We'll highlight it in our channels as well @marcellovictorino! 

@elenasamuylova just raised a PR:
https://github.com/evidentlyai/evidently/pull/468

Thanks for the contribution @marcellovictorino! Merged and already in the docs: https://github.com/evidentlyai/evidently/issues/464

Feel free to join our Discord community, too, would be glad to give the contributor role there too! 
https://discord.com/invite/xZjKRaNp8b