Repository: tensorflow/model-analysis
Issue: Feature Request: Support viewing the slicing metrics widget outside of a Jupyter notebook
Root cause: Unimplemented Feature Gap

### System information

-   **Have I written custom code (as opposed to using a stock example script
    provided in TensorFlow Model Analysis)**: Yes
-   **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: N/A
-   **TensorFlow Model Analysis installed from (source or binary)**: PyPI
-   **TensorFlow Model Analysis version (use command below)**: 0.6.0
-   **Python version**: 2.7
-   **Jupyter Notebook version**: N/A
-   **Exact command to reproduce**:

```py
import tensorflow_model_analysis as tfma
from ipywidgets.embed import embed_minimal_html

analysis_path = 'gs://<TFMA_OUTPUT_DIRECTORY>'
result = tfma.load_eval_result(output_path=analysis_path)
slicing_metrics_view = tfma.view.render_slicing_metrics(result)
embed_minimal_html('tfma_export.html', views=[slicing_metrics_view], title='Slicing Metrics')
```

### Describe the problem

Jupyter notebook widgets support embedding the widget in a static HTML file that can be loaded outside of a Jupyter notebook (see [here](https://ipywidgets.readthedocs.io/en/latest/embedding.html) for details).

This *almost* works for the TFMA widgets, except that the `tfma_widget_js.js` file assumes that it is running in the context of a notebook page, and tries to load the `vulcanized_template.html` file from the notebook server (which won't exist in the case of a static HTML file).

Thus, I am filing a feature request to put that file on a CDN somewhere, and to teach the `tfma_widget_js.js` code to load it from the CDN if necessary.

### Source code / logs

[Here](https://github.com/tensorflow/model-analysis/blob/master/tensorflow_model_analysis/static/index.js#L141) is the line in the JS file where it tries to load the vulcanized_template.html file from the notebook server.

The simplest way I could suggest to enhance this would be to check of the `data-base-url` document attribute is null (indicating that the code is running outside of a notebook), and in that case have the [`__webpack_require__.p`](https://github.com/tensorflow/model-analysis/blob/master/tensorflow_model_analysis/static/index.js#L107) location resolve to a CDN URL.

Hi Ojarjur, This is not connected to the feature here. Could you please let me know how you were able to import tfma. 
` import tensorflow_model_analysis as tfma`
I am working on that for few days with no much success. Many thanks, eilalan

The error from datalab (kernel 2) is:
```
ImportErrorTraceback (most recent call last)
<ipython-input-30-f85e4d8fbd99> in <module>()
----> 1 import tensorflow_model_analysis as tfma

/usr/local/envs/py2env/lib/python2.7/site-packages/tensorflow_model_analysis/__init__.py in <module>()
     15 
     16 
---> 17 from tensorflow_model_analysis import view
     18 # pylint: disable=unused-import
     19 from tensorflow_model_analysis.api import tfma_unit as test

ImportError: cannot import name view


```

@OrielResearchCure The above snippet was not run inside of a notebook, but the process in general is to first install the library from PyPI:

```sh
pip install tensorflow-model-analysis
```

Thank you. This will be very helpful. I am using datalab for all the product development and willing to make any adjustment to make it work on datalab. The tensorflow model analysis is something that I was not able to work with on datalab and had to create a VM with Jupyter notebook. I would appreciate if you could also include the tf_debug option in the tensorboard as well - I think that it is only a redirection of the port as part of the datalab tensorboard library (installation and import work working fine). Many thanks. eilalan

Hello,
Is there an update on this issue?
Is it doable now?

Any updates here?

Could those who want to use TFMA visualization outside of Jupyter / Colab let us know where you are using it? Maybe we can look into why they do not work in those environment.

It is not difficult to implement the sugegsted change of loading from CDN (when static source is not there). However, the visualziation code changes over time so versioning is required to ensure thigns work as time goes by. This requires ongoing support that we currently do not have reousrces for.

Thanks!

I would use it on my website & presentations to show the analysis of the data to anyone who might be interested (in my case pharma/ patients / healthcare).I can show, response to therapy by different slices. Or screening data of patients (cancer & others) by different genomic properties. It is the number one visualization tool for big data that I am familiar with.

Thank you so much for considering it portability to many other platforms.

eilalan

I did some investigation on this issue to try to support https://github.com/kubeflow/pipelines/issues/2283.

A demo of seeing TFMA slicing metrics widget outside of notebook: https://bobgy.github.io/tfma-vis-example/. See below for details.

[The line](https://github.com/tensorflow/model-analysis/blob/a964d2e8430b447c898d271fb6e6d8f5b99adf4b/tensorflow_model_analysis/notebook/jupyter/js/lib/index.js#L23) that assigns notebook public path to webpack public path as mentioned in [the first comment](https://github.com/tensorflow/model-analysis/issues/10#issue-328701594) belongs to a bundle that specializes for notebook (no surprise).

However, I found there is another existing bundle prepared for making the widget portable: https://github.com/tensorflow/model-analysis/blob/a964d2e8430b447c898d271fb6e6d8f5b99adf4b/tensorflow_model_analysis/notebook/jupyter/js/webpack.config.js#L58
Looks like it was planned to be published in npm and loaded by unpkg cdn, but no one ever continued that work and managed the release process.

I just made a fork of this repo and built a bundle using that configuration in my branch: https://github.com/Bobgy/model-analysis/pull/1
So it's now possible to use a public cdn to use my built content like the following (for tfma v0.21.3):
```python
import tensorflow_model_analysis as tfma
from ipywidgets.embed import embed_minimal_html

analysis_path = 'gs://<TFMA_OUTPUT_DIRECTORY>'
result = tfma.load_eval_result(output_path=analysis_path)
slicing_metrics_view = tfma.view.render_slicing_metrics(result)
embed_minimal_html('tfma_export.html', views=[slicing_metrics_view], title='Slicing Metrics')
html = None
with open('tfma_export.html', 'r') as view:
    html = view.read()
# HACK: patch the generated html to use js files in my branch
html = html.replace('dist/embed-amd.js" crossorigin="anonymous"></script>', 'dist/embed-amd.js" crossorigin="anonymous" data-jupyter-widgets-cdn="https://cdn.jsdelivr.net/gh/Bobgy/model-analysis@kfp/tensorflow_model_analysis/notebook/jupyter/js/dist/" crossorigin="anonymous"></script>')
display(HTML(html))
# you can also save this html string to a file again, it is a portable html file that can be used outside of notebook
```

Of course, long term solution is managing npm releases of tfma_widget_js package together with tfma releases. I hope we can do that instead.

tfma 0.21.5 has been released with portable js widget support in https://www.npmjs.com/package/tensorflow_model_analysis

Now we can use the below snippet to generate static html content without any hack.
```
import tensorflow_model_analysis as tfma
from ipywidgets.embed import embed_minimal_html

analysis_path = 'gs://<TFMA_OUTPUT_DIRECTORY>'
result = tfma.load_eval_result(output_path=analysis_path)
slicing_metrics_view = tfma.view.render_slicing_metrics(result)
embed_minimal_html('tfma_export.html', views=[slicing_metrics_view], title='Slicing Metrics')
html = None
with open('tfma_export.html', 'r') as view:
    html = view.read()
display(HTML(html))
```


@mdreves for closing the issue