Package: tensorflow-model-analysis
TensorFlow Model AnalysisTensorFlow Model Analysis(TFMA) is a library for evaluating TensorFlow
models.  It allows users to evaluate their models on large amounts of data in a
distributed manner, using the same metrics defined in their trainer. These
metrics can be computed over different slices of data and visualized in Jupyter
notebooks.Caution: TFMA may introduce backwards incompatible changes before version 1.0.InstallationThe recommended way to install TFMA is using thePyPI package:pip install tensorflow-model-analysispip install fromhttps://pypi-nightly.tensorflow.orgpip install -i https://pypi-nightly.tensorflow.org/simple tensorflow-model-analysispip install from the HEAD of the git:pip install git+https://github.com/tensorflow/model-analysis.git#egg=tensorflow_model_analysispip install from a released version directly from git:pip install git+https://github.com/tensorflow/model-analysis.git@v0.21.3#egg=tensorflow_model_analysisIf you have cloned the repository locally, and want to test your local change,
pip install from a local folder.pip install -e $FOLDER_OF_THE_LOCAL_LOCATIONNote that protobuf must be installed correctly for the above option since it is
building TFMA from source and it requires protoc and all of its includes
reference-able. Please seeprotobuf install instructionfor see the latest install instructions.Currently, TFMA requires that TensorFlow is installed but does not have an
explicit dependency on the TensorFlow PyPI package. See theTensorFlow install guidesfor
instructions.Build TFMA from sourceTo build from source follow the following steps:Install the protoc as per the link mentioned:protocCreate a virtual environment by running the commandspython3 -m venv <virtualenv_name>
source <virtualenv_name>/bin/activate
pip3 install setuptools wheel
git clone https://github.com/tensorflow/model-analysis.git
cd model-analysis
python3 setup.py bdist_wheelThis will build the TFMA wheel in the dist directory. To install the wheel from
dist directory run the commandscd dist
pip3 install tensorflow_model_analysis-<version>-py3-none-any.whlJupyter LabAs of writing, because ofhttps://github.com/pypa/pip/issues/9187,pip installmight never finish. In that case, you should revert pip to version 19 instead of
20:pip install "pip<20".Using a JupyterLab extension requires installing dependencies on the command
line. You can do this within the console in the JupyterLab UI or on the command
line. This includes separately installing any pip package dependencies and
JupyterLab labextension plugin dependencies, and the version numbers must be
compatible.  JupyterLab labextension packages refer to npm packages
(eg,tensorflow_model_analysis.The examples below use 0.32.0. Check availableversionsbelow to use the latest.Jupyter Lab 3.0.xpipinstalltensorflow_model_analysis==0.32.0
jupyterlabextensioninstalltensorflow_model_analysis@0.32.0
pipinstalljupyterlab_widgets==1.0.0Jupyter Lab 2.2.xpipinstalltensorflow_model_analysis==0.32.0
jupyterlabextensioninstalltensorflow_model_analysis@0.32.0
jupyterlabextensioninstall@jupyter-widgets/jupyterlab-manager@2Jupyter Lab 1.2.xpipinstalltensorflow_model_analysis==0.32.0
jupyterlabextensioninstalltensorflow_model_analysis@0.32.0
jupyterlabextensioninstall@jupyter-widgets/jupyterlab-manager@1.1Classic Jupyter NotebookTo enable TFMA visualization in the classic Jupyter Notebook (either throughjupyter notebookorthrough the JupyterLab UI),
you'll also need to run:jupyternbextensionenable--pywidgetsnbextension
jupyternbextensionenable--pytensorflow_model_analysisNote: If Jupyter notebook is already installed in your home directory, add--userto these commands. If Jupyter is installed as root, or using a virtual
environment, the parameter--sys-prefixmight be required.Building TFMA from sourceIf you want to build TFMA from source and use the UI in JupyterLab, you'll need
to make sure that the source contains valid version numbers.  Check that the
Python package version number and npm package version number are exactly the
same, and that both valid version numbers (eg, remove the-devsuffix).TroubleshootingCheck pip packages:piplistCheck JupyterLab extensions:jupyterlabextensionlist# for JupyterLabjupyternbextensionlist# for classic Jupyter NotebookStandalone HTML page withembed_minimal_htmlTFMA notebook extension can be built into a standalone HTML file that also
bundles data into the HTML file.  See the Jupyter Widgets docs onembed_minimal_html.Kubeflow PipelinesKubeflow Pipelinesincludes integrations that embed the TFMA notebook extension (code).
This integration relies on network access at runtime to load a variant of the
JavaScript build published on unpkg.com (seeconfigandloader code).Notable DependenciesTensorFlow is required.Apache Beamis required; it's the way that efficient
distributed computation is supported. By default, Apache Beam runs in local
mode but can also run in distributed mode usingGoogle Cloud Dataflowand other Apache
Beamrunners.Apache Arrowis also required. TFMA uses Arrow to
represent data internally in order to make use of vectorized numpy functions.Getting StartedFor instructions on using TFMA, see theget started
guide.Compatible VersionsThe following table is the TFMA package versions that are compatible with each
other. This is determined by our testing framework, but otheruntestedcombinations may also work.tensorflow-model-analysisapache-beam[gcp]pyarrowtensorflowtensorflow-metadatatfx-bslGitHub master2.47.010.0.0nightly (2.x)1.14.01.14.00.45.02.47.010.0.02.131.14.01.14.00.44.02.40.06.0.02.121.13.11.13.00.43.02.40.06.0.02.111.12.01.12.00.42.02.40.06.0.01.15.5 / 2.101.11.01.11.10.41.02.40.06.0.01.15.5 / 2.91.10.01.10.10.40.02.38.05.0.01.15.5 / 2.91.9.01.9.00.39.02.38.05.0.01.15.5 / 2.81.8.01.8.00.38.02.36.05.0.01.15.5 / 2.81.7.01.7.00.37.02.35.05.0.01.15.5 / 2.71.6.01.6.00.36.02.34.05.0.01.15.5 / 2.71.5.01.5.00.35.02.33.05.0.01.15 / 2.61.4.01.4.00.34.12.32.02.0.01.15 / 2.61.2.01.3.00.34.02.31.02.0.01.15 / 2.61.2.01.3.10.33.02.31.02.0.01.15 / 2.51.2.01.2.00.32.12.29.02.0.01.15 / 2.51.1.01.1.10.32.02.29.02.0.01.15 / 2.51.1.01.1.00.31.02.29.02.0.01.15 / 2.51.0.01.0.00.30.02.28.02.0.01.15 / 2.40.30.00.30.00.29.02.28.02.0.01.15 / 2.40.29.00.29.00.28.02.28.02.0.01.15 / 2.40.28.00.28.00.27.02.27.02.0.01.15 / 2.40.27.00.27.00.26.12.28.00.17.01.15 / 2.30.26.00.26.00.26.02.25.00.17.01.15 / 2.30.26.00.26.00.25.02.25.00.17.01.15 / 2.30.25.00.25.00.24.32.24.00.17.01.15 / 2.30.24.00.24.10.24.22.23.00.17.01.15 / 2.30.24.00.24.00.24.12.23.00.17.01.15 / 2.30.24.00.24.00.24.02.23.00.17.01.15 / 2.30.24.00.24.00.23.02.23.00.17.01.15 / 2.30.23.00.23.00.22.22.20.00.16.01.15 / 2.20.22.20.22.00.22.12.20.00.16.01.15 / 2.20.22.20.22.00.22.02.20.00.16.01.15 / 2.20.22.00.22.00.21.62.19.00.15.01.15 / 2.10.21.00.21.30.21.52.19.00.15.01.15 / 2.10.21.00.21.30.21.42.19.00.15.01.15 / 2.10.21.00.21.30.21.32.17.00.15.01.15 / 2.10.21.00.21.00.21.22.17.00.15.01.15 / 2.10.21.00.21.00.21.12.17.00.15.01.15 / 2.10.21.00.21.00.21.02.17.00.15.01.15 / 2.10.21.00.21.00.15.42.16.00.15.01.15 / 2.0n/a0.15.10.15.32.16.00.15.01.15 / 2.0n/a0.15.10.15.22.16.00.15.01.15 / 2.0n/a0.15.10.15.12.16.00.15.01.15 / 2.0n/a0.15.00.15.02.16.00.15.01.15n/an/a0.14.02.14.0n/a1.14n/an/a0.13.12.11.0n/a1.13n/an/a0.13.02.11.0n/a1.13n/an/a0.12.12.10.0n/a1.12n/an/a0.12.02.10.0n/a1.12n/an/a0.11.02.8.0n/a1.11n/an/a0.9.22.6.0n/a1.9n/an/a0.9.12.6.0n/a1.10n/an/a0.9.02.5.0n/a1.9n/an/a0.6.02.4.0n/a1.6n/an/aQuestionsPlease direct any questions about working with TFMA toStack Overflowusing thetensorflow-model-analysistag.