Repository: tensorflow/model-analysis
Issue: MultiClassConfusionMatrixPlot - no plot data found
Root cause: Validation Gap

-   **Have I written custom code (as opposed to using a stock example script
    provided in TensorFlow Model Analysis)**: Yes. See below.
-   **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: macOS 10.14.6
-   **TensorFlow Model Analysis installed from (source or binary)**: pip
-   **TensorFlow Model Analysis version (use command below)**: 0.21.2
-   **Python version**: 3.6.8
-   **Jupyter Notebook version**: 6.0.3
-   **Exact command to reproduce**:

### Describe the problem

I have an `EvalResult` that contains data for a multiclass confusion matrix but `tfma.view.render_plot` can't seem to find it.

### Source code / logs

```python
import tensorflow as tf
import tensorflow_model_analysis as tfma
m = tfma.default_eval_shared_model('train', tags='serve', example_weight_key='sample_weight', include_default_metrics=False)
eval_config = tfma.EvalConfig(
    model_specs=[tfma.ModelSpec(signature_name='eval', label_key='activity1', example_weight_key='sample_weight')],
    metrics_specs=tfma.metrics.specs_from_metrics([
        tfma.metrics.MultiClassConfusionMatrixPlot(),
    ], include_weighted_example_count=False),
    slicing_specs=[
        tfma.SlicingSpec(),
    ]
)

import numpy as np
from tensorflow_model_analysis import types
import apache_beam as beam

LABELS = [...]

eye = np.eye(len(LABELS))

@beam.ptransform_fn
@beam.typehints.with_input_types(types.Extracts)
@beam.typehints.with_output_types(types.Extracts)
def f(extracts):
    def _f(ex):
        ex['labels'] = eye[LABELS.index(ex['labels'][0].decode())]
        return ex
    return extracts | 'OHE' >> beam.Map(_f)

integerize_labels = tfma.extractors.Extractor('ohe', f())

result = tfma.run_model_analysis(
    eval_shared_model=m,
    eval_config=eval_config,
    data_location='...',
    extractors=tfma.default_extractors(eval_shared_model=m, eval_config=eval_config) + [integerize_labels],
    evaluators=[
         tfma.evaluators.metrics_and_plots_evaluator_v2.MetricsAndPlotsEvaluator(eval_config=eval_config, eval_shared_model=m, run_after='ohe')
    ]
)
```

```python
tfma.view.render_plot(result)
```

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-84-348d3cfecd7f> in <module>
----> 1 tfma.view.render_plot(result)

/anaconda3/envs/gcp_test_env/lib/python3.6/site-packages/tensorflow_model_analysis/view/widget_view.py in render_plot(result, slicing_spec, output_name, class_id, top_k, k, label)
    109   data, config = util.get_plot_data_and_config(result.plots, slice_spec_to_use,
    110                                                output_name, class_id, top_k, k,
--> 111                                                label)
    112   return visualization.render_plot(data, config)

/anaconda3/envs/gcp_test_env/lib/python3.6/site-packages/tensorflow_model_analysis/view/util.py in get_plot_data_and_config(results, slicing_spec, output_name, class_id, top_k, k, label)
    335     if not contains_supported_plot_data:
    336       raise ValueError('No plot data found. Maybe provide a label? %s' %
--> 337                        json.dumps(plot_data))
    338 
    339   plot_data = _replace_nan_with_none(plot_data, _SUPPORTED_PLOT_KEYS)

ValueError: No plot data found. Maybe provide a label? {"multiClassConfusionMatrixAtThresholds": {"matrices": [{"entries": [{"numWeightedExamples": 23878.0}, {"predictedClassId": 2, "numWeightedExamples": 4.0}, {"predictedClassId": 3, "numWeightedExamples": 873.0}, {"actualClassId": 1, "predictedClassId": 1, "numWeightedExamples": 306.0}, {"actualClassId": 2, "predictedClassId": 2, "numWeightedExamples": 1173.0}, {"actualClassId": 2, "predictedClassId": 3, "numWeightedExamples": 22.0}, {"actualClassId": 3, "predictedClassId": 3, "numWeightedExamples": 52.0}, {"actualClassId": 4, "predictedClassId": 4, "numWeightedExamples": 1754.0}, {"actualClassId": 5, "predictedClassId": 1, "numWeightedExamples": 37.0}, {"actualClassId": 5, "predictedClassId": 2, "numWeightedExamples": 273.0}, {"actualClassId": 5, "predictedClassId": 5, "numWeightedExamples": 9.0}, {"actualClassId": 5, "predictedClassId": 6, "numWeightedExamples": 4.0}, {"actualClassId": 6, "predictedClassId": 1, "numWeightedExamples": 361.0}, {"actualClassId": 6, "predictedClassId": 2, "numWeightedExamples": 106.0}, {"actualClassId": 6, "predictedClassId": 6, "numWeightedExamples": 5.0}]}]}}
```


@mdreves any update on this?

I'm working on a potential fix, but can you confirm whether you can see other plots such as the CalibrationPlot.

Yes, I can see the calibration plot, precision/recall plot, etc.

I've updated the missing plot specs, but I haven't fully verified the fix yet. If you are blocked and can't wait, you can try at head and it may or may not fix the issue.

Awesome, thank you!

This should be fixed at head.