Repository: tensorflow/model-analysis
Issue: Build configuration is missing definitions
Root cause: Architectural Constraint

### System information

-   **Have I written custom code (as opposed to using a stock example script
    provided in TensorFlow Model Analysis)**: N/A
-   **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: N/A
-   **TensorFlow Model Analysis installed from (source or binary)**: N/A
-   **TensorFlow Model Analysis version (use command below)**: N/A
-   **Python version**: N/A
-   **Jupyter Notebook version**: N/A
-   **Exact command to reproduce**: N/A

### Describe the problem

The [HEAD](https://github.com/tensorflow/model-analysis/commit/2fb97d4d6bac1b800c340ca9a6f2393bd5169cf2) version of TFMA seems to be missing some definitions for third-party dependencies. The easy-to-fix one is ProtoBuf:

```
ERROR: error loading package 'tensorflow_model_analysis/proto': Extension file not found. Unable to load package for '@protobuf_bzl//:protobuf.bzl': The repository could not be resolved
```

which can be fixed by changing the load call slightly 

```diff
diff --git a/tensorflow_model_analysis/proto/BUILD b/tensorflow_model_analysis/proto/BUILD
index af3386c..af787da 100644
--- a/tensorflow_model_analysis/proto/BUILD
+++ b/tensorflow_model_analysis/proto/BUILD
@@ -2,7 +2,7 @@ licenses(["notice"])  # Apache 2.0

 package(default_visibility = ["//visibility:public"])

-load("@protobuf_bzl//:protobuf.bzl", "py_proto_library")
+load("@com_google_protobuf//:protobuf.bzl", "py_proto_library")
```

The other one is a bit more difficult since the `BUILD` file for `third_party/py/typing` is truly missing from the repo

```
ERROR: [...]/model-analysis/tensorflow_model_analysis/slicer/BUILD:3:1: no such package 'third_party/py/typing': BUILD file not found on package path and referenced by '//tensorflow_model_analysis/slicer:slicer'
```

Finally, some of the TFMA `BUILD` files reference `third_party/py/numpy` which is missing as well.

Hi Sergei,

What command did you run? If you would like to generate whl file from source codes, you could simply run `python bdist setup.py`.

I wanted to run tests with Bazel. Would you recommend running them via e.g. pytest instead?

It turns out that our BUILD files need to be fixed quite a bit to get them to work. We'll work on fixing the BUILD files and pushing a new version with the fixed BUILD files soon. In the meantime, you can try running the tests directly with Python. Sorry for the inconvenience, and thanks for your patience!

Okay, after some discussion, we'll be removing the BUILD files for the Python targets for now, since they don't really add much. Could you try running the tests with Python directly, and let us know if you run into any issues?

I have tried running the tests locally:

```
============== 23 failed, 85 passed, 1 warnings in 247.46 seconds ==============
```

most failures involve the repr of dilll internals and are tricky to diagnose. Full output of pytest: [failures.txt](https://github.com/tensorflow/model-analysis/files/2307867/failures.txt)


My guess is that they have something to do with Beam. Which version of Beam do you have installed?

2.6.0 which matches the `apache-beam[gcp]>=2.5,<3` constraint in setup.py.

I have the same number of failures with 2.5.0 and 2.4.0, and the error messages look similar. Could it be the dill version then?

The failures are caused by pytest. I think it instruments unittest classes (or possibly instances) in a way which makes them non-pickleable. Therefore, the error: "cannot pickle generators".

Interestingly, the error goes away if pytest is executed with the`--pdb` flag :)

Would you accept a PR adding Travis CI support?

I see, thanks for looking into this!

It would be nice to have CI support! Let me check with some folks to see whether there's a preferred solution or whether Travis CI would be fine.

Okay, so I checked with some other folks, and it looks like the plan is to set up a CI solution similar to what we have with the TensorFlow repository. Thanks for offering to add Travis CI support though!

Thanks for checking, closing the issue.