Issue 1:
Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS, Linux

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A

TensorFlow installed from (source or binary): Anaconda binary

TensorFlow version (use command below): 1.14

Python version: 3.7.3

Bazel version (if compiling from source): N/A

GCC/Compiler version (if compiling from source): N/A

CUDA/cuDNN version: N/A

GPU model and memory: N/A

You can collect some of this information using our environment capture

script

You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

$ python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

unknown 1.14.0

Describe the current behavior

cloudpickle cannot unpickle tf.keras because the deprecation_wrapper introduced in 1.14.

In one python session:

import cloudpickle
import tensorflow.keras as K
with open("/tmp/K.pkl", "wb") as f:
cloudpickle.dump(K, f)

Then start another python session:

import cloudpickle
with open("/tmp/K.pkl", "rb") as f:
cloudpickle.load(f)

Error:

Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/Users/meng/conda/envs/tf-1.14/lib/python3.7/site-packages/tensorflow/python/util/deprecation_wrapper.py", line 148, in __setstate__
sys.modules[d]._dw_wrapped_module,
KeyError: 'tensorflow.keras'

Describe the expected behavior

The same code worked in TensorFlow 1.13. In TensorFlow 1.13, K.__module__ does not exist, while in TensorFlow 1.14, K.__module__ is

'tensorflow.python.util.deprecation_wrapper'

It makes the tensorflow.keras module not loaded during unpickling.

There are multiple applications use pickle to ship TensorFlow code to a remote machine to unpickle and execute, e.g., Spark, horovod.spark. So the behavior change would fail the jobs.

Btw, one workaround is to do from tensorflow import keras as K instead of import tensorflow.keras as K. The diff is that K.__name__ is 'tensorflow.python.keras.api._v1.keras' in the first case and tensorflow.keras in the second. And pickle uses __name__ to get global names: https://github.com/python/cpython/blob/3.7/Lib/pickle.py#L952

cc: @alsrgv @hanyucui @annarev

Code to reproduce the issue

Provide a reproducible test case that is the bare minimum necessary to generate the problem.

See above.

Other info / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

Issue 2:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes (code included below in the issue)

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS 10.14.3

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A

TensorFlow installed from (source or binary): pip

TensorFlow version (use command below): v2.0.0-beta1-5101-gc75bb66a99 2.0.0-rc0

Python version: Python 3.6.7 :: Anaconda, Inc.

Bazel version (if compiling from source): N/A

GCC/Compiler version (if compiling from source): N/A

CUDA/cuDNN version: N/A

GPU model and memory: N/A

Using cloudpickle to serialize a Python function that uses tf.keras.Sequential fails with a recursion error.

Note that this works with tensorflow==1.14.0.

I imagine it also fails with other things, not just tf.keras.Sequential.

import cloudpickle # cloudpickle.__version__ == '1.2.1'
import tensorflow as tf # tf.__version__ == '2.0.0-rc0'

def f():
tf.keras.Sequential

cloudpickle.loads(cloudpickle.dumps(f)) # This fails.

The last line fails with

---------------------------------------------------------------------------
RecursionError Traceback (most recent call last)
<ipython-input-23-25cc307e6227> in <module>
----> 1 cloudpickle.loads(cloudpickle.dumps(f))

~/anaconda3/lib/python3.6/site-packages/tensorflow/__init__.py in __getattr__(self, item)
48
49 def __getattr__(self, item):
---> 50 module = self._load()
51 return getattr(module, item)
52

~/anaconda3/lib/python3.6/site-packages/tensorflow/__init__.py in _load(self)
42 def _load(self):
43 """Import the target module and insert it into the parent's namespace."""
---> 44 module = _importlib.import_module(self.__name__)
45 self._parent_module_globals[self._local_name] = module
46 self.__dict__.update(module.__dict__)

... last 2 frames repeated, from the frame below ...

~/anaconda3/lib/python3.6/site-packages/tensorflow/__init__.py in __getattr__(self, item)
48
49 def __getattr__(self, item):
---> 50 module = self._load()
51 return getattr(module, item)
52

RecursionError: maximum recursion depth exceeded while calling a Python object

See https://stackoverflow.com/questions/57750920/ray-tensorflow-gpu-2-0-recursionerror/57761034#57761034