Repository: tensorflow
Issue: Concrete Function output shape sometimes changes after save/load cycle
Labels: stat:awaiting tensorflower,type:bug,comp:apis,TF 2.3

Output of environment capture script:

```
== check python ===================================================
python version: 3.7.8
python branch:
python build version: ('default', 'Aug 10 2020 13:15:25')
python compiler version: Clang 10.0.0 (clang-1000.10.44.4)
python implementation: CPython


== check os platform ===============================================
os: Darwin
os kernel version: Darwin Kernel Version 17.7.0: Thu Dec 20 21:47:19 PST 2018; root:xnu-4570.71.22~1/RELEASE_X86_64
os release version: 17.7.0
os platform: Darwin-17.7.0-x86_64-i386-64bit
linux distribution: ('', '', '')
linux os distribution: ('', '', '')
mac version: ('10.13.6', ('', '', ''), 'x86_64')
uname: uname_result(system='Darwin', node='...', release='17.7.0', version='Darwin Kernel Version 17.7.0: Thu Dec 20 21:47:19 PST 2018; root:xnu-4570.71.22~1/RELEASE_X86_64', machine='x86_64', processor='i386')
architecture: ('64bit', '')
machine: x86_64


== are we in docker =============================================
No

== compiler =====================================================
Apple LLVM version 10.0.0 (clang-1000.10.44.4)
Target: x86_64-apple-darwin17.7.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

== check pips ===================================================
numpy                         1.18.5
protobuf                      3.10.0
tensorflow                    2.3.1
tensorflow-estimator          2.3.0

== check for virtualenv =========================================
False

== tensorflow import ============================================
tf.version.VERSION = 2.3.1
tf.version.GIT_VERSION = v2.3.0-54-gfcc4b966f1
tf.version.COMPILER_VERSION = 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
./tf_env_collect.sh: line 147: nvidia-smi: command not found

== cuda libs  ===================================================

== tensorflow installed from info ==================
Name: tensorflow
Version: 2.3.1
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: .../python3.7/site-packages
Required-by: 

== python version  ==============================================
(major, minor, micro, releaselevel, serial)
(3, 7, 8, 'final', 0)

== bazel version  ===============================================
```

**Describe the current behavior**

The `structured_outputs` of a concrete function might not match after saving and restoring a model containing that function via tf.saved_model. In the example below, the function doesn't change the shape of the tensor, but, when requesting a concrete function for a tensor with shape [None, 3], a more specific output [7, 3] is returned instead.

**Describe the expected behavior**

The output shape should not be more specific than the true output.

**Standalone code to reproduce the issue**
Reproduction script and stdout of `for i in $(seq 10); do echo $i; python repro.py; done` in this Gist: https://gist.github.com/gmacon/057cc64bf849c8d65974daec56a037b7

**Other info / logs**
Initially, I thought this bug was deterministic, but it's clearly actually random. Based on running the reproduction script in a loop, it appears to work correctly about half of the time. Question: does this have something to do with the traversal order of a randomized hash table?

Edit again: I initially thought it was deterministic because it was happening every time in the system I'm building. However, I was reusing the same serialized model over and over in those tests. It appears to work in about half the *saves*, not half the *loads*.

~As far as I can tell, this was working correctly in Tensorflow 2.2.~ Edit: I'm not sure about that any more.

Edit again again: I reproduced this in Tensorflow 2.0.0.