Repository: tensorflow
Issue: Model checkpoint load ignores wrong shape
Labels: stat:awaiting tensorflower,type:bug,comp:ops,2.6.0

**System information**
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOSX 11.6 + Ubuntu 20.04
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
- TensorFlow installed from (source or binary): pip binary
- TensorFlow version (use command below):  v2.6.0-rc2-32-g919f693420e 2.6.0
- Python version: 3.9.6 + 3.8
- Bazel version (if compiling from source): -
- GCC/Compiler version (if compiling from source): -
- CUDA/cuDNN version: None
- GPU model and memory: None

**Describe the current behavior**

Saving a variable of shape (3,3,40,32) into a checkpoint, and then loading the variable with shape (3,3,1,32) from this checkpoint does not cause an error. Instead, it loads fine. The variable shape still claims to be (3,3,1,32), but when evaluated, one can see that it is indeed (3,3,40,32).

So there does not seem to be any validation of the shape during checkpoint loading.

**Describe the expected behavior**

In general, if a tensor or variable claims to be of some static shape, I think it should never be possible that its actual shape when evaluated is different.

**Standalone code to reproduce the issue**
```

import tensorflow as tf

print("TF:", tf.__version__)
tf.compat.v1.disable_eager_execution()

filename = "test-ckpt-diff-shape.model"


with tf.Graph().as_default() as graph:
    with tf.compat.v1.Session(graph=graph) as session:
        shape1 = (3,3,40,32)
        v = tf.compat.v1.get_variable(name="W", shape=shape1)
        print(v)
        saver = tf.compat.v1.train.Saver(var_list=[v])
        session.run(tf.compat.v1.global_variables_initializer())
        saver.save(sess=session, save_path=filename)


with tf.Graph().as_default() as graph:
    with tf.compat.v1.Session(graph=graph) as session:
        shape2 = (3,3,1,32)
        v = tf.compat.v1.get_variable(name="W", shape=shape2)
        print(v)
        saver = tf.compat.v1.train.Saver(var_list=[v])
        saver.restore(sess=session, save_path=filename)
        v_raw = session.run(v)
        print(v)
        print(v_raw.shape)
        assert v.shape.as_list() == list(shape2)
        assert v.shape.as_list() == list(v_raw.shape)
```

**Other info / logs**

I stumbled upon this problem because it causes a seemingly unrelated error for some following 2D convolution where this variable is used as a kernel:
```
2021-09-30 12:52:25.768174: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at conv_ops.cc:1
115 : Not found: No algorithm worked!
TensorFlow exception: 2 root error(s) found.
  (0) Not found: No algorithm worked!
         [[node conv0/convolution (defined at u/schmitt/src/returnn/returnn/tf/layers/basic.py:4061) ]]
         [[output/rec/while/Switch_14/_553]]
  (1) Not found: No algorithm worked!
         [[node conv0/convolution (defined at u/schmitt/src/returnn/returnn/tf/layers/basic.py:4061) ]]
0 successful operations.
```
This was reported here: https://github.com/rwth-i6/returnn/issues/703

There are a couple of these errors also reported here:
* https://github.com/tensorflow/tensorflow/issues/43174
* https://github.com/tensorflow/tensorflow/issues/45044
* https://github.com/tensorflow/tensorflow/issues/48117

I wonder if some of them have a similar source.

But anyway, this issue is not really about the convolution error, but about the checkpoint loading behavior, and the variable shape.