Issue 1:
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10

Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A

TensorFlow installed from (source or binary): Binary, pip install

TensorFlow version (use command below): tensorflow-gpu==2.0.0-beta1

Python version: 3.6

Bazel version (if compiling from source): N/A

GCC/Compiler version (if compiling from source): N/A

CUDA/cuDNN version: N/A

GPU model and memory: N/A

Describe the current behavior

Error arises during Concatenate when I run the following code:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Conv2D, Concatenate

inputs = keras.Input(shape=(256,256,3))
x = Conv2D(16,3, padding='same',activation='relu')(inputs)
x_list = [x]
for i in range(3):
x = Conv2D(16,3, padding='same',activation='relu')(x)
x_list.append(x)
x = Concatenate(3)(x_list)

model = keras.Model(inputs=inputs, outputs=x)
model.summary()

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("conv2d_31/Identity:0", shape=(None, 256, 256, 16), dtype=float32) at layer "concatenate_8". The following previous layers were accessed without issue: ['input_9', 'conv2d_29', 'conv2d_30']

This issue does not occur in a Tensorflow 1.X environment, only TF 2.0

Describe the expected behavior

Now the Concatenate function works properly when using a sequential model. That is, if I swap in "for i in range(1):" rather than "for i in range(3):" above, the code executes cleanly. However, the non-sequential repeated Concatenation in the loop leaves the a Graph disconnected error.

Furthermore, the error is also eliminated when using tf.concat, so the following code also executes cleanly.

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Conv2D, Concatenate

inputs = keras.Input(shape=(256,256,3))
x = Conv2D(16,3, padding='same',activation='relu')(inputs)
x_list = [x]
for i in range(3):
x = Conv2D(16,3, padding='same',activation='relu')(x)
x_list.append(x)
x = tf.concat(x_list, 3)

model = keras.Model(inputs=inputs, outputs=x)
model.summary()

Therefore, I do have a working alternative, but there does appear to be an issue with the keras Concatenate function

Issue 2:
tf.keras.layers.Concatenate used to operate in a very straightforward way with the Keras functional API for building DenseNet-esque feedforward networks.

The code below works in TF 1.13 but fails in 1.14, 2.0.0a0 and beyond.

I also tested replacing the Concatenate layer with its functional alternative, concatenate, and produced the same error.

from tensorflow.keras.layers import Dense, Input, Concatenate
from tensorflow.keras.models import Model

inputs = Input(shape=(10,))

all_layers = []

x1 = Dense(512)(inputs)
all_layers.append(x1)

# all layers: [x1]
x2 = Dense(256, activation='relu')(x1)
all_layers.append(x2)

# all layers: [x1, x2]
conc = Concatenate()(all_layers)
x3 = Dense(128, activation='relu')(conc)
all_layers.append(x3)

# all layers: [x1, x2, x3]
conc = Concatenate()(all_layers)
prediction = Dense(1)(conc)
model = Model(inputs=inputs, outputs=prediction)

The error output is

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("dense_2/Identity:0", shape=(None, 128), dtype=float32) at layer "concatenate". The following previous layers were accessed without issue: ['input_1', 'dense', 'dense_1']

Visualization of model