Repository: keras-team/keras

Issue #5934: CNN-LSTM with video frame sequence: InvalidArgumentError: You must feed a value for placeholder tensor
Created: 2017-03-22T21:18:09Z
I'm building a CNN-LSTM network in Keras (v2.02) + Tensorflow (v1.0.1) using video frames as input. I'm setting up the network as shown below:
import tensorflow as tf
import keras
import cv2

video = keras.layers.Input(shape=(None, 299,299,3),name='video_input')

cnn = keras.applications.InceptionV3(weights='imagenet',
                                 include_top='False',
                                 pooling='avg')

cnn.trainable = False
encoded_frame = keras.layers.TimeDistributed(cnn)(video)
encoded_vid = keras.layers.LSTM(256)(encoded_frame)
outputs = keras.layers.Dense(128, activation='relu')(encoded_vid)

Some of the tensor properties are below:
video
<tf.Tensor 'video_input:0' shape=(?, ?, 299, 299, 3) dtype=float32>

cnn.input
<tf.Tensor 'input_1:0' shape=(?, 299, 299, 3) dtype=float32>

cnn.output
<tf.Tensor 'predictions/Softmax:0' shape=(?, 1000) dtype=float32>    

encoded_frame
<tf.Tensor 'time_distributed_1/Reshape_1:0' shape=(?, ?, 1000) dtype=float32>

encoded_vid
<tf.Tensor 'lstm_1/TensorArrayReadV3:0' shape=(?, 256) dtype=float32>

outputs
<tf.Tensor 'dense_1/Relu:0' shape=(?, 128) dtype=float32>

Now I build the model and fit the data:
model = keras.models.Model(inputs=[video],outputs=outputs)
model.compile(optimizer='adam',
          loss='mean_squared_logarithmic_error')
# Generate random targets
y = np.random.random(size=(128,)) 
y = np.reshape(y,(-1,128))
model.fit(x=frame_sequence, y=y, validation_split=0.0,shuffle=False, batch_size=1)

where frame_sequence is a sequence of video frames from one video:
frame_sequence.shape
(1, 48, 299, 299, 3)

All seems well up to the training step model.fit, where I get an error attributed to the input_1 placeholder in the InceptionV3 model input:
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_1' with dtype float
 [[Node: input_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Training works without error if I build my CNN from scratch instead of loading InceptionV3. For example, replacing InceptionV3 with:
cnn = Sequential()
cnn.add(Conv2D(64, (3, 3), activation='relu', padding='same', input_shape=(229, 229, 3)))
cnn.add(Conv2D(64, (3, 3), activation='relu'))
cnn.add(MaxPooling2D((2, 2)))
cnn.add(Conv2D(128, (3, 3), activation='relu', padding='same'))
cnn.add(Conv2D(128, (3, 3), activation='relu'))
cnn.add(MaxPooling2D((2, 2)))
cnn.add(Conv2D(256, (3, 3), activation='relu', padding='same'))
cnn.add(Conv2D(256, (3, 3), activation='relu'))
cnn.add(Conv2D(256, (3, 3), activation='relu'))
cnn.add(MaxPooling2D((2, 2)))
cnn.add(Flatten())

Here is some minimal code to reproduce the issue.

Duplicate issue #6490: Time Distributed wrapper: You must feed a value for tensor for dtype=bool, Invalid argument error
Created: 2017-05-03T14:49:33Z
I'm trying to build a network with both convolutional and lstm layers. I'm wrapping the Convolutional,activation,max pooling, drop out and flatten layer in a TimeDistributed() layer. When my code reaches the train_on_batch() method I have a illegal argument exception thrown by tensorflow, so I think it is some keras problem. Here it is my code:
def get_model(): 
    model=Sequential()
    model.add(TimeDistributed(Conv2D(10,(3,3)),input_shape=(100,60,80,1)))
    model.add(TimeDistributed(Activation('tanh')))
    model.add(TimeDistributed(Conv2D(10,(3,3))))
    model.add(TimeDistributed(Activation('tanh')))
    model.add(TimeDistributed(MaxPooling2D(pool_size=(2,2))))
    model.add(TimeDistributed(Dropout(0.25)))
    model.add(TimeDistributed(Flatten()))
    model.add(LSTM(1000,return_sequences=True))
    model.add(TimeDistributed(Dense(1,activation='relu')))
    return model
    model=get_model()
model.summary()
rms=RMSprop(lr=0.001)
model.compile(optimizer=rms,loss='mean_squared_error')
for e in range(10):#epoch
    for X,Y in gen('data/train.mp4','train'):#<- this is my custom batch generator
        l=model.train:_on_batch(X,Y)
    model.reset_states()

It gives me :
> Using TensorFlow backend.
> _________________________________________________________________
> Layer (type)                 Output Shape              Param #   
> =================================================================
> time_distributed_1 (TimeDist (None, 100, 58, 78, 10)   100       
> _________________________________________________________________
> time_distributed_2 (TimeDist (None, 100, 58, 78, 10)   0         
> _________________________________________________________________
> time_distributed_3 (TimeDist (None, 100, 56, 76, 10)   910       
> _________________________________________________________________
> time_distributed_4 (TimeDist (None, 100, 56, 76, 10)   0         
> _________________________________________________________________
> time_distributed_5 (TimeDist (None, 100, 28, 38, 10)   0         
> _________________________________________________________________
> time_distributed_6 (TimeDist (None, 100, 28, 38, 10)   0         
> _________________________________________________________________
> time_distributed_7 (TimeDist (None, 100, 10640)        0         
> _________________________________________________________________
> lstm_1 (LSTM)                (None, 100, 1000)         46564000  
> _________________________________________________________________
> time_distributed_8 (TimeDist (None, 100, 1)            1001      
> =================================================================
> Total params: 46,566,011
> Trainable params: 46,566,011
> Non-trainable params: 0
> _________________________________________________________________
> epoca:0
> (1, 100, 60, 80, 1)
> (1, 100, 1)
> 2017-05-03 16:39:04.437304: W tensorflow/core/framework/op_kernel.cc:1152] Invalid argument: You must feed a value for placeholder tensor 'time_distributed_6/keras_learning_phase' with dtype bool
> 	 [[Node: time_distributed_6/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1039, in _do_call
>     return fn(*args)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1021, in _run_fn
>     status, run_metadata)
>   File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
>     next(self.gen)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
>     pywrap_tensorflow.TF_GetCode(status))
> tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'time_distributed_6/keras_learning_phase' with dtype bool
> 	 [[Node: time_distributed_6/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
> 	 [[Node: mul_1/_43 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_1414_mul_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "/home/crow/PycharmProjects/Speed_challenge/main.py", line 78, in <module>
>     l=model.train_on_batch(X,Y)
>   File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 944, in train_on_batch
>     class_weight=class_weight)
>   File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 1633, in train_on_batch
>     outputs = self.train_function(ins)
>   File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 2229, in __call__
>     feed_dict=feed_dict)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 778, in run
>     run_metadata_ptr)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 982, in _run
>     feed_dict_string, options, run_metadata)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1032, in _do_run
>     target_list, options, run_metadata)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1052, in _do_call
>     raise type(e)(node_def, op, message)
> tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'time_distributed_6/keras_learning_phase' with dtype bool
> 	 [[Node: time_distributed_6/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
> 	 [[Node: mul_1/_43 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_1414_mul_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
> 
> Caused by op 'time_distributed_6/keras_learning_phase', defined at:
>   File "/home/crow/PycharmProjects/Speed_challenge/main.py", line 65, in <module>
>     model=get_model()
>   File "/home/crow/PycharmProjects/Speed_challenge/main.py", line 57, in get_model
>     model.add(TimeDistributed(Dropout(0.25)))
>   File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 466, in add
>     output_tensor = layer(self.outputs[0])
>   File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 585, in __call__
>     output = self.call(inputs, **kwargs)
>   File "/usr/local/lib/python3.5/dist-packages/keras/layers/wrappers.py", line 177, in call
>     y = self.layer.call(inputs)  # (num_samples * timesteps, ...)
>   File "/usr/local/lib/python3.5/dist-packages/keras/layers/core.py", line 111, in call
>     training=training)
>   File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 2559, in in_train_phase
>     training = learning_phase()
>   File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 112, in learning_phase
>     name='keras_learning_phase')
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py", line 1507, in placeholder
>     name=name)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 1997, in _placeholder
>     name=name)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
>     op_def=op_def)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
>     original_op=self._default_original_op, op_def=op_def)
>   File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1228, in __init__
>     self._traceback = _extract_stack()
> 
> InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'time_distributed_6/keras_learning_phase' with dtype bool
> 	 [[Node: time_distributed_6/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
> 	 [[Node: mul_1/_43 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_1414_mul_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]



 Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps


 If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.


 If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps


 Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

Closing comment (2017-05-22T08:33:38Z): I'm closing the issue because it is the same as #5934 .