Issue 1:
Hi,

For example, I'd like to insert some new layers to VGG model before the dense layers, load the parameters, freeze them and continue training.

I followed some old issues, which are popping up the top dense and outupt layers, adding new layers and the dense and output layers again.

When I try to add new layers, I failed with this error:

Exception: Input 0 is incompatible with layer pool6_zp: expected ndim=4, found ndim=2

Actually I also tried just add back the flatten layer and failed too.

I'm using Keras 1.0.4 with tensorflow as backend.

I'm really confused. How can I do that?

Thanks a lot!

Here is the test code:

from keras.layers import (AveragePooling2D, Convolution2D, Dense, Dropout,
Flatten, Input, MaxPooling2D, ZeroPadding2D)
from keras.models import Model, Sequential

vgg_model = Sequential()
vgg_model.add(Convolution2D(64, 3, 3, activation='relu', input_shape=(3, 224, 224), border_mode='same', name='conv1_1'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv1_1_zp'))
vgg_model.add(Convolution2D(64, 3, 3, activation='relu', name='conv1_2'))
vgg_model.add(MaxPooling2D((2, 2), strides=(2, 2), name='pool1'))

vgg_model.add(ZeroPadding2D((1, 1), name='pool1_zp'))
vgg_model.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_1'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv2_1_zp'))
vgg_model.add(Convolution2D(128, 3, 3, activation='relu', name='conv2_2'))
vgg_model.add(MaxPooling2D((2, 2), strides=(2, 2), name='pool2'))

vgg_model.add(ZeroPadding2D((1, 1), name='pool2_zp'))
vgg_model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_1'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv3_1_zp'))
vgg_model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_2'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv3_2_zp'))
vgg_model.add(Convolution2D(256, 3, 3, activation='relu', name='conv3_3'))
vgg_model.add(MaxPooling2D((2, 2), strides=(2, 2), name='pool3'))

vgg_model.add(ZeroPadding2D((1, 1), name='conv4_1_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_2'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv4_2_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv4_3'))
vgg_model.add(MaxPooling2D((2, 2), strides=(2, 2), name='pool4'))

vgg_model.add(ZeroPadding2D((1, 1), name='pool4_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_1'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv5_1_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_2'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv5_2_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv5_3'))
vgg_model.add(MaxPooling2D((2, 2), strides=(2, 2), name='pool5'))

vgg_model.add(Flatten())
vgg_model.add(Dense(4096, activation='relu', name='fc6'))
vgg_model.add(Dropout(0.5, name='fc6do'))
vgg_model.add(Dense(4096, activation='relu', name='fc7'))
vgg_model.add(Dropout(0.5, name='fc7do'))
vgg_model.add(Dense(1000, activation='softmax', name='vgg_classes'))

del vgg_model.layers[-6:]
'''
vgg_model.add(ZeroPadding2D((1, 1), name='pool6_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv6_1'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv6_1_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv6_2'))
vgg_model.add(ZeroPadding2D((1, 1), name='conv6_2_zp'))
vgg_model.add(Convolution2D(512, 3, 3, activation='relu', name='conv6_3'))
vgg_model.add(MaxPooling2D((2, 2), strides=(2, 2), name='pool5'))
'''
vgg_model.add(Flatten())
vgg_model.add(Dense(4096, activation='relu', name='fc6'))
vgg_model.add(Dropout(0.5, name='fc6do'))
vgg_model.add(Dense(4096, activation='relu', name='fc7'))
vgg_model.add(Dropout(0.5, name='fc7do'))
vgg_model.add(Dense(1000, activation='softmax', name='vgg_classes'))

Issue 2:
i want to apply timedistributed to dropout - the reason is i want each timestep to have exactly x% of the features dropped.

i am also using batch_shape for my input with a given batch size since this is required for stateful rnn in my full code. when batch size is given the timedistributed wrapper uses rnn. my backend is theano

this gist recreates the issue

https://gist.github.com/eyaler/6801eee9b2134bcd518ffd34004c1475

may be related to:

#3834

see first comment for error message