An Approach Towards Convolutional Recurrent Neural Networks | by Chandra Churh Chatterjee | Towards Data Science

The Convolutional Recurrent Neural Networks is the combination of two of the most prominent neural networks. The CRNN (convolutional recurrent neural network) involves CNN(convolutional neural network) followed by the RNN(Recurrent neural networks). The proposed network is similar to the CRNN but generates better or optimal results especially towards audio signal processing.
The network starts with the traditional 2D convolutional neural network followed by batch normalization, ELU activation, max-pooling and dropout with a dropout rate of 50%. Three such convolution layers are placed in a sequential manner with their corresponding activations. The convolutional layers are followed by the permute and the reshape layer which is very necessary for CRNN as the shape of the feature vector differs from CNN to RNN. The convolutional layers are developed on 3-dimensional feature vectors, whereas the recurrent neural networks are developed on 2-dimensional feature vectors.
The permute layers change the direction of the axes of the feature vectors, which is followed by the reshape layers, which convert the feature vector to a 2-dimensional feature vector. The RNN is compatible with the 2-dimensional feature vectors. The proposed network consists of two bidirectional GRU layers with ’n’ no of GRU cells in each layer where ’n’ depends on the no of classes of the classification performed using the corresponding network. The bidirectional GRU (Gated recurrent unit) is used instead of the unidirectional RNN layers because the bidirectional layers take into account not only the future timestamps but also the future timestamp representations as well. Incorporating two-dimensional representations from both the timestamps allows incorporating the time dimensional features in a very optimal manner.
Finally, the output of the bidirectional layers is fed to the time distributed dense layers followed by the Fully connected layer.
The implementation of the proposed network is as follows:
def get_model(data_in, data_out, _cnn_nb_filt, _cnn_pool_size, _rnn_nb, _fc_nb):spec_start = Input(shape=(data_in.shape[-3], data_in.shape[-2], data_in.shape[-1])) spec_x = spec_start for _i, _cnt in enumerate(_cnn_pool_size): spec_x = Conv2D(filters = cnn_nb_filt, kernel_size=(2, 2), padding=’same’)(spec_x) spec_x = BatchNormalization(axis=1)(spec_x) spec_x = Activation(‘relu’)(spec_x) spec_x = MaxPooling2D(pool_size=(1, _cnn_pool_size[_i]))(spec_x) spec_x = Dropout(dropout_rate)(spec_x) spec_x = Permute((2, 1, 3))(spec_x) spec_x = Reshape((data_in.shape[-2], -1))(spec_x)for _r in _rnn_nb: spec_x = Bidirectional( GRU(_r, activation=’tanh’, dropout=dropout_rate, recurrent_dropout=dropout_rate, return_sequences=True), merge_mode=’concat’)(spec_x)for _f in _fc_nb: spec_x = TimeDistributed(Dense(_f))(spec_x) spec_x = Dropout(dropout_rate)(spec_x)spec_x = TimeDistributed(Dense(data_out.shape[-1]))(spec_x) out = Activation(‘sigmoid’, name=’strong_out’)(spec_x)_model = Model(inputs=spec_start, outputs=out) _model.compile(optimizer=’Adam’, loss=’binary_crossentropy’,metrics = [‘accuracy’]) _model.summary() return _model
The model summary can be displayed, which is as follows:
# Load model”model = get_model(X, Y, cnn_nb_filt, cnn_pool_size, rnn_nb, fc_nb)