Keras to Pytorch Convolutional Neural Network

username: PyAP
cooked: <p>I am trying to convert this keras model to pytorch.</p>
<h1>
<a name="build-vt-cnn2-neural-net-model-using-keras-primitives-1" class="anchor" href="#build-vt-cnn2-neural-net-model-using-keras-primitives-1"></a>Build VT-CNN2 Neural Net model using Keras primitives –</h1>
<h1>
<a name="reshape-n2128-to-n12128-on-input-2" class="anchor" href="#reshape-n2128-to-n12128-on-input-2"></a>- Reshape [N,2,128] to [N,1,2,128] on input</h1>
<h1>
<a name="pass-through-2-2dconvrelu-layers-3" class="anchor" href="#pass-through-2-2dconvrelu-layers-3"></a>- Pass through 2 2DConv/ReLu layers</h1>
<h1>
<a name="pass-through-2-dense-layers-relu-and-softmax-4" class="anchor" href="#pass-through-2-dense-layers-relu-and-softmax-4"></a>- Pass through 2 Dense layers (ReLu and Softmax)</h1>
<h1>
<a name="perform-categorical-cross-entropy-optimization-5" class="anchor" href="#perform-categorical-cross-entropy-optimization-5"></a>- Perform categorical cross entropy optimization</h1>
<p>dr = 0.5 # dropout rate (%)<br>
model = models.Sequential()<br>
model.add(Reshape([1]+in_shp, input_shape=in_shp))<br>
model.add(ZeroPadding2D((0, 2)))<br>
model.add(Convolution2D(256, 1, 3, border_mode=‘valid’, activation=“relu”, name=“conv1”, init=‘glorot_uniform’))<br>
model.add(Dropout(dr))<br>
model.add(ZeroPadding2D((0, 2)))<br>
model.add(Convolution2D(80, 2, 3, border_mode=“valid”, activation=“relu”, name=“conv2”, init=‘glorot_uniform’))<br>
model.add(Dropout(dr))<br>
model.add(Flatten())<br>
model.add(Dense(256, activation=‘relu’, init=‘he_normal’, name=“dense1”))<br>
model.add(Dropout(dr))<br>
model.add(Dense( len(classes), init=‘he_normal’, name=“dense2” ))<br>
model.add(Activation(‘softmax’))<br>
model.add(Reshape([len(classes)]))<br>
model.compile(loss=‘categorical_crossentropy’, optimizer=‘adam’)<br>
model.summary()</p>
<h1>
<a name="layer-type-output-shape-param-connected-to-6" class="anchor" href="#layer-type-output-shape-param-connected-to-6"></a>Layer (type)                     Output Shape          Param #     Connected to</h1>
<p>reshape_1 (Reshape)              (None, 1, 2, 128)     0           reshape_input_1[0][0]</p>
<hr>
<p>zeropadding2d_1 (ZeroPadding2D)  (None, 1, 2, 132)     0           reshape_1[0][0]</p>
<hr>
<p>conv1 (Convolution2D)            (None, 256, 2, 130)   1024        zeropadding2d_1[0][0]</p>
<hr>
<p>dropout_1 (Dropout)              (None, 256, 2, 130)   0           conv1[0][0]</p>
<hr>
<p>zeropadding2d_2 (ZeroPadding2D)  (None, 256, 2, 134)   0           dropout_1[0][0]</p>
<hr>
<p>conv2 (Convolution2D)            (None, 80, 1, 132)    122960      zeropadding2d_2[0][0]</p>
<hr>
<p>dropout_2 (Dropout)              (None, 80, 1, 132)    0           conv2[0][0]</p>
<hr>
<p>flatten_1 (Flatten)              (None, 10560)         0           dropout_2[0][0]</p>
<hr>
<p>dense1 (Dense)                   (None, 256)           2703616     flatten_1[0][0]</p>
<hr>
<p>dropout_3 (Dropout)              (None, 256)           0           dense1[0][0]</p>
<hr>
<p>dense2 (Dense)                   (None, 11)            2827        dropout_3[0][0]</p>
<hr>
<p>activation_1 (Activation)        (None, 11)            0           dense2[0][0]</p>
<hr>
<p>reshape_2 (Reshape)              (None, 11)            0           activation_1[0][0]</p>
<p>Here is my Pytorch code: The input has already been converted to [N,1,2,128] , The number of classes is 11. Think the issue is with the forward pass. Thanks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p><span class="hashtag">#Input</span> [N , 1 , 2, 128]<br>
class ConvNet(nn.Module):<br>
def <strong>init</strong>(self,dr=0.5):<br>
super(ConvNet,self).<strong>init</strong>()<br>
self.pad = nn.ConstantPad2d((2,2,0,0),0)<br>
self.conv1 = nn.Conv2d(1,256,kernel_size=(1,3)) #(Input_channels, Output_channels, Kernel)<br>
self.drop = nn.Dropout(dr)<br>
self.zeropad2 = nn.ConstantPad2d((2,2,0,0),0)<br>
self.conv2 = nn.Conv2d(256,80,kernel_size=(2,3))<br>
self.flatten = nn.Flatten()<br>
self.Dense = nn.Linear(10560,128) # at the point will get flatten, the number of neurons required will be 10560<br>
self.Dense2 = nn.Linear(256,11)<br>
self.softmax = nn.Softmax()</p>
<pre><code>def forward(self, x):
    x = self.pad(x) #pad
    x = F.relu(self.conv1(x)) #conv1
    x = self.drop(x) #dropout
    x = self.zeropad2(x) #zeropadding2
    x = F.relu(self.conv2(x)) #conv2
    x = x.view(x.size(0),-1) #flatten
    x = self.relu.Dense(x) #Linear
    x = x.view(x.shape[0],-1) #Linear
    x = self.relu.Dense2(x)
    x = F.softmax(x) #softmax
    return x</code></pre>

username: ptrblck
cooked: <p>I don’t know which arguments <code>Convolution2D</code> expect, but based on:</p>
<pre><code class="lang-python"># Keras
model.add(Convolution2D(256, 1, 3, border_mode=‘valid’, activation=“relu”, name=“conv1”, init=‘glorot_uniform’))

# PyTorch
self.conv1 = nn.Conv2d(1,256,kernel_size=(1,3)) #(Input_channels, Output_channels, Kernel)
</code></pre>
<p>it seems as if you assume the 2. and 3. arguments in Keras are defining the kernel size?<br>
Could you double check it, as I would expect to see a <code>tuple</code> argument instead of separate inputs.</p>
<p>Also,</p>
<pre><code class="lang-python">self.Dense = nn.Linear(10560,128)
self.Dense2 = nn.Linear(256,11)
</code></pre>
<p>won’t work since <code>Dense</code> returns <code>128</code> features while <code>Dense2</code> expects <code>256</code>.<br>
You wouldn’t need to flatte nthe activation again after the first linear layer as you’ve already flattened it after <code>conv2</code>.<br>
Also, remove the <code>softmax</code> layer in PyTorch as <code>nn.CrossEntropyLoss</code> expects raw logits.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier.</p>