Converting Keras code to pytorch

username: Rituraj_19_Dutta_100
cooked: <pre><code class="lang-auto">from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import keras
import keras.layers as layers 
from keras.models import Model
from keras.layers.core import Lambda
import encoder_models as EM
import cv2
import numpy as np


    
def GlobalAveragePooling2D_r(f):
    def func(x):
    
        repc =  int(x.shape[4])
        m    =  keras.backend.repeat_elements(f, repc, axis = 4)
        x    =  layers.multiply([x, m])
        repx =  int(x.shape[2])
        repy =  int(x.shape[3])
        x    = (keras.backend.sum(x, axis=[2, 3], keepdims=True) / (keras.backend.sum(m, axis=[2, 3], keepdims=True)))
        x    =  keras.backend.repeat_elements(x, repx, axis = 2)
        x    =  keras.backend.repeat_elements(x, repy, axis = 3)    
        return x
    return Lambda(func)

def Rep_mask(f):
    def func(x):
        x    =  keras.backend.repeat_elements(x, f, axis = 1)   
        return x
    return Lambda(func)
    
def common_representation(x1, x2): 
    repc =  int(x1.shape[1])
    x2   =  keras.layers.Reshape(target_shape=(1, np.int32(x2.shape[1]), np.int32(x2.shape[2]), np.int32(x2.shape[3]))) (x2) 
    x2   =  Rep_mask(repc)(x2)
    x    =  layers.concatenate([x1, x2], axis=4) 
    x    =  layers.TimeDistributed(layers.Conv2D(128, 3, padding = 'same', kernel_initializer = 'he_normal'))(x)
    x    =  layers.TimeDistributed(layers.BatchNormalization(axis=3))(x) 
    x    =  layers.TimeDistributed(layers.Activation('relu'))(x) 
    return x
</code></pre>
<p>I’m new to Pytorch and I’m having some problems converting this code written in Keras to PyTorch. I have converted half of the code but I’m stuck in this part. Any help would be greatly appreciated.</p>

username: eqy
cooked: <p>Are you referring to the <code>TimeDistributed</code> layer? afaik there have been a few previous threads on how to implement this e.g., <a href="https://discuss.pytorch.org/t/timedistributed-cnn/51707/2" class="inline-onebox">Timedistributed CNN - #2 by ilyes</a></p>

username: Rituraj_19_Dutta_100
cooked: <p>I got the intuition for the Time Distributed layer but I’m actually stuck in layers.multiply(), keras.backend.sum and all</p>

username: eqy
cooked: <p>Is <code>layers.multiply()</code> an elementwise mul? that corresponds to <code>torch.mul</code>. Similarly if <code>keras.backend.sum</code> corresponds to a reduction across some axes that corresponds to <code>torch.sum</code>.</p>

username: Rituraj_19_Dutta_100
cooked: <p>And<code> keras.backend.repeat_elements</code> corresponds to t<code>orch.repeat()</code> or <code>torch.repeat_interleave()</code> ?</p>

username: eqy
cooked: <p>***I believe it should correspond to <code>torch.repeat_interleave</code>, as the docs say it is like numpy’s repeat.</p>

username: Rituraj_19_Dutta_100
cooked: <p>I have tried <code>torch.repeat_interleave()</code> for a rank 4 tensor like <code>[batch,channel,height,width]</code>  along axis=1 (channel) but the ouput tensor is 1D.</p>

username: eqy
cooked: <p>I think the default behavior is to use the flattened input unless a dimension is passed.<br>
Can you share a code snippet which demonstrates your issue?</p>

username: Rituraj_19_Dutta_100
cooked: <p>Okay this worked similar to <code>keras.backend.repeat_elements()</code></p>
<pre><code class="lang-auto">import torch

A = torch.randn([5,3,224,224])

print(A.shape)

B = torch.repeat_interleave(A,4,dim=1)

print(B.shape)  ## (5,12,224,224)
</code></pre>
<p>Probably I was missing something<br>
Thanks !!</p>

username: Rituraj_19_Dutta_100
cooked: <p>Okay one last Question how to write the below function in Pytorch’s way</p>
<pre><code class="lang-auto">def GlobalAveragePooling2D_r(f):
    def func(x):
    
        repc =  int(x.shape[4])
        m    =  keras.backend.repeat_elements(f, repc, axis = 4)
        x    =  layers.multiply([x, m])
        repx =  int(x.shape[2])
        repy =  int(x.shape[3])
        x    = (keras.backend.sum(x, axis=[2, 3], keepdims=True) / (keras.backend.sum(m, axis=[2, 3], keepdims=True)))
        x    =  keras.backend.repeat_elements(x, repx, axis = 2)
        x    =  keras.backend.repeat_elements(x, repy, axis = 3)    
        return x
    return Lambda(func)
</code></pre>

username: eqy
cooked: <p>If you mean the organization of the operator invocations into <code>Module</code>s, I think a good reference is some of the implementations in torchvision e.g., the resnet implementation here:<br>
<a href="https://pytorch.org/vision/stable/_modules/torchvision/models/resnet.html#resnet18" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/vision/stable/_modules/torchvision/models/resnet.html#resnet18</a></p>