Is it still possible to get layer parameters like kernel_size, pad and stride in grad_fn in torch 1.2?

username: yxchng
cooked: <p>Is it still possible to get layer parameters like <code>kernel_size</code>, <code>pad</code> and <code>stride</code> in <code>grad_fn</code> in torch 1.2?</p>

username: ptrblck
cooked: <p><code>kernel_size</code>, <code>pad</code>, and <code>stride</code> are attributes of a convolution layer, not the <code>grad_fn</code>.<br>
You could directly access them using the layer instance.</p>
<p>Could you explain your use case using <code>grad_fn</code> a bit?</p>

username: yxchng
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>I actually want to do some computations that require me to know the entire chain of layers (and their parameters) leading to each leaf l_i of the network graph. This is possible to do in frameworks like Caffe that require users to specify input layer name and output layer name for each layer.</p>
<p>However, I can’t find an easy way to do this in PyTorch.</p>
<p>The only way I can think of right now is to do this via <code>grad_fn</code> which contains <code>next_functions</code>. Getting the chain then is just a matter of tracing the <code>next_functions</code> recursively.</p>
<p>Do you think there is a better way to do this?</p>
<p>I am actually trying to implement this <a href="https://github.com/BVLC/caffe/blob/master/python/caffe/coord_map.py" rel="noopener nofollow ugc">https://github.com/BVLC/caffe/blob/master/python/caffe/coord_map.py</a>, which is used in FCN for cropping. Currently, what others do to reimplement this kind of cropping is to manually specify the entire chain for each leaf like this (as in <a href="https://github.com/xwjabc/hed/blob/master/networks.py" rel="noopener nofollow ugc">https://github.com/xwjabc/hed/blob/master/networks.py</a>):</p>
<blockquote>
<p>def prepare_aligned_crop(self):<br>
“”" Prepare for aligned crop. “”"<br>
# Re-implement the logic in deploy.prototxt and<br>
#   /hed/src/caffe/layers/crop_layer.cpp of official repo.<br>
# Other reference materials:<br>
#   hed/include/caffe/layer.hpp<br>
#   hed/include/caffe/vision_layers.hpp<br>
#   hed/include/caffe/util/coords.hpp<br>
#   <a href="https://groups.google.com/forum/#!topic/caffe-users/YSRYy7Nd9J8" class="inline-onebox" rel="noopener nofollow ugc">Redirecting to Google Groups</a></p>
<pre><code>    def map_inv(m):
        """ Mapping inverse. """
        a, b = m
        return 1 / a, -b / a

    def map_compose(m1, m2):
        """ Mapping compose. """
        a1, b1 = m1
        a2, b2 = m2
        return a1 * a2, a1 * b2 + b1

    def deconv_map(kernel_h, stride_h, pad_h):
        """ Deconvolution coordinates mapping. """
        return stride_h, (kernel_h - 1) / 2 - pad_h

    def conv_map(kernel_h, stride_h, pad_h):
        """ Convolution coordinates mapping. """
        return map_inv(deconv_map(kernel_h, stride_h, pad_h))

    def pool_map(kernel_h, stride_h, pad_h):
        """ Pooling coordinates mapping. """
        return conv_map(kernel_h, stride_h, pad_h)

    x_map = (1, 0)
    conv1_1_map = map_compose(conv_map(3, 1, 35), x_map)
    conv1_2_map = map_compose(conv_map(3, 1, 1), conv1_1_map)
    pool1_map = map_compose(pool_map(2, 2, 0), conv1_2_map)

    conv2_1_map = map_compose(conv_map(3, 1, 1), pool1_map)
    conv2_2_map = map_compose(conv_map(3, 1, 1), conv2_1_map)
    pool2_map = map_compose(pool_map(2, 2, 0), conv2_2_map)

    conv3_1_map = map_compose(conv_map(3, 1, 1), pool2_map)
    conv3_2_map = map_compose(conv_map(3, 1, 1), conv3_1_map)
    conv3_3_map = map_compose(conv_map(3, 1, 1), conv3_2_map)
    pool3_map = map_compose(pool_map(2, 2, 0), conv3_3_map)

    conv4_1_map = map_compose(conv_map(3, 1, 1), pool3_map)
    conv4_2_map = map_compose(conv_map(3, 1, 1), conv4_1_map)
    conv4_3_map = map_compose(conv_map(3, 1, 1), conv4_2_map)
    pool4_map = map_compose(pool_map(2, 2, 0), conv4_3_map)

    conv5_1_map = map_compose(conv_map(3, 1, 1), pool4_map)
    conv5_2_map = map_compose(conv_map(3, 1, 1), conv5_1_map)
    conv5_3_map = map_compose(conv_map(3, 1, 1), conv5_2_map)

    score_dsn1_map = conv1_2_map
    score_dsn2_map = conv2_2_map
    score_dsn3_map = conv3_3_map
    score_dsn4_map = conv4_3_map
    score_dsn5_map = conv5_3_map

    upsample2_map = map_compose(deconv_map(4, 2, 0), score_dsn2_map)
    upsample3_map = map_compose(deconv_map(8, 4, 0), score_dsn3_map)
    upsample4_map = map_compose(deconv_map(16, 8, 0), score_dsn4_map)
    upsample5_map = map_compose(deconv_map(32, 16, 0), score_dsn5_map)

    crop1_margin = int(score_dsn1_map[1])
    crop2_margin = int(upsample2_map[1])
    crop3_margin = int(upsample3_map[1])
    crop4_margin = int(upsample4_map[1])
    crop5_margin = int(upsample5_map[1])

    return crop1_margin, crop2_margin, crop3_margin, crop4_margin, crop5_margin
</code></pre>
</blockquote>
<p>This is problematic because it does not allow easy switching of backbones. I hope to be able to compute this kind of crop offsets automatically.</p>