Architecture inconsistency in Zoo models in different frameworks

nitin2212: <p>Hi, I am comparing few zoo models implementation in DL4J with Pytorch zoo models and found<br>
that the padding in Convolution layers does not match most of time ?</p>
<p>For  <strong>Resnet50 and SqueezeNet</strong>  :</p>
<p>In DL4J, they  <strong>do not</strong>  apply padding : [0, 0] ; while in PyTorch they have padding [1, 1].<br>
This results in different output in layers</p>
<p>In DL4J, they apply  <strong>Bias</strong>  in Conv layers while in PyTorch, they do not.</p>
<p>Why such irregularities in Network structure across frameworks ?</p>

ptrblck: <p>I guess PyTorch might have used the <a href="https://github.com/KaimingHe/deep-residual-networks/blob/master/prototxt/ResNet-50-deploy.prototxt">original Caffe implementation</a> as the base?<br>
As you can see in the <code>prototxt</code>, the conv layers do not use the <code>bias</code>, as it would be cancelled by the following batchnorm layers.</p>

nitin2212: <p>Alright ! But about the inconsistency in padding ?</p>

ptrblck: <p>I don’t know which reference DL4J used. Did you compare the Caffe model with the torchvision implementation and did you find any mismatches?</p>

nitin2212: <p>See :<br>
<strong>DL4J Resnet50</strong>: <a href="https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-zoo/src/main/java/org/deeplearning4j/zoo/model/ResNet50.java" class="inline-onebox" rel="noopener nofollow ugc">deeplearning4j/deeplearning4j/deeplearning4j-zoo/src/main/java/org/deeplearning4j/zoo/model/ResNet50.java at master · deeplearning4j/deeplearning4j · GitHub</a></p>
<p><strong>Torch Resnet50</strong> :<br>
<a href="https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py" class="inline-onebox" rel="noopener nofollow ugc">vision/torchvision/models/resnet.py at main · pytorch/vision · GitHub</a></p>
<p>I got the layers information and printed them on console. As you can see in the image <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/9/a92dfb4e7a9fa32c24116f68e1f26e4c3c567edc.png" data-download-href="https://discuss.pytorch.org/uploads/default/a92dfb4e7a9fa32c24116f68e1f26e4c3c567edc" title="Zoo_Models_Resnet50"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/9/a92dfb4e7a9fa32c24116f68e1f26e4c3c567edc.png" alt="Zoo_Models_Resnet50" data-base62-sha1="o8DcSFL4kct0U3jFN7n8jjater2" width="510" height="500" data-dominant-color="EBEBEC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Zoo_Models_Resnet50</span><span class="informations">702×688 14.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> -  the stride mismatch in second convolution operation (<strong>Torch uses 1</strong> while **dl4j uses 2! ) . This results in decreasing the output volume rapidly in dl4j.</p>

ptrblck: <p>As I said, I’m not familiar with the DL4J implementation, so you would need to ask the authors in their discussion board of on GitHub etc.</p>
<p>If you think that the <code>torchvision</code> model doesn’t reflect the expected implementation from the paper, please let us know and we’ll look into it.</p>

nitin2212: <p>Okay. Thanks !! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>