Does AlexNet Architecture match the paper definition?

username: PabloRR100
cooked: <p>Hi community,</p>
<p>I have two question regarding the structure of the AlexNet:</p>
<p>I opened a question on <a href="https://stackoverflow.com/questions/52821355/alexnet-paper-does-not-match-dimensions-they-reported" rel="nofollow noopener">SO</a> about it because if you read the specification on the <a href="https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf" rel="nofollow noopener">paper</a> and then the diagram, the dimensions of the volumes on the networks does not match. I assume I am missing something but it just does not work in this publication (which is amazing, don’t get me wrong)</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://i.stack.imgur.com/hoXIq.png" title="hoXIq.png" rel="nofollow noopener"><img src="https://i.stack.imgur.com/hoXIq.png" alt width="690" height="349"><div class="meta">
<svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use xlink:href="#far-image"></use></svg><span class="filename">hoXIq.png</span><span class="informations">1964×994</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use xlink:href="#discourse-expand"></use></svg>
</div></a></div></p>
<p>I took a look at how <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py" rel="nofollow noopener">PyTorch implemented it</a> with the hope to find out what was I missing, and I just see the implementation is totally different from the original paper. For instance, the first layer in the paper is a 11x11 convolution with a stride of 4 and <strong>96 kernels</strong>, whereas the PyTorch implementation has <strong>64</strong>. I think this could be because for a single GPU it may not be just gathering the 96 kernels (48 each of the 2 defined in the paper), but where can I find those specifications then?</p>
<p>Thanks in advance!<br>
Pablo</p>

username: cyberwillis
cooked: <pre><code class="lang-auto">def alexnet(pretrained=False, **kwargs):
    r"""AlexNet model architecture from the
    `"One weird trick..." &lt;https://arxiv.org/abs/1404.5997&gt;`_ paper.
    Args:
        pretrained (bool): If True, returns a model pre-trained on ImageNet
    """
</code></pre>

username: EricWiener
cooked: <p>Answered on SO, but re-posting here (since this is the first result that comes up when I looked up the same question):</p>
<p>I found a <a href="https://github.com/pytorch/vision/pull/463#issuecomment-379214941" rel="noopener nofollow ugc">comment in a GitHub issue</a> that explained this well:</p>
<blockquote>
<p>current implementation comes from <a href="https://github.com/soumith/imagenet-multiGPU.torch/blob/master/models/alexnetowt.lua" rel="noopener nofollow ugc">the torch re-implementation</a>, which is based on the <a href="https://github.com/akrizhevsky/cuda-convnet2/blob/master/layers/layers-imagenet-1gpu.cfg#L81" rel="noopener nofollow ugc">single GPU implementation from Alex</a> which uses 256 filters, while the <a href="https://github.com/akrizhevsky/cuda-convnet2/blob/master/layers/layers-imagenet-2gpu-model.cfg#L163-L189" rel="noopener nofollow ugc">2GPU implementation from Krizhevsky uses 384 filters</a>.</p>
</blockquote>
<p>The original AlexNet model needed to be trained across 2 GTX 580 3GB GPUs because the model was too large to be trained on a single GPU (at the time). Krizhevsky (the author of the paper) released a single GPU implementation with different dimensions and the PyTorch implementation comes from there.</p>