AlexNet for cifar from " Understanding deep learning requires rethinking generalization"

username: Brando_Miranda
cooked: <p>I wanted to train an AlexNet model on cifar with the architecture from:</p>
<p><a href="https://arxiv.org/abs/1611.03530" rel="nofollow noopener">“Understanding deep learning requires rethinking generalization”</a></p>
<p>Is the following the recommended way to do it:</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/bearpaw/pytorch-classification/blob/master/models/cifar/alexnet.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/bearpaw/pytorch-classification/blob/master/models/cifar/alexnet.py" target="_blank" rel="nofollow noopener">bearpaw/pytorch-classification/blob/master/models/cifar/alexnet.py</a></h4>
<pre><code class="lang-py">'''AlexNet for CIFAR10. FC layers are removed. Paddings are adjusted.
Without BN, the start learning rate should be 0.01
(c) YANG, Wei 
'''
import torch.nn as nn


__all__ = ['alexnet']


class AlexNet(nn.Module):

    def __init__(self, num_classes=10):
        super(AlexNet, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=5),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=2, stride=2),
            nn.Conv2d(64, 192, kernel_size=5, padding=2),
            nn.ReLU(inplace=True),
</code></pre>

  This file has been truncated. <a href="https://github.com/bearpaw/pytorch-classification/blob/master/models/cifar/alexnet.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>or is there a standard way to do this in pytorch for <strong>cifar</strong>?</p>

username: Brando_Miranda
cooked: <p>(note I don’t need pre-trained models)</p>

username: aplassard
cooked: <p>That looks like a reasonable way to initialize it, though your model is not exactly the same as AlexNet both in the layer definitions and the fully connected layers.</p>
<p>There is also torchvision which has a pre-setup version of the model - <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py" rel="nofollow noopener">https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py</a></p>

username: Brando_Miranda
cooked: <p>is the alexnet on that paper the alex net on Cifar 10 or Imagenet? (I want one for Cifar 10, I know the original one was on imagenet)</p>

username: Brando_Miranda
cooked: <p>Crossposted:</p>
<p><a href="https://qr.ae/TWnjL4" class="onebox" target="_blank" rel="nofollow noopener">https://qr.ae/TWnjL4</a></p>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/57192391/how-to-build-alexnet-for-cifar10-from-understanding-deep-learning-requires-reth" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/1601580/charlie-parker" target="_blank" rel="nofollow noopener">
    <img alt="Charlie Parker" src="https://i.stack.imgur.com/2B6rV.png?s=128&amp;g=1" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/57192391/how-to-build-alexnet-for-cifar10-from-understanding-deep-learning-requires-reth" target="_blank" rel="nofollow noopener">How to build AlexNet for Cifar10 from "Understanding deep learning requires rethinking generalization” for Pytorch?</a>
</h4>

<div class="tags">
  <strong>python, machine-learning, deep-learning, conv-neural-network</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/1601580/charlie-parker" target="_blank" rel="nofollow noopener">
    Charlie Parker
  </a>
  on <a href="https://stackoverflow.com/questions/57192391/how-to-build-alexnet-for-cifar10-from-understanding-deep-learning-requires-reth" target="_blank" rel="nofollow noopener">11:15PM - 24 Jul 19 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>