Guidance on Training Model with PyTorch's ImageNet Dataset

HassanBinHaroon: <p>Hello PyTorch community,</p>
<p>I’m seeking guidance on utilizing PyTorch’s <code>torchvision.datasets.ImageNet</code> class for training my model. Specifically, I’m interested in understanding how to effectively leverage the functionalities provided by this class for training purposes. My goal is to train a CNN model on the ImageNet dataset.</p>
<p>Your insights and guidance would be highly appreciated. Thank you for your assistance!</p>
<p>Best,<br>
Hassan Bin Haroon</p>

HassanBinHaroon: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> <a class="mention" href="/u/alband">@albanD</a> Please provide your insights. Thanks!</p>

ptrblck: <p>I don’t understand the question, and which functionalities you are asking about. The dataset provides input arguments as:</p>
<ul>
<li><strong>root</strong> (<em>string</em>) – Root directory of the ImageNet Dataset.</li>
<li><strong>split</strong> (<em>string</em> <em>,</em> <em>optional</em>) – The dataset split, supports <code>train</code>, or <code>val</code>.</li>
<li><strong>transform</strong> (<em>callable</em> <em>,</em> <em>optional</em>) – A function/transform that takes in an PIL image and returns a transformed version. E.g, <code>transforms.RandomCrop</code></li>
<li><strong>target_transform</strong> (<em>callable</em> <em>,</em> <em>optional</em>) – A function/transform that takes in the target and transforms it.</li>
<li><strong>loader</strong> – A function to load an image given its path.</li>
</ul>
<p>which are used toe properly load and transform the samples.</p>

HassanBinHaroon: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Let me specify the functionality.</p>
<ol>
<li>How to use torchvision.datasets.ImageNet to <strong>access the images and corresponding labels</strong> for PyTorch network training loop. I have the ILSVRC 2012 dataset downloaded. But I want a simple example resource that exhibits the correct utilization of <em>torchvision.datasets.ImageNet</em> effectively.</li>
</ol>
<p>In other words, how to leverage PyTorch for training the model on <em>ILSVRC 2012</em>. Given that I have the dataset ILSVRC 2012 downloaded. But replying just on PyTorch code for parsing and other pre-processing.</p>
<p>As PyTorch provides some of the ImageNet parsing code, my intention is not to create a custom dataset class from scratch. Instead, I aim to utilize PyTorch for the same purpose.</p>
<p>Let’s elaborate the question further,<br>
“<strong>imagenet_data = torchvision.datasets.ImageNet(‘path/to/imagenet_root/’)</strong><br>
<strong>data_loader = torch.utils.data.DataLoader(imagenet_data, batch_size=4, shuffle=True, num_workers=args.nThreads)</strong>”</p>
<p>How can we get the images and labels from <strong>data_loader</strong>? Is it the same as in the other dataset cases?</p>
<p>I hope this clarifies my request for the help.</p>

J_Johnson: <p>This tutorial shows an example of training a model using the dataset class with CIFAR10. Shouldn’t be too different for any other dataset.</p>
<p><a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html</a></p>

HassanBinHaroon: <p><a class="mention" href="/u/j_johnson">@J_Johnson</a> Thanks for the CIFAR10 example. You’re right; it shouldn’t be too different. Since the ImageNet dataset isn’t publicly available, users need to download it before instantiating the dataset class. This differs from datasets like CIFAR10, which can be downloaded and preprocessed entirely by PyTorch.</p>

J_Johnson: <p>For any additional details, please check this PyTorch example here using ImageNet:</p>
<aside class="onebox githubfolder" data-onebox-src="https://github.com/pytorch/examples/tree/main/imagenet">
  <header class="source">

      <a href="https://github.com/pytorch/examples/tree/main/imagenet" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h3><a href="https://github.com/pytorch/examples/tree/main/imagenet" target="_blank" rel="noopener nofollow ugc"></a></h3>

  <p><a href="https://github.com/pytorch/examples/tree/main/imagenet" target="_blank" rel="noopener nofollow ugc">//github.com/pytorch/examples/tree/main/imagenet</a></p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


HassanBinHaroon: <p>Considerable Resource. I am looking at this implementation. Thanks <a class="mention" href="/u/j_johnson">@J_Johnson</a>!</p>