AlexNet With ImageNet Testing

username: Sami_Hassan
cooked: <p>This is probably a human error but i would like to note down accuracy of AlexNet with already trained networks and then replace conv layers with my custom layers and note down results again.</p>
<p>I can find AlexNet and pre_trained weights here <a href="https://pytorch.org/docs/stable/_modules/torchvision/models/alexnet.html" rel="nofollow noopener">[AlexNet]</a></p>
<p>The Datasets are downloaded from here <a href="http://academictorrents.com/collection/imagenet-lsvrc-2015" rel="nofollow noopener">[AT]</a></p>
<p>Main Folder Name : imagenet2012<br>
Sub Folder 1:  ILSVRC2012_img_train Contains different folder (n01443537,n01484850… 15 of them with images inside)<br>
Sub Folder 2: ILSVRC2012_img_val (ILSVRC2012_val_00000001.JPEG etc…Contains all images)</p>
<pre><code class="lang-auto">import torch.nn as nn
from torch.hub import load_state_dict_from_url
import torchvision
from torchvision import transforms
from torch.utils import data
from torchvision.datasets import ImageFolder
import torch
__all__ = ['AlexNet', 'alexnet']


data_transforms = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])


data_dir = "/home/Sami/Documents/imagenet2012/"

model_urls = {
    'alexnet': 'https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth',
}

train_dataset = torchvision.datasets.ImageFolder(root=val_dir, transform=data_transforms, target_transform=None, is_valid_file=None)
test_dataset  = torchvision.datasets.ImageFolder(root=val_dir, transform=data_transforms, is_valid_file=None)

test_loader = torch.utils.data.DataLoader(dataset=test_dataset,  batch_size=100, shuffle=True)
test_ds  = torch.utils.data.Subset(test_loader.dataset, range(0,50))
test_loader_1 = torch.utils.data.DataLoader(dataset=test_ds,  batch_size=100, shuffle=True)




class AlexNet(nn.Module):

    def __init__(self, num_classes=1000):
        super(AlexNet, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(64, 192, kernel_size=5, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(192, 384, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(384, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
        )
        self.avgpool = nn.AdaptiveAvgPool2d((6, 6))
        self.classifier = nn.Sequential(
            nn.Dropout(),
            nn.Linear(256 * 6 * 6, 4096),
            nn.ReLU(inplace=True),
            nn.Dropout(),
            nn.Linear(4096, 4096),
            nn.ReLU(inplace=True),
            nn.Linear(4096, num_classes),
        )

    def forward(self, x):
        x = self.features(x)
        x = self.avgpool(x)
        x = torch.flatten(x, 1)
        x = self.classifier(x)
        return x

model = AlexNet()
state_dict = load_state_dict_from_url(model_urls['alexnet'],  progress=True)
model.load_state_dict(state_dict)
print(model)



with torch.no_grad():
    correct = 0
    total = 0
    
    for images,labels in test_loader_1:
        out = model(images)
        _,predicted = torch.max(out.data,1)
        total += labels.size(0)
        correct += (predicted==labels).sum().item()
    
    
print('Accuracy of the network on the 50 test images: {} %'.format(100 * correct / total))

</code></pre>
<p>But the accuracy stays 0.0 something like that.</p>

username: ptrblck
cooked: <p>Could you please check, if <code>correct</code> and <code>total</code> are stores as float values or as int?<br>
In the latter case, your accuracy might get rounded down to zero.</p>

username: Sami_Hassan
cooked: <p>You are right it was int, after changing it to float i get 2.0 % on 50 images is this alright ?</p>

username: ptrblck
cooked: <p>2% accuracy on 50 images would be a single correct sample.<br>
It depends what you define as “alright”, but I would say it’s a bit low.</p>

username: Sami_Hassan
cooked: <p>I am not sure but i was thinking may be around 56-57% would be alright out of 1000 images <a href="https://www.researchgate.net/figure/Accuracy-for-ImageNet-classification-challenge-with-different-DL-models_fig1_331540139" rel="nofollow noopener">Comparisons or Accuracy</a>.    ILSVRC2012_img_train/ folder has only 15 sub-folders with in.</p>
<p>May be my data loaders have some problems in them ? Directory structure is as follows</p>
<pre><code class="lang-auto">imagenet2012/
     ILSVRC2012_img_train/
                         n01443537/
                                  n01443537_2.JPEG
                                  n01443537_16.JPEG
                                  .
                                  .
                         n01484850/
                                  n01484850_17.JPEG
                                  n01484850_76.JPEG
                                  .
                                  .
                                  .
                                  .
     ILSVRC2012_img_val 
         ILSVRC2012_val_00000001.JPEG
         ILSVRC2012_val_00000002.JPEG
                                  .
                                  .
                                  .</code></pre>

username: ptrblck
cooked: <p>Based on your code it looks like you are using <code>ImageFolder</code> on your validation directory, which seem to contain only the images without any subfolders.<br>
<code>ImageFolder</code> creates the targets based on subfolders, so your current <code>Datasets</code> might contains only a single class label. Could you check that?</p>

username: Sami_Hassan
cooked: <p>Yes you are right. That seems to be the problem.<br>
In reply to the commands</p>
<pre><code class="lang-auto">print(test_dataset.classes)
print(test_dataset.class_to_idx)
</code></pre>
<p>Yields</p>
<blockquote>
<p>[‘ILSVRC2012_img_train’, ‘ILSVRC2012_img_val’]</p>
</blockquote>
<blockquote>
<p>{‘ILSVRC2012_img_train’: 0, ‘ILSVRC2012_img_val’: 1}</p>
</blockquote>
<p>Do i need to have some official dataset ? As i cannot seem to download it when i use this one</p>
<pre><code class="lang-auto">torchvision.datasets.ImageNet(root, split='train', download=True, **kwargs)
</code></pre>

username: ptrblck
cooked: <p>I think the download URL used in <code>datasets.ImageNet</code> was disabled (as described <a href="https://github.com/pytorch/vision/issues/1563#issuecomment-554236943">here</a>), so you would need to register on their site and download the data separately.</p>

username: Sami_Hassan
cooked: <p>Thank you very much. I have already applied to register on their site. I will update this post as soon as i move ahead. In the meantime i will try this dataset <a href="https://www.kaggle.com/c/imagenet-object-localization-challenge/data" rel="nofollow noopener">[kaggle]</a> and let you know. Thanks a ton really</p>
<p>Update:<br>
Above kaggle dataset works fine i have used this script <a href="https://github.com/pytorch/examples/tree/master/imagenet" rel="nofollow noopener">sh</a> to put images in folders for “val” so that pytorch’s dataloader can easily identify labels. For test data folder may be a similar script is needed to put them in folders.</p>
<p>Obtained Accuracy on Val Set 70.1% (1000 images)</p>

username: Ajinkya.Bankar
cooked: <p>Hi,<br>
Can you please share more details on the structure of the “test data folder” and test-dataloader for ImageNet dataset? Thanks.</p>

username: Sami_Hassan
cooked: <p>Hi, are you asking about this one .</p>
<pre><code class="lang-auto">_all__ = ['AlexNet', 'alexnet']


data_transforms = transforms.Compose([
    transforms.Resize(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])


val_dirr = "~/Alexnet/val"

model_urls = {
    'alexnet': 'https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth',
}

#train_dataset = torchvision.datasets.ImageFolder(root=..., transform=data_transforms, target_transform=None, is_valid_file=None)
test_dataset  = torchvision.datasets.ImageFolder(root=val_dirr, transform=data_transforms, is_valid_file=None)

test_loader = torch.utils.data.DataLoader(dataset=test_dataset,  batch_size=100, shuffle=True)

#If making subsets
#test_ds  = torch.utils.data.Subset(test_loader.dataset, range(0,2))
#test_loader_1 = torch.utils.data.DataLoader(dataset=test_ds,  batch_size=1, shuffle=True)
</code></pre>