How to Classify Single Image using Loaded Net

nikmentenson: <p>Hi,</p>
<p>I trained, saved, and can load a resnet50 net, but am not sure of how to feed in a single image for classification using the loaded net.</p>
<p>I’ve tried using the dataloader function, but I think that is more appropriate for testing a group of images that already have labels, whereas a single image that you are trying to classify obviously would not have a label.</p>
<p>If you could point me to the right function to use or to a general methodology to follow, I would greatly appreciate it.</p>
<p>Sorry if this is a bit basic of a question, but for some reason I could not find much online to guide me on this.</p>
<p>Thanks</p>

tymokvo: <p>I use something like this. I think it’s a mashup of code from various tutorials/examples so my apologies if I should be crediting someone…</p>
<p>uses PIL and torchvision.transforms</p>
<pre><code>imsize = 256
loader = transforms.Compose([transforms.Scale(imsize), transforms.ToTensor()])

def image_loader(image_name):
    """load image, returns cuda tensor"""
    image = Image.open(image_name)
    image = loader(image).float()
    image = Variable(image, requires_grad=True)
    image = image.unsqueeze(0)  #this is for VGG, may not be needed for ResNet
    return image.cuda()  #assumes that you're using GPU

image = image_loader(PATH TO IMAGE)

your_trained_net(image)
</code></pre>
<p>hope that helps</p>

nikmentenson: <p>Awesome, thank you so much. That worked.</p>

nikmentenson: <p>Hi again,</p>
<p>One more quick question.<br>
So I am able to classify the image using the code you listed above, but my output is the following:</p>
<pre><code>Variable containing:
0
[torch.LongTensor of size 1]
</code></pre>
<p>How do I extract the value of this variable and convert it to, say, an <strong>int</strong>?<br>
I want to use that <strong>int</strong> to print out the string description of the category it belongs to.<br>
As I understand it, the value in this variable is the index of the class (folder) that it predicts the image belongs to?</p>
<p>Many thanks</p>

fmassa: <p>You can extract the tensor present in a Variable by accessing the <code>.data</code> field.<br>
Then, you can take the value corresponding to it by simply indexing it. Here is an example</p>
<pre><code class="lang-python">variable = Variable(torch.rand(1))
num = variable.data[0]
</code></pre>

nikmentenson: <p>Great, but once I have extracted the LongTensor, how do I convert it to an <strong>int</strong>?</p>
<p>Thanks</p>

tymokvo: <p>standard casting</p>
<pre><code>int(variable.data[0])</code></pre>

nikmentenson: <p>ERROR PyTorch 0.1.11</p>

xwgeng: <p>I think the predict is 2-d tensor, so variable.data[0] is <code>LongTensor</code>. You should  use <code>predict.data[0][0]</code> if <code>batch_size</code> is 1</p>

nikmentenson: <p>One last thing-wanted to confirm this is in fact correct:</p>
<p>As I understand it, the value of the variable mentioned above is the index of the class (folder) that it predicts the image belongs to?<br>
So, that <strong>int</strong> is the index of the class name (where the names of the classes are in alphabetical order)?</p>

tymokvo: <p>yes, provided the second argument to torch.max is looking at the correct dimension of the output Variable. Which yours should be, if you’re using a pretrained resnet.</p>
<p><code>int(predicted.data[0])</code></p>
<p>will be casting the index of the maximum value in the tensor contained in the Variable that is output from the network to an integer. Though, in this case, you don’t need the cast to int. Indexing to a specific location in a LongTensor will return an int.<br>
Sorry, I know I suggested to use that before. Didn’t understand what you were asking <img src="//discuss.pytorch.org/images/emoji/apple/no_mouth.png?v=5" title=":no_mouth:" class="emoji" alt=":no_mouth:"></p>

FuriouslyCurious: <p>I keep getting an error “Expected 3D tensor” when trying to classify image with a model saved by <a class="mention" href="/u/smth">@smth</a> ImageNet example.  What am I doing wrong?  Model was trained on GPU with CUDA and saved in the pth.tar format.</p>
<pre><code># Bunch of imports go here

# Convert image to Variable
def Torchify( aImage ):
    ptLoader = transforms.Compose([transforms.ToTensor()])
    aImage = ptLoader( aImage ).float()
    aImage = Variable( aImage, volatile=True  )
    return aImage.cuda()

# Load model from Checkpoint
print("=&gt; Loading Network")
ptModelAxial = densenet.__dict__['densenet161'](pretrained=False, num_classes=5)
ptModelAxial.classifier = nn.Linear(8832, 5)
ptModelAxial = torch.nn.DataParallel(ptModelAxial).cuda()
dTemp = torch.load("best.pth.tar")
ptModelAxial.load_state_dict(dTemp['state_dict'])
for p in ptModelAxial.parameters():
    p.requires_grad = False
ptModelAxial.eval()

InputImg = skimage.img_as_float(skimage.io.imread(sFileName))
ptModelPreds = ptModelAxial( Torchify(InputImg) )
print( ptModelPreds )
</code></pre>
<p>Error:</p>
<pre><code>Traceback (most recent call last):
  File "extract.py", line 298
      ptModelPreds = ptModelAxial( Torchify(InputImg) )
File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/parallel/data_parallel.py", line 61, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/parallel/data_parallel.py", line 71, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/parallel/parallel_apply.py", line 45, in parallel_apply
    raise output
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/parallel/parallel_apply.py", line 25, in _worker
    output = module(*input, **kwargs)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/keyur/kaggle/densenet.py", line 153, in forward
    features = self.features(x)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/modules/container.py", line 64, in forward
    input = module(input)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/modules/conv.py", line 237, in forward
    self.padding, self.dilation, self.groups)
  File "/conda3/envs/idp/lib/python3.5/site-packages/torch/nn/functional.py", line 39, in conv2d
    return f(input, weight, bias)
RuntimeError: expected 3D tensor</code></pre>

smth: <p>send a tensor in of <code>1 x C x H x W</code>. Even for a single image, you have to send in the mini-batch dimension.</p>

vipin14119: <p>thanks for the answer it worked , but yeah <code>image.unsqueeze(0)</code> is still needed in ResNet otherwise you might get this error  <code>RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0</code></p>

isalirezag: <p>So I have a couple of questions here:</p>
<ol>
<li>I am not sure why when I use the line  <code>image = loader(image).float()</code> I receive an error. Do you have any idea? but if i dont use it, it works.</li>
</ol>
<p>Here is the error that I get:<br>
<code>RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same</code></p>
<ol start="2">
<li>when I do the classification, e.g. <code>resnet = models.resnet18(pretrained = True);</code><br>
<code>predicted = resnet(inputVar)</code> and plot the predicted values, the min and max values are -2.3611, and 2.4880, respectively.<br>
However, shouldnot the min and max values be between 0 and 1?</li>
</ol>

InnovArul: <ol>
<li>
<p>one of them (model or input) is in cpu and the other is in GPU.<br>
Upload them to GPU using <code>model = model.cuda(), image = image.cuda()</code> commands.</p>
</li>
<li>
<p>The output that we get from pretrained models are unnormalized linear layer outputs.<br>
you can use <code>F.softmax()</code> to convert them to probabilities.</p>
</li>
</ol>

micklexqg: <p>but why still get error, the code is as follow:<br>
loader = transforms.Compose([ transforms.Scale(311),<br>
transforms.CenterCrop(299), transforms.ToTensor()])<br>
def image_loader(image_name):<br>
“”“load image, returns cuda tensor”""<br>
image = Image.open(image_name)<br>
image = loader(image).float()<br>
image = Variable(image, requires_grad=True)<br>
image = image.unsqueeze(0)  <span class="hashtag">#this</span> is for VGG, may not be needed for ResNet<br>
return image<br>
# return image.cuda()  <span class="hashtag">#assumes</span> that you’re using GPU<br>
image = image_loader(‘test.jpg’)<br>
print(image.shape)<br>
model = models.inception_v3(pretrained=True)<br>
model(image)</p>
<p>the image shape is torch size [1,3,299,299]<br>
but get error:<br>
Expected more than 1 value per channel when training, got input size [1, 768, 1, 1]</p>

ptrblck: <p>Could you try to set <code>model.eval()</code> and run it again?<br>
Probably your model is still in training and thus probably a <code>BatchNorm</code> layer needs more than one pixel / sample to calculate the running stats.</p>

micklexqg: <p>sorry for so late reply, thanks!</p>

alejodosr: <p>Found a weird behavior here (bug), any advice?</p>
<p>Common code:</p>
<pre><code class="lang-auto">normalize = transforms.Normalize(
   mean=[0.485, 0.456, 0.406],
   std=[0.229, 0.224, 0.225]
)
preprocess = transforms.Compose([
   transforms.Resize(256),
   transforms.CenterCrop(224),
   transforms.ToTensor(),
   normalize
   # transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])

resnet_model = torchvision.models.resnet50(pretrained=True, num_classes=1000)
resnet_model.eval()

for i in range(4):
    # An instance of your model.
    img_pil = Image.open("/home/alejandro/workspace/uav_detection/images/" + str(i + 1) + ".jpg")
    # img_pil.show()
    img_tensor = preprocess(img_pil).float()
    img_tensor = img_tensor.unsqueeze_(0)

    fc_out = resnet_model(Variable(img_tensor))

    output = fc_out.detach().numpy()
    print(output.argmax())
</code></pre>
<p>Results with:<br>
Python 2.7 Anaconda and pytorch 1.0.0a0+37627a1 (built from source):</p>
<pre><code class="lang-auto">918
918
918
918
</code></pre>
<p>Python 2.7 Anaconda and pytorch 0.4.1.post2 (downloaded with pip):</p>
<pre><code class="lang-auto">834
208
285
478
</code></pre>