Imagenet classes

arunkumar: <p>I am trying to use a pretrained resnet model to test on a elephant image. How do we get the class name after getting class id. Also I am not sure I am doing preprocessing correctly. Is this the right approach?</p>
<pre><code>import torch
import torchvision.transforms as transforms
from torch.autograd import Variable
from torchvision.models import resnet50
from PIL import Image

net = resnet50(pretrained=True)
centre_crop = transforms.Compose([
    transforms.Scale(256),
    transforms.CenterCrop(224),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])
img = Image.open('elephant.jpg')
out = net(Variable(centre_crop(img).unsqueeze(0)))
print(out[0].sort()[1][-10:])</code></pre>

moskomule: <p>I found a map of id -&gt; label <a href="https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a" rel="nofollow noopener">https://gist.github.com/yrevar/942d3a0ac09ec9e5eb3a</a> .<br>
So for example</p>
<pre><code class="lang-auto">with open("imagenet1000_clsid_to_human.txt") as f:
    idx2label = eval(f.read())

for idx in out[0].sort()[1][-10:]:
    print(idx2label(idx))
</code></pre>
<p>will work, though <code>eval</code> may be not good way.</p>

tom: <p>After downloading this URL used by <a href="https://github.com/fchollet/keras/blob/master/keras/applications/imagenet_utils.py" rel="nofollow noopener">keras</a>:<br>
<a href="https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json" class="onebox" target="_blank" rel="nofollow noopener">https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json</a></p>
<p>you could use:</p>
<pre><code class="lang-python">import json
class_idx = json.load("imagenet_class_index.json")
idx2label = [class_idx[str(k)][1] for k in range(len(class_idx))]

for idx in out[0].sort()[1][-10:]:
    print(idx2label[idx])
</code></pre>
<p>Best regards</p>
<p>Thomas</p>

Zijun_Wei: <p><a class="mention" href="/u/tom">@tom</a>, thanks!<br>
A tiny update:slight_smile:</p>
<pre><code>import json
class_idx = json.load(open("imagenet_class_index.json"))</code></pre>

sytelus: <p>This works:</p>
<pre><code class="lang-auto">import json
idx2label = []
cls2label = {}
with open("../../data/imagenet_class_index.json", "r") as read_file:
    class_idx = json.load(read_file)
    idx2label = [class_idx[str(k)][1] for k in range(len(class_idx))]
    cls2label = {class_idx[str(k)][0]: class_idx[str(k)][1] for k in range(len(class_idx))}
</code></pre>

quantum-fusion: <p>I want to know where these imagenet_class_index.json files originate from, and how to generate them.  I find it frustrating that the ONNX model zoo does not publish these artifacts, and I find little documentation on how to create them myself from scratch.</p>
<p>I need to know how to generate them, because I wish to model serve, and I see currently 2 options/choices for this purpose. It appears in the first example it states it needs the index_to_name.json file and in the second the model definition file.  However, in the case of the second, it also requires a signature.json file input as well. This is all to convert an ONNX model from the model zoo to .MAR format for the model server.</p>
<ol>
<li>PTH to MAR format using TorchServe</li>
</ol>
<p>Pre-requisites to create a torch model archive (.mar) :</p>
<p>serialized-file (.pt) : This file represents the state_dict in case of eager mode model.</p>
<p>model-file (.py) : This file contains model class extended from torch nn.modules representing the model architecture. This parameter is mandatory for eager mode models. This file must contain only one class definition extended from torch.nn.modules</p>
<p>index_to_name.json : This file contains the mapping of predicted index to class. The default TorchServe handles returns the predicted index and probability. This file can be passed to model archiver using —extra-files parameter.</p>
<p>version : Model’s version.</p>
<p>handler : TorchServe default handler’s name or path to custom inference handler(.py)</p>
<p>PTH to MAR format (TorchServe)<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/serve/blob/master/examples/README.md" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/serve/blob/master/examples/README.md" target="_blank" rel="nofollow noopener">pytorch/serve/blob/master/examples/README.md</a></h4>
<pre><code class="lang-md"># Contents of this Document
* [Creating mar file for an eager mode model](#creating-mar-file-for-eager-mode-model)
* [Creating mar file for torchscript mode model](#creating-mar-file-for-torchscript-mode-model)
* [Serving torchvision image classification models in TorchServe](#examples-torchvision-image-classification-models-in-torchserve)
* [Serving custom model with custom service handler](#example-to-serve-a-custom-model-with-custom-service-handler)
* [Serving text classification model](#example-to-serve-text-classification-model)
* [Serving object detection model](#example-to-serve-object-detection-model)
* [Serving image segmentation model](#example-to-serve-image-segmentation-model)

# TorchServe Examples

The following are examples on how to create and serve model archives with TorchServe.

## Creating mar file for eager mode model

Following are the steps to create a torch-model-archive (.mar) to execute an eager mode torch model in TorchServe :

* Pre-requisites to create a torch model archive (.mar) :
    * serialized-file (.pt) : This file represents the state_dict in case of eager mode model.
    * model-file (.py) : This file contains model class extended from torch nn.modules representing the model architecture. This parameter is mandatory for eager mode models. This file must contain only one class definition extended from torch.nn.modules
</code></pre>

  This file has been truncated. <a href="https://github.com/pytorch/serve/blob/master/examples/README.md" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>2.) ONNX to MAR using Multi-model-server</p>
<p>The downloaded model artifact files are:</p>
<p>Model Definition (json file) - contains the layers and overall structure of the neural network.</p>
<p>Model Params and Weights (params file) - contains the parameters and the weights.</p>
<p>Model Signature (json file) - defines the inputs and outputs that MMS is expecting to hand-off to the API.</p>
<p>assets (text files) - auxiliary files that support model inference such as vocabularies, labels, etc. These vary depending on the model.</p>
<p>ONNX to MAR format (Multi-Model-Server)<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/awslabs/multi-model-server/blob/master/model-archiver/docs/convert_from_onnx.md" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/awslabs/multi-model-server/blob/master/model-archiver/docs/convert_from_onnx.md" target="_blank" rel="nofollow noopener">awslabs/multi-model-server/blob/master/model-archiver/docs/convert_from_onnx.md</a></h4>
<pre><code class="lang-md"># Converting an ONNX Model

## Install model-archiver with ONNX support
ONNX support is optional in `model-archiver` tool. It's not installed by default with `model-archiver`.

To install MMS with ONNX support, you will need to have the [protobuf compiler](https://github.com/onnx/onnx#installation) installed:

for Ubuntu run:

```bash
sudo apt-get install protobuf-compiler libprotoc-dev

pip install model-archiver[onnx]
```

Or for Mac run:

```bash
conda install -c conda-forge protobuf numpy

</code></pre>

  This file has been truncated. <a href="https://github.com/awslabs/multi-model-server/blob/master/model-archiver/docs/convert_from_onnx.md" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


Mona_Jalal: <p>Hi Thomas,</p>
<p>I got this error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [47], in &lt;module&gt;
      1 import json
----&gt; 2 class_idx = json.load("imagenet_class_index.json")
      3 idx2label = [class_idx[str(k)][1] for k in range(len(class_idx))]
      5 for idx in out[0].sort()[1][-10:]:

File /usr/lib/python3.8/json/__init__.py:293, in load(fp, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    274 def load(fp, *, cls=None, object_hook=None, parse_float=None,
    275         parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):
    276     """Deserialize ``fp`` (a ``.read()``-supporting file-like object containing
    277     a JSON document) to a Python object.
    278 
   (...)
    291     kwarg; otherwise ``JSONDecoder`` is used.
    292     """
--&gt; 293     return loads(fp.read(),
    294         cls=cls, object_hook=object_hook,
    295         parse_float=parse_float, parse_int=parse_int,
    296         parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)

AttributeError: 'str' object has no attribute 'read'
</code></pre>
<pre><code class="lang-auto">import json
class_idx = json.load("imagenet_class_index.json")
idx2label = [class_idx[str(k)][1] for k in range(len(class_idx))]

for idx in out[0].sort()[1][-10:]:
    print(idx2label[idx])

</code></pre>
<p>for</p>
<pre><code class="lang-auto">! wget  'https://www.freeiconspng.com/thumbs/cat-png/cat-png-26.png' 
from PIL import Image

cat_img = Image.open('cat-png-26.png')
cat_img_preprocessed = test_transforms(cat_img)
batch_img_cat_tensor = torch.unsqueeze(cat_img_preprocessed, 0)
out = model_ft(batch_img_cat_tensor.cuda())
</code></pre>
<p>I have this when I print out:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/9/29f3297e35886a3cd54bba78aeed2a789625dbb8.png" data-download-href="https://discuss.pytorch.org/uploads/default/29f3297e35886a3cd54bba78aeed2a789625dbb8" title="Screen Shot 2022-03-03 at 1.02.15 AM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/9/29f3297e35886a3cd54bba78aeed2a789625dbb8_2_690x126.png" alt="Screen Shot 2022-03-03 at 1.02.15 AM" data-base62-sha1="5Z6y0ZP2w0VLtNlytPhreFNnjKg" width="690" height="126" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/9/29f3297e35886a3cd54bba78aeed2a789625dbb8_2_690x126.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/9/29f3297e35886a3cd54bba78aeed2a789625dbb8_2_1035x189.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/9/29f3297e35886a3cd54bba78aeed2a789625dbb8.png 2x" data-dominant-color="242424"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2022-03-03 at 1.02.15 AM</span><span class="informations">1066×195 12.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

tom: <p>That seems more generally Python and I’m not the best person for this type of question, but the error seems to be that json.load expects an open file instead of a filename.</p>
<p>Best regards</p>
<p>Thomas</p>

Rubiel: <p>I am surprised Pytorch doesn’t have already a dictionary from outputs to classes implemented. Each image classification model should have such a dictionary.</p>

barkermrl: <p>I came across this same problem! This page in the docs was really helpful: <a href="https://pytorch.org/vision/stable/models.html" class="inline-onebox" rel="noopener nofollow ugc">Models and pre-trained weights — Torchvision main documentation</a></p>
<p>The <code>resnet50(pretrained=True)</code> command is deprecated in the latest version (v0.13 as time of writing) and can be replaced by an imported weights object:</p>
<pre><code class="lang-auto">from torchvision.models import resnet50, ResNet50_Weights

weights = ResNet50_Weights.DEFAULT
net = resnet50(weights=weights)
</code></pre>
<p>The preprocessing transforms and categories are bundled into the weights object:</p>
<pre><code class="lang-auto">centre_crop = weights.transforms()
categories = weights.meta["categories"]
</code></pre>
<p>Hopefully this is useful! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

RickeyPonting: <p>Yes, it is<a href="https://myjdfaccount.top/" rel="noopener nofollow ugc">,</a> thanks for sharing it.</p>

GrantElliot: <p>Does it really work<a href="https://raiderlinkttu.us/" rel="noopener nofollow ugc">,</a> can anybody also help me with this?</p>

andrewashere: <p>further up in the thread <a class="mention" href="/u/tom">@tom</a> gives a link for the S3 bucket used by keras that is still active. I used that to receive the class names with this code snippet. I have omitted the processing logic</p>
<pre><code class="lang-auto">import json

with open('imagenet_class_index.json', 'r') as f:
    idx_to_class = json.load(f)


for i, (prob, idx) in enumerate(zip(top5_probabilities[0], top5_class_indices[0])):
    class_name = idx_to_class[str(idx.item())][1]  # Convert index to string, get class name
    print(f"Rank {i+1}: {class_name} with probability {prob.item():.2f}%")
</code></pre>