Error deploying torchserve: model load failed

DeepManuPy: <p>Here is the dropbox link to the zip containing log file, model.py, handler.py etc.<br>
<a href="https://www.dropbox.com/s/gi8wmyxmpy1zx84/Files.zip?dl=0" rel="noopener nofollow ugc">https://www.dropbox.com/s/gi8wmyxmpy1zx84/Files.zip?dl=0</a></p>
<p>Model trained:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c9358abd1be5692610052b36e57477f5bddcd69.png" data-download-href="https://discuss.pytorch.org/uploads/default/9c9358abd1be5692610052b36e57477f5bddcd69" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c9358abd1be5692610052b36e57477f5bddcd69.png" alt="image" data-base62-sha1="ml8aL4ojdVY7TcCvtmBONF8RF3j" width="690" height="199" data-dominant-color="262726"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">794×230 8.89 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Commands used:</p>
<pre><code class="lang-auto">torch-model-archiver --model-name foodnet_resnet18 \
                     --version 1.0 \
                     --model-file model/model.py \
                     --serialized-file model/foodnet_resnet18.pth \
                     --handler model/handler.py \
                     --extra-files model/index_to_name.json
</code></pre>
<pre><code class="lang-auto">torchserve --start \
           --ncs \
           --ts-config deployment/config.properties \
           --model-store deployment/model-store \
           --models foodnet=foodnet_resnet18.mar
</code></pre>

ptrblck: <p>What kind of error are you seeing? Could you post the complete stack trace, please?</p>

DeepManuPy: <p>Could you please check out the log file from the dropbox link? The webpage isn’t allowing me to post a complete stack trace.</p>

DeepManuPy: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> ,</p>
<p>I’ve traced the error to model.load_state_dict( ). To give you context,</p>
<pre><code class="lang-auto">import torch.nn as nn
from torchvision.models.resnet import ResNet,BasicBlock

class ImageClassifier(ResNet):
	def __init__(self):
		super(ImageClassifier,self).__init__(BasicBlock,[2,2,2,2],num_classes=2)
		self.fc = nn.Linear(512,2)
</code></pre>
<pre><code class="lang-auto">import torch
model = ImageClassifier()
model.load_state_dict(torch.load("trainedRes18.pth"))
</code></pre>
<p>I’ve trained and saved the “trainedRes18.pth” using:<br>
The code snippet I’ve posted.</p>
<pre><code class="lang-auto">torch.save(model_ft.state_dict(),"trainedRes18.pth")
</code></pre>

ptrblck: <p>I’m not familiar enough with the <code>torchserve</code> backend, and don’t know why the loading is failing as it doesn’t seem to provide enough information in the error message (i.e. is the file missing?).</p>

DeepManuPy: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b0ce1d0e1c8ca9466e6222b064b4d8c35047dcf.png" data-download-href="https://discuss.pytorch.org/uploads/default/8b0ce1d0e1c8ca9466e6222b064b4d8c35047dcf" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/8/b/8b0ce1d0e1c8ca9466e6222b064b4d8c35047dcf.png" alt="image" data-base62-sha1="jQ5Y1znxa0TLBUeJsDu17PJhTwb" width="690" height="279" data-dominant-color="643835"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">908×368 24.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I’ve tried to run the load separately to see why its failing. This is what it shows.</p>

ptrblck: <p>That’s more helpful and points towards a PyTorch version, which is too old to load the file.<br>
I guess you might have stored the <code>state_dict</code> (or any other file) using a new PyTorch release and are trying to load it in <code>torchserve</code> using an older PyTorch version.<br>
Could this be the case and if so, could you update <code>torchserve</code> to the latest release?</p>

DeepManuPy: <p>The torch version on which I am running the torchserve is:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/6/c/6c19ed158f46c50717258216bb22c9a2f4d09f26.png" alt="image" data-base62-sha1="fqj71j5EF8voE5rBeCJA5KD9dhI" width="240" height="60">    cpu only.</p>
<p>And I’ve trained the model using torch 1.8.0+cu101</p>

ptrblck: <p>That would explain the error. Could you update the PyTorch version in your <code>torchserve</code> environment to <code>1.8.0</code> as well?</p>

DeepManuPy: <p>After updating PyTorch, model load is working but only if map_location is used. How do I make torchserve know that it needs to load model on cpu and not on gpu?</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/d/3d9785be44cbfd69497ddd01aa06fdabe39a3c21.png" data-download-href="https://discuss.pytorch.org/uploads/default/3d9785be44cbfd69497ddd01aa06fdabe39a3c21" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/d/3d9785be44cbfd69497ddd01aa06fdabe39a3c21.png" alt="image" data-base62-sha1="8MRLblsl6qaUVQ0spqYWQq6RDgJ" width="690" height="101" data-dominant-color="222223"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">720×106 3.25 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>The loading behavior is not defined by <code>torchserve</code> but by PyTorch depending how you’ve stored the <code>state_dict</code>. You could either push the model to the CPU before storing the <code>state_dict</code> or use the <code>map_location</code> argument as explained <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-model-across-devices">here</a>.</p>