I am getting the following error while using pickle

Nilesh_Pandey1: <hr>
<p>AttributeError                            Traceback (most recent call last)<br>
 in ()<br>
1<br>
2 with open(fname, ‘rb’) as fid:<br>
----&gt; 3     model = pickle.load(fid)</p>
<p>AttributeError: Can’t get attribute ‘_load_from_bytes’ on &lt;module ‘torch.storage’ from ‘/home/np9207/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/storage.py’&gt;</p>
<p><strong>The Pickel file is pre-trained model as .pkl</strong></p>

vmirly1: <p>Can you try loading the model using the pytorch function <code>torch.load()</code>? Here is an example:</p>
<pre><code class="lang-python">model = MyAutoEncoderClass()
model.load_state_dict(torch.load('./model.pkl'))
</code></pre>
<p>More examples can also be found at <a href="https://pytorch.org/tutorials/beginner/saving_loading_models.html" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/saving_loading_models.html</a></p>

nile649: <p>That win’t work for him, it is the pickle error. Pickle for encoding used is different from the one used in pytorch.</p>

nile649: <p><strong>I tried, I am getting error with pytorch 0.4.1 and 1.00</strong></p>
<p>**---------------------------------------------------------------------------<br>
ModuleNotFoundError                       Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 torch.load(’./Downloads/Human3.6M-17J-ResNet50.pkl’)</p>
<p>~/anaconda3/envs/torch/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module)<br>
356         f = open(f, ‘rb’)<br>
357     try:<br>
–&gt; 358         return _load(f, map_location, pickle_module)<br>
359     finally:<br>
360         if new_fd:</p>
<p>~/anaconda3/envs/torch/lib/python3.6/site-packages/torch/serialization.py in _load(f, map_location, pickle_module)<br>
530             f.seek(0)<br>
531<br>
–&gt; 532     magic_number = pickle_module.load(f)<br>
533     if magic_number != MAGIC_NUMBER:<br>
534         raise RuntimeError(“Invalid magic number; corrupt file?”)</p>
<p>ModuleNotFoundError: No module named ‘utils’</p>
<p>**</p>
<p><strong>With pytorch 0.4.0, the error is</strong><br>
**AttributeError: Can’t get attribute ‘_load_from_bytes’ on &lt;module ‘torch.storage’ from ‘/home/user/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/storage.py’&gt;<br>
**</p>

nile649: <p>link to the data if anyone wants to help <a href="http://pascal.inrialpes.fr/data2/grogez/LCR-Net/pthmodels/Human3.6M-17J-ResNet50.pkl" rel="nofollow noopener">http://pascal.inrialpes.fr/data2/grogez/LCR-Net/pthmodels/Human3.6M-17J-ResNet50.pkl</a></p>

rasbt: <blockquote>
<p>link to the data if anyone wants to help <a href="http://pascal.inrialpes.fr/data2/grogez/LCR-Net/pthmodels/Human3.6M-17J-ResNet50.pkl" rel="noopener nofollow ugc">http://pascal.inrialpes.fr/data2/grogez/LCR-Net/pthmodels/Human3.6M-17J-ResNet50.pkl</a></p>
</blockquote>
<p>I don’t recommend anyone to load pickle files from the internet – it’s a huge security risk. Also, I don’t think anyone would be able to load the file if the person doesn’t have your model code … which leads me to the point: Make sure you also defined all the necessary classes and functions in the script where you are loading the file. In future, I recommend using <code>state_dict</code> approach, which is a bit more robust regarding dependencies and file structures.</p>

nile649: <p>Any idea or information on how can I extract the dict data from the pkl file with or without python? I am using the out of box code from the team behind LCR NET++.</p>

rasbt: <p>I don’t think this is easily possible because pickle is a binary format, so you have to load the file as a whole as far as I know. The easiest way to accomplish that is to use the exam same functions and classes you defined in the script that you used to generate the pickle file and then try to load it. Once you have loaded the model, you can then access the dict via model.state_dict().</p>
<p>(Also, like vahid suggested, are you sure that this is a regular pickle file and not one created via pytorch?)</p>
<p>Also note that you need to use the exact same PyTorch version etc.</p>

nile649: <p>This definitely created by pytorch, because of the errors. The pytorch version used for creation of the file is 0.4.1, but I am not able to figure out utils module needed for it. Thanks, I will try contacting the authors.</p>

rasbt: <p>Yes, restoring pkl files on some other machine is usually tricky. If the authors provide a model.pkl file, it could technically be at least any of the three</p>
<p>a) a model pickled via regular pickle use<br>
b) a model saved via pickle using torch.save<br>
c) a model state_dict saved via pickle using torch.save</p>
<p>So, it’s maybe best to ask the authors for recommendation for loading it. Also, you may look through their other source code and see if you can find a file that created this pickle file, which could give you additional clues.</p>

ajaymaity: <p>Hey, did you find the solution for this? I am running the LCRNet code too, and facing this exact same issue!</p>