ModuleNotFoundError: No module named '_lzma'

Xanthan: <p>I’m using python 3.7.10 with torch 1.81 and torchvision 0.9.1 in my machine and while importing dset from torchvision.dataset this error shows up:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "train.py", line 23, in &lt;module&gt;
    from dataloader import load_data
  File "/home/rdb/Prithviraj/Image-Restoration-in-Occluded-Images-using-GANs/dataloader.py", line 3, in &lt;module&gt;
    import torchvision.datasets as dset
  File "/home/rdb/.local/lib/python3.7/site-packages/torchvision/__init__.py", line 7, in &lt;module&gt;
    from torchvision import datasets
  File "/home/rdb/.local/lib/python3.7/site-packages/torchvision/datasets/__init__.py", line 6, in &lt;module&gt;
    from .mnist import MNIST, EMNIST, FashionMNIST, KMNIST, QMNIST
  File "/home/rdb/.local/lib/python3.7/site-packages/torchvision/datasets/mnist.py", line 11, in &lt;module&gt;
    import lzma
  File "/usr/local/lib/python3.7/lzma.py", line 27, in &lt;module&gt;
    from _lzma import *
ModuleNotFoundError: No module named '_lzma'
</code></pre>
<p>This means that the torchvision I have is broken or buggy. Can someone suggest a fix for this error.<br>
If u need a demo code to run then please use <a href="https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html" rel="noopener nofollow ugc">this</a> tutorial .</p>
<p>Any help would be great!!</p>

ptrblck: <p>It seems that the usage of <code>lzma</code> was removed in the <a href="https://github.com/pytorch/vision/blob/58afa5110ce65c718dea676eccc5783d7e56d07a/torchvision/datasets/mnist.py#L9">master branch</a>, but for <code>0.9.1</code> you could try to install it via <code>pip</code> or <code>conda</code>.</p>

Xanthan: <p>is it this command?<br>
<code>pip install backports.lzma</code></p>

ptrblck: <p>This could work. However, note that <code>lzma</code> should be a default Python package starting from <a href="https://docs.python.org/3.5/library/lzma.html">Python 3.5</a>, so unsure why it’s missing in your environment.<br>
A related issue is <a href="https://github.com/pytorch/vision/issues/2921">here</a> with more suggestions.</p>

Xanthan: <p>Ya, i have read this thread and it says that the fix is to do lazy import or something. But I don’t really understand it as _lmza should be imported by default when torchvision is imported. and as you said it is a default python package but the difference is I’m using python version 3.7.10  and you have mentioned about python3.5<br>
Any suggestions what I should do next?</p>

ptrblck: <p>You could follow <a href="https://github.com/pytorch/vision/issues/2921#issuecomment-811767491">this advice</a>, install the missing library, and then reinstall Python so that it’ll add the support. Alternatively, you could also search for other standalone Python libs (such as in your previous suggestion), but I haven’t tried to install any of them.</p>

Xanthan: <p>I tried to install that package but it says that the package is already up to date</p>
<pre><code class="lang-auto">sudo apt-get install -y liblzma-dev

Reading package lists... Done
Building dependency tree       
Reading state information... Done
liblzma-dev is already the newest version (5.1.1alpha+20120614-2ubuntu2).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
</code></pre>
<p>So it wasn’t missing in the first place…</p>

Xanthan: <p>So there might a issue in torchvision.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="119894">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>A related issue is <a href="https://github.com/pytorch/vision/issues/2921" rel="noopener nofollow ugc">here</a> with more suggestions.</p>
</blockquote>
</aside>
<p>The thread here says that torchvision works fine with version 0.7.0</p>
<p>do u recommend downgrading the version? (if so then please also show how to)</p>

ptrblck: <p>I don’t think it’s an issue in <code>torchvision</code>, as <code>lzma</code> should be a default Python package, and thus it would be a valid dependency.<br>
I don’t know why your Python setup isn’t recognizing it, so you might need to create a new environment maybe.</p>

Xanthan: <p>I listed down all the packages of python using <code>help("modules")</code></p>
<pre><code class="lang-auto">PIL                 _weakrefset         heapq               scipy
__future__          _xxtestfuzz         hmac                seaborn
_abc                abc                 html                secrets
_ast                absl                http                select
_asyncio            aifc                idlelib             selectors
_bisect             antigravity         imaplib             setuptools
_blake2             argparse            imghdr              shelve
_bootlocale         array               imp                 shlex
_bz2                ast                 importlib           shutil
_codecs             asynchat            inspect             signal
_codecs_cn          asyncio             io                  site
_codecs_hk          asyncore            ipaddress           six
_codecs_iso2022     atexit              itertools           smtpd
_codecs_jp          audioop             json                smtplib
_codecs_kr          base64              keyword             sndhdr
_codecs_tw          bdb                 kiwisolver          socket
_collections        binascii            lib2to3             socketserver
_collections_abc    binhex              linecache           spwd
_compat_pickle      bisect              locale              sqlite3
_compression        builtins            logging             sre_compile
_contextvars        bz2                 lzma                sre_constants
_crypt              cProfile            macpath             sre_parse
_csv                caffe2              mailbox             ssl
_ctypes             calendar            mailcap             stat
_ctypes_test        cgi                 marshal             statistics
_curses             cgitb               math                string
_curses_panel       chunk               matplotlib          stringprep
_datetime           cmath               mimetypes           struct
_dbm                cmd                 mmap                subprocess
_decimal            code                modulefinder        sunau
_dummy_thread       codecs              multiprocessing     symbol
_elementtree        codeop              netrc               symtable
_functools          collections         nis                 sys
_gdbm               colorsys            nntplib             sysconfig
_hashlib            compileall          ntpath              syslog
_heapq              concurrent          nturl2path          tabnanny
_imp                configparser        numbers             tarfile
_io                 contextlib          numpy               telnetlib
_json               contextvars         opcode              tempfile
_locale             copy                operator            termios
_lsprof             copyreg             optparse            test
_markupbase         crypt               os                  textwrap
_md5                csv                 ossaudiodev         this
_multibytecodec     ctypes              pandas              threading
_multiprocessing    curses              parser              time
_opcode             cycler              pathlib             timeit
_operator           dataclasses         pdb                 tkinter
_osx_support        datetime            pickle              token
_pickle             dateutil            pickletools         tokenize
_posixsubprocess    dbm                 pip                 torch
_py_abc             decimal             pipes               torchaudio
_pydecimal          difflib             pkg_resources       torchvision
_pyio               dis                 pkgutil             trace
_queue              distutils           platform            traceback
_random             doctest             plistlib            tracemalloc
_remote_module_non_sriptable dummy_threading     poplib              tty
_sha1               easy_install        posix               turtle
_sha256             email               posixpath           turtledemo
_sha3               encodings           pprint              types
_sha512             ensurepip           profile             typing
_signal             enum                pstats              typing_extensions
_sitebuiltins       errno               pty                 unicodedata
_socket             faulthandler        pwd                 unittest
_sqlite3            fcntl               py_compile          urllib
_sre                filecmp             pyclbr              uu
_ssl                fileinput           pydoc               uuid
_stat               fnmatch             pydoc_data          venv
_string             formatter           pyexpat             warnings
_strptime           fractions           pylab               wave
_struct             ftplib              pyparsing           weakref
_symtable           functools           pytz                webbrowser
_sysconfigdata_m_linux_x86_64-linux-gnu gc                  queue               wsgiref
_testbuffer         genericpath         quopri              xdrlib
_testcapi           get-pip             random              xml
_testimportmultiple getopt              re                  xmlrpc
_testmultiphase     getpass             readline            xxlimited
_thread             gettext             reprlib             xxsubtype
_threading_local    glob                resource            zipapp
_tracemalloc        grp                 rlcompleter         zipfile
_warnings           gzip                runpy               zipimport
_weakref            hashlib             sched               zlib

</code></pre>
<p>and as you can see there is no package called _lmza<br>
So know that we have established that I don’t have this package, how should I add it ?<br>
By now, I have tried using the following commands:</p>
<pre><code class="lang-auto">pip install backports.lzma
pip install pylzma
sudo apt-get install liblzma-dev
</code></pre>
<p>For the first two there is a error with a long traceback call.<br>
while the 3rd command give this output:</p>
<aside class="quote no-group" data-username="Xanthan" data-post="7" data-topic="119894">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xanthan/48/37484_2.png" class="avatar"> Xanthan:</div>
<blockquote>
<pre><code class="lang-auto">Reading package lists... Done
Building dependency tree       
Reading state information... Done
liblzma-dev is already the newest version (5.1.1alpha+20120614-2ubuntu2).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
</code></pre>
</blockquote>
</aside>

ptrblck: <p>The <code>_lzma</code> module is internal and your list shows indeed an available <code>lzma</code> module.<br>
In my current environment it’s located at:</p>
<pre><code class="lang-auto">/anaconda3/envs/pytorch_181_cuda102/lib/python3.8/lzma.py
</code></pre>
<p>Could you check, if a direct import also fails:</p>
<pre><code class="lang-python">import lzma
</code></pre>

Xanthan: <p>yes the import fails.</p>
<pre><code class="lang-auto">import lmza
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
ModuleNotFoundError: No module named 'lmza'

</code></pre>

ptrblck: <p>Something still seems to be strange in your Python setup.<br>
Which Python version are you using and are you seeing the <code>lzma.py</code> file in the <code>pythonMAJOR.MINOR</code> dir?</p>

Xanthan: <p>I’m using python 3.7.10.</p>
<aside class="quote no-group" data-username="ptrblck" data-post="13" data-topic="119894">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>lzma.py</code> file in the <code>pythonMAJOR.MINOR</code> dir?</p>
</blockquote>
</aside>
<p>Can you please mention where can I find the pythonMAJOR.MINOR?</p>

Qazaq159: <p>Reinstall env libs helped me to solve this issue. Recommend to try again(resinstall python, reinstall virtualenv and its requirements libs).</p>

bolkogor: <p>I had the same issue using debian, and solved it by removing lzma import in <code>{venv_dir}/.../torchvision/datasets/utils.py</code><br>
On line ~240 of <code>utils.py</code> I replaced lzma with:</p>
<pre><code class="lang-auto">def raise_lzma(*args, **kwargs):
    raise Exception('lzma removed error')

_COMPRESSED_FILE_OPENERS: Dict[str, Callable[..., IO]] = {
    ".bz2": bz2.open,
    ".gz": gzip.open,
    ".xz": raise_lzma,
}
</code></pre>
<p>It works if you don’t need compression and do not have sudo access to install <code>liblzma-dev</code> and recompile python.</p>