Auto-gptq installation

username: dannn
cooked: <p>Hello, I am trying to install auto-gptq locally and I receive this error (apparently torch is not installed, but it is):</p>
<p>Collecting auto-gptq<br>
Using cached auto_gptq-0.3.2.tar.gz (63 kB)<br>
Installing build dependencies … done<br>
Getting requirements to build wheel … error<br>
error: subprocess-exited-with-error</p>
<p>× Getting requirements to build wheel did not run successfully.<br>
│ exit code: 4294967295<br>
╰─&gt; [1 lines of output]<br>
torch is not installed, please install torch first!<br>
[end of output]</p>
<p>note: This error originates from a subprocess, and is likely not a problem with pip.<br>
error: subprocess-exited-with-error</p>
<p>× Getting requirements to build wheel did not run successfully.<br>
│ exit code: 4294967295<br>
╰─&gt; See above for output.</p>
<p>note: This error originates from a subprocess, and is likely not a problem with pip.</p>
<p>I installed torch:</p>
<p>PS C:\Users\danie&gt; C:\Users\danie\AppData\Local\Programs\Python\Python311\python.exe -m pip install torch<br>
Requirement already satisfied: torch in c:\users\danie\appdata\local\programs\python\python311\lib\site-packages (2.0.1)<br>
…</p>
<p>But still I recieve the error. Why is that? I would appreciate your help very much:)</p>

username: ptrblck
cooked: <p>You could of course remove <a href="https://github.com/PanQiWei/AutoGPTQ/blob/a7167b108c438f570938f0ced46a52fe515f4a59/setup.py#L16-L20">this check</a> if you are convinced PyTorch is properly installed in your environment, but I would guess the build would fail at a later stage and the error is real, so check if different envs are used/created.</p>

username: dannn
cooked: <p>Thank you for your reply. I am only using visual code studio to install everything… I do not have different envs. I do not have conda or so</p>

username: dannn
cooked: <p>Does cuda 12.2 work with the latest torch version? I have cude 12.2</p>

username: ptrblck
cooked: <p>Yes, the PyTorch binaries ship with their own CUDA toolkit and only a properly installed NVIDIA driver is needed. Your locally installed CUDA toolkit will be used if you build PyTorch from source or a custom CUDA extension. The error is also unrelated to CUDA since no installed binary can be found at all.</p>

username: dannn
cooked: <p>Okay, I work in the same environment, have cuda, have a NVIDIA driver (I bought the laptop one week ago and updated everything). Is there anything else I could try to make it work?</p>

username: ptrblck
cooked: <p>Yes, you could install PyTorch manually and run and smoke test to make sure that PyTorch itself can be imported and your GPU used. Something like this should do it: <code>import torch; print(torch.randn(1).cuda())</code>. If this prints a proper <code>CUDATensor</code> your setup looks good. Afterwards try to install <code>auto_gptq</code>. If this then wipes your PyTorch installation or complains again, you could either debug their setup or ask the authors of this package.</p>

username: dannn
cooked: <p>Hi ptrblck,</p>
<p>Thank you for all your help.</p>
<p><code>import torch; print(torch.randn(1).cuda())</code> showd that cuda and torch are working together, but I still had the “torch is not installed, please install torch first!” error installing auto-gptq. I ended up downloading the auto-gptq from github and installing it with setup.py, after some minor errors it worked out well and there was no complaining about “torch is not installed, please install torch first!” despite it being in the setup.py code.</p>
<p>My model is working and I hope everything is fine now, so thank you very much for all your help <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"> I appreciate it very much!</p>

username: ptrblck
cooked: <p>Cool! Thanks for the update and good to hear it’s working now <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>