Model.cuda() takes long time

ming: Even for a small model, calling its cuda() method takes minutes to finish. Is it normal? If yes what it does behind the scene that takes so long? Thanks!

apaszke: We’re aware that there are problems with binary packages we’ve been shipping, that only appear on certain GPU architectures. Would you mind sharing what’s your GPU? I believe this has been already fixed, so a reinstall might help you.

ming: I upgraded pytorch from 0.1.6-py35_19 to 0.1.6-py35_22. It still took 2.5 minutes to run my_small_model.cuda(). I am using Conda Python 3.5 env, GTX 1080 with cuda 8.0. Thanks for your quick reply!

smth: i will update back when this is fixed. I hope to get a fix out today or tomorrow.

ming: 

bellatoris: I am using Conda  Python 2.7 env, GTX 1080 with cuda 8.0 too. And pytorch was installed by binary packages. Even when learning, it was much slower than lua’s torch.nn. Is this problem related to my GPU?  Or is it just a matter of my code? Thanks.

bellatoris: During learning, there is a part where data calls cuda (). Does data.cuda() have same problem? Thanks

smth: This should be fixed now if you use the command (website is updated): <SCODE>conda install pytorch torchvision cuda80 -c soumith
<ECODE>

ming: Tested the new release and it works great. Thanks a lot for the super responsive fix!

chenyuntc: 

stepelu: 

jtang10: Hello I just encountered this problem as well. I am running pytorch on a cluster. Here is the information about the sysmtem and code I am running: Thanks in advance for any help!!!

pipehappy1: I don’t want to hijack the thread, but at least the title fits. The case here is that the home fold where my script lives is NFS mounted from another server. When comes to model.cuda(), pytorch takes some time to move something over the NFS link. But only my script is on NFS. Both pytorch under conda and the data is on local disk. The question is is there a way(or simple modification) to ask pytorch to use a local path to do compiling work if all the hypothesis is correct?

smth: 

Cognac: <SCODE>torch                     0.2.0+59e0472             <pip>
pytorch                   0.2.0           py36hf0d2509_4cu75    soumith
<ECODE> Running a simple script like this gives me the following results: <SCODE>import torch
from datetime import datetime

for i in range(10):
    x = torch.randn(3, 4)
    t1 = datetime.now()
    x.cuda()
    print(i, datetime.now() - t1)
<ECODE> <SCODE>0 0:06:24.108245
1 0:00:00.000110
2 0:00:00.000055
3 0:00:00.000048
4 0:00:00.000046
5 0:00:00.000046
6 0:00:00.000044
7 0:00:00.000044
8 0:00:00.000044
9 0:00:00.000044
<ECODE> Sorry that I’m new to PyTorch so I might be doing something incorrectly. Thanks in advance.

richard: In my experience installing torchvision after installing conda overrides the pytorch source install. However you can re-install pytorch from source and you’ll be using the latest pytorch. <SCODE>import torch
torch.__version__
torch.version.cuda
torch.version.cudnn
<ECODE> ?

Cognac: Thanks for the immediate response! <SCODE>>>> import torch
>>> torch.__version__
'0.2.0_4'
>>> torch.version.cuda
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'torch.version' has no attribute 'cuda'
>>> torch.version.cudnn
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'torch.version' has no attribute 'cudnn'
<ECODE>

richard: Interesting. You’re definitely not on pytorch master, you’re on the version that comes with conda. If you want to build from source again (to see if your problem will go away on master, but I don’t know if it will), you can try the following: (I’m assuming you have the pytorch source code somewhere): <SCODE>pip uninstall torch 
pip uninstall torch # yes, this is intentional
cd pytorch
python setup.py install
<ECODE>

Cognac: 

richard: I’m trying to get torch removed from your pip list. I think you’ll still see pytorch installed via conda – you shouldn’t remove that because I believe removing that will uninstall torchvision as well.