Repository: pykeen/pykeen
Issue: Train on GPU, run on CPU
Root cause: Configuration Error

### Describe the bug

I have successfuly trained a model on GPU and transferred it to a server, where the predictions will be made using CPU.
I get the following error when I try to make a prediction:

> Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

How do I tell it to use CPU, I have used the map_location to load the model, not sure what more can I do.


### How to reproduce

Train a model on GPU.
Save the directory folder and triples and copy them to a CPU machine.
My code on the CPU machine is:

```python
device = torch.device('cpu')


my_pykeen_model = torch.load('rotate-small/trained_model.pkl', map_location=device)
data = pd.read_csv('triples.csv')
data.dropna(inplace=True, subset=["source", "target", "type"])

tf = TriplesFactory.from_labeled_triples(
  data[["source", "type", "target"]].values,
)

def get_prediction(head, tail):
   result = get_relation_prediction_df(my_pykeen_model, head, tail, triples_factory=tf)
   return result
```

When I execute the get_prediction function I get the following error:

```python-traceback
File "test.py", line 20, in <module>
  print(get_prediction("C0057488","C0086543"))
File "test.py", line 17, in get_prediction
  result = get_relation_prediction_df(my_pykeen_model, head, tail, triples_factory=tf)
File "/home/tomaz/anaconda3/lib/python3.8/site-packages/pykeen/models/predict.py", line 203, in get_relation_prediction_df
  batch = torch.as_tensor([[head_id, tail_id]], dtype=torch.long, device=model.device)
File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/cuda/__init__.py", line 214, in _lazy_init
  torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
```

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.11.0-44-generic        |
| Time            | Sat Jan 15 19:45:04 2022 |
| Python          | 3.8.8                    |
| PyKEEN          | 1.6.1-dev                |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 1.10.1+cu102             |
| CUDA Available? | false                    |
| CUDA Version    | 10.2                     |
| cuDNN Version   | 7605                     |


### Additional information

I've also tried 

my_pykeen_model = torch.load('rotate-small/trained_model.pkl', map_location=device).to(device)

but it doesn't help

Can you please provide the full traceback?

```python-traceback
File "test.py", line 20, in <module>
  print(get_prediction("C0057488","C0086543"))
File "test.py", line 17, in get_prediction
  result = get_relation_prediction_df(my_pykeen_model, head, tail, triples_factory=tf)
File "/home/tomaz/anaconda3/lib/python3.8/site-packages/pykeen/models/predict.py", line 203, in get_relation_prediction_df
  batch = torch.as_tensor([[head_id, tail_id]], dtype=torch.long, device=model.device)
File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/cuda/__init__.py", line 214, in _lazy_init
  torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx


Is `map_location` the right way to do this in `my_pykeen_model = torch.load('rotate-small/trained_model.pkl', map_location=device)`? Shouldn't it be the following?

```python
my_pykeen_model = torch.load('rotate-small/trained_model.pkl').to(device)
```

this produces an error during loading:

```
raceback (most recent call last):
  File "test.py", line 8, in <module>
    my_pykeen_model = torch.load('rotate-small/trained_model.pkl').to(device)
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 607, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 882, in _load
    result = unpickler.load()
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 857, in persistent_load
    load_tensor(data_type, size, key, _maybe_decode_ascii(location))
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 846, in load_tensor
    loaded_storages[key] = restore_location(storage, location)
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 175, in default_restore_location
    result = fn(storage, location)
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 151, in _cuda_deserialize
    device = validate_cuda_device(location)
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 135, in validate_cuda_device
    raise RuntimeError('Attempting to deserialize object on a CUDA '
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.
```

And using both:

```
my_pykeen_model = torch.load('rotate-small/trained_model.pkl', map_location=torch.device('cpu')).to(device)
```

produces the same error as in the start

```
Traceback (most recent call last):
  File "test.py", line 20, in <module>
    print(get_prediction("C0057488","C0086543"))
  File "test.py", line 17, in get_prediction
    result = get_relation_prediction_df(my_pykeen_model, head, tail, triples_factory=tf)
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/pykeen/models/predict.py", line 203, in get_relation_prediction_df
    batch = torch.as_tensor([[head_id, tail_id]], dtype=torch.long, device=model.device)
  File "/home/tomaz/anaconda3/lib/python3.8/site-packages/torch/cuda/__init__.py", line 214, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
```

Maybe you also need to map the device of the Triples factory? Unfortunately I just looked through the api documentation ans it doesn’t look like there’s a way to specify it explicitly with the from_labeled_triples constructor 

There was no problem when training was done on CPU and then transferred to a CPU machine. It specifically doesn't work with a combination of training GPU and running on CPU. Luckily, the dataset is not that big and I'll just train it on CPU and let it run over night, but for future it would be nice if this would be possible.

Thanks for swift replies and great library!

@mberr mentioned he might have an idea to address this. See #730 for our potential solution.

@tomasonjo I wanted to comment on this (hope you don't mind) as I primarily use pykeen in way you are describing (train on GPU, eval on CPU). This is the code I use when loading my cuda model back onto a cpu machine:

``` python 

model = torch.load("trained_model.pkl", map_location=torch.device('cpu'))
model.device = 'cpu'
```

Obviously a very small difference to your current approach so I would be interested to see if this works for you.

@sbonner0 this works for me as well, thanks!

Probably best to add something in the docs to let other people know.

Max has kindly solved this issue in #730. @tomasonjo please let us know if there are any more hiccups