Repository: pykeen/pykeen
Issue: Device error for filtered negative samples
Root cause: Resource Mishandling

### Describe the bug

When I follow the guide[https://pykeen.readthedocs.io/en/v1.7.0/reference/negative_sampling.html#identifying-false-negatives-during-training](url) to add `negative_sampler_kwargs=dict(filtered=True)` at pipeline, this error caused. When I remove it, it seems back to work well.

### How to reproduce

```
result=pipeline(
    #stopper='early',
    device="cuda:0",
    training=training,
    testing=testing,
    validation=validation,
    optimizer='Adam',
    model="RotatE",
    loss='softplus',
    training_loop='sLCWA',
    negative_sampler='basic',
    negative_sampler_kwargs=dict(
        filtered=True,          
    ),
    model_kwargs=dict(embedding_dim=200),
    optimizer_kwargs=dict(lr=1.0e-1),
    training_kwargs=dict(
        num_epochs=1000,
        checkpoint_name='RotatE.pt',
        batch_size=128,
        use_tqdm_batch=True,
    ),
)
```

### Environment

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Linux                    |
| Release         | 5.13.0-30-generic        |
| Time            | Mon Feb 21 16:36:56 2022 |
| Python          | 3.9.7                    |
| PyKEEN          | 1.7.0                    |
| PyKEEN Hash     | UNHASHED                 |
| PyKEEN Branch   |                          |
| PyTorch         | 1.10.0+cu113             |
| CUDA Available? | true                     |
| CUDA Version    | 11.3                     |
| cuDNN Version   | 8200                     |

### Additional information

This is part of the error.
```
[File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py:709, in TrainingLoop._train(self, triples_factory, training_instances, num_epochs, batch_size, slice_size, label_smoothing, sampler, continue_training, only_size_probing, use_tqdm, use_tqdm_batch, tqdm_kwargs, stopper, result_tracker, sub_batch_size, num_workers, save_checkpoints, checkpoint_path, checkpoint_frequency, checkpoint_on_failure_file_path, best_epoch_model_file_path, last_best_epoch, drop_last, callbacks, gradient_clipping_max_norm, gradient_clipping_norm_type, gradient_clipping_max_abs_value)
    ]()[707](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=706)[     if best_epoch_model_file_path is not None and best_epoch_model_file_path.is_file():
    ]()[708](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=707)[         os.remove(best_epoch_model_file_path)
--> ]()[709](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=708)[     raise e
    ]()[711](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=710)[ # Includes a call to result_tracker.log_metrics
    ]()[712](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=711)[ callback.post_epoch(epoch=epoch, epoch_loss=epoch_loss)]()
[File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py:606, in TrainingLoop._train(self, triples_factory, training_instances, num_epochs, batch_size, slice_size, label_smoothing, sampler, continue_training, only_size_probing, use_tqdm, use_tqdm_batch, tqdm_kwargs, stopper, result_tracker, sub_batch_size, num_workers, save_checkpoints, checkpoint_path, checkpoint_frequency, checkpoint_on_failure_file_path, best_epoch_model_file_path, last_best_epoch, drop_last, callbacks, gradient_clipping_max_norm, gradient_clipping_norm_type, gradient_clipping_max_abs_value)
    ]()[603](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=602)[ stop = min(start + _sub_batch_size, current_batch_size)
    ]()[605](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=604)[ # forward pass call
--> ]()[606](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=605)[ batch_loss = self._forward_pass(
    ]()[607](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=606)[     batch,
    ]()[608](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=607)[     start,
    ]()[609](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=608)[     stop,
    ]()[610](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=609)[     current_batch_size,
    ]()[611](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=610)[     label_smoothing,
    ]()[612](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=611)[     slice_size,
    ]()[613](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=612)[ )
    ]()[614](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=613)[ current_epoch_loss += batch_loss
    ]()[615](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=614)[ callback.on_batch(epoch=epoch, batch=batch, batch_loss=batch_loss)]()
[File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py:763, in TrainingLoop._forward_pass(self, batch, start, stop, current_batch_size, label_smoothing, slice_size)
    ]()[753](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=752)[ def _forward_pass(
    ]()[754](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=753)[     self,
    ]()[755](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=754)[     batch: BatchType,
   (...)
    ]()[761](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=760)[ ) -> float:
    ]()[762](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=761)[     # forward pass
--> ]()[763](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=762)[     loss = self._process_batch(
    ]()[764](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=763)[         batch=batch,
    ]()[765](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=764)[         start=start,
    ]()[766](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=765)[         stop=stop,
    ]()[767](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=766)[         label_smoothing=label_smoothing,
    ]()[768](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=767)[         slice_size=slice_size,
    ]()[769](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=768)[     )
    ]()[771](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=770)[     # raise error when non-finite loss occurs (NaN, +/-inf)
    ]()[772](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/training_loop.py?line=771)[     if not torch.isfinite(loss):]()
[File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/slcwa.py:79, in SLCWATrainingLoop._process_batch(self, batch, start, stop, label_smoothing, slice_size)
     ]()[76](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/slcwa.py?line=75)[ positive_batch = batch[start:stop].to(device=self.device)
     ]()[78](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/slcwa.py?line=77)[ # Create negative samples, shape: (batch_size, num_neg_per_pos, 3)
---> ]()[79](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/slcwa.py?line=78)[ negative_batch, positive_filter = self.negative_sampler.sample(positive_batch=positive_batch)
     ]()[81](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/slcwa.py?line=80)[ # apply filter mask
     ]()[82](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/training/slcwa.py?line=81)[ if positive_filter is None:

File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/negative_sampler.py:95, in NegativeSampler.sample(self, positive_batch)
     ]()[92](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/negative_sampler.py?line=91)[     return negative_batch, None
     ]()[94](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/negative_sampler.py?line=93)[ # If filtering is activated, all negative triples that are positive in the training dataset will be removed
---> ]()[95](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/negative_sampler.py?line=94)[ return negative_batch, self.filterer(negative_batch=negative_batch)

File ~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py:1102, in Module._call_impl(self, *input, **kwargs)
   ]()[1098](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1097)[ # If we don't have any hooks, we want to skip the rest of the logic in
   ]()[1099](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1098)[ # this function, and just call forward.
   ]()[1100](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1099)[ if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   ]()[1101](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1100)[         or _global_forward_hooks or _global_forward_pre_hooks):
-> ]()[1102](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1101)[     return forward_call(*input, **kwargs)
   ]()[1103](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1102)[ # Do not call functions when jit is used
   ]()[1104](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1103)[ full_backward_hooks, non_full_backward_hooks = [], []]()
[File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py:171, in Filterer.forward(self, negative_batch)
    ]()[148](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=147)[ def forward(
    ]()[149](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=148)[     self,
    ]()[150](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=149)[     negative_batch: MappedTriples,
    ]()[151](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=150)[ ) -> torch.BoolTensor:
    ]()[152](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=151)[     """Filter all proposed negative samples that are positive in the training dataset.
    ]()[153](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=152)[ 
    ]()[154](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=153)[     Normally there is a low probability that proposed negative samples are positive in the training datasets and
   (...)
    ]()[169](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=168)[         A mask, where True indicates that the negative sample is valid.
    ]()[170](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=169)[     """
--> ]()[171](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=170)[     return ~self.contains(batch=negative_batch)

File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py:348, in BloomFilterer.contains(self, batch)
    ]()[337](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=336)[ """
    ]()[338](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=337)[ Check whether a triple is contained.
    ]()[339](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=338)[ 
   (...)
    ]()[345](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=344)[     erroneous.
    ]()[346](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=345)[ """
    ]()[347](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=346)[ result = batch.new_ones(batch.shape[:-1], dtype=torch.bool)
--> ]()[348](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=347)[ for i in self.probe(batch):
    ]()[349](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=348)[     result &= self.bit_array[i]
    ]()[350](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=349)[ return result]()
[File ~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py:321, in BloomFilterer.probe(self, batch)
    ]()[312](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=311)[ """
    ]()[313](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=312)[ Iterate over indices from the probes.
    ]()[314](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=313)[ 
   (...)
    ]()[318](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=317)[ :yields: Indices of the k-th round, shape: (batch_size,).
    ]()[319](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=318)[ """
    ]()[320](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=319)[ # pre-hash
--> ]()[321](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=320)[ x = (self.mersenne * batch).sum(dim=-1)
    ]()[322](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=321)[ for _ in range(self.rounds):
    ]()[323](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=322)[     # cf. https://github.com/skeeto/hash-prospector#two-round-functions
    ]()[324](file:///~/miniconda3/envs/torch/lib/python3.9/site-packages/pykeen/sampling/filtering.py?line=323)[     x = x ^ (x >> 16)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!]()
```

@lovefxyo 

this should be fixed with #801 being merged. Until our next release, you need to install directly from GitHub, cf. https://github.com/pykeen/pykeen#installation--