Repository: pykeen/pykeen
Issue: PairRE model and  self-adversarial negative sampling
GitHub labels: question

### What is your question

I have a question related to PairRE model in general and How you implement in Pykeen. 

As in the paper of PairRE and RotatE , they used self-adversarial negative sampling loss function to produce negative samples which is  NSSALoss in the Pykeen. 

paper of PairRE: https://arxiv.org/abs/2011.03798 

I think Pykeen use uniform negative sampling (negative_sampler) with MarginRankingLoss as default with PairRE model or not ? and why not NSSALoss ? 

Generally, the  idea of PairRE 

1- generate negative sampling by self-adversarial negative sampling and adjust adversarial  temperature to make the random sampling not uniform and then predict the probability of the negative sample given the positive triple 

2- using the probability of this negative triples as a score for the negative sample and then use the modifies of the margin loss function
are these right ? why you do not use NSSALoss  with PairRE? or may I misunderstood something 

### Environment

<html>
<body>
<!--StartFragment-->

Key | Value
-- | --
OS | posix
Platform | Linux
Release | 5.10.107+
Time | Sun May 15 21:17:41 2022
Python | 3.7.12
PyKEEN | 1.8.1
PyKEEN Hash | UNHASHED
PyKEEN Branch |  
PyTorch | 1.11.0
CUDA Available? | true
CUDA Version | 11.0
cuDNN Version | 8005

<!--EndFragment-->
</body>
</html>

### Issue Template Checks

- [X] This is not a bug report (use a different issue template if it is)
- [X] This is not a feature request (use a different issue template if it is)
- [X] I've read the text explaining why including environment information is important and understand if I omit this information that my issue will be dismissed

Hi @Ahmed-fub ,

this should be fixed with #925. 

In general, the implementations in `pykeen.models` should be some sensible default configurations for using the interaction functions. However, since settings differ for datasets, we cannot guarantee that they are suitable for a particular dataset at hand. For reproducing results from publications, we curate *some* experimential configurations under [./src/pykeen/experiments](https://github.com/pykeen/pykeen/tree/master/src/pykeen/experiments); PairRE is not yet there.

Thanks @mberr for your quick reply. 

I am not sure if here is a suitable question here or not, but I misunderstand something about the negative sampling in PairRE. 

Why use uniform sampling with self-adversarial negative sampling loss function? 
I thought that SAN is alternative to uniform sampling.  

SO , Generally, the idea of PairRE as I mentioned in the two points in my first post or Not ? Could you mention the idea of negative sampling in PairRE briefly? Or Could you provide me another source, explain the idea of SAN than the original paper / RotatE paper, if you have ? 


Hi @Ahmed-fub ,

the [original paper](https://arxiv.org/abs/1902.10197) discusses it in chapter 3.3.

In particular, the first propose that we should sample negatives non-uniformly

> We also propose a new approach for drawing negative samples.

But afterwards they note that this would be too costly, and thus they continue to use the uniform samples, but re-weight them according to the probability in the self-adversarial distribution

> Moreover, since the sampling procedure may be costly, we treat the above probability as the weight of the negative sample.

Thanks @mberr 

So they first used uniform sampling and then take the sample with higher probability from the self-adversarial distribution and consider its probability as a score to calculate the loss function with the positive sample


They keep all negative samples, but weight them differently based on how likely the would have been sampled from the self-adversarial distribution. It is somewhat similar to [importance sampling](https://en.wikipedia.org/wiki/Importance_sampling).

Okay, it is more clear now. Thanks