Repository: pykeen/pykeen
Issue: `AdversarialBCEWithLogits` seems to not work properly
Root cause: Algorithmic Error

### Describe the bug

The current implementation of the `AdversarialBCEWithLogits` degrades the performance with the growing number of negative samples (see the illustration) - presumably because of incorrect weighting?

Some basic analysis: 
* pink - the NSSA baseline
* orange - current AdvBCE implementation with 1 negative sample
* purple - current AdvBCE implementation with 128 negative samples
* brown - new (see below) implementation with 64 negatives and `inverse_adversarial_temperature = 5.0`
* red - new (see below) implementation with 64 negatives and `inverse_adversarial_temperature = 10.0`

<img width="1481" alt="Screen Shot 2022-06-27 at 1 21 18 AM" src="https://user-images.githubusercontent.com/4430986/175866004-96ac5da7-fe2c-4f6f-b428-b23ed8a3d361.png">

In the new version, I re-wrote the loss computation process to use **the sum** of `neg_weights` to weight **the sum** of each row in the batch (the batch has a shape `batch_size, 1 + num_negatives`, and only then use the final reduction (often `mean`)
```python
    def forward(
        self,
        pos_scores: torch.FloatTensor,
        neg_scores: torch.FloatTensor,
        neg_weights: torch.FloatTensor,
        label_smoothing: Optional[float] = None,
        num_entities: Optional[int] = None,
    ) -> torch.FloatTensor:  # noqa: D102
        # neg scores might be -inf for masked values -> get rid of those
        mask = torch.isfinite(neg_scores)
        # neg_weights, neg_scores = neg_weights[mask], neg_scores[mask]  # NOT USED
        # create targets
        return self._reduction_method(
            torch.cat([
                neg_weights
                * functional.binary_cross_entropy_with_logits(
                    neg_scores,
                    apply_label_smoothing(torch.zeros_like(neg_scores), epsilon=label_smoothing, num_classes=num_entities),
                    reduction="none",
                ),
                functional.binary_cross_entropy_with_logits(
                pos_scores,
                apply_label_smoothing(torch.ones_like(pos_scores), epsilon=label_smoothing, num_classes=num_entities),
                reduction="none",
            )], dim=-1).sum(dim=-1) / neg_weights.sum(dim=-1)  # new weighting happens here before the final reduction (should be mean anyways)
        )
```


Still, even the fixed version performs worse than the simple NSSA baseline (with margin 9.0 and temperature 1.0) 🤔 

### How to reproduce

Experimenting with a simple pipeline

```python
from pykeen.pipeline import pipeline
from pykeen.datasets import FB15k237
from pykeen.losses import AdversarialBCEWithLogitsLoss

dataset = FB15k237(create_inverse_triples=True)

result = pipeline(
    dataset=dataset,
    model="Rotate",
    model_kwargs=dict(
        embedding_dim=100,
    ),
    loss="AdversarialBCEWithLogits",  # or "nssa"
    loss_kwargs=dict(reduction="mean", inverse_softmax_temperature=5.0),
    optimizer="Adam",
    optimizer_kwargs=dict(lr=0.0005),
    training_loop="sLCWA",
    negative_sampler="BasicNegativeSampler",
    negative_sampler_kwargs=dict(
        num_negs_per_pos=64
    ),

    training_kwargs=dict(
        num_epochs=100,
        callbacks="Evaluation",
        callback_kwargs=dict(
            evaluation_triples=dataset.validation.mapped_triples,
            additional_filter_triples=[dataset.training.mapped_triples],
            frequency=1,
            use_tqdm=False
        )
    ),
)
```

### Environment

The latest master

| Key             | Value                    |
|-----------------|--------------------------|
| OS              | posix                    |
| Platform        | Darwin                   |
| Release         | 21.5.0                   |
| Time            | Mon Jun 27 01:35:04 2022 |
| Python          | 3.8.5                    |
| PyKEEN          | 1.8.3-dev                |
| PyKEEN Hash     | 62220dfe                 |
| PyKEEN Branch   | master                   |
| PyTorch         | 1.10.0                   |
| CUDA Available? | false                    |
| CUDA Version    | N/A                      |
| cuDNN Version   | N/A                      |

### Additional information

_No response_

Could you also run new with 1/128 negatives, or old with 64? Just to rule out that there is something special about 64 samples 😉 

Also, having the new implementation as (draft) PR would make it easier to directly compare the two variants.

Yes, added a few runs with the same temperature:
* Green - old with 64 negs (very bad perf either)
* Blue - new with 1 neg 
* Red - new with 128 negs

New version is consistently better, I'll draft a PR

![Screen Shot 2022-06-27 at 1 37 50 PM](https://user-images.githubusercontent.com/4430986/176002460-b88bfc97-d67f-49e4-95f1-47cff23c751a.png)