Is AUROC from lightning metrics package intentionally different from the sklearn's?
❓ Questions and Help
Faced this weird behaviour trying to figure out why my model's auroc metric sucks balls deep, while all the others (e. g. accuracy, sensitivity, specificity) are relatively high.
Given two exact same tensors of targets and model's outputs, sklearn metric version outputs high score as intended, whereas plain tensors version gives me something like the opposite of that score. For example, if sklearn metric outputs a roc score of 0.975, tensor metric gives me 0.025.
What is your question?
Is it intentional or some sort of a bug, or am I dumb ass? Sorry for the latter though... I have only used sklearn metrics before, so I kinda got used to them.
Code
>>> import torch
>>> from pytorch_lightning.metrics.classification import AUROC
>>> from pytorch_lightning.metrics.sklearns import AUROC as SKAUROC

>>> auroc = AUROC()
>>> sk_auroc = SKAUROC()

>>> outs = torch.Tensor([[0.8, 0.2], [0.8, 0.2], [0.9, 0.1], [0.1, 0.9]])
>>> targets = torch.Tensor([0, 0, 0, 1])
>>> auroc(pred=outs, target=targets)
tensor(0.)

>>> sk_auroc(y_score=outs[:, 1], y_pred=targets)
tensor(1.)
What's your environment?
* CUDA:
        - GPU:
                - GeForce GTX 1080 Ti
        - available:         True
        - version:           10.2
* Packages:
        - numpy:             1.18.5
        - pyTorch_debug:     False
        - pyTorch_version:   1.5.0
        - pytorch-lightning: 0.8.5
        - tensorboard:       2.2.2
        - tqdm:              4.45.0
* System:
        - OS:                Linux
        - architecture:
                - 64bit
                -
        - processor:         x86_64
        - python:            3.7.5
        - version:           #107-Ubuntu SMP Thu Jun 4 11:27:52 UTC 2020