Slow accuracy metric
🐛 Bug

Accuracy class from metrics.classification is too slow, compared to pytorch vanilla implementation.
To Reproduce
Try the following demo code.

Code sample

from time import time
import torch
from pytorch_lightning.metrics.classification import Accuracy


def test_acc(pred, targ, get_acc, steps=1000, rep=10):
    tsum, mean_res = 0, []
    for i in range(rep):
        res = []
        ts = time()
        for j in range(steps):
            res.append(get_acc(pred, targ))
        te = time()
        tsum += te - ts
        mean_acc = torch.stack(res).mean().item()
        mean_res.append(mean_acc)
    
    return (tsum / rep), mean_res


def demo(batch_size=32, num_classes=10, device='cuda', steps=100, rep=5):
    print("Testing  bs: %d,  num_cls: %d,  device: %s,  steps: %d,  rep: %d" % (batch_size, num_classes, device, steps, rep))

    pred = torch.randint(0, num_classes, (batch_size, )).to(device=device)
    targ = torch.randint(0, num_classes, (batch_size, )).to(device=device)

    pl_acc = Accuracy(num_classes=num_classes)
    def pt_acc(pred, targ):
        return pred.eq(targ).to(dtype=torch.float).mean()

    pl_time, pl_res = test_acc(pred, targ, pl_acc, steps, rep)
    pt_time, pt_res = test_acc(pred, targ, pt_acc, steps, rep)

    if pl_res != pt_res:
        print("Results mismatch!")
    print("Lightning: %2.6fs" % pl_time)
    print("Vanilla  : %2.6fs" % pt_time)
    print("")

if __name__ == "__main__":
    demo(32, 10)
    demo(512, 10)
    demo(32, 100)
    demo(512, 100)
    demo(32, 1000)
    demo(512, 1000)

    demo(32, 10, 'cpu')
    demo(512, 10, 'cpu')
    demo(32, 100, 'cpu')
    demo(512, 100, 'cpu')
    demo(32, 1000, 'cpu')
    demo(512, 1000, 'cpu')
On my machine, the results are like the following.
Testing  bs: 32,  num_cls: 10,  device: cuda,  steps: 100,  rep: 5
Lightning: 0.378017s
Vanilla  : 0.002766s

Testing  bs: 512,  num_cls: 10,  device: cuda,  steps: 100,  rep: 5
Lightning: 0.372391s
Vanilla  : 0.002776s

Testing  bs: 32,  num_cls: 100,  device: cuda,  steps: 100,  rep: 5
Lightning: 3.346141s
Vanilla  : 0.002812s

Testing  bs: 512,  num_cls: 100,  device: cuda,  steps: 100,  rep: 5
Lightning: 3.335049s
Vanilla  : 0.002660s

Testing  bs: 32,  num_cls: 1000,  device: cuda,  steps: 100,  rep: 5
Lightning: 32.751692s
Vanilla  : 0.002666s

Testing  bs: 512,  num_cls: 1000,  device: cuda,  steps: 100,  rep: 5
Lightning: 32.967594s
Vanilla  : 0.002762s

Testing  bs: 32,  num_cls: 10,  device: cpu,  steps: 100,  rep: 5
Lightning: 0.128131s
Vanilla  : 0.001317s

Testing  bs: 512,  num_cls: 10,  device: cpu,  steps: 100,  rep: 5
Lightning: 0.128481s
Vanilla  : 0.001403s

Testing  bs: 32,  num_cls: 100,  device: cpu,  steps: 100,  rep: 5
Lightning: 1.117403s
Vanilla  : 0.001275s

Testing  bs: 512,  num_cls: 100,  device: cpu,  steps: 100,  rep: 5
Lightning: 1.166188s
Vanilla  : 0.001330s

Testing  bs: 32,  num_cls: 1000,  device: cpu,  steps: 100,  rep: 5
Lightning: 10.927701s
Vanilla  : 0.001229s

Testing  bs: 512,  num_cls: 1000,  device: cpu,  steps: 100,  rep: 5
Lightning: 11.560013s
Vanilla  : 0.001310s

100x ~ 10000x more time is needed for pl's accuracy, depending on the num_classes argument.
Expected behavior
Pytorch-lightning's accuracy function should take similar time compared to the pytorch vanilla implementation, and it should not depend on num_classes too much.

Environment
* CUDA:
	- GPU:
		- GeForce GTX 1050 Ti
	- available:         True
	- version:           10.2
* Packages:
	- numpy:             1.19.0
	- pyTorch_debug:     False
	- pyTorch_version:   1.5.1
	- pytorch-lightning: 0.8.1
	- tensorboard:       2.2.2
	- tqdm:              4.47.0
* System:
	- OS:                Linux
	- architecture:
		- 64bit
		- ELF
	- processor:         x86_64
	- python:            3.6.9
	- version:           #46~18.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020

I tested the same code from two other machines (each with 2080 ti and with V100), and they give similar results.
Additional context

I'm not sure this issue falls into the bug report or the feature request category. Please relocate this as you like. :)