Repository: pytorch
Issue: ```torch.distributions.Categorical``` unintended  ```log_prob``` gradient w.r.t ```probs```
Labels: module: distributions,triaged

## 🐛 Bug

When ```torch.distributions.Categorical``` is initialized with ```probs```, the implementation normalizes it even if it is already normalized. However, if we give normalized values to ```probs```, this normalization leads to incorrect gradients. This happens when we have the softmax output of some tensor and we want to use Categorical distribution with the normalized prob values that we have.

## To Reproduce

```Python
import torch

normalized_probs = torch.tensor([[0.2, 0.8]], dtype=torch.float32, requires_grad=True)

dist = torch.distributions.Categorical(probs=normalized_probs)
sample = dist.sample()
dist.log_prob(sample).sum().backward()

print(normalized_probs.grad, sample)
>>> tensor([[ 4., -1.]]) tensor([0])
```
We expect to see zero in the gradient at index 1 since that index is not sampled, but we observed -1.0

## Expected behavior

The correct gradient can be obtained by ignoring the additional normalization performed by ```torch.distributions.Categorical``` at the initialization.

```Python
normalized_probs = torch.tensor([[0.2, 0.8]], dtype=torch.float32, requires_grad=True)

dist = torch.distributions.Categorical(probs=normalized_probs)
dist.probs = normalized_probs  # This line solves the issue
dist.log_prob(sample).sum().backward()

print(normalized_probs.grad, sample)
>>> tensor([[5., 0.]]) tensor([0])
```
This time the gradient at index 1 is zero.

## Environment

PyTorch version: 1.7.0
Is debug build: True
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 10.15.7 (x86_64)
GCC version: Could not collect
Clang version: 12.0.0 (clang-1200.0.32.27)
CMake version: version 3.19.0
Libc version: N/A

Python version: 3.8.3 (default, Jul  2 2020, 11:26:31)  [Clang 10.0.0 ] (64-bit runtime)
Python platform: macOS-10.15.7-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.5
[pip3] numpydoc==1.1.0
[pip3] torch==1.7.0
[pip3] torchaudio==0.7.0a0+ac17b64
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.8.1
[conda] blas                      1.0                         mkl
[conda] mkl                       2019.4                      233
[conda] mkl-service               2.3.0            py38hfbe908c_0
[conda] mkl_fft                   1.1.0            py38hc64f4ea_0
[conda] mkl_random                1.1.1            py38h959d312_0
[conda] numpy                     1.18.5           py38h1da2735_0
[conda] numpy-base                1.18.5           py38h3304bdc_0
[conda] numpydoc                  1.1.0                      py_0
[conda] pytorch                   1.7.0                   py3.8_0    pytorch
[conda] torchaudio                0.7.0                      py38    pytorch
[conda] torchsummary              1.5.1                    pypi_0    pypi
[conda] torchvision               0.8.1                  py38_cpu    pytorch



cc @fritzo @neerajprad @alicanb @nikitaved @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @agolynski @SciPioneer @H-Huang @mrzzd @cbalioglu @gcramer23