Repository: pytorch
Issue: Standardized Distributions
Labels: module: distributions,triaged

## 🚀 Feature
<!-- A clear and concise description of the feature proposal -->

`torch.normal`, `torch.distributions.normal`, and `torch.tensor.normal_` should allow the same parameters and execute the same function/method in the backend.

## Motivation
<!-- Please outline the motivation for the proposal. Is your feature request related to a problem? e.g., I'm always frustrated when [...]. If this is related to another GitHub issue, please link here too -->

This issue is related to pytorch/opacus/issues/59. The motivation behind it is to allow any distribution to be used by the noise generation of differential privacy in `opacus`, formerly known as `pytorch-dp`.

There are multiple ways to create random samples from the normal (or Gaussian) distribution, three at least that I know of (see above).
However not all allow the same parameters and therefore configuration.

- `torch.distributions.normal.Normal` does not allow a custom `generator` and therefore does not support `urandom`.
  - same goes for the other distributions in `torch.distributions`
- **but** distributions in `torch.tensor` allow a `generator`
- **however** `torch.tensor` only supports a subset of the distributions available in `torch.distributions`
  - in-place sampling from Laplace is not possible
- `torch.distributions` only allow the device to be passed explicitly
  - one needs to pass it through one of the other parameters and use a `Tensor` instead of a `float`
- the in-place sampling gets the device from the `Tensor` it samples for

## Pitch
<!-- A clear and concise description of what you want to happen. -->

1. The distributions in `torch.distributions` should allow the `generator` parameter and (optionally) the explicit `device` parameter.
    - this ensures compatibility for the differential privacy implementation, that was based on `torch.normal`, which as stated [here](https://github.com/pytorch/pytorch/issues/4620#issuecomment-359732513) is no longer the "best practice" (and also does not allow other distributions as only normal is available
2. All other methods and functions to the distributions should allow the same parameters and call the same code in the back end.
    - this would reduce performance and result differences, if multiple researchers (and/or developers) use different methods to produce samples of the same distribution
3. Ideally the [**in-place** distributions](https://pytorch.org/docs/stable/torch.html#in-place-random-sampling) in `torch.tensor` should be complete, meaning include all the distributions available in `torch.distributions`.

## Alternatives

<!-- A clear and concise description of any alternative solutions or features you've considered, if any. -->
If **1.** is not possible, at least each distribution in `torch.distributions` should also be available through `torch.tensor.<distribution_name>_` as stated in **3.**, as it allows the use of the **in-place** sampling to replace `torch-normal` and allows addition of Laplace as an alternative noise generation.

## Additional context
<!-- Add any other context or screenshots about the feature request here. -->

I highly recommend to read the other issue pytorch/opacus/issues/59 for more context.

cc @vincentqb @fritzo @neerajprad @alicanb @vishwakftw @nikitaved