Repository: pytorch
Issue: LayerNorm and GroupNorm with num_groups=1 not equivalent
Labels: module: nn,triaged,module: norms and normalization

### 🐛 Describe the bug

`LayerNorm` and `GroupNorm` with `num_groups=1` should be equivalent but they are not

```python
from torch import nn
import torch
import random
random.seed(0)
torch.manual_seed(0)
x = torch.randn((1, 8, 2, 2))


x_norm = x.permute(0, 2, 3, 1)
x_norm = nn.LayerNorm(8, eps=1e-6)(x_norm)
x_norm.permute(0, 3, 1, 2)

print(x_norm[0,:2,:2,:2])
```

```
tensor([[[-1.4655,  0.7272],
         [-0.6575,  0.8390]],

        [[-0.8825, -1.0009],
         [-0.3389, -1.8741]]], grad_fn=<SliceBackward0>)
```


```python
x_norm = nn.GroupNorm(1, 8, eps=1e-6)(x)

print(x_norm[0,:2,:2,:2])
```

```
tensor([[[-1.1255, -1.1518],
         [-0.2556, -0.4378]],

        [[ 0.8369,  0.6812],
         [-0.3206, -2.1088]]], grad_fn=<SliceBackward0>)
````


### Versions

```
Collecting environment information...
PyTorch version: 1.11.0+cu102
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.3 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.13.0-39-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1080 Ti
Nvidia driver version: 510.54
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.3
[pip3] pytorch-lightning==1.5.10
[pip3] pytorch-pretrained-biggan==0.1.1
[pip3] torch==1.11.0
[pip3] torchaudio==0.11.0
[pip3] torchcam==0.3.1
[pip3] torchinfo==1.6.5
[pip3] torchmetrics==0.7.3
[pip3] torchvision==0.12.0
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.22.3                   pypi_0    pypi
[conda] pytorch-lightning         1.5.10                   pypi_0    pypi
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] pytorch-pretrained-biggan 0.1.1                    pypi_0    pypi
[conda] torch                     1.11.0                   pypi_0    pypi
[conda] torchaudio                0.11.0                   pypi_0    pypi
[conda] torchcam                  0.3.1                    pypi_0    pypi
[conda] torchinfo                 1.6.5                    pypi_0    pypi
[conda] torchmetrics              0.7.3                    pypi_0    pypi
[conda] torchvision               0.12.0                   pypi_0    pypi
```

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345