gopikrsmscs/torch-issues huggingface dataset:
How to handle __module__  attribute for Public API bindings
Labels: module: tests, triaged

While working on the NN onboarding lab (with corresponding closed PR: #77425 ), after registering the functional version of new module in `torch/nn/functional.py` The following test would fail ` pytest test/test_public_bindings.py` with:
```Bash
Full list:
# torch.nn.functional.bias:
  - Is public: it is an attribute that does not start with `_` on a module that does not have `__all__` defined
  - Does NOT look public: because its `__module__` attribute (`torch._C._nn`) is not within the torch library or does not start with the submodule where it is defined (`torch.nn.functional`)
  - You can do either of these two things to fix this problem:
    - To make it NOT public: either define a `__all__` for `torch.nn.functional` or add a `_` at the beginning of the name
    - To make it look public: make sure the `__module__` is properly set and points to a submodule of `torch.nn.functional`
```    
I defined the functional version analogously to the linear module:
```Python
bias = _add_docstr(
    torch._C._nn.bias,
    r"""
bias(input, bias) -> Tensor

Adds a bias vector the last dimension of input tensor

Shape:
    - Input: math:`(*, num\_features)` where `*` means any number of
      additional dimensions, including none
    - Bias: :math:`(num\_features)` or :math:`()`
    - Output: :math:`(*, num\_features)` where `*` means any number of
      additional dimensions, including none, same shape as Input
""")
```

I add this function 'bias' to the allowlist here: `test/allowlist_for_publicAPI.json` in the list for `"torch.nn.functional"`

When reading the test function though it says that no new functions should be added to this list. If I def bias above and then implement `bias.__module__ = 'torch.nn.functional'` This does indeed work. 

Is that the correct solution?
 Would it be a nicer API if there was a function analogous to `_add_docstr` which also defined the `__module__` attribute when setting the doc string.


cc @mruberry
FSDP: test mixed precision with checkpoint
Labels: oncall: distributed, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

When training with mixed precision, we should ensure we take checkpoint with parameters, buffers in full precision so that it can be loaded in for full precision retraining. We currently cast params and buffers to their full precision dtype before taking checkpoint, but this should be validated via unittests.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
`stateless.functional_call` doesn't work with `nn.DataParallel`
Labels: module: nn, triaged, module: data parallel, actionable

### 🐛 Describe the bug

Encountered in https://github.com/pytorch/pytorch/pull/77137#issuecomment-1125694907 
```
import torch
import torch.nn as nn
import torch.nn.utils.stateless as stateless

class Foo(nn.Module):
    def __init__(self):
        super().__init__()
        self.weight = nn.Parameter(torch.ones(5))

    def forward(self, x):
        return self.weight + x

mod = Foo().cuda()
mod = nn.DataParallel(mod, [0, 1])
print(stateless.functional_call(mod, {'module.weight': torch.zeros(5, device='cuda')}, (torch.ones(2, 5, device='cuda'),)))
```
errors with
``
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1!
```

### Versions

Nightly

cc @ezyang @gchanan @zou3519 @albanD @mruberry @jbschlosser @walterddr @kshitij12345
Investigate sharded gradscaler OOM on CPU workloads
Labels: oncall: distributed, module: fsdp

### 🐛 Describe the bug

Sharded grad scaler for FSDP + mixed precision is landed: https://github.com/pytorch/pytorch/pull/76918, although we're seeing an OOM on workloads with CPU offload at the moment. We should investigate and fix this OOM. 

### Versions

main

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
lintrunner doesn't give good error message suggesting lintrunner init
Labels: module: lint, triaged

### 🐛 Describe the bug

```
  Error (CLANGTIDY) Linter failed
    Linter failed. This a bug, please file an issue against the linter
    maintainer.
    
    CONTEXT:
    Linter command failed with non-zero exit code.
    STDERR:
    Traceback (most recent call last):
      File "tools/linter/adapters/clangtidy_linter.py", line 138, in <module>
        ] + clang_search_dirs()
      File "tools/linter/adapters/clangtidy_linter.py", line 106, in
    clang_search_dirs
        result = subprocess.run(
      File "/scratch/ezyang/pytorch-scratch2-env/lib/python3.8/subprocess.py",
    line 512, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['clang', '-E', '-x', 'c++', '-',
    '-v']' returned non-zero exit status 1.
    
    
    STDOUT:
    

```

cc @suo 

I have to say this is a very useless error message lol. I think the problem is I didn't run lintrunner init.

### Versions

master
batch Kronecker product 
Labels: triaged, enhancement

### 🚀 The feature, motivation and pitch

I have two tensors that are batches of matrices:
```
x = torch.randn(100,10,10) 
y = torch.randn(100,2,2)
```

I want to parallelize the kronecker on each matrix, not doing the kronecker product of the tensors. torch.kron(x,y) gives me a tensor of size (10000, 20,20), but I want a shape out of size (100, 20, 20) that computes the kronecker product for each matrix. Is there any way to do so ?

Something like torch.kron(x,y, start_dim = 1) is what I tried but it does not seem to be implemented.

Thanks


### Alternatives

A parameter start_dim that tells form which dimension we want to do perform a kronecker product. 

### Additional context

_No response_
DDP window TCP bug [socket.cpp:558] [c10d] The client socket has failed to 
Labels: oncall: distributed

### 🐛 Describe the bug

Hi   

I have two window boxes, each host has a single GPU.  Both can communicate and I disabled the firewall etc.  
Both hosts added A recond in the DNS server.  I added both while troubleshooting.  added in both host's file

Master node.  (IP address 192.168.254.230)

```python
torch.distributed.init_process_group(
            backend="gloo",
            init_method="tcp://localhost:54321"   // I tried to us 192.168.254.230 , actual hostname.  win11 , win11.mydomain_name
            world_size=2,
            rank=0)
```
Second node.
Master node.  (IP address 192.168.254.230)

```python
torch.distributed.init_process_group(
            backend="gloo",
            init_method="tcp://192.168.254.230:54321" actual hostname.  win11 , win11.mydomain_name
            world_size=2,
            rank=1)
```

Tried to use the os.env variable etc, In all cases, I see this error on the client node ( i.e rank 1)

In all cases, I see the error and both master and node never return from the init call.
Meanwhile, I run Wireshark and I see a TCP session and handshake on the same port  54321.

```
[W C:\cb\pytorch_1000000000000\work\torch\csrc\distributed\c10d\socket.cpp:558] [c10d] The client socket has failed to connect to [win11.vmwarelab.edu]:54321 (system error: 10049 - The requested address is not valid in its context.).
```

```
>>> import torch
>>> print(torch.__version__)
1.11.0
```

If you need Wireshark PCAPI can send but I seek the last packet from the master ACK 

PCAP screenshoot

<img width="839" alt="image" src="https://user-images.githubusercontent.com/11797329/168544258-d191881d-ce10-4b87-a4d9-f6d3041d7874.png">


### Versions

1.11.0


cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
Inplace Bool API + `sum` will trigger INTERNAL ASSERT FAILED
Labels: module: autograd, triaged, module: edge cases

### 🐛 Describe the bug

Inplace Bool API + `sum` will trigger INTERNAL ASSERT FAILED, like `eq_, le_, ne_, ge_, gt_` 

```python
import torch
input = torch.randint(-4, 8, [4, 3], dtype=torch.int64)
other = torch.rand([4, 3], dtype=torch.float64, requires_grad=True)

input.eq_(other)
print(input)

input.sum()
# tensor([[0, 0, 0],
#         [0, 0, 0],
#         [0, 0, 0],
#         [0, 0, 0]], grad_fn=<EqBackward1>)
# 
# RuntimeError: isDifferentiableType(variable.scalar_type())INTERNAL ASSERT FAILED at "/Users/distiller/project/pytorch/torch/csrc/autograd/functions/utils.h":65, please report a bug to PyTorch. 
```

By the way, `index_fill_` can also trigger such bug

```python
import torch
input = torch.tensor(True, dtype=torch.bool)
index = torch.tensor(0, dtype=torch.int64)
dim = 0
value = torch.rand([], dtype=torch.float64, requires_grad=True)

input.index_fill_(dim, index, value).sum()
# RuntimeError: isDifferentiableType(variable.scalar_type())INTERNAL ASSERT FAILED at "/Users/distiller/project/pytorch/torch/csrc/autograd/functions/utils.h":65, please report a bug to PyTorch. 
```

### Versions

pytorch: 1.11.0

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
`max_pool1d` can succeed when padding is negative for tensor requiring grad
Labels: module: nn, triaged, actionable, module: pooling, module: edge cases

### 🐛 Describe the bug

`max_pool1d` can succeed when padding is negative for tensor requiring grad

```python
import torch

input = torch.rand([20, 16, 50], dtype=torch.float32, requires_grad=True)

kernel_size = 3
stride = 2
padding = -1
torch.nn.functional.max_pool1d(input, kernel_size, stride=stride, padding=padding)
# work
```

```python
import torch

input = torch.rand([20, 16, 50], dtype=torch.float32, requires_grad=False)

kernel_size = 3
stride = 2
padding = -1
torch.nn.functional.max_pool1d(input, kernel_size, stride=stride, padding=padding)
# RuntimeError: max_pool1d() padding must be non-negative, but got -1
```

### Versions

pytorch: 1.11.0

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345
Standalone unittests for checkpoint_wrapper
Labels: oncall: distributed, module: bootcamp, triaged, better-engineering, module: fsdp

### 🚀 The feature, motivation and pitch

New features to checkpoint_wrapper are being added such as in https://github.com/pytorch/pytorch/pull/77224, but the file is missing standalone unittests and is instead mostly tested through FSDP (`test_fsdp_checkpoint` and `test_fsdp_state_dict`). To ensure more robust support, we should have standalone unittests independent of FSDP.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501
More clarity in doc for `torch.cuda.Event.record`?
Labels: module: docs, triaged

### 📚 The doc issue

Hi, the documentation for `torch.cuda.Event.record` describes it as "Records the event in a given stream." For someone who is new to CUDA Events, this description is not very clear to me. 

https://pytorch.org/docs/stable/generated/torch.cuda.Event.html#torch.cuda.Event.record

### Suggest a potential alternative/fix

Could we add more details to this description so it is clear to a n00b what "record" actually does?

cc @svekars @holly1238
FSDP: Mixed precision should not cast ignored buffers
Labels: oncall: distributed, triaged, module: fsdp

### 🐛 Describe the bug

Currently, FSDP mixed precision casts all buffers, but if user uses `ignore_modules` API, then these buffers (and parameters) should not be casted.

This currently works for ignored parameters but buffers are still casted. We should add testing as well to ensure params and buffers are not casted if they are part of ignored modules.

### Versions

main

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @pietern @rohan-varma @SciPioneer
Suboptimal error message - nn.Linear with double argument
Labels: module: error checking, triaged

### 🐛 Describe the bug

Following code fails with error message below:
```python
from torch import nn


net = nn.Linear(10, 10)

t = torch.tensor([1.]*10, dtype=torch.double)
net(t)
```

Error:
```
Traceback (most recent call last):
  File "~/Library/Application Support/JetBrains/PyCharm2022.1/scratches/ntuple.py", line 15, in <module>
    net(t)
  File "~/venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "~/venv/lib/python3.9/site-packages/torch/nn/modules/linear.py", line 103, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: expected scalar type Double but found Float
```

There are two things I don't get with this message:
1. I'm passing in a dtype=torch.double, but the message complains inversely that it *expects* Double and found Float. This must be due to order of operations inside nn.Linear, but as a user I am responsible in the first line for type of my inputs, so saying "expects Float, found Double" would make me discover my bugs faster.
2. Why does it say `scalar type`? weight, bias and input are all non-scalars.

In my opinion it would make sense to say explicitly what had what time in this mismatch. Something along lines: "nn.Linear of type `torch.float` expects input of type `torch.float`, but input of type `torch.double` was given."

### Versions

PyTorch version: 1.11.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 11.6.5 (x86_64)
GCC version: Could not collect
Clang version: 12.0.5 (clang-1205.0.22.9)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.12 (main, Mar 26 2022, 15:52:10)  [Clang 13.0.0 (clang-1300.0.29.30)] (64-bit runtime)
Python platform: macOS-11.6.5-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.942
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.6
[pip3] torch==1.11.0
[conda] blas                      1.0                         mkl    https://repo.anaconda.com/pkgs/main
[conda] mkl                       2021.2.0           hecd8cb5_269    https://repo.anaconda.com/pkgs/main
[conda] mkl-service               2.3.0            py38h9ed2024_1    https://repo.anaconda.com/pkgs/main
[conda] mkl_fft                   1.3.0            py38h4a7008c_2    https://repo.anaconda.com/pkgs/main
[conda] mkl_random                1.2.1            py38hb2f4e1b_2    https://repo.anaconda.com/pkgs/main
[conda] numpy                     1.22.2                   pypi_0    pypi
[conda] numpydoc                  1.1.0              pyhd3eb1b0_1    https://repo.anaconda.com/pkgs/main
[conda] torch                     1.11.0                   pypi_0    pypi
[conda] torchvision               0.10.0                   pypi_0    pypi
Allow force building with/without AVX
Labels: module: build, triaged

### 🚀 The feature, motivation and pitch

Currently it seems impossible to force a non-AVX build on an AVX-available machine, or vice versa. This poses a problem on packaging a non-AVX version of pytorch on Arch Linux ([FS#73012](https://bugs.archlinux.org/task/73012)), since it involves building the package for non-AVX machines on a machine with AVX.

I'd like to suggest adding some CMake options for force building with/without AVX or specifying the desired target CPU features.

### Alternatives

_No response_

### Additional context

_No response_

cc @malfet @seemethere
Large numerical inconsistency for `torch.einsum` on RTX30 series GPU.
Labels: module: cuda, triaged, module: tf32

### 🐛 Describe the bug

```python
import torch

print(torch.__version__)

a = torch.rand(1, 8192*10, 3) * 100
b = torch.rand(1, 3, 3)

c_1 = torch.einsum('c...i,cji->c...j', a, b)
c_2 = torch.einsum('c...i,cji->c...j', a.cuda(), b.cuda())
c_3 = (a.cuda()[...,None,:] * b.cuda()[:,None,...]).sum(-1)

print((c_1 - c_2.cpu()).abs().max(), (c_1 - c_3.cpu()).abs().max())
```

Output:
```
1.11.0+cu113
tensor(0.0833), tensor(3.0518e-05)
```

The difference between c_1 and c_2 is ridiculously large. However, when I rewrote cuda einsum to `( _ * _ ).sum(-1)`, the result is normal.

I tested this short script on RTX3090, RTX3060, and they both show similar problematic results.
I also tested on RTX2070, Telsa T4 (colab), they do not have such issue.
 

### Versions

Collecting environment information...
PyTorch version: 1.11.0+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.2 LTS (x86_64)
GCC version: (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.31

Python version: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.11.0-40-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 11.4.48
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090
GPU 3: NVIDIA GeForce RTX 3090

Nvidia driver version: 470.82.00
cuDNN version: Probably one of the following:
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn.so.8.2.1
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_infer.so.8.2.1
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_adv_train.so.8.2.1
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_infer.so.8.2.1
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_cnn_train.so.8.2.1
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_infer.so.8.2.1
/usr/local/cuda-11.4/targets/x86_64-linux/lib/libcudnn_ops_train.so.8.2.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.11.0+cu113
[pip3] torch-ema==0.3
[pip3] torch-scatter==2.0.9
[pip3] torch-tb-profiler==0.4.0
[pip3] torchvision==0.12.0
[conda] cudatoolkit               11.1.1               h6406543_8    conda-forge
[conda] numpy                     1.22.3                   pypi_0    pypi
[conda] torch                     1.11.0+cu113             pypi_0    pypi
[conda] torch-ema                 0.3                      pypi_0    pypi
[conda] torch-scatter             2.0.9                    pypi_0    pypi
[conda] torch-tb-profiler         0.4.0                    pypi_0    pypi
[conda] torchvision               0.12.0                   pypi_0    pypi

cc @ezyang @gchanan @zou3519 @ngimel @zasdfgbnm @ptrblck
microbenchmark-style tests
Labels: module: tests, triaged

### 🚀 The feature, motivation and pitch

We can add a set of tests that are based lists of IR, like the microbenchmarks in torchbench.

Reason - a lot of the microbenchmarks have revealed non-performance issues - so it might be good to add these into the pytorch tests to catch failures, instead of waiting for torchbench failures.

### Alternatives

_No response_

### Additional context

_No response_

cc @mruberry
DISABLED test_ddp_profiling_autograd_profiler (__main__.TestDistBackendWithSpawn)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](http://torch-ci.com/failure/test_ddp_profiling_autograd_profiler%2C%20TestDistBackendWithSpawn) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/6399576647).

Over the past 3 hours, it has been determined flaky in 1 workflow(s) with 1 red and 4 green.

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
Disable issue doesn't disable multiple dtypes correctly
Labels: module: ci, triaged

### 🐛 Describe the bug

I wanted to disable `test_ref_small_input_prod_cuda` and `test_ref_small_input__masked_prod_cuda`, but creating a disabling issue for them didn't actually stop the tests from running (there are int8, int16, and int32 variants for both). I ended up having to create 6 separate issues.


cc @seemethere @malfet @pytorch/pytorch-dev-infra
wrong overload resolved for `torch.mul(x, 4)` in `__torch_dispatch__`
Labels: triaged, module: __torch_dispatch__

### 🐛 Describe the bug

minimal repro

```python
import torch
class ElementwiseMulScalarIntModule(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x):
        return torch.mul(x, 4)

class TrivTensor(torch.Tensor):
    @staticmethod
    def __new__(cls, elem, *, requires_grad=None):
        if requires_grad is None:
            return super().__new__(cls, elem)
        else:
            return cls._make_subclass(cls, elem, requires_grad)

    @classmethod
    def __torch_dispatch__(cls, func, _types, args=(), kwargs=None):
        print(func)
        for arg in args:
            if isinstance(arg, cls): continue
            print(arg, type(arg))


ElementwiseMulScalarIntModule()(TrivTensor(torch.randint(10, (3, 4))))

> aten.mul.Tensor
> 4 <class 'int'>
```

might be related to https://github.com/pytorch/pytorch/issues/77223

cc @Chillee @ezyang @zou3519 @albanD @samdow @silvasean @cathyzhyi @powderluv 

### Versions

PyTorch version: 1.12.0.dev20220511+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: 10.0.0-4ubuntu1 
CMake version: version 3.21.2
Libc version: glibc-2.31

Python version: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.13.0-40-generic-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: Could not collect
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3080
Nvidia driver version: 510.39.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.12.0.dev20220511+cpu
[pip3] torchvision==0.13.0.dev20220511+cpu
[conda] No relevant packages
DISABLED test_DistributedDataParallel (__main__.TestDistBackendWithSpawn)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](http://torch-ci.com/failure/test_DistributedDataParallel%2C%20TestDistBackendWithSpawn) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/6396724783).

Over the past 3 hours, it has been determined flaky in 1 workflow(s) with 1 red and 4 green.

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
CrossEntropyLoss computes SoftMax always across the second dimension
Labels: module: nn, module: loss, triaged, actionable

### 🚀 The feature, motivation and pitch

When working with recurrent neural networks, such as `torch.nn.RNN`,  the shape of the output (the logits) of that network is `NxLxD` where `N` is the batch size, `L` is the sequence length and `D` is the feature dimension. When using one-hot encodings for our targets in the same dimensions, e.g., for text processing, we want to use categorical cross entropy-loss on top of softmax activation (i.e. `torch.nn.CrossEntropyLoss`). 

According to the documentation, `torch.nn.CrossEntropyLoss` performs sodtmax **always** on the second dimension of the given logits. In our case, however, we need to compute softmax across the **third** dimension. Currently, it is required to `permute(0,2,1)` both the logits and the targets to artificially turn the third dimension second.

The requested feature would be a `dim` parameter in the constructor of the `torch.nn.CrossEntropyLoss` that enables to select the dimension over which softmax is computed. Similarly, for `torch.nn.functional.cross_entropy`, a `dim` parameter would be required.

### Alternatives

I guess, a quick-and-dirty implementation would simply apply the `permute` internally before passing the data to the C implementation.

### Additional context

_No response_

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345
The codegen unconditionaly generate code even when it is not going to be used
Labels: triaged, module: codegen

The codegen today generate cpp and header for all the dispatch keys listed in https://github.com/pytorch/pytorch/blob/2083b16f68963e563c6fbf6bf1084e78a7f66139/torchgen/model.py#L144-L163

We should be able to skip all the cuda-related ones when doing a CPU-only build and skip the mkldnn ones when it is not available.
This would speed up codegen for such builds.

cc @ezyang @bhosmer @bdhirsh
libtorch1.8 torch::sigmoid is wrong
Labels: module: cpp, triaged, module: jetson

### 🐛 Describe the bug

when I use libtorch1.8 for NVIDIA Jetson AGX,  I find the problem that torch::sigmoid() function is wrong. When I use libtorch1.7, the result is currect!

```
#include <torch/torch.h>
#include<torch/csrc/autograd/function.h>
#include <iostream>
using namespace std;

int main(){
    // at::Tensor input = torch::tensor(5.5);
    at::Tensor input = torch::ones({1, 2, 152, 152});
    at::Tensor output = input.sigmoid_();
    cout << output.sum() << endl;
    return 0;
}
```
the result:

libtorch1.8
```
27271.1
[ CPUFloatType{} ]
```

libtorch1.7
```
33780.8
[ CPUFloatType{} ]
```

### Versions

libtorch1.8 for NVIDIA Jetson AGX

cc @jbschlosser @ptrblck @puririshi98
`tensordot` does check the dtype of empty tensor
Labels: triaged, module: linear algebra

### 🐛 Describe the bug

`tensordot` does check the dtype of empty tensor

```python
import torch

a = torch.rand([3, 2, 1], dtype=torch.float64)
b = torch.rand([1, 2, 0], dtype=torch.complex128)
torch.tensordot(a, b)
# tensor([], size=(3, 0), dtype=torch.float64)
```

By contrast,
```python
import torch

a = torch.rand([3, 2, 1], dtype=torch.float64)
b = torch.rand([1, 2, 1], dtype=torch.complex128)
torch.tensordot(a, b)
# RuntimeError: expected scalar type Double but found ComplexDouble
```

Interestingly, when the tensor is 2d, it will raise the error even `b` is empty

```python
import torch

a = torch.rand([2, 1], dtype=torch.float64)
b = torch.rand([1, 0], dtype=torch.complex128)
torch.tensordot(a, b)
# RuntimeError: expected scalar type Double but found ComplexDouble
```

### Versions

pytorch: 1.11.0

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
Write decomposition conditionals in a way that leads to simpler shape expressions
Labels: triaged, module: primTorch

### 🚀 The feature, motivation and pitch

Take code like
```
assert x.numel() != 0
```

vs. 
```
assert all([i > 0 for i in x.shape])
```

Both of these are mathematically equivalent. But, the second one will be more friendly to a dynamic shapes tracing model that guards on 0/1. It doesn't lead to less recompiles, but it could lead to simpler guard expressions.

cc: @jansel @ezyang 

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @mruberry @ngimel
`torch.scatter_add` will succeed when the `index` is a complex tensor
Labels: triaged, module: scatter & gather ops

### 🐛 Describe the bug

`torch.scatter_add` will succeed when the `index` is a complex tensor without any elements.

```python
import torch

input_tensor = torch.rand([10, 5], dtype=torch.float64)
index_tensor = torch.rand([13, 0], dtype=torch.complex128)
src_tensor = torch.rand([10, 2], dtype=torch.float64)

def fn(input, index, src):
    dim = -1
    fn_res = torch.scatter_add(input, dim, index, src, )
    return fn_res

fn(input_tensor, index_tensor, src_tensor)
# tensor([[0.9610, 0.2629, 0.8555, 0.7965, 0.3472],
#         [0.7140, 0.6187, 0.4872, 0.3589, 0.7170],
#         [0.3184, 0.3303, 0.8061, 0.6865, 0.5176],
#         [0.6451, 0.1152, 0.4974, 0.0535, 0.0350],
#         [0.1497, 0.7439, 0.7563, 0.8654, 0.6401],
#         [0.1090, 0.9057, 0.2156, 0.3272, 0.6849],
#         [0.8402, 0.4956, 0.4937, 0.9882, 0.1275],
#         [0.0889, 0.8429, 0.3421, 0.1373, 0.1697],
#         [0.1318, 0.0984, 0.1662, 0.4122, 0.1132],
#         [0.9094, 0.2276, 0.8924, 0.3781, 0.7588]], dtype=torch.float64)
```

However, when trying to compute the gradient

```python
import torch
from torch.autograd import gradcheck

input_tensor = torch.rand([10, 5], dtype=torch.float64, requires_grad=True)
index_tensor = torch.rand([13, 0], dtype=torch.complex128, requires_grad=True)
src_tensor = torch.rand([10, 2], dtype=torch.float64, requires_grad=True)

def fn(input, index, src):
    dim = -1
    fn_res = torch.scatter_add(input, dim, index, src, )
    return fn_res

gradcheck(fn, (input_tensor, index_tensor, src_tensor))
# RuntimeError: Function ScatterAddBackward0 returned an invalid gradient at index 1 - got [13, 0] but expected shape compatible with [10, 2]
```

### Versions

pytorch: 1.11.0

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
fast `gradcheck` fails when outputs that do not require grad precede outputs that do
Labels: module: autograd, triaged, module: linear algebra, actionable

### 🐛 Describe the bug

`gradcheck` fails for `torch.linalg.slogdet` when `fast_mode=True`

```python
import torch
from torch.autograd import gradcheck

A = torch.rand([1, 1], dtype=torch.float64, requires_grad=True)
gradcheck(torch.linalg.slogdet, (A), fast_mode=True)
```

### Versions

pytorch: 1.11.0

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @jianyuh @mruberry @walterddr @IvanYashchuk @xwang233
torch.ops.aten.ceil(1.5) returns Tensor rather than scalar
Labels: triaged, module: type promotion

### 🐛 Describe the bug

```
torch.ops.aten.ceil(1.5) 
```
gives `tensor(2., dtype=torch.float64)` 

However, the result is expected to be scalar type since the input is scalar type.

Moreover, when using this operator in torchscript
```
import torch

@torch.jit.script
def forward(self, lhs, rhs):
  return torch.ops.aten.ceil(float(lhs-rhs))

print(forward.graph)
```

The torchscript IR shows
```
graph(%self : Tensor,
      %lhs.1 : Tensor,
      %rhs.1 : Tensor):
  %5 : int = prim::Constant[value=1]()
  %6 : Tensor = aten::sub(%lhs.1, %rhs.1, %5) # /usr/local/google/home/cathyzhyi/items/test/test.py:5:35
  %8 : float = aten::Float(%6) # /usr/local/google/home/cathyzhyi/items/test/test.py:5:29
  %9 : int = aten::ceil(%8) # /usr/local/google/home/cathyzhyi/items/test/test.py:5:9
  return (%9)
```

It seems the output is supposed to be an int scalar not the Tensor dtype `torch.float64`

### Versions

Collecting environment information...
PyTorch version: 1.12.0.dev20220510+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux rodete (x86_64)
GCC version: (Debian 11.2.0-16+build1) 11.2.0
Clang version: 13.0.1-3+build2
CMake version: version 3.22.1
Libc version: glibc-2.33

Python version: 3.9.7 (default, Sep 16 2021, 13:09:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.15.15-1rodete2-amd64-x86_64-with-glibc2.33
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.22.3
[pip3] torch==1.12.0.dev20220510+cpu
[pip3] torchvision==0.13.0.dev20220510+cpu
[conda] numpy                     1.22.3                   pypi_0    pypi
[conda] torch                     1.12.0.dev20220510+cpu          pypi_0    pypi
[conda] torchvision               0.13.0.dev20220510+cpu          pypi_0    pypi

cc @nairbv @mruberry
[primTorch] Reduction references don't return views consistent with their original operators
Labels: triaged, module: primTorch

This occurs because the reduction references model the keepdim arg as a broadcast following the reduction. This makes a lot of sense but causes view inconsistency. For example:

```
a = torch.randn(2, 2)
result = torch.amin(a, keepdim=True, dim=0)
print(result._is_view())
: False

result = refs.amin(a, keepdim=True, dim=0)
print(result._is_view())
: True
```

We should think about a way to resolve this -- making the operations consistent is one option, but maybe primTorch can have a different view behavior in this case. 

cc @ezyang @mruberry @ngimel
[RFC] Allow device override during Tensor unpickling without torch.load
Labels: module: pickle, module: serialization, triaged, enhancement

### 🚀 The feature, motivation and pitch

Enable the behavior of `map_location` argument of `torch.load` when unpickling a tensor directly.

This is a recurring problem with torch.distributed and its reliance on pickle. Currently object collectives(1) that send around tensors produce undefined behavior as tensor can potentially surface on the wrong device or on an invalid device.

To address this we should offer a context manager that overrides the device tensors are placed at:
```python
data: BytesIO = ....
pickle.dump(torch.rand(1, device="cuda:0", data))

data.seek(0)

with pickle_device_override("cuda:1"):
   t = pickle.load(data)
   assert str(x.device) =="cuda:1"
```

(1) https://pytorch.org/docs/stable/distributed.html#torch.distributed.broadcast_object_list

### Alternatives

Tensors could use python 3.8 support for OOB buffers: https://docs.python.org/3/library/pickle.html#out-of-band-buffers

This is a much more desirable approach as users that need special care for tensor location during pickling and unpickling could do it with custom Picklers without PT having to introduce a new special purpose hook.

One nice property of this alternative is that it would allow object collectives / rpc to send tensors around without copying them to the CPU first.

### Additional context

This is a known issue for PTD: https://github.com/pytorch/pytorch/issues/76865

cc @mruberry
Don't populate f_locals to check guards
Labels: triaged, enhancement, oncall: pt2, module: dynamo

Currently, TorchDynamo calls [PyFrame_FastToLocalsWithError](https://github.com/pytorch/torchdynamo/blob/7aeb1d4516a1b792cfb04807f6ea9faa14461147/torchdynamo/_eval_frame.c#L323) ([docs](https://bugs.python.org/file32537/c_api_frame.patch)).

Internally in cpython, this converts between the "frame fast variables" (a flat C array of PyObject*) and `frame->f_locals` (a python dictionary).

`frame->f_locals` starts as NULL, and is only populated by this function.

This works, but is higher overhead than it needs to be.

We should rewrite the [generated guards](https://github.com/pytorch/torchdynamo/blob/7aeb1d4516a1b792cfb04807f6ea9faa14461147/torchdynamo/guards.py#L350) to use the "fast variables" data structure directly.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
Improve the overall design of MPSGraphCache
Labels: triaged, enhancement, module: backend, module: mps

### 🐛 Describe the bug

Currently the MPSGraphCache uses string keys to retrieve the Graphs which can be error-prone. Instead, storing the string in the Graph can alleviate some of these issues and takes care of situations where collisions happen. This issue tracks to improve the design of MPSGraphCache. 

```
struct MPSGraphCache
{
  typedef MPSCachedGraph * (^CreateCachedGraphBlock)();

  struct CacheEntry {
    CacheEntry(std::string key, MPSCachedGraph *cachedGraph) : cachedGraph_(cachedGraph), key_(key) {}
    MPSCachedGraph* cachedGraph_ = nullptr;
    std::string key_ = nullptr;
  };

 public:

  static MPSGraphCache* getInstance() {
    if(_instance_cache == nullptr) {
      _instance_cache = new MPSGraphCache();
    }
    return _instance_cache;
  }

....
```


### Versions

Python version: 3.8.11 (default, Aug  6 2021, 08:56:27)  [Clang 10.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] mypy==0.812
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.20.0
[pip3] torch==1.12.0a0+gitcfd4b64
[conda] mkl                       2021.4.0           hecd8cb5_637
[conda] mkl-include               2022.0.0           hecd8cb5_105
[conda] mkl-service               2.4.0            py38h9ed2024_0
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] nomkl                     3.0                           0
[conda] numpy                     1.21.2           py38h0fa1045_0
[conda] numpy-base                1.21.2           py38hbbe2e76_0
[conda] torch                     1.12.0a0+git64704d5          pypi_0    pypi


cc @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Allow users to express fused matmul/bias/relu
Labels: triaged, enhancement, needs research, module: python frontend

### 🚀 The feature, motivation and pitch

We should let users have the benefit of `_addmm_activation` (fused matmul/bias/relu; also matmul/bias/gelu if/when that is fixed in cublasLt) from #74490 in their models. Some ideas on how we might do this:

- add a LinearReLU module that calls addmm + relu in training and just _addmm_activation in inference
- add a backward & documentation for _addmm_activation and make it public
- both?
- something else?

CC @ngimel 

### Alternatives

- Do nothing and rely on nvfuser; doesn't help eager mode
- Do nothing and let users implement their own solution if they want it

### Additional context

_No response_
Move the MPSGuardImpl to inherit from NoOpDeviceGuardImpl
Labels: triaged, module: backend, module: mps

### 🐛 Describe the bug

Currently the MPSGuardImpl is inheriting from:

```
struct TORCH_API MPSGuardImpl final : public c10::impl::DeviceGuardImplInterface {
  static constexpr DeviceType static_type = DeviceType::MPS;
```

This could be moved to NoOpDeviceGuardImpl. This issue is tracking that cleanup.

### Versions

```
Python version: 3.8.11 (default, Aug  6 2021, 08:56:27)  [Clang 10.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] mypy==0.812
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.20.0
[pip3] torch==1.12.0a0+gitcfd4b64
[conda] mkl                       2021.4.0           hecd8cb5_637
[conda] mkl-include               2022.0.0           hecd8cb5_105
[conda] mkl-service               2.4.0            py38h9ed2024_0
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] nomkl                     3.0                           0
[conda] numpy                     1.21.2           py38h0fa1045_0
[conda] numpy-base                1.21.2           py38hbbe2e76_0
[conda] torch                     1.12.0a0+git64704d5          pypi_0    pypi
```
torch.cholesky has been deprecated in favour of torch.linalg.cholesky. However, torch.cholesky_inverse remains as is. It should also be moved to torch.linalg
Labels: triaged, module: linear algebra

### 🐛 Describe the bug

Not sure where this should be filed under. But I noticed that, in the most recent version of pytorch `1.11.0+cu102`, when using `torch.cholesky`, it says it's deprecated in favour of `torch.linalg.cholesky`, and will getting phased out in future versions. However, the complementary function `torch.cholesky_inverse` still runs with no deprecation warning. Indeed, there's currently no `torch.linalg.cholesky_inverse`, nor does it seem like there are any such plans. If you saw it fit to move `cholesky` to `torch.linalg`, then it seems only right that `cholesky_inverse` should join it there.

### Versions

Running PyTorch 1.11.0+cu102 on WSL.

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
Automate cleanup of header includes
Labels: module: internals, triaged

Many files currently include unnecessary headers, or include non-specific headers like `ATen.h` or `Functions.h` instead of the specific per-operator header e.g. `ATen/ops/sum.h`. It would be nice to have tools which can clean these up automatically. A fully automatic solution would also greatly simplify syncing header changes with meta-internal.

Currently there are configs for running `include-what-you-use` in `tools/iwyu` but this still requires a lot of manual tweaks,
- sometimes need to manually remove system-specific headers like `<bits/foo.h>` instead of `<foo>` for C++ stdlib
- it isn't aware of build configurations e.g.  `ATen/ops` includes need to be guarded with `#ifdef AT_PER_OPERATOR_HEADERS`
- it can't be run on files in `ATen/native/cpu/` since these files aren't built by the build system directly, so it can't determine build flags

cc @ezyang @bhosmer @smessmer @ljk53 @bdhirsh @albanD
FSDP: ability to ignore parameters
Labels: high priority, triage review, oncall: distributed, module: fsdp

### 🚀 The feature, motivation and pitch

https://github.com/pytorch/pytorch/issues/75255 implemented the ability to ignore FSDP parameters at the module level, i.e. by passing in an `ignore_module` list to the FSDP API.

Although, we've also seen the use case where users don't want to shard particular parameters of a module, but want to ensure the rest of the module still has parameters sharded. To support use cases such as these, we'd like to be able to ignore parameters specifically and not have them coupled to be all the parameters of a module. 

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @gchanan @zou3519 @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
Add Tensor compare support for MPS backend
Labels: triaged, module: backend, module: testing, module: mps

### 🐛 Describe the bug

Currently the MPS backend doesn't support all the operations needed for assertEqual. So currently the tensors on `mps` are mapped to cpu and the Equal operation is performed on CPU. This issue tracks enabling all the necessary operations for MPS backend and not-falling back to CPU. This issue was currently identified [here.](https://github.com/pytorch/pytorch/pull/76725/files#r866238060)


### Versions

Python version: 3.8.11 (default, Aug  6 2021, 08:56:27)  [Clang 10.0.0 ] (64-bit runtime)
Python platform: macOS-10.16-x86_64-i386-64bit
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] mypy==0.812
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.20.0
[pip3] torch==1.12.0a0+gitcfd4b64
[conda] mkl                       2021.4.0           hecd8cb5_637
[conda] mkl-include               2022.0.0           hecd8cb5_105
[conda] mkl-service               2.4.0            py38h9ed2024_0
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] nomkl                     3.0                           0
[conda] numpy                     1.21.2           py38h0fa1045_0
[conda] numpy-base                1.21.2           py38hbbe2e76_0
[conda] torch                     1.12.0a0+gitcfd4b64          pypi_0    pypi
[FSDP] `ignored_modules` follow-ups
Labels: oncall: distributed, module: fsdp

This issue is to track a few follow-ups regarding `ignored_modules`.

1. Users may want to ignore specific parameters or buffers within a module. How should we modify the API to accommodate this?
2. What should happen if a user passes a module into `ignored_modules` that contains a submodule that is an `FSDP` instance?
    - The current implementation ignores it without warning, but alternatives are to issue a warning or to error.
3. How should `ignored_modules` interact with shared parameters and buffers? For example, what should happen if a parameter is in a module in `ignored_modules` but also in a module that is not in `ignored_modules`? What if that latter module is an `FSDP` instance and already flattened its parameters? 

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
__name__ on OpOverload should not contain period
Labels: triaged, module: __torch_dispatch__

### 🐛 Describe the bug

Conventionally, `__name__` refers to the name of the Python object without any namespacing. However, for OpOverloads as retrieved from `torch.ops.aten.abs.default` these have the `__name__` `abs.default` which violates this

cc @Chillee @ezyang @zou3519 @albanD @samdow @anjali411 

### Versions

master
broadcast_object_list with GPU tensors can lead to deadlock on PyTorch CI machines
Labels: high priority, triage review, oncall: distributed, module: fsdp

### 🐛 Describe the bug

See for example: https://github.com/pytorch/pytorch/runs/6357736633?check_suite_focus=true

Here, we are using `broadcast_object_list` on a dict with GPU tensors, and running into deadlocks which cause the process to hang and eventually terminate. This issue is not reproducible on V100 / A100 HW architecture and only appears to manifest itself on M60 PyTorch CI machines. https://github.com/pytorch/pytorch/issues/76865 runs into a similar deadlock.

### Versions

main

cc @ezyang @gchanan @zou3519 @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang
Unable to continue adding modules to `nn.Sequential` after using `del` method
Labels: module: nn, triaged, enhancement

### 🐛 Describe the bug

When the module deleted by the `del` method is not the last module of the `torch.nn.Sequential` object, the index will be discontinuous and the module cannot be correctly added.

```python
from torch import nn

net = nn.Sequential(
    nn.Linear(20, 10),
    nn.ReLU(),
    nn.Linear(10, 5)
)

del net[1]
print(net)
```
Output:
```python
Sequential(
  (0): Linear(in_features=20, out_features=10, bias=True)
  (2): Linear(in_features=10, out_features=5, bias=True)
)
```
It can be seen that the index is no longer consecutive. Continue to add modules:
```python
net.append(nn.ReLU())
net.append(nn.Sigmoid())
print(net)
```
Output:
```python
Sequential(
  (0): Linear(in_features=20, out_features=10, bias=True)
  (2): Sigmoid()
)
```
This does not match the expected result.

---

If the last module is removed, other modules can be added correctly later:
```python
from torch import nn

net = nn.Sequential(
    nn.Linear(20, 10),
    nn.ReLU(),
    nn.Linear(10, 5)
)

del net[2]
net.append(nn.ReLU())
net.append(nn.Sigmoid())
print(net)
```
Output:
```python
Sequential(
  (0): Linear(in_features=20, out_features=10, bias=True)
  (1): ReLU()
  (2): ReLU()
  (3): Sigmoid()
)
```



### Versions

Pytorch Version: 1.11.0

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345