gopikrsmscs/torch-issues huggingface dataset:
[Quant] Reference get_default_qconfig_mapping in docs/tutorials
Labels: oncall: quantization, triaged

Right now the recommended flow for FX graph mode quantization is:
```
from torch.ao.quantization import get_default_qconfig_mapping
from torch.ao.quantization.quantize_fx import prepare_fx, convert_fx
...
qconfig_mapping = get_default_qconfig_mapping("fbgemm")
model = prepare_fx(model, qconfig_mapping, example_inputs)
model(...) # calibrate
model = convert_fx(model)
```
However, `get_default_qconfig_mapping` is not documented or referenced (same for `get_default_qat_qconfig_mapping`). We should make sure this is easily accessible in the following places:
- https://pytorch.org/docs/master/quantization-support.html
- https://pytorch.org/docs/master/generated/torch.quantization.quantize_fx.prepare_fx.html#torch.quantization.quantize_fx.prepare_fx
- https://pytorch.org/tutorials/prototype/fx_graph_mode_ptq_static.html

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
Please include virtual/physical batch sizes in the tutorials
Labels: module: docs, triaged

### 📚 The doc issue

The tutorials in the website, e.g. the [CIFAR10 tutorial](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html) only refer to the batch size when initializing the dataloader. The batch size used for the dataloader is the same batch size used for the optimization step:

```
batch_size = 4

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
                                          shuffle=True, num_workers=2)
```

However, when using a single GPU, people may sometimes want to simulate rather large virtual batch sizes, i.e., compute the gradients of many small "physical" batches, which are feasible to compute on their GPU, but only take the optimization step when reaching the desired amount of samples in the "virtual" batch. It is not trivial, since it turns out that the [gradients of the small batches are summed](https://discuss.pytorch.org/t/how-are-losses-aggregated-over-multiple-computed-batches/160041). This means that the optimization step size taken using the final gradient will be K times larger than the user-specified learning rate, which may be undesirable in most cases. The losses of the small batches should therefore be multiplied by their relative size, so the final gradient will exactly simulate a large batch.



### Suggest a potential alternative/fix

### I'd change the code as follows:

## Dataloader initialization
```
optimization_batch_size = 16
physical_batch_size = 4

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=optimization_batch_size,
                                          shuffle=True, num_workers=2)
```

## Training
```
running_loss = 0.0
for i, data in enumerate(trainloader, 0):
    # get the inputs; data is a list of [inputs, labels]
    inputs, labels = data
    
    # zero the parameter gradients
    optimizer.zero_grad()

    # run batch
    batch_loss = 0.0
    # compute all physical batches
    for j in range(0, len(inputs), physical_batch_size):
        end_index = min(j + physical_batch_size, len(inputs))
        inputs_j = inputs[j:end_index]
        labels_j = labels[j:end_index]
        # forward + backward
        outputs_j = net(inputs_j)
        loss = criterion(outputs_j, labels_j) * len(inputs_j) / float(len(inputs))
        batch_loss += loss.item()
        loss.backward()
    # optimize whole batch
    optimizer.step()

    # print statistics
    running_loss += batch_loss
    if i % 2000 == 1999:    # print every 2000 mini-batches
        print(f'[{epoch + 1}, {i + 1:5d}] loss: {running_loss / 2000:.3f}')
        running_loss = 0.0
```


cc @svekars @holly1238
MPS convolution is sometimes returning NaNs for valid inputs.
Labels: triaged, module: mps

Splitting https://github.com/pytorch/pytorch/issues/81185 into two. This one focusses on the MPS side of the problem with the following repro:
```python
# bug_demo.py

import torch

n_trials = 100
for ii in range(n_trials):
    a = torch.randn(1024, device='mps')
    b = torch.randn(499, device='mps')
    c = torch.nn.functional.conv1d(a.view(1, 1, -1), b.view(1, 1, -1))
    if torch.isnan(torch.sum(c)):
        print(f'mps: trial {ii}, nan elements {torch.isnan(c.squeeze()).nonzero().view(-1).cpu().numpy()}')
```

cc @kulinseth
Move self.subtest calls in FSDP test suite to run_subtests utility
Labels: oncall: distributed, triaged, better-engineering, module: fsdp

### 🚀 The feature, motivation and pitch

As per the discussion in https://github.com/pytorch/pytorch/pull/83195#discussion_r952931023, we can use the `self.run_subtests` functionality which will require some refactoring of test code. 

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @ezyang
Better error message for qlinear_prepack
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

See https://discuss.pytorch.org/t/index-out-of-bounds-error-with-perchannel-quantization/159937
linear_prepack only works for per channel quantized weight with ch_axis=0, we should have a better error message.

### Versions

master

cc @jianyuh @raghuramank100 @jamesr66a @vkuzo
Expose API for Registering Post-Gradient-Computation Hook
Labels: module: autograd, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

To preface, this is not urgent or blocking.

Context: For FSDP to reduce-scatter gradients, it wants to register a hook that fires _after_ the gradient is computed. Currently, this is achieved by registering a hook on the `AccumulateGrad` object. This issue relates to how to access that `AccumulateGrad` object.

FSDP currently uses an `expand_as()` trick to bypass the fact that the `FlatParameter` (a leaf tensor) does not have a `grad_fn`. In the following code, `p_tmp` will have a `grad_fn` of `ExpandBackward0`, which then gives access to the `AccumulateGrad` object via `next_functions[0][0]`.
https://github.com/pytorch/pytorch/blob/00cb184512f3a636d87793f46d3f9c7fea406b25/torch/distributed/fsdp/fully_sharded_data_parallel.py#L2825-L2835

In my `IndirectParameter` prototype for FSDP, I changed `FlatParameter: nn.Parameter` to `FlatTensor: Tensor` since we no longer want to expose the `FlatParameter` to `nn.Module` methods like `named_parameters()`. In doing this, I still need to use the `expand_as()` trick since the `FlatTensor` is a leaf tensor. However, I find that the resulting tensor's `grad_fn` chain is now `AliasBackward0` -> `ExpandBackward0` -> `AccumulateGrad`; i.e. there is an additional `AliasBackward0`. I am not sure if this is expected. Either way, this provides some motivation for an API to register a post-gradient-computation hook (i.e. a hook that runs _after_ the gradient is computed) since this `expand_as()` trick is arguably hacky.




### Alternatives

The alternative is to continue using the `expand_as()` trick. When registering the post-backward hook for FSDP, we will need some logic to handle both the `FlatParameter` and `FlatTensor` case since we will need to preserve the existing code path in the near term. This is not a real problem.

### Additional context

In case this is relevant to the appearance of the `AliasBackward0`, the `FlatTensor`, which is just a `torch.Tensor`, is initialized like:
```
class FlatTensor(torch.Tensor):
    ...

@staticmethod
def flatten_params(
    params: List[torch.Tensor],
    requires_grad: bool,
) -> FlatTensor:
    with torch.no_grad():
        flat_tensors = [
            p.detach().reshape(-1) if isinstance(p, nn.Parameter) else p.reshape(-1)
            for p in params
        ]
        flat_tensor_data = torch.cat(flat_tensors, dim=0)
    flat_tensor = FlatTensor(flat_tensor_data)
    flat_tensor.requires_grad_(requires_grad)
    return flat_tensor
```

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @zhaojuanmao @mrshenli @rohan-varma
Large number of WONT CONVERTs on detectron2 model
Labels: triaged, oncall: pt2

Repro: n2409433.

Dynamo is really choking on this model—there's many WONT CONVERTS and as a result we only get 17 ops out of the whole thing. I pulled out pytorch/torchdynamo#1010 and pytorch/torchdynamo#1009 into self-contained repros, but the rest of them probably need someone to look at the model directly.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
fill_ OpInfo code not used, also, doesn't test the case where the second argument is a Tensor
Labels: module: tests, triaged

Two observations:
1. `sample_inputs_fill_` is no longer used. Can be deleted (https://github.com/pytorch/pytorch/blob/master/torch/testing/_internal/common_methods_invocations.py#L1798-L1807)
2. The new OpInfo for fill doesn't actually test the `tensor.fill_(other_tensor)` case. Previously we did test this, as shown by `sample_inputs_fill_`

cc @mruberry
[Nested Tensor] Enable Nestedtensor to work with OpInfos
Labels: triaged, module: nestedtensor

### 🐛 Describe the bug

We currently use [test/test_nestedtensor.py](https://github.com/pytorch/pytorch/blob/master/test/test_nestedtensor.py) to test nested tensor functions. Ideally, we would like to integrate this with OpInfos as has been done for sparse layouts, etc.

Creating this issue to track progress on this.

### Versions

n/a

cc @cpuhrsch @jbschlosser @bhosmer @drisspg
Linux cuda-11.x binary build  jobs intermittently take more than 4 hours
Labels: high priority, oncall: releng, module: ci, triaged

### 🐛 Describe the bug

 When we do:
```
conda install pytorch cudatoolkit=11.3 -c pytorch-nightly
```
Conda installs the CPU version:
```
pytorch            pytorch-nightly/linux-64::pytorch-1.13.0.dev20220824-py3.9_cpu_0
```

The Linux jobs upload have failed:
<img width="1058" alt="Screen Shot 2022-08-24 at 9 15 41 AM" src="https://user-images.githubusercontent.com/7563158/186431127-c16e5dab-d1dc-4e33-a32c-cbaf6bbc5bb7.png">

Github upload failed:
https://github.com/pytorch/pytorch/runs/7989185420?check_suite_focus=true

```
Uploaded /home/ec2-user/actions-runner/_work/_temp/artifacts/pytorch-1.13.0.dev20220824-py3.9_cuda11.3_cudnn8.3.2_0.tar.bz2 (90.2%) bytes 1191182336:1199570943
Total file count: 1 ---- Processed file #0 (0.0%)
Error: The operation was canceled.
```

Root cause timeout more then 4hrs:

<img width="396" alt="Screen Shot 2022-08-24 at 9 00 59 PM" src="https://user-images.githubusercontent.com/7563158/186550393-e49cb402-39dd-4d8a-8b54-1e9931e2cd3f.png">



### Versions

cuda 11.3
cuda 11.6

cc @ezyang @gchanan @zou3519 @seemethere @malfet @pytorch/pytorch-dev-infra
General NestedTensor op coverage tracking issue
Labels: feature, triaged, module: nestedtensor

### This issue is to have a centralized place to list and track work on adding support to new ops for the NestedTensor backend.
There are  large number of operators in pytorch and so they are not all implemented yet for the NestedTensor backends as it is still in the prototype phase. We will be prioritizing adding new operators based on user feedback. If possible, please also provide link to the network or use-case where this op is getting used.

If you want to work on adding support for such op, feel free to comment below to get assigned one. Please avoid picking up an op that is already being worked on or that already has a PR associated with it.

#### Op coverage requests
- [ ] aten::convolution
- [ ] aten::max.dim
- [x] detach #84078 
- [x] to #87146
- [ ] eq
- [ ] masked_select
- [ ] index_select
- [ ] narrow
- [ ] alias
- [ ] Broadcasting Ufuncs along implicit NT dim
- [ ] Zero-copy Nt construction from Size info
- [ ] BCE/ Other loss functions for NT
- [ ] nested tensor creation from arbitrary masks rather than left-aligned

#### Backward op coverage requests
- [ ] gelu, relu backward
- [ ] layernorm backward

cc @cpuhrsch @jbschlosser @bhosmer
PyTorch EC2 runners can not be used with standard actions
Labels: module: ci, triaged

### 🐛 Describe the bug

I.e. even setup-python action fails, see
https://github.com/pytorch/torchdynamo/runs/7996989843?check_suite_focus=true

for following simple workflow file:
```
 test-inductor:
    runs-on: linux.4xlarge.nvidia.gpu
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-python@v4
      with:
        python-version: '3.8.13'
        cache: 'pip'
```

### Versions

CI

cc @seemethere @pytorch/pytorch-dev-infra
Undefined reference in libtorch_cpu.so `...std::__cxx11::basic_string...`
Labels: module: build, triaged

### 🐛 Describe the bug

When I try to build app that should link with libtorch I get following (output of make):
```
g++ -c -pipe -D_GLIBCXX_USE_CXX11_ABI=1 -O2 -std=gnu++1y -Wall -W -fPIC  -I. -o main.o ../main.cpp
g++ -Wl,-O1 -o test main.o -lc10 -ltorch_cpu 
//usr/local/lib/libtorch_cpu.so: undefined reference to `std::allocator<std::pair<long, std::tuple<torch::jit::SourceRange, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, c10::intrusive_ptr<torch::jit::InlinedCallStack, c10::detail::intrusive_target_default_null_type<torch::jit::InlinedCallStack> > > > >::allocator()'

```

libtorch (1.11.0) has been builded successfully with commands:

```
wget https://github.com/pytorch/pytorch/releases/download/v1.11.0/pytorch-v1.11.0.tar.gz && \
tar -xvzf pytorch-v1.11.0.tar.gz && rm pytorch-v1.11.0.tar.gz && \
cd pytorch-v1.11.0 && mkdir build && cd build && \
cmake -D CMAKE_BUILD_TYPE=Release \
      -D USE_AVX=OFF \
      -D USE_NNPACK=OFF \
      -D USE_MKLDNN=OFF \
      -D BUILD_PYTHON=OFF \
      .. && \
make -j1 && \
sudo make install && \
sudo ldconfig
```

what is the reason? 

I already try:
  - build app with -D_GLIBCXX_USE_CXX11_ABI=0 - another errors appear
  - use Ubuntu18.04 - success, all works, but I can not understand why on RED OS (my target) it is not working...

### Versions

Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: RED OS (7.2)  (x86_64)
GCC version: (GCC) 7.2.1 20180116 (Red Hat 7.2.1-7)
Clang version: Could not collect
CMake version: version 3.9.0
Libc version: glibc-2.3.4

Python version: 3.6.13 (default, Apr  5 2022, 16:16:18)  [GCC 7.2.1 20180116 (Red Hat 7.2.1-7)] (64-bit runtime)
Python platform: Linux-4.19.204-2.el7.x86_64-x86_64-with-centos-7.2
Is CUDA available: N/A
CUDA runtime version: Could not collect
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] No relevant packages
[conda] Could not collect```
```

cc @malfet @seemethere
[FSDP] Make sharded / unsharded check more robust
Labels: oncall: distributed, triaged, module: fsdp

### 🚀 The feature, motivation and pitch

As per the comment in https://github.com/pytorch/pytorch/pull/83195#discussion_r952937179, we currently check if we are using the local shard for the param to check if a param is sharded or unsharded:

```
p.data.data_ptr() == p._local_shard.data_ptr():
```

This assumes that if we are using the local shard, we already correctly freed the full parameter. This assumption holds true today but if it were to break this would be a silent correctness issue. We should figure out a robust, future-proof approach to checking if we need to reshard a parameter or not.

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @ezyang
Are PyTorch Android nightly builds getting automatically published
Labels: module: ci, triaged, module: android

### 🐛 Describe the bug

https://github.com/pytorch/pytorch/pull/83930 kills logic for uploading nightly builds, because apparently it were never used. 
Lets track whether we need to publish nightly builds for Android.... And if  do not, let's close this issue


Our [documentation](https://pytorch.org/mobile/android/#using-the-nightly-pytorch-android-libraries) claims that Android nightly builds exist, but still references Python-1.8.0 and the last non-trivial update to this page were in Jun 2021: https://github.com/pytorch/pytorch.github.io/commits/site/_mobile/android.md
### Versions

CI

cc @seemethere @pytorch/pytorch-dev-infra
empty_quantized should probably be new_empty_quantized
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

the current empty_quantized takes a Tensor as input, so it matches the semantics of new_empty instead of empty (which does not take Tensor input), we should merge the empty_quantized implementation under new_empty instead

### Versions

master

cc @jianyuh @raghuramank100 @jamesr66a @vkuzo
Add torch nightly builds pipeline for aarch64 linux
Labels: module: ci, triaged, enhancement, module: arm

 There are no aarch64_linux nightly wheels here:
https://download.pytorch.org/whl/nightly/cpu/torch_nightly.html

However, I see the wheels are published for [PT1.12.1 release](https://pypi.org/project/torch/#files) on Aug 5th, so, the issue with the nightly builds might be with some infrastructure not the codebase.

cc @seemethere @malfet @pytorch/pytorch-dev-infra
primTorch: support refs and decompositions when ATen and Python disagree
Labels: triaged, module: primTorch

### 🐛 Describe the bug

1. Policy (related): We need to agree on how to handle refs/decomps like `binary_cross_entropy` where the ATen op and the Python frontend have different number/type of arguments. Right now, `binary_cross_entropy` just defines a decomp. Do we want to split the ref/decomp implementations for these (with a shared core/conversion logic) or handle in some other way?

2. Bug: if `register_decomposition` is used with ops like these to define _one_ Python function that's both registered as a ref and a decomp, there are no type signature checks in `register_decomposition` to catch this. So it'll just run and we might not notice unless it breaks for some other reason.

I have a demo branch with this issue here, see the commit message and notes in the topmost commit:
https://github.com/nkaretnikov/pytorch/commits/primtorch-l1-loss-decomp-ref-compat-issue

### Versions

master (e0f2eba93d2804d22cd53ea8c09a479ae546dc7f)

cc @ezyang @mruberry @ngimel
Support primtorch view ops in functionalization
Labels: triaged, module: viewing and reshaping, module: functionalization, module: primTorch

I was starting to look at failing inductor models with `functorch.config.use_functionalize=True` turned on, and one failure that I noticed is:
```
// run this
python benchmarks/timm_models.py --float32 -dcuda --no-skip --training --inductor --use-eval-mode --only=mobilevit_s

// output
    return forward_call(*input, **kwargs)
  File "<eval_with_key>.4", line 394, in forward
    broadcast_in_dim_default = torch.ops.prims.broadcast_in_dim.default(var_default, [256, 256, 1], [0, 1]);  var_default = None
  File "/scratch/hirsheybar/work/benchmark/pytorch/torch/_ops.py", line 60, in __call__
    return self._op(*args, **kwargs or {})
RuntimeError: !schema.hasAnyAliasInfo() INTERNAL ASSERT FAILED at "/scratch/hirsheybar/work/benchmark/pytorch/aten/src/ATen/FunctionalizeFallbackKernel.cpp":30, please report a bug to PyTorch. mutating and aliasing ops should all have codegen'd kernels
```

This error didn't show up in my AOT + eager tests, probably because the inductor backend is choosing to run extra primtorch decompositions. This causes functionalization to see `prims.broadcast_in_dim`, which is a new "view" op that it doesn't know how to handle.

For what it's worth - the code above fails with `use_functionalize=False` turned off today, too, because of some issue's in dynamo' `normalize_ir()` code. So this technically isn't a regression, although it seems like time would be better spent fixing the problem with functionalization instead of fixing dynamo's normalization logic.


Two potential ways to handle this are:
(1) write a custom functionalization kernel for `prims.broadcast_in_dims.default`. We'd need to expose the right C++ API's to python in order to do this
(2) Beef up the functionalization boxed fallback to handle views. This would be very useful, but it's a bit unclear how this would work: for every primtorch view, the boxed fallback needs to know how to map it to a "view inverse" function.

cc @ezyang @mruberry @ngimel
Tracking nested tensor functions with backward kernels registered in derivatives.yaml
Labels: triaged, module: nestedtensor

### 🐛 Describe the bug

Context:

Backward formulas for certain nested tensor functions do not work as they call `.sizes()`. As a workaround, we can register formulas specific to the AutogradNestedTensor dispatch key in derivatives.yaml. This should be removed when SymInts for nested tensor sizes are ready for use. This issue serves as a tracker for functions for which we have added this workaround so that we can remove them in the future.

- [ ] [_nested_sum_backward](https://github.com/pytorch/pytorch/pull/82625)
- [ ] [_select_backward](https://github.com/pytorch/pytorch/pull/83875)

### Versions

n/a

cc @cpuhrsch @jbschlosser @bhosmer @drisspg @albanD
Bug in batch names with matmul (result tensor has names=('i', 'i', 'k')).
Labels: triaged, module: named tensor

### 🐛 Describe the bug

The following code should fail, as it gives duplicate names in the output.  
```
A = t.ones((3,3), names=('i', 'j'))
B = t.ones((3,3,3), names=('i', 'j', 'k'))
print((A@B).names)
```
Instead, it gives duplicated names (which is not allowed e.g. in the named constructors):
```
('i', 'i', 'k')
```
I'm on PyTorch version 1.12.1 on CPU.

### Versions

Collecting environment information...
PyTorch version: 1.12.1
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: CentOS Linux release 7.9.2009 (Core) (x86_64)
GCC version: (GCC) 9.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.17

Python version: 3.9.12 (main, Apr  5 2022, 06:56:58)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-3.10.0-1160.45.1.el7.x86_64-x86_64-with-glibc2.17
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] numpydoc==1.2
[pip3] torch==1.12.1
[pip3] torchaudio==0.12.1+cu113
[pip3] torchvision==0.13.1+cu113
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.3.1               h2bc3f7f_2
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py39h7f8727e_0
[conda] mkl_fft                   1.3.1            py39hd3c417c_0
[conda] mkl_random                1.2.2            py39h51133e4_0
[conda] numpy                     1.21.5           py39he7a7128_1
[conda] numpy-base                1.21.5           py39hf524024_1
[conda] numpydoc                  1.2                pyhd3eb1b0_0
[conda] pytorch                   1.12.1          py3.9_cuda11.3_cudnn8.3.2_0    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchaudio                0.12.1+cu113             pypi_0    pypi
[conda] torchvision               0.13.1+cu113             pypi_0    pypi

cc @zou3519
pytorch 1.12.1 Adam Optimizer Malfunction!!!
Labels: needs reproduction, module: optimizer, triaged

If you have a question or would like help and support, please ask at our
[forums](https://discuss.pytorch.org/).

If you are submitting a feature request, please preface the title with [feature request].
If you are submitting a bug report, please fill in the following details.

## Issue description

Provide a short description.
In pytorch 1.12.1, Adam optimization doesn't work well.
I think, It seems that the internal behavior has changed as the version is upgraded, please check 

## Code example

Please try to provide a minimal example to repro the bug.
Error messages and stack traces are also helpful.

## System Info
Please copy and paste the output from our
[environment collection script](https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py)
(or fill out the checklist below manually).

You can get the script and run it with:
```
wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
```

- PyTorch 
- How you installed PyTorch Conda
- Build command you used (if compiling from source):
- OS: window10, Ubuntu 18.04
- PyTorch version: 1.12.1
- Python version: 3.8.0
- CUDA/cuDNN version: Cuda11.3
- GPU models and configuration: gtx1080ti, gtx3070
- GCC version (if compiling from source):
- CMake version:
- Versions of any other relevant libraries:


cc @vincentqb @jbschlosser @albanD
Improve FSDP error msg on wrong attr access
Labels: oncall: distributed, module: bootcamp, triaged, pt_distributed_rampup, module: fsdp

### 🚀 The feature, motivation and pitch

If FSDP does not have an attr, it will dispatch into the contained module to try to get the attribute. However if this fails, the error raised is confusing, since it prints out a msg containing info about the wrapped module not having the attribute, instead of the FSDP module. An example:

```
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/testing/_internal/common_distributed.py", line 622, in run_test
    getattr(self, test_name)()
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/testing/_internal/common_distributed.py", line 503, in wrapper
    fn()
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/testing/_internal/common_distributed.py", line 145, in wrapper
    return func(*args, **kwargs)
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/test/distributed/fsdp/test_fsdp_misc.py", line 150, in test_fsdp_not_all_outputs_used_in_loss
    loss.backward()
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/_tensor.py", line 484, in backward
    torch.autograd.backward(
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/autograd/__init__.py", line 191, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/distributed/fsdp/fully_sharded_data_parallel.py", line 2880, in _post_backward_hook
    if self._should_free_full_params():
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/distributed/fsdp/fully_sharded_data_parallel.py", line 3483, in _should_free_full_params
    self.sharding_stratagy == ShardingStrategy.FULL_SHARD
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/distributed/fsdp/fully_sharded_data_parallel.py", line 1536, in __getattr__
    return getattr(self._fsdp_wrapped_module, name)
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/distributed/fsdp/flatten_params_wrapper.py", line 146, in __getattr__
    return getattr(self.module, name)  # fall back to the wrapped module
  File "/fsx/users/rvarm1/rvarm1/repos/pytorch/torch/nn/modules/module.py", line 1260, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'Linear' object has no attribute 'sharding_stratagy'
```

In this case, we have a typo `sharding_stratagy` but it is harder to debug since the error `AttributeError: 'Linear' object has no attribute 'sharding_stratagy'` is misleading. We should improve this to include information about `FullyShardedDataParallel`. 

### Alternatives

_No response_

### Additional context

_No response_

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @SciPioneer @H-Huang @kwen2501 @ezyang
Pytorch/Nova CI should monitor service outages for major dependencies
Labels: module: ci, triaged, needs design

### 🐛 Describe the bug

Followup after https://github.com/pytorch/vision/issues/6466 
There should be a mechanism one can rely to tell whether some of the components CI depends on were experiencing outage at the time CI job were run.
This includes, but not limited:
 - https://www.githubstatus.com/
 - https://status.circleci.com/
 - https://developer.download.nvidia.com
 - https://download.pytorch.org
 - https://anaconda.org/ and its CDN
 - PyPI and its CDN

### Versions

CI

cc @seemethere @malfet @pytorch/pytorch-dev-infra
Explore TorchInductor optimization pass to reorder kernel bodies
Labels: triaged, oncall: pt2

See pytorch/torchdynamo#934 for more context.   We found an example of a 10% performance difference from very similar kernels, where the biggest difference seemed to be instruction ordering.

Interesting, my first guess looking at the two kernels is just the ordering of ops.  The loads on the faster kernel are "spread out" while the loads in the slow kernel are "bunched up".

Perhaps we should explore a compiler pass that reorders ops within a kernel.

Our current inductor kernels usually look like:
```
<all of the loads>
<all of the compute>
<all of the stores>
````

When you have indirect loads, it moves those indirect loads into the "compute" section, because they must come after the address computation.  Thus allowing that spread out pattern to be generated.

My thinking of doing that ordering was it makes compiler analysis easier for Triton/LLVM.  I may have been wrong there... especially if Triton doesn't yet have an instruction reordering pass.

This is just one theory though, we should test it.

_Originally posted by @jansel in https://github.com/pytorch/torchdynamo/issues/934#issuecomment-1221583598_

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
[feature request] Add new device type works on CPU
Labels: triaged, enhancement

### 🚀 The feature, motivation and pitch

When we write code on a CPU machine and it runs on a GPU machine, we sometimes forget to transfer tensor GPU to CPU or opposite.
Because we write `device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")` on head of code.
When we test it on a CPU machine, all tensors are put on the CPU and do not tell us whether the tensor will transfer as we expect or not.
So, how about adding a new device type that works on a CPU? (like a fake-gpu)

In my assumption, 

Now(no GPU machine)
```
device = torch.device("cuda:0" if torch.cuda.is_available else "cpu")
a = torch.arange(10).to(device)
a.numpy()  # actually need a.cpu() before numpy()
>>> No Error
```

New Feature(no GPU machine)
```
device = torch.device("cuda:0" if torch.cuda.is_available else "fake-gpu")
a = torch.arange(10).to(device)
a.numpy() # need a.cpu() before numpy()
>>> Error
```


### Alternatives

_No response_

### Additional context

_No response_
Error on installation
Labels: module: rocm, triaged

### 🐛 Describe the bug

Hello, after "python setup.py install" the script shows this error and the installation fails:

fatal error: error in backend: Cannot select: intrinsic %llvm.amdgcn.ds.bpermute
clang-14: error: clang frontend command failed with exit code 70 (use -v to see invocation)
AMD clang version 14.0.0 (https://github.com/RadeonOpenCompute/llvm-project roc-5.2.3 22324 d6c88e5a78066d5d7a1e8db6c5e3e9884c6ad10e)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm/hip/../llvm/bin
clang-14: note: diagnostic msg: Error generating preprocessed source(s).
CMake Error at torch_hip_generated_cub-RadixSortPairs.hip.o.cmake:200 (message):
  Error generating file
  /home/ferna/pytorch/build/caffe2/CMakeFiles/torch_hip.dir/__/aten/src/ATen/hip/./torch_hip_generated_cub-RadixSortPairs.hip.o

Something is missing? Thanks

### Versions

roc-5.2.3
python 3.9

cc @malfet @seemethere @jeffdaily @sunway513 @jithunnair-amd @ROCmSupport @KyleCZH
Inductor Error: aten.fill_.Tensor
Labels: triaged, oncall: pt2

`python /scratch/eellison/work/torchdynamo/benchmarks/microbenchmarks/operatorbench.py --op=aten.fill_.Tensor --dtype=float16 --suite=huggingface`

> CUDA error: operation failed due to a previous error during capture
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
[Nested Tensor] view + inplace for autograd. 
Labels: module: autograd, triaged, module: nestedtensor

## Summary
This is currently erroring:
```
import torch
a = torch.randn(1, 2, 4, requires_grad=True)
b = torch.randn(2, 2, 4, requires_grad=True)
c = torch.randn(3, 2, 4, requires_grad=True)
nt = torch.nested_tensor([a,b,c])
buffer = nt.values()
buffer.mul_(2)
```
This is
1. Creating a view nt -> buffer 
2. Applying an inplace op on buffer. 


This triggers rebase_history() which in turn creates copy_slices. Copy slices utilize a struct called TensorGeometry to track geometry information about the base and the view.  The view_fn will be used instead of as_strided. However we apply this function on result which is an empty of clone of base. We don't currently have a factory function that can do this. This will be unblocked when we add this.


cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @cpuhrsch @jbschlosser @bhosmer
HuggingFace Slow Operators
Labels: triaged, oncall: pt2

Operators with 50th percentile below 98% of aten perf or 20th percentile below 96%:

Float16:

```
aten._log_softmax.default:: [0.8083701248515153, 0.9260282672431682, 1.0251716763358925]
aten._log_softmax_backward_data.default:: [0.9195609454113238, 0.9607073111684497, 1.0512267863036555]
aten.addmm.default:: [0.7886046418843882, 0.9329217798915578, 0.9962013717422963]
aten.sum.SymInt:: [0.9358205974583087, 0.9814126709379405, 1.006583550282628]
aten.add_.Tensor:: [0.7253489773412456, 0.7703528781962926, 0.7976537424898803]
aten.new_empty_strided.default:: [0.9366916208071663, 0.9398957719201622, 0.9472405067590517]
aten.sqrt.default:: [0.9791511684613813, 0.9791511684613813, 0.9791511684613813]
aten.new_empty.default:: [0.7994959370210444, 0.804862283140364, 0.8102286292596836]
aten.native_layer_norm_backward.default:: [0.8506269179221464, 1.1152916772486603, 1.6212585844412815]
```

Float32:

```
aten._log_softmax.default:: [0.9550605426043154, 0.9823529203640421, 1.002335693504097]
aten._log_softmax_backward_data.default:: [0.9260106786584326, 0.9406878586461753, 1.0006503787776375]
aten.addmm.default:: [0.9509063155376642, 0.9920652896706708, 1.0371791371367722]
aten.sum.SymInt:: [0.9487470525571405, 0.9901798410457018, 1.05810554621502]
aten.new_empty_strided.default:: [0.7239798153666939, 0.7404751002819113, 0.7636612269539008]
aten.sqrt.default:: [0.9371196135483814, 0.9371196135483814, 0.9371196135483814]
aten.new_empty.default:: [0.8338423596933202, 0.850186671512076, 0.866530983330832]
```

`_log_softmax` and `_log_softmax_backward_data`, and `aten.sum` in fp16 stand out to me. 


Full log: [float32](https://gist.github.com/eellison/b312dc4adc512599084486df667b6264), [float16](https://gist.github.com/eellison/5e3ec5064a19100f95a67b7c69c9ca0a)




cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Timm Model Slow Operators
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

Operators with 50th percentile below 98% of aten perf or 20th percentile below 96%:


```
aten.addmm.default:: [0.8386316138954961, 0.9255533387715138, 0.955919097384518]
aten.new_empty_strided.default:: [0.7074878925134296, 0.7366914699489878, 0.7576066949031502]
aten._softmax.default:: [0.844983630149154, 0.9835961036033891, 1.068020755737445]
aten.max_pool2d_with_indices.default:: [0.6342950044218125, 1.0382350718486175, 1.1183607334882082]
aten.hardsigmoid_backward.default:: [0.9559467667087947, 1.0058027523555526, 1.0368018754050459]
```

```
aten.addmm.default:: [0.8430439948974364, 0.9937072943974172, 1.1970515808766191]
aten.new_empty_strided.default:: [0.715191214805386, 0.7424496557446567, 0.7541450693325825]
aten.max_pool2d_with_indices.default:: [0.836623979229799, 1.0084812810590111, 1.033377775031802]
aten._softmax.default:: [0.8147407919961176, 1.004666288386921, 1.1519087656954243]
aten.select_backward.default:: [0.8844741032127397, 1.0799256067174667, 1.1887779335651258]
aten.new_empty.default:: [0.9331701212881051, 0.9612216337535711, 0.9760060763695062]
aten.div.Tensor:: [0.9588477230714266, 0.9588477230714266, 0.9588477230714266]
aten.rsqrt.default:: [0.9006916404896046, 0.9011194114944925, 0.9032443478821974]
```

`max_pool2d_with_indices`, and `_softmax` stand out. `rsqrt` is probably because we are using the decomposition and not the `rsqrt` ptx instruction.

Full logs: [float16](https://gist.github.com/eellison/1af4f518318d8892159765cb20fd3299), [float32](https://gist.github.com/eellison/3d2c9139c164d3190c8da520018dc99f)

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
[TorchTidy] Check if `set_to_none` would change optimizer semantics.
Labels: oncall: profiler

### 🚀 The feature, motivation and pitch

Several PyTorch optimizers offer a `set_to_none` argument to delete gradients rather than zeroing them. This is particularly important for CUDA where each operation incurs a separate `cudaLaunchKernel` call. [GradNotSetToNonePattern](https://github.com/pytorch/pytorch/blob/master/torch/profiler/_pattern_matcher.py#L429) attempts to identify this opportunity for improvement; however it is currently overzealous. The issue is that certain optimizers will interpret `None` gradients as cause to reset optimizer state (e.g. [SGD momentum](https://github.com/pytorch/pytorch/blob/master/torch/optim/sgd.py#L234-L238)), and as a result `set_to_none` changes the semantics of the optimizer. This task is to extend `GradNotSetToNonePattern` to check for such exceptions, and only recommend `set_to_none` when it can be safely applied.

CC @tiffzhaofb (you can grab the issue when you get your github in the pytorch org)

### Alternatives

_No response_

### Additional context

_No response_

cc @robieta @chaekit @aaronenyeshi @ngimel @nbcsm @guotuofeng @guyang3532 @gaoteng-git @tiffzhaofb
Missing header file
Labels: triaged

### 🚀 The feature, motivation and pitch

Hi, I always build PyTorch from source and link my library to it. Today, I tried link my library to PyTorch nightly package, the error message below came out.
```
Cannot open include file: 'torch/csrc/jit/passes/onnx/constant_map.h': No such file or directory
```
Could we add this header to the package? Or do you have specific policy regarding what should be included and what no? Thanks.

### Alternatives

_No response_

### Additional context

_No response_
tabulate.tabulate causes a lot of memory to be allocated in yolov3
Labels: triaged, oncall: pt2

Calling tabulate.tabulate to render the output graph causes 1 GB of additional memory allocation in yolov3.

This is gated on the log level now, but as a result when DEBUG or INFO log levels are set, this will cause additional memory to be allocated.



https://github.com/pytorch/torchdynamo/blob/0e4d5ee9db6dfeba4d424b2e5d2ad96c79f1dffa/torchdynamo/utils.py#:~:text=tabulate(node_specs%2C%20headers%3D%5B%22opcode%22%2C%20%22name%22%2C%20%22target%22%2C%20%22args%22%2C%20%22kwargs%22%5D) 

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
[Nested Tensor] Update TestCase.AssertEqual
Labels: triaged, module: nestedtensor, module: testing

## TLDR: Update `TestCase.assertEqual` for nested tensors 

We are currently unbinding nested tensors and comparing elements to assert equality. This works but I think that when the metadata for nested tensors if fully ironed out this can be more direct. This ticket is to track follow up work when the metadata scenario has been solidified.

cc @cpuhrsch @jbschlosser @bhosmer @pmeier
BCELoss results in autocast CUDA warning
Labels: triaged, module: amp (automated mixed precision)

### 🐛 Describe the bug

Hi folks, BCELoss on calling *backward* method creates warning UserWarning: User provided device_type of 'cuda', but CUDA is not available. Disabling   warnings.warn('User provided device_type of \'cuda\', but CUDA is not available. Disabling'). The code works normally but it is very irritant. torch/cuda/amp/autocast_mode.py creates a warning because it gets called with `device_type` "cuda". I can get around by commenting line 199 but that is not the solution. Although I am using `torch` together with `DGL` library, the error source seems to be in PyTorch.

```
 # Define training EdgeDataLoader
    train_dataloader = dgl.dataloading.DataLoader(
        graph,                                  # The graph
        train_eid_dict,  # The edges to iterate over
        sampler,                                # The neighbor sampler
        batch_size=batch_size,    # Batch size
        shuffle=True,       # Whether to shuffle the nodes for every epoch
        drop_last=False,    # Whether to drop the last incomplete batch
        num_workers=sampling_workers, # Number of sampling processes
    )

    # Define validation EdgeDataLoader
    validation_dataloader = dgl.dataloading.DataLoader(
        graph,                                  # The graph
        val_eid_dict,  # The edges to iterate over
        sampler,                                # The neighbor sampler
        batch_size=batch_size,    # Batch size
        shuffle=True,       # Whether to shuffle the nodes for every epoch
        drop_last=False,    # Whether to drop the last incomplete batch
        num_workers=sampling_workers,       # Number of sampler processes
    )

    print(f"Canonical etypes: {graph.canonical_etypes}")

    # Initialize loss
    loss = torch.nn.BCELoss()

    # Initialize activation func
    m, threshold = torch.nn.Sigmoid(), 0.5

    # Iterate for every epoch
    for epoch in range(1, num_epochs+1):
        model.train()
        tr_finished = False
        for _, pos_graph, neg_graph, blocks in train_dataloader:
            input_features = blocks[0].ndata[node_features_property]
            # Perform forward pass
            probs, labels, loss_output = batch_forward_pass(model, predictor, loss, m, target_relation, input_features, pos_graph, neg_graph, blocks)
            # Make an optimization step
            optimizer.zero_grad()
            loss_output.backward()  # ***This line generates warning***
            optimizer.step()
  
```

### Versions

o CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.22.2
[pip3] numpydoc==1.4.0
[pip3] torch==1.12.1+cpu
[pip3] torchaudio==0.12.1+cpu
[pip3] torchvision==0.13.1+cpu
[conda] Could not collect


cc @mcarilli @ptrblck
Build from source failed on MacOS 10.6 with CUDA 10.1 
Labels: module: build, triaged, module: macos

### 🐛 Describe the bug

I am trying to build Pytorch from source with CUDA enabled. I followed the steps at https://gist.github.com/brokeyourbike/def7a3313c1e4fdf1334b5c4f0e239f3, including installing from https://github.com/pytorch/pytorch/tree/eeb43ffab9200ea046f29fecf914d8e71d00d474

I am not able to complete `MACOSX_DEPLOYMENT_TARGET=10.13 CC=clang CXX=clang++ python setup.py install` 

Here are some details of my setup:
MacBook Pro (Retina, 15-inch, Late 2013)
NVIDIA GeForce GT 750M 2048 MB
macOS 10.13.6 (17G14042)
Xcode 10.1
CUDA toolkit 10.1
CUDA Driver Version: 418.163
GPU Driver Version: 387.10.10.10.40.140

I tried installing with Python 3.7, 3.8, 3.8, 3.10. They give different warnings, but all fail. 

I've also tried installing, with Python 3.7, eeb43ffab9200ea046f29fecf914d8e71d00d474, 87061875239d0694779255b55e50250b272169ff, and f094113ebf5b4e5281ab1a134220a1a985f03964 (most recent commit today).

My goal is to be able to do the verify installation steps a la https://gist.github.com/brokeyourbike/def7a3313c1e4fdf1334b5c4f0e239f3 with some PyTorch master branch - doesn't have to be the most recent version of PyTorch, but something within the last few years.

For more details see these CMake logs (for build with Python 3.7, PyTorch eeb43ffab9200ea046f29fecf914d8e71d00d474)

[CMakeError.log](https://github.com/pytorch/pytorch/files/9378367/CMakeError.log)
[CMakeOutput.log](https://github.com/pytorch/pytorch/files/9378368/CMakeOutput.log)

Note that I can use CUDA (10.1) on my machine, and compile .cu files and successfully run them.

### Versions

```
(torch_20220818_py37) gws-MacBook-Pro-6:developer gw$ python collect_env.py
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: macOS 10.13.6 (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.24.20220817-gb6cfa38
Libc version: N/A

Python version: 3.7.13 (default, Mar 28 2022, 07:24:34)  [Clang 12.0.0 ] (64-bit runtime)
Python platform: Darwin-17.7.0-x86_64-i386-64bit
Is CUDA available: N/A
CUDA runtime version: 10.1.168
GPU models and configuration: Could not collect
Nvidia driver version: 1.1.0
cuDNN version: Probably one of the following:
/usr/local/cuda/lib/libcudnn.7.dylib
/usr/local/cuda/lib/libcudnn.dylib
/usr/local/cuda/lib/libcudnn_static.a
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.5
[conda] mkl                       2020.2                      260
[conda] mkl-include               2022.0.0           hecd8cb5_105
[conda] numpy                     1.21.5           py37h0f1bd0b_3
[conda] numpy-base                1.21.5           py37hbda7086_3
```

cc @malfet @seemethere @albanD