gopikrsmscs/torch-issues huggingface dataset:
Transformer.generate_square_subsequent_mask has nan values on MPS device
Labels: triaged, module: NaNs and Infs, module: mps

### 🐛 Describe the bug

Square subsequent mask has nan values in place of zeroes when created on MPS device. To reproduce:

```
import torch.nn as nn
print(nn.modules.transformer.Transformer.generate_square_subsequent_mask(4, device='mps'))
```
prints:
```
tensor([[nan, -inf, -inf, -inf],
        [nan, nan, -inf, -inf],
        [nan, nan, nan, -inf],
        [nan, nan, nan, nan]], device='mps:0')
```

However, the correct tensor is printed when generated on CPU,  `nn.modules.transformer.Transformer.generate_square_subsequent_mask(4, device='cpu')` prints:
```
tensor([[0., -inf, -inf, -inf],
        [0., 0., -inf, -inf],
        [0., 0., 0., -inf],
        [0., 0., 0., 0.]])
```

After looking into the `Transformer.generate_square_subsequent_mask` function, the problem appears to come from `torch.triu` when called with `diagonal=1`, since `torch.full` works as expected on MPS device.

### Versions

PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.4 (arm64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.9.17 (main, Jul  5 2023, 15:35:09)  [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.4-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2 Max

Versions of relevant libraries:
[pip3] numpy==1.25.0
[pip3] torch==2.0.1
[pip3] torchvision==0.15.2
[conda] numpy                     1.25.0                   pypi_0    pypi
[conda] torch                     2.0.1                    pypi_0    pypi
[conda] torchvision               0.15.2                   pypi_0    pypi
```[tasklist]
### Tasks
```


cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
add private API for generating all CompositeImplicit decomps from the dispatcher
Labels: Stale

What do people think of this API / location?

Part of the reason it's a bit ugly is because there are 3 different places that decomp can live - the C++ dispatcher, the Python dispatcher, and the python global decomp table.

This should be useful to QAT (cc @andrewor14) - the idea is that they want to be able to use pre_dispatch tracing to trace out an ATen graph above the dispatcher, but they want an easy way to be able to say "specify a decomp table that includes every CompositeImplicitAutograd op that would have decomposed if we had traced through the dispatcher".

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #105240
* __->__ #106330
* #106329
* #105865
pre_dispatch tracing: fix for nn.MultiheadAttention
Labels: Stale

Fixes https://github.com/pytorch/pytorch/issues/106302, see the issue for details.

One minor risk in this PR is that: if you pre_dispatch trace `nn.MultiheadAttention`, **and** your inputs are some custom torch_function subclass, we'll incorrectly assume that the fast-path is okay to use. Given that pre_dispatch is only used underneath our `torch.export` and `torch.compile` subsystems, and it runs in a second pass **after** dynamo has captured the graph, then I think the risk here is pretty low - in both of those cases, dynamo should have already desugared away any torch_function subclasses/modes before we get to our second round of tracing to run with pre_dispatch.

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #106315
* #106461
* #106460
Don't set CUDA_HOME when not compiled with CUDA support
Labels: open source, Stale, ciflow/trunk, release notes: cpp, topic: bug fixes

It doesn't make sense to set this (on import!) as CUDA cannot be used with PyTorch in this case but leads to messages like
> No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'
when CUDA happens to be installed which is at least confusing.
DISABLED test_cuda_assert_should_not_stop_common_distributed_test_suite_cuda (__main__.TestTestingCUDA)
Labels: oncall: distributed, module: ci, triaged, skipped

## Reason
PR https://github.com/pytorch/pytorch/pull/105206 broke one of the slow test "test_testing.py::TestTestingCUDA::test_cuda_assert_should_not_stop_common_distributed_test_suite_cuda"

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @seemethere @malfet @pytorch/pytorch-dev-infra
`torch.nn.modules.MultiheadAttention` yields different graph under pre_dispatch tracing
Labels: triaged, module: __torch_function__, module: export, pre_dispatch tracing

**The problem**

Repro posted at the bottom of the issue. Why does `pre_dispatch` tracing yield a different graph? Answer:

(1) `pre_dispatch` tracing uses `PreDispatchTorchFunctionMode` so that it can properly intercept and capture autograd and autocast API's during tracing, like no_grad/enable_grad ([here](https://github.com/pytorch/pytorch/blob/0af3203c728278920f6e64b22f63181c6f88628a/torch/fx/experimental/proxy_tensor.py#L520))

(2) `MultiheadAttention` has a series of checks to decide whether or not it can use a fast-path kernel during inference. One of them: it will ignore the fast path if any `TorchFunctionMode` is active ([here](https://github.com/pytorch/pytorch/blob/0af3203c728278920f6e64b22f63181c6f88628a/torch/nn/modules/activation.py#L1166))

repro:
```
import torch
from torch.nn.modules import MultiheadAttention
from torch.fx.experimental.proxy_tensor import make_fx

m = MultiheadAttention(4, 4, batch_first=True)
m.eval()

inp = torch.randn(4, 4, 4)
with torch.no_grad():
    gm1 = make_fx(m, pre_dispatch=False)(inp, inp, inp)
    # First graph has "torch.ops.aten._native_multi_head_attention.default" in the trace
    print(gm1.code)
    # Second graph does not
    gm2 = make_fx(m, pre_dispatch=True)(inp, inp, inp)
    print(gm2.code)
```

**Solutions**

The main question here is: "Is `MultiheadAttention` doing something unusual that we're okay with applying a one-off fix for"? User code can do all sorts of branching on whether or not its inputs have a `__torch_function__` or `__torch_dispatch__` enabled (or if there is a mode active), which can cause us to execute a different set of code when we trace compared to running eager mode.

(1) one-off fix for `MultiheadAttention`: we could augment the check [here](https://github.com/pytorch/pytorch/blob/0af3203c728278920f6e64b22f63181c6f88628a/torch/nn/modules/activation.py#L1165) to ignore the case when a pre_dispatch tracing TorchFunctionMode is active. This is actually a bit annoying, since we'd still probably like that check to return true if pre_dispatch is tracing **and** any other torch_function modes are active.

(2) Agree that "when users ask if torchfunction is enabled in some form, they're really asking about **user** torch_function code, and not our tracing infra - augment `has_torch_function(...)` to explicitly ignore the torchfunctionmode used by pre_dispatch tracing.


Open to suggestions - I don't have a strong opinion, although (2) is probably easier to implement.

cc @hameerabbasi @rgommers @peterbell10 @ezyang
Can't build with non-static protobuf
Labels: module: build, triaged

### 🐛 Describe the bug

When building with SYSTEM protobuf (i.e. `-DBUILD_CUSTOM_PROTOBUF=OFF`) and a shared protobuf library the build will fail with link errors such as 

```
ld: /tmp/pytorch-v1.13.1/build/lib/libtorch_cpu.so: undefined reference to `google::protobuf::internal::ThreadSafeArena::thread_cache_'
```

Reason for this is the same as #106206: The generated protobuf files are not correctly linked against the `protobuf::libprotobuf` target which would have defined `PROTOBUF_USE_DLLS` which leads to different places where that variable is defined and expected, see e.g. https://github.com/protocolbuffers/protobuf/blob/v23.0/src/google/protobuf/arena.cc#L522-L530

### Versions

1.13.1 - master

cc @malfet @seemethere
torch DDP oom caused by weak protocol
Labels: oncall: distributed, module: memory usage, module: ddp

### 🐛 Describe the bug

Current torch DDP distributed trainiing protocol is weak, that it use simple tcp protocol listening over master port to choose action to exectue. While it is efficient, but it may cause issue when abonormal network traffic comes.

Here we met an OOM issue, which cause machine quickly run out cpu memory. And we root cause this issue to be torch DDP mistake nmap scan message as addHandler message, and create a very large mmaped pool hold to prepare the incoming message.

Process to reproduce DDP OOM:
```
nmap -p[master port] -sS -sV [torch DDP master node IP]
```

The trigger trace is as:
```
https://github.com/pytorch/pytorch/blob/v2.0.1/torch/csrc/distributed/c10d/TCPStore.cpp#L588
-> https://github.com/pytorch/pytorch/blob/v2.0.1/torch/csrc/distributed/c10d/TCPStore.cpp#L281
-> https://github.com/pytorch/pytorch/blob/v2.0.1/torch/csrc/distributed/c10d/TCPStore.cpp#L384
-> https://github.com/pytorch/pytorch/blob/v2.0.1/torch/csrc/distributed/c10d/Utils.hpp#L659

inline std::string recvString(int socket) {
  SizeType valueSize;
  recvBytes<SizeType>(socket, &valueSize, 1);
  std::vector<char> value(valueSize);
  recvBytes<char>(socket, value.data(), value.size());
  return std::string(value.data(), value.size());
}
```
@recvString, it would parse nmap message into a very large mesage to recv, more than 1T, which lead torch to request 1T+ memory to system, leading to OOM.

It is very common to use nmap to scan port in data center, so shall we consider to make torch DDP protocol more robust to such kind of "attack"? I think only add some magic number would greatly reduce such issue.

Thx

### Versions

Seems to me, since DDP is supported, this issue is existed.

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
Added More Information About Adadelta Optimizer
Labels: triaged, open source, Stale, release notes: optimizer

I have added more information about Adadelta Optimizer to developers understand faster ways to what is doing that.
It's my changes code looks like this:
![Screenshot from 2023-07-31 10-01-54](https://github.com/pytorch/pytorch/assets/93595990/72d7cd00-8acb-4ab0-820b-7ece4943c7c1)
Many tests in test/dynamo fail if run in the context of just 'pytest test/dynamo'
Labels: module: ci, triaged, module: dynamo

### 🐛 Describe the bug

```
FAILED [0.0085s] test/dynamo/test_misc.py::MiscTests::test_cond_nested - AssertionError: <class 'torch._dynamo.variables.torch.TorchVariable'>
FAILED [0.0073s] test/dynamo/test_recompile_ux.py::RecompileUxTests::test_drop_cache_on_skip - AssertionError: False is not true
FAILED [0.0126s] test/dynamo/test_replay_record.py::ReplayRecordTests::test_fn_call_args - AssertionError: no logs of level ERROR or higher triggered on torch._dynamo
FAILED [0.0277s] test/dynamo/test_replay_record.py::ReplayRecordTests::test_local_module - AssertionError: no logs of level ERROR or higher triggered on torch._dynamo
FAILED [0.0234s] test/dynamo/test_replay_record.py::ReplayRecordTests::test_nonlocal_fn_call - AssertionError: no logs of level ERROR or higher triggered on torch._dynamo
FAILED [0.0202s] test/dynamo/test_replay_record.py::ReplayRecordTests::test_nonlocal_module_class - AssertionError: no logs of level ERROR or higher triggered on torch._dynamo
FAILED [0.0238s] test/dynamo/test_replay_record.py::ReplayRecordTests::test_nonlocal_module_fn_call - AssertionError: no logs of level ERROR or higher triggered on torch._dynamo
FAILED [0.0234s] test/dynamo/test_replay_record.py::ReplayRecordTests::test_successful_inline - AssertionError: no logs of level ERROR or higher triggered on torch._dynamo
FAILED [0.0173s] test/dynamo/test_replay_record.py::ReplayRecordTests::test_unsuccessful_inline - AssertionError: no logs of level ERROR or higher triggered on torch._dynamo
FAILED [0.0941s] test/dynamo/test_repros.py::ReproTests::test_reformer_train - AssertionError: '3' != '1'
```

Many of these do not fail if you run them individually. Some of these tests have already been flagged as flaky by CI https://github.com/pytorch/pytorch/issues/101731

I already filed the ReplayRecordTests five days ago https://github.com/pytorch/pytorch/issues/105944

### Versions

main

cc @seemethere @malfet @pytorch/pytorch-dev-infra @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov
RuntimeError: GlooDeviceFactory::makeUVDevice(): interface or hostname can't be empty
Labels: oncall: distributed

### 🐛 Describe the bug

File "C:\Users\me\Desktop\rvc5\RVC-beta0717\runtime\lib\site-packages\torch\distributed\distributed_c10d.py", line 994, in _new_process_group_helper
backend_class = ProcessGroupGloo(backend_prefix_store, group_rank, group_size, timeout=timeout)
RuntimeError: GlooDeviceFactory::makeUVDevice(): interface or hostname can't be empty

### Versions

Collecting environment information...
PyTorch version: 2.0.1+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Home
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr  5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 536.67
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=2496
DeviceID=CPU0
Family=198
L2CacheSize=4096
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=2496
Name=11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==2.0.1
[pip3] torchaudio==2.0.1
[pip3] torchcrepe==0.0.20
[pip3] torchgen==0.0.1
[pip3] torchvision==0.15.2
[conda] Could not collect

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
Fix sym node printing
Labels: triaged, open source, Stale, topic: not user facing, module: dynamo, ciflow/inductor

Fixes #103602 


cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @anijain2305 @ipiszy
Runtime Error: Empty tensor
Labels: needs reproduction, triaged, module: mps

### 🐛 Describe the bug

M1 Max 64 GB RAM running Stable Diffusion in Automatic1111 with SDXL 1.0 model:

RuntimeError: [srcBuf length] > 0 INTERNAL ASSERT FAILED at "/Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/mps/OperationUtils.mm":277, please report a bug to PyTorch. Placeholder tensor is empty!

### Versions

Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: macOS 14.0 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.0.34.3)
CMake version: version 3.27.1
Libc version: N/A

Python version: 3.9.6 (default, Jul  7 2023, 20:26:47)  [Clang 15.0.0 (clang-1500.0.34.3)] (64-bit runtime)
Python platform: macOS-14.0-arm64-arm-64bit
Is CUDA available: N/A
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

CPU:
Apple M1 Max

Versions of relevant libraries:
[pip3] No relevant packages
[conda] Could not collect


cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Network module momory is not released in  C++ libtorch 2.0.1 
Labels: module: windows, module: cpp, triaged

### 🐛 Describe the bug

  
Easily we can define the network that consisting of various nn::modules is defined as a class.
Calling a module's destructor fails to free memory.
void Dispose()
{
  delete model;
 }

At this time, the exception is not caught and the program terminates.

We separately implemented the function of allocating and removing an object in this network module.
When using torch version 1.13, it was confirmed that the GPU memory was completely removed normally with the "delete model" command.

However, when this code with torch 2.0.1 is executed, the Network class defined as Module::holder fails to release the memory normally and the program terminates abnormally.


Can I free the model memory of pytorch for developing object-oriented C++ software.

thank you



### Versions

 Libtorch C++ 2.0.1
Windows 10, CUDA 11.8, RTX 3090 environment.

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @iremyux @Blackhex @cristianPanaite @jbschlosser
Change behavior of .users and .args of SchedulerNode to match the same API in fx.Node
Labels: fb-exported, Stale, module: inductor, module: dynamo, ciflow/inductor

Summary: Following the discussion in https://github.com/pytorch/pytorch/pull/100762/files#r1214655330, this PR changes behavior of `.users` and `.args` of SchedulerNode to match the same API in fx.Node, so that it's easier for people who are familiar with fx.Node APIs to get started on Inductor IR (SchedulerNode).

Differential Revision: D47892324



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @ngimel @anijain2305
[inductor] AOTInductor w/ saved weights
Labels: module: inductor, module: dynamo, ciflow/inductor, module: export, release notes: export

When generating the C++ code, we will unlift the parameters from the module inputs, and output the constants as a dictionary mapping constant name to the constant tensor to a separate file (the same file name where the code is generated, suffixed with `_constants.pt`). These constants can then be loaded through the `AOTInductorModelContainerSetConstants` function, which can be called before `AOTInductorModel.run_impl` to update the global `AOTInductorModel.constants_info_` dictionary of constant name to constant tensor. During `run_impl`, we can query `constants_info_` for the tensor values. 

An example of the resulting C++ code is: [P805713957](https://www.internalfb.com/phabricator/paste/view/P805713957?lines=233%2C242).

This diff is mainly addressing the frontend of AOTInductor, where we save the constants to a different file. The changes to the cpp wrapper are for some initial testing to make sure the flow works. @muchulee8 will follow up with more legit changes on the codegen/cpp wrapper related to `AOTInductorModelContainer`.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @anijain2305
ghstack + mergebot race condition
Labels: module: ci, triaged

### 🐛 Describe the bug

"@pytorchbot merge" command was issued for both https://github.com/pytorch/pytorch/pull/105713 and https://github.com/pytorch/pytorch/pull/105808, which are stacked on top of each other resulted in 3 commits instead of two:
- https://github.com/pytorch/pytorch/commit/ac6d8fb16e64868ec78548882ef5951a05e437f8
- https://github.com/pytorch/pytorch/commit/2e02dfae9a23664f351c49d4a1b4106e45c9f4e1
- https://github.com/pytorch/pytorch/commit/f15b6ec6d6d983cfc399b49b2f72e7ab5635c4e8 (which is a duplicate of https://github.com/pytorch/pytorch/commit/ac6d8fb16e64868ec78548882ef5951a05e437f8 )

Merge-bot should detect such slightly incorrect usecases and cancel concurrent merges on all but topmost PR on the stack

### Versions

CI

cc @seemethere @pytorch/pytorch-dev-infra
[Compiled Autograd] Refactor duplicate symint handling
Labels: module: inductor, module: dynamo

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #106234



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @anijain2305
RProp improvement tracker
Labels: module: optimizer, triaged, actionable

As I was reading through RProp, I realize there are several things that would make it better:
- [ ] Finish converting step to be a Tensor instead of a Scalar (every other optimizer in torch.optim has made this leap a few years ago)
- [ ] The foreach implementation can be faster with the introduction of
     - [ ] torch._foreach_sign
     - [ ] torch._foreach_clamp

cc @vincentqb @jbschlosser @albanD @crcrpar
torch compile does not work with torch.nn.functional.softmax ?
Labels: triaged, oncall: pt2

### 🐛 Describe the bug

Hi,

I'm trying to compile our model with torch.compile stack.
It looks like its not compatible with softmax.

Here is a full error message.
```
rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]   File "/mnt/xarfuse/uid-194481/e0143f7f-seed-bac47847-3826-49af-9334
-48541a87a0c7-ns-4026535691/torch/_subclasses/fake_tensor.py", line 1382, in dispatch
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]     return decomposition_table[func](*args, **kwargs)
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]   File "/mnt/xarfuse/uid-194481/e0143f7f-seed-bac47847-3826-49af-9334
-48541a87a0c7-ns-4026535691/torch/_refs/__init__.py", line 2309, in amax
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]     return _reduction(
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]   File "/mnt/xarfuse/uid-194481/e0143f7f-seed-bac47847-3826-49af-9334
-48541a87a0c7-ns-4026535691/torch/_refs/__init__.py", line 2097, in _reduction
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]     raise RuntimeError(
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] torch._dynamo.exc.BackendCompilerFailed: backend='inductor' raised:
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] RuntimeError: reducing over zero-size dimension for reduction operation
 without identity
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] 
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] While executing %amax : [num_users=1] = call_function[target=torch.ops.aten
.amax.default](args = (%view_4, [-1], True), kwargs = {})
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] Original traceback:
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]   File "/mnt/xarfuse/uid-194481/e0143f7f-seed-bac47847-3826-49af-9334
-48541a87a0c7-ns-4026535691/pyspeech/modules/emformer_attention.py", line 955, in <resume in forward>
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]     rc_output_memory = self.prepare_attention_output(
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]   File "/mnt/xarfuse/uid-194481/e0143f7f-seed-bac47847-3826-49af-9334
-48541a87a0c7-ns-4026535691/pyspeech/modules/emformer_attention.py", line 811, in prepare_attention_output
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING]     attention_weights_float = torch.nn.functional.softmax(
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] 
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] 
[rank11]:[2023-07-28 04:35:51,728] torch._dynamo.convert_frame: [WARNING] 
[rank11]:[2023-07-28 04:35:51,729] torch._dynamo.convert_frame: [WARNING] converting frame raised error, suppressing error

```

Can you help me to look?

### Versions

Meta internal

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
Decomposition of bmm, addmm, mm for dot product
Labels: fb-exported, module: inductor, ciflow/inductor, release notes: inductor

Summary: Decomposition of bmm, addmm, mm for dot product 

Test Plan:
sandcastle ; github ci

Differential Revision: D47881983



cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
[dynamo.export] Assertion Error: Mutating module attribute during export.
Labels: triaged, oncall: pt2, module: export

Examples: #105530, #105531.

Some thoughts:

1. For concern over model being mutated during export, maybe we can backup the original attribute value and revert afterwards.
2. For concern over exported model soundness.
    1. If the original value of the mutated attribute was never used, the graph can safely emit a variable to be used instead of the attribute.
    2. Otherwise, the attribute can be lifted as additional graph input and output, with hinted connection to the original attribute which can be processed by downstream users of export.

Some of the functionalities already exist. The remaining work seems to be mainly formalizing the existence of 2.2. and making it play well with `Dynamo input/output is not consistent with traced input/output` check.

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
dynamo struggles with list of tensor elements
Labels: good first issue, triaged, module: dynamo

### 🐛 Describe the bug

```python
def fn():
    x = torch.tensor([1, 2, 3, 4, 5, 6], dtype=torch.int64)
    y = [x[0], x[2], x[4]]
    return torch.LongTensor(y)
```

fails to compile throwing a
```
torch._dynamo.exc.TorchRuntimeError: Failed running call_function <class 'torch.LongTensor'>(*([FakeTensor
(..., size=(), dtype=torch.int64), FakeTensor(..., size=(), dtype=torch.int64), FakeTensor(..., size=(), d
type=torch.int64)],), **{}):
The tensor has a non-zero number of elements, but its data is not allocated yet. Caffe2 uses a lazy alloca
tion, so you will need to call mutable_data() or raw_mutable_data() to actually allocate memory.
```

Same same if we use a `torch.as_tensor` in place of the `LongTensor` call. It actually passes if we change it by a `torch.tensor`.

### Versions

master

cc @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @aakhundov
Build failure due to C++ version mismatch
Labels: module: build, triaged

### 🐛 Describe the bug

I'm building with GCC 12.2 and "system"/preinstalled protobuf.

The build fails on `caffe2.pb.cc` which includes protobuf which includes abseil which tries to use `std::string_view`.

Abseil was build without specific C++-standard flags so it uses the compilers default, which is C++17. However PyTorchs C++ standard (used to) be C++14: https://github.com/pytorch/pytorch/blob/dff70a5e1a6e7a184a2e75c1b22a8e0ce0944b41/CMakeLists.txt#L40

Both Abseil and Protobuf correctly export (via INTERFACE properties) their required C++ standard but PyTorchs doesn't consistently use the CMake target `protobuf::libprotobuf`

The 2 affected locations are:

- https://github.com/pytorch/pytorch/blob/dffa4e14b912914d461c1af5cea632460ca545aa/caffe2/proto/CMakeLists.txt#L9 which adds an (object-)library containing (generated) protobuf C++ files but that doesn't link against  `protobuf::libprotobuf` and hence doesn't know about it's C++ standard
- https://github.com/pytorch/pytorch/blob/dffa4e14b912914d461c1af5cea632460ca545aa/cmake/ProtoBuf.cmake#L120-L122 which hides the issue by adding the protobuf include path (read from the target) globally (which is discouraged CMake practice)

I was able to resolve this by adding `target_link_libraries(Caffe2_PROTO PUBLIC protobuf::libprotobuf)` and removing https://github.com/pytorch/pytorch/blob/dffa4e14b912914d461c1af5cea632460ca545aa/cmake/ProtoBuf.cmake#L120-L122 

However I'm wondering why the object-library is necessary at all (the objects are used once to create `caffe2_protos`)

Although this (currently) might not be an issue anymore after 36ac095ff8918bf7c208029bf6ad28418f1620c1 raised the standard to C++17 I still think it is wrong to just ignore the C++ standard of (in this case) libprotobuf and any other INTERFACE options it might have set and which may be required to successfully compile it on all systems.

### Versions

1.13.1 - master

cc @malfet @seemethere
[caffe2] Clean up platform-specific fbobjc deps/flags
Labels: fb-exported, Stale, topic: not user facing

Summary:
Replace `platform_deps` with `select()`s and use to make sure the `cpukernel_avx2`
dep is x86-specific.

https://fb.prod.workplace.com/groups/buck2users/posts/3469961626593527/

Test Plan:
```
$ buck2 build //xplat/rtc/media/tools/newton:newton_pcAppleMac#macosx-arm64 --target-platforms //xplat/rtc/webrtc/platforms:bwe-dbg-arm64
```

Differential Revision: D47844555
Revert "[quant][pt2e] store scale/zero_point as tensor attributes to support serialization (#105894)"
Labels: Stale, release notes: quantization

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #106291
* __->__ #106184

manual revert as mergebot refuses to revert https://github.com/pytorch/pytorch/pull/105894

This reverts commit 3ca71ed735257cb7ad377b57a45057c265893a40.
[dynamo.export] symbolic_shapes.GuardOnDataDependentSymNode
Labels: triaged, oncall: pt2, module: dynamic shapes

This was observed when exporting `BertForMaskedLM`. It turned out the model code is doing a input value check to produce a warning. Hence an intuitive workaround appears to be applying `@torch._dynamo.assume_constant_result` on the checker function. That indeed worked around the issue and enables export.

I'm wondering if it is technically feasible to create an "unsafe" config that, when opt in, automatically applies `assume_constant_result` onto the call that triggers `GuardOnDataDependentSymNode`, to essentially consider it a constant and prevent the graph break?

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 

Repro script
```python
import torch
import transformers

model = transformers.AutoModelForMaskedLM.from_pretrained("bert-base-uncased")
gm, _ = torch._dynamo.export(model, aten_graph=True, return_dict=False, input_ids=torch.tensor([[1, 2, 3]]))

gm.print_readable()

# Or below after the api change
# torch._dynamo.export(model)(return_dict=False, input_ids=torch.tensor([[1, 2, 3]]))
```
```
GuardOnDataDependentSymNode: It appears that you're trying to get a value out of symbolic int/float whose value is data-dependent (and
thus we do not know the true value.)  The expression we were trying to evaluate is Eq(i0, 1) (unhinted: Eq(i0, 1)).  Scroll up to see 
where each of these data-dependent accesses originally occurred.

from user code:
   File "transformers/models/bert/modeling_bert.py", line 970, in 
forward
    self.warn_if_padding_and_no_attention_mask(input_ids, attention_mask)
  File "transformers/modeling_utils.py", line 3491, in 
warn_if_padding_and_no_attention_mask
    if self.config.pad_token_id in input_ids[:, [-1, 0]]:

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information
```
[ONNX] Set dynamic_shapes default to True
Labels: module: onnx, open source, Stale, release notes: onnx, topic: improvements

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #106179
* #106178

TODO: Check benchmark result.
- Investigate regressions.
- Consolidate how internal dynamo dynamic shapes api should be exposed.
- Diagnostics.
`torch.ops.aten.split.Tensor._schema` return alias annotations are wrong
Labels: triaged, module: viewing and reshaping

The schema of `split.Tensor` is:
```
split.Tensor(Tensor(a -> *) self, SymInt split_size, int dim=0) -> Tensor(a)[]
```

where the output is a TensorList with the same alias annotation as the input (`a`).

It looks like `torch.ops.aten.split.Tensor`'s schema info doesn't capture this information:
```
>>> torch.ops.aten.split.Tensor._schema.returns[0].alias_info.before_set
set()
>>> torch.ops.aten.split.Tensor._schema.arguments[0].alias_info.before_set
{'a'}
>>> torch.ops.aten.split.Tensor._schema.arguments[0].alias_info.after_set
{'*'}

# This should return {'a'}, probably?
>>> torch.ops.aten.split.Tensor._schema.returns[0].alias_info.before_set
set()
>>> torch.ops.aten.split.Tensor._schema.returns[0].alias_info.after_set
set()
```
Automated submodule update: FBGEMM
Labels: triaged, open source, topic: not user facing

This is an automated pull request to update the first-party submodule for [pytorch/FBGEMM](https://github.com/pytorch/FBGEMM).

New submodule commit: https://github.com/pytorch/FBGEMM/commit/3579b4d627a17e0cbff39b553038e95830fc4685

Test Plan: Ensure that CI jobs succeed on GitHub before landing.
[RFC][WIP] Extend convert_to_unspecialized for module attr mutation to module fields mutated through BINARY_SUBSCR
Labels: Stale, module: inductor, module: dynamo, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #106163

fixes

cc @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @ngimel @anijain2305
rocm support for windows
Labels: module: windows, module: rocm, triaged, enhancement

### 🚀 The feature, motivation and pitch

https://github.com/RadeonOpenCompute/ROCm/discussions/2347 now it's avaible

### Alternatives

_No response_

### Additional context

_No response_

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @vladimir-aubrecht @iremyux @Blackhex @cristianPanaite @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang
Automated submodule update: kineto
Labels: triaged, open source, topic: not user facing

This is an automated pull request to update the first-party submodule for [pytorch/kineto](https://github.com/pytorch/kineto).

New submodule commit: https://github.com/pytorch/kineto/commit/465ff4cd715aa81f08a4320820ea4965bb2fe9c1

Test Plan: Ensure that CI jobs succeed on GitHub before landing.
Pytorch nighlty and openAI/triton cuda
Labels: high priority, needs reproduction, triaged, oncall: pt2

### 🐛 Describe the bug

If I am not wrong I think that the nightly pytorch (Cuda 11.8) wheels are not compatible with the Triton pinned commit as I am seeing something like https://github.com/openai/triton/issues/1955

See more:
https://discuss.pytorch.org/t/any-change-of-using-cuda-12-2/184461/6

If it is true why the CI is not failing with tests on pytorch nighlty cuda 11.8?




### Versions

nightly

cc @ezyang @gchanan @zou3519 @msaroufim @wconstab @bdhirsh @anijain2305
Sparse COO indices are torch.Int64 -- is this necessary?
Labels: module: sparse, triaged

### 🚀 The feature, motivation and pitch

```
import torch
import numpy as np

a = np.eye(10)
t = torch.Tensor(a).to_sparse_coo()
print(t.indices().dtype)
```
returns:
```torch.int64```

A 64-bit int seems excessive for storing coordinate indices, especially given the main intent of sparse formats is size reduction. Perhaps a torch.int32 would suffice? 

### Alternatives

_No response_

### Additional context

_No response_

cc @alexsamardzic @nikitaved @pearu @cpuhrsch @amjames @bhosmer
`export(..., pre_dispatch=True)` for model in eval mode still inserts autograd ops
Labels: triaged, module: export, pre_dispatch tracing

Support for `export(..., pre_dispatch=True)` was added to make sure autograd (and autocast) functionality (e.g. `torch.no_grad`) continue to work on exported graphs. This is needed for training on an exported model. The way this works is it inserts these `torch._C._set_grad_enabled` ops [into the graph](https://github.com/pytorch/pytorch/blob/646fa36875821f2bcf4fbfbf669c1f4f9f69700d/torch/fx/experimental/proxy_tensor.py#L517-L536).

We have a use case where we want to use `export(..., pre_dispatch=True)` for models in eval mode. However, I just tried this out and looks like I'm still seeing these `torch._C._set_grad_enabled` ops in the graph. This may lead to unexpected behavior during eval, where some gradients may be computed but we actually don't need them.

**Minimal repro:**
```
import torch
import torch._dynamo

class ToyModel(torch.nn.Module):
    def __init__(self):
        super(ToyModel, self).__init__()
        self.linear1 = torch.nn.Linear(10, 10)
        self.linear2 = torch.nn.Linear(10, 5)

    def forward(self, x):
        x = self.linear1(x)
        with torch.no_grad():
            x = self.linear2(x)
        return x

example_inputs = (torch.randn(10, 10),)
m = ToyModel().eval()
m, _ = torch._dynamo.export(
    m,
    *example_inputs,
    aten_graph=True,
    pre_dispatch=True,
)
print(m)
```

**Output:**
```
def forward(self, x):
    arg0, = fx_pytree.tree_flatten_spec(([x], {}), self._in_spec)
    _param_constant0 = self._param_constant0
    _param_constant1 = self._param_constant1
    linear_default = torch.ops.aten.linear.default(arg0, _param_constant0, _param_constant1);  arg0 = _param_constant0 = _param_constant1 = None
    _set_grad_enabled = torch._C._set_grad_enabled(False)
    _param_constant2 = self._param_constant2
    _param_constant3 = self._param_constant3
    linear_default_1 = torch.ops.aten.linear.default(linear_default, _param_constant2, _param_constant3);  linear_default = _param_constant2 = _param_constant3 = None
    _set_grad_enabled_1 = torch._C._set_grad_enabled(True)
    return pytree.tree_unflatten([linear_default_1], self._out_spec)
```
bc-linter false positive with TypeAliases
Labels: triaged, module: devx

### 🐛 Describe the bug

The BC-linter warned on my PR even though the actual type has not changed from `Dict[str, Any]`. In actuality, I just introduced a TypeAlias called StateDict for clarity. 

https://github.com/pytorch/pytorch/pull/105953#issuecomment-1653715848

### Versions

main

cc @ZainRizvi @kit1980 @huydhn @clee2000
Registering function that takes const std::vector<c10::SymInt>& to SymInt[] schema gives confusing error message
Labels: triaged, module: dynamic shapes

### 🐛 Describe the bug

```
  what():  Inferred operator schema for a C++ kernel function doesn't match the expected function schema.                                                                         
  operator: fbgemm::jagged_to_padded_dense                                                                                                                                        
  expected schema: fbgemm::jagged_to_padded_dense(Tensor values, Tensor[] offsets, int[] max_lengths, float padding_value=0.) -> Tensor                                           
    registered at fbcode/deeplearning/fbgemm/fbgemm_gpu/src/jagged_tensor_ops/jagged_tensor_ops_cpu.cpp:1601                                                                      
  inferred schema: (Tensor _0, Tensor[] _1, SymInt[] _2, float _3) -> Tensor _0                                                                                                   
    registered at fbcode/deeplearning/fbgemm/fbgemm_gpu/src/jagged_tensor_ops/jagged_tensor_ops_autograd.cpp:867                                                                    reason: Type mismatch in argument 3: int[] vs SymInt[]   
```

### Versions

main
torch._subclasses.fake_tensor.DynamicOutputShapeException: aten.nonzero.default
Labels: triaged, module: fakeTensor

Hi team, I'm trying to grab a computational graph using FakeTensor, but encountered DynamicOutputShapeException exception.

PyTorch Version: 2.0.1+cu118


```
import torch
from torch._subclasses.fake_tensor import FakeTensorMode
import traceback
from torch.fx.experimental.symbolic_shapes import ShapeEnv
from torch._dynamo.output_graph import config

a = torch.ones([1, 0])
mode = FakeTensorMode()

a = mode.from_tensor(a)
try:
    with mode:
        b = torch.nonzero(a)
except Exception as e:
    traceback.print_exc() 
```

The stacktrace is:

```
Traceback (most recent call last):
  File "<ipython-input-15-5efd30f1231c>", line 13, in <cell line: 11>
    b = torch.nonzero(a)
  File "/usr/local/lib/python3.10/dist-packages/torch/utils/_stats.py", line 20, in wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py", line 987, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py", line 1162, in dispatch
    op_impl_out = op_impl(self, func, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py", line 403, in dyn_shape
    raise DynamicOutputShapeException(func)
torch._subclasses.fake_tensor.DynamicOutputShapeException: aten.nonzero.default
```


So, anyone can help me to bypass this exception with nonzero operator?

Any response will be appreciated, thanks.
Allow settingsetGraphExecutorOptimize default for all threads
Labels: oncall: jit

### 🚀 The feature, motivation and pitch

Our teams have had numerous issues with torchscipt optimization and have had to disabled it to avoid breaking the code.

I recently debugged one such issue here: https://github.com/pytorch/pytorch/issues/106128

`torch::jit::setGraphExecutorOptimize` sets a thread-local variable, but the model may be run on a different thread.  This was difficult to debug when a change to our test execution framework started running unit tests on new threads, but the model was only loaded once and reused across the tests.

Disabling optimization as a workaround would be much easier if it could be set globally for all threads.  

### Alternatives

Surfacing and resolving optimizer bugs instead of working around them and sacrificing performance would be ideal, but when it blocks research work we need a stable workaround.

### Additional context

Refer to https://github.com/pytorch/pytorch/issues/106128 for an example.

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
Libtorch report C10 error when compiling on my own project
Labels: needs reproduction, oncall: binaries, module: cpp, triaged

### 🐛 Describe the bug

Hi, guys

Some errors occur when i compiled Libtorch with other third party, such as onnx. The errors are as following:

I don't know if it's about the torch flags, D_GLIBCXX_USE_CXX11_ABI is 1 or 0.
`-- TORCH_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=1`

### Error logs

```
`/home/xxx/code/dev/nn_tools/third_party/libtorch/include/c10/util/Exception.h:220:83: error: exception handling disabled, use ‘-fexceptions’ to enable
  220 |   throw ::c10::err_type({__func__, __FILE__, static_cast<uint32_t>(__LINE__)}, msg)

/home/xxx/code/dev/nn_tools/third_party/libtorch/include/c10/core/DispatchKeySet.h:147:38: error: ‘findFirstSet’ is not a member of ‘llvm’
  147 |       uint64_t firstKeyIndex = llvm::findFirstSet(masked_data);

/home/xxx/code/dev/nn_tools/third_party/libtorch/include/ATen/core/ivalue_inl.h:555:14: error: ‘e’ was not declared in this scope; did you mean ‘llvm::numbers::e’?
  555 |       return e.what();
      |              ^
      |              llvm::numbers::e`
```
As the log says, if i add `add_compile_options(-fexceptions)`,  it will report other c10::detail::torchInternalAssertFail errors:

```
quantizationCalibration.cpp:(.text._ZN3c1020intrusive_ptr_targetD2Ev[_ZN3c1020intrusive_ptr_targetD5Ev]+0xdd): undefined reference to `c10::detail::torchInternalAssertFail(char const*, char const*, unsigned int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/usr/bin/ld: quantizationCalibration.cpp:(.text._ZN3c1020intrusive_ptr_targetD2Ev[_ZN3c1020intrusive_ptr_targetD5Ev]+0x137): undefined reference to `c10::detail::torchInternalAssertFail(char const*, char const*, unsigned int, char const*, char const*)'
/usr/bin/ld: lib/libMLIRTop.a(quantizationCalibration.cpp.o): in function `c10::intrusive_ptr<c10::TensorImpl, c10::UndefinedTensorImpl>::reset_()':
quantizationCalibration.cpp:(.text._ZN3c1013intrusive_ptrINS_10TensorImplENS_19UndefinedTensorImplEE6reset_Ev[_ZN3c1013intrusive_ptrINS_10TensorImplENS_19UndefinedTensorImplEE6reset_Ev]+0xa): undefined reference to `c10::UndefinedTensorImpl::_singleton'
/usr/bin/ld: lib/libMLIRTop.a(quantizationCalibration.cpp.o): in function `std::_MakeUniq<torch::autograd::AutogradMeta>::__single_object std::make_unique<torch::autograd::AutogradMeta, c10::TensorImpl*, bool&>(c10::TensorImpl*&&, bool&)':
```

I have test Libtorch standalone, it can compile and run correctly. So why it report above errors when i compile on my own project?

Thanks in advance.

### Minified repro

_No response_

### Versions

The Libtorch version is 2.0.1, CPU and GPU version



cc @seemethere @malfet @jbschlosser @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
[feature request] Better argument checks and error messaging for `tensor.repeat`
Labels: module: error checking, triaged, actionable, release notes: python_frontend

### 🐛 Describe the bug

```python
erb_indices = [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 5, 5, 7, 7, 8, 10, 12, 13, 15, 18, 20, 24, 28, 31, 37, 42, 50, 56, 67]

torch.tensor(erb_indices).reciprocal().repeat(erb_indices)
# RuntimeError: Storage size calculation overflowed with sizes=[2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 5, 5, 7, 7, 8, 10, 12, 13, 15, 18, 20, 24, 28, 31, 37, 42, 50, 56, 2144]
```

It would be good if the error is more clear.

also, strangely `torch.repeat` does not exist as a module function as opposed to `torch.repeat_interleave`: `AttributeError: module 'torch' has no attribute 'repeat'`

Btw, I also noticed that shape checks for torch.hstack are much more permissive compared to torch.stack. This might be okay for compat with NumPy (or maybe there should even be a `strict` mode that would enable stricter checks; or that pytorch should be stricter for consistency sake between torch.stack and torch.hstack which appear similar enough), but I suggest that this should be mentioned in docs.

### Versions

'2.0.0+cpu'

cc @malfet @svekars @carljparker
Removal of Object Check in CUDAPluggableAllocator::raw_delete()
Labels: triaged, open source, Stale, topic: not user facing

I've observed that the current sanity check is designed to prevent the deallocation of an illegal address and ensure the correctness of subsequent size and device information. The allocation_metadata_ is an unordered_map, indicating that a unique address can only correspond to a unique active object.

However, if users employ custom allocators based on a priori knowledge or profiling-based strategies, combined with Python's irregular garbage collection, it's plausible for multiple active objects to possess the same address. This doesn't necessarily lead to actual conflicts, but rather is due to Python not deallocating them in time.

In light of this situation, I would like to propose a consideration to possibly remove or at least downgrade this mandatory check to a mere warning. This is not to undermine the importance of maintaining object metadata integrity or checking for illegal addresses. Instead, it's a suggestion to potentially allow the responsibility to rest more with the user's custom allocator, since mature memory allocators should be capable of managing the object metadata independently. Their free() methods would primarily require the pointer information, and their methods would already include sufficient checks. 

This proposal is meant to foster a more flexible design space for user custom allocators. Please take this as a well-intended recommendation for an open discussion.