gopikrsmscs/torch-issues huggingface dataset:
Edge case: CPU bool abs is not supported
Labels: triaged, module: numpy, module: edge cases

### 🐛 Describe the bug

Test on the CPU:

    import torch
    m = torch.nn.Softsign()
    input = torch.randint(0, 2, [2], dtype=torch.bool)
    print(m(input))

result:RuntimeError: "abs_cpu" not implemented for 'Bool'


Test on the GPU:

    import torch
    m = torch.nn.Softsign().cuda()
    input = torch.randint(0, 2, [2], dtype=torch.bool).cuda()
    print(m(input))

result: tensor([0.5000, 0.0000], device='cuda:0')

When torch.nn.Softsign runs on the CPU, it will throw an exception of "abs_cpu" not implemented for 'Bool', but on the GPU, it can run normally.

The same situation exists for torch.nn.functional.softsign.

### Versions

pytorch: 1.8
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows

cc @mruberry @rgommers @albanD @jbschlosser @walterddr @kshitij12345 @saketh-are
How can i patch the torch.jit in the second solution? Could not figure out entrypoint ?
Labels: oncall: jit

        Here are the workarounds I found to make things work.

### Solution 1: patch torchvision 0.2.2 (dirty and painful)
Pyinstaller works well with torchvision 0.2.2, thus if you do not use too many functionnalities of more recent version of torchvision, you can "patch" old torchvision 0.2.2 by adding the needed new tools. Note that this may need you first use compiled file of torchvision >= 3.0 (_C.so).

### Solution 2: Keep recent torchvision & patch torch.jit in your entry point
I found some monkey patch on stackoverflow and dev blogs that simply overwrite 2 methods of torch.jit.
Simply add this path at the beginning of your entry point:
```
def script_method(fn, _rcb=None):
    return fn
def script(obj, optimize=True, _frames_up=0, _rcb=None):
    return obj    
import torch.jit
torch.jit.script_method = script_method 
torch.jit.script = script
```


I still think there should be a more direct way to do that with pyinstaller, but this works for me.
I hope it will help others too.

_Originally posted by @Sstrap in https://github.com/pytorch/vision/issues/1899#issuecomment-598200938_
      

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
torch.nn.ReplicationPad1d:The description of the exception information thrown is not accurate
Labels: module: error checking, triaged, module: padding

### 🐛 Describe the bug

code1: 

    import torch
    m = torch.nn.ReplicationPad1d("max")
    input = torch.arange(8, dtype=torch.float).reshape(1, 2, 4)
    print(m(input))

result: TypeError: pad(): argument 'pad' (position 2) must be tuple of ints, not tuple

For the padding in torch.nn.ReplicationPad1d(padding), the input of the test code is of type str, but the exception information thrown is of type tuple.

The same problem exists with torch.nn.ReplicationPad2d.

### Versions

pytorch: 1.12.1
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows
prod_cpu not implemented for 'BFloat16'
Labels: module: cpu, triaged, module: bfloat16

### 🐛 Describe the bug

Test on the CPU:

    import torch
    m = torch.nn.Hardshrink()
    input = torch.rand([2], dtype=torch.bfloat16)
    print(m(input))

result: RuntimeError: "hardshrink_cpu" not implemented for 'BFloat16'


Test on the GPU:

    import torch
    m = torch.nn.Hardshrink()
    input = torch.rand([2], dtype=torch.bfloat16).cuda()
    print(m(input))

result: tensor([0.9102, 0.0000], device='cuda:0', dtype=torch.bfloat16)

In the pytorch 1.8 version, hardshrink_cpu does not support BFloat16 and Half, but cuda does. Then in the latest version, hardshrink_cpu already supports BFloat16, but it still does not support Half.

torch.nn.Softshrink has the same problem.

### Versions

pytorch: 1.8
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows 

cc @VitalyFedyunin @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
torch.nn.functional.normalize: whether true is equal to 1
Labels: triaged, module: norms and normalization

### 🐛 Describe the bug

pytorch 1.8:

    import torch
    input = torch.rand([2, 3], dtype=torch.float32)
    print(torch.nn.functional.normalize(input, p=2.0, dim=True))

result : tensor([[0.7107, 0.6418, 0.2883], [0.0826, 0.2365, 0.9681]])


pytorch 1.12: 

    import torch
    input = torch.rand([2, 3], dtype=torch.float32)
    print(torch.nn.functional.normalize(input, p=2.0, dim=True))

result : TypeError: norm() received an invalid combination of arguments - got (Tensor, float, list, keepdim=bool), but expected one of: ... ....


pytorch 1.12: 

    import torch
    input = torch.rand([2, 3], dtype=torch.float32)
    print(torch.nn.functional.normalize(input, p=2.0, dim=1))

result : tensor([[0.1417, 0.4098, 0.9011], [0.3879, 0.4451, 0.8071]])


pytorch 1.12: 

    print(True == 1)

result : True
   
In both pytorch1.8 and pytorch1.12, the same code yields different results. Both pytorch versions work successfully when dim is changed from True to 1, but from a python perspective, the default value of True is 1. True==1 results in True, so I think the advanced version of torch.nn.functional.normalizeis handling True improperly.

### Versions

pytorch: 1.12
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows 

pytorch: 1.8.0
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.1
GPU models and configuration: RTX3060
Operating System：Windows
RuntimeError: CUDA error: device-side assert triggered
Labels: module: loss, triaged

### 🐛 Describe the bug

    import torch
    input = torch.rand([210, 6], dtype=torch.float32).cuda()
    target = torch.randint(0, 8, [210], dtype=torch.uint8).cuda()
    try:
        print(torch.nn.functional.nll_loss(input, target))
    except Exception as e:
        print(str(e))

result :  RuntimeError: CUDA error: device-side assert triggered.CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

When running on cuda, the test code problem should be Target 7 is out of bounds. The exception thrown is out of bounds. Expected object of scalar type Long but got scalar type Byte for argument #2 'target' in call to _thnn_nll_loss_forward. Expected object of scalar type long but got scalar type byte for argument #2 'target' in call to _thnn_nLL_loss_forward.

torch.nn.NLLLoss also has the same problem.

### Versions

pytorch: 1.12.1
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows
torch.nn.functional.embedding_bag throws an exception when it runs on a CPU, but it runs successfully on a GPU.
Labels: module: nn, module: cuda, module: error checking, triaged, actionable, module: embedding

### 🐛 Describe the bug

Test on the CPU:

    import torch
    input = torch.randint(-5, 1, [1, 2], dtype=torch.int64)
    weight = torch.rand([2, 3], dtype=torch.float32)
    print(torch.nn.functional.embedding_bag(input, weight))

result: RuntimeError: Index 0 is out of bounds: -1, range 0 to 2

Test on the GPU:

    import torch
    input = torch.randint(-5, 1, [1, 2], dtype=torch.int64).cuda()
    weight = torch.rand([2, 3], dtype=torch.float32).cuda()
    print(torch.nn.functional.embedding_bag(input, weight))

result: tensor([[0., 0., 0.]], device='cuda:0')
embedding_bag execution on a CPU throws an exception, but it runs successfully on a GPU.

### Versions

pytorch: 1.12.1
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows 

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are @ngimel
Documentation: torch.nn.functional.embedding docs could more clearly state the requirement that weight be a 2D tensor
Labels: module: docs, module: nn, triaged, module: embedding, topic: docs

### 🐛 Describe the bug

    import torch
    input = torch.tensor([[1, 2, 4, 5], [4, 3, 2, 9]])
    embedding_matrix = torch.rand(10)
    print(torch.nn.functional.embedding(input, embedding_matrix))

pytorch 1.9.0:
result: tensor([[0.3005, 0.3053, 0.9081, 0.4288],  [0.9081, 0.8599, 0.3053, 0.1734]])

pytorch 1.12.1:
result: RuntimeError: 'weight' must be 2-D

The descriptions in the pytorch1.8 and pytorch1.12 official documentation are the same, but 1.8 will run successfully with the same parameters, while 1.12 will throw an exception.

### Versions

pytorch: 1.9.0
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.1
GPU models and configuration: RTX3060
Operating System：Windows 

pytorch: 1.12.1
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.3
GPU models and configuration: RTX3060
Operating System：Windows 

cc @svekars @carljparker @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Per-sample input xfail / test generation
Labels: triaged, module: testing

### 🐛 Describe the bug

In https://github.com/pytorch/pytorch/pull/89332 I added a new sample input, and as a result, I had to disable cat testing on MPS entirely, because only *that* sample input was computed incorrectly on MPS. It would be nice if I could have only xfail'ed that sample input.

In general, running all sample inputs in a single test is problematic, as you cannot easily tell if there is only one sample input that is problematic / which sample input is problematic. Creating a separate test per sample input feels like it would be better.

### Versions

master
AdaptiveAvgPool1d failed in the lower version
Labels: module: nn, triaged, module: pooling

### 🐛 Describe the bug

pytorch 1.8 :

    import torch
    m = torch.nn.AdaptiveAvgPool1d(5)
    input = [torch.randn(1, 64),]
    print(m(*input))

result: RuntimeError: Expected 3-dimensional tensor, but got 2-dimensional tensor for argument #1 'self' (while checking arguments for adaptive_avg_pool1d)

pytorch 1.12 :

    import torch
    m = torch.nn.AdaptiveAvgPool1d(5)
    input = [torch.randn(1, 64),]
    print(m(*input))

result: tensor([[-0.2048,  0.0213,  0.1549,  0.5774, -0.3389]])

It is not clear why an exception is raised in pytorch 1.8, but not in other versions.




### Versions

pytorch: 1.8.0
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.1
GPU models and configuration: RTX3060
Operating System：Windows 

pytorch: 1.12
Python version: 3.9
CUDA/cuDNN version: cuDNN 11.6
GPU models and configuration: RTX3060
Operating System：Windows 

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
AdaptiveAvgPool1d throws different exceptions when using the gpu
Labels: module: nn, module: cuda, module: error checking, triaged, actionable, module: pooling

### 🐛 Describe the bug

Test on the CPU:

    import torch
    m = torch.nn.AdaptiveAvgPool1d(-2)
    input = torch.rand([1, 64, 8], dtype=torch.float32)
    m(input)

result: RuntimeError: Trying to create tensor with negative dimension -2: [1, 64, 1, -2]

Test on the GPU:

    import torch
    m = torch.nn.AdaptiveAvgPool1d(-2).cuda()
    input = torch.rand([1, 64, 8], dtype=torch.float32).cuda()
    m(input)

result:RuntimeError: setStorage: sizes [1, 64, -2], strides [64, 1, 1], storage offset 0, and itemsize 4 requiring a storage size of 244 are out of bounds for storage of size 0

As we can see, torch.nn.AdaptiveAvgPool1d running the same code on the CPU and GPU throws different exceptions.

### Versions

pytorch: 1.8.0
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.1
GPU models and configuration: RTX3060
Operating System：Windows 

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are @ngimel
torch.mm: Exceptions thrown on the CPU and GPU are inconsistent
Labels: module: cuda, module: error checking, triaged

### 🐛 Describe the bug

Example 1:

        Test on the CPU:
        
            import torch
            input = torch.rand([5, 3], dtype=torch.float32)
            mat2 = torch.rand([64, 1], dtype=torch.float32)
            resCPU = torch.mm(input, mat2)
        
        result: RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x3 and 64x1)
        
        Test on the GPU:
        
            import torch
            input = torch.rand([5, 3], dtype=torch.float32).cuda()
            mat2 = torch.rand([64, 1], dtype=torch.float32).cuda()
            resCPU = torch.mm(input, mat2)
        
        result: RuntimeError: mat1 dim 1 must match mat2 dim 0

Example 2:

        Test on the CPU:
        
            import torch
            input = torch.rand([2], dtype=torch.float32)
            mat2 = torch.rand([64, 1], dtype=torch.float32)
            resCPU = torch.mm(input, mat2)
        
        result: RuntimeError: self must be a matrix
        
        Test on the GPU:
        
            import torch
            input = torch.rand([2], dtype=torch.float32).cuda()
            mat2 = torch.rand([64, 1], dtype=torch.float32).cuda()
            resCPU = torch.mm(input, mat2)
        
        result: RuntimeError: tensors must be 2-D


As we can see, torch.mm running the same code on the CPU and GPU throws different exceptions.

torch.matmul and torch.bmm also have the same situation.

### Versions

pytorch: 1.8.0
Python version: 3.8
CUDA/cuDNN version: cuDNN 11.1
GPU models and configuration: RTX3060
Operating System：Windows 

cc @ngimel
Should torchdynamo specialize on nn.Module
Labels: triaged, oncall: pt2

This is an open design question with several discussion threads, and unclear motivations and perf/risk tradeoffs.  I'm opening the issue in part to organize the discussion, and see if we can align on a direction.

There are 2 behaviors that are interesting and somewhat orthogonal.

1. dynamo **specializes** on nn.module instances, meaning it places a (ref?) to the module in its traced graphmodule's `.parameters` field, and guards only on the id() of the nn.module matching at runtime.  It also monkeypatches the module's __setattr__ so that it can detect (most) changes to the module at runtime, and invalidate the specialization, causing a recompile
2. **`call_module`** nodes are inserted in the graph for nn.Modules, rather than tracing into the module and recording torch function calls.

(1) is a potential issue in that we cannot detect modifications to C++ part of nn.Module that don't go through python API, e.g. some sketchy stuff FSDP does).  But it is also an important perf optimization because it reduces the number of guards in the system, and provides a way for backend compilers to keep a list of .parameters() tensors separate from input tensors, and apply specializing optimizations just to the parameters.

(2) could be an issue if code inside nn.modules isn't well behaved- tracing deeper would be safer, at the cost of more exposure to having graph-breaks.  It has been proposed to change just (2) to unify dynamo's IR trace level, even if (1) is left as it is.  


For now i'm not proposing to rush to a change or a decision, but i'd like to sharpen the above description, including the set of constraints and problems and goals based on your feedback.

cc @ezyang @soumith @msaroufim @ngimel @bdhirsh
`masked_fill` with `FloatTensor` mask will never mask but fails silently.
Labels: triaged, module: correctness (silent), module: masked operators

### 🐛 Describe the bug

Passing `bool` or `int` tensors as a mask input to `masked_fill` returns the excepted result, but a `float` tensor will not mask and returns input tensor silently.

```
import torch

x = torch.rand(4, 4)
m = torch.rand(4, 4) > 0.5

print(m, '\n')
print(x.masked_fill(m , 0.), '\n')
print(x.masked_fill(m.int() , 0.), '\n')
print(x.masked_fill(m.float() , 0.), '\n')
```

Output:
```
tensor([[False, False,  True,  True],
        [ True, False, False, False],
        [ True,  True, False, False],
        [ True,  True, False, False]])

tensor([[0.8455, 0.7714, 0.0000, 0.0000],
        [0.0000, 0.1767, 0.9352, 0.1114],
        [0.0000, 0.0000, 0.7616, 0.2102],
        [0.0000, 0.0000, 0.4796, 0.1690]])

tensor([[0.8455, 0.7714, 0.0000, 0.0000],
        [0.0000, 0.1767, 0.9352, 0.1114],
        [0.0000, 0.0000, 0.7616, 0.2102],
        [0.0000, 0.0000, 0.4796, 0.1690]])

tensor([[0.8455, 0.7714, 0.7397, 0.3867],
        [0.8923, 0.1767, 0.9352, 0.1114],
        [0.9112, 0.1297, 0.7616, 0.2102],
        [0.9136, 0.0941, 0.4796, 0.1690]])
```

I feel this behaviour is inconsistent with python's behaviour: `bool(0.0) == False` and `bool(1.0) == True` and should return the same as the other two examples. In any case, it should at least fail when a `FloatTensor` is passed as a mask.

### Versions

```
$ python collect_env.py
Collecting environment information...
PyTorch version: 1.13.0
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Arch Linux (x86_64)
GCC version: (GCC) 12.2.0
Clang version: 14.0.6
CMake version: version 3.24.3
Libc version: glibc-2.36

Python version: 3.10.8 (main, Nov  1 2022, 14:18:21) [GCC 12.2.0] (64-bit runtime)
Python platform: Linux-6.0.8-arch1-1-x86_64-with-glibc2.36
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce GTX 1080 Ti
Nvidia driver version: 520.56.06
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.8.6.0
/usr/lib/libcudnn_adv_infer.so.8.6.0
/usr/lib/libcudnn_adv_train.so.8.6.0
/usr/lib/libcudnn_cnn_infer.so.8.6.0
/usr/lib/libcudnn_cnn_train.so.8.6.0
/usr/lib/libcudnn_ops_infer.so.8.6.0
/usr/lib/libcudnn_ops_train.so.8.6.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.982
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.4
[pip3] rotary-embedding-torch==0.1.5
[pip3] torch==1.13.0
[pip3] torchtext==0.10.0a0+0d670e0
[conda] nomkl                     3.0                           0
```

I also reproduced this on a Ubuntu 20 internal node, but don't have access to the env right now.
code sharing for fundamental ops in quantization
Labels: oncall: quantization, triaged

### 🐛 Describe the bug

As @vkuzo mentioned in https://github.com/pytorch/pytorch/pull/89269#discussion_r1026672280 we should share the code for ops that's critical for numerics in quantization, such as quantize_per_tensor, quantize_per_channel, fake quantize, and dequantize ops, this will help us to have identical numerics across different flows (QAT, PTQ) and different APIs (convert_to_reference_fx, _convert_to_reference_decomposed_fx)

### Versions

master

cc @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel
Meta implementation for copy_ is wrong
Labels: triaged, module: meta tensors

### 🐛 Describe the bug

Currently, the meta impl for copy_ just returns self without performing any checks. This is wrong and we should fix this. 
CPU tensor: (correctly fails)
```
>>> x=torch.randn([5, 1]).expand(5, 5)
>>> x.stride()
(1, 0)
>>> sqrt=x.sqrt()
>>> sqrt.stride()
(5, 1)
>>> x.copy_(sqrt)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: unsupported operation: more than one element of the written-to tensor refers to a single memory location. Please clone() the tensor before performing the operation.
```

Meta (incorrectly passes):
```
>>> x=torch.randn([5, 1], device='meta').expand(5, 5)
>>> sqrt=x.sqrt()
>>> x.copy_(sqrt)
tensor(..., device='meta', size=(5, 5))
```

### Versions

master

cc @ezyang @eellison @bdhirsh @soumith
[dynamic shapes] detectron2 dynamic shapes fails
Labels: triaged, oncall: pt2

- Install dynamo
- Install D2 (full instruction is [here](https://docs.google.com/document/d/1iOXyMHAkfWw0eKOV-E0yzvMNLIUB0sBRacvEfDNJC88/edit?usp=sharing))

```bash
pip3 install opencv-python
pip3 install scipy
pip3 install shapely
sudo apt install ninja-build  # make compilation faster
git clone git@github.com:facebookresearch/detectron2.git
cd detectron2 
TORCH_CUDA_ARCH_LIST="Pascal;Volta" pip3 install -e .
Add dynamo model optimization after this line https://github.com/facebookresearch/detectron2/blob/main/tools/lazyconfig_train_net.py#L73
cd projects/ViTDet
```

```bash
python lazy_train_net.py --config-file configs/sanity_check.py --num-gpus 2 dataloader.train.total_batch_size=2
```

It will throw error here https://github.com/facebookresearch/detectron2/blob/main/detectron2/structures/image_list.py#L122

cc @ezyang @msaroufim @wconstab @ngimel @bdhirsh
torch.randn and torch.normal sometimes produce NaN on mps device
Labels: triaged, module: random, module: mps

### 🐛 Describe the bug

Using the function torch.randn (and torch.normal)  on the mps device sometimes produces unexpected NaN in the generated tensor.

```python

import torch

for i in range(10000):
    jitter = torch.randn((32, 1000), device="mps")
    if torch.any(torch.isnan(jitter)):
        print(f"NaN generated at iter: {i}")
        break
```

```bash
>>> NaN generated at iter: 246
```


### Versions

```bash
>>> python collect_env.py
Collecting environment information...
PyTorch version: 1.13.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.102)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.7 (main, Nov  9 2022, 12:38:27) [Clang 14.0.0 (clang-1400.0.29.102)] (64-bit runtime)
Python platform: macOS-12.6-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.4
[pip3] torch==1.13.0
[pip3] torchmetrics==0.10.2
[conda] Could not collect
```

cc @pbelevich @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
torch.addcdiv: input, tensor1, and tensor2 parameters should be of the same type
Labels: module: docs, triaged, module: assert failure

### 📚 The doc issue

In the latest article, for torch.addcdiv(input, tensor1, tensor2, *, value=1, out=None) operator, the input, tensor1, tensor2 types are not required to be consistent, but simply referred to as tensors. But in a real test, something goes wrong.

For example：

    import torch
    input = torch.rand([1], dtype=torch.float64)
    tensor1 = torch.rand([5, 5, 1], dtype=torch.float64)
    tensor2 = torch.rand([1, 5], dtype=torch.float32)
    value = 1
    print(torch.addcdiv(input, tensor1, tensor2, value=value))

result:  RuntimeError: !(has_different_input_dtypes && !config.promote_inputs_to_common_dtype_ && (has_undefined_outputs || config.enforce_safe_casting_to_output_ || config.cast_common_dtype_to_outputs_))INTERNAL ASSERT FAILED at "..\\aten\\src\\ATen\\TensorIterator.cpp":331, please report a bug to PyTorch. 


### Suggest a potential alternative/fix

Add input, tensor1, and tensor2 parameters of the same type.

cc @svekars @carljparker
torch.lobpcg should support black-box linear operators like SciPy
Labels: feature, triaged, module: linear algebra

### 🚀 The feature, motivation and pitch

Currently, the `torch.lobpcg` matrix-free eigensolver only supports fully materialized dense or sparse tensors as input. This makes it impossible to, for example, pass in a Hessian-vector product callable from `functorch` when you know that the Hessian of the function you're using will be positive definite. This is in contrast to [the SciPy implementation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.linalg.lobpcg.html), which allows the user to provide a black-box `LinearOperator` callable as input. Since the algorithm itself is matrix-free and only relies on matrix-vector products to compute its solution, there's no _algorithmic_ reason why support for black-box callables couldn't be added relatively easily AFAICT.

### Alternatives

_No response_

### Additional context

_No response_

cc @jianyuh @nikitaved @pearu @mruberry @walterddr @IvanYashchuk @xwang233 @Lezcano
`torch.nn.ReplicationPad2D` Report "invalid configuration argument" Error under Compute Sanitizer
Labels: module: nn, module: error checking, triaged, module: padding

### 🐛 Describe the bug

A test of `torch.nn.Replication2D` reports "invalid configuration argument" error when it's run under compute sanitizer. Without sanitizers it terminates normally on GPU.
Test:
```python
import torch

def test():
    arg_class = torch.nn.ReplicationPad2d([0,0,30,1024,-1,0])
    arg_tensor = torch.rand([1, 1, 3, 3], dtype=torch.float32).clone().cuda()
    arg = [arg_tensor,]
    res = arg_class(*arg)

test()
```
Error log:
```
========= COMPUTE-SANITIZER
========= Program hit cudaErrorInvalidConfiguration (error 9) due to "invalid configuration argument" on CUDA API call to cudaLaunchKernel.
========= 
========= Program hit cudaErrorInvalidConfiguration (error 9) due to "invalid configuration argument" on CUDA API call to cudaGetLastError.
========= 
========= ERROR SUMMARY: 2 errors
```

### Versions

```
PyTorch version: 1.14.0a0+gitbdc9911
Is debug build: False
CUDA used to build PyTorch: 11.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: 11.1.0-6
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov  2 2022, 18:53:38) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.6.124
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: 
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090

Nvidia driver version: 515.65.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.14.0a0+gitbdc9911
[pip3] torchvision==0.13.1
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               11.3.1               h2bc3f7f_2  
[conda] mkl                       2021.4.0           h06a4308_640  
[conda] mkl-service               2.4.0            py39h7f8727e_0  
[conda] mkl_fft                   1.3.1            py39hd3c417c_0  
[conda] mkl_random                1.2.2            py39h51133e4_0  
[conda] numpy                     1.21.5           py39he7a7128_1  
[conda] numpy-base                1.21.5           py39hf524024_1  
[conda] numpydoc                  1.2                pyhd3eb1b0_0  
[conda] torch                     1.14.0a0+gitce2f870          pypi_0    pypi
```

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @saketh-are
Dynamo + NNC: incorrect results with in-place ops on inputs
Labels: triaged, NNC, module: dynamo

### 🐛 Describe the bug

```python
import torch
import torch._dynamo

def fn(x, y):
    x += y
    return x


fn2 = torch._dynamo.optimize("nnc")(fn)
print(fn(torch.ones(1), torch.ones(1)))
print(fn2(torch.ones(1), torch.ones(1)))
```

prints:
```
tensor([2.])
tensor([6.])
```

Clearly the FX -> NNC conversion (through tracing?) is not sound wrt to in-place ops.

### Versions

1.14.0.dev20221117+cpu

cc @EikanWang @jgong5 @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
[BF16] Visit all the type cast from integer to BF16 type for potential accuracy loss
Labels: module: cpu, triaged, intel

## Issue description

BF16 cannot accurately represent all integer values outside the range of [-255,255]. When there is a cast from an integer value from BF16 data type, there would be potential accuracy loss. Typically, such a type cast is used in pixel index calculation in ops such as upsample, pooling etc. FP32 is preferred in these cases. See https://github.com/pytorch/pytorch/issues/88939 for a related functionality issue.

## Code example

`area_pixel_compute_scale` (https://github.com/jgong5/pytorch/blob/fc60a1865eafc985217eccc0251f82014041e6a7/aten/src/ATen/native/UpSample.h#L263) is one of the examples and used in a lot of places. There are also direct usage of `static_cast<scalar_t>` from integer to bf16, such as https://github.com/jgong5/pytorch/blob/fc60a1865eafc985217eccc0251f82014041e6a7/aten/src/ATen/native/FractionalMaxPool2d.cpp#L139


cc @VitalyFedyunin @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10
[Inductor] [CPU] accuracy failure in torchbench model detectron2_fcos_r_50_fpn
Labels: triaged, bug, oncall: pt2, module: cpu inductor

### 🐛 Describe the bug

This failure was found in latest **TorchInductor CPU Performance Dashboard** refresh test in https://github.com/pytorch/pytorch/issues/93531

SW information

SW	| Nightly commit	| Master/Main commit
----|--------|-----
Pytorch	|[637228b](https://github.com/blzheng/pytorch/commit/637228bcc4d2566fb617bbf1c4abeff69b3bdae7)	|[46796fe](https://github.com/blzheng/pytorch/commit/46796fe5e9b74602d45927304773fdcda1c3215a)
Torchbench|	/	|[022dfe3](https://github.com/pytorch/benchmark/commit/022dfe3d89db9278119bf1cd179997076b5ec5b5)
torchaudio	|[4b10b6a](https://github.com/pytorch/audio/commit/4b10b6adece473ad70e221bb3f9442d11906111f)	|[74f9a89](https://github.com/pytorch/audio/commit/74f9a894fcd4e7d635919803b364e47577a2b6e8)
torchtext	|[71e4561](https://github.com/pytorch/text/commit/71e456119d39cf72a511ce6acb10b737e02102d3)	|[c047efe](https://github.com/pytorch/text/commit/c047efeba813ac943cb8046a49e858a8b529d577)
torchvision	|[797e1ac](https://github.com/pytorch/vision/commit/797e1acf043621c5b53319032a269cbe1190b018)	|[ffd5a56](https://github.com/pytorch/vision/commit/ffd5a567eb90abf6b5555063da434d3c130d540f)

detail info reference the [Dashboard](https://github.com/pytorch/pytorch/issues/93531) 

### Error logs

`RuntimeError: Storage size calculation overflowed with sizes=[2, 3, 140639896720224, 140639896720384]`

### Minified repro

`python benchmarks/dynamo/torchbench.py --accuracy --float32 -dcpu -n50 --inductor  --no-skip --dashboard --only=detectron2_fcos_r_50_fpn --batch_size 2 `

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Collective operations do not work with `torch.BoolTensor`s on `gloo` and raise `Invalid scalar type`
Labels: oncall: distributed

### 🐛 Describe the bug

Hi, collective operations do not work with `torch.BoolTensor`s on backend `gloo`. Actually `GENERATE_ALL_TYPES` macro in `torch/csrc/distributed/c10d/ProcessGroupGloo.cpp` seems to raise `RuntimeError: Invalid scalar type`.

To reproduce:
```python
import torch.distributed as dist

tensor = torch.empty((5,),dtype=torch.bool)
if dist.get_rank() == 0:
    gather_list = [torch.empty((5,),dtype=torch.bool) for _ in range(dist.get_world_size())]
else:
    gather_list = None
dist.gather(tensor, gather_list)
```
This is also the case for `all_gather`, `reduce`, `all_reduce` and `broadcast`. Changing `dtype=torch.bool` to `dtype=torch.uint8` resolves the error, while `torch.bool == torch.uint8` evaluates to `True`.

### Versions

PyTorch version: 1.13.0.dev20220927+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.10 (default, Jun 22 2022, 20:18:18)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.4.0-126-generic-x86_64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: GeForce 820M
Nvidia driver version: 340.108
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.991
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.5
[pip3] pytorch-fid==0.1.1
[pip3] pytorch-ignite==0.4.10
[pip3] pytorch-sphinx-theme==0.0.24
[pip3] torch==1.13.0.dev20220927+cpu
[pip3] torch-model-archiver==0.6.0
[pip3] torch-tb-profiler==0.4.0
[pip3] torch-workflow-archiver==0.2.4
[pip3] torchaudio==0.13.0.dev20220923+cpu
[pip3] torchfile==0.1.0
[pip3] torchserve==0.6.0
[pip3] torchvision==0.14.0.dev20220926+cpu
[conda] Could not collect

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu
[aot-autograd] [hf_BigBird] Output 0 of CompiledFunctionBackward is a view and is being modified inplace
Labels: triaged, ezyang's list, bug, module: aotdispatch

### 🐛 Describe the bug

Following repro fails.  

cc @ezyang @bdhirsh @Chillee 

### Error logs

~~~
Traceback (most recent call last):
  File "/scratch/anijain/work/pytorch/failures/bigbird.py", line 23, in <module>
    opt_fn(a, b).sum().backward()
  File "/scratch/anijain/work/pytorch/torch/_dynamo/eval_frame.py", line 174, in _fn
    return fn(*args, **kwargs)
  File "/scratch/anijain/work/pytorch/failures/bigbird.py", line 15, in fn
    c.unsqueeze_(2)
RuntimeError: Output 0 of CompiledFunctionBackward is a view and is being modified inplace. This view was created inside a custom Function (or because an input was returned as-is) and the autograd logic to handle view+inplace would override the custom backward associated with the custom Function, leading to incorrect gradients. This behavior is forbidden. You can fix this by cloning the output of the custom Function.
~~~

### Minified repro

~~~
import torch
import torch._dynamo


def torch_bmm_nd(inp_1, inp_2, ndim=4):
    """Fast nd matrix multiplication"""
    # faster replacement of torch.einsum ("bhqk,bhkd->bhqd")
    return torch.bmm(inp_1.reshape((-1,) + inp_1.shape[-2:]), inp_2.reshape((-1,) + inp_2.shape[-2:])).view(
        inp_1.shape[: ndim - 2] + (inp_1.shape[ndim - 2], inp_2.shape[ndim - 1])
    )

@torch._dynamo.skip
def fn(x, y):
    c = torch_bmm_nd(x, y)
    c.unsqueeze_(2)
    return c

a = torch.randn(torch.Size([2, 12, 64, 1024]), requires_grad=True)
b = torch.randn(torch.Size([2, 12, 1024, 64]), requires_grad=True)
ref = fn(a, b).sum().backward()

opt_fn = torch._dynamo.optimize("aot_eager")(fn) 
opt_fn(a, b).sum().backward()
~~~
[feature request] Add ability to preserve traced shape during torch.jit.save and torch.jit.load
Labels: oncall: jit

## Issue description

I would like to be able to access the shape information of a traced model after loading the traced version of a model. It is my understanding that this isn't always desirable so to preserve current workflows, I propose adding an optional parameter that defaults to false in torch.jit.load that will allow this information to be restored. I also understand that this can add more data to the saved model so I think that there should be a parameter that allows this information to be saved, but that it default to True so that this feature will work when loading models by default as in most models the extra space should be negligible.

## Code example
I would like to support the below example

 ```
import os
 import torch
 from torchvision.models import resnet50, ResNet50_Weights

 model=resnet50(weights=ResNet50_Weights.IMAGENET1K_V1)
 traced = torch.jit.trace(model, torch.rand(1, 3, 224, 224))
 traced.forward(torch.rand(1,3,224,224))
 input_sizes = [ x.type().sizes() for x in traced.graph.inputs() ]
 torch.jit.save(traced, "/tmp/resnet50pt")
 loaded = torch.jit.load("/tmp/resnet50pt", _retrace=True)
 loaded_input_sizess  = [ x.type().sizes() for x in loaded.graph.inputs() ]
 assert loaded_inputs == inputs
```

## System Info
This shouldn't be relevant in this case.

## Extra Notes
I'm already working on implementing this, just wanted to confirm that this seemed reasonable


cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
Complex Not Supported in Torchinductor
Labels: triaged, bug, oncall: pt2

### 🐛 Describe the bug

A linear layer with complex valued weights (e.g. nn.Linear(inp_size, out_size, dtype=torch.complex64)) produces the following NotImplementedError with `dynamo.optimize()(model)`

### Error logs

```
Traceback (most recent call last):
  
  File "*/lib/python3.10/site-packages/torch/_dynamo/eval_frame.py", line 169, in _fn
    return fn(*args, **kwargs)
  File "*/lib/python3.10/site-packages/functorch/_src/aot_autograd.py", line 951, in forward
    return compiled_f(
  File "*/lib/python3.10/site-packages/functorch/_src/aot_autograd.py", line 937, in new_func
    compiled_fn = create_aot_dispatcher_function(
  File "*/lib/python3.10/site-packages/functorch/_src/aot_autograd.py", line 657, in create_aot_dispatcher_function
    aot_dispatch_autograd(flat_fn, fake_flat_tensor_args, aot_config)
  File "*/lib/python3.10/site-packages/functorch/_src/aot_autograd.py", line 501, in aot_dispatch_autograd
    fw_module, bw_module = aot_config.partition_fn(fx_g, joint_inputs)
  File "*/lib/python3.10/site-packages/functorch/_src/partitioners.py", line 420, in min_cut_rematerialization_partition
    weight = get_node_weight(node)
  File "*/lib/python3.10/site-packages/functorch/_src/partitioners.py", line 382, in get_node_weight
    mem_sz = _size_of(node)
  File "*/lib/python3.10/site-packages/functorch/_src/partitioners.py", line 225, in _size_of
    return _tensor_nbytes(to_size_hint(val.numel()), val.dtype)
  File "*/lib/python3.10/site-packages/functorch/_src/partitioners.py", line 206, in _tensor_nbytes
    raise NotImplementedError("Don't know the size of dtype ", dtype)
NotImplementedError: ("Don't know the size of dtype ", torch.complex64)
```

### Minified repro

_No response_

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Support disallowing calls to certain instance methods in TorchDynamo
Labels: triaged, module: dynamo

### 🚀 The feature, motivation and pitch

Currently TorchDynamo supports disallowing functions in captured graphs through `torch._dynamo.disallow_in_graph`. However, there is no way to disallow instance methods, for example, `Tensor.copy_`. 

It would be a useful feature to exclude method calls as well, so that the model can be optimized by the backend even if the backend lacks support for some ops used in the model. Currently I have to patch the internal class of TorchDynamo, like
```py
import functools

import torch
import torch._dynamo

torch._dynamo.disallow_in_graph([torch.sub, torch.Tensor.__setitem__, torch.Tensor.copy_])


def backend(gm, example_inputs):
    print(gm.graph)
    return gm.forward


@torch._dynamo.optimize(backend)
def fn(x, y):
    x = torch.add(x, 1)
    x[0:2] = torch.zeros(2)
    x = torch.add(x, 2)
    x = torch.sub(x, 0)
    x = torch.add(x, 3)
    x.copy_(y)
    x = torch.add(x, 4)
    return x


fn(torch.ones(8), torch.zeros(8))


def _patch():
    cls = torch._dynamo.variables.tensor.TensorVariable
    old_method = cls.call_method
    @functools.wraps(old_method)
    def call_method(self, tx, name, args, kwargs):
        if name == "__setitem__" or name == "copy_":
            raise torch._dynamo.exc.Unsupported("Tensor __setitem__ not supported")
        return old_method(self, tx, name, args, kwargs)

    cls.call_method = call_method


_patch()
print("TensorVariable call_method patched")

torch._dynamo.reset()
fn(torch.ones(8), torch.zeros(8))
```

cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire @junrushao

### Alternatives

_No response_

### Additional context

_No response_
[FSDP] Investigate Unit Testing when Gradient Computation Differs on CPU/GPU
Labels: oncall: distributed, triaged, module: fsdp

Consider the following unit test (relying on some imports from `common_fsdp.py`):
```
def test_cpu_gpu_parity(self):
    cpu_model = TransformerWithSharedParams.init(
        self.process_group,
        FSDPInitMode.NO_FSDP,
        CUDAInitMode.CUDA_NEVER,
        deterministic=True,
    )
    gpu_model = copy.deepcopy(cpu_model).cuda()
    cpu_inp = cpu_model.get_input(torch.device("cpu"))
    gpu_inp = gpu_model.get_input(torch.device("cuda"))
    for t1, t2 in zip(cpu_inp, gpu_inp):
        assert torch.equal(t1, t2.cpu())  # same input except device
    for p1, p2 in zip(cpu_model.parameters(), gpu_model.parameters()):
        assert torch.equal(p1, p2.cpu())  # same parameters except device
    cpu_out = cpu_model(*cpu_inp)
    cpu_out.sum().backward()
    gpu_out = gpu_model(*gpu_inp)
    gpu_out.sum().backward()
    for p1, p2 in zip(cpu_model.parameters(), gpu_model.parameters()):
        assert torch.equal(p1, p2.cpu())
        assert torch.equal(p1.grad, p2.grad.cpu()), f"{torch.linalg.vector_norm(p1.grad - p2.grad.cpu())}"
```
This fails on the last line:
```
assert torch.equal(p1.grad, p2.grad.cpu()), f"{torch.linalg.vector_norm(p1.grad - p2.grad.cpu())}"
AssertionError: 2.398389005975332e-05
```
This slight difference in the computed gradients makes downstream testing infeasible (e.g. testing `clip_grad_norm_()`). This is relevant when testing DDP and FSDP parity when only FSDP is offloading parameters to CPU (since DDP does not support that natively).

We need to find a workaround for this for the unit testing coverage.

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501
torch.normal(...) on MPS sometimes produces NaN's
Labels: triaged, module: mps

### 🐛 Describe the bug

PyTorch: 1.14.0.dev20221020. 

Calling torch.normal(...) on MPS (Apple M1 Pro chip) sometimes produces nan's. I have found the issue exists with a variety of means and std's. Problem does not seem to occur when generating the tensor on the CPU then moving to MPS. 

Minimal example:

```
import torch

mps = torch.device('mps')
cpu = torch.device('cpu')

for i in range(100):
    print(torch.any(torch.isnan(torch.normal(0, 1, size=(1000, 1000), device=mps))))
# occasionally prints True

for i in range(100):
    print(torch.any(torch.isnan(torch.normal(0, 1, size=(1000, 1000), device=cpu).to(mps))))
# doesn't print True
```




### Versions

Collecting environment information...
PyTorch version: 1.14.0.dev20221020
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.6 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.102)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.6 (main, Sep  9 2022, 11:31:09) [Clang 13.1.6 (clang-1316.0.21.2.5)] (64-bit runtime)
Python platform: macOS-12.6-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.910
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.23.2
[pip3] torch==1.14.0.dev20221020
[pip3] torchaudio==0.13.0.dev20220911
[pip3] torchinfo==1.7.0
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.14.0.dev20220911
[conda] Could not collect


cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
binary_cross_entropy/bce_with_logits (+ other loss functions) for nested_tensor
Labels: module: loss, triaged, module: nestedtensor

### 🚀 The feature, motivation and pitch

For multi-task learning, it could be useful to have loss functions support nested_tensors. Some of the reductions might not be so useful for some loss functions as currently implemented since they seem to assume regularly sized inputs, but at least something could be implemented when reduce="none".

```
import torch
from torch.nn.functional import binary_cross_entropy

nt = torch.nested.nested_tensor([[1.0, 0.4, 0.2], [0.1]])
print(binary_cross_entropy(nt, nt, reduce="none"))
```

Current output

```
/home/frankier/edu/doc/bert_ordinal/nightlytorchplay/losstest.py:5: UserWarning: The PyTorch API of nested tensors is in prototype stage and will change in the near future. (Triggered internally at ../aten/src/ATen/NestedTensorImpl.cpp:177.)
  nt = torch.nested.nested_tensor([[1.0, 0.4, 0.2], [0.1]])
Traceback (most recent call last):
  File "/home/frankier/edu/doc/bert_ordinal/nightlytorchplay/losstest.py", line 7, in <module>
    print(binary_cross_entropy(nt, nt, reduction="none"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/frankier/edu/doc/bert_ordinal/nightlytorchplay/venv/lib/python3.11/site-packages/torch/nn/functional.py", line 3087, in binary_cross_entropy
    if target.size() != input.size():
       ^^^^^^^^^^^^^
RuntimeError: Internal error: NestedTensorImpl doesn't support sizes. Please file an issue on https://github.com/pytorch/nestedtensor
```

### Alternatives

If this were possible https://github.com/pytorch/pytorch/issues/89124 it might provide a good/easy workaround.

### Additional context

_No response_

cc @cpuhrsch @jbschlosser @bhosmer @drisspg @mikaylagawarecki
Zero-copy way to make flat tensor into a nested_tensor given a shape
Labels: triaged, module: nestedtensor

### 🚀 The feature, motivation and pitch

If it were possible to create a nested_tensor from a flat tensor with the correct layout, we could always work around elementwise functions which are not (yet) supported by nested_tensor e.g.

```
a_shape = [t.size() for t in a.unbind()]

tensor.nested.nested_tensor_from_values_as(elementwise_function(a.values()), a)

# OR

tensor.nested.nested_tensor_from_values(elementwise_function(a.values()), a_shape)
```

### Alternatives

Copying or trying to work with the flat tensor directly.

### Additional context

I am currently trying to use unsupported loss functions in their unreduced form

cc @cpuhrsch @jbschlosser @bhosmer @drisspg @mikaylagawarecki