gopikrsmscs/torch-issues huggingface dataset:
ValueError: args contained 2 None's after flattening. When exporting a ScriptModule or ScriptFunction, no args may be None because that breaks type propagation.
Labels: module: onnx, triaged, module: multi-headed-attention

### 🐛 Describe the bug

when I export model to ONNX I found the error `RuntimeError: ScalarType UNKNOWN_SCALAR is an unexpected tensor scalar type`.
then I debug 1 line to 1 line , I found `torch.nn.MultiheadAttention` thrown it !

I pick a small unit to debug it 
```python
import torch
from torch import Tensor, nn
d_model = 256
dropout = 0.0
n_heads = 8
model = nn.MultiheadAttention(d_model, n_heads, dropout=dropout) 

tgt = torch.randn(6, 1, 256, dtype=torch.float32)
q = torch.randn(6, 1, 256, dtype=torch.float32)
k = torch.randn(6, 1, 256, dtype=torch.float32)

args = (
    tgt, q, k
)

scripted_model = torch.jit.script(model)
scripted_model.save("model.pt")
torch.onnx.export(model=scripted_model, args=args, f="model.onnx", verbose=True,opset_version=17)
```
then an error thrown 
```sh
=========== Diagnostic Run torch.onnx.export version 2.1.0a0+b5021ba ===========
verbose: False, log level: 40
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Traceback (most recent call last):
  File "script_save_export_onnx.py", line 331, in <module>
    torch.onnx.export(model=scripted_model, args=args, f="model.onnx", verbose=True,opset_version=17)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 507, in export
    _export(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1567, in _export
    graph, params_dict, torch_out = _model_to_graph(
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1124, in _model_to_graph
    graph, params, torch_out, module = _create_jit_graph(model, args)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 968, in _create_jit_graph
    _check_flatten_did_not_remove(args, flattened_args)
  File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 956, in _check_flatten_did_not_remove
    raise ValueError(
ValueError: args contained 2 None's after flattening. When exporting a ScriptModule or ScriptFunction, no args may be None because that breaks type propagation.
```


### Versions

docker pull nvcr.io/nvidia/pytorch:23.07-py3
Some ONNX tests have been disabled because of new tensor.split signature
Labels: module: onnx, triaged, onnx-triaged

### 🐛 Describe the bug

#107484 adds a new drop_remainder=False attribute to tensor.split.
This causes some ONNX tests to fail, because the [corresponding op in ONNXScript](https://github.com/microsoft/onnxscript/blob/b6c5e3bb9729f939e979a41d8d724009df9f3b58/onnxscript/function_libs/torch_lib/ops/core.py#L6918) has the previous signature.

`torch.onnx._internal.diagnostics.infra.context.RuntimeErrorWithDiagnostic: Cannot find any perfect/nearest match of symbolic function for aten::split.Tensor,which should be registered under aten.split.Tensor.
`

Once the PR is merged, and the ONNX op script op is updated, I will fix this issue and re-enable the tests.

### Versions

Not needed

cc @seemethere @malfet @pytorch/pytorch-dev-infra
Create a new heuristic TD rule for failures coming from base commit of the pull requests
Labels: triaged, module: devx

Here is the thoughts behind this feature request:

* If a test is failing on the base commit, we might want to run it **last** on the PR knowing that it's going to fail.  This allows other failures to surface and prevent the situation where reverting a change surfaces other failures hidden by the broken trunk.
* ~~On the other hand, if a pull request is deliberately created to forward fix trunk failures, running base failures **first** makes sense as this help validate the fix faster.~~  However, this should be the minority because:
  * We revert first, ask question later
  * Even if the fix works and the failed tests pass right away.  It's the usual requirement to wait for the whole job finishes before merging, i.e. dynamo job has passed 

cc @ZainRizvi @kit1980 @clee2000
[MPS] Fix output stride holistically when input isn't contiguous
Labels: open source, release notes: mps, ciflow/mps

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #110120
* #109584
* #109574
* #109557

This is a long lasting bug, related issues: https://github.com/pytorch/pytorch/issues/107694, https://github.com/pytorch/pytorch/issues/94396 (and more).

In general, we want to gather cached contiguous input tensors so that we don't need to make non-contiguous input tensors contiguous before the mps graph is run, since making tensors contiguous is an inefficient operation. However, when the output has been pre-set a non-contiguous stride, which usually happens when the kernel is structured or in-place, the result becomes incorrect because the gathered input tensors are contiguous, thereby producing a contiguous output.

Therefore, this PR makes output tensor's stride contiguous when cached input tensors are gathered.

This is a POC PR which only applies the fix to `clamp` & `masked_fill_`. If it is feasible, we could think about how to apply it to every MPS Graph kernel.
Skip cuda kernel launch with torch.sum when dimension length is 0
Labels: module: performance, module: cuda, triaged

### 🚀 The feature, motivation and pitch

If there is a torch tensor, e.g. `a = torch.ones([1, 1000])` and I call `a.sum(dim=0)`, torch will launch a cuda kernel for this when it's really just a view. Torch should special case this or include it inside the compiler in `torch.compile` as it is leads to significant wins in certain settings.

### Alternatives

_No response_

### Additional context

_No response_

cc @ptrblck
torch._dynamo.reset() before and after running each test
Labels: ciflow/trunk, release notes: releng, module: dynamo, ciflow/inductor, keep-going

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #110107

Previously, running tests with PYTORCH_TEST_WITH_DYNAMO=1 ran some tests in eager-mode pytorch because the Dynamo cache filled up. We need to reset the Dynamo state before each test in order to actually run each test using Dynamo instead of non-deterministically falling back to eager-mode.

Test Plan:
- CI

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Dynamo tests in CI seem to not run at times
Labels: high priority, triaged, oncall: pt2

We landed https://github.com/pytorch/pytorch/pull/109427. ~10-20 commits later, The Dynamo CI shard consistently started failing (repro: `PYTORCH_TEST_WITH_DYNAMO=1 python test_ops.py -k test_out_warning__refs_cat_cpu`, [example logs](https://github.com/pytorch/pytorch/actions/runs/6315629259/job/17148986023)). We are not sure why this happened.

Possibly related:
- https://github.com/pytorch/pytorch/issues/107444

cc @ezyang @gchanan @kadeng @seemethere @malfet @pytorch/pytorch-dev-infra @msaroufim @wconstab @bdhirsh @anijain2305
GPT2ForSequenceClassification, LayoutLMForSequenceClassification: "torch._dynamo.exc.Unsupported: call_function BuiltinVariable(setattr) [HFPretrainedConfigVariable(), ConstantVariable(str), ConstantVariable(str)] {}"
Labels: triaged, oncall: pt2

Repro:
```
python benchmarks/dynamo/huggingface.py --bfloat16 --accuracy --inference --device cuda --export-aot-inductor  --only GPT2ForSequenceClassification
```

Error:
```
  File "/home/binbao/pytorch/torch/_dynamo/convert_frame.py", line 559, in _compile
    guarded_code = compile_inner(code, one_graph, hooks, transform)
  File "/home/binbao/pytorch/torch/_dynamo/utils.py", line 190, in time_wrapper
    r = func(*args, **kwargs)
  File "/home/binbao/pytorch/torch/_dynamo/convert_frame.py", line 481, in compile_inner
    out_code = transform_code_object(code, transform)
  File "/home/binbao/pytorch/torch/_dynamo/bytecode_transformation.py", line 1028, in transform_code_object
    transformations(instructions, code_options)
  File "/home/binbao/pytorch/torch/_dynamo/convert_frame.py", line 451, in transform
    tracer.run()
  File "/home/binbao/pytorch/torch/_dynamo/symbolic_convert.py", line 2103, in run
    super().run()
  File "/home/binbao/pytorch/torch/_dynamo/symbolic_convert.py", line 743, in run
    and self.step()
  File "/home/binbao/pytorch/torch/_dynamo/symbolic_convert.py", line 706, in step
    getattr(self, inst.opname)(inst)
  File "/home/binbao/pytorch/torch/_dynamo/symbolic_convert.py", line 1234, in STORE_ATTR
    .call_function(
  File "/home/binbao/pytorch/torch/_dynamo/variables/builtin.py", line 656, in call_function
    return super().call_function(tx, args, kwargs)
  File "/home/binbao/pytorch/torch/_dynamo/variables/base.py", line 306, in call_function
    unimplemented(f"call_function {self} {args} {kwargs}")
  File "/home/binbao/pytorch/torch/_dynamo/exc.py", line 176, in unimplemented
    raise Unsupported(msg)
torch._dynamo.exc.Unsupported: call_function BuiltinVariable(setattr) [HFPretrainedConfigVariable(), ConstantVariable(str), ConstantVariable(str)] {}
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
[inductor][Optimus]Improve logging for Optimus
Labels: fb-exported, module: inductor, ciflow/inductor

Summary: It is based on the diff D49340843. We add more logs for better debug and logging purposes.

Test Plan: N4248219

Differential Revision: D49422563




cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
sam: AssertionError at torch/_inductor/graph.py `assert isinstance(value, (TensorBox, sympy.Expr))`
Labels: triaged, oncall: pt2, module: inductor

Repro:
```
python benchmarks/dynamo/torchbench.py --bfloat16 --accuracy --inference --device cuda  --inductor  --only sam
```

Error:
```
  File "/home/binbao/pytorch/torch/_inductor/graph.py", line 464, in run
    return super().run(*args)
  File "/home/binbao/pytorch/torch/fx/interpreter.py", line 138, in run
    self.env[node] = self.run_node(node)
  File "/home/binbao/pytorch/torch/_inductor/graph.py", line 735, in run_node
    result = super().run_node(n)
  File "/home/binbao/pytorch/torch/fx/interpreter.py", line 195, in run_node
    return getattr(self, n.op)(n.target, args, kwargs)
  File "/home/binbao/pytorch/torch/_inductor/graph.py", line 674, in output
    assert isinstance(value, (TensorBox, sympy.Expr))
AssertionError
```

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305 @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
scatter_add: Mixing 0-dim and 1-dim tensors
Labels: module: docs, triaged, module: python frontend, module: edge cases

### 🐛 Describe the bug

scatter_add currently allows 0-dim and 1-dim tensors to be mixed together

```
import torch
op = torch.ops.aten.scatter_add.default

op(torch.zeros([1]), 0, torch.zeros([], dtype=torch.long), torch.ones([2]))
op(torch.zeros([]), 0, torch.zeros([2], dtype=torch.long), torch.ones([3]))

torch.zeros([1]).scatter_add_(0, torch.zeros([], dtype=torch.long), torch.ones([2]))
torch.zeros([]).scatter_add_(0, torch.zeros([2], dtype=torch.long), torch.ones([3]))
```

However, documentation explicitly says that all three tensors should have the same number of dimensions
https://pytorch.org/docs/stable/generated/torch.Tensor.scatter_add_.html

Code comments and error messages also state that the tensors should have the same dimensionality.

The behavior is caused by the use of `ensure_nonempty_dim` in line 73 here:
https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/ScatterGatherChecks.h#L66-L74
Notice code comment (line 66) and error message (line 74) stating number of dimensions must be the same

Is this behavior desired? If so, then we need to update code comment/error message/documentation.


### Versions

.

cc @svekars @carljparker @albanD
Devices API
Labels: feature, triaged, needs design, topic: new features, module: python frontend

### 🚀 The feature, motivation and pitch

Hi,

I would like to propose a new device API that is device-type (cuda, xla, npu, …) agnostic. For example, you could do:
```
torch.devices.count(type='cuda')
torch.devices.is_available(type='npu')
torch.devices.seed(42, type='xla')
```

Instead of:
```
torch.cuda.device_count()
torch_npu.npu.is_available()
torch_xla.set_rng_state(42)
```

This would greatly help with making 3rd party code device-type agnostic because, at the moment, each device type has its own API.

I would be happy to implement such an API if there is support from the community.

#108046 is loossly related and could be put under the same API namespace

(cc team: @AnthonyBarbier @HMellor)

### Alternatives

_No response_

### Additional context

_No response_

cc @albanD
Fix linalg_vector_norm ONNX export with wrong output dtype
Labels: module: onnx, triaged, open source, release notes: onnx

Up to now, the export of `aten::linalg_vector_norm` was wrongly giving as output dtype `torch.float32` although the input may be of other dtype. This does not match PyTorch behavior (e.g. see https://github.com/pytorch/pytorch/blob/23938640706214921c0c0c29b1ed4855a0536cd8/torch/_refs/linalg/__init__.py#L72 or https://github.com/pytorch/pytorch/blob/23938640706214921c0c0c29b1ed4855a0536cd8/aten/src/ATen/native/LinearAlgebra.cpp#L2713 - though the second one is quite unreadable to me, I would need to compile pytorch with `DEBUG=1` and use `TORCH_SHOW_DISPATCH_TRACE=1` to understand where the output is allocated).

Example of the issue:
```python
import torch
import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = nn.Linear(10, 20)
        self.eps = 1e-12
        self.dim = -1
        
    def forward(self, x):
        denom = x.norm(2.0, self.dim, keepdim=True)
        print("denom", denom.dtype)
        return denom

model = MyModel()
model = model.eval().to(torch.float16).to("cuda")

inp = torch.rand(8, 10, device="cuda", dtype=torch.float16)

with torch.no_grad():
    res = model(inp)

    torch.onnx.export(model, (inp,), f="normalize.onnx")
```

giving out
![image](https://github.com/pytorch/pytorch/assets/9808326/44b3ea0b-e1d2-4f23-a963-e8bc8e7cb9df)
with the current implementation, while giving
![image](https://github.com/pytorch/pytorch/assets/9808326/dc737a45-bcc1-464e-bce0-d68728033a79)
in this proposed PR.

WDYT @justinchuby @BowenBao? This is an issue for the ONNX export in fp16 for some models, where ORT later raise at load time the error `FAIL : Load model from speecht5_onnx/decoder_model.onnx failed:Type Error: Type parameter (T) of Optype (MatMul) bound to different types (tensor(float) and tensor(float16)` due to this bug.
test_complex_div_underflow_overflow: increase amount of input data
Labels: triaged, open source, topic: not user facing

This ensures that even for complex float and AVX512 there would be enough data to use vectorized path. Without this change
test test_complex_div_underflow_overflow_cpu_complex64 would use non-vectorized algorithm while
test test_complex_div_underflow_overflow_cpu_complex128 would use vectorized algorithm on s390x.

Same behaviour is happening on x86_64 with AVX2.
Add IWYU pragma for umbrella header
Labels: fb-exported

Summary:
Include cleaner tools would benefit from IWYU pragmas that define implementation headers and exported headers.   See for more information.
```
https://github.com/include-what-you-use/include-what-you-use/blob/master/docs/IWYUPragmas.md
```

Test Plan: sandcastle_pass

Differential Revision: D49632855
DISABLED test_tags_recomputed_rand (__main__.ActivationCheckpointingViaTagsTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo

Platforms: linux, rocm, slow

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_tags_recomputed_rand&suite=ActivationCheckpointingViaTagsTests) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17123686365).

Over the past 3 hours, it has been determined flaky in 15 workflow(s) with 45 failures and 15 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_tags_recomputed_rand`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `dynamo/test_activation_checkpointing.py`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[dynamo] Fix comparison between SymNodeVariable and ListVariable
Labels: triaged, open source, topic: not user facing, module: dynamo, ciflow/inductor

Fixes #109504

Shuffled around an if statement to cause mis-matching types with 'is' operator to return False instead of triggering unimplemented exception

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
[Dynamo][Test] reland testcase with state again
Labels: triaged, open source, topic: not user facing, module: dynamo

The PR https://github.com/pytorch/pytorch/pull/109713 was reverted due to a mistake. Reland the code again.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng @anijain2305 @PaliC
DISABLED test_tags_rand (__main__.ActivationCheckpointingViaTagsTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo

Platforms: linux, rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_tags_rand&suite=ActivationCheckpointingViaTagsTests) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17118349065).

Over the past 3 hours, it has been determined flaky in 10 workflow(s) with 30 failures and 10 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_tags_rand`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `dynamo/test_activation_checkpointing.py`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Grouped query attention

# Summary
Closed: #106730 in favor of this PR since the rebase would have been more difficult then just recreating.

There is still some edge cases I need to appropriately handle but this is how it could look.#109887 

## Design choices
This is currently implemented by looking at the "num heads" dimension, size(1) and determining if it can be repeat_interleaved to fill the query num_heads. 

Another option would be to pass in an explicit `groups` parameter to the top level SDPA and passing down.
DISABLED test_circular_dependencies (__main__.TestImports)
Labels: module: rocm, module: tests, triaged, module: flaky-tests, skipped

Platforms: rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_circular_dependencies&suite=TestImports) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17106258267).

Over the past 3 hours, it has been determined flaky in 3 workflow(s) with 3 failures and 3 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_circular_dependencies`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `test_testing.py`

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @mruberry
DISABLED test_tags_multiple_checkpoints (__main__.ActivationCheckpointingViaTagsTests)
Labels: triaged, skipped, oncall: pt2

Platforms: linux, rocm, slow

https://github.com/pytorch/pytorch/actions/runs/6303967068/job/17115576334

This test was disabled because it is failing on main branch ([recent examples](http://torch-ci.com/failure/dynamo%2Ftest_activation_checkpointing.py%3A%3AActivationCheckpointingViaTagsTests%3A%3Atest_tags_multiple_checkpoints)).

cc @ezyang @msaroufim @wconstab @bdhirsh @anijain2305
Update caffe2 with LLVM-18 API change to CreateMalloc/CreateFree
Labels: fb-exported, NNC, release notes: jit

Summary: https://reviews.llvm.org/D158861 and https://reviews.llvm.org/D159418 modified the APIs for `CreateMalloc` and `CreateFree` respectively from being based off of `CallInst` to `IRBuilderBase`. Use the new API under LLVM version guards

Test Plan: CI

Differential Revision: D49612136




cc @EikanWang @jgong5
Dataloader resetting with num_workers=1 and persistent_workers=True
Labels: module: dataloader, triaged

### 🐛 Describe the bug

I hit the following bug when initializing a pytorch Dataloader with `num_workers=1` and `persistent_workers=True`. It seems the Dataloader gets reset:

`UserWarning: Length of IterableDataset <ConstantLengthDataset object at 0x7f4f3d028790> was reported to be 55936 (when accessing len(dataloader)), but 69087 samples have been fetched. For multiprocessing data-loading, this could be caused by not properly configuring the IterableDataset replica at each worker.`

I also tried using the `worker_init_fn` as suggested, where `dataset.start` and `.end` are the start and end indices. It still hits the above warning: 

```
def worker_init_fn(worker_id):
    worker_info = torch.utils.data.get_worker_info()
    dataset = worker_info.dataset  # the dataset copy in this worker process
    overall_start = dataset.start
    overall_end = dataset.end
    # configure the dataset to only process the split workload
    per_worker = int(math.ceil((overall_end - overall_start) / float(worker_info.num_workers)))
    worker_id = worker_info.id
    dataset.start = overall_start + worker_id * per_worker
    dataset.end = min(dataset.start + per_worker, overall_end)
    logger.info(f"Initializing worker with split of data from {dataset.start} to {dataset.end}")

```

From the training perspective, I hit this error about every 2.46 epochs and the epochs get reset to 1 and then it continues training from there, but I have a suspicion it's skipping part of the data. I worry this might have affected training. 

### Versions

PyTorch version: 2.0.1+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

Versions of relevant libraries:
[pip3] numpy==1.24.4
[pip3] pytorch-lightning==2.0.8
[pip3] pytorch-ranger==0.1.1
[pip3] torch==2.0.1+cu118
[pip3] torch-optimizer==0.3.0
[pip3] torchdata==0.6.1
[pip3] torchmetrics==0.11.4
[pip3] torchtext==0.15.2+cpu
[pip3] torchvision==0.15.2+cu118
[pip3] triton==2.0.0
[pip3] triton-pre-mlir==2.0.0
[conda] Could not collect

cc @SsnL @VitalyFedyunin @ejguan @dzhulgakov
DISABLED test_raises_mesh_dim_less_than_2 (__main__.TestDeviceMeshGetItem)
Labels: oncall: distributed, module: flaky-tests, skipped

Platforms: linux

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_raises_mesh_dim_less_than_2&suite=TestDeviceMeshGetItem) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17104082187).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 2 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_raises_mesh_dim_less_than_2`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `distributed/_tensor/test_device_mesh.py`

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
[aotinductor] Pass TorchIR to AOTInductor
Labels: fb-exported, ciflow/trunk, module: inductor, module: dynamo, ciflow/inductor, release notes: inductor, module: export

Updates `_export.aot_compile` to pass a torch IR graph to inductor, allowing inductor to now run the pre_grad_passes, and reuse more of inductor's code. 
Also updates the API to only return the `so_path`, and not returning the exported program. The pytree call spec is now serialized and placed inside of the generated model code. When calling the model, because there is no c++ pytree implementation linked yet, we can access the call specs through `get_call_spec()`, and call pytree flatten/unflattenin python.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @avikchaudhuri @gmagogsfm @zhxchen17 @tugsbayasgalan
tan/tanh discrepancies with complex due to jiterator
Labels: module: cuda, triaged, module: jiterator

### 🐛 Describe the bug

There is perhaps a large discrepancy (requiring atol being increased to 3e-4) between the before+after #102427 implementation of tan/tanh on CUDA with complex values.

Repro:
```
import torch
x = torch.tensor(-7.8167-0.0451j, device='cuda:0')
torch.set_printoptions(precision=10)
print(torch.tan(x))
print(torch._foreach_tan([x])[0])
print(torch._foreach_tan([x.to(device="cpu")])[0])
```

Before:
![image](https://github.com/pytorch/pytorch/assets/31798555/0666808c-eb85-490d-b12e-51720a9f828c)

After:
![image](https://github.com/pytorch/pytorch/assets/31798555/193b312e-ff18-4b8e-9a65-3597f702c41f)

I happened to observe this when debugging why some tan tests were failing when I added new sample inputs to the foreach tests in https://github.com/pytorch/pytorch/pull/109402#discussion_r1333562975. 

### Versions

trunk

cc @ptrblck @mruberry @parth-desai @peterbell10
DISABLED test_tags_module (__main__.ActivationCheckpointingViaTagsTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo

Platforms: linux, slow

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_tags_module&suite=ActivationCheckpointingViaTagsTests) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17093971077).

Over the past 3 hours, it has been determined flaky in 2 workflow(s) with 6 failures and 2 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_tags_module`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `dynamo/test_activation_checkpointing.py`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Change signature of CompilerFn for register_backend decorator
Labels: triaged, open source, topic: not user facing, module: dynamo, ciflow/inductor

## Description
Add `...` to show that CompilerFn for custom backend could take additional options

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
Please offer packages with local version `torch==2.1.0+cpu` for macOS
Labels: module: build, oncall: releng, triaged

### 🚀 The feature, motivation and pitch

https://download.pytorch.org/whl/test/torch/ offers CPU builds of version 2.1.0 RC. For Intel Linux and Windows both packages are available with local version `2.1.0+cpu`. For macOS there is only version `2.1.0` without an appended local version.

This currently makes it very hard to add PyTorch as a dependency to a project managed with poetry if compatibility to both Linux and macOS is desired. In poetry it's currently not possible to prefer `2.1.0` over `2.1.0+cpu` (https://github.com/python-poetry/poetry/issues/7256) and there's no working way to configure packages from this same source for different systems with different local version numbers.

### Alternatives

Users could host the packages themselves (with controlled version numbers), but that is not possible in all situations.

### Additional context

Poetry is a very popular solution to manage dependencies of complex Python projects. It would be great if PyTorch worked well with it.

cc @malfet @seemethere
[aotinductor] support _scaled_dot_product_flash_attention fallback
Labels: module: inductor, ciflow/inductor

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* __->__ #110003

This PR supports _scaled_dot_product_flash_attention fallback kernel.
Note that in the abi_compatible mode, we retrieve outputs by passing
output argument pointers rather than relying on std::get.

It also fixes an issue related to dynamic shapes, where we wrongfully
query undefined dynamic symbols.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @kadeng @muchulee8 @aakhundov
DISABLED test_mandelbrot_numpy (__main__.MiscTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo

Platforms: asan, linux, mac, macos, rocm, slow

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_mandelbrot_numpy&suite=MiscTests) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17086622044).

Over the past 3 hours, it has been determined flaky in 37 workflow(s) with 111 failures and 37 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_mandelbrot_numpy`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `dynamo/test_misc.py`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
DISABLED test_tags_function_with_kwargs (__main__.ActivationCheckpointingViaTagsTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo

Platforms: linux, rocm, slow

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_tags_function_with_kwargs&suite=ActivationCheckpointingViaTagsTests) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17085019023).

Over the past 3 hours, it has been determined flaky in 8 workflow(s) with 24 failures and 8 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_tags_function_with_kwargs`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `dynamo/test_activation_checkpointing.py`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
DISABLED test_mandelbrot_numpy_dynamic_shapes (__main__.DynamicShapesMiscTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo

Platforms: asan, linux, mac, macos, rocm, slow

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/flakytest?name=test_mandelbrot_numpy_dynamic_shapes&suite=DynamicShapesMiscTests) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17085074563).

Over the past 3 hours, it has been determined flaky in 32 workflow(s) with 96 failures and 32 successes.

**Debugging instructions (after clicking on the recent samples link):**
DO NOT ASSUME THINGS ARE OKAY IF THE CI IS GREEN. We now shield flaky tests from developers so CI will thus be green but it will be harder to parse the logs.
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Click on the Test step of the job so that it is expanded. Otherwise, the grepping will not work.
3. Grep for `test_mandelbrot_numpy_dynamic_shapes`
4. There should be several instances run (as flaky tests are rerun in CI) from which you can study the logs.


Test file path: `dynamo/test_dynamic_shapes.py`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
add gradscaler on CPU
Labels: module: cpu, open source, module: amp (automated mixed precision), ciflow/trunk, release notes: distributed (sharded), ciflow/periodic, ciflow/inductor, ciflow/slow

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom):
* #109994
* __->__ #109993
* #109281



cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @mcarilli @ptrblck @leslie-fang-intel
Implmenet kthvalue for bfloat16 on CUDA
Labels: module: cuda, triaged, module: bfloat16

### 🚀 The feature, motivation and pitch

[`torch.kthvalue`](https://pytorch.org/docs/stable/generated/torch.kthvalue.html) isn't currently supported on CUDA for `bfloat16`:

```python
>>> import torch
>>> torch.tensor([1., 2., 3.], dtype=torch.bfloat16, device="cuda:0").kthvalue(1)
RuntimeError: "kthvalue_cuda" not implemented for 'BFloat16'
```


### Alternatives

_No response_

### Additional context

_No response_

cc @ptrblck
DISABLED test_tags_function_via_global_checkpoint (__main__.ActivationCheckpointingViaTagsTests)
Labels: triaged, module: flaky-tests, skipped, module: dynamo

Platforms: linux, rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/failure/test_tags_function_via_global_checkpoint) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17056185819).

Over the past 72 hours, it has flakily failed in 2 workflow(s).

**Debugging instructions (after clicking on the recent samples link):**
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Grep for `test_tags_function_via_global_checkpoint`


Test file path: `dynamo/test_activation_checkpointing.py`

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @chenyang78 @aakhundov @kadeng
DISABLED test_cat_addmm (__main__.TestMaxAutotune)
Labels: module: rocm, triaged, module: flaky-tests, skipped, module: inductor

Platforms: rocm

This test was disabled because it is failing in CI. See [recent examples](https://hud.pytorch.org/failure/test_cat_addmm) and the most recent trunk [workflow logs](https://github.com/pytorch/pytorch/runs/17058274254).

Over the past 72 hours, it has flakily failed in 9 workflow(s).

**Debugging instructions (after clicking on the recent samples link):**
To find relevant log snippets:
1. Click on the workflow logs linked above
2. Grep for `test_cat_addmm`


Test file path: `inductor/test_max_autotune.py`

cc @jeffdaily @sunway513 @jithunnair-amd @pruthvistony @ROCmSupport @dllehr-amd @jataylo @hongxiayang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
torch-<version>.dist-info WHEEL file contains incorrect metadata for M1/M2 macOS platform
Labels: oncall: binaries, oncall: releng, triaged, module: m1

### 🐛 Describe the bug

The `pip install` explicitly downloads `torch-<version>-cp310-none-macosx_11_0_arm64.whl`, however, the `WHEEL` metadata file has the tag: `Tag: cp310-cp310-macosx_11_0_x86_64`. Since the `arm64.whl` is downloaded, the tag should contain the `arm64` tag instead of the `x86_64` tag.

### System

```bash
Chip: Apple M2 Max
macOS: 13.5.1
```

### Steps to reproduce

```bash
python3 --version
# Python 3.10.12
python3 -m venv .venv
source .venv/bin/activate
pip install --no-cache-dir torch==2.0.1
# Collecting torch==2.0.1
# Downloading torch-2.0.1-cp310-none-macosx_11_0_arm64.whl (55.8 MB)
# ...
cat .venv/lib/python3.10/site-packages/torch-2.0.1.dist-info/WHEEL
# Wheel-Version: 1.0
# Generator: bdist_wheel (0.38.4)
# Root-Is-Purelib: false
# Tag: cp310-cp310-macosx_11_0_x86_64
```

The same behaviour is seen with `torch==1.13.1` as well. I haven't tested more versions, it could be present there as well.

### Versions

Collecting environment information...
PyTorch version: 2.0.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.5.1 (arm64)
GCC version: Could not collect
Clang version: 14.0.3 (clang-1403.0.22.14.1)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.12 (main, Aug 23 2023, 13:33:00) [Clang 14.0.3 (clang-1403.0.22.14.1)] (64-bit runtime)
Python platform: macOS-13.5.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M2 Max

Versions of relevant libraries:
[pip3] torch==2.0.1
[conda] Could not collect

Process finished with exit code 0

cc @seemethere @malfet
Dtype hard-coded in DataLoader (for python floats).
Labels: triaged, module: data

### 🐛 Describe the bug

The default dtype is not respected in `DataLoader` when Dataset `__getitem__()` returns a float. `DataLoader` uses `collate_float_fn` ([link to the source](https://github.com/pytorch/pytorch/blob/a683bc54fdc130537c1d3d81d7290ac9c49df3b2/torch/utils/data/_utils/collate.py#L179)) which has `torch.float64` hardcoded as follows: 


```python
def collate_float_fn(batch, *, collate_fn_map: Optional[Dict[Union[Type, Tuple[Type, ...]], Callable]] = None):
    return torch.tensor(batch, dtype=torch.float64)
```

Is there any reason to force it to `float64`? This seems inconsistent with other types that seem to respect `torch.set_default_dtype()`.

### Versions

PyTorch version: 2.0.1+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro N
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.10 | packaged by Anaconda, Inc. | (main, Mar 21 2023, 18:39:17) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.22621-SP0
Is CUDA available: False
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4070
Nvidia driver version: 535.98
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture=9
CurrentClockSpeed=3500
DeviceID=CPU0
Family=205
L2CacheSize=20480
L2CacheSpeed=
Manufacturer=GenuineIntel
MaxClockSpeed=3500
Name=13th Gen Intel(R) Core(TM) i5-13600K
ProcessorType=3
Revision=

Versions of relevant libraries:
[pip3] numpy==1.26.0
[pip3] torch==2.0.1
[conda] No relevant packages

cc @VitalyFedyunin @ejguan @dzhulgakov
[WIP] Make ONNX OpSchema function matcher more robust
Labels: module: onnx, open source, release notes: onnx

Make the algorithm for "finding the perfect/nearest matched OnnxFunction for the given FX node, arguments, and keyword arguments" more robust in the case when keyword arguments differ, but are set to the default value.

For example:
- fx node is: "split" (tensor,dim,drop_remainder=False (default value))
- ONNX op is: "split" (tensor,dim)

(see PR #107484 which adds an optional drop_remainder=False to torch.split). The [corresponding torch op in ONNX script](https://github.com/microsoft/onnxscript/blob/b6c5e3bb9729f939e979a41d8d724009df9f3b58/onnxscript/function_libs/torch_lib/ops/core.py#L6918) doesn't yet have the new attribute "drop_remainder", this causes the following error during ONNX dynamo_export:

`torch.onnx._internal.diagnostics.infra.context.RuntimeErrorWithDiagnostic: Cannot find any perfect/nearest match of symbolic function for aten::split.Tensor,which should be registered under aten.split.Tensor.`

(because of the attribute list mismatch between ONNX script torch_op and the Pytorch fx node).

This PR improves the lookup of a any perfect/nearest match of symbolic function, by ignoring keyword attributes that are set to a default value (which should be backward compatible).

Fixes https://github.com/pytorch/pytorch/issues/110131
WelfordReduction seems to have invalid/dead code when reduction_numel <= 1
Labels: triaged, module: inductor

I uncovered this trying to add types to `ir.py`. I'm not sure if this ever worked, but I'm pretty sure it doesn't successfully execute now due to the fact that it references an undefined `dst_dtype` local:

https://github.com/pytorch/pytorch/blob/b7a95f4fdb8a79dc459cc757dafcdbd0953b1a62/torch/_inductor/ir.py#L1179
https://github.com/pytorch/pytorch/blob/b7a95f4fdb8a79dc459cc757dafcdbd0953b1a62/torch/_inductor/ir.py#L1203

and also calls `const` with 1 instead of the required 2 arguments:

https://github.com/pytorch/pytorch/blob/b7a95f4fdb8a79dc459cc757dafcdbd0953b1a62/torch/_inductor/ir.py#L1189

Moreover, I wasn't able to find a test that triggered those code paths.

It seems like the code was added in 18b1c2907d672759a2b9c909004636b460fd9a5c. @peterbell10, would you mind taking a look to see if it's actually needed?

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov
How to compile torch 2.0.1 version from source?
Labels: oncall: releng, triaged

### 🐛 Describe the bug

While I was using 'git clone --branch v2.0.1 https://github.com/pytorch/pytorch.git  & python setup.py develop', and 'Building wheel torch-1.14.0a0+410ce96' version was being built. 

### Versions

I also checked the version.txt, it shows '2.0.0a0' which should be the version in v2.0.1 tag branch.

So how should I compile torch 2.0.1 version from source? Thanks!
Simple script segfaulting when grad is enabled
Labels: module: autograd, triaged, needs design, module: edge cases

### 🐛 Describe the bug

I was just toying around a bit with `torch` and came across this strange bug:

```python
import torch
print(torch.__version__)

FEATURE_SIZE = 768
BATCH_SIZE = 512
NUM_ITERS = 171
has_grad = True

with torch.set_grad_enabled(has_grad):
    torch.random.manual_seed(0)
    linear_layer = torch.nn.Linear(FEATURE_SIZE, FEATURE_SIZE, bias=True)
    input_tensor = torch.randn(BATCH_SIZE, FEATURE_SIZE)
    output = torch.zeros_like(input_tensor)

    for iter_num in range(NUM_ITERS):
        print(f"\rForward pass {iter_num}", end='')
        for batch_idx in range(BATCH_SIZE):
            output[batch_idx] = linear_layer(input_tensor[batch_idx])
```

This results in the following output on `torch==2.0.1`:
```
2.0.1
Forward pass 170[1]    59045 segmentation fault (core dumped)  python3 test.py
```
and also on recent nightly builds
```
2.2.0.dev20230912+cu121
Forward pass 170[1]    59424 segmentation fault (core dumped)  python3 test.py
```

In both cases, the script runs until after the last forward pass and then segfaults. On my setup, this happens for `NUM_ITERS>=171`. With `NUM_ITERS=170`, everything works fine. Similarly, setting `has_grad=False` also prevents the segfault.

### Versions

Versions of relevant libraries:
[pip3] numpy==1.24.1
[pip3] torch==2.0.1
[pip3] torchaudio==2.0.2
[pip3] torchvision==0.15.2

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Need latest NCCL support to reduce GPU HBM consumption
Labels: oncall: distributed, module: nccl

### 🚀 The feature, motivation and pitch

Since NCCL 2.18.1 opened a new API "ncclCommSplit" which allows communicators share GPU HBM on same GPU.
We want this feature support to run larger models on same GPU clusters.


### Alternatives

_No response_

### Additional context

_No response_

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu
Batching for is_in
Labels: triaged, module: functorch

### 🚀 The feature, motivation and pitch

I was trying to use torch.vmap for running is_in on a batch of vectors and got this:

UserWarning: There is a performance drop because we have not yet implemented the batching rule for aten::isin.Tensor_Tensor. Please file us an issue on GitHub so that we can prioritize its implementation. (Triggered internally at /opt/conda/conda-bld/pytorch_1682343997789/work/aten/src/ATen/functorch/BatchedFallback.cpp:82.)

So filing and issue because it told me to essentially

### Alternatives

_No response_

### Additional context

_No response_

cc @zou3519 @Chillee @samdow @kshitij12345 @janeyx99
Fix S367052 to unblock ICVR MC3
Labels: fb-exported, release notes: fx

Summary: Somehow "getitem" started to get Tensor starting from ads_ranking:996 and broke SDD pipelining FX-transformer. We need to skip the Tensor node in annotation.

Test Plan:
N4326037
with ads_ranking kernel
# Before
ads_ranking:v996
 {F1100009226}
# With this diff
 {F1100009310}

Differential Revision: D49567615
test test_2d_fsdp_integration_fsdp_nested_param_groups failed
Labels: oncall: distributed, triaged

### 🐛 Describe the bug

This test in `test_fsdp_2d_parallel.py` is trying to test 2D (TP + FSDP) with nested wrapping and multiple param groups. Somehow if I disable nested wrapping or comment the second param works, test all passed. 

We need to investigate why this happens and filed an issue to track it.

### Versions

PyTorch nightly build

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu