gopikrsmscs/torch-issues huggingface dataset:
Add local version identifier to wheel file names
Labels: module: build, triaged

### 🚀 The feature, motivation and pitch

Hello 👋  Thank you for maintaining pytorch 🙇 It's a joy to use and I'm very excited about the 2.0 release!
I've found a tiny quality of life improvement surrounding the precompiled wheel available at https://download.pytorch.org/whl/ .

Currently users are unable to tell `pip` that a specific compiled version of torch should be re-installed from a different whl.
For example; 

```
# Install torch
$ pip install 'torch>=1.13.0+cpu' --find-links https://download.pytorch.org/whl/cpu/torch_stable.html
Looking in indexes: https://alexander.vaneck:****@artifactory.paigeai.net/artifactory/api/pypi/pypi/simple
Looking in links: https://download.pytorch.org/whl/cpu/torch_stable.html
Collecting torch>=1.13.0+cpu
  Using cached https://artifactory.paigeai.net/artifactory/api/pypi/pypi/packages/packages/82/d8/0547f8a22a0c8aeb7e7e5e321892f1dcf93ea021829a99f1a25f1f535871/torch-1.13.1-cp310-none-macosx_10_9_x86_64.whl (135.3 MB)
...
Installing collected packages: typing-extensions, torch
Successfully installed torch-1.13.1 typing-extensions-4.5.0
# Then later install torch for cu116
$ pip install 'torch>=1.13.0+cu116' --find-links https://download.pytorch.org/whl/cu116/torch_stable.html
Looking in indexes: https://alexander.vaneck:****@artifactory.paigeai.net/artifactory/api/pypi/pypi/simple
Looking in links: https://download.pytorch.org/whl/cu116/torch_stable.html
Requirement already satisfied: torch>=1.13.0+cu116 in ./.virtualenvs/torch-links/lib/python3.10/site-packages (1.13.1)
Requirement already satisfied: typing-extensions in ./.virtualenvs/torch-links/lib/python3.10/site-packages (from torch>=1.13.0+cu116) (4.5.0)
# We now hold CPU torch for CUDA 11.6 enabled environment.
```

It would be mighty handy if the wheels provided would include the [local version identifiers](https://peps.python.org/pep-0440/#local-version-identifiers) for the environment they are built in, f.e. `torch-1.13.1+cu116-cp39-cp39-manylinux2014_aarch64.whl`. And as local version identifiers are optional it would have no impact on the current dependency resolution. 

### Alternatives

_No response_

### Additional context

_No response_

cc @malfet @seemethere
Differentiate with regard a subset of the input
Labels: feature, module: autograd, triaged

### 🚀 The feature, motivation and pitch

Hi,

When I want to differentiate with regard to the first element in a batch, I get an error, as shown in the following example:

```python
n, d = 10, 3
lin1 = nn.Linear(d, d)

# case 1
x = torch.randn(d).requires_grad_()
y = lin1(x)
vec = torch.ones(d)
gr = torch.autograd.grad(y, x, vec, retain_graph=True)[0]
print(gr.shape)  # returns: torch.Size([3])

# case 2
x = torch.randn(n, d).requires_grad_()
y = lin1(x)
vec = torch.ones(d)
gr = torch.autograd.grad(y[0], x[0], vec, retain_graph=True)[0]

# Raises Error:
# RuntimeError: One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.
```
so taking x[i] removes it from the computation graph, I tried to clone it or to use .narrow but in vain!

y[0] only depends on x[0], so I don’t want to compute the gradient with regard to the full input!

Is there a way to do this efficiently?

### Alternatives

_No response_

### Additional context

_No response_

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7
Default value of `validate_args` is set to `True` when passed as `None` in `Multinomial`
Labels: module: distributions, triaged

Running
https://github.com/pytorch/pytorch/blob/17d0b7f532c4b3cd4af22ee0cb25ff12dada85cb/torch/distributions/multinomial.py#L40-L41

produces
```
ValueError: Expected value argument (Tensor of shape (4,)) to be within the support (Multinomial()) of the distribution Multinomial(), but found invalid values:
tensor([2, 5, 2, 1])
```

I believe it is because the default value of `validate_args` is set to `True`.

cc @fritzo @neerajprad @alicanb @nikitaved
CosineAnnealingWarmRestarts but restarts are becoming more frequent
Labels: triaged, module: LrScheduler

I'm training a model on a large dataset (1 epoch is more than 3000 iterations). An ideal learning rate scheduler is CosineAnnealingWarmRestarts in my case but it would be better if it would make restarts more frequently as training goes further. I tried to pass 0.5 for `T_mult` but it didn't work. I got this error `ValueError: Expected integer T_mult >= 1, but got 0.5`. I think it can be implemented safely with integer division `T_0 // T_mult`. Is there any special reason not to implement this functionality?
Proposal: `@capture`: Unified API for capturing functions across `{fx, proxy_tensor, dynamo}`
Labels: module: onnx, feature, triaged, oncall: pt2, module: functorch, module: dynamo

> ```python
> # replaces `@fx.wrap`
> @fx.capture 
> def my_traced_op(x):
>   pass
> 
> # new API, similar to `@fx.wrap`. `make_fx` will capture this function.
> @proxy_tensor.capture 
> def my_traced_op(x):
>   pass
> 
> # replaces `dynamo.allow_in_graph`
> @dynamo.capture 
> def my_traced_op(x):
>   pass
> ```
> 
> Can we break backwards compatibility on `wrap`? It is a terrible API due to its name. 
> `capture` could be the preferred new API, and `wrap` will simply alias it, then we can deprecate `wrap` slowly.

_Originally posted by @jon-chuang in https://github.com/pytorch/pytorch/issues/94461#issuecomment-1435530121_
            
Examples of user demand for custom op tracing:
1. https://github.com/pytorch/pytorch/issues/95021
2.  https://github.com/pytorch/pytorch/pull/94867

Question: is there overlap between `@dynamo.capture` and `@proxy_tensor.capture`? Whether the fx graph traced by dynamo is traced again by `make_fx` depends on the backend (e.g. `aot_eager`, `inductor`). So I suppose that to trace an op through `torch.compile` and into a backend like aot_eager which uses `make_fx`, one should wrap with both, correct? @jansel 

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @zou3519 @Chillee @samdow @kshitij12345 @janeyx99 @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
AOTAutograd can add extra as_strided() calls when graph outputs alias inputs
Labels: module: autograd, triaged, module: functionalization, oncall: pt2

A simple example:

```
import torch

@torch.compile(backend="aot_eager")
def f(x):
    return x.unsqueeze(-1)

a = torch.ones(4)
b = a.view(2, 2)
out = f(b)
```

If you put a breakpoint at [this line](https://github.com/pytorch/pytorch/blob/c16b2916f15d7160c0254580f18007eb0c373abc/torch/_functorch/aot_autograd.py#L498) of aot_autograd.py, you'll see it getting hit.

Context: When AOTAutograd sees a graph with an output that aliases an input, it tries to "replay" the view on the input in eager mode, outside of the compiled `autograd.Function` object. It makes a best effort to replay the exact same set of views as the original code, but it isn't always able to.

The reason it can't in this case is because the input to the graph, `b`, is itself a view of another tensor, `a`. Morally, all we want to do is replay the call to `b.unsqueeze()`. Autograd has some view replay logic, but it requires us to replay *all* of the views from `out`'s base, including the views from the base (a) to the graph input (b). Since that view was created in eager mode, the view that autograd records is an as_strided, so we're forced to run an as_strided() call as part of view replay.

as_strided has a slower backward formula compared to most other view ops. If we want to optimize this, there are a few things that we can do:

(1) Wait for `view_strided()`: @dagitses is working on a compositional view_strided() op that we should be able to replace this as_strided() call with, that should have a more efficient backward formula

(2) make autograd's view replay API smarter (`_view_func`). What we could do is allow for you to do "partial view-replay". E.g. in the above example, if I ran `out._view_func(b)`, it would be nice if it just performed a single call to `.unsqueeze()` instead of needing to go through the entire view chain.

(3) Manually keep track of view chains in AOTAutograd. This would take more work, but in theory we could track all the metadata we need on exactly what the chain of view ops from an input to an output is during tracing, and manually replay it at runtime, instead of relying on autograd's view-replay.

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @soumith @msaroufim @wconstab @ngimel @davidberard98
test_torchinductor.py test isolation problems
Labels: triaged, module: inductor

### 🐛 Describe the bug

In this CI run https://hud.pytorch.org/pytorch/pytorch/pull/95063?sha=c059823d525078a0fe7e8f96040a78c6ec3b42c3 you can see that test_upsample_nearest3d_cpu has failed. Actually, it has nothing to do with this test; if you run that test locally it will pass. It turns out that way earlier there were some failures in test_inplace_unsqueeze, and then somehow this triggered a bunch of downstream tests to fail. This is not great.

I tried inducing the cascading failure with `pytest test/inductor/test_torchinductor.py -k 'test_inplace_unsqueeze or test_inplace_unsqueeze2 or test_inplace_unsqueeze3 or test_baddbmm_cpu' -v` but baddbm_cpu didn't fail that way. So not sure how to nail this.

Branch for permanent repro at https://github.com/ezyang/pytorch/pull/new/cascading-error

### Versions

master

cc @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
Implement a `torch.cuda.visible_device_indexes` function. 
Labels: module: cuda, triaged

### 🚀 The feature, motivation and pitch

A `visible_device_indexes` function should be added to the `torch.cuda` module. This function's implementation would be nearly identical to how `torch.cuda.device_count` is implemented, but is not cached, and would not include the call to `len` at the end. The implementation of `torch.cuda.device_count` would then call `len()` on this new `visible_device_indexes` function and could be cached up to the other issue described in #95073.

I would love to implement this myself to contribute to Pytorch if the team would allow it! 

### Alternatives

Happy to iterate on design here.

### Additional context

I have a version of this function implemented currently, but thought it really would belong better in the official Pytorch package since it really is nearly identical to the `device_count` already existing withinside `torch.cuda`.

cc @ngimel
Make artifacts easier to discover on HUD
Labels: module: ci, triaged

## Feature request:

See description in https://github.com/pytorch/pytorch/issues/95075#issuecomment-1435171782

---

## Original request:

See this job run for reference.
https://github.com/pytorch/pytorch/actions/runs/4196727169

The main issues are
1) discoverability: 

i found it straightforward to identify the perf job and open it up, look for the 'upload' step of the job and read the logs.  I found filenames here, but was hoping for download URLs or some instructions about where to look for them.
<img width="1948" alt="image" src="https://user-images.githubusercontent.com/4984825/219737660-bee902a9-19da-47f8-93a8-9f4185401587.png">

2) When I assumed the artifacts are hosted in github (as opposed to S3 bucket or something), I still can't find them
* I followed this instruction: https://docs.github.com/en/actions/managing-workflow-runs/downloading-workflow-artifacts
* But when I clicked around in the github UI I never saw the 'artifacts' pane, and assumed maybe we weren't using gh artifacts at all.
* The issue is that the artifacts pane renders below some warnings so it does not show up unless you scroll down.


Not sure what to do about this, but perhaps putting some message in the job log step saying where to look?  (or, if you can generate a download URL to throw in the log even better!) 

cc @seemethere @malfet @pytorch/pytorch-dev-infra
A100 Perf Job artifact zipfiles unzip to generic folder that loses job information
Labels: module: ci, triaged

<img width="754" alt="image" src="https://user-images.githubusercontent.com/4984825/219733418-4359a47f-fcd0-4da2-b443-f5b856e794b5.png">

My use case is probably typical: i want to compare perf on my PR with perf on a baseline, so I may download 8 artifact zips.
* 4 zips per job (one per benchmark suite)
* 2 jobs (baseline and PR)

The first issue is that once i double click one of the downloaded zips, it generates a folder that has a generic name like `test-reports` or `test-reports2` etc.  Now when i'm working with the files I'm getting confused.

Luckily the .csv files inside the folders bear the name of the benchmark suite, but they do not bear the name of the PR or the baseline, so I can't actually tell what is what unless I am very careful to rename the `test-reports` folders as i unzip them.

It's probably a good idea to be as redundant as possible here, and put the info about the PR/job AND benchmark suite into both the unzipped folder AND csv filenames, just so no matter what workflow or scripts people write, the info is easy to find.



cc @seemethere @malfet @pytorch/pytorch-dev-infra
[Inductor] [CPU] Huggingface model AllenaiLongformerBase performance regression > 10% on ww07.4
Labels: triaged, module: inductor, module: cpu inductor

### 🐛 Describe the bug

There is a performance regression on Huggingface model AllenaiLongformerBase  on ww07.4 [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531#issuecomment-1434019577)  compare with ww07.2 as bellow:

|  ww07.4   |   |     |   |  ww07.2   |   |     |   | Comp  |     |   |
|  ----  | ----  |  ----  | ----  |  ----  | ----  |  ----  | ----  | ----  |  ----  | ----  |
| batch_size  | speedup | inductor  | eager | batch_size  | speedup | inductor  | eager | speedup ratio | eager ratio  | inductor ratio |
|1	|0.9901	|2.3819678	|2.358386319	|1	|1.1392	|2.0668015	|2.354500269	|0.87	|1	|0.87

Please find sw info from [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531) 


Graph dump by cosim:

WW07.4:
[graph.txt](https://github.com/pytorch/pytorch/files/10765350/graph.txt)

WW07.2:
[graph.txt](https://github.com/pytorch/pytorch/files/10765352/graph.txt)

Repro:

```
python -m torch.backends.xeon.run_cpu --core_list 0 --ncores_per_instance 1 benchmarks/dynamo/huggingface.py --performance --float32 -dcpu -n50 --inductor  --no-skip --dashboard --only AllenaiLongformerBase  --cold_start_latency --batch_size 1 --threads 1
```

cc @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
[Inductor] [CPU] Huggingface model MT5ForConditionalGeneration &T5ForConditionalGeneration & T5Small performance regression > 10% on ww07.4
Labels: triaged, module: inductor

### 🐛 Describe the bug
There is a performance regression on Huggingface models **MT5ForConditionalGeneration &T5ForConditionalGeneration & T5Small** on ww07.4 [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531#issuecomment-1434019397)  compare with ww07.2 as bellow:

| |  ww07.4   |   |     |   |  ww07.2   |   |     |   | Comp  |     |   |
| ---- |  ----  | ----  |  ----  | ----  |  ----  | ----  |  ----  | ----  | ----  |  ----  | ----  |
| name | batch_size  | speedup | inductor  | eager | batch_size  | speedup | inductor  | eager | speedup ratio | eager ratio  | inductor ratio |
|MT5ForConditionalGeneration	|16	|1.0417	|0.2628685	|0.273830116	|16	|1.1846	|0.2305143	|0.27306724	|0.88	|1	|0.88
|T5ForConditionalGeneration	|4	|0.7722	|0.5160617	|0.398502845	|4	|1.0301	|0.3849303	|0.396516702	|0.75	|1	|0.75
|T5Small	|4	|0.7736	|0.5160818	|0.39924088	|4	|1.0333	|0.3849365	|0.397754885	|0.75	|1	|0.75

Please find sw info from [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531) 

Graph dump by cosim:

WW07.4:
[MT5ForConditionalGeneration graph.txt](https://github.com/pytorch/pytorch/files/10765405/graph.txt)
[T5ForConditionalGeneration graph.txt](https://github.com/pytorch/pytorch/files/10765393/graph.txt)
[T5Small graph.txt](https://github.com/pytorch/pytorch/files/10765363/graph.txt)

WW07.2:
[MT5ForConditionalGeneration graph.txt](https://github.com/pytorch/pytorch/files/10765406/graph.txt)
[T5ForConditionalGeneration graph.txt](https://github.com/pytorch/pytorch/files/10765396/graph.txt)
[T5Small graph.txt](https://github.com/pytorch/pytorch/files/10765367/graph.txt)

Repro:

```
python -m torch.backends.xeon.run_cpu --node_id 0 benchmarks/dynamo/huggingface.py --performance --float32 -dcpu  -n50 --inductor  --no-skip --dashboard --only MODEL  --cold_start_latency
```

cc @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
[Inductor] [CPU] Torchbench model hf_Longformer performance regression > 10% on ww07.4
Labels: triaged, module: inductor

### 🐛 Describe the bug

There is a performance regression on Torchbench model hf_Longformer on ww07.4 [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531#issuecomment-1434019577)  compare with ww07.2 as bellow:

|  ww07.4   |   |     |   |  ww07.2   |   |     |   | Comp  |     |   |
|  ----  | ----  |  ----  | ----  |  ----  | ----  |  ----  | ----  | ----  |  ----  | ----  |
| batch_size  | speedup | inductor  | eager | batch_size  | speedup | inductor  | eager | speedup ratio | eager ratio  | inductor ratio |
|1	|1.0426	|1.0742891	|1.120053816	|1	|1.1938	|0.943502	 |1.126352688	|0.87	|1.01	|0.88

Please find sw info from [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531) 

Graph dump by cosim:

WW07.4:
[graph.txt](https://github.com/pytorch/pytorch/files/10765423/graph.txt)


WW07.2:
[graph.txt](https://github.com/pytorch/pytorch/files/10765435/graph.txt)

Repro:

```
python -m torch.backends.xeon.run_cpu --core_list 0 --ncores_per_instance 1 benchmarks/dynamo/torchbench.py --performance --float32 -dcpu -n50 --inductor  --no-skip --dashboard --only hf_Longformer  --cold_start_latency --batch_size 1 --threads 1
```

cc @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
[Inductor] [CPU] Torchbench model hf_T5 & hf_T5_large & hf_T5_base performance regression > 10% on ww07.4
Labels: triaged, module: inductor

### 🐛 Describe the bug

There is a performance regression on Torchbench models **hf_T5 & hf_T5_large & hf_T5_base** on ww07.4 [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531#issuecomment-1434019397)  compare with ww07.2 as bellow:

|  |  ww07.4   |   |     |   |  ww07.2   |   |     |   | Comp  |     |   |
|  ---- |  ----  | ----  |  ----  | ----  |  ----  | ----  |  ----  | ----  | ----  |  ----  | ----  |
| name| batch_size  | speedup | inductor  | eager | batch_size  | speedup | inductor  | eager | speedup ratio | eager ratio  | inductor ratio 
| hf_T5	| 1	| 0.9775	| 0.0487881	| 0.047690368	| 1	| 1.3737	| 0.0351173	| 0.048240635	| 0.71	| 1.01	| 0.72
| hf_T5_base	| 1	| 0.7094	| 1.4970444	| 1.062003297	| 1	| 1.073	| 0.9889897	| 1.061185948	| 0.66	| 1	| 0.66
| hf_T5_large	| 1	| 1.1104	| 0.4428939	| 0.491789387	| 1	| 1.4794	| 0.3326359	| 0.49210155	| 0.75	| 1	| 0.75

Please find sw info from [TorchInductor CPU Performance Dashboard](https://github.com/pytorch/pytorch/issues/93531) 

Graph dump by cosim:

WW07.4:
[hf_T5 graph.txt](https://github.com/pytorch/pytorch/files/10765472/graph.txt)
[hf_T5_large graph.txt](https://github.com/pytorch/pytorch/files/10765455/graph.txt)
[hf_T5_base graph.txt](https://github.com/pytorch/pytorch/files/10765463/graph.txt)

WW07.2:
[hf_T5 graph.txt](https://github.com/pytorch/pytorch/files/10765474/graph.txt)
[hf_T5_large graph.txt](https://github.com/pytorch/pytorch/files/10765458/graph.txt)
[hf_T5_base graph.txt](https://github.com/pytorch/pytorch/files/10765465/graph.txt)

Repro:

```
python -m torch.backends.xeon.run_cpu --node_id 0 benchmarks/dynamo/torchbench.py --performance --float32 -dcpu -n50 --inductor  --no-skip --dashboard --only MODEL  --cold_start_latency

```

cc @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
Custom operations in inductor
Labels: triaged, oncall: pt2, module: inductor, module: dynamo

### 🚀 The feature, motivation and pitch

I'm working on a backend for Dynamo/Inductor and I would like to "catch" modules at a higher level than the lowered inductor graph. Lowering a high-level operator to ATen primitives misses out on semantics that could be used for efficient backend implementations. A few examples:
* `nn.Dropout` is immediately lowered to Philox random + `aten.where`, which makes certain assumptions
* `nn.BatchNorm` is reduced to its core operations, which disallows calling something like cuDNN
* Higher-level modules from PyTorch extensions cannot always be compiled to a single graph, due to their use of Pythonic features

I would still want to use the awesome optimizations and inlining features that inductor does on the original GraphModule, just not on every operator.

Ideally, the solution would entail taking components in their `nn.Module` level and replacing them with custom lowered operations that can be used internally. Dynamo will, in turn, [optionally] skip trying to trace through the module so as to not crash because of missing Pythonic features [if shape/type inference is provided].

### Alternatives

I tried wrapping modules with specific types with some wrapper class, which diverges in the `forward` function based on `dynamo.is_compiling()`. While verbose (and forcing me to redo shape inference), this allows me to change some internal behavior.

Following this step, none of the approaches I tried work. I tried several things, including registering custom operations via `torch.fx` and `torch.ops` (as in the custom ops tutorial https://pytorch.org/tutorials/advanced/torch_script_custom_ops.html). torch.fx wrapped functions are ignored, custom ops are lowered to ATen on the C++ level, and the lowered inductor graph always ends up with the primitives.

It is worth mentioning that in the ONNX exporter this used to be possible via registering custom schemata.

Thanks in advance!

### Additional context

_No response_

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire
interactions between views + autograd.Function + AOTAutograd causes memory leak
Labels: module: autograd, triaged, actionable

I have a repro here (actively working on making it smaller). What's interesting is that it's non-trivial to make smaller - removing some extra ops, and e.g. changing the `autograd.function.apply()` to instead just call `torch.ops.test.foo_fwd()` directly all remove the leak.

```
import torch

def foo_fwd(v, o):
    out = torch.randn(512 * 256 * 128, device=v.device)
    return out.view(512, 256, 128)

def foo_bwd(grad_output, o):
    out = torch.zeros(64656, 128, device=grad_output.device)
    return out.squeeze(-1)

lib = torch.library.Library("test", "DEF")
lib.define("foo_fwd(Tensor v, Tensor[] o) -> Tensor")
lib.define("foo_bwd(Tensor grad_output, Tensor[] o) -> Tensor")

cuda_lib = torch.library.Library("test", "IMPL", "CUDA")
cuda_lib.impl("foo_fwd", foo_fwd)
cuda_lib.impl("foo_bwd", foo_bwd)

meta_lib = torch.library.Library("test", "IMPL", "Meta")
meta_lib.impl("foo_fwd", foo_fwd)
meta_lib.impl("foo_bwd", foo_bwd)

class Foo(torch.autograd.Function):
    @staticmethod
    def forward(ctx, v, o):
        return torch.ops.test.foo_fwd(v, o)

    @staticmethod
    def backward(ctx, grad_outputs):
        o = torch.ones(513, device='cuda', dtype=torch.int32)
        return torch.ops.test.foo_bwd(grad_outputs[0], [o])


def f(a, b, c, d, e, f):
    n: int = c.size(1)
    g = torch.mm(a, d)
    _, out, h = torch.split(g, [256, 256, 128], dim=1)
    i = h.unsqueeze(-1) * e + f
    j, _ = torch.unbind(i, dim=-1)
    k = Foo.apply(j, [b])
    torch._dynamo.graph_break()
    tmp = k + 1
    return out

f_compiled = torch.compile(f, backend="aot_eager")
inps = [
    torch.randn(64656, 256, dtype=torch.float32, device='cuda'),
    torch.arange(513, dtype=torch.int32, device='cuda'),
    torch.randn(512, 256, dtype=torch.float32, device='cuda'),
    torch.empty((256, 640), device='cuda', requires_grad=True),
    torch.empty(1, 128, 2, device='cuda', requires_grad=True),
    torch.zeros(1, 128, 2, device='cuda', requires_grad=True),
]

# This eventually OOMs
for i in range(300):
    print(torch.cuda.memory_allocated())
    #output = f(*inps)
    output = f_compiled(*inps)
    del output
    print(torch.cuda.memory_allocated())
#    print()
#print("@@@@@")
```

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @Lezcano @Varal7 @gchanan
[libtorh]Consistency problem of gpu computing
Labels: module: cuda, triaged, module: determinism

### 🐛 Describe the bug

Run the same input 100 times, the first time and the other 99 times the result is inconsistent, 2 to 100 times the result is the same.
The code to set the random seed is as follows:
``` c++
        int seed = 123456;
        srand(seed);
        torch::manual_seed(seed);
        if (m_gpu_id >= 0) {
            torch::cuda::manual_seed(seed);
            torch::cuda::manual_seed_all(seed);
            at::globalContext().setDeterministicCuDNN(true);
            at::globalContext().setDeterministicAlgorithms(true);
            at::globalContext().setBenchmarkCuDNN(false);
        }
```

The environment variables are set as follows
```
CUBLAS_WORKSPACE_CONFIG=":4096:8"
CUDA_LAUNCH_BLOCKING="1"
PYTHONHASHSEED="123456"
```


### Versions

cuda: 10.2
libtorch:  1.10.0-cu102
os: linux


cc @ngimel @mruberry @kurtamohler
Slow inference of torchscript model in C++
Labels: oncall: jit

### 🐛 Describe the bug

Torchscript same model have very slow inference in c++ as compared to python 

In Python 

    ```
    model = jit.load("parseq.ts")
    model.eval()
    ......
    .......
    image = Image.open('cropped.jpeg').convert('RGB')
    image = img_transform(image).unsqueeze(0).to('cpu')
    print('input shape : {}'.format(image.shape))
    start_time = time.time()
    p = model(image)
    end_time = time.time()
    elapsed_time = (end_time - start_time) * 1000
    print('Inference Time : {} ms'.format(elapsed_time))
	```

OUTPUT 

```
input shape : torch.Size([1, 3, 32, 128])
Inference Time : 665.2061939239502 ms
```
In C++ both 1.13.1 and 2.0.0

    ```
     module = torch::jit::load("parseq.ts");
     module.to(at::kCPU);
     .........................
     .........................
    std::cout << "input shape : " << tensor_image.sizes() << std::endl;
    // Create a vector of inputs.
    std::vector<torch::jit::IValue> inputs;
    inputs.push_back(tensor_image);
    
    // Execute the model and turn its output into a tensor.
    
    start_time = std::chrono::steady_clock::now();
    
    auto output = module.forward(inputs).toTensor();
    
    end = std::chrono::steady_clock::now();
    std::chrono::duration<double, std::milli> elapsed_seconds = end - start_time;
    std::cout << "Inference Time : " << elapsed_seconds.count() << " ms\n";

	```

OUTPUT

```
input shape : [1, 3, 32, 128]
elapsed time: 41168.8 ms
```

### Versions


PyTorch version: 1.13.1+cu117 on python 
PyTorch version: 2.0.0 on C++ 
PyTorch version: 1.13.1 on C++
os: Linux


cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel
PR #88607 breaks build for POWER9 CPU
Labels: module: build, triaged, module: POWER, actionable

### 🐛 Describe the bug

PR #88607 adds shift operators to vector types. In the general case, those shifts are implemented as for loops:

https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/cpu/vec/vec_base.h#L805

However, the access with `operator[]` conflicts, in the POWER9 case (and likely others), with the fact that they are not implemented:
https://github.com/pytorch/pytorch/blob/master/aten/src/ATen/cpu/vec/vec256/vsx/vec256_int32_vsx.h#L219

and so the code cannot compile.

### Versions

Git revision ae57bd663061cbd6bca3a03d12bb70a24913a84c

Relevant info from script:

```
GCC version: (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3)
Clang version: 12.0.1
CMake version: version 3.25.1

CPU:
Architecture:          ppc64le
Byte Order:            Little Endian
Model name:            POWER9, altivec supported
```


cc @malfet @seemethere
[numpy] mean & nanmean should support int dtypes
Labels: triaged, module: numpy, module: reductions

### 🚀 The feature, motivation and pitch

#### NumPy
```python
>>> import numpy as np
>>> k = np.array([1, 2, 3], dtype=np.int32)
>>> np.mean(k)
2.0
>>> np.nanmean(k)
2.0
```
#### PyTorch
```python
>>> import torch
>>> k = torch.tensor([1, 2, 3], dtype=torch.int32)
>>> torch.mean(k)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: mean(): could not infer output dtype. Input dtype must be either a floating point or complex dtype. Got: Int
>>> torch.nanmean(k)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: nanmean(): expected input to have floating point dtype but got Int
```
As discussed here: https://github.com/pytorch/pytorch/pull/93199#issuecomment-1431169963

### Alternatives

_No response_

### Additional context

_No response_

cc @mruberry @rgommers
Optimizer "Lion" in  Symbolic Discovery of Optimization Algorithms
Labels: module: optimizer, triaged, needs research

### 🚀 The feature, motivation and pitch

A new optimizer "Lion" was proposed in Symbolic Discovery of Optimization Algorithms
https://arxiv.org/pdf/2302.06675.pdf

It is simple, memory efficient and have a faster runtime.
I think it can be a great addition to the Pytorch optimizer algorithms' collection as 
it directly compares to AdamW which is already available. 

the optimizer is a single file
https://github.com/google/automl/blob/master/lion/lion_pytorch.py

Pseudo Code:
$$
\begin{algorithm}
    \caption{\name{} Lion}
    \begin{algorithmic}
    \State \textbf{given} $\beta_1$, $\beta_2$, $\lambda$, $\eta$, $f$
    \State \textbf{initialize} $\theta_0$, $m_0\leftarrow 0$
    \While{$\theta_t$ not converged}
        \State $g_t \leftarrow \nabla_\theta f(\theta_{t-1})$
        \State \textbf{update model parameters}
        \State $c_t \leftarrow \beta_1 m_{t-1} + (1-\beta_1)g_t$
        \State $\theta_t \leftarrow \theta_{t-1} - \eta_t(\text{sign}(c_t) + \lambda\theta_{t-1})$
        \State \textbf{update EMA of $g_t$}
        \State $m_t \leftarrow \beta_2 m_{t-1} + (1 - \beta_2)g_t$
    \EndWhile
    \State \textbf{return} $\theta_t$
    \end{algorithmic}
    \label{alg:ours}
\end{algorithm}
$$

### Alternatives

_No response_

### Additional context

_No response_

cc @vincentqb @jbschlosser @albanD @janeyx99
[FSDP] Gradients not propagating for mixed precision case
Labels: oncall: distributed, triaged, module: fsdp

```
import torch
from torch import nn
import torch.distributed as dist
torch.use_deterministic_algorithms(True, warn_only=True)
torch.manual_seed(0)
dist.init_process_group(backend="nccl")
global_rank = dist.get_rank()
torch.cuda.set_device(global_rank)
import functools

from torch.distributed.fsdp import FullyShardedDataParallel as FSDP
from torch.distributed.fsdp import ShardingStrategy
from torch.distributed.fsdp import MixedPrecision
from torch.distributed.fsdp.wrap import ModuleWrapPolicy
mixed_precision_config = MixedPrecision(
    param_dtype=torch.float16,
    reduce_dtype=torch.float32,
    buffer_dtype=torch.float32,
)
sharding_strategy_config = ShardingStrategy.SHARD_GRAD_OP
teacher_backbone = nn.Linear(24, 24)
student_backbone = nn.Linear(24, 24)
head = nn.Linear(24, 24)
x_half = torch.randn(32, 24, device="cuda", dtype=torch.float16)
x_half2 = torch.randn(32, 24, device="cuda", dtype=torch.float16)
fsdp_wrapper = functools.partial(
    FSDP, 
    sharding_strategy=sharding_strategy_config,
    mixed_precision=mixed_precision_config,
    device_id=global_rank,
    use_orig_params=True,
)
s_teacher_backbone = fsdp_wrapper(teacher_backbone)
s_student_backbone = fsdp_wrapper(student_backbone)
s_head = fsdp_wrapper(head)
t1 = s_teacher_backbone(x_half)
t2 = s_head(t1).detach()
s1 = s_student_backbone(x_half2)
s2 = s_head(s1)
loss = (s2 * t2).sum()
loss.backward()
```

Even if `t2` has `detach()` called, the gradient should propagate through `s2` to `s_head`. However, none of `s_head`'s parameters receive gradients.

If you remove the FSDP from this repro, the gradients are propagated as expected.

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @rohan-varma @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501
Dynamo.export should support formatting tensor value within a string
Labels: feature, triaged, oncall: pt2, module: dynamo

### 🚀 The feature, motivation and pitch

Code like this would fail to trace in dynamo.export

```
torch._assert(x.numel() > 0, f"My message, tensor value is: {x}")
```

Error message:
```
Unsupported: call_function BuiltinVariable(format) [ConstantVariable(str), TensorVariable()] {}
```

### Alternatives

N/A

### Additional context

It is desired because the code pattern is very common in modeling and there isn't really a work-around for its functionality.

cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
Rationalize specialize_int_float handling
Labels: triaged, oncall: pt2, module: dynamo

### 🐛 Describe the bug

There are a few things going on here, writing them down in no particular order:

* dynamic turns off specialize_int_float by default, but this is the wrong choice for export (as all module parameters become new inputs, oof)
* specialize_int_float = False is too aggressive, a ton of things turn into float inputs when they shouldn't be (eager mode specific problem); observed by Horace
* unspecialized int/float are represented as 0d tensor in dynamo, but it is more uniform to treat them as symint/symfloat (but there is some debate about this, because 0d tensor rep is more convenient for backends to handle UNLESS the int ends up being used in a shape context)

### Versions

master

cc @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire
Allow Dynamo backends to use Inductor as fallback instead of eager mode
Labels: triaged, oncall: pt2, module: inductor

### 🚀 The feature, motivation and pitch

I'm working on Dynamo oneDNN backend (being integrated at the mid-layer, like nvfuser or AITemplate) and would like to run models with mixture of Dynamo oneDNN backend + Inductor CPU backend. 

Based on oneDNN graph's programming model, we prefer to pass the full graph to oneDNN backend first, which decides its supported subgraphs, and run the rest of the graph with Inductor CPU afterward. 

This prototype PR [#90356] from @SherlockNoMad allows inductor to accepts opaque callable. This aligns with our design, where oneDNN backend will fuse subgraphs to call_module nodes and set callable for each fused node. The fused graph will be passed to Inductor CPU for compile and execute.

We'd like to check the plan of this feature and if the PR would be merged.   



cc @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @jiayisunx @peterbell10 @desertfire @mlazos
Linking error with Libtorch
Labels: module: cpp, triaged, module: mkl

### 🐛 Describe the bug

Hello I'm trying to create an example-app case to run with libtorch as in the https://pytorch.org/cppdocs/installing.html, however I have a linking error.

### Error logs

LINK : fatal error LNK1104: cannot open file 'mkl_intel_ilp64.lib'

### Minified repro

_No response_

### Versions

libtorch 1.13.1+cpu 

cc @jbschlosser @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh
Make `torch.onnx.utils._optimize_graph` use several CPU cores
Labels: module: performance, module: onnx, triaged

### 🚀 The feature, motivation and pitch

As per title, the function `_optimize_graph` from `torch/onnx/utils.py` uses a single core. This can be checked using `mpstat -P ALL 1` and putting some prints to pin down where the export is hanging.

https://github.com/pytorch/pytorch/blob/d1d5d16df3766bc6a6c9ebe1168ff652bcdcaa5e/torch/onnx/utils.py#L564

This is probably a factor in some exports being extremely slow, maybe linked https://github.com/pytorch/pytorch/issues/63734

In particular, I notice that I am hanging in https://github.com/pytorch/pytorch/blob/d1d5d16df3766bc6a6c9ebe1168ff652bcdcaa5e/torch/onnx/utils.py#L665 which apparently maps to https://github.com/pytorch/pytorch/blob/d1d5d16df3766bc6a6c9ebe1168ff652bcdcaa5e/torch/csrc/jit/passes/onnx.cpp#L165 that takes a very long time, with a single core active.

It would be useful for the method to use several cores.

### Alternatives

Stop using `torch.onnx.export` altogether, stay in PyTorch all the way.

### Additional context

PyTorch version: 1.13.1

cc @ngimel
grid_sample with relative grid
Labels: feature, module: nn, triaged, actionable, module: interpolation

### 🚀 grid_sample with relative grid, to make code simpler, and optimize speed and accuracy

Currently, "grid_sample" requires a "grid" with (sort of) "absolute" values. Meaning -1 is the left image edge and +1 is the right image edge. That seems fine in theory. However, practically, if we're talking about big images, e.g. 4K video, the pixels become so small that the addressing becomes highly inaccurate when using Float16.

E.g. let's play around with Float16 numbers here:

https://evanw.github.io/float-toy/

Let's say we want to move pixel 0 to the side by 0.5 pixels. So when talking about a 4K image, the "grid" would need to have a value of -1919.5 / 1920 = -0.99973958333. If you enter this number into the float calculator above, due to the limited accuracy of Float16, the number changes to just -0.9996. Now let's multiply by 1920 and we get -1919.232. So instead of the desired pixel shift of 0.5 we actually get a pixel shift of 0.768! You can see how inaccurate this becomes, and that creates very noticeable image quality degradations. Which means that when using a Float16 grid, grid_sample becomes totally unusable because at high image resolutions, Float16 becomes so inaccurate that it's basically utterly broken.

Now we could argue that we should simply create a Float32 grid. It works well. So why not do that? Well, it consumes more RAM, and it's slower. So it would be much nicer if we could use a Float16 grid.

**So here comes my wish**: There should be a grid_sample alternative which accepts a relative grid. With the "grid" all zero, basically input and output images should be identical. If you want to read a pixel from 0.1 pixels from the right, that pixel in the grid would simply have a value of "0.1". Such a relative grid should work perfectly fine with Float16.

Plus, most neural networks (especially convolutional ones) which create a "grid" actually calculate relative values natively, anyway! They don't usually calculate absolute values. So if you look through all the public neural networks which use grid_sample, many (most?) of them painfully convert their internal relative grid to an absolute grid, to make grid_sample happy. Which is a completely useless conversion, which only makes the use of grid_sample more complex, plus it introduces the Float16 accuracy problem explained above.

Practically, in all my neural networks which use grid_sample, if there was a relative addressing version of grid_sample available, my code would be simpler, and I could stick to Float16, which my neural network natively calculates, anyway (due to Automatic Mixed Precision training). So it would also perform better, with no relative -> absolute conversion needed, and no Float16 -> Float32 conversion needed, either.

### Additional context

This feature wish is loosely related to issue # 94733.

cc @albanD @mruberry @jbschlosser @walterddr @saketh-are
[mta] Implement fused SGD
Labels: module: cpu, triaged, module: mkldnn, open source, module: amp (automated mixed precision), Stale, ciflow/trunk, release notes: quantization, release notes: nn, ciflow/mps, module: inductor, module: dynamo, ciflow/inductor, module: export

rel:
- https://github.com/pytorch/pytorch/issues/58833
- https://github.com/pytorch/pytorch/issues/68041

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @gujinghui @PenghuiCheng @jianyuh @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen @mcarilli @ptrblck @leslie-fang-intel @voznesenskym @penguinwu @EikanWang @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @avikchaudhuri @gmagogsfm
pytorch log level API and env var
Labels: high priority, feature, module: logging, triaged

### 🚀 The feature, motivation and pitch

I'm requesting to please implement an API to allow users to set logging level across pytorch 

For example here is how I set a consistent log level in my applications across multiple packages on a multi-gpu training setup:
```
    log_level = logging.WARNING if dist.get_rank() == 0 else logging.ERROR
    
    datasets.utils.logging.set_verbosity(log_level)
    transformers.utils.logging.set_verbosity(log_level)
    
    from deepspeed.utils import logger as ds_logger
    ds_logger.setLevel(log_level)
```

Would it be possible to be able to do:
```
    torch.utils.logging.set_verbosity(log_level)
```

Each of the above examples supports `DATASETS_VERBOSITY`, `TRANSFORMERS_VERBOSITY` env vars to accomplish the same.

`torch.distributed` uses `LOGLEVEL`, which already helps, but unfortunately doesn't use a domain specific prefix. Ideally it would be some sort of `PYTORCH_LOGLEVEL` so that it won't potentially clash with other packages who too didn't think of using a prefixed env var with a unique package-related prefix.

So to summarize:

1. `torch.utils.logging.set_verbosity(log_level)`
2. `PYTORCH_LOGLEVEL` as another way of setting the log level.

I'm not attached to naming, these are just examples.

wrt implementation you are more than welcome to adapt our feature-full [logging.py](https://github.com/huggingface/transformers/blob/main/src/transformers/utils/logging.py) module. All you need to do is `s/transformers/pytorch/g` and `s/TRANSFORMERS/PYTORCH/g`. All the work is already done.

Thank you.



cc @ezyang @gchanan @zou3519
MPS internal error in `torch.gather` when last dimension is a singleton dimension
Labels: module: crash, triaged, module: mps

### 🐛 Describe the bug

I'm trying to port my PyTorch code to use MPS acceleration on a MacBook Pro M1 Pro and I'm facing a MPS internal error I cannot debug:

```
/AppleInternal/Library/BuildRoots/c651a45f-806e-11ed-a221-7ef33c48bc85/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSNDArray/Kernels/MPSNDArrayGatherND.mm:234: failed assertion `Rank of updates array (1) must be greater than or equal to inner-most dimension of indices array (20)'
[1]    11201 abort      python src/evaluate.py --valid_dir database/valid --load_model  --anchor_name
/Users/louislac/miniconda3/envs/sdnet/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
```

The code to reproduce the error is available [here](https://github.com/laclouis5/StructureDetector/tree/apple-silicon-mps). the precise environment can be installed with Anaconda using the `environment_mps.yml` environment file. the command I'm using is `python src/evaluate.py --valid_dir valid --load_model model_best_classif.pth --anchor_name stem --conf_threshold 0.4 --decoder_dist_thresh 0.1 --dist_threshold 0.05`. The `valid` directory can be found [here](https://www.icloud.com/iclouddrive/05ae8fGB4-wknNuWEkOleMlsA#valid) and the model parameters `model_best_classif.pth` can be found [here](https://www.icloud.com/iclouddrive/0e0HWpCssX9orjNJ-Xq79b2OQ#model_best_classif).

This program works fine using the  CPU or using  CUDA with a compatible GPU.

### Versions

```
Collecting environment information...
PyTorch version: 2.0.0.dev20230213
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 13.2 (arm64)
GCC version: Could not collect
Clang version: 14.0.0 (clang-1400.0.29.202)
CMake version: Could not collect
Libc version: N/A

Python version: 3.10.9 (main, Jan 11 2023, 09:18:18) [Clang 14.0.6 ] (64-bit runtime)
Python platform: macOS-13.2-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Apple M1 Pro

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==2.0.0.dev20230213
[pip3] torchvision==0.15.0.dev20230213
[conda] numpy                     1.23.5          py310hb93e574_0  
[conda] numpy-base                1.23.5          py310haf87e8b_0  
[conda] pytorch                   2.0.0.dev20230213        py3.10_0    pytorch-nightly
[conda] torchvision               0.15.0.dev20230213       py310_cpu    pytorch-nightly
```

cc @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev
Update PyTorch's default C standard to C17 from C11
Labels: module: build, triaged

### 🚀 The feature, motivation and pitch

We just updated the codebase to C++17 to be compatible with the latest CUDA. However, we left the C standard at C11: https://github.com/pytorch/pytorch/blob/216f88d084f290020785d2719c20c0d3acc510aa/CMakeLists.txt#L41 

C17 is also available and should be supported by any compiler that supports C++17. C17 is mostly a bugfix / defect fixes for C11 and doesn't introduce any / major new features: https://en.cppreference.com/w/c/17 . I don't see any reason why wouldn't upgrade to C17 as well, and I just want to make sure it wasn't an oversight during the C++17 upgrade process. It doesn't seem to be discussed here: https://github.com/pytorch/pytorch/issues/56055

### Alternatives

Keep it as C11.

### Additional context

_No response_

cc @malfet @seemethere