zhiminy/EvalEng huggingface dataset:
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'b48a2494be5e5e88feb6d1d46f36ed84610ddd8e'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 1d3bf3da2129a882434dc8a44c14f21e826aac7b

Affected PyTorch commit: b48a2494be5e5e88feb6d1d46f36ed84610ddd8e

Affected Tests:
- test_train[dcgan-cuda-eager]_latency: -12.71842%
- test_train[detectron2_fasterrcnn_r_101_fpn-cuda-eager]_latency: -10.60742%
- test_train[detectron2_maskrcnn-cuda-eager]_latency: -14.61259%
- test_eval[functorch_dp_cifar10-cuda-eager]_latency: -10.70315%
- test_eval[hf_Bert-cuda-eager]_latency: -10.26816%
- test_eval[llama-cuda-eager]_latency: -16.41389%
- test_eval[maml_omniglot-cuda-eager]_latency: -22.13216%
- test_train[moco-cuda-eager]_latency: -31.53001%
- test_eval[moco-cuda-eager]_latency: -33.23979%
- test_eval[yolov3-cuda-eager]_latency: -10.64466%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5966145314

cc @xuzhao9


Bisection workflow: https://github.com/pytorch/benchmark/actions/runs/5967010600

Result is empty, closed.https://github.com/pytorch/benchmark/actions/runs/5967010600
Repository: pytorch/benchmark
Issue: why I can not choose the compiler??
Root cause: Documentation Deficiency

![image](https://github.com/pytorch/benchmark/assets/58350195/1140aafe-b461-41f0-adef-534a098642c5)

if i use this command, it will select noting
BUT

![image](https://github.com/pytorch/benchmark/assets/58350195/7b4a1816-3efd-4af5-905e-03de81a41779)

if I delete the compiler in "test_train" , it can work , can you explain why?

`test_bench.py` doesn't support choosing compiler backends. You need to use run.py:

```
python run.py BERT_pytorch -d cuda -t train --torchdynamo inductor
```

Checkout https://github.com/pytorch/benchmark/issues/1825 for details

> `test_bench.py` doesn't support choosing compiler backends. You need to use run.py:
> 
> ```
> python run.py BERT_pytorch -d cuda -t train --torchdynamo inductor
> ```
> 
> Checkout #1825 for details

so that means the readme is wrong?


> > `test_bench.py` doesn't support choosing compiler backends. You need to use run.py:
> > ```
> > python run.py BERT_pytorch -d cuda -t train --torchdynamo inductor
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > Checkout #1825 for details
> 
> so that means the readme is wrong?

Sorry can I have the reference from the readme? `test_bench.py` only supports eager mode for now. We should fix readme if it claims the opposite.

> > > `test_bench.py` doesn't support choosing compiler backends. You need to use run.py:
> > > ```
> > > python run.py BERT_pytorch -d cuda -t train --torchdynamo inductor
> > > ```
> > > 
> > > 
> > >     
> > >       
> > >     
> > > 
> > >       
> > >     
> > > 
> > >     
> > >   
> > > Checkout #1825 for details
> > 
> > 
> > so that means the readme is wrong?
> 
> Sorry can I have the reference from the readme? `test_bench.py` only supports eager mode for now. We should fix readme if it claims the opposite.

Here:
<img width="1076" alt="image" src="https://github.com/pytorch/benchmark/assets/58350195/17994750-33ba-4570-b308-f5b2134e528a">
if I type the "Bert_pytorch-cuda-eager", it can not choose any test 

Sorry I figured out that we need to update our README to remove the `-eager` part of the test. test_bench.py does not support customizing compiler backend.

Submitted https://github.com/pytorch/benchmark/pull/1878
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on 'b48a2494be5e5e88feb6d1d46f36ed84610ddd8e'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 1d3bf3da2129a882434dc8a44c14f21e826aac7b

Affected PyTorch commit: b48a2494be5e5e88feb6d1d46f36ed84610ddd8e

Affected Tests:


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5971720988

cc @janeyx99


Seems related to https://github.com/pytorch/benchmark/issues/1839
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '91bde862b20971c6c0176ae10759a3412751b304'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: b48a2494be5e5e88feb6d1d46f36ed84610ddd8e

Affected PyTorch commit: 91bde862b20971c6c0176ae10759a3412751b304

Affected Tests:
- timm_vision_transformer_large, ASGD, cuda, differentiable: +33.04270%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5982697770

cc @janeyx99


Seems related to https://github.com/pytorch/benchmark/issues/1839
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'c0a66455c03fd83cdc4edef9d1e26953e393dc68'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: b48a2494be5e5e88feb6d1d46f36ed84610ddd8e

Affected PyTorch commit: c0a66455c03fd83cdc4edef9d1e26953e393dc68

Affected Tests:
- test_train[detectron2_maskrcnn-cuda-eager]_latency: +10.76028%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5985737224

cc @xuzhao9


Bisection: https://github.com/pytorch/benchmark/actions/runs/6000572381

Failed to checkout the commit, closed.
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'f54acf099180d680076065f7fdcc7c8be578a91e'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: c0a66455c03fd83cdc4edef9d1e26953e393dc68

Affected PyTorch commit: f54acf099180d680076065f7fdcc7c8be578a91e

Affected Tests:
- test_eval[pyhpc_turbulent_kinetic_energy-cuda-eager]_latency: +20.63511%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5991921274

cc @xuzhao9


Bisection: https://github.com/pytorch/benchmark/actions/runs/6000577390

```
{
  "target_repo": "pytorch",
  "start": "95cacb7fa97881b7e830d1ee0bcb4729b50632c0",
  "end": "bad3f2db40ffa491181fc7df5dc310830326d9ec",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '2c3aa09cde00fe4b45307770ebc86178d61773e7'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: f54acf099180d680076065f7fdcc7c8be578a91e

Affected PyTorch commit: 2c3aa09cde00fe4b45307770ebc86178d61773e7

Affected Tests:
- test_eval[pyhpc_turbulent_kinetic_energy-cuda-eager]_latency: -18.39092%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6002173378

cc @xuzhao9


https://github.com/pytorch/benchmark/actions/runs/6004329436

```
{
  "target_repo": "pytorch",
  "start": "bad3f2db40ffa491181fc7df5dc310830326d9ec",
  "end": "138e2895d08a6517c5718b2a0118c1b23ff4664c",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: Benchmarks leak files
Root cause: Resource Mishandling

I observed that when you run benchmarks many files are created in the current working directory.  This is bad because:
1) It may increases benchmark variance because disk access can have less predictable performance
2) It it subjectively ugly/messy

Reproduce with:
```
rm -f Video_data_train_processed.csv labels.json results.png results.txt train_batch0.jpg checkpoints/*
pytest test.py
ls Video_data_train_processed.csv labels.json results.png results.txt train_batch0.jpg checkpoints/*
```

Actual output of `ls`
```
labels.json
results.png
results.txt
train_batch0.jpg
Video_data_train_processed.csv

checkpoints/horse2zebra:
loss_log.txt
train_opt.txt
web

checkpoints/horse2zebra_pretrained:
test_opt.txt
```

Desired output of `ls`:  empty with many "No such file or directory" errors





Closed by https://github.com/pytorch/benchmark/pull/792
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '2c3aa09cde00fe4b45307770ebc86178d61773e7'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: f54acf099180d680076065f7fdcc7c8be578a91e

Affected PyTorch commit: 2c3aa09cde00fe4b45307770ebc86178d61773e7

Affected Tests:


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6007752817

cc @janeyx99


Seems related to https://github.com/pytorch/benchmark/issues/1839
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '8c684663042a61de20f163ab6a292cd43c67e294'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 2c3aa09cde00fe4b45307770ebc86178d61773e7

Affected PyTorch commit: 8c684663042a61de20f163ab6a292cd43c67e294

Affected Tests:
- mobilenet_v3_large, Adadelta, cuda, differentiable: +43.35396%
- mobilenet_v3_large, AdamW, cuda, no_foreach: +47.85973%
- mobilenet_v3_large, AdamW, cuda, differentiable: +35.74592%
- mobilenet_v3_large, SGD, cuda, no_foreach: +41.62527%
- mobilenet_v3_large, RMSprop, cuda, no_foreach: +52.92799%
- opacus_cifar10, Adam, cuda, no_foreach: +44.78348%
- opacus_cifar10, RMSprop, cuda, differentiable: +48.32945%
- phlippe_densenet, Adamax, cuda, maximize: +35.26371%
- phlippe_densenet, RMSprop, cuda, differentiable: +52.33228%
- doctr_reco_predictor, Adagrad, cuda, differentiable: +31.70632%
- mobilenet_v2, Adam, cuda, differentiable: +35.75587%
- moco, RMSprop, cuda, differentiable: +49.00914%
- DALLE2_pytorch, Adam, cuda, amsgrad, maximize: +31.66236%
- DALLE2_pytorch, AdamW, cuda, amsgrad, maximize: +32.70095%
- DALLE2_pytorch, RAdam, cuda, default: +33.38843%
- hf_Whisper, ASGD, cuda, foreach: +53.27004%
- hf_Whisper, RMSprop, cuda, differentiable: +35.19851%
- mnasnet1_0, AdamW, cuda, no_foreach: +36.39512%
- hf_Reformer, AdamW, cuda, no_foreach: +47.05424%
- hf_T5_base, Adadelta, cuda, no_foreach: +40.63222%
- hf_T5_base, AdamW, cuda, no_foreach: +42.40198%
- hf_T5_base, ASGD, cuda, foreach: +34.62303%
- hf_T5_large, Rprop, cuda, no_foreach: +44.36867%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6020392233

cc @janeyx99


Seems related to https://github.com/pytorch/benchmark/issues/1839
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'b3874abf04e0579fdc0025195593d90b97178f0e'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 8c684663042a61de20f163ab6a292cd43c67e294

Affected PyTorch commit: b3874abf04e0579fdc0025195593d90b97178f0e

Affected Tests:
- test_eval[mobilenet_v3_large-cuda-eager]_latency: -13.73389%
- test_eval[yolov3-cuda-eager]_latency: -11.76849%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6027298051

cc @xuzhao9


Bisection: https://github.com/pytorch/benchmark/actions/runs/6042017533

Result:

```
{
  "target_repo": "pytorch",
  "start": "e68b3ad14f9598334a97c1aff15408e1d8f09002",
  "end": "620d267ef30b199e0ba1b37407dcf8670448fb6a",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on 'b5b99fe13b890232bb61155a46239922661f4695'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: b3874abf04e0579fdc0025195593d90b97178f0e

Affected PyTorch commit: b5b99fe13b890232bb61155a46239922661f4695

Affected Tests:


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- timm_vision_transformer_large, Adadelta, cuda, (pt2) default: 0.02814946771832183
- timm_vision_transformer_large, Adadelta, cuda, (pt2) no_foreach: 0.038562088971957564
- timm_vision_transformer_large, Adadelta, cuda, (pt2) foreach: 0.02896553324535489
- timm_vision_transformer_large, Adadelta, cuda, foreach: 0.08906917941446106
- timm_vision_transformer_large, Adagrad, cuda, (pt2) default: 0.023091250823603734
- timm_vision_transformer_large, Adagrad, cuda, (pt2) no_foreach: 0.030158286315522025
- timm_vision_transformer_large, Adagrad, cuda, (pt2) foreach: 0.02312654722481966
- timm_vision_transformer_large, Adam, cuda, (pt2) default: 0.030340568561639105
- timm_vision_transformer_large, Adam, cuda, (pt2) no_foreach: 0.05908456491306424
- timm_vision_transformer_large, Adam, cuda, (pt2) foreach: 0.03233537796352591
- timm_vision_transformer_large, AdamW, cuda, (pt2) default: 0.03256231539749673
- timm_vision_transformer_large, AdamW, cuda, (pt2) no_foreach: 0.05833212053403258
- timm_vision_transformer_large, AdamW, cuda, (pt2) foreach: 0.031883069447108676
- timm_vision_transformer_large, AdamW, cuda, foreach: 0.06900479132309556
- timm_vision_transformer_large, Adamax, cuda, (pt2) no_foreach: 0.07628471342225869
- timm_vision_transformer_large, ASGD, cuda, foreach: 0.024120567785575985
- timm_vision_transformer_large, Rprop, cuda, (pt2) default: 0.030385738837399652
- timm_vision_transformer_large, Rprop, cuda, (pt2) no_foreach: 0.040980339888483285
- timm_vision_transformer_large, Rprop, cuda, (pt2) foreach: 0.029263119945036515
- timm_vision_transformer_large, Rprop, cuda, foreach: 0.11135207861661911
- timm_vision_transformer_large, NAdam, cuda, (pt2) no_foreach: 0.06171368760988116


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6045259308

cc @janeyx99


Seems related to #1839
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '73c794dadccfcc204b7db42dc0c915978fc61434'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: b5b99fe13b890232bb61155a46239922661f4695

Affected PyTorch commit: 73c794dadccfcc204b7db42dc0c915978fc61434

Affected Tests:
- test_eval[pytorch_CycleGAN_and_pix2pix-cuda-eager]_latency: +19.00387%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6051589243

cc @xuzhao9


Bisection: https://github.com/pytorch/benchmark/actions/runs/6052176562

```
{
  "target_repo": "pytorch",
  "start": "0e4752bafc24755893bd1657c42d8937c275ce74",
  "end": "4a9c6f1b731e86b1d4fe31c6753ce7efdc522efa",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '753f5a11bf46f2c5f52c31bbb0bca5c1771fb509'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 73c794dadccfcc204b7db42dc0c915978fc61434

Affected PyTorch commit: 753f5a11bf46f2c5f52c31bbb0bca5c1771fb509

Affected Tests:
- test_eval[pytorch_CycleGAN_and_pix2pix-cuda-eager]_latency: -14.91456%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6059275226

cc @xuzhao9


Bisection workflow: https://github.com/pytorch/benchmark/actions/runs/6104757433

Bisection workflow: https://github.com/pytorch/benchmark/actions/runs/6313591947

Result:
```
{
  "target_repo": "pytorch",
  "start": "4a9c6f1b731e86b1d4fe31c6753ce7efdc522efa",
  "end": "4084d039b7a4b6e55ee6ca05f2789d7ed03b9d73",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on 'ea7e22973572b5fcf43ea05897c2cf56ace392da'
Root cause: Others


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 753f5a11bf46f2c5f52c31bbb0bca5c1771fb509

Affected PyTorch commit: ea7e22973572b5fcf43ea05897c2cf56ace392da

Affected Tests:
- resnet18, Adamax, cuda, (pt2) no_foreach: +364743.84980%
- timm_vision_transformer, Adamax, cuda, (pt2) no_foreach: -99.97365%
- timm_vision_transformer_large, Adamax, cuda, (pt2) no_foreach: -99.97646%
- timm_vision_transformer_large, NAdam, cuda, (pt2) no_foreach: +836399.68657%
- DALLE2_pytorch, Adam, cuda, fused, amsgrad, maximize: +47.50999%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6068945904

cc @janeyx99


closing due to high variance, similar as #1876
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'cf9eb00513769966ee1f0ac556b83ccefb9cd047'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: ea7e22973572b5fcf43ea05897c2cf56ace392da

Affected PyTorch commit: cf9eb00513769966ee1f0ac556b83ccefb9cd047

Affected Tests:
- test_train[dcgan-cuda-eager]_latency: +23.72781%
- test_train[lennard_jones-cuda-eager]_latency: +12.11775%
- test_train[maml_omniglot-cuda-eager]_latency: +10.48792%
- test_train[mobilenet_v3_large-cuda-eager]_latency: +11.62848%
- test_eval[opacus_cifar10-cuda-eager]_latency: +14.05594%
- test_train[phlippe_resnet-cuda-eager]_latency: +16.55267%
- test_eval[phlippe_resnet-cuda-eager]_latency: +12.21230%
- test_train[pytorch_CycleGAN_and_pix2pix-cuda-eager]_latency: +11.41554%
- test_eval[shufflenet_v2_x1_0-cuda-eager]_latency: +12.95073%
- test_train[tts_angular-cuda-eager]_latency: +14.72015%
- test_eval[yolov3-cuda-eager]_latency: +15.24206%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6075646454

cc @xuzhao9


Bisection workflow: https://github.com/pytorch/benchmark/actions/runs/6104759637

```
{
  "target_repo": "pytorch",
  "start": "e5548f81956157e1fdb76cb52579bffe5915dee1",
  "end": "621463a3e6b488b2bff04e355a1abd9a4c5bb2cd",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '9ea5b31c6fd45cd3139540d3da74a2a248f644d9'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: ea7e22973572b5fcf43ea05897c2cf56ace392da

Affected PyTorch commit: 9ea5b31c6fd45cd3139540d3da74a2a248f644d9

Affected Tests:
- timm_vision_transformer_large, Adamax, cuda, (pt2) no_foreach: +433296.71577%
- timm_vision_transformer_large, NAdam, cuda, (pt2) no_foreach: -99.98595%
- DALLE2_pytorch, Adam, cuda, fused, amsgrad, maximize: -31.65240%
- DALLE2_pytorch, AdamW, cuda, fused, capturable, amsgrad: +46.68095%
- resnet18, Adamax, cuda, (pt2) no_foreach: -99.97085%
- resnet18, NAdam, cuda, (pt2) no_foreach: -99.98824%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6092573815

cc @janeyx99


Weird that there's huge variance for NAdam and Adamax, will keep seeing if this happens once I land the PR to tease out the compile time
<img width="386" alt="image" src="https://github.com/pytorch/benchmark/assets/31798555/6e231202-c1a6-4d16-beae-f3e984e5855d">