zhiminy/EvalEng huggingface dataset:
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '806d1a871ddfd2d38e1791489892009feaec8425'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 806d1a871ddfd2d38e1791489892009feaec8425

Affected PyTorch commit: 806d1a871ddfd2d38e1791489892009feaec8425

Affected Tests:
- timm_vision_transformer, Adadelta, cuda, (pt2) no_foreach: +820455.44220%
- timm_vision_transformer, Adam, cuda, (pt2) no_foreach: +568127.78268%
- timm_vision_transformer, Adam, cuda, (pt2) foreach: +124.23588%
- timm_vision_transformer, AdamW, cuda, (pt2) no_foreach: +564497.22671%
- timm_vision_transformer, AdamW, cuda, (pt2) foreach: +208.75838%
- timm_vision_transformer_large, ASGD, cuda, (pt2) no_foreach: -99.98306%
- timm_vision_transformer_large, ASGD, cuda, (pt2) foreach: -69.20373%
- timm_vision_transformer_large, RAdam, cuda, differentiable: -44.58829%
- compile_time, timm_vision_transformer_large, AdamW, cuda, (pt2) foreach: +1230.58213%
- compile_time, timm_vision_transformer_large, ASGD, cuda, (pt2) no_foreach: +2013.29060%
- resnet18, Adam, cuda, (pt2) no_foreach: -99.98449%
- resnet18, Adam, cuda, (pt2) foreach: -57.05718%
- compile_time, resnet18, Adam, cuda, (pt2) no_foreach: +500.11103%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- timm_vision_transformer_large, RAdam, cuda, (pt2) default: 0.07144256508909166
- timm_vision_transformer_large, RAdam, cuda, default: 0.07678249742214878
- timm_vision_transformer_large, RAdam, cuda, (pt2) foreach: 0.07665289286524057
- timm_vision_transformer_large, RAdam, cuda, foreach: 0.07637819166605671
- timm_vision_transformer_large, NAdam, cuda, default: 0.07561325514689088
- compile_time, timm_vision_transformer_large, RAdam, cuda, (pt2) default: 0.05993408725286523
- compile_time, timm_vision_transformer_large, RAdam, cuda, (pt2) foreach: 0.054936480397979416


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6134970889

cc @janeyx99


redundant issues with #1888
Repository: pytorch/benchmark
Issue: [optim][ub] pt2 values keep flip-flopping nondeterministically
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 806d1a871ddfd2d38e1791489892009feaec8425

Affected PyTorch commit: 3316374d14f2e488b65dfb4d0a2560d8414fa19f

Affected Tests:
- timm_vision_transformer_large, Adadelta, cuda, (pt2) no_foreach: +482341.57958%
- timm_vision_transformer_large, RAdam, cuda, differentiable: +81.55424%
- timm_vision_transformer, AdamW, cuda, (pt2) no_foreach: -99.98224%
- timm_vision_transformer, AdamW, cuda, (pt2) foreach: -66.83298%
- compile_time, timm_vision_transformer, AdamW, cuda, (pt2) no_foreach: +1891.78438%
- compile_time, timm_vision_transformer, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +895.46175%
- compile_time, timm_vision_transformer, RMSprop, cuda, (pt2) foreach: +2183.02906%
- DALLE2_pytorch, AdamW, cuda, fused, capturable, amsgrad: +38.77857%
- resnet18, Adadelta, cuda, (pt2) no_foreach: +735169.67855%
- resnet18, AdamW, cuda, (pt2) no_foreach: +606511.29533%
- resnet18, AdamW, cuda, (pt2) foreach: +192.09611%
- compile_time, resnet18, Adadelta, cuda, (pt2) foreach: +1824.75319%
- compile_time, resnet18, Adagrad, cuda, (pt2) foreach: +2646.24568%
- compile_time, resnet18, AdamW, cuda, (pt2) foreach: +1720.93538%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6141749176

cc @janeyx99
Repository: pytorch/benchmark
Issue: Add TorchBench to HUD and Run Periodically
Root cause: Unimplemented Feature Gap


It would be great if we could add a simple dashboard that covers all userbenchmarks through fetching data from S3.

TODO:
- [x] Add pytorch version date to userbenchmarks so that we can track the commits by their dates
- [x] Update the dashboard and Rockset query to track the metrics by date

Fixed by https://github.com/pytorch/test-infra/pull/4743

Demo URL: https://hud.pytorch.org/torchbench/userbenchmark
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '7b14f0ff8fabe2185e4ff9551e858e46c7784536'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 3316374d14f2e488b65dfb4d0a2560d8414fa19f

Affected PyTorch commit: 7b14f0ff8fabe2185e4ff9551e858e46c7784536

Affected Tests:
- resnet50_quantized_qat, Adadelta, cuda, differentiable: -30.48702%
- resnet18, Adadelta, cuda, (pt2) no_foreach: -99.98456%
- resnet18, Adam, cuda, (pt2) no_foreach: -99.98356%
- resnet18, Adam, cuda, (pt2) foreach: -56.96719%
- compile_time, resnet18, Adadelta, cuda, (pt2) no_foreach: +431.22333%
- compile_time, resnet18, Adam, cuda, (pt2) no_foreach: +521.22345%
- timm_vision_transformer_large, Adam, cuda, (pt2) no_foreach: -99.97464%
- compile_time, timm_vision_transformer_large, Adadelta, cuda, (pt2) no_foreach: -252.74299%
- compile_time, timm_vision_transformer_large, Adam, cuda, (pt2) no_foreach: +4187.44263%
- timm_vision_transformer, SGD, cuda, no_foreach: -31.76677%
- timm_resnest, Adadelta, cuda, no_foreach: -33.33582%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- timm_vision_transformer_large, Adam, cuda, (pt2) foreach: 0.030929460589374815
- timm_vision_transformer_large, Adam, cuda, foreach: 0.058241735119372605
- timm_vision_transformer_large, Adam, cuda, foreach, maximize, capturable: 0.07816018629819155
- timm_vision_transformer_large, Adam, cuda, (pt2) fused: 0.030626087582537105
- timm_vision_transformer_large, Adam, cuda, fused: 0.02658559228293598
- timm_vision_transformer_large, Adam, cuda, fused, amsgrad, maximize: 0.032691909199846644
- timm_vision_transformer_large, Adam, cuda, fused, capturable: 0.02660306040197611
- timm_vision_transformer_large, Adam, cuda, fused, capturable, amsgrad: 0.03027761382982135
- compile_time, timm_vision_transformer_large, Adam, cuda, (pt2) foreach: 19.04951794523123
- compile_time, timm_vision_transformer_large, Adam, cuda, (pt2) fused: 4.2639227368248


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6167804843

cc @janeyx99


Closing, nothing new reported. Will be implicitly fixed by https://github.com/pytorch/benchmark/pull/1931 and https://github.com/pytorch/benchmark/pull/1932
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '5c903fb12cd09ddb306364d06b2c8da581bf481b'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 2baa4c49288efeded2fad677b2f28570b0ce858b

Affected PyTorch commit: 5c903fb12cd09ddb306364d06b2c8da581bf481b

Affected Tests:
- resnet18, Adam, cuda, (pt2) no_foreach: -99.98417%
- resnet18, Adam, cuda, (pt2) foreach: -58.20034%
- resnet18, AdamW, cuda, (pt2) no_foreach: +621285.20671%
- resnet18, AdamW, cuda, (pt2) foreach: +167.85725%
- compile_time, resnet18, Adam, cuda, (pt2) no_foreach: +576.39475%
- compile_time, resnet18, Adamax, cuda, (pt2) foreach: +235.54790%
- compile_time, resnet18, RMSprop, cuda, (pt2) foreach: +2546.37004%
- compile_time, resnet18, NAdam, cuda, (pt2) foreach: +231.13998%
- timm_vision_transformer_large, Adadelta, cuda, (pt2) no_foreach: +476683.80457%
- timm_vision_transformer_large, Adagrad, cuda, (pt2) no_foreach: -99.98632%
- timm_vision_transformer_large, ASGD, cuda, (pt2) no_foreach: -99.98328%
- timm_vision_transformer_large, ASGD, cuda, (pt2) foreach: -67.36265%
- compile_time, timm_vision_transformer_large, Adagrad, cuda, (pt2) no_foreach: +10562.16702%
- compile_time, timm_vision_transformer_large, ASGD, cuda, (pt2) no_foreach: +817.51312%


Tests that were no longer run on affected commit:
- timm_vision_transformer_large, Adamax, cuda, default: 0.08233604347333312
- timm_vision_transformer_large, Adamax, cuda, (pt2) no_foreach: 0.07465639725948374
- timm_vision_transformer_large, Adamax, cuda, no_foreach: 0.07512599943826596
- timm_vision_transformer_large, Adamax, cuda, differentiable: 0.07935263402760029
- timm_vision_transformer_large, Adamax, cuda, (pt2) foreach: 0.08465497987344861
- timm_vision_transformer_large, Adamax, cuda, foreach: 0.0820629281612734
- compile_time, timm_vision_transformer_large, Adamax, cuda, (pt2) no_foreach: 314.7724319839229
- compile_time, timm_vision_transformer_large, Adamax, cuda, (pt2) foreach: 11.282057445806762


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6193735023

cc @janeyx99


Closing, nothing new reported. Will be implicitly fixed by https://github.com/pytorch/benchmark/pull/1931 and https://github.com/pytorch/benchmark/pull/1932
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '4a4af8fb1cc638ea249fb5afc2b2d1ade091d204'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 4a4af8fb1cc638ea249fb5afc2b2d1ade091d204

Affected PyTorch commit: 4a4af8fb1cc638ea249fb5afc2b2d1ade091d204

Affected Tests:
- timm_vision_transformer_large, Adadelta, cuda, (pt2) no_foreach: -99.97922%
- compile_time, timm_vision_transformer_large, Adadelta, cuda, (pt2) no_foreach: -13934.39166%
- compile_time, timm_vision_transformer_large, Adam, cuda, (pt2) no_foreach: -204.86572%
- resnet18, Adagrad, cuda, (pt2) no_foreach: -99.98401%
- resnet18, Adam, cuda, (pt2) no_foreach: -99.98377%
- resnet18, Adam, cuda, (pt2) foreach: -57.11772%
- resnet18, AdamW, cuda, (pt2) no_foreach: -99.98388%
- resnet18, AdamW, cuda, (pt2) foreach: -64.42269%
- compile_time, resnet18, Adagrad, cuda, (pt2) no_foreach: +358.75174%
- compile_time, resnet18, Adam, cuda, (pt2) no_foreach: +480.23172%
- compile_time, resnet18, AdamW, cuda, (pt2) no_foreach: +599.73875%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- timm_vision_transformer_large, Adam, cuda, (pt2) fused: 0.03037401388532349
- timm_vision_transformer_large, Adam, cuda, fused: 0.02655341257341206
- timm_vision_transformer_large, Adam, cuda, fused, capturable: 0.026600698893889784
- compile_time, timm_vision_transformer_large, Adam, cuda, (pt2) fused: 4.400100964276741


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6211451398

cc @janeyx99


Closing, nothing new reported. Will be implicitly fixed by https://github.com/pytorch/benchmark/pull/1931 and https://github.com/pytorch/benchmark/pull/1932
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '4a4af8fb1cc638ea249fb5afc2b2d1ade091d204'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 4a4af8fb1cc638ea249fb5afc2b2d1ade091d204

Affected PyTorch commit: 4a4af8fb1cc638ea249fb5afc2b2d1ade091d204

Affected Tests:
- timm_vision_transformer_large, Adam, cuda, foreach: +41.62049%
- DALLE2_pytorch, AdamW, cuda, default: +32.02154%
- DALLE2_pytorch, AdamW, cuda, amsgrad, maximize: +33.02614%
- resnet18, Adam, cuda, (pt2) no_foreach: -99.98338%
- resnet18, Adam, cuda, (pt2) foreach: -55.38268%
- compile_time, resnet18, Adam, cuda, (pt2) no_foreach: +543.08771%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6230983259

cc @janeyx99


Closing, nothing new reported. Will be implicitly fixed by https://github.com/pytorch/benchmark/pull/1931 and https://github.com/pytorch/benchmark/pull/1932
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on '00ae5fa31130b0c97b541c18a773d35405d46bb2'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 4d1773d0c5ca48575ea0c53e3a043a5bc434776b

Affected PyTorch commit: 00ae5fa31130b0c97b541c18a773d35405d46bb2

Affected Tests:
- test_train[lennard_jones-cuda-eager]_latency: -14.46901%
- test_eval[pyhpc_isoneutral_mixing-cuda-eager]_latency: +18.06381%
- test_eval[pyhpc_turbulent_kinetic_energy-cuda-eager]_latency: +59.41351%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6251197388

cc @xuzhao9


Bisection test: https://github.com/pytorch/benchmark/actions/runs/6305459798

Output:
```
{
  "target_repo": "pytorch",
  "start": "98208e51603a27e4b88d933a7c7edc18aada4b9e",
  "end": "1b3e5b53f336e782d6706eb6e61ec363f2fb2c30",
  "result": []
}
```

Closed.
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '00ae5fa31130b0c97b541c18a773d35405d46bb2'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 4d1773d0c5ca48575ea0c53e3a043a5bc434776b

Affected PyTorch commit: 00ae5fa31130b0c97b541c18a773d35405d46bb2

Affected Tests:
- timm_vision_transformer_large, Adam, cuda, differentiable: +41.88491%
- phlippe_resnet, SGD, cuda, no_foreach: -33.90851%
- resnet18, Adam, cuda, (pt2) no_foreach: -99.98218%
- resnet18, Adam, cuda, (pt2) foreach: -55.36035%
- resnet18, NAdam, cuda, (pt2) default: -45.29823%
- resnet18, NAdam, cuda, (pt2) no_foreach: +1016451.48978%
- resnet18, NAdam, cuda, (pt2) foreach: +122.87167%
- compile_time, resnet18, Adam, cuda, (pt2) no_foreach: +411.52155%
- compile_time, resnet18, NAdam, cuda, (pt2) default: +974.04609%
- compile_time, resnet18, NAdam, cuda, (pt2) foreach: +1263.48322%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6257022033

cc @janeyx99


Closing, nothing new reported. Will be implicitly fixed by https://github.com/pytorch/benchmark/pull/1931 and https://github.com/pytorch/benchmark/pull/1932
Repository: pytorch/benchmark
Issue: CI: No space left on device error
Root cause: Environment Incompatibility

https://github.com/pytorch/benchmark/actions/runs/6279881658/job/17056289916?pr=1921

Showing up here https://github.com/pytorch/benchmark/actions/runs/6279881658/job/17056289916?pr=1921

The runner is A10G, which uses AWS linux.g5.4xlarge.nvidia.gpu type instance.

@malfet Do you think we could increase the disk size for this type of runner?

@xuzhao9 depends on how much disk space we might need. Perhaps we can create a specific runner type for torchbench runs if we need a really large disk space...

@malfet I use this command to get the docker size:
```
$ docker manifest inspect -v ghcr.io/pytorch/torchbench:latest| grep size | awk -F ':' '{sum+=$NF} END {print sum}'| numfmt --to=iec-i
27Gi
```

During CI, it might require more space than the docker image. But I am thinking 150G should be safe...
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '90c406a3a198b8f45682a9979b4c091ec5dc647e'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: d8fd7f96b9537ce5a88601e7cf410b790fdd12c1

Affected PyTorch commit: 90c406a3a198b8f45682a9979b4c091ec5dc647e

Affected Tests:
- hf_Whisper, Adamax, cuda, foreach: +37.88576%
- timm_vision_transformer_large, RMSprop, cuda, (pt2) no_foreach: +312279.40870%
- compile_time, timm_vision_transformer_large, Adam, cuda, (pt2) foreach: +1304.15035%
- compile_time, resnet18, Adadelta, cuda, (pt2) foreach: +1542.71367%
- compile_time, resnet18, AdamW, cuda, (pt2) foreach: +1136.77162%
- compile_time, resnet18, AdamW, cuda, (pt2) fused: +296.44309%
- compile_time, resnet18, ASGD, cuda, (pt2) foreach: +986.44063%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- timm_vision_transformer_large, Adamax, cuda, default: 0.08410019232542254
- timm_vision_transformer_large, Adamax, cuda, maximize: 0.08961313066538423


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6281496870

cc @janeyx99


Closing, nothing new reported. Will be implicitly fixed by https://github.com/pytorch/benchmark/pull/1931 and https://github.com/pytorch/benchmark/pull/1932
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '90c406a3a198b8f45682a9979b4c091ec5dc647e'
Root cause: Others


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 90c406a3a198b8f45682a9979b4c091ec5dc647e

Affected PyTorch commit: 90c406a3a198b8f45682a9979b4c091ec5dc647e

Affected Tests:
- timm_vision_transformer_large, RMSprop, cuda, (pt2) no_foreach: -99.96645%
- compile_time, timm_vision_transformer_large, RMSprop, cuda, (pt2) no_foreach: -9387.62493%
- resnet18, Adagrad, cuda, (pt2) no_foreach: +597185.21617%
- resnet18, ASGD, cuda, (pt2) no_foreach: +503823.35555%
- resnet18, ASGD, cuda, (pt2) foreach: +238.29607%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6287880146

cc @janeyx99


Closing, nothing new reported. Will be implicitly fixed by https://github.com/pytorch/benchmark/pull/1931 and https://github.com/pytorch/benchmark/pull/1932
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '90c406a3a198b8f45682a9979b4c091ec5dc647e'
Root cause: External Dependency Breakage


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 90c406a3a198b8f45682a9979b4c091ec5dc647e

Affected PyTorch commit: 90c406a3a198b8f45682a9979b4c091ec5dc647e

Affected Tests:
- timm_vision_transformer_large, NAdam, cuda, (pt2) foreach: +714836.98800%
- compile_time, timm_vision_transformer_large, RMSprop, cuda, (pt2) foreach: +920.61117%
- resnet18, Adagrad, cuda, (pt2) no_foreach: -99.98313%
- resnet18, ASGD, cuda, (pt2) no_foreach: -99.97986%
- resnet18, ASGD, cuda, (pt2) foreach: -69.88764%
- compile_time, resnet18, Adagrad, cuda, (pt2) no_foreach: +370.52887%
- compile_time, resnet18, ASGD, cuda, (pt2) no_foreach: +724.13615%


Tests that were no longer run on affected commit:
- timm_vision_transformer_large, Adamax, cuda, default: 0.08446503132775736
- timm_vision_transformer_large, Adamax, cuda, maximize: 0.08891550899716094
- timm_vision_transformer_large, Adamax, cuda, (pt2) no_foreach: 0.07448057901153031
- timm_vision_transformer_large, Adamax, cuda, no_foreach: 0.07214673500857316
- timm_vision_transformer_large, Adamax, cuda, differentiable: 0.08244337833214861
- timm_vision_transformer_large, Adamax, cuda, (pt2) foreach: 0.08642790532515694
- timm_vision_transformer_large, Adamax, cuda, foreach: 0.08295293099945411
- compile_time, timm_vision_transformer_large, Adamax, cuda, (pt2) no_foreach: 308.4128124323518
- compile_time, timm_vision_transformer_large, Adamax, cuda, (pt2) foreach: 10.28842167367111


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6294720433

cc @janeyx99


Closing, nothing new reported.
Repository: pytorch/benchmark
Issue: benchmarks/dynamo/common.py should be identical to one in core
Root cause: Architectural Constraint

I noticed this file was checked in https://github.com/pytorch/benchmark/blob/main/userbenchmark/dynamo/common.py

This is a fork of the equivalent file in core, `common.py` changes a lot so it might best if we package up the upstream `common.py` into its own package so code will be identical

@desertfire and I thinks we should setup code synchronization between 2 repos from the same internal copy.

Currently this is blocked by an internal sync ShipIt bug, will continue working on this after the bug is resolved.

Fixed by https://github.com/pytorch/benchmark/pull/1956
Repository: pytorch/benchmark
Issue: V3 Performance Signal Detected by TorchBench Userbenchmark "torch-nightly" on 'd7520d8668dc08f7bed27a64f006c909006e653a'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: ab61acc20ccd35835b9cd7f587f6a909839cf57f

Affected PyTorch commit: d7520d8668dc08f7bed27a64f006c909006e653a

Affected Tests:
- test_train[phlippe_resnet-cuda-eager]_latency: +11.59415%
- test_eval[pyhpc_turbulent_kinetic_energy-cuda-eager]_latency: +87.47630%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6328520935

cc @xuzhao9


Bisection workflow: https://github.com/pytorch/benchmark/actions/runs/6331648366

```
{
  "target_repo": "pytorch",
  "start": "ac60638c6c74149f05c7980f571afa6fe0661b0c",
  "end": "869226bf947bad0c4ba740c7b56ffe38acdddf52",
  "result": []
}
```
Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on 'd7520d8668dc08f7bed27a64f006c909006e653a'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: ab61acc20ccd35835b9cd7f587f6a909839cf57f

Affected PyTorch commit: d7520d8668dc08f7bed27a64f006c909006e653a

Affected Tests:


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6334375922

cc @janeyx99