Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on '6c94bfc7da1b91f110929dfb7f8b5fa6d6baaa0a'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: abcc3fadc88502dd64dc52a0a208dc0606401f33

Affected PyTorch commit: 6c94bfc7da1b91f110929dfb7f8b5fa6d6baaa0a

Affected Tests:
- sam, RAdam, cuda, no_foreach: +55.36671%
- timm_vision_transformer_large, RAdam, cuda, differentiable: +79.71807%
- DALLE2_pytorch, RAdam, cuda, foreach: +31.34282%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- phi_1_5, Adadelta, cuda, default: 0.12255744752474129
- phi_1_5, Adadelta, cuda, maximize: 0.13190429750829935
- phi_1_5, Adadelta, cuda, no_foreach: 0.11824519420042634
- phi_1_5, Adadelta, cuda, differentiable: 0.13133868179284036
- phi_1_5, Adadelta, cuda, foreach: 0.12240869761444628
- phi_1_5, Adagrad, cuda, default: 0.0581908165011555
- phi_1_5, Adagrad, cuda, maximize: 0.06844953897719581
- phi_1_5, Adagrad, cuda, no_foreach: 0.047819876577705145
- phi_1_5, Adagrad, cuda, differentiable: 0.047863224893808363
- phi_1_5, Adagrad, cuda, foreach: 0.05816376032307744
- phi_1_5, Adam, cuda, default: 0.08269835961982608
- phi_1_5, Adam, cuda, amsgrad, maximize: 0.10444057499989867
- phi_1_5, Adam, cuda, no_foreach: 0.07898491869370143
- phi_1_5, Adam, cuda, differentiable: 0.07926848980908592
- phi_1_5, Adam, cuda, foreach: 0.08247320866212249
- phi_1_5, Adam, cuda, foreach, maximize, capturable: 0.10084142140112817
- phi_1_5, Adam, cuda, foreach, maximize, capturable, amsgrad: 0.11402787826955318
- phi_1_5, Adam, cuda, fused: 0.03636672869324684
- phi_1_5, Adam, cuda, fused, amsgrad, maximize: 0.04160186969675124
- phi_1_5, Adam, cuda, fused, capturable: 0.03635919946245849
- phi_1_5, Adam, cuda, fused, capturable, amsgrad: 0.04166154838167131
- phi_1_5, AdamW, cuda, default: 0.09202117302144568
- phi_1_5, AdamW, cuda, amsgrad, maximize: 0.11349813546985388
- phi_1_5, AdamW, cuda, no_foreach: 0.08788888063281775
- phi_1_5, AdamW, cuda, differentiable: 0.08820240339264274
- phi_1_5, AdamW, cuda, foreach: 0.0916012991219759
- phi_1_5, AdamW, cuda, foreach, maximize, capturable: 0.10943374643102288
- phi_1_5, AdamW, cuda, foreach, maximize, capturable, amsgrad: 0.12308404175564647
- phi_1_5, AdamW, cuda, fused: 0.0373490396887064
- phi_1_5, AdamW, cuda, fused, amsgrad, maximize: 0.04202162390574813
- phi_1_5, AdamW, cuda, fused, capturable: 0.03736672466620803
- phi_1_5, AdamW, cuda, fused, capturable, amsgrad: 0.04207098446786404
- phi_1_5, Adamax, cuda, default: 0.10861710552126169
- phi_1_5, Adamax, cuda, maximize: 0.11384346662089229
- phi_1_5, Adamax, cuda, no_foreach: 0.08919075690209866
- phi_1_5, Adamax, cuda, differentiable: 0.09840354835614562
- phi_1_5, Adamax, cuda, foreach: 0.1051824779715389
- phi_1_5, ASGD, cuda, default: 0.06576364010106772
- phi_1_5, ASGD, cuda, maximize: 0.0747058962782224
- phi_1_5, ASGD, cuda, no_foreach: 0.05903661088086665
- phi_1_5, ASGD, cuda, differentiable: 0.05928382102865726
- phi_1_5, ASGD, cuda, foreach: 0.06564374943263829
- phi_1_5, SGD, cuda, default: 0.013404742977581919
- phi_1_5, SGD, cuda, maximize: 0.022514259023591877
- phi_1_5, SGD, cuda, no_foreach: 0.013356827455572783
- phi_1_5, SGD, cuda, differentiable: 0.013383566820994019
- phi_1_5, SGD, cuda, foreach: 0.013373248768039048
- phi_1_5, SGD, cuda, foreach, momentum=0.9, nesterov: 0.049138896726071836
- phi_1_5, SGD, cuda, foreach, momentum=0.9: 0.035813794611021875
- phi_1_5, RAdam, cuda, default: 0.1018761140294373
- phi_1_5, RAdam, cuda, no_foreach: 0.0664774333126843
- phi_1_5, RAdam, cuda, differentiable: 0.06647269940003753
- phi_1_5, RAdam, cuda, foreach: 0.10166986496187747
- phi_1_5, Rprop, cuda, default: 0.14392798556946218
- phi_1_5, Rprop, cuda, maximize: 0.16215909365564585
- phi_1_5, Rprop, cuda, no_foreach: 0.15061860950663686
- phi_1_5, Rprop, cuda, differentiable: 0.1597048882395029
- phi_1_5, Rprop, cuda, foreach: 0.1437057692091912
- phi_1_5, RMSprop, cuda, default: 0.058108772011473776
- phi_1_5, RMSprop, cuda, maximize: 0.06829443899914622
- phi_1_5, RMSprop, cuda, no_foreach: 0.05643115141429007
- phi_1_5, RMSprop, cuda, differentiable: 0.056448604073375465
- phi_1_5, RMSprop, cuda, foreach: 0.05810281769372523
- phi_1_5, NAdam, cuda, default: 0.10023320419713855
- phi_1_5, NAdam, cuda, no_foreach: 0.09622718393802643
- phi_1_5, NAdam, cuda, differentiable: 0.11452715983614326
- phi_1_5, NAdam, cuda, foreach: 0.10001550056040287


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6542362524

cc @janeyx99


:0 new model