Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on 'fecde478ac83edf78e7d0e9d11ab73cb1580f6cf'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: d7520d8668dc08f7bed27a64f006c909006e653a

Affected PyTorch commit: fecde478ac83edf78e7d0e9d11ab73cb1580f6cf

Affected Tests:
- stable_diffusion_unet, RAdam, cuda, no_foreach: -47.85513%
- stable_diffusion_unet, RAdam, cuda, differentiable: -48.20885%
- DALLE2_pytorch, Adam, cuda, amsgrad, maximize: +32.97666%
- DALLE2_pytorch, Adam, cuda, fused, amsgrad, maximize: +39.28866%
- DALLE2_pytorch, Adam, cuda, fused, capturable, amsgrad: +44.78136%
- DALLE2_pytorch, AdamW, cuda, default: +37.12394%
- DALLE2_pytorch, AdamW, cuda, amsgrad, maximize: +37.40143%
- DALLE2_pytorch, AdamW, cuda, foreach: +37.27161%
- DALLE2_pytorch, AdamW, cuda, fused, amsgrad, maximize: +43.04382%
- DALLE2_pytorch, AdamW, cuda, fused, capturable, amsgrad: +40.58533%
- DALLE2_pytorch, RAdam, cuda, default: +35.76401%
- DALLE2_pytorch, RAdam, cuda, foreach: +35.42852%


Tests that were no longer run on affected commit:
- fambench_xlmr, Adadelta, cuda, default: 0.04931840759236365
- fambench_xlmr, Adadelta, cuda, maximize: 0.05153135990258306
- fambench_xlmr, Adadelta, cuda, no_foreach: 0.04757571630179882
- fambench_xlmr, Adadelta, cuda, differentiable: 0.0546946327900514
- fambench_xlmr, Adadelta, cuda, foreach: 0.047904940496664494
- fambench_xlmr, Adagrad, cuda, default: 0.02324453688925132
- fambench_xlmr, Adagrad, cuda, maximize: 0.026643992902245374
- fambench_xlmr, Adagrad, cuda, no_foreach: 0.01963424375280738
- fambench_xlmr, Adagrad, cuda, differentiable: 0.020073896297253667
- fambench_xlmr, Adagrad, cuda, foreach: 0.023189703293610364
- fambench_xlmr, Adam, cuda, default: 0.032094168791081755
- fambench_xlmr, Adam, cuda, amsgrad, maximize: 0.04074071650393307
- fambench_xlmr, Adam, cuda, no_foreach: 0.03387390018906444
- fambench_xlmr, Adam, cuda, differentiable: 0.06436306480318307
- fambench_xlmr, Adam, cuda, foreach: 0.03208067340310663
- fambench_xlmr, Adam, cuda, foreach, maximize, capturable: 0.04175691159907728
- fambench_xlmr, Adam, cuda, foreach, maximize, capturable, amsgrad: 0.047025109408423305
- fambench_xlmr, Adam, cuda, fused: 0.014776056603295729
- fambench_xlmr, Adam, cuda, fused, amsgrad, maximize: 0.017104668548563497
- fambench_xlmr, Adam, cuda, fused, capturable: 0.014802309195511043
- fambench_xlmr, Adam, cuda, fused, capturable, amsgrad: 0.017137673654360697
- fambench_xlmr, AdamW, cuda, default: 0.03573932839790359
- fambench_xlmr, AdamW, cuda, amsgrad, maximize: 0.04440390840172768
- fambench_xlmr, AdamW, cuda, no_foreach: 0.039004817698150875
- fambench_xlmr, AdamW, cuda, differentiable: 0.06728450329974293
- fambench_xlmr, AdamW, cuda, foreach: 0.03569157550809905
- fambench_xlmr, AdamW, cuda, foreach, maximize, capturable: 0.04534340070094913
- fambench_xlmr, AdamW, cuda, foreach, maximize, capturable, amsgrad: 0.050630067195743325
- fambench_xlmr, AdamW, cuda, fused: 0.015210027300054208
- fambench_xlmr, AdamW, cuda, fused, amsgrad, maximize: 0.01735305179608986
- fambench_xlmr, AdamW, cuda, fused, capturable: 0.015243320155423134
- fambench_xlmr, AdamW, cuda, fused, capturable, amsgrad: 0.017356748902238907
- fambench_xlmr, Adamax, cuda, default: 0.04442148520611226
- fambench_xlmr, Adamax, cuda, maximize: 0.04790715900016949
- fambench_xlmr, Adamax, cuda, no_foreach: 0.04123727260157466
- fambench_xlmr, Adamax, cuda, differentiable: 0.05030385409481823
- fambench_xlmr, Adamax, cuda, foreach: 0.044373169704340396
- fambench_xlmr, ASGD, cuda, default: 0.044388337910640986
- fambench_xlmr, ASGD, cuda, maximize: 0.04762052539736032
- fambench_xlmr, ASGD, cuda, no_foreach: 0.06063354030484334
- fambench_xlmr, ASGD, cuda, differentiable: 0.061320055602118374
- fambench_xlmr, ASGD, cuda, foreach: 0.04394585069967434
- fambench_xlmr, SGD, cuda, default: 0.0052511856600176545
- fambench_xlmr, SGD, cuda, maximize: 0.008986131734369943
- fambench_xlmr, SGD, cuda, no_foreach: 0.006254356020363048
- fambench_xlmr, SGD, cuda, differentiable: 0.005249961640220136
- fambench_xlmr, SGD, cuda, foreach: 0.0052385661704465744
- fambench_xlmr, SGD, cuda, foreach, momentum=0.9, nesterov: 0.019441048003500327
- fambench_xlmr, SGD, cuda, foreach, momentum=0.9: 0.014189434406580404
- fambench_xlmr, RAdam, cuda, default: 0.039337284991052004
- fambench_xlmr, RAdam, cuda, no_foreach: 0.06273950749309734
- fambench_xlmr, RAdam, cuda, differentiable: 0.06426761120092124
- fambench_xlmr, RAdam, cuda, foreach: 0.03932071949820966
- fambench_xlmr, Rprop, cuda, default: 0.06308145800139756
- fambench_xlmr, Rprop, cuda, maximize: 0.07025696609634906
- fambench_xlmr, Rprop, cuda, no_foreach: 0.0849860606637473
- fambench_xlmr, Rprop, cuda, differentiable: 0.09017868731946994
- fambench_xlmr, Rprop, cuda, foreach: 0.06307932320050895
- fambench_xlmr, RMSprop, cuda, default: 0.0231006411020644
- fambench_xlmr, RMSprop, cuda, maximize: 0.0266125755966641
- fambench_xlmr, RMSprop, cuda, no_foreach: 0.02264648169511929
- fambench_xlmr, RMSprop, cuda, differentiable: 0.023086359398439525
- fambench_xlmr, RMSprop, cuda, foreach: 0.023027285002171993
- fambench_xlmr, NAdam, cuda, default: 0.03897200330393389
- fambench_xlmr, NAdam, cuda, no_foreach: 0.042920765199232845
- fambench_xlmr, NAdam, cuda, differentiable: 0.06999223566769312
- fambench_xlmr, NAdam, cuda, foreach: 0.038871330209076405


Tests that were newly added on affected commit:


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6347022695

cc @janeyx99


conglomerating noise in #1988