Repository: pytorch/benchmark
Issue: Optim Perf Signal Detected by TorchBench CI on 'ab61acc20ccd35835b9cd7f587f6a909839cf57f'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 90c406a3a198b8f45682a9979b4c091ec5dc647e

Affected PyTorch commit: ab61acc20ccd35835b9cd7f587f6a909839cf57f

Affected Tests:


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- hf_clip, Adadelta, cuda, default: 0.01312899379990995
- hf_clip, Adadelta, cuda, maximize: 0.0141005489975214
- hf_clip, Adadelta, cuda, no_foreach: 0.034115590003784745
- hf_clip, Adadelta, cuda, differentiable: 0.040964748099213466
- hf_clip, Adadelta, cuda, foreach: 0.013030943399644456
- hf_clip, Adagrad, cuda, default: 0.006216102920006961
- hf_clip, Adagrad, cuda, maximize: 0.0071839390997774895
- hf_clip, Adagrad, cuda, no_foreach: 0.018905463302507995
- hf_clip, Adagrad, cuda, differentiable: 0.019783602454117498
- hf_clip, Adagrad, cuda, foreach: 0.006219089880469255
- hf_clip, Adam, cuda, default: 0.009016100264852868
- hf_clip, Adam, cuda, amsgrad, maximize: 0.011321785749169066
- hf_clip, Adam, cuda, no_foreach: 0.03156509170075879
- hf_clip, Adam, cuda, differentiable: 0.062278598599368705
- hf_clip, Adam, cuda, foreach: 0.008962108365570506
- hf_clip, Adam, cuda, foreach, maximize, capturable: 0.013933931002975443
- hf_clip, Adam, cuda, foreach, maximize, capturable, amsgrad: 0.015344426897354425
- hf_clip, Adam, cuda, fused: 0.004888425309909508
- hf_clip, Adam, cuda, fused, amsgrad, maximize: 0.005012183240032755
- hf_clip, Adam, cuda, fused, capturable: 0.004883712059818208
- hf_clip, Adam, cuda, fused, capturable, amsgrad: 0.005007396010332741
- hf_clip, AdamW, cuda, default: 0.00999225180130452
- hf_clip, AdamW, cuda, amsgrad, maximize: 0.012354723800672218
- hf_clip, AdamW, cuda, no_foreach: 0.034762969403527674
- hf_clip, AdamW, cuda, differentiable: 0.06518846510443836
- hf_clip, AdamW, cuda, foreach: 0.009914563330433642
- hf_clip, AdamW, cuda, foreach, maximize, capturable: 0.014872383000329137
- hf_clip, AdamW, cuda, foreach, maximize, capturable, amsgrad: 0.016331211599754168
- hf_clip, AdamW, cuda, fused: 0.005032310499809682
- hf_clip, AdamW, cuda, fused, amsgrad, maximize: 0.00506931910989806
- hf_clip, AdamW, cuda, fused, capturable: 0.005037551290006377
- hf_clip, AdamW, cuda, fused, capturable, amsgrad: 0.005087913139723241
- hf_clip, Adamax, cuda, default: 0.03292286570067517
- hf_clip, Adamax, cuda, maximize: 0.031934282300062476
- hf_clip, Adamax, cuda, no_foreach: 0.04073155950172804
- hf_clip, Adamax, cuda, differentiable: 0.04841220210073516
- hf_clip, Adamax, cuda, foreach: 0.02976354170241393
- hf_clip, ASGD, cuda, default: 0.028487998998025434
- hf_clip, ASGD, cuda, maximize: 0.0294245017983485
- hf_clip, ASGD, cuda, no_foreach: 0.05224683689884842
- hf_clip, ASGD, cuda, differentiable: 0.05297557759913616
- hf_clip, ASGD, cuda, foreach: 0.028194729797542097
- hf_clip, SGD, cuda, default: 0.0014253706749877892
- hf_clip, SGD, cuda, maximize: 0.002434341360349208
- hf_clip, SGD, cuda, no_foreach: 0.0031756557896733283
- hf_clip, SGD, cuda, differentiable: 0.0014277938098530285
- hf_clip, SGD, cuda, foreach: 0.0014286950847599655
- hf_clip, SGD, cuda, foreach, momentum=0.9, nesterov: 0.005231500529916957
- hf_clip, SGD, cuda, foreach, momentum=0.9: 0.003813607519841753
- hf_clip, RAdam, cuda, default: 0.01101170004985761
- hf_clip, RAdam, cuda, no_foreach: 0.05173892619786784
- hf_clip, RAdam, cuda, differentiable: 0.05368491469998844
- hf_clip, RAdam, cuda, foreach: 0.010976311549893581
- hf_clip, Rprop, cuda, default: 0.04535560359945521
- hf_clip, Rprop, cuda, maximize: 0.04737445369828493
- hf_clip, Rprop, cuda, no_foreach: 0.0683637936017476
- hf_clip, Rprop, cuda, differentiable: 0.07380215869634413
- hf_clip, Rprop, cuda, foreach: 0.045027171494439244
- hf_clip, RMSprop, cuda, default: 0.006218328850227408
- hf_clip, RMSprop, cuda, maximize: 0.0071756272501079365
- hf_clip, RMSprop, cuda, no_foreach: 0.01706607024825644
- hf_clip, RMSprop, cuda, differentiable: 0.018339644352090546
- hf_clip, RMSprop, cuda, foreach: 0.006207530539832078
- hf_clip, NAdam, cuda, default: 0.010804415849270298
- hf_clip, NAdam, cuda, no_foreach: 0.03718638860154897
- hf_clip, NAdam, cuda, differentiable: 0.0655736547021661
- hf_clip, NAdam, cuda, foreach: 0.010767862200737


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/6321234747

cc @janeyx99


Yay hf_clip