Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '61239df555df02e8c60a2ad63363878a2a57c161'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 329bb2a33e40f4bc76b2e061b180d3234984c91b

Affected PyTorch commit: 61239df555df02e8c60a2ad63363878a2a57c161

Affected Tests:
- timm_vision_transformer, Adadelta, cuda, differentiable: +29.87629%
- timm_vision_transformer, Adagrad, cuda, maximize: +23.82667%
- timm_vision_transformer, Adam, cuda, (pt2) differentiable: +11.50897%
- timm_vision_transformer, AdamW, cuda, foreach, maximize, capturable: +12.23035%
- timm_vision_transformer, Adamax, cuda, foreach: +10.09680%
- timm_vision_transformer, ASGD, cuda, default: +21.40052%
- timm_vision_transformer, ASGD, cuda, no_foreach: -15.23271%
- timm_vision_transformer, ASGD, cuda, (pt2) differentiable: +15.58055%
- detectron2_maskrcnn_r_101_fpn, Adagrad, cuda, foreach: +24.35258%
- detectron2_maskrcnn_r_101_fpn, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +13.32298%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) default: +32.15407%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) maximize: +13.61581%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) foreach: +20.32258%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, default: +14.93400%
- tts_angular, Adagrad, cuda, default: +11.94997%
- tts_angular, AdamW, cuda, foreach, maximize, capturable: +18.74591%
- maml_omniglot, Adam, cuda, (pt2) amsgrad, maximize: +19.85630%
- maml_omniglot, ASGD, cuda, (pt2) default: +42.69573%
- maml_omniglot, ASGD, cuda, default: +34.52687%
- maml_omniglot, NAdam, cuda, foreach: +18.36359%
- basic_gnn_sage, Adagrad, cuda, maximize: +11.16043%
- basic_gnn_sage, Adam, cuda, foreach, maximize, capturable, amsgrad: +12.24314%
- basic_gnn_sage, Adam, cuda, (pt2) fused: +12.45441%
- basic_gnn_sage, AdamW, cuda, (pt2) amsgrad, maximize: +16.02690%
- basic_gnn_sage, AdamW, cuda, foreach: +12.04578%
- basic_gnn_sage, Rprop, cuda, (pt2) default: +37.11663%
- basic_gnn_sage, Rprop, cuda, (pt2) foreach: +21.89400%
- dcgan, Adagrad, cuda, (pt2) foreach: +31.19294%
- dcgan, Adam, cuda, foreach, maximize, capturable: +11.73770%
- dcgan, AdamW, cuda, amsgrad, maximize: +11.51673%
- dcgan, AdamW, cuda, (pt2) foreach: +20.93707%
- dcgan, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -10.10027%
- dcgan, Adamax, cuda, (pt2) foreach: -14.24871%
- dcgan, Adamax, cuda, foreach: +33.33300%
- dcgan, ASGD, cuda, foreach: +20.04249%
- dcgan, Rprop, cuda, maximize: +34.12127%
- hf_BigBird, Adadelta, cuda, (pt2) maximize: -12.07825%
- mnasnet1_0, Adagrad, cuda, (pt2) default: +11.99654%
- mnasnet1_0, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +18.51464%
- mnasnet1_0, Adamax, cuda, (pt2) default: +29.94030%
- mnasnet1_0, Rprop, cuda, (pt2) default: +10.90039%
- timm_resnest, Adadelta, cuda, (pt2) maximize: -13.07751%
- timm_resnest, Adagrad, cuda, (pt2) maximize: +19.46394%
- timm_resnest, Adam, cuda, foreach, maximize, capturable: +13.23123%
- timm_resnest, AdamW, cuda, default: +11.65157%
- fambench_xlmr, ASGD, cuda, (pt2) default: +13.05915%
- phlippe_densenet, Adadelta, cuda, maximize: -10.45729%
- phlippe_densenet, Adam, cuda, default: +10.85168%
- phlippe_densenet, Adam, cuda, foreach, maximize, capturable: +11.66973%
- detectron2_maskrcnn_r_50_fpn, ASGD, cuda, (pt2) default: +40.17876%
- detectron2_maskrcnn_r_50_fpn, Rprop, cuda, (pt2) default: +43.76327%
- detectron2_maskrcnn_r_50_fpn, Rprop, cuda, default: -11.01132%
- detectron2_maskrcnn_r_50_fpn, Rprop, cuda, (pt2) maximize: -14.54373%
- hf_T5_generate, Adagrad, cuda, (pt2) foreach: +30.20985%
- hf_T5_generate, Rprop, cuda, foreach: +46.46026%
- resnet18, Adagrad, cuda, foreach: -10.17351%
- resnet18, Adam, cuda, foreach: +14.51785%
- resnet18, Adamax, cuda, (pt2) default: +13.55498%
- resnet18, ASGD, cuda, (pt2) maximize: +15.90979%
- resnet18, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +13.56605%
- resnet18, SGD, cuda, (pt2) foreach, momentum=0.9: +12.31270%
- torchrec_dlrm, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +17.76496%
- torchrec_dlrm, SGD, cuda, (pt2) foreach, momentum=0.9: +13.48018%
- demucs, SGD, cuda, (pt2) foreach, momentum=0.9: +19.59262%
- Super_SloMo, Adamax, cuda, (pt2) default: -19.02363%
- Super_SloMo, ASGD, cuda, foreach: +10.76259%
- basic_gnn_edgecnn, Rprop, cuda, (pt2) default: +17.18296%
- basic_gnn_edgecnn, Rprop, cuda, maximize: -13.41306%
- mobilenet_v2, Adagrad, cuda, maximize: +44.28140%
- mobilenet_v2, Adagrad, cuda, foreach: +25.62562%
- mobilenet_v2, ASGD, cuda, foreach: +12.40204%
- mobilenet_v2, Rprop, cuda, foreach: +15.98176%
- detectron2_fasterrcnn_r_50_c4, ASGD, cuda, (pt2) foreach: +21.48187%
- detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +67.79239%
- detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9: +21.58385%
- detectron2_fasterrcnn_r_50_c4, Rprop, cuda, (pt2) maximize: +46.65272%
- resnet152, Adagrad, cuda, foreach: +14.45022%
- resnet152, Adamax, cuda, maximize: +10.97403%
- resnet152, ASGD, cuda, (pt2) default: -22.14231%
- resnet152, ASGD, cuda, (pt2) maximize: +15.66355%
- resnet152, RAdam, cuda, (pt2) default: +26.59496%
- resnet152, NAdam, cuda, (pt2) default: +13.99849%
- resnet152, NAdam, cuda, default: +15.41091%
- soft_actor_critic, Adadelta, cuda, default: -11.29230%
- soft_actor_critic, AdamW, cuda, default: +22.52360%
- soft_actor_critic, RMSprop, cuda, (pt2) foreach: +11.69501%
- timm_nfnet, Adagrad, cuda, foreach: +11.04522%
- timm_nfnet, ASGD, cuda, (pt2) default: +52.32632%
- timm_nfnet, ASGD, cuda, default: +21.43094%
- timm_nfnet, ASGD, cuda, maximize: +18.40929%
- densenet121, Adadelta, cuda, (pt2) default: -14.94077%
- densenet121, Adagrad, cuda, foreach: -10.07686%
- densenet121, Adamax, cuda, maximize: +19.47509%
- densenet121, Rprop, cuda, foreach: +39.87730%
- llama, Adam, cuda, amsgrad, maximize: +11.07379%
- mobilenet_v2_quantized_qat, Adagrad, cuda, default: +17.63492%
- timm_efficientdet, Adagrad, cuda, (pt2) foreach: -12.10186%
- timm_efficientdet, AdamW, cuda, (pt2) amsgrad, maximize: +12.38378%
- timm_efficientdet, Adamax, cuda, (pt2) default: +27.78125%
- timm_efficientdet, Rprop, cuda, (pt2) default: -24.04180%
- moco, Rprop, cuda, (pt2) foreach: +20.00088%
- resnext50_32x4d, Adam, cuda, (pt2) default: +10.62591%
- resnext50_32x4d, Adam, cuda, (pt2) foreach: -10.06393%
- fastNLP_Bert, AdamW, cuda, foreach, maximize, capturable: +11.50904%
- fastNLP_Bert, ASGD, cuda, (pt2) default: -14.21155%
- fastNLP_Bert, ASGD, cuda, (pt2) maximize: +12.67264%
- fastNLP_Bert, Rprop, cuda, (pt2) foreach: +15.60391%
- LearningToPaint, Adagrad, cuda, (pt2) default: -10.11772%
- LearningToPaint, Adam, cuda, (pt2) foreach: -13.18049%
- LearningToPaint, Adam, cuda, foreach, maximize, capturable, amsgrad: +11.89989%
- LearningToPaint, Adamax, cuda, (pt2) maximize: -12.72140%
- LearningToPaint, Adamax, cuda, (pt2) foreach: -14.56813%
- LearningToPaint, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +22.02774%
- LearningToPaint, SGD, cuda, (pt2) foreach, momentum=0.9: +25.60262%
- hf_Bart, ASGD, cuda, maximize: +15.26274%
- tacotron2, Adagrad, cuda, (pt2) foreach: +35.23871%
- tacotron2, ASGD, cuda, (pt2) foreach: +11.44371%
- tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +13.20373%
- tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9: +12.42022%
- tacotron2, Rprop, cuda, default: +28.17270%
- detectron2_fasterrcnn_r_101_fpn, Adamax, cuda, (pt2) default: +13.65030%
- detectron2_fasterrcnn_r_101_fpn, Adamax, cuda, (pt2) maximize: +16.53718%
- functorch_dp_cifar10, Adagrad, cuda, (pt2) default: +40.74818%
- functorch_dp_cifar10, Adagrad, cuda, default: +31.40623%
- functorch_dp_cifar10, Adagrad, cuda, foreach: +26.78851%
- functorch_dp_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +10.03327%
- functorch_dp_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9: +33.79975%
- squeezenet1_1, Adagrad, cuda, (pt2) foreach: +15.83943%
- squeezenet1_1, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -17.31265%
- squeezenet1_1, Adamax, cuda, (pt2) maximize: +38.50645%
- squeezenet1_1, ASGD, cuda, (pt2) foreach: +14.70139%
- squeezenet1_1, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +15.62235%
- squeezenet1_1, SGD, cuda, (pt2) foreach, momentum=0.9: +14.23887%
- squeezenet1_1, Rprop, cuda, foreach: +25.67648%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) default: +34.98251%
- hf_Reformer, ASGD, cuda, (pt2) foreach: +51.04420%
- hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +150.25907%
- hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9: +852.43180%
- phlippe_resnet, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -13.75466%
- phlippe_resnet, Adam, cuda, foreach, maximize, capturable, amsgrad: +14.46257%
- phlippe_resnet, Adamax, cuda, (pt2) default: +35.93306%
- phlippe_resnet, ASGD, cuda, (pt2) maximize: -22.41150%
- phlippe_resnet, ASGD, cuda, maximize: +20.14299%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +25.55209%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9: +29.30938%
- functorch_maml_omniglot, Adagrad, cuda, (pt2) default: +10.84665%
- functorch_maml_omniglot, Adagrad, cuda, maximize: +42.02219%
- functorch_maml_omniglot, Adam, cuda, foreach, maximize, capturable, amsgrad: -12.42640%
- functorch_maml_omniglot, AdamW, cuda, (pt2) default: +12.54426%
- functorch_maml_omniglot, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +14.77801%
- functorch_maml_omniglot, SGD, cuda, (pt2) default: +10.72208%
- functorch_maml_omniglot, SGD, cuda, (pt2) foreach: +12.40626%
- functorch_maml_omniglot, RAdam, cuda, foreach: +10.99758%
- pytorch_unet, Adamax, cuda, default: +10.86043%
- pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -29.80095%
- pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9: +68.75163%
- timm_efficientnet, AdamW, cuda, (pt2) foreach: +11.02021%
- timm_efficientnet, Adamax, cuda, (pt2) default: +10.85712%
- detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, (pt2) default: +25.10940%
- detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, default: +21.95017%
- detectron2_fasterrcnn_r_50_fpn, ASGD, cuda, default: +12.20150%
- pytorch_struct, Adagrad, cuda, default: +11.91020%
- pytorch_struct, Adagrad, cuda, maximize: +21.24154%
- pytorch_struct, AdamW, cuda, amsgrad, maximize: +10.77508%
- pytorch_struct, SGD, cuda, (pt2) foreach: +10.88017%
- pytorch_struct, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +16.04580%
- pytorch_struct, Rprop, cuda, (pt2) default: +46.71463%
- pytorch_struct, Rprop, cuda, default: +27.53447%
- hf_Bert, Adagrad, cuda, (pt2) foreach: -12.08794%
- hf_GPT2, ASGD, cuda, maximize: +12.54874%
- hf_GPT2, Rprop, cuda, (pt2) maximize: +17.94305%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, foreach: +31.38523%
- detectron2_maskrcnn_r_101_c4, ASGD, cuda, (pt2) default: +17.95032%
- hf_T5_base, Adagrad, cuda, maximize: +20.12939%
- hf_T5_base, Adamax, cuda, default: +21.80839%
- hf_T5_base, ASGD, cuda, default: +30.48980%
- basic_gnn_gin, Adagrad, cuda, default: +14.48688%
- basic_gnn_gin, Adam, cuda, (pt2) default: +17.72103%
- basic_gnn_gin, Adamax, cuda, (pt2) maximize: +11.09148%
- basic_gnn_gin, Rprop, cuda, maximize: +11.54226%
- basic_gnn_gin, Rprop, cuda, (pt2) foreach: +10.25500%
- shufflenet_v2_x1_0, Adadelta, cuda, (pt2) maximize: -13.06028%
- shufflenet_v2_x1_0, Adadelta, cuda, foreach: -14.55013%
- hf_Bert_large, Adagrad, cuda, foreach: +13.18638%
- hf_Bert_large, Rprop, cuda, maximize: +11.49977%
- resnet50_quantized_qat, Adadelta, cuda, (pt2) foreach: -10.80213%
- resnet50_quantized_qat, Adamax, cuda, foreach: +14.75559%
- resnet50_quantized_qat, Rprop, cuda, (pt2) foreach: -15.24115%
- detectron2_maskrcnn_r_50_c4, Adagrad, cuda, maximize: +13.78059%
- detectron2_maskrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +18.67060%
- detectron2_maskrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9: +15.32828%
- alexnet, Adadelta, cuda, (pt2) default: -11.10004%
- doctr_det_predictor, Adagrad, cuda, maximize: +39.31794%
- doctr_det_predictor, AdamW, cuda, foreach: +10.39087%
- doctr_det_predictor, Rprop, cuda, maximize: +17.54402%
- pytorch_CycleGAN_and_pix2pix, Adam, cuda, (pt2) foreach, maximize, capturable: +16.29817%
- pytorch_CycleGAN_and_pix2pix, ASGD, cuda, default: +10.10914%
- pytorch_CycleGAN_and_pix2pix, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +10.44466%
- pytorch_CycleGAN_and_pix2pix, SGD, cuda, (pt2) foreach, momentum=0.9: +10.12218%
- hf_Albert, ASGD, cuda, default: +17.57927%
- hf_Albert, ASGD, cuda, (pt2) foreach: +26.31712%
- detectron2_maskrcnn, Adagrad, cuda, (pt2) maximize: -10.67664%
- detectron2_maskrcnn, Adamax, cuda, (pt2) default: +16.34526%
- resnet50, Adadelta, cuda, (pt2) default: -10.94830%
- resnet50, Adadelta, cuda, (pt2) differentiable: +14.26704%
- resnet50, Adam, cuda, default: +10.95496%
- resnet50, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -16.84814%
- resnet50, AdamW, cuda, default: +14.70268%
- resnet50, Adamax, cuda, (pt2) maximize: +26.50778%
- resnet50, Adamax, cuda, (pt2) foreach: +22.02447%
- resnet50, RAdam, cuda, (pt2) differentiable: +16.94807%
- resnet50, RAdam, cuda, differentiable: +10.17915%
- resnet50, RAdam, cuda, (pt2) foreach: +22.94225%
- resnet50, Rprop, cuda, (pt2) no_foreach: +12.24908%
- resnet50, RMSprop, cuda, no_foreach: +17.99141%
- hf_DistilBert, ASGD, cuda, (pt2) default: +10.20016%
- hf_DistilBert, ASGD, cuda, (pt2) maximize: +16.51209%
- hf_DistilBert, ASGD, cuda, maximize: +38.86585%
- hf_DistilBert, Rprop, cuda, (pt2) foreach: +22.17492%
- basic_gnn_gcn, Adamax, cuda, default: +34.54067%
- basic_gnn_gcn, Rprop, cuda, foreach: +10.60907%
- doctr_reco_predictor, Adagrad, cuda, foreach: +14.83708%
- doctr_reco_predictor, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -10.75229%
- doctr_reco_predictor, Adamax, cuda, (pt2) maximize: +37.05120%
- doctr_reco_predictor, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -20.99821%
- attention_is_all_you_need_pytorch, Adadelta, cuda, (pt2) default: -10.62029%
- attention_is_all_you_need_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable: +18.19045%
- attention_is_all_you_need_pytorch, ASGD, cuda, (pt2) foreach: +18.10095%
- drq, Adagrad, cuda, default: +42.03590%
- drq, Adamax, cuda, maximize: +30.42514%
- drq, ASGD, cuda, maximize: +15.28668%
- drq, RAdam, cuda, (pt2) foreach: +13.66745%
- yolov3, Adam, cuda, (pt2) no_foreach: -10.99434%
- yolov3, Adam, cuda, differentiable: +11.55830%
- yolov3, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -12.15729%
- yolov3, AdamW, cuda, no_foreach: +13.21242%
- yolov3, AdamW, cuda, (pt2) foreach, maximize, capturable: -10.55678%
- yolov3, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -19.77660%
- yolov3, AdamW, cuda, foreach, maximize, capturable, amsgrad: -14.18354%
- yolov3, ASGD, cuda, (pt2) no_foreach: -16.77884%
- yolov3, RAdam, cuda, (pt2) no_foreach: +12.15803%
- yolov3, Rprop, cuda, no_foreach: +45.91666%
- yolov3, Rprop, cuda, (pt2) differentiable: +18.39044%
- detectron2_fasterrcnn_r_101_dc5, Adadelta, cuda, (pt2) maximize: -13.40607%
- lennard_jones, Adadelta, cuda, maximize: +11.43903%
- lennard_jones, Adam, cuda, foreach: +12.25737%
- lennard_jones, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +14.51531%
- lennard_jones, Adam, cuda, (pt2) fused, amsgrad, maximize: +12.30569%
- lennard_jones, ASGD, cuda, (pt2) foreach: +28.58997%
- lennard_jones, ASGD, cuda, foreach: +18.57151%
- lennard_jones, RAdam, cuda, default: +18.29510%
- lennard_jones, Rprop, cuda, foreach: +19.54759%
- lennard_jones, NAdam, cuda, default: +10.20431%
- lennard_jones, NAdam, cuda, foreach: +11.84110%
- mobilenet_v3_large, Adadelta, cuda, (pt2) foreach: -11.24278%
- mobilenet_v3_large, Adagrad, cuda, (pt2) foreach: +34.65377%
- mobilenet_v3_large, Adamax, cuda, default: +15.40265%
- mobilenet_v3_large, ASGD, cuda, (pt2) default: +22.16980%
- mobilenet_v3_large, ASGD, cuda, foreach: +42.46519%
- mobilenet_v3_large, RAdam, cuda, default: +13.23417%
- DALLE2_pytorch, Adagrad, cuda, (pt2) maximize: +39.63974%
- DALLE2_pytorch, Adagrad, cuda, maximize: +39.73919%
- DALLE2_pytorch, Adagrad, cuda, foreach: +13.72574%
- DALLE2_pytorch, AdamW, cuda, fused, amsgrad, maximize: +43.12966%
- DALLE2_pytorch, Rprop, cuda, (pt2) foreach: +11.20274%
- maml, ASGD, cuda, maximize: +22.88520%
- maml, SGD, cuda, (pt2) foreach, momentum=0.9: +10.69514%
- maml, RAdam, cuda, (pt2) default: +28.60684%
- hf_Longformer, ASGD, cuda, (pt2) maximize: -29.55975%
- hf_Longformer, Rprop, cuda, default: -10.07437%
- Background_Matting, Adadelta, cuda, (pt2) maximize: -10.73454%
- Background_Matting, NAdam, cuda, (pt2) foreach: +11.01888%
- hf_T5_large, AdamW, cuda, differentiable: +11.25036%
- hf_T5_large, Adamax, cuda, (pt2) no_foreach: +12.23026%
- hf_T5_large, RAdam, cuda, foreach: +11.37265%
- hf_T5_large, Rprop, cuda, no_foreach: -13.83590%
- hf_T5_large, NAdam, cuda, foreach: +13.67086%
- hf_GPT2_large, Adam, cuda, no_foreach: +11.74810%
- pytorch_stargan, Adadelta, cuda, (pt2) maximize: -10.71856%
- pytorch_stargan, Adam, cuda, amsgrad, maximize: +16.36709%
- pytorch_stargan, AdamW, cuda, (pt2) foreach, maximize, capturable: +14.84146%
- pytorch_stargan, AdamW, cuda, foreach, maximize, capturable: +12.49350%
- pytorch_stargan, Adamax, cuda, (pt2) default: +35.02400%
- pytorch_stargan, ASGD, cuda, maximize: +29.79039%
- opacus_cifar10, Adagrad, cuda, (pt2) maximize: -15.18746%
- opacus_cifar10, Adam, cuda, foreach: +15.74611%
- opacus_cifar10, Adam, cuda, foreach, maximize, capturable, amsgrad: -11.58500%
- opacus_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +44.97638%
- opacus_cifar10, Rprop, cuda, (pt2) maximize: +26.05681%
- hf_T5, Adadelta, cuda, (pt2) maximize: -11.10476%
- hf_T5, Adagrad, cuda, foreach: +45.37757%
- hf_T5, Rprop, cuda, (pt2) foreach: -13.74365%
- detectron2_fasterrcnn_r_50_dc5, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -18.31732%
- detectron2_fasterrcnn_r_50_dc5, SGD, cuda, (pt2) foreach, momentum=0.9: +15.69011%
- timm_vovnet, Adadelta, cuda, (pt2) maximize: -10.43216%
- timm_vovnet, Adagrad, cuda, maximize: +36.23421%
- timm_vovnet, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -10.98324%
- timm_vovnet, Adamax, cuda, (pt2) maximize: +25.53056%
- vision_maskrcnn, Adagrad, cuda, default: +19.95618%
- vision_maskrcnn, Adagrad, cuda, (pt2) maximize: -20.97249%
- vision_maskrcnn, Adam, cuda, (pt2) foreach, maximize, capturable: +14.39880%
- vision_maskrcnn, Adamax, cuda, (pt2) foreach: +27.31809%
- vision_maskrcnn, ASGD, cuda, (pt2) foreach: +18.43528%
- BERT_pytorch, Adadelta, cuda, differentiable: +56.57138%
- BERT_pytorch, Adamax, cuda, no_foreach: +23.89152%
- BERT_pytorch, ASGD, cuda, (pt2) maximize: -13.45787%
- BERT_pytorch, Rprop, cuda, (pt2) default: +18.20076%
- timm_regnet, Adadelta, cuda, (pt2) default: -12.03245%
- timm_regnet, Adadelta, cuda, (pt2) maximize: -16.19504%
- timm_regnet, Adadelta, cuda, (pt2) foreach: -13.82701%
- timm_regnet, Adagrad, cuda, foreach: +30.90645%
- timm_regnet, ASGD, cuda, (pt2) foreach: -10.42559%
- timm_regnet, ASGD, cuda, foreach: +27.51899%
- speech_transformer, Adagrad, cuda, default: +21.55573%
- speech_transformer, Adam, cuda, (pt2) foreach, maximize, capturable: +26.42426%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5030299568

cc @janeyx99