Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '61239df555df02e8c60a2ad63363878a2a57c161'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 61239df555df02e8c60a2ad63363878a2a57c161

Affected PyTorch commit: 61239df555df02e8c60a2ad63363878a2a57c161

Affected Tests:
- hf_T5_large, Adadelta, cuda, differentiable: +13.60454%
- hf_T5_large, AdamW, cuda, no_foreach: +22.47358%
- hf_T5_large, RAdam, cuda, (pt2) foreach: -11.14815%
- hf_T5_large, RMSprop, cuda, no_foreach: +38.66255%
- speech_transformer, Adagrad, cuda, default: -18.22282%
- speech_transformer, Adam, cuda, (pt2) foreach, maximize, capturable: -20.42337%
- speech_transformer, AdamW, cuda, foreach, maximize, capturable: +21.70409%
- speech_transformer, ASGD, cuda, (pt2) default: +26.28217%
- speech_transformer, ASGD, cuda, (pt2) foreach: +18.62524%
- speech_transformer, Rprop, cuda, (pt2) foreach: +33.17368%
- resnet152, Adagrad, cuda, (pt2) default: -16.15718%
- resnet152, ASGD, cuda, (pt2) maximize: +17.12808%
- resnet152, ASGD, cuda, (pt2) foreach: -12.05825%
- resnet152, RAdam, cuda, (pt2) default: -19.17358%
- resnet152, Rprop, cuda, (pt2) maximize: +17.61649%
- resnet152, NAdam, cuda, (pt2) default: -14.12475%
- resnet152, NAdam, cuda, default: -14.79072%
- pytorch_struct, Adagrad, cuda, default: -10.95862%
- pytorch_struct, Adagrad, cuda, maximize: -17.71966%
- pytorch_struct, AdamW, cuda, (pt2) amsgrad, maximize: +13.53300%
- pytorch_struct, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +23.93497%
- pytorch_struct, AdamW, cuda, foreach, maximize, capturable, amsgrad: +14.15635%
- pytorch_struct, Rprop, cuda, (pt2) default: -30.78712%
- pytorch_struct, Rprop, cuda, default: -22.48682%
- LearningToPaint, Adagrad, cuda, maximize: +16.84442%
- LearningToPaint, Adam, cuda, foreach, maximize, capturable, amsgrad: -11.67310%
- LearningToPaint, AdamW, cuda, amsgrad, maximize: +10.30468%
- LearningToPaint, AdamW, cuda, foreach: +14.97183%
- LearningToPaint, NAdam, cuda, (pt2) foreach: +10.06510%
- timm_resnest, Adagrad, cuda, (pt2) maximize: -15.13953%
- timm_resnest, Adagrad, cuda, foreach: +11.73446%
- timm_resnest, Adam, cuda, foreach, maximize, capturable: -13.74881%
- timm_resnest, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -13.23167%
- timm_resnest, Rprop, cuda, maximize: +21.74018%
- timm_nfnet, Adagrad, cuda, foreach: -10.77845%
- timm_nfnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +13.12893%
- timm_nfnet, ASGD, cuda, (pt2) default: -19.26312%
- timm_nfnet, ASGD, cuda, default: -17.34360%
- timm_nfnet, ASGD, cuda, maximize: -14.62167%
- densenet121, Adagrad, cuda, foreach: +22.43232%
- densenet121, Adamax, cuda, maximize: -22.54708%
- densenet121, Rprop, cuda, maximize: +29.38693%
- densenet121, Rprop, cuda, foreach: -34.95967%
- torchrec_dlrm, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -14.45357%
- torchrec_dlrm, SGD, cuda, (pt2) foreach, momentum=0.9: -12.03607%
- hf_T5, Adagrad, cuda, (pt2) maximize: +12.49735%
- hf_T5, Adagrad, cuda, foreach: -31.39563%
- hf_T5, Adam, cuda, foreach, maximize, capturable: +17.29529%
- hf_T5, Adamax, cuda, (pt2) default: +31.25084%
- hf_T5, Rprop, cuda, maximize: +15.05087%
- detectron2_fasterrcnn_r_101_fpn, Adamax, cuda, (pt2) maximize: -13.24207%
- fastNLP_Bert, AdamW, cuda, foreach, maximize, capturable: -11.34146%
- Background_Matting, Adagrad, cuda, (pt2) maximize: +15.44612%
- Background_Matting, AdamW, cuda, (pt2) default: +14.85950%
- doctr_det_predictor, Adagrad, cuda, maximize: -28.51256%
- doctr_det_predictor, AdamW, cuda, foreach: -11.91730%
- doctr_det_predictor, AdamW, cuda, (pt2) foreach, maximize, capturable: +11.17356%
- doctr_det_predictor, AdamW, cuda, foreach, maximize, capturable: +22.57804%
- doctr_det_predictor, ASGD, cuda, (pt2) default: +13.07918%
- doctr_det_predictor, Rprop, cuda, maximize: -16.71013%
- mnasnet1_0, Adagrad, cuda, default: +11.33607%
- mnasnet1_0, Adam, cuda, default: +12.69658%
- mnasnet1_0, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -19.85623%
- mnasnet1_0, Adamax, cuda, (pt2) default: -25.29040%
- mnasnet1_0, ASGD, cuda, (pt2) maximize: +15.28812%
- mnasnet1_0, Rprop, cuda, (pt2) default: -12.66443%
- detectron2_fasterrcnn_r_50_c4, Adamax, cuda, (pt2) maximize: -12.23157%
- detectron2_fasterrcnn_r_50_c4, ASGD, cuda, (pt2) foreach: -15.23407%
- detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -27.15889%
- detectron2_fasterrcnn_r_50_c4, Rprop, cuda, (pt2) maximize: -30.22095%
- detectron2_fasterrcnn_r_50_c4, Rprop, cuda, (pt2) foreach: +35.17799%
- timm_vovnet, Adagrad, cuda, maximize: -30.08798%
- timm_vovnet, Adamax, cuda, default: +30.26049%
- timm_vovnet, Adamax, cuda, (pt2) maximize: -15.23673%
- timm_vovnet, Adamax, cuda, maximize: +28.61749%
- timm_vovnet, RAdam, cuda, default: +18.71114%
- timm_vovnet, Rprop, cuda, (pt2) default: +22.53426%
- timm_vovnet, Rprop, cuda, (pt2) foreach: +44.75472%
- timm_vovnet, Rprop, cuda, foreach: +21.76370%
- phlippe_resnet, Adam, cuda, (pt2) default: +14.10820%
- phlippe_resnet, Adam, cuda, foreach, maximize, capturable, amsgrad: -12.74643%
- phlippe_resnet, Adamax, cuda, (pt2) default: -26.51051%
- phlippe_resnet, Adamax, cuda, (pt2) maximize: +32.45039%
- phlippe_resnet, ASGD, cuda, maximize: -15.60158%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9: -10.03034%
- phlippe_densenet, Adam, cuda, foreach: +10.65175%
- phlippe_densenet, Adam, cuda, foreach, maximize, capturable: -10.98485%
- phlippe_densenet, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +15.39777%
- phlippe_densenet, Rprop, cuda, (pt2) maximize: +11.40993%
- phlippe_densenet, Rprop, cuda, maximize: +13.60870%
- squeezenet1_1, Adagrad, cuda, default: +14.66994%
- squeezenet1_1, Adagrad, cuda, (pt2) foreach: -11.50462%
- squeezenet1_1, Adam, cuda, (pt2) foreach, maximize, capturable: +16.25525%
- squeezenet1_1, Adamax, cuda, (pt2) maximize: -25.69500%
- squeezenet1_1, Adamax, cuda, foreach: +23.13726%
- squeezenet1_1, RAdam, cuda, (pt2) default: +11.68151%
- squeezenet1_1, Rprop, cuda, (pt2) maximize: +37.85334%
- squeezenet1_1, Rprop, cuda, maximize: +10.00621%
- squeezenet1_1, Rprop, cuda, foreach: -22.33141%
- squeezenet1_1, NAdam, cuda, (pt2) foreach: +10.97497%
- fambench_xlmr, ASGD, cuda, (pt2) default: -11.41906%
- fambench_xlmr, Rprop, cuda, default: +10.29543%
- mobilenet_v2_quantized_qat, Adagrad, cuda, default: -13.70634%
- mobilenet_v2_quantized_qat, Adagrad, cuda, (pt2) foreach: +11.59934%
- mobilenet_v2_quantized_qat, AdamW, cuda, default: +11.92230%
- mobilenet_v2_quantized_qat, Rprop, cuda, maximize: +10.19087%
- detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, (pt2) default: -26.19251%
- detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, default: -17.72972%
- detectron2_fasterrcnn_r_50_fpn, ASGD, cuda, default: -11.05838%
- detectron2_fasterrcnn_r_50_fpn, Rprop, cuda, maximize: +18.29244%
- timm_regnet, Adagrad, cuda, foreach: -31.48542%
- timm_regnet, Adam, cuda, foreach, maximize, capturable: -11.32439%
- timm_regnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +14.77964%
- timm_regnet, ASGD, cuda, maximize: +10.27234%
- timm_regnet, ASGD, cuda, foreach: -25.83032%
- timm_regnet, Rprop, cuda, default: +11.85468%
- resnet50, Adadelta, cuda, (pt2) differentiable: -12.23494%
- resnet50, Adagrad, cuda, (pt2) no_foreach: +10.04018%
- resnet50, Adagrad, cuda, differentiable: +47.46322%
- resnet50, Adagrad, cuda, (pt2) foreach: +44.43562%
- resnet50, Adam, cuda, (pt2) differentiable: +11.25322%
- resnet50, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +10.26715%
- resnet50, AdamW, cuda, default: -11.16829%
- resnet50, Adamax, cuda, (pt2) default: +11.47901%
- resnet50, Adamax, cuda, (pt2) maximize: -16.38561%
- resnet50, Adamax, cuda, (pt2) foreach: -14.60888%
- resnet50, ASGD, cuda, (pt2) maximize: +28.51260%
- resnet50, RAdam, cuda, (pt2) differentiable: -15.08969%
- resnet50, RAdam, cuda, (pt2) foreach: -16.58168%
- resnet50, Rprop, cuda, foreach: +27.03858%
- resnet50, NAdam, cuda, (pt2) default: +10.26112%
- hf_Bert, Adagrad, cuda, (pt2) foreach: +19.31554%
- hf_Bert, Adamax, cuda, default: +23.80594%
- lennard_jones, Adadelta, cuda, maximize: -10.65196%
- lennard_jones, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -12.54647%
- lennard_jones, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +10.95844%
- lennard_jones, ASGD, cuda, (pt2) default: +29.08472%
- lennard_jones, ASGD, cuda, maximize: +35.90132%
- lennard_jones, ASGD, cuda, (pt2) foreach: -19.84921%
- lennard_jones, ASGD, cuda, foreach: -14.93836%
- lennard_jones, SGD, cuda, foreach, momentum=0.9: +14.93877%
- lennard_jones, RAdam, cuda, default: -11.16003%
- lennard_jones, Rprop, cuda, (pt2) maximize: +10.55758%
- lennard_jones, Rprop, cuda, foreach: -17.00680%
- lennard_jones, RMSprop, cuda, foreach: +10.84504%
- timm_efficientnet, Adagrad, cuda, (pt2) default: +17.82986%
- timm_efficientnet, Adagrad, cuda, (pt2) maximize: +14.37078%
- timm_efficientdet, Adagrad, cuda, maximize: +11.55644%
- timm_efficientdet, Adam, cuda, amsgrad, maximize: -12.94190%
- timm_efficientdet, Adamax, cuda, (pt2) default: -20.94779%
- timm_efficientdet, Adamax, cuda, maximize: +16.95827%
- timm_efficientdet, ASGD, cuda, (pt2) default: +29.36164%
- timm_efficientdet, Rprop, cuda, (pt2) maximize: +10.00228%
- timm_efficientdet, Rprop, cuda, (pt2) foreach: +13.73209%
- timm_efficientdet, NAdam, cuda, default: +16.40506%
- drq, Adagrad, cuda, default: -29.40278%
- drq, AdamW, cuda, (pt2) foreach: +10.80673%
- drq, Adamax, cuda, maximize: -23.21681%
- drq, ASGD, cuda, maximize: -14.23941%
- drq, RAdam, cuda, (pt2) default: +14.06625%
- drq, RAdam, cuda, (pt2) foreach: -15.32356%
- hf_T5_generate, Adagrad, cuda, (pt2) maximize: +13.75225%
- hf_T5_generate, Adagrad, cuda, (pt2) foreach: -22.28028%
- hf_T5_generate, Adamax, cuda, default: +14.27014%
- hf_T5_generate, RAdam, cuda, (pt2) default: +11.47564%
- hf_T5_generate, Rprop, cuda, maximize: -11.78328%
- hf_T5_generate, Rprop, cuda, (pt2) foreach: -10.57932%
- hf_T5_generate, Rprop, cuda, foreach: -32.73970%
- detectron2_maskrcnn_r_50_c4, Adagrad, cuda, maximize: -18.85365%
- moco, ASGD, cuda, (pt2) foreach: +29.08947%
- moco, Rprop, cuda, (pt2) foreach: -18.29600%
- basic_gnn_gin, Adagrad, cuda, default: -16.43837%
- basic_gnn_gin, Adam, cuda, (pt2) default: -10.71454%
- basic_gnn_gin, Rprop, cuda, maximize: -10.41602%
- resnet50_quantized_qat, Adagrad, cuda, (pt2) maximize: -10.06705%
- resnet50_quantized_qat, Adamax, cuda, foreach: -12.15751%
- resnet50_quantized_qat, ASGD, cuda, default: +18.22748%
- resnet50_quantized_qat, ASGD, cuda, maximize: +20.63210%
- vision_maskrcnn, Adagrad, cuda, default: -13.73216%
- vision_maskrcnn, Adam, cuda, (pt2) foreach, maximize, capturable: -12.29962%
- vision_maskrcnn, Adamax, cuda, (pt2) foreach: -25.53220%
- vision_maskrcnn, Adamax, cuda, foreach: +20.03636%
- vision_maskrcnn, ASGD, cuda, (pt2) foreach: -17.80287%
- vision_maskrcnn, ASGD, cuda, foreach: +26.06824%
- vision_maskrcnn, Rprop, cuda, maximize: +48.69072%
- vision_maskrcnn, Rprop, cuda, foreach: +48.50271%
- hf_GPT2, Adagrad, cuda, default: -10.43195%
- hf_GPT2, Rprop, cuda, (pt2) default: +40.37547%
- hf_GPT2, Rprop, cuda, (pt2) maximize: -21.21464%
- DALLE2_pytorch, Adagrad, cuda, (pt2) maximize: -28.31618%
- DALLE2_pytorch, Adagrad, cuda, maximize: -29.68450%
- DALLE2_pytorch, Adagrad, cuda, foreach: -13.56571%
- DALLE2_pytorch, AdamW, cuda, fused, amsgrad, maximize: -29.81193%
- DALLE2_pytorch, Rprop, cuda, default: +14.65963%
- DALLE2_pytorch, Rprop, cuda, (pt2) foreach: -10.47113%
- attention_is_all_you_need_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable: -13.58209%
- attention_is_all_you_need_pytorch, ASGD, cuda, (pt2) foreach: -13.25113%
- attention_is_all_you_need_pytorch, ASGD, cuda, foreach: +25.25339%
- functorch_maml_omniglot, Adagrad, cuda, maximize: -27.84574%
- functorch_maml_omniglot, AdamW, cuda, (pt2) default: -10.29754%
- functorch_maml_omniglot, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -12.11789%
- functorch_maml_omniglot, ASGD, cuda, default: -13.95462%
- functorch_maml_omniglot, RAdam, cuda, foreach: -10.81749%
- detectron2_maskrcnn, Adagrad, cuda, (pt2) default: +10.03808%
- detectron2_maskrcnn, Adamax, cuda, (pt2) default: -11.67045%
- detectron2_maskrcnn, ASGD, cuda, maximize: +25.72164%
- detectron2_maskrcnn, Rprop, cuda, maximize: +30.01440%
- basic_gnn_gcn, Adam, cuda, foreach: +10.09730%
- basic_gnn_gcn, Adam, cuda, (pt2) foreach, maximize, capturable: +20.37854%
- basic_gnn_gcn, Adam, cuda, foreach, maximize, capturable: +11.82814%
- basic_gnn_gcn, Adamax, cuda, default: -24.98751%
- basic_gnn_gcn, NAdam, cuda, foreach: +11.66020%
- opacus_cifar10, Adam, cuda, foreach: -12.82273%
- opacus_cifar10, Adamax, cuda, default: +21.08431%
- opacus_cifar10, Adamax, cuda, (pt2) maximize: +32.02716%
- opacus_cifar10, Adamax, cuda, foreach: +27.23378%
- opacus_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -28.65616%
- opacus_cifar10, Rprop, cuda, (pt2) maximize: -23.25056%
- hf_DistilBert, Adagrad, cuda, foreach: +10.68220%
- hf_DistilBert, Adamax, cuda, (pt2) foreach: +12.05817%
- hf_DistilBert, ASGD, cuda, (pt2) maximize: -13.33010%
- hf_DistilBert, ASGD, cuda, maximize: -26.60854%
- hf_DistilBert, Rprop, cuda, (pt2) foreach: -18.89753%
- hf_DistilBert, Rprop, cuda, foreach: -10.80559%
- functorch_dp_cifar10, Adagrad, cuda, (pt2) default: -28.36952%
- functorch_dp_cifar10, Adagrad, cuda, default: -22.25236%
- functorch_dp_cifar10, Adagrad, cuda, foreach: -18.63088%
- functorch_dp_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +24.07712%
- maml_omniglot, Adadelta, cuda, maximize: +15.74202%
- maml_omniglot, Adam, cuda, (pt2) amsgrad, maximize: -13.92704%
- maml_omniglot, Adam, cuda, fused: -10.21255%
- maml_omniglot, ASGD, cuda, (pt2) default: -26.65473%
- maml_omniglot, ASGD, cuda, default: -20.93049%
- maml_omniglot, NAdam, cuda, foreach: -15.05091%
- mobilenet_v3_large, Adagrad, cuda, (pt2) foreach: -26.48557%
- mobilenet_v3_large, AdamW, cuda, foreach, maximize, capturable: +10.80723%
- mobilenet_v3_large, Adamax, cuda, default: -12.13923%
- mobilenet_v3_large, Adamax, cuda, (pt2) foreach: +29.29962%
- mobilenet_v3_large, ASGD, cuda, (pt2) default: -18.29511%
- mobilenet_v3_large, ASGD, cuda, default: +14.85064%
- mobilenet_v3_large, ASGD, cuda, foreach: -29.75856%
- mobilenet_v3_large, RAdam, cuda, default: -10.62867%
- nanogpt_generate, Rprop, cuda, (pt2) default: +41.21039%
- nanogpt_generate, Rprop, cuda, (pt2) maximize: +41.27364%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, foreach: -26.89252%
- detectron2_maskrcnn_r_101_c4, ASGD, cuda, (pt2) default: -12.44815%
- detectron2_maskrcnn_r_101_c4, Rprop, cuda, default: +11.11972%
- detectron2_fasterrcnn_r_50_dc5, SGD, cuda, (pt2) foreach, momentum=0.9: +20.00735%
- hf_Longformer, Adamax, cuda, default: +15.61983%
- hf_Longformer, Rprop, cuda, (pt2) default: +41.45902%
- basic_gnn_edgecnn, Rprop, cuda, (pt2) default: -13.05964%
- hf_Albert, ASGD, cuda, default: -14.27121%
- hf_Albert, ASGD, cuda, (pt2) foreach: -19.49115%
- yolov3, Adagrad, cuda, (pt2) no_foreach: +16.03149%
- yolov3, Adagrad, cuda, foreach: +18.63486%
- yolov3, Adam, cuda, no_foreach: +14.38061%
- yolov3, Adam, cuda, differentiable: -16.54344%
- yolov3, AdamW, cuda, (pt2) differentiable: +10.71704%
- yolov3, AdamW, cuda, differentiable: +37.27744%
- yolov3, AdamW, cuda, foreach, maximize, capturable, amsgrad: +21.15375%
- yolov3, Adamax, cuda, differentiable: +12.35434%
- yolov3, ASGD, cuda, (pt2) differentiable: +10.44449%
- yolov3, ASGD, cuda, (pt2) foreach: +49.38943%
- yolov3, ASGD, cuda, foreach: +32.95941%
- yolov3, RAdam, cuda, (pt2) default: -12.58476%
- yolov3, RAdam, cuda, (pt2) no_foreach: -11.84660%
- yolov3, RAdam, cuda, no_foreach: +32.02452%
- yolov3, RAdam, cuda, foreach: +22.91353%
- yolov3, Rprop, cuda, no_foreach: -30.77830%
- yolov3, Rprop, cuda, (pt2) differentiable: -10.01303%
- yolov3, Rprop, cuda, (pt2) foreach: +14.83490%
- yolov3, Rprop, cuda, foreach: +15.79699%
- soft_actor_critic, AdamW, cuda, default: -17.33461%
- soft_actor_critic, AdamW, cuda, (pt2) foreach, maximize, capturable: +12.61778%
- soft_actor_critic, Adamax, cuda, (pt2) default: +22.45962%
- soft_actor_critic, Adamax, cuda, default: +14.58660%
- soft_actor_critic, RMSprop, cuda, foreach: +11.66893%
- soft_actor_critic, NAdam, cuda, (pt2) default: +13.45134%
- detectron2_maskrcnn_r_50_fpn, AdamW, cuda, (pt2) foreach, maximize, capturable: +10.67271%
- detectron2_maskrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: +14.53275%
- detectron2_maskrcnn_r_50_fpn, ASGD, cuda, (pt2) default: -27.12756%
- detectron2_maskrcnn_r_50_fpn, Rprop, cuda, (pt2) default: -31.44503%
- resnet18, Adam, cuda, (pt2) foreach: +13.00038%
- resnet18, Adam, cuda, foreach: -12.64608%
- resnet18, ASGD, cuda, (pt2) maximize: -12.30010%
- resnet18, RAdam, cuda, (pt2) foreach: +19.26040%
- resnext50_32x4d, Adagrad, cuda, maximize: -10.48879%
- resnext50_32x4d, RAdam, cuda, foreach: +13.25582%
- mobilenet_v2, Adagrad, cuda, maximize: -26.17869%
- mobilenet_v2, Adagrad, cuda, foreach: -10.65649%
- mobilenet_v2, Adam, cuda, (pt2) default: +10.97607%
- mobilenet_v2, ASGD, cuda, (pt2) foreach: +51.72550%
- mobilenet_v2, Rprop, cuda, (pt2) maximize: +10.77751%
- mobilenet_v2, Rprop, cuda, foreach: -15.58003%
- hf_Reformer, Adagrad, cuda, maximize: +11.16544%
- hf_Reformer, ASGD, cuda, (pt2) foreach: -32.21486%
- hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +19.80901%
- hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9: -49.98759%
- tts_angular, Adagrad, cuda, default: -11.57109%
- tts_angular, AdamW, cuda, foreach, maximize, capturable: -21.02603%
- tts_angular, Rprop, cuda, default: +18.57440%
- tts_angular, Rprop, cuda, maximize: -10.53156%
- tts_angular, Rprop, cuda, foreach: +43.81550%
- hf_Bert_large, Adagrad, cuda, foreach: -12.47307%
- hf_Bert_large, ASGD, cuda, default: +28.44674%
- hf_Bert_large, Rprop, cuda, maximize: -11.82824%
- hf_T5_base, Adagrad, cuda, maximize: -14.03934%
- hf_T5_base, Adamax, cuda, (pt2) default: +21.21990%
- hf_T5_base, Adamax, cuda, default: -20.78679%
- hf_T5_base, ASGD, cuda, default: -23.59413%
- timm_vision_transformer, Adadelta, cuda, (pt2) no_foreach: +11.41676%
- timm_vision_transformer, Adadelta, cuda, no_foreach: +10.20763%
- timm_vision_transformer, Adadelta, cuda, differentiable: -22.33240%
- timm_vision_transformer, Adagrad, cuda, maximize: -17.42072%
- timm_vision_transformer, AdamW, cuda, (pt2) foreach, maximize, capturable: +18.71245%
- timm_vision_transformer, AdamW, cuda, foreach, maximize, capturable: -12.43316%
- timm_vision_transformer, Adamax, cuda, (pt2) maximize: -10.24628%
- timm_vision_transformer, Adamax, cuda, (pt2) no_foreach: +10.77915%
- timm_vision_transformer, ASGD, cuda, default: -13.83146%
- timm_vision_transformer, ASGD, cuda, (pt2) foreach: +13.61679%
- timm_vision_transformer, SGD, cuda, no_foreach: +48.24445%
- timm_vision_transformer, RAdam, cuda, no_foreach: +21.56532%
- timm_vision_transformer, Rprop, cuda, (pt2) foreach: +46.90374%
- hf_BigBird, ASGD, cuda, maximize: +16.82669%
- hf_BigBird, Rprop, cuda, (pt2) default: +34.88466%
- maml, Adam, cuda, (pt2) default: +14.16109%
- maml, Adamax, cuda, (pt2) foreach: +33.64934%
- maml, ASGD, cuda, maximize: -16.86106%
- maml, ASGD, cuda, (pt2) foreach: +41.76913%
- maml, RAdam, cuda, (pt2) default: -11.01540%
- maml, Rprop, cuda, (pt2) maximize: +46.33812%
- pytorch_stargan, Adam, cuda, amsgrad, maximize: -13.45159%
- pytorch_stargan, AdamW, cuda, (pt2) foreach, maximize, capturable: -18.92852%
- pytorch_stargan, AdamW, cuda, foreach, maximize, capturable: -11.42102%
- pytorch_stargan, Adamax, cuda, (pt2) default: -25.06524%
- pytorch_stargan, ASGD, cuda, maximize: -22.20560%
- pytorch_stargan, Rprop, cuda, default: +16.42667%
- doctr_reco_predictor, Adagrad, cuda, default: +11.81406%
- doctr_reco_predictor, Adagrad, cuda, foreach: -13.42570%
- doctr_reco_predictor, Adamax, cuda, (pt2) maximize: -21.20705%
- doctr_reco_predictor, ASGD, cuda, (pt2) default: +12.56589%
- doctr_reco_predictor, ASGD, cuda, maximize: +13.22545%
- doctr_reco_predictor, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +11.98373%
- doctr_reco_predictor, Rprop, cuda, (pt2) default: +30.54472%
- hf_GPT2_large, Rprop, cuda, (pt2) foreach: +12.82935%
- hf_GPT2_large, NAdam, cuda, default: -12.72739%
- pytorch_CycleGAN_and_pix2pix, Adam, cuda, (pt2) foreach, maximize, capturable: -13.52605%
- pytorch_CycleGAN_and_pix2pix, RAdam, cuda, (pt2) foreach: +11.45351%
- pytorch_CycleGAN_and_pix2pix, RAdam, cuda, foreach: +27.26347%
- pytorch_CycleGAN_and_pix2pix, Rprop, cuda, (pt2) maximize: +25.12776%
- shufflenet_v2_x1_0, Adagrad, cuda, (pt2) foreach: +10.18825%
- shufflenet_v2_x1_0, Rprop, cuda, (pt2) foreach: +13.46989%
- BERT_pytorch, Adadelta, cuda, differentiable: -37.41114%
- BERT_pytorch, Adagrad, cuda, foreach: +16.35415%
- BERT_pytorch, AdamW, cuda, (pt2) differentiable: +11.26834%
- BERT_pytorch, Adamax, cuda, no_foreach: -17.72422%
- BERT_pytorch, Adamax, cuda, differentiable: +18.66413%
- BERT_pytorch, RMSprop, cuda, no_foreach: +50.91613%
- dcgan, Adagrad, cuda, (pt2) foreach: -22.44141%
- dcgan, Adam, cuda, default: -11.44944%
- dcgan, Adam, cuda, foreach, maximize, capturable: -11.43638%
- dcgan, AdamW, cuda, (pt2) foreach: -12.85182%
- dcgan, Adamax, cuda, foreach: -22.16777%
- dcgan, ASGD, cuda, (pt2) default: +30.91377%
- dcgan, ASGD, cuda, foreach: -20.35816%
- dcgan, Rprop, cuda, maximize: -27.10498%
- detectron2_maskrcnn_r_101_fpn, Adagrad, cuda, foreach: -17.31563%
- detectron2_maskrcnn_r_101_fpn, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -11.97197%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) default: -24.95229%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, maximize: +32.51212%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) foreach: -14.26331%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, foreach: +31.28277%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, foreach: +14.06535%
- Super_SloMo, Adamax, cuda, (pt2) foreach: +17.66255%
- Super_SloMo, ASGD, cuda, (pt2) default: +11.44557%
- Super_SloMo, ASGD, cuda, default: +20.54556%
- tacotron2, Adagrad, cuda, (pt2) foreach: -24.03062%
- tacotron2, ASGD, cuda, (pt2) foreach: -11.73387%
- tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9: -13.48581%
- tacotron2, Rprop, cuda, default: -23.21057%
- llama, Adam, cuda, amsgrad, maximize: -10.00944%
- llama, Adamax, cuda, default: +19.25434%
- llama, Rprop, cuda, maximize: +41.28893%
- llama, NAdam, cuda, (pt2) default: +13.53627%
- basic_gnn_sage, Adagrad, cuda, (pt2) default: +27.84624%
- basic_gnn_sage, Adagrad, cuda, maximize: -13.68657%
- basic_gnn_sage, Adam, cuda, foreach, maximize, capturable, amsgrad: -13.48494%
- basic_gnn_sage, AdamW, cuda, (pt2) amsgrad, maximize: -10.66960%
- basic_gnn_sage, SGD, cuda, foreach, momentum=0.9: +13.97110%
- basic_gnn_sage, Rprop, cuda, (pt2) default: -11.87209%
- basic_gnn_sage, Rprop, cuda, (pt2) foreach: -18.34679%
- pytorch_unet, Adagrad, cuda, (pt2) foreach: +13.24542%
- pytorch_unet, Adamax, cuda, default: -10.03720%
- pytorch_unet, ASGD, cuda, foreach: +28.25222%
- pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -16.88537%
- pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9: -18.45504%
- pytorch_unet, Rprop, cuda, (pt2) default: +47.62757%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) default: +34.30178%
- detectron2_fasterrcnn_r_101_c4, Adam, cuda, foreach, maximize, capturable: +16.00049%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) default: -19.76335%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) maximize: +20.19919%
- detectron2_fasterrcnn_r_101_c4, ASGD, cuda, default: +11.44166%
- hf_Bart, ASGD, cuda, maximize: -11.23174%
- hf_Bart, Rprop, cuda, (pt2) maximize: +12.43215%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/5035713765

cc @janeyx99


In the process of figuring out a better threshold