Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '1b6d6e98fed5cdb0a0c2fedd5354dd8b83dd07f1'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 1b6d6e98fed5cdb0a0c2fedd5354dd8b83dd07f1

Affected PyTorch commit: 1b6d6e98fed5cdb0a0c2fedd5354dd8b83dd07f1

Affected Tests:
- fambench_xlmr, AdamW, cuda, foreach: +0.09202%
- fambench_xlmr, ASGD, cuda, (pt2) default: +0.12329%
- hf_Albert, Adamax, cuda, (pt2) maximize: +0.07297%
- hf_Albert, RMSprop, cuda, (pt2) maximize: +0.09371%
- timm_vision_transformer, Adamax, cuda, (pt2) maximize: +0.16770%
- timm_vision_transformer, Adamax, cuda, maximize: +0.07752%
- timm_vision_transformer, Adamax, cuda, (pt2) foreach: +0.14745%
- timm_vision_transformer, Rprop, cuda, (pt2) no_foreach: +0.10521%
- timm_vision_transformer, Rprop, cuda, (pt2) foreach: +0.58680%
- timm_vision_transformer, Rprop, cuda, foreach: +0.09055%
- attention_is_all_you_need_pytorch, Adagrad, cuda, maximize: +0.16281%
- attention_is_all_you_need_pytorch, AdamW, cuda, foreach, maximize, capturable: +0.20054%
- attention_is_all_you_need_pytorch, Adamax, cuda, (pt2) default: +0.15414%
- attention_is_all_you_need_pytorch, RAdam, cuda, (pt2) foreach: +0.08750%
- attention_is_all_you_need_pytorch, Rprop, cuda, default: +0.19557%
- mobilenet_v2_quantized_qat, AdamW, cuda, (pt2) foreach: +0.11199%
- mobilenet_v2_quantized_qat, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.12649%
- mobilenet_v2_quantized_qat, Rprop, cuda, foreach: +0.07330%
- timm_resnest, Adadelta, cpu, (pt2) foreach: +0.13354%
- timm_resnest, Adadelta, cpu, foreach: +0.09823%
- timm_resnest, Adagrad, cpu, (pt2) foreach: +0.14583%
- timm_resnest, Adagrad, cpu, foreach: +0.31224%
- timm_resnest, Adamax, cpu, foreach: +0.41316%
- timm_resnest, SGD, cpu, foreach, momentum=0.9, nesterov: +0.07648%
- timm_resnest, RMSprop, cpu, (pt2) foreach: +0.09723%
- timm_resnest, NAdam, cpu, (pt2) foreach: +0.14571%
- detectron2_fasterrcnn_r_50_c4, Adagrad, cuda, maximize: +0.07045%
- detectron2_fasterrcnn_r_50_c4, Adam, cuda, (pt2) foreach, maximize, capturable: +0.08680%
- detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.16327%
- detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9: +0.14455%
- detectron2_fasterrcnn_r_50_c4, Rprop, cuda, (pt2) default: +0.13569%
- detectron2_fasterrcnn_r_50_c4, Rprop, cuda, maximize: +0.10976%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, default: +0.34681%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) maximize: +0.21333%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, maximize: +0.08350%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) foreach: +0.32346%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, (pt2) default: +0.11819%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, default: +0.19819%
- hf_BigBird, Adamax, cuda, (pt2) foreach: +0.33418%
- maml, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.09657%
- fastNLP_Bert, Adamax, cuda, maximize: +0.38077%
- vision_maskrcnn, Adagrad, cpu, (pt2) foreach: +0.07844%
- vision_maskrcnn, Adagrad, cpu, foreach: +0.22839%
- vision_maskrcnn, Adam, cpu, (pt2) foreach: +0.39782%
- vision_maskrcnn, Adam, cpu, foreach: +0.53464%
- vision_maskrcnn, AdamW, cpu, (pt2) foreach: +0.47539%
- vision_maskrcnn, AdamW, cpu, foreach: +0.50056%
- vision_maskrcnn, Adamax, cpu, (pt2) foreach: +0.16403%
- vision_maskrcnn, Adamax, cpu, foreach: +0.22268%
- vision_maskrcnn, RAdam, cpu, (pt2) foreach: +0.08855%
- phlippe_resnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.11242%
- phlippe_resnet, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.17775%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9: +0.10370%
- hf_GPT2_large, Adamax, cuda, (pt2) no_foreach: +0.09223%
- hf_GPT2_large, Adamax, cuda, (pt2) differentiable: +0.07229%
- hf_GPT2_large, RAdam, cuda, (pt2) default: +0.09546%
- hf_GPT2_large, NAdam, cuda, default: +0.13096%
- hf_GPT2_large, NAdam, cuda, foreach: +0.12477%
- pytorch_struct, Adagrad, cuda, (pt2) default: +0.07740%
- pytorch_struct, Adagrad, cuda, (pt2) foreach: +0.09474%
- pytorch_struct, Adamax, cuda, default: +0.09362%
- hf_Albert, Adadelta, cpu, (pt2) foreach: +0.44129%
- hf_Albert, Adadelta, cpu, foreach: +0.46254%
- hf_Albert, Adagrad, cpu, (pt2) foreach: +0.71751%
- hf_Albert, Adagrad, cpu, foreach: +0.62515%
- hf_Albert, Adam, cpu, (pt2) foreach: +0.37719%
- hf_Albert, Adam, cpu, foreach: +0.39084%
- hf_Albert, AdamW, cpu, (pt2) foreach: +0.37455%
- hf_Albert, AdamW, cpu, foreach: +0.33461%
- hf_Albert, Adamax, cpu, (pt2) foreach: +0.29564%
- hf_Albert, Adamax, cpu, foreach: +0.35900%
- resnet18, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.43065%
- moco, Adagrad, cuda, (pt2) default: +0.11411%
- moco, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.09263%
- moco, ASGD, cuda, foreach: +0.18699%
- moco, Rprop, cuda, foreach: +0.10129%
- Background_Matting, Adagrad, cuda, default: +0.11742%
- Background_Matting, Adamax, cuda, maximize: +0.10161%
- Background_Matting, RAdam, cuda, default: +0.20445%
- Background_Matting, RAdam, cuda, (pt2) foreach: +0.18092%
- phlippe_densenet, Adagrad, cuda, maximize: +0.29361%
- pytorch_stargan, Adagrad, cpu, (pt2) foreach: +0.24946%
- pytorch_stargan, SGD, cpu, foreach: +0.07068%
- pytorch_stargan, RAdam, cpu, (pt2) foreach: +0.10308%
- pytorch_stargan, Rprop, cpu, (pt2) foreach: +0.08324%
- pytorch_stargan, RMSprop, cpu, foreach: +0.07545%
- hf_GPT2, Adagrad, cuda, default: +0.12724%
- hf_GPT2, Adamax, cuda, maximize: +0.09274%
- shufflenet_v2_x1_0, Adadelta, cuda, maximize: +0.07656%
- shufflenet_v2_x1_0, Rprop, cuda, (pt2) default: +0.08418%
- shufflenet_v2_x1_0, Rprop, cuda, maximize: +0.07036%
- opacus_cifar10, Adadelta, cpu, foreach: +0.07267%
- opacus_cifar10, AdamW, cpu, (pt2) foreach: +0.17090%
- opacus_cifar10, AdamW, cpu, foreach: +0.14954%
- opacus_cifar10, Adamax, cpu, (pt2) foreach: +0.52533%
- opacus_cifar10, SGD, cpu, foreach, momentum=0.9, nesterov: +0.09703%
- opacus_cifar10, RAdam, cpu, foreach: +0.12908%
- opacus_cifar10, RMSprop, cpu, (pt2) foreach: +0.07582%
- opacus_cifar10, NAdam, cpu, (pt2) foreach: +0.13881%
- opacus_cifar10, NAdam, cpu, foreach: +0.13969%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, (pt2) foreach: +0.10687%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, foreach: +0.14369%
- detectron2_fasterrcnn_r_50_c4, RMSprop, cpu, (pt2) foreach: +0.10779%
- detectron2_fasterrcnn_r_50_c4, NAdam, cpu, foreach: +0.08724%
- pytorch_struct, ASGD, cpu, (pt2) foreach: +0.07233%
- pytorch_struct, ASGD, cpu, foreach: +0.10231%
- pytorch_struct, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.20596%
- pytorch_struct, SGD, cpu, foreach, momentum=0.9, nesterov: +0.13630%
- pytorch_struct, SGD, cpu, (pt2) foreach, momentum=0.9: +0.07809%
- pytorch_struct, SGD, cpu, foreach, momentum=0.9: +0.10901%
- pytorch_struct, RAdam, cpu, (pt2) foreach: +0.07081%
- pytorch_struct, RMSprop, cpu, foreach: +0.08483%
- drq, Adagrad, cuda, maximize: +0.32943%
- drq, Adam, cuda, amsgrad, maximize: +0.09060%
- drq, AdamW, cuda, (pt2) amsgrad, maximize: +0.11001%
- drq, AdamW, cuda, foreach, maximize, capturable: +0.07656%
- drq, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.61861%
- drq, RAdam, cuda, default: +0.11325%
- BERT_pytorch, Adagrad, cuda, differentiable: +0.34038%
- BERT_pytorch, Adamax, cuda, maximize: +0.16104%
- BERT_pytorch, ASGD, cuda, (pt2) maximize: +0.08482%
- BERT_pytorch, ASGD, cuda, maximize: +0.08031%
- BERT_pytorch, SGD, cuda, no_foreach: +0.35186%
- BERT_pytorch, Rprop, cuda, no_foreach: +0.48031%
- speech_transformer, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.08117%
- speech_transformer, Adamax, cuda, default: +0.07306%
- squeezenet1_1, Adadelta, cpu, (pt2) foreach: +0.14694%
- squeezenet1_1, Adadelta, cpu, foreach: +0.13176%
- squeezenet1_1, Adagrad, cpu, (pt2) foreach: +0.11479%
- squeezenet1_1, Adagrad, cpu, foreach: +0.08424%
- squeezenet1_1, Adam, cpu, (pt2) foreach: +0.13331%
- squeezenet1_1, AdamW, cpu, (pt2) foreach: +0.09047%
- squeezenet1_1, AdamW, cpu, foreach: +0.08631%
- squeezenet1_1, Adamax, cpu, (pt2) foreach: +0.08963%
- squeezenet1_1, Adamax, cpu, foreach: +0.09383%
- squeezenet1_1, RAdam, cpu, (pt2) foreach: +0.16466%
- squeezenet1_1, RAdam, cpu, foreach: +0.17059%
- squeezenet1_1, Rprop, cpu, (pt2) foreach: +0.11001%
- squeezenet1_1, Rprop, cpu, foreach: +0.07355%
- resnext50_32x4d, Adagrad, cuda, (pt2) maximize: +0.29118%
- resnext50_32x4d, Adamax, cuda, (pt2) default: +0.15980%
- LearningToPaint, RAdam, cpu, (pt2) foreach: +0.23849%
- LearningToPaint, RMSprop, cpu, foreach: +0.09440%
- maml_omniglot, Adam, cuda, (pt2) fused, capturable: +0.09493%
- maml_omniglot, AdamW, cuda, (pt2) fused: +0.08400%
- maml_omniglot, Adamax, cuda, (pt2) maximize: +0.09475%
- maml_omniglot, Adamax, cuda, maximize: +0.07469%
- tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.21699%
- tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9: +0.16226%
- drq, Adagrad, cpu, (pt2) foreach: +0.07707%
- drq, Adagrad, cpu, foreach: +0.57642%
- drq, AdamW, cpu, (pt2) foreach: +0.08673%
- drq, Adamax, cpu, (pt2) foreach: +0.65119%
- drq, ASGD, cpu, foreach: +0.12409%
- drq, SGD, cpu, foreach, momentum=0.9: +0.08101%
- drq, RAdam, cpu, (pt2) foreach: +0.15851%
- timm_regnet, Adagrad, cuda, (pt2) default: +0.17045%
- timm_regnet, Adagrad, cuda, maximize: +0.10894%
- timm_regnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.07037%
- timm_regnet, Adamax, cuda, (pt2) maximize: +0.12068%
- timm_regnet, ASGD, cuda, default: +0.14216%
- timm_regnet, ASGD, cuda, foreach: +0.13934%
- timm_regnet, RAdam, cuda, foreach: +0.07416%
- detectron2_fasterrcnn_r_101_fpn, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.10141%
- detectron2_fasterrcnn_r_101_fpn, AdamW, cuda, (pt2) default: +0.09193%
- detectron2_fasterrcnn_r_101_fpn, Adamax, cuda, default: +0.09849%
- detectron2_fasterrcnn_r_101_fpn, ASGD, cuda, maximize: +0.17490%
- timm_efficientnet, Adagrad, cuda, (pt2) maximize: +0.11317%
- timm_efficientnet, Adagrad, cuda, maximize: +0.36261%
- timm_efficientnet, Adam, cuda, (pt2) default: +0.19098%
- timm_efficientnet, Adam, cuda, default: +0.10568%
- timm_efficientnet, AdamW, cuda, (pt2) foreach: +0.07339%
- timm_efficientnet, Adamax, cuda, (pt2) foreach: +0.08962%
- dcgan, Adam, cuda, (pt2) amsgrad, maximize: +0.07369%
- dcgan, Adamax, cuda, (pt2) foreach: +0.27341%
- mnasnet1_0, Rprop, cuda, (pt2) default: +0.07760%
- mnasnet1_0, NAdam, cuda, (pt2) default: +0.07989%
- detectron2_maskrcnn_r_50_c4, Adam, cpu, (pt2) foreach: +0.32196%
- detectron2_maskrcnn_r_50_c4, AdamW, cpu, (pt2) foreach: +0.07005%
- detectron2_maskrcnn_r_50_c4, AdamW, cpu, foreach: +0.21304%
- detectron2_maskrcnn_r_50_c4, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.07309%
- detectron2_maskrcnn_r_50_c4, SGD, cpu, (pt2) foreach, momentum=0.9: +0.67483%
- detectron2_maskrcnn_r_50_c4, NAdam, cpu, foreach: +0.07352%
- hf_T5_large, Adagrad, cuda, differentiable: +0.07452%
- hf_T5_large, Adam, cuda, no_foreach: +0.07096%
- hf_T5_large, Adam, cuda, differentiable: +0.09099%
- hf_T5_large, AdamW, cuda, foreach, maximize, capturable: +0.10970%
- hf_T5_large, Adamax, cuda, differentiable: +0.09213%
- hf_T5_large, ASGD, cuda, (pt2) maximize: +0.08592%
- detectron2_maskrcnn_r_101_fpn, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.15669%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, maximize: +0.17238%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, foreach: +0.11089%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) default: +0.49721%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) maximize: +0.45401%
- timm_resnest, Adagrad, cuda, default: +0.19442%
- timm_resnest, Adamax, cuda, maximize: +0.12281%
- timm_resnest, ASGD, cuda, default: +0.10122%
- timm_resnest, ASGD, cuda, (pt2) maximize: +0.10073%
- timm_resnest, SGD, cuda, (pt2) default: +0.10330%
- timm_resnest, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +1.00346%
- timm_resnest, SGD, cuda, (pt2) foreach, momentum=0.9: +0.10800%
- resnet18, Adagrad, cpu, foreach: +0.07503%
- resnet18, AdamW, cpu, (pt2) foreach: +0.11106%
- resnet18, AdamW, cpu, foreach: +0.07413%
- resnet18, Adamax, cpu, foreach: +0.11767%
- resnet18, ASGD, cpu, (pt2) foreach: +0.09401%
- resnet18, SGD, cpu, foreach, momentum=0.9, nesterov: +0.08023%
- resnet18, RMSprop, cpu, foreach: +0.07822%
- resnet18, NAdam, cpu, (pt2) foreach: +0.26376%
- yolov3, Adagrad, cuda, foreach: +0.07130%
- yolov3, Adam, cuda, no_foreach: +0.10407%
- yolov3, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.08393%
- yolov3, AdamW, cuda, (pt2) no_foreach: +0.07466%
- yolov3, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.20906%
- yolov3, AdamW, cuda, foreach, maximize, capturable: +0.07060%
- yolov3, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.12993%
- yolov3, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.11876%
- yolov3, Adamax, cuda, (pt2) foreach: +0.09331%
- yolov3, ASGD, cuda, (pt2) default: +0.13586%
- yolov3, ASGD, cuda, (pt2) no_foreach: +0.10871%
- yolov3, RAdam, cuda, default: +0.07816%
- yolov3, RAdam, cuda, (pt2) no_foreach: +0.08807%
- yolov3, RAdam, cuda, foreach: +0.10251%
- yolov3, Rprop, cuda, maximize: +0.08135%
- yolov3, NAdam, cuda, foreach: +0.08415%
- llama, Adagrad, cuda, maximize: +0.10586%
- llama, Adamax, cuda, (pt2) foreach: +0.08198%
- llama, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +2.01158%
- llama, Rprop, cuda, default: +0.07216%
- torchrec_dlrm, AdamW, cpu, (pt2) foreach: +0.37612%
- torchrec_dlrm, Adamax, cpu, foreach: +0.11973%
- torchrec_dlrm, Rprop, cpu, foreach: +0.10003%
- torchrec_dlrm, NAdam, cpu, (pt2) foreach: +0.10446%
- resnet50, Adagrad, cuda, foreach: +0.07086%
- resnet50, Adam, cuda, (pt2) foreach: +0.07207%
- resnet50, AdamW, cuda, (pt2) no_foreach: +0.09016%
- resnet50, Adamax, cuda, (pt2) maximize: +0.09715%
- resnet50, Rprop, cuda, default: +0.08121%
- resnet50, Rprop, cuda, (pt2) foreach: +0.07157%
- resnet50, NAdam, cuda, differentiable: +0.14546%
- soft_actor_critic, Adadelta, cpu, (pt2) foreach: +0.13631%
- soft_actor_critic, Adagrad, cpu, (pt2) foreach: +0.13549%
- soft_actor_critic, Adagrad, cpu, foreach: +0.10875%
- soft_actor_critic, Adam, cpu, (pt2) foreach: +0.13023%
- soft_actor_critic, Adam, cpu, foreach: +0.15709%
- soft_actor_critic, AdamW, cpu, (pt2) foreach: +0.17293%
- soft_actor_critic, AdamW, cpu, foreach: +0.14239%
- soft_actor_critic, Adamax, cpu, (pt2) foreach: +0.15373%
- soft_actor_critic, Adamax, cpu, foreach: +0.14470%
- soft_actor_critic, ASGD, cpu, (pt2) foreach: +0.07942%
- soft_actor_critic, RAdam, cpu, (pt2) foreach: +0.14157%
- soft_actor_critic, RAdam, cpu, foreach: +0.12484%
- soft_actor_critic, Rprop, cpu, (pt2) foreach: +0.12407%
- soft_actor_critic, Rprop, cpu, foreach: +0.11863%
- soft_actor_critic, NAdam, cpu, (pt2) foreach: +0.08345%
- soft_actor_critic, NAdam, cpu, foreach: +0.10671%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.08280%
- detectron2_fasterrcnn_r_50_fpn, ASGD, cuda, maximize: +0.12859%
- detectron2_fasterrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.67619%
- hf_Bert_large, Adagrad, cuda, foreach: +0.08304%
- hf_Bert_large, ASGD, cuda, maximize: +0.09124%
- hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9: +0.23111%
- densenet121, AdamW, cuda, amsgrad, maximize: +0.08223%
- densenet121, ASGD, cuda, (pt2) default: +0.37824%
- densenet121, ASGD, cuda, default: +0.10745%
- densenet121, ASGD, cuda, maximize: +0.08761%
- densenet121, RAdam, cuda, foreach: +0.08444%
- densenet121, Rprop, cuda, default: +0.09504%
- densenet121, Rprop, cuda, (pt2) foreach: +0.16875%
- soft_actor_critic, Adam, cuda, (pt2) amsgrad, maximize: +0.07652%
- soft_actor_critic, ASGD, cuda, (pt2) default: +0.10248%
- speech_transformer, ASGD, cpu, (pt2) foreach: +0.08374%
- alexnet, Adagrad, cuda, (pt2) foreach: +0.07827%
- dcgan, SGD, cpu, (pt2) foreach, momentum=0.9: +0.07336%
- dcgan, SGD, cpu, foreach, momentum=0.9: +0.09655%
- detectron2_fasterrcnn_r_50_dc5, Adagrad, cpu, (pt2) foreach: +0.07417%
- DALLE2_pytorch, Adagrad, cuda, (pt2) default: +0.42231%
- DALLE2_pytorch, Adagrad, cuda, maximize: +0.12828%
- DALLE2_pytorch, Adam, cuda, foreach, maximize, capturable: +0.25147%
- DALLE2_pytorch, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.08160%
- DALLE2_pytorch, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.08927%
- DALLE2_pytorch, RAdam, cuda, (pt2) default: +0.25834%
- DALLE2_pytorch, RAdam, cuda, default: +0.12094%
- DALLE2_pytorch, RAdam, cuda, (pt2) foreach: +0.07485%
- DALLE2_pytorch, RAdam, cuda, foreach: +0.28425%
- DALLE2_pytorch, Rprop, cuda, maximize: +0.10046%
- functorch_maml_omniglot, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.09789%
- Super_SloMo, Adagrad, cuda, default: +0.11507%
- Super_SloMo, Rprop, cuda, default: +0.50406%
- detectron2_maskrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: +0.18755%
- detectron2_maskrcnn_r_50_fpn, Adamax, cuda, (pt2) maximize: +0.08474%
- detectron2_maskrcnn_r_50_fpn, ASGD, cuda, (pt2) default: +0.15193%
- detectron2_maskrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +2.10492%
- vision_maskrcnn, Adamax, cuda, (pt2) maximize: +0.09711%
- vision_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9: +0.50023%
- vision_maskrcnn, Rprop, cuda, (pt2) maximize: +0.07734%
- doctr_det_predictor, Adam, cuda, foreach, maximize, capturable: +0.08186%
- doctr_det_predictor, Adamax, cuda, foreach: +0.23054%
- doctr_det_predictor, Rprop, cuda, (pt2) default: +0.51379%
- doctr_det_predictor, NAdam, cuda, (pt2) foreach: +0.12579%
- hf_Bart, Adagrad, cuda, default: +0.25047%
- hf_Bart, Adagrad, cuda, (pt2) maximize: +0.24501%
- hf_Bart, Adagrad, cuda, foreach: +0.07813%
- hf_Bart, Adamax, cuda, foreach: +0.25149%
- hf_Bart, ASGD, cuda, (pt2) default: +0.08095%
- hf_Bart, Rprop, cuda, maximize: +0.10891%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) default: +0.36810%
- timm_efficientdet, Adadelta, cuda, (pt2) default: +0.12777%
- timm_efficientdet, Adam, cuda, (pt2) default: +0.09676%
- timm_efficientdet, Adam, cuda, default: +0.11994%
- timm_efficientdet, AdamW, cuda, amsgrad, maximize: +0.08774%
- timm_efficientdet, AdamW, cuda, foreach: +0.07090%
- timm_efficientdet, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.07085%
- timm_efficientdet, AdamW, cuda, foreach, maximize, capturable: +0.07446%
- timm_efficientdet, ASGD, cuda, (pt2) maximize: +0.09170%
- timm_efficientdet, RAdam, cuda, foreach: +0.07037%
- timm_efficientdet, Rprop, cuda, default: +0.55212%
- timm_efficientdet, RMSprop, cuda, (pt2) maximize: +0.07440%
- timm_efficientdet, NAdam, cuda, (pt2) default: +0.07281%
- pytorch_CycleGAN_and_pix2pix, AdamW, cpu, foreach: +0.17129%
- pytorch_CycleGAN_and_pix2pix, SGD, cpu, (pt2) foreach: +0.07910%
- pytorch_CycleGAN_and_pix2pix, SGD, cpu, foreach, momentum=0.9, nesterov: +0.09726%
- pytorch_CycleGAN_and_pix2pix, SGD, cpu, foreach, momentum=0.9: +0.08770%
- timm_nfnet, Adamax, cuda, maximize: +0.07743%
- timm_nfnet, Adamax, cuda, (pt2) foreach: +0.08243%
- timm_nfnet, Rprop, cuda, (pt2) default: +0.11373%
- doctr_reco_predictor, AdamW, cuda, (pt2) amsgrad, maximize: +0.10141%
- doctr_reco_predictor, ASGD, cuda, (pt2) default: +0.38156%
- doctr_reco_predictor, RAdam, cuda, (pt2) default: +0.07621%
- doctr_reco_predictor, Rprop, cuda, (pt2) maximize: +0.39441%
- doctr_reco_predictor, NAdam, cuda, (pt2) foreach: +0.08171%
- functorch_dp_cifar10, Adadelta, cpu, (pt2) foreach: +0.10278%
- functorch_dp_cifar10, Adagrad, cpu, foreach: +0.07680%
- functorch_dp_cifar10, Adamax, cpu, foreach: +0.40035%
- functorch_dp_cifar10, SGD, cpu, (pt2) foreach, momentum=0.9: +0.29724%
- functorch_dp_cifar10, NAdam, cpu, foreach: +0.08834%
- timm_vovnet, Adadelta, cpu, foreach: +0.07327%
- pytorch_CycleGAN_and_pix2pix, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.11156%
- hf_DistilBert, Adamax, cuda, foreach: +0.13725%
- hf_DistilBert, ASGD, cuda, maximize: +0.27243%
- hf_DistilBert, Rprop, cuda, maximize: +0.15266%
- hf_DistilBert, Rprop, cuda, foreach: +0.07476%
- functorch_maml_omniglot, Adagrad, cuda, default: +0.10093%
- functorch_maml_omniglot, Adagrad, cuda, maximize: +0.08255%
- functorch_maml_omniglot, Adagrad, cuda, (pt2) foreach: +0.40360%
- functorch_maml_omniglot, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.07867%
- functorch_maml_omniglot, Adamax, cuda, default: +0.08016%
- functorch_maml_omniglot, ASGD, cuda, maximize: +0.10084%
- functorch_maml_omniglot, ASGD, cuda, (pt2) foreach: +0.31621%
- functorch_maml_omniglot, SGD, cuda, foreach, momentum=0.9: +0.09523%
- functorch_maml_omniglot, Rprop, cuda, maximize: +0.26895%
- functorch_dp_cifar10, Adam, cuda, (pt2) foreach: +0.11505%
- functorch_dp_cifar10, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.07566%
- functorch_dp_cifar10, Rprop, cuda, maximize: +0.53347%
- functorch_dp_cifar10, RMSprop, cuda, foreach: +0.09338%
- mobilenet_v2, ASGD, cuda, (pt2) foreach: +0.07836%
- hf_T5, Adamax, cuda, maximize: +0.16838%
- tts_angular, Adadelta, cpu, (pt2) foreach: +0.33848%
- tts_angular, AdamW, cpu, foreach: +0.16873%
- tts_angular, Adamax, cpu, (pt2) foreach: +0.79444%
- tts_angular, Adamax, cpu, foreach: +0.75701%
- tts_angular, SGD, cpu, foreach: +0.11262%
- tts_angular, SGD, cpu, foreach, momentum=0.9, nesterov: +0.08543%
- tts_angular, SGD, cpu, (pt2) foreach, momentum=0.9: +0.08886%
- tts_angular, SGD, cpu, foreach, momentum=0.9: +0.10050%
- tts_angular, RAdam, cpu, foreach: +0.20879%
- tts_angular, Rprop, cpu, foreach: +0.16483%
- tts_angular, RMSprop, cpu, (pt2) foreach: +0.11213%
- LearningToPaint, AdamW, cuda, (pt2) foreach: +0.07468%
- LearningToPaint, ASGD, cuda, maximize: +0.19123%
- LearningToPaint, Rprop, cuda, (pt2) maximize: +0.13697%
- LearningToPaint, Rprop, cuda, maximize: +0.52529%
- LearningToPaint, Rprop, cuda, (pt2) foreach: +0.44068%
- resnet50_quantized_qat, Adagrad, cuda, (pt2) default: +0.07391%
- resnet50_quantized_qat, Adagrad, cuda, (pt2) maximize: +0.10374%
- pytorch_stargan, Adam, cuda, (pt2) foreach, maximize, capturable: +0.08716%
- pytorch_stargan, AdamW, cuda, default: +0.08030%
- pytorch_stargan, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.08923%
- mobilenet_v3_large, Adagrad, cuda, (pt2) default: +0.44211%
- mobilenet_v3_large, Adamax, cuda, (pt2) default: +0.33264%
- mobilenet_v3_large, ASGD, cuda, (pt2) foreach: +0.16634%
- mobilenet_v3_large, Rprop, cuda, (pt2) maximize: +0.08127%
- mobilenet_v3_large, RMSprop, cuda, default: +0.07493%
- opacus_cifar10, Adagrad, cuda, (pt2) maximize: +0.37667%
- opacus_cifar10, Adamax, cuda, (pt2) foreach: +0.07085%
- opacus_cifar10, Rprop, cuda, (pt2) default: +0.14021%
- pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.20723%
- pytorch_unet, RAdam, cuda, (pt2) foreach: +0.09800%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, foreach: +0.12667%
- resnet152, Adagrad, cuda, (pt2) default: +0.12700%
- resnet152, Adagrad, cuda, default: +0.07659%
- resnet152, Adagrad, cuda, foreach: +0.10497%
- resnet152, Adam, cuda, (pt2) foreach, maximize, capturable: +0.07875%
- resnet152, Adam, cuda, foreach, maximize, capturable: +0.15075%
- resnet152, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.07376%
- resnet152, AdamW, cuda, (pt2) amsgrad, maximize: +0.08386%
- resnet152, ASGD, cuda, default: +0.12838%
- resnet152, ASGD, cuda, (pt2) maximize: +0.10536%
- squeezenet1_1, ASGD, cuda, default: +0.07605%
- squeezenet1_1, NAdam, cuda, default: +0.08029%
- phlippe_resnet, SGD, cpu, (pt2) foreach, momentum=0.9: +0.15421%
- phlippe_resnet, Rprop, cpu, (pt2) foreach: +0.10097%
- detectron2_maskrcnn, Adagrad, cuda, maximize: +0.13221%
- detectron2_maskrcnn, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.07875%
- detectron2_maskrcnn, Rprop, cuda, (pt2) default: +0.26405%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/4911275907

cc @janeyx99


I believe this is still just noise..I may need to up the threshold but would like to see some more data before doing that. Some observations:
Total: 417
CUDA: 269  |  CPU: 138
foreach: 255 | fused: 3 | ...
SGD:  43 | (N/R)Adam(W): 136 | ...
pt2: 216

These were..still...all slowdowns :/ Okay this is sus.

Fix for why we  only see slowdowns: https://github.com/pytorch/benchmark/pull/1621