Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '9a8c6559561431401ed10d063c9d28a87aaa5db2'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 1b6d6e98fed5cdb0a0c2fedd5354dd8b83dd07f1

Affected PyTorch commit: 9a8c6559561431401ed10d063c9d28a87aaa5db2

Affected Tests:
- maml_omniglot, Adagrad, cuda, (pt2) maximize: +0.07953%
- maml_omniglot, Adamax, cuda, maximize: -0.07111%
- maml_omniglot, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.07419%
- fastNLP_Bert, Adagrad, cuda, (pt2) default: +0.11089%
- fastNLP_Bert, Adamax, cuda, maximize: -0.27513%
- tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.16154%
- tacotron2, SGD, cuda, (pt2) foreach, momentum=0.9: -0.11634%
- timm_vovnet, Adamax, cuda, (pt2) foreach: +0.31107%
- timm_vovnet, SGD, cuda, maximize: +0.07083%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) default: -0.26897%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, default: +0.10523%
- detectron2_fasterrcnn_r_101_c4, Rprop, cuda, (pt2) foreach: +0.11956%
- resnet152, Adam, cuda, foreach, maximize, capturable: -0.09229%
- resnet152, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.07949%
- resnet152, AdamW, cuda, (pt2) default: +0.11986%
- resnet152, ASGD, cuda, default: -0.11398%
- resnet152, ASGD, cuda, (pt2) maximize: -0.09027%
- resnet152, ASGD, cuda, (pt2) foreach: +0.14042%
- resnet152, RAdam, cuda, (pt2) default: +0.12332%
- resnet152, Rprop, cuda, maximize: +0.20105%
- lennard_jones, Rprop, cuda, foreach: +0.07515%
- doctr_det_predictor, Adagrad, cuda, (pt2) default: +0.07676%
- doctr_det_predictor, Adam, cuda, foreach, maximize, capturable: -0.08555%
- doctr_det_predictor, Adamax, cuda, foreach: -0.19444%
- doctr_det_predictor, Rprop, cuda, (pt2) default: -0.33395%
- doctr_det_predictor, NAdam, cuda, (pt2) foreach: -0.10966%
- yolov3, Adadelta, cuda, no_foreach: +0.09145%
- yolov3, Adadelta, cuda, differentiable: +0.24767%
- yolov3, AdamW, cuda, (pt2) no_foreach: -0.07712%
- yolov3, AdamW, cuda, differentiable: +0.09312%
- yolov3, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.15681%
- yolov3, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.08491%
- yolov3, ASGD, cuda, (pt2) default: -0.08021%
- yolov3, ASGD, cuda, (pt2) maximize: +0.09098%
- yolov3, SGD, cuda, no_foreach: +0.07453%
- yolov3, Rprop, cuda, (pt2) maximize: +0.15933%
- yolov3, RMSprop, cuda, differentiable: +0.07163%
- hf_Albert, Adagrad, cuda, default: +0.07487%
- hf_Albert, Adagrad, cuda, (pt2) maximize: +0.08133%
- hf_Albert, Adagrad, cuda, maximize: +0.17982%
- hf_Albert, RMSprop, cuda, (pt2) maximize: -0.08253%
- hf_DistilBert, Adamax, cuda, foreach: -0.09425%
- hf_DistilBert, ASGD, cuda, (pt2) default: +0.07346%
- hf_DistilBert, ASGD, cuda, (pt2) maximize: +0.14105%
- hf_DistilBert, ASGD, cuda, maximize: -0.20211%
- hf_DistilBert, Rprop, cuda, maximize: -0.13217%
- timm_vovnet, Adagrad, cpu, (pt2) foreach: +0.15834%
- timm_vovnet, Adagrad, cpu, foreach: +0.07232%
- lennard_jones, Adadelta, cpu, (pt2) foreach: +0.19945%
- lennard_jones, ASGD, cpu, (pt2) foreach: +0.12228%
- vgg16, Adadelta, cpu, (pt2) foreach: +0.07180%
- LearningToPaint, Adamax, cpu, foreach: +0.19250%
- LearningToPaint, Rprop, cpu, foreach: -0.12298%
- Background_Matting, Adagrad, cuda, default: -0.11770%
- Background_Matting, AdamW, cuda, foreach: +0.11550%
- Background_Matting, Adamax, cuda, maximize: -0.10353%
- Background_Matting, ASGD, cuda, maximize: +0.08371%
- Background_Matting, ASGD, cuda, (pt2) foreach: +0.10255%
- Background_Matting, RAdam, cuda, default: -0.20787%
- Background_Matting, RAdam, cuda, (pt2) foreach: -0.18555%
- Background_Matting, Rprop, cuda, (pt2) default: +0.07907%
- Background_Matting, Rprop, cuda, maximize: +0.10131%
- Background_Matting, Rprop, cuda, foreach: +0.26442%
- Background_Matting, NAdam, cuda, default: +0.08567%
- Background_Matting, NAdam, cuda, foreach: +0.10394%
- soft_actor_critic, Adam, cuda, (pt2) default: +0.08167%
- soft_actor_critic, ASGD, cuda, (pt2) default: -0.10085%
- hf_Bart, Adadelta, cuda, (pt2) maximize: +0.07516%
- hf_Bart, Adagrad, cuda, default: -0.20204%
- hf_Bart, Adagrad, cuda, (pt2) maximize: -0.20047%
- hf_Bart, Adagrad, cuda, foreach: -0.07261%
- hf_Bart, Adamax, cuda, foreach: -0.23704%
- hf_Bart, Rprop, cuda, maximize: -0.07358%
- hf_Bart, Rprop, cuda, foreach: +0.13353%
- detectron2_maskrcnn_r_101_fpn, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.12476%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, maximize: -0.14102%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, foreach: -0.09595%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) default: -0.30411%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) maximize: -0.31839%
- drq, Adagrad, cuda, maximize: -0.16446%
- drq, Adam, cuda, amsgrad, maximize: -0.07321%
- drq, AdamW, cuda, (pt2) amsgrad, maximize: -0.11390%
- drq, AdamW, cuda, foreach, maximize, capturable: -0.08493%
- timm_resnest, Adadelta, cpu, (pt2) foreach: -0.34345%
- timm_resnest, Adadelta, cpu, foreach: -0.20941%
- timm_resnest, Adagrad, cpu, (pt2) foreach: -0.15615%
- timm_resnest, Adagrad, cpu, foreach: -0.19673%
- timm_resnest, Adamax, cpu, foreach: -0.29224%
- timm_resnest, SGD, cpu, foreach, momentum=0.9: -0.08573%
- timm_resnest, Rprop, cpu, (pt2) foreach: +0.09069%
- timm_resnest, NAdam, cpu, (pt2) foreach: -0.11392%
- mobilenet_v2_quantized_qat, AdamW, cuda, (pt2) foreach: -0.08894%
- mobilenet_v2_quantized_qat, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.09141%
- mobilenet_v2_quantized_qat, ASGD, cuda, (pt2) default: +0.11568%
- mobilenet_v2_quantized_qat, Rprop, cuda, maximize: +0.07179%
- nvidia_deeprecommender, SGD, cuda, (pt2) foreach: +0.10867%
- functorch_maml_omniglot, Adagrad, cuda, maximize: -0.07751%
- functorch_maml_omniglot, Adagrad, cuda, (pt2) foreach: -0.25926%
- functorch_maml_omniglot, ASGD, cuda, maximize: -0.08835%
- functorch_maml_omniglot, ASGD, cuda, (pt2) foreach: -0.21603%
- functorch_maml_omniglot, SGD, cuda, foreach, momentum=0.9, nesterov: +0.08477%
- functorch_maml_omniglot, SGD, cuda, foreach, momentum=0.9: -0.08615%
- functorch_maml_omniglot, Rprop, cuda, (pt2) maximize: +0.13710%
- functorch_maml_omniglot, Rprop, cuda, maximize: -0.15690%
- resnet50, Adadelta, cuda, no_foreach: +0.13558%
- resnet50, Adam, cuda, no_foreach: +0.07384%
- resnet50, Adamax, cuda, (pt2) differentiable: -0.07220%
- resnet50, NAdam, cuda, (pt2) differentiable: -0.17092%
- resnet50, NAdam, cuda, differentiable: -0.09587%
- hf_Albert, Adadelta, cpu, foreach: +0.07164%
- hf_Albert, Adagrad, cpu, (pt2) foreach: -0.13935%
- hf_Albert, Adagrad, cpu, foreach: -0.14391%
- hf_Albert, Adam, cpu, (pt2) foreach: -0.27209%
- hf_Albert, AdamW, cpu, (pt2) foreach: -0.28028%
- hf_Albert, AdamW, cpu, foreach: -0.23593%
- hf_Albert, Adamax, cpu, foreach: -0.22067%
- hf_Albert, Rprop, cpu, (pt2) foreach: +0.32544%
- hf_Albert, Rprop, cpu, foreach: +0.35119%
- hf_T5, Adamax, cuda, maximize: -0.11632%
- hf_T5, NAdam, cuda, (pt2) foreach: +0.08740%
- detectron2_fasterrcnn_r_50_dc5, Rprop, cpu, (pt2) foreach: +0.12085%
- LearningToPaint, Adagrad, cuda, (pt2) maximize: +0.38392%
- LearningToPaint, AdamW, cuda, (pt2) foreach: -0.08318%
- LearningToPaint, Adamax, cuda, (pt2) default: +0.17796%
- LearningToPaint, ASGD, cuda, maximize: -0.17004%
- LearningToPaint, RAdam, cuda, (pt2) default: +0.09648%
- LearningToPaint, Rprop, cuda, default: +0.24456%
- LearningToPaint, Rprop, cuda, (pt2) maximize: +0.32448%
- LearningToPaint, Rprop, cuda, maximize: -0.28881%
- LearningToPaint, Rprop, cuda, (pt2) foreach: -0.31121%
- resnet18, Adadelta, cpu, (pt2) foreach: -0.09659%
- resnet18, Adadelta, cpu, foreach: -0.08632%
- resnet18, Adagrad, cpu, foreach: +0.08595%
- resnet18, Adam, cpu, (pt2) foreach: -0.07052%
- resnet18, Adamax, cpu, foreach: +0.29102%
- resnet18, ASGD, cpu, (pt2) foreach: -0.08654%
- resnet18, SGD, cpu, foreach, momentum=0.9, nesterov: -0.07432%
- resnet18, RAdam, cpu, (pt2) foreach: -0.11561%
- resnet18, Rprop, cpu, (pt2) foreach: -0.12680%
- resnet18, NAdam, cpu, foreach: +0.07242%
- maml_omniglot, NAdam, cpu, (pt2) foreach: +0.09298%
- timm_vision_transformer_large, Adam, cuda, default: -0.07515%
- timm_vision_transformer_large, Adam, cuda, foreach: -0.07551%
- torchrec_dlrm, Adam, cpu, (pt2) foreach: -0.36801%
- torchrec_dlrm, Adamax, cpu, (pt2) foreach: +0.10934%
- torchrec_dlrm, Adamax, cpu, foreach: -0.10297%
- torchrec_dlrm, RAdam, cpu, foreach: -0.20039%
- torchrec_dlrm, Rprop, cpu, (pt2) foreach: +0.08824%
- torchrec_dlrm, Rprop, cpu, foreach: -0.07429%
- torchrec_dlrm, RMSprop, cpu, (pt2) foreach: -0.34892%
- torchrec_dlrm, RMSprop, cpu, foreach: -0.32622%
- torchrec_dlrm, NAdam, cpu, foreach: -0.31778%
- densenet121, Adagrad, cuda, maximize: +0.18547%
- densenet121, Adamax, cuda, (pt2) foreach: +0.19222%
- densenet121, ASGD, cuda, (pt2) default: -0.27151%
- densenet121, ASGD, cuda, default: -0.07667%
- densenet121, ASGD, cuda, maximize: -0.08713%
- densenet121, SGD, cuda, (pt2) default: +0.08115%
- densenet121, Rprop, cuda, (pt2) foreach: -0.09533%
- hf_GPT2_large, Adam, cuda, differentiable: +0.22729%
- hf_GPT2_large, AdamW, cuda, no_foreach: +0.07751%
- hf_GPT2_large, RAdam, cuda, default: -0.08915%
- hf_GPT2_large, RAdam, cuda, foreach: +0.09474%
- hf_GPT2_large, NAdam, cuda, foreach: -0.10950%
- timm_vision_transformer, Adam, cuda, no_foreach: +0.08278%
- timm_vision_transformer, Adam, cuda, differentiable: +0.07615%
- timm_vision_transformer, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.07194%
- timm_vision_transformer, Adamax, cuda, (pt2) maximize: -0.12491%
- timm_vision_transformer, Adamax, cuda, (pt2) foreach: -0.11485%
- timm_vision_transformer, Rprop, cuda, (pt2) no_foreach: -0.07057%
- timm_vision_transformer, Rprop, cuda, (pt2) foreach: -0.34017%
- timm_vision_transformer, NAdam, cuda, (pt2) default: +0.09661%
- timm_vision_transformer, NAdam, cuda, (pt2) differentiable: -0.18939%
- hf_Longformer, Adamax, cuda, (pt2) foreach: +0.07243%
- resnext50_32x4d, Adagrad, cuda, (pt2) maximize: -0.24528%
- resnext50_32x4d, Adamax, cuda, (pt2) default: -0.11911%
- resnext50_32x4d, Rprop, cuda, maximize: +0.07519%
- soft_actor_critic, Adagrad, cpu, (pt2) foreach: -0.08405%
- soft_actor_critic, Adagrad, cpu, foreach: -0.10675%
- soft_actor_critic, Adam, cpu, (pt2) foreach: -0.07435%
- soft_actor_critic, AdamW, cpu, (pt2) foreach: -0.12204%
- soft_actor_critic, SGD, cpu, foreach, momentum=0.9: +0.08583%
- soft_actor_critic, RAdam, cpu, foreach: +0.07242%
- soft_actor_critic, Rprop, cpu, foreach: +0.08831%
- speech_transformer, Adagrad, cuda, (pt2) foreach: +0.07062%
- speech_transformer, Adam, cuda, foreach, maximize, capturable: +0.07265%
- BERT_pytorch, Adagrad, cuda, (pt2) no_foreach: +0.24731%
- BERT_pytorch, Adagrad, cuda, differentiable: -0.24039%
- BERT_pytorch, Adagrad, cuda, foreach: +0.46070%
- BERT_pytorch, AdamW, cuda, foreach, maximize, capturable: +0.16247%
- BERT_pytorch, Adamax, cuda, maximize: -0.11144%
- BERT_pytorch, Adamax, cuda, differentiable: +0.07258%
- BERT_pytorch, ASGD, cuda, (pt2) default: +0.22130%
- BERT_pytorch, ASGD, cuda, (pt2) maximize: -0.08397%
- BERT_pytorch, ASGD, cuda, foreach: +0.09995%
- BERT_pytorch, SGD, cuda, (pt2) no_foreach: +0.13359%
- BERT_pytorch, SGD, cuda, no_foreach: -0.21425%
- BERT_pytorch, Rprop, cuda, (pt2) maximize: +0.26828%
- BERT_pytorch, Rprop, cuda, no_foreach: -0.28799%
- BERT_pytorch, Rprop, cuda, (pt2) foreach: +0.42324%
- BERT_pytorch, RMSprop, cuda, no_foreach: +0.15285%
- BERT_pytorch, NAdam, cuda, no_foreach: +0.37374%
- BERT_pytorch, NAdam, cuda, (pt2) differentiable: -0.12487%
- phlippe_resnet, Adagrad, cuda, (pt2) default: +0.11160%
- phlippe_resnet, Adagrad, cuda, default: +0.07042%
- phlippe_resnet, AdamW, cuda, (pt2) fused, amsgrad, maximize: +0.07073%
- phlippe_resnet, SGD, cuda, foreach: +0.07168%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9: -0.08730%
- vision_maskrcnn, Adagrad, cpu, (pt2) foreach: -0.27169%
- vision_maskrcnn, Adagrad, cpu, foreach: -0.31668%
- vision_maskrcnn, Adam, cpu, (pt2) foreach: -0.27348%
- vision_maskrcnn, Adam, cpu, foreach: -0.33056%
- vision_maskrcnn, AdamW, cpu, (pt2) foreach: -0.30964%
- vision_maskrcnn, AdamW, cpu, foreach: -0.29317%
- vision_maskrcnn, Adamax, cpu, (pt2) foreach: -0.12996%
- vision_maskrcnn, Adamax, cpu, foreach: -0.17923%
- drq, Adagrad, cpu, (pt2) foreach: +0.33376%
- drq, Adagrad, cpu, foreach: +0.13961%
- drq, Adamax, cpu, foreach: +0.51834%
- drq, ASGD, cpu, (pt2) foreach: +0.08583%
- drq, RAdam, cpu, (pt2) foreach: -0.13238%
- drq, Rprop, cpu, foreach: -0.09776%
- doctr_reco_predictor, AdamW, cuda, (pt2) amsgrad, maximize: -0.09330%
- doctr_reco_predictor, ASGD, cuda, (pt2) default: -0.26344%
- doctr_reco_predictor, RAdam, cuda, (pt2) default: -0.08912%
- doctr_reco_predictor, Rprop, cuda, (pt2) maximize: -0.28864%
- resnet50_quantized_qat, Adagrad, cuda, (pt2) default: -0.07380%
- resnet50_quantized_qat, Adam, cuda, (pt2) foreach, maximize, capturable: +0.09531%
- resnet50_quantized_qat, ASGD, cuda, (pt2) default: -0.07604%
- dcgan, Adagrad, cpu, (pt2) foreach: +0.27755%
- dcgan, Adam, cpu, (pt2) foreach: +0.07933%
- dcgan, Rprop, cpu, (pt2) foreach: -0.09376%
- hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9: -0.16279%
- hf_Reformer, Rprop, cuda, (pt2) default: +0.09388%
- pytorch_stargan, AdamW, cuda, foreach, maximize, capturable: +0.10547%
- pytorch_stargan, ASGD, cuda, foreach: -0.08553%
- moco, Adagrad, cuda, (pt2) default: -0.07385%
- moco, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.11906%
- moco, AdamW, cuda, (pt2) foreach: +0.10981%
- moco, AdamW, cuda, foreach, maximize, capturable: +0.10286%
- moco, ASGD, cuda, maximize: +0.14737%
- moco, ASGD, cuda, foreach: -0.17099%
- moco, Rprop, cuda, maximize: +0.10947%
- hf_GPT2, AdamW, cuda, (pt2) fused, capturable: +0.08962%
- hf_GPT2, Adamax, cuda, maximize: -0.07502%
- resnet18, Adagrad, cuda, foreach: +0.11372%
- resnet18, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.26345%
- resnet18, Rprop, cuda, (pt2) default: +0.35266%
- detectron2_fasterrcnn_r_50_c4, Adam, cpu, (pt2) foreach: +0.17724%
- detectron2_fasterrcnn_r_50_c4, AdamW, cpu, (pt2) foreach: +0.12184%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, (pt2) foreach: -0.07602%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, foreach: -0.10456%
- detectron2_fasterrcnn_r_50_c4, RAdam, cpu, (pt2) foreach: +0.10373%
- detectron2_fasterrcnn_r_50_c4, RAdam, cpu, foreach: +0.26240%
- detectron2_fasterrcnn_r_50_c4, Rprop, cpu, (pt2) foreach: +0.07998%
- detectron2_fasterrcnn_r_50_c4, Rprop, cpu, foreach: +0.14642%
- detectron2_fasterrcnn_r_50_c4, RMSprop, cpu, (pt2) foreach: -0.07023%
- detectron2_fasterrcnn_r_50_c4, NAdam, cpu, (pt2) foreach: +0.22136%
- detectron2_fasterrcnn_r_50_c4, NAdam, cpu, foreach: +0.12104%
- detectron2_fasterrcnn_r_50_c4, Adam, cuda, (pt2) foreach, maximize, capturable: -0.07624%
- detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.13668%
- detectron2_fasterrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9: -0.11062%
- detectron2_fasterrcnn_r_50_c4, Rprop, cuda, (pt2) default: -0.09704%
- detectron2_fasterrcnn_r_50_c4, Rprop, cuda, maximize: -0.10467%
- detectron2_maskrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: -0.13985%
- detectron2_maskrcnn_r_50_fpn, Adamax, cuda, (pt2) maximize: -0.07760%
- detectron2_maskrcnn_r_50_fpn, ASGD, cuda, (pt2) default: -0.14330%
- detectron2_maskrcnn_r_50_fpn, ASGD, cuda, foreach: -0.08154%
- detectron2_maskrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.51885%
- detectron2_maskrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9: +1.99380%
- attention_is_all_you_need_pytorch, Adagrad, cuda, (pt2) maximize: +0.08928%
- attention_is_all_you_need_pytorch, Adagrad, cuda, maximize: -0.10104%
- attention_is_all_you_need_pytorch, Adagrad, cuda, (pt2) foreach: +0.11106%
- attention_is_all_you_need_pytorch, AdamW, cuda, foreach, maximize, capturable: -0.11887%
- attention_is_all_you_need_pytorch, Adamax, cuda, (pt2) default: -0.13761%
- attention_is_all_you_need_pytorch, ASGD, cuda, (pt2) foreach: +0.09049%
- attention_is_all_you_need_pytorch, RAdam, cuda, (pt2) foreach: -0.07053%
- attention_is_all_you_need_pytorch, Rprop, cuda, default: -0.14393%
- mnasnet1_0, Adadelta, cuda, (pt2) maximize: +0.07141%
- mnasnet1_0, Adam, cuda, (pt2) amsgrad, maximize: +0.07804%
- mnasnet1_0, Adam, cuda, (pt2) foreach, maximize, capturable: +0.07890%
- mnasnet1_0, Adam, cuda, (pt2) fused, amsgrad, maximize: +0.11452%
- mnasnet1_0, AdamW, cuda, (pt2) amsgrad, maximize: +0.10830%
- mnasnet1_0, AdamW, cuda, (pt2) foreach: +0.16618%
- mnasnet1_0, AdamW, cuda, foreach, maximize, capturable: +0.12197%
- mnasnet1_0, ASGD, cuda, (pt2) foreach: -0.07032%
- detectron2_fasterrcnn_r_101_fpn, AdamW, cuda, (pt2) default: -0.07265%
- detectron2_fasterrcnn_r_101_fpn, ASGD, cuda, maximize: -0.13692%
- detectron2_maskrcnn, Adam, cuda, foreach, maximize, capturable: +0.09107%
- detectron2_maskrcnn, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.07506%
- detectron2_maskrcnn, Adamax, cuda, (pt2) foreach: +0.15394%
- detectron2_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +1.01538%
- detectron2_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9: +0.49408%
- detectron2_maskrcnn, Rprop, cuda, (pt2) default: -0.21936%
- dcgan, Adamax, cuda, (pt2) foreach: -0.21202%
- opacus_cifar10, Adadelta, cpu, foreach: -0.07132%
- opacus_cifar10, Adagrad, cpu, foreach: +0.25652%
- opacus_cifar10, Adam, cpu, (pt2) foreach: -0.07272%
- opacus_cifar10, AdamW, cpu, (pt2) foreach: -0.22074%
- opacus_cifar10, AdamW, cpu, foreach: -0.11899%
- opacus_cifar10, Adamax, cpu, (pt2) foreach: -0.30851%
- opacus_cifar10, Adamax, cpu, foreach: -0.16602%
- opacus_cifar10, SGD, cpu, foreach, momentum=0.9, nesterov: -0.10121%
- opacus_cifar10, RAdam, cpu, (pt2) foreach: -0.07235%
- opacus_cifar10, Rprop, cpu, foreach: +0.12339%
- opacus_cifar10, RMSprop, cpu, (pt2) foreach: -0.07156%
- opacus_cifar10, NAdam, cpu, (pt2) foreach: -0.07846%
- opacus_cifar10, NAdam, cpu, foreach: -0.09827%
- timm_vision_transformer_large, Adadelta, cpu, (pt2) foreach: -0.12553%
- timm_efficientdet, Adadelta, cuda, (pt2) default: -0.10343%
- timm_efficientdet, Adam, cuda, (pt2) default: -0.07541%
- timm_efficientdet, Adam, cuda, default: -0.09220%
- timm_efficientdet, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.07013%
- timm_efficientdet, NAdam, cuda, (pt2) foreach: +0.07501%
- timm_regnet, Adagrad, cuda, (pt2) default: -0.13756%
- timm_regnet, Adagrad, cuda, maximize: -0.09569%
- timm_regnet, Adamax, cuda, (pt2) maximize: -0.09662%
- timm_regnet, ASGD, cuda, default: -0.14323%
- timm_regnet, ASGD, cuda, foreach: -0.12524%
- timm_regnet, RAdam, cuda, foreach: -0.07075%
- squeezenet1_1, Adamax, cuda, default: +0.14480%
- squeezenet1_1, ASGD, cuda, (pt2) maximize: +0.15550%
- squeezenet1_1, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.12035%
- hf_Bert, Adagrad, cuda, maximize: +0.07346%
- hf_Bert, Rprop, cuda, (pt2) maximize: +0.11475%
- functorch_dp_cifar10, Adadelta, cpu, foreach: +0.20683%
- functorch_dp_cifar10, Adagrad, cpu, (pt2) foreach: -0.10367%
- functorch_dp_cifar10, Adam, cpu, (pt2) foreach: +0.10549%
- functorch_dp_cifar10, AdamW, cpu, (pt2) foreach: +0.19218%
- functorch_dp_cifar10, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.08149%
- functorch_dp_cifar10, SGD, cpu, (pt2) foreach, momentum=0.9: -0.23438%
- pytorch_CycleGAN_and_pix2pix, Adadelta, cpu, (pt2) foreach: -0.35032%
- pytorch_CycleGAN_and_pix2pix, Adadelta, cpu, foreach: -0.34643%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cpu, (pt2) foreach: -0.38201%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cpu, foreach: -0.35521%
- pytorch_CycleGAN_and_pix2pix, Adam, cpu, (pt2) foreach: -0.20654%
- pytorch_CycleGAN_and_pix2pix, AdamW, cpu, (pt2) foreach: -0.12238%
- pytorch_CycleGAN_and_pix2pix, AdamW, cpu, foreach: -0.22300%
- pytorch_CycleGAN_and_pix2pix, SGD, cpu, (pt2) foreach: -0.07832%
- pytorch_CycleGAN_and_pix2pix, RAdam, cpu, (pt2) foreach: +0.22344%
- pytorch_CycleGAN_and_pix2pix, Rprop, cpu, (pt2) foreach: -0.09053%
- pytorch_CycleGAN_and_pix2pix, Rprop, cpu, foreach: -0.27917%
- pytorch_CycleGAN_and_pix2pix, NAdam, cpu, foreach: +0.19077%
- speech_transformer, ASGD, cpu, (pt2) foreach: -0.07925%
- nvidia_deeprecommender, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.14446%
- hf_T5_large, Adam, cuda, no_foreach: +0.07264%
- hf_T5_large, Adamax, cuda, default: -0.07487%
- hf_T5_large, Adamax, cuda, no_foreach: +0.08822%
- hf_T5_large, ASGD, cuda, (pt2) maximize: -0.07399%
- hf_T5_large, RAdam, cuda, default: -0.10158%
- hf_T5_large, RAdam, cuda, (pt2) no_foreach: +0.07520%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, default: +0.09759%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, foreach: -0.09712%
- detectron2_maskrcnn_r_101_c4, ASGD, cuda, (pt2) maximize: +0.10954%
- pytorch_struct, ASGD, cpu, (pt2) foreach: +0.18875%
- pytorch_struct, ASGD, cpu, foreach: +0.08029%
- pytorch_struct, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.08662%
- pytorch_struct, RMSprop, cpu, foreach: -0.11484%
- pytorch_unet, ASGD, cuda, (pt2) default: +0.10596%
- pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.09103%
- pytorch_unet, RAdam, cuda, (pt2) foreach: -0.10438%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, (pt2) amsgrad, maximize: +0.08118%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.08029%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.08432%
- detectron2_fasterrcnn_r_50_fpn, ASGD, cuda, maximize: -0.08948%
- detectron2_fasterrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.30618%
- detectron2_fasterrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9: +0.19843%
- phlippe_densenet, Adagrad, cuda, (pt2) default: +0.09864%
- phlippe_densenet, Adagrad, cuda, (pt2) maximize: +0.11408%
- phlippe_densenet, Adagrad, cuda, maximize: -0.22361%
- phlippe_densenet, AdamW, cuda, (pt2) foreach: +0.12856%
- phlippe_densenet, NAdam, cuda, foreach: -0.07596%
- timm_efficientnet, Adagrad, cuda, maximize: -0.22634%
- timm_efficientnet, Adam, cuda, (pt2) default: -0.16999%
- timm_efficientnet, Adam, cuda, default: -0.09673%
- timm_efficientnet, AdamW, cuda, (pt2) foreach: -0.07373%
- timm_efficientnet, Adamax, cuda, (pt2) foreach: +0.13061%
- timm_efficientnet, ASGD, cuda, (pt2) maximize: +0.08044%
- hf_BigBird, Adamax, cuda, (pt2) foreach: -0.25746%
- hf_BigBird, ASGD, cuda, (pt2) maximize: +0.42442%
- pytorch_struct, Adagrad, cuda, default: +0.09200%
- pytorch_struct, Adagrad, cuda, maximize: +0.08636%
- pytorch_struct, Adagrad, cuda, foreach: +0.09366%
- pytorch_struct, AdamW, cuda, foreach, maximize, capturable: +0.09170%
- pytorch_struct, Adamax, cuda, (pt2) default: +0.08846%
- pytorch_struct, Adamax, cuda, (pt2) maximize: +0.07138%
- pytorch_struct, Adamax, cuda, (pt2) foreach: +0.07606%
- hf_T5_base, Rprop, cuda, (pt2) foreach: +0.09045%
- tts_angular, Adam, cpu, foreach: -0.23107%
- tts_angular, AdamW, cpu, (pt2) foreach: -0.19433%
- tts_angular, AdamW, cpu, foreach: -0.27487%
- tts_angular, Adamax, cpu, (pt2) foreach: -0.40583%
- tts_angular, SGD, cpu, (pt2) foreach: -0.11082%
- tts_angular, SGD, cpu, foreach, momentum=0.9, nesterov: -0.07579%
- tts_angular, SGD, cpu, foreach, momentum=0.9: -0.08679%
- tts_angular, RAdam, cpu, (pt2) foreach: +0.12993%
- tts_angular, RMSprop, cpu, (pt2) foreach: -0.10131%
- tts_angular, NAdam, cpu, (pt2) foreach: +0.27352%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cuda, default: -0.09339%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cuda, maximize: -0.08112%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cuda, foreach: -0.07603%
- pytorch_CycleGAN_and_pix2pix, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.12074%
- pytorch_CycleGAN_and_pix2pix, Rprop, cuda, (pt2) maximize: +0.10328%
- hf_Bert_large, Adagrad, cuda, foreach: -0.07383%
- hf_Bert_large, ASGD, cuda, maximize: -0.09762%
- opacus_cifar10, Adagrad, cuda, (pt2) maximize: -0.26323%
- opacus_cifar10, Adam, cuda, (pt2) default: +0.07120%
- opacus_cifar10, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.08106%
- opacus_cifar10, ASGD, cuda, foreach: -0.07436%
- opacus_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.08911%
- opacus_cifar10, Rprop, cuda, (pt2) default: -0.13876%
- opacus_cifar10, Rprop, cuda, (pt2) maximize: +0.10832%
- squeezenet1_1, Adam, cpu, (pt2) foreach: -0.08878%
- squeezenet1_1, Adam, cpu, foreach: -0.08992%
- squeezenet1_1, AdamW, cpu, (pt2) foreach: -0.07408%
- squeezenet1_1, RAdam, cpu, foreach: -0.08831%
- squeezenet1_1, NAdam, cpu, (pt2) foreach: -0.08061%
- Super_SloMo, Adagrad, cuda, default: -0.08280%
- Super_SloMo, ASGD, cuda, (pt2) default: +0.08888%
- Super_SloMo, ASGD, cuda, maximize: +0.19874%
- Super_SloMo, Rprop, cuda, default: -0.33311%
- vision_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9: +1.06221%
- tts_angular, Adagrad, cuda, (pt2) foreach: +0.21638%
- tts_angular, Adagrad, cuda, foreach: +0.18090%
- tts_angular, SGD, cuda, (pt2) maximize: +0.07463%
- timm_nfnet, AdamW, cuda, foreach, maximize, capturable: +0.10117%
- timm_nfnet, Adamax, cuda, default: -0.07482%
- timm_nfnet, RAdam, cuda, (pt2) foreach: -0.08100%
- timm_nfnet, Rprop, cuda, (pt2) default: -0.08563%
- pytorch_stargan, Adagrad, cpu, foreach: +0.78862%
- pytorch_stargan, Adam, cpu, (pt2) foreach: +0.17620%
- pytorch_stargan, Adam, cpu, foreach: -0.28734%
- pytorch_stargan, AdamW, cpu, (pt2) foreach: -0.24707%
- pytorch_stargan, AdamW, cpu, foreach: -0.21278%
- pytorch_stargan, RAdam, cpu, foreach: +0.55810%
- pytorch_stargan, NAdam, cpu, (pt2) foreach: -0.31133%
- functorch_dp_cifar10, Adam, cuda, (pt2) foreach: -0.10213%
- functorch_dp_cifar10, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.07189%
- functorch_dp_cifar10, RAdam, cuda, foreach: +0.09055%
- functorch_dp_cifar10, Rprop, cuda, maximize: -0.31422%
- functorch_dp_cifar10, RMSprop, cuda, foreach: -0.07509%
- fambench_xlmr, AdamW, cuda, default: -0.09177%
- fambench_xlmr, ASGD, cuda, (pt2) default: -0.10362%
- maml, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.08325%
- maml, Adamax, cuda, foreach: -0.07861%
- timm_resnest, Adagrad, cuda, (pt2) default: +0.09011%
- timm_resnest, Adagrad, cuda, default: -0.11607%
- timm_resnest, Adagrad, cuda, maximize: +0.09456%
- timm_resnest, ASGD, cuda, default: -0.10029%
- timm_resnest, ASGD, cuda, (pt2) maximize: -0.07442%
- timm_resnest, SGD, cuda, (pt2) default: -0.09012%
- timm_resnest, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.46981%
- timm_resnest, SGD, cuda, (pt2) foreach, momentum=0.9: +0.80975%
- shufflenet_v2_x1_0, Adadelta, cuda, (pt2) default: +0.08478%
- shufflenet_v2_x1_0, Adagrad, cuda, (pt2) default: +0.07492%
- shufflenet_v2_x1_0, Adamax, cuda, (pt2) foreach: +0.09850%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, default: -0.23008%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) maximize: -0.14036%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) foreach: -0.22964%
- detectron2_maskrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.19471%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, (pt2) default: -0.07696%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, default: -0.11740%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, (pt2) maximize: +0.09537%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, (pt2) foreach: +0.09556%
- phlippe_resnet, ASGD, cpu, (pt2) foreach: +0.07579%
- phlippe_resnet, ASGD, cpu, foreach: +0.07688%
- phlippe_resnet, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.14598%
- phlippe_resnet, SGD, cpu, (pt2) foreach, momentum=0.9: -0.08979%
- detectron2_fasterrcnn_r_101_dc5, ASGD, cuda, (pt2) default: +0.07933%
- mobilenet_v3_large, Adagrad, cuda, (pt2) default: -0.28731%
- mobilenet_v3_large, Adamax, cuda, (pt2) default: -0.24930%
- mobilenet_v3_large, ASGD, cuda, (pt2) foreach: -0.11382%
- mobilenet_v3_large, RMSprop, cuda, default: -0.07745%
- detectron2_maskrcnn_r_50_c4, Adagrad, cpu, (pt2) foreach: +0.50266%
- detectron2_maskrcnn_r_50_c4, Adagrad, cpu, foreach: +0.66783%
- detectron2_maskrcnn_r_50_c4, Adam, cpu, (pt2) foreach: +0.14061%
- detectron2_maskrcnn_r_50_c4, Adam, cpu, foreach: +0.16726%
- detectron2_maskrcnn_r_50_c4, AdamW, cpu, foreach: +0.34038%
- detectron2_maskrcnn_r_50_c4, Adamax, cpu, (pt2) foreach: +0.39555%
- detectron2_maskrcnn_r_50_c4, Adamax, cpu, foreach: +0.39314%
- detectron2_maskrcnn_r_50_c4, SGD, cpu, (pt2) foreach, momentum=0.9: -0.39527%
- detectron2_maskrcnn_r_50_c4, RAdam, cpu, (pt2) foreach: +0.17844%
- detectron2_maskrcnn_r_50_c4, NAdam, cpu, (pt2) foreach: +0.10037%
- llama, Adagrad, cuda, default: +0.14606%
- llama, Adamax, cuda, default: +0.19653%
- llama, Adamax, cuda, (pt2) maximize: +0.27801%
- llama, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.50184%
- detectron2_fasterrcnn_r_50_dc5, SGD, cuda, (pt2) foreach, momentum=0.9: +0.18533%
- detectron2_fasterrcnn_r_50_dc5, RMSprop, cuda, (pt2) default: +0.07259%
- DALLE2_pytorch, Adagrad, cuda, (pt2) default: -0.26878%
- DALLE2_pytorch, Adagrad, cuda, default: +0.09181%
- DALLE2_pytorch, Adagrad, cuda, maximize: -0.12657%
- DALLE2_pytorch, Adagrad, cuda, foreach: +0.21274%
- DALLE2_pytorch, Adam, cuda, foreach, maximize, capturable: -0.20089%
- DALLE2_pytorch, AdamW, cuda, foreach, maximize, capturable: -0.07517%
- DALLE2_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.18733%
- DALLE2_pytorch, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.08413%
- DALLE2_pytorch, Adamax, cuda, default: -0.07739%
- DALLE2_pytorch, RAdam, cuda, default: -0.12801%
- DALLE2_pytorch, RAdam, cuda, foreach: -0.21375%
- DALLE2_pytorch, Rprop, cuda, maximize: -0.07463%
- DALLE2_pytorch, NAdam, cuda, foreach: +0.09479%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/4922187952

cc @janeyx99


Observations:
Total: ~502
Speedups: 275 | Slowdowns: 227
CUDA: 348 | CPU: 154
foreach: 309 | fused: 3 | default: 91 | ...
ASGD: 60 | SGD: 50 | (N/R)Adam(W): 161 | ...
pt2: 283
within .1: 217 | .1-.2: 164 | ...