Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '9a8c6559561431401ed10d063c9d28a87aaa5db2'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 9a8c6559561431401ed10d063c9d28a87aaa5db2

Affected PyTorch commit: 9a8c6559561431401ed10d063c9d28a87aaa5db2

Affected Tests:
- densenet121, Adagrad, cuda, maximize: -0.12551%
- densenet121, Adam, cuda, foreach, maximize, capturable: +0.13147%
- densenet121, AdamW, cuda, amsgrad, maximize: -0.07067%
- densenet121, Adamax, cuda, (pt2) default: +0.11828%
- densenet121, Adamax, cuda, maximize: +0.32015%
- densenet121, Adamax, cuda, (pt2) foreach: -0.17531%
- densenet121, NAdam, cuda, (pt2) default: -0.07630%
- densenet121, NAdam, cuda, default: +0.16600%
- pytorch_stargan, Adadelta, cpu, (pt2) foreach: -0.35552%
- pytorch_stargan, Adadelta, cpu, foreach: -0.34724%
- pytorch_stargan, Adagrad, cpu, (pt2) foreach: -0.52386%
- pytorch_stargan, Adagrad, cpu, foreach: -0.46929%
- pytorch_stargan, Adam, cpu, (pt2) foreach: -0.19757%
- pytorch_stargan, ASGD, cpu, (pt2) foreach: -0.09647%
- pytorch_stargan, ASGD, cpu, foreach: -0.09903%
- pytorch_stargan, SGD, cpu, foreach, momentum=0.9: -0.12002%
- pytorch_stargan, RAdam, cpu, (pt2) foreach: -0.12703%
- pytorch_stargan, RAdam, cpu, foreach: -0.12730%
- pytorch_stargan, Rprop, cpu, (pt2) foreach: -0.22607%
- pytorch_stargan, RMSprop, cpu, foreach: -0.08340%
- detectron2_maskrcnn_r_50_fpn, Adam, cuda, foreach, maximize, capturable: +0.21556%
- detectron2_maskrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: +0.10395%
- detectron2_maskrcnn_r_50_fpn, Adamax, cuda, (pt2) default: +0.08106%
- detectron2_maskrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.33246%
- detectron2_maskrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9: -0.66614%
- detectron2_maskrcnn_r_50_fpn, Rprop, cuda, (pt2) maximize: +0.36602%
- DALLE2_pytorch, Adagrad, cuda, (pt2) default: +0.12649%
- DALLE2_pytorch, Adagrad, cuda, (pt2) maximize: +0.08104%
- DALLE2_pytorch, Adagrad, cuda, maximize: +0.36586%
- DALLE2_pytorch, Adagrad, cuda, foreach: -0.12179%
- DALLE2_pytorch, Adam, cuda, default: +0.13485%
- DALLE2_pytorch, Adam, cuda, (pt2) foreach: +0.13968%
- DALLE2_pytorch, Adam, cuda, foreach: +0.08234%
- DALLE2_pytorch, Adam, cuda, (pt2) foreach, maximize, capturable: +0.13823%
- DALLE2_pytorch, Adam, cuda, foreach, maximize, capturable: +0.16264%
- DALLE2_pytorch, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.14788%
- DALLE2_pytorch, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.13009%
- DALLE2_pytorch, Adam, cuda, (pt2) fused: +0.07214%
- DALLE2_pytorch, Adam, cuda, (pt2) fused, amsgrad, maximize: +0.09157%
- DALLE2_pytorch, Adam, cuda, (pt2) fused, capturable: +0.07419%
- DALLE2_pytorch, AdamW, cuda, foreach: +0.15720%
- DALLE2_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.11022%
- DALLE2_pytorch, AdamW, cuda, foreach, maximize, capturable: +0.25566%
- DALLE2_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.10138%
- DALLE2_pytorch, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.13004%
- DALLE2_pytorch, Adamax, cuda, (pt2) foreach: +0.09384%
- DALLE2_pytorch, Adamax, cuda, foreach: +0.21819%
- DALLE2_pytorch, ASGD, cuda, (pt2) default: +0.11747%
- DALLE2_pytorch, ASGD, cuda, default: +0.13890%
- DALLE2_pytorch, ASGD, cuda, (pt2) maximize: +0.09153%
- DALLE2_pytorch, ASGD, cuda, maximize: +0.11299%
- DALLE2_pytorch, ASGD, cuda, (pt2) foreach: +0.07551%
- DALLE2_pytorch, ASGD, cuda, foreach: +0.09653%
- DALLE2_pytorch, SGD, cuda, maximize: +0.08326%
- DALLE2_pytorch, RAdam, cuda, (pt2) default: -0.15976%
- DALLE2_pytorch, RAdam, cuda, foreach: +0.10455%
- DALLE2_pytorch, Rprop, cuda, (pt2) foreach: +0.51305%
- DALLE2_pytorch, Rprop, cuda, foreach: +0.08759%
- functorch_dp_cifar10, Adagrad, cuda, (pt2) default: +0.07366%
- functorch_dp_cifar10, Adagrad, cuda, maximize: +0.09551%
- functorch_dp_cifar10, Adagrad, cuda, (pt2) foreach: +0.11235%
- functorch_dp_cifar10, Adam, cuda, default: +0.09188%
- functorch_dp_cifar10, AdamW, cuda, (pt2) foreach: +0.11785%
- functorch_dp_cifar10, AdamW, cuda, foreach, maximize, capturable: +0.12932%
- functorch_dp_cifar10, ASGD, cuda, foreach: +0.08531%
- hf_T5, ASGD, cuda, maximize: +0.07677%
- tts_angular, Adadelta, cpu, foreach: +0.11071%
- tts_angular, Adam, cpu, foreach: +0.19303%
- tts_angular, AdamW, cpu, (pt2) foreach: +0.08265%
- tts_angular, AdamW, cpu, foreach: +0.33730%
- tts_angular, Adamax, cpu, (pt2) foreach: +0.70025%
- tts_angular, Adamax, cpu, foreach: -0.15506%
- tts_angular, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.14517%
- tts_angular, RAdam, cpu, (pt2) foreach: -0.08707%
- tts_angular, RAdam, cpu, foreach: -0.25512%
- tts_angular, Rprop, cpu, (pt2) foreach: +0.20585%
- tts_angular, NAdam, cpu, foreach: +0.30758%
- hf_Bart, Adagrad, cuda, default: +0.09482%
- hf_Bart, Adagrad, cuda, maximize: +0.07401%
- hf_Bart, Adagrad, cuda, (pt2) foreach: +0.07841%
- hf_Bart, ASGD, cuda, default: +0.09284%
- hf_Bart, Rprop, cuda, foreach: -0.10734%
- dcgan, Adadelta, cuda, (pt2) default: +0.09071%
- dcgan, Adadelta, cuda, (pt2) foreach: +0.09322%
- dcgan, Adagrad, cuda, (pt2) maximize: +0.11435%
- dcgan, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.07249%
- dcgan, SGD, cuda, (pt2) maximize: -0.18413%
- alexnet, Adagrad, cuda, (pt2) maximize: +0.07091%
- alexnet, Adagrad, cuda, (pt2) foreach: -0.09244%
- doctr_reco_predictor, Adam, cuda, foreach, maximize, capturable: +0.13138%
- doctr_reco_predictor, AdamW, cuda, default: +0.11279%
- doctr_reco_predictor, AdamW, cuda, (pt2) amsgrad, maximize: +0.07615%
- doctr_reco_predictor, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.12389%
- doctr_reco_predictor, RAdam, cuda, (pt2) foreach: +0.08524%
- doctr_reco_predictor, Rprop, cuda, maximize: +0.14152%
- doctr_reco_predictor, NAdam, cuda, default: +0.14281%
- doctr_reco_predictor, NAdam, cuda, foreach: +0.07323%
- pytorch_unet, Adagrad, cuda, maximize: +0.07582%
- pytorch_unet, Adam, cuda, default: +0.10640%
- pytorch_unet, Adam, cuda, foreach, maximize, capturable: +0.13114%
- pytorch_unet, AdamW, cuda, (pt2) default: +0.13388%
- pytorch_unet, AdamW, cuda, default: +0.12640%
- pytorch_unet, AdamW, cuda, amsgrad, maximize: +0.07883%
- pytorch_unet, AdamW, cuda, foreach: +0.09295%
- pytorch_unet, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.15821%
- pytorch_unet, AdamW, cuda, foreach, maximize, capturable: +0.14657%
- pytorch_unet, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.07545%
- pytorch_unet, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.09430%
- pytorch_unet, Adamax, cuda, (pt2) default: +0.35678%
- pytorch_unet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.08626%
- pytorch_unet, RAdam, cuda, default: +0.09026%
- pytorch_unet, Rprop, cuda, foreach: +0.09891%
- pytorch_unet, NAdam, cuda, default: +0.08917%
- pytorch_struct, Adagrad, cuda, (pt2) foreach: +0.34338%
- pytorch_struct, Adam, cuda, (pt2) foreach: +0.08619%
- pytorch_struct, Adam, cuda, (pt2) fused, amsgrad, maximize: +0.11253%
- pytorch_struct, Adam, cuda, (pt2) fused, capturable: +0.07166%
- pytorch_struct, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.16366%
- pytorch_struct, Adamax, cuda, (pt2) maximize: +0.08706%
- pytorch_struct, Adamax, cuda, (pt2) foreach: +0.07669%
- pytorch_struct, ASGD, cuda, (pt2) maximize: -0.08283%
- pytorch_struct, ASGD, cuda, foreach: +0.27066%
- pytorch_struct, SGD, cuda, (pt2) maximize: +0.07881%
- pytorch_struct, Rprop, cuda, foreach: +0.12094%
- opacus_cifar10, Adagrad, cpu, (pt2) foreach: -0.12509%
- opacus_cifar10, Adagrad, cpu, foreach: -0.13935%
- opacus_cifar10, Adamax, cpu, foreach: -0.15975%
- opacus_cifar10, SGD, cpu, (pt2) foreach, momentum=0.9: +0.35270%
- opacus_cifar10, SGD, cpu, foreach, momentum=0.9: -0.08476%
- opacus_cifar10, RAdam, cpu, foreach: -0.10989%
- hf_T5_base, Adagrad, cuda, default: +0.15890%
- hf_T5_base, SGD, cuda, (pt2) default: +0.07101%
- hf_T5_base, Rprop, cuda, (pt2) foreach: -0.07823%
- detectron2_maskrcnn_r_101_c4, Adagrad, cuda, default: +0.19489%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, default: -0.10846%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, maximize: +0.29027%
- detectron2_maskrcnn_r_101_c4, ASGD, cuda, (pt2) default: +0.11530%
- detectron2_maskrcnn_r_101_c4, ASGD, cuda, (pt2) maximize: -0.07739%
- functorch_maml_omniglot, Adam, cpu, foreach: +0.13302%
- functorch_maml_omniglot, AdamW, cpu, (pt2) foreach: +0.14462%
- functorch_maml_omniglot, SGD, cpu, (pt2) foreach: +0.07499%
- functorch_maml_omniglot, NAdam, cpu, foreach: +0.12183%
- fastNLP_Bert, Adamax, cuda, default: +0.07024%
- fastNLP_Bert, ASGD, cuda, default: +0.11752%
- fastNLP_Bert, ASGD, cuda, maximize: +0.10515%
- vgg16, Adadelta, cuda, (pt2) foreach: +0.07158%
- vgg16, AdamW, cuda, (pt2) fused, capturable, amsgrad: +0.07427%
- timm_vovnet, Adadelta, cpu, (pt2) foreach: -0.33486%
- timm_vovnet, Adadelta, cpu, foreach: -0.33531%
- timm_vovnet, Adagrad, cpu, (pt2) foreach: -0.33261%
- timm_vovnet, Adagrad, cpu, foreach: -0.21383%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) maximize: +0.08784%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) default: +0.11107%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, default: +0.30705%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) maximize: +0.08081%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, maximize: +0.07270%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) foreach: +0.23805%
- detectron2_maskrcnn_r_101_fpn, Rprop, cuda, (pt2) foreach: +0.07123%
- functorch_maml_omniglot, ASGD, cuda, maximize: +0.10377%
- functorch_maml_omniglot, SGD, cuda, (pt2) maximize: -0.07080%
- functorch_maml_omniglot, SGD, cuda, foreach, momentum=0.9, nesterov: -0.07767%
- functorch_maml_omniglot, Rprop, cuda, (pt2) default: +0.28455%
- hf_Longformer, Rprop, cuda, maximize: +0.07174%
- resnet18, Adadelta, cuda, (pt2) default: +0.10250%
- resnet18, Adagrad, cuda, (pt2) default: +0.08670%
- resnet18, Adagrad, cuda, (pt2) foreach: +0.08704%
- resnet18, Adagrad, cuda, foreach: -0.09143%
- resnet18, SGD, cuda, (pt2) foreach, momentum=0.9: +0.19213%
- resnet18, RAdam, cuda, foreach: +0.09279%
- resnet18, Rprop, cuda, (pt2) default: -0.22133%
- resnet18, NAdam, cuda, (pt2) foreach: +0.14552%
- nvidia_deeprecommender, SGD, cuda, (pt2) foreach: -0.09907%
- timm_vision_transformer, Adagrad, cuda, (pt2) foreach: +0.07357%
- timm_vision_transformer, Adagrad, cuda, foreach: +0.37626%
- timm_vision_transformer, Adam, cuda, (pt2) default: +0.08471%
- timm_vision_transformer, RAdam, cuda, (pt2) differentiable: +0.08036%
- timm_vision_transformer, RAdam, cuda, (pt2) foreach: +0.07019%
- timm_vision_transformer, NAdam, cuda, (pt2) default: -0.10482%
- detectron2_fasterrcnn_r_101_fpn, Adagrad, cuda, default: +0.16005%
- detectron2_fasterrcnn_r_101_fpn, Adagrad, cuda, foreach: +0.27826%
- detectron2_fasterrcnn_r_101_fpn, Adam, cuda, (pt2) foreach, maximize, capturable: +0.07764%
- detectron2_fasterrcnn_r_101_fpn, ASGD, cuda, default: +0.20465%
- detectron2_fasterrcnn_r_101_fpn, RAdam, cuda, (pt2) default: +0.09400%
- detectron2_maskrcnn, Adagrad, cuda, maximize: +0.16066%
- detectron2_maskrcnn, Adagrad, cuda, (pt2) foreach: +0.12062%
- detectron2_maskrcnn, Adamax, cuda, default: +0.19318%
- detectron2_maskrcnn, Adamax, cuda, (pt2) maximize: +0.32357%
- detectron2_maskrcnn, Adamax, cuda, maximize: +0.12672%
- detectron2_maskrcnn, Adamax, cuda, (pt2) foreach: -0.12453%
- detectron2_maskrcnn, ASGD, cuda, default: +0.41130%
- detectron2_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9: -0.33115%
- detectron2_maskrcnn, Rprop, cuda, maximize: +0.09369%
- soft_actor_critic, Adam, cpu, (pt2) foreach: +0.07003%
- soft_actor_critic, RAdam, cpu, foreach: -0.09756%
- soft_actor_critic, Rprop, cpu, (pt2) foreach: -0.08667%
- soft_actor_critic, Rprop, cpu, foreach: -0.09782%
- hf_Albert, Adagrad, cuda, maximize: -0.10859%
- hf_Albert, AdamW, cuda, foreach, maximize, capturable: +0.09309%
- hf_Albert, Adamax, cuda, default: +0.08348%
- hf_Albert, ASGD, cuda, foreach: +0.08143%
- resnet152, Adadelta, cuda, (pt2) default: +0.07898%
- resnet152, Adagrad, cuda, (pt2) default: -0.07823%
- resnet152, AdamW, cuda, (pt2) default: -0.07017%
- resnet152, AdamW, cuda, foreach: +0.13079%
- resnet152, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.08858%
- resnet152, ASGD, cuda, (pt2) maximize: +0.15257%
- resnet152, ASGD, cuda, (pt2) foreach: -0.10327%
- resnet152, RAdam, cuda, (pt2) default: -0.10073%
- resnet152, Rprop, cuda, maximize: -0.17434%
- resnet152, NAdam, cuda, default: +0.08895%
- resnet152, NAdam, cuda, (pt2) foreach: +0.15837%
- resnet152, NAdam, cuda, foreach: +0.08634%
- detectron2_fasterrcnn_r_50_dc5, SGD, cuda, (pt2) foreach, momentum=0.9: -0.19973%
- detectron2_fasterrcnn_r_50_dc5, Rprop, cuda, (pt2) default: +0.08851%
- pytorch_struct, Adagrad, cpu, (pt2) foreach: -0.07739%
- pytorch_struct, Adagrad, cpu, foreach: -0.07323%
- pytorch_struct, Adam, cpu, (pt2) foreach: -0.07004%
- pytorch_struct, Adam, cpu, foreach: -0.07200%
- pytorch_struct, AdamW, cpu, (pt2) foreach: -0.11297%
- pytorch_struct, ASGD, cpu, (pt2) foreach: -0.27195%
- pytorch_struct, ASGD, cpu, foreach: -0.18016%
- pytorch_struct, SGD, cpu, (pt2) foreach: -0.10873%
- pytorch_struct, SGD, cpu, foreach: -0.10614%
- pytorch_struct, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.08546%
- pytorch_struct, NAdam, cpu, (pt2) foreach: +0.08111%
- detectron2_maskrcnn_r_50_c4, Adagrad, cpu, foreach: -0.28835%
- detectron2_maskrcnn_r_50_c4, Adam, cpu, (pt2) foreach: -0.26440%
- detectron2_maskrcnn_r_50_c4, Adam, cpu, foreach: -0.11904%
- detectron2_maskrcnn_r_50_c4, AdamW, cpu, (pt2) foreach: -0.09260%
- detectron2_maskrcnn_r_50_c4, AdamW, cpu, foreach: -0.38479%
- detectron2_maskrcnn_r_50_c4, Adamax, cpu, (pt2) foreach: -0.21826%
- detectron2_maskrcnn_r_50_c4, Adamax, cpu, foreach: -0.31753%
- detectron2_maskrcnn_r_50_c4, RAdam, cpu, (pt2) foreach: -0.08749%
- detectron2_maskrcnn_r_50_c4, NAdam, cpu, (pt2) foreach: -0.09387%
- detectron2_maskrcnn_r_50_c4, NAdam, cpu, foreach: -0.09297%
- timm_vision_transformer_large, Adam, cuda, default: +0.07167%
- timm_vision_transformer_large, Adam, cuda, foreach: +0.07361%
- doctr_det_predictor, Adam, cuda, (pt2) foreach, maximize, capturable: +0.08517%
- doctr_det_predictor, Adam, cuda, foreach, maximize, capturable: +0.25069%
- doctr_det_predictor, ASGD, cuda, default: +0.47928%
- fambench_xlmr, AdamW, cuda, default: +0.09170%
- fambench_xlmr, AdamW, cuda, foreach: -0.09863%
- fambench_xlmr, Rprop, cuda, (pt2) foreach: +0.13189%
- mnasnet1_0, Adagrad, cuda, maximize: +0.08721%
- mnasnet1_0, Adam, cuda, (pt2) amsgrad, maximize: -0.09042%
- mnasnet1_0, Adam, cuda, (pt2) foreach: +0.08004%
- mnasnet1_0, Adam, cuda, (pt2) foreach, maximize, capturable: -0.08730%
- mnasnet1_0, Adam, cuda, foreach, maximize, capturable: -0.07232%
- mnasnet1_0, Adam, cuda, (pt2) fused, amsgrad, maximize: -0.11749%
- mnasnet1_0, AdamW, cuda, default: -0.07566%
- mnasnet1_0, AdamW, cuda, (pt2) foreach: -0.12828%
- mnasnet1_0, AdamW, cuda, foreach, maximize, capturable: -0.10663%
- mnasnet1_0, Adamax, cuda, maximize: +0.31258%
- mnasnet1_0, ASGD, cuda, (pt2) default: +0.43219%
- mnasnet1_0, ASGD, cuda, maximize: +0.16277%
- mnasnet1_0, RAdam, cuda, (pt2) default: +0.20765%
- mnasnet1_0, Rprop, cuda, (pt2) maximize: -0.07037%
- mnasnet1_0, Rprop, cuda, (pt2) foreach: +0.34798%
- hf_Bert, Adagrad, cuda, (pt2) maximize: +0.08659%
- hf_Bert, Adamax, cuda, (pt2) maximize: +0.07906%
- hf_Bert, Rprop, cuda, (pt2) maximize: -0.08753%
- opacus_cifar10, Adagrad, cuda, (pt2) default: +0.14013%
- opacus_cifar10, Adagrad, cuda, maximize: -0.07188%
- opacus_cifar10, AdamW, cuda, default: +0.11573%
- opacus_cifar10, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.07180%
- opacus_cifar10, Adamax, cuda, default: +0.25608%
- opacus_cifar10, ASGD, cuda, (pt2) default: +0.15872%
- opacus_cifar10, ASGD, cuda, (pt2) maximize: -0.07498%
- opacus_cifar10, Rprop, cuda, (pt2) maximize: -0.10489%
- hf_T5_large, AdamW, cuda, no_foreach: -0.07068%
- hf_T5_large, Adamax, cuda, default: +0.07135%
- hf_T5_large, Adamax, cuda, no_foreach: -0.10082%
- hf_T5_large, ASGD, cuda, (pt2) maximize: +0.18153%
- hf_T5_large, ASGD, cuda, foreach: +0.27834%
- hf_T5_large, RAdam, cuda, default: +0.10297%
- hf_T5_large, RAdam, cuda, (pt2) no_foreach: -0.29179%
- hf_T5_large, Rprop, cuda, default: +0.22153%
- hf_T5_large, Rprop, cuda, differentiable: -0.07320%
- LearningToPaint, Adagrad, cuda, (pt2) maximize: -0.28576%
- LearningToPaint, Adagrad, cuda, foreach: +0.10570%
- LearningToPaint, Adam, cuda, default: +0.07132%
- LearningToPaint, Adamax, cuda, (pt2) default: -0.10294%
- LearningToPaint, RAdam, cuda, (pt2) default: -0.11466%
- LearningToPaint, Rprop, cuda, default: -0.20237%
- LearningToPaint, Rprop, cuda, maximize: +0.21862%
- attention_is_all_you_need_pytorch, Adagrad, cuda, (pt2) maximize: -0.07359%
- attention_is_all_you_need_pytorch, Adagrad, cuda, maximize: +0.37178%
- attention_is_all_you_need_pytorch, Adagrad, cuda, (pt2) foreach: -0.09347%
- attention_is_all_you_need_pytorch, Adam, cuda, (pt2) foreach: -0.08448%
- attention_is_all_you_need_pytorch, ASGD, cuda, (pt2) foreach: +0.34337%
- attention_is_all_you_need_pytorch, Rprop, cuda, (pt2) maximize: +0.11283%
- pytorch_stargan, Adagrad, cuda, (pt2) maximize: -0.08341%
- pytorch_stargan, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.12628%
- pytorch_stargan, AdamW, cuda, (pt2) default: +0.07821%
- pytorch_stargan, AdamW, cuda, foreach, maximize, capturable: -0.08528%
- pytorch_stargan, ASGD, cuda, default: +0.07541%
- pytorch_stargan, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.10521%
- pytorch_stargan, SGD, cuda, (pt2) foreach, momentum=0.9: +0.11184%
- pytorch_stargan, Rprop, cuda, (pt2) foreach: +0.12522%
- pytorch_stargan, NAdam, cuda, (pt2) default: +0.09886%
- maml_omniglot, Adamax, cpu, foreach: +0.14693%
- maml_omniglot, SGD, cpu, (pt2) foreach: +0.09742%
- maml_omniglot, RAdam, cpu, foreach: +0.07937%
- hf_Reformer, Adam, cuda, amsgrad, maximize: +0.09801%
- hf_Reformer, Adam, cuda, (pt2) foreach, maximize, capturable: +0.08729%
- resnext50_32x4d, Adam, cuda, (pt2) default: +0.07328%
- resnext50_32x4d, Rprop, cuda, maximize: -0.09134%
- maml, Adagrad, cuda, (pt2) foreach: +0.38893%
- maml, Adam, cuda, (pt2) default: +0.10622%
- maml, Adam, cuda, (pt2) amsgrad, maximize: +0.08463%
- maml, Adam, cuda, (pt2) foreach: +0.13424%
- maml, Adam, cuda, (pt2) foreach, maximize, capturable: +0.08187%
- maml, AdamW, cuda, (pt2) fused, capturable: +0.10829%
- maml, AdamW, cuda, (pt2) fused, capturable, amsgrad: +0.07513%
- maml, Adamax, cuda, default: +0.10856%
- maml, Adamax, cuda, (pt2) maximize: +0.07017%
- maml, ASGD, cuda, (pt2) maximize: +0.28462%
- maml, SGD, cuda, foreach, momentum=0.9, nesterov: +0.12739%
- maml, Rprop, cuda, default: +0.09202%
- maml, Rprop, cuda, (pt2) foreach: +0.40790%
- maml, Rprop, cuda, foreach: +0.09030%
- maml, NAdam, cuda, (pt2) foreach: +0.21272%
- phlippe_densenet, AdamW, cuda, amsgrad, maximize: +0.08413%
- phlippe_densenet, AdamW, cuda, (pt2) foreach: -0.12779%
- phlippe_densenet, ASGD, cuda, (pt2) foreach: +0.07261%
- phlippe_densenet, Rprop, cuda, (pt2) default: -0.08987%
- phlippe_densenet, NAdam, cuda, (pt2) default: +0.15450%
- hf_GPT2_large, Adam, cuda, differentiable: -0.17382%
- hf_GPT2_large, AdamW, cuda, foreach, maximize, capturable: +0.07155%
- hf_GPT2_large, SGD, cuda, (pt2) no_foreach: +0.08947%
- hf_GPT2_large, RAdam, cuda, foreach: -0.08640%
- hf_GPT2_large, NAdam, cuda, default: -0.09889%
- hf_GPT2_large, NAdam, cuda, foreach: +0.11518%
- maml_omniglot, Adadelta, cuda, foreach: -0.07315%
- maml_omniglot, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.18376%
- maml_omniglot, AdamW, cuda, (pt2) foreach: +0.16970%
- maml_omniglot, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.07174%
- maml_omniglot, Adamax, cuda, (pt2) default: +0.27720%
- maml_omniglot, ASGD, cuda, (pt2) foreach: +0.09697%
- maml_omniglot, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.08025%
- maml_omniglot, RAdam, cuda, (pt2) foreach: +0.07389%
- maml_omniglot, RAdam, cuda, foreach: +0.12779%
- maml_omniglot, Rprop, cuda, (pt2) default: +0.39780%
- maml_omniglot, NAdam, cuda, (pt2) foreach: +0.07185%
- timm_efficientdet, Adam, cuda, amsgrad, maximize: +0.10605%
- timm_efficientdet, Adamax, cuda, (pt2) default: +0.34569%
- timm_efficientdet, ASGD, cuda, (pt2) foreach: +0.37602%
- timm_efficientdet, RAdam, cuda, (pt2) foreach: +0.23891%
- timm_efficientdet, Rprop, cuda, default: -0.33394%
- timm_efficientdet, Rprop, cuda, maximize: +0.24264%
- pytorch_CycleGAN_and_pix2pix, Adadelta, cpu, (pt2) foreach: +0.51641%
- pytorch_CycleGAN_and_pix2pix, Adadelta, cpu, foreach: +0.50467%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cpu, (pt2) foreach: +0.38784%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cpu, foreach: +0.39627%
- pytorch_CycleGAN_and_pix2pix, Adam, cpu, (pt2) foreach: +0.25652%
- pytorch_CycleGAN_and_pix2pix, AdamW, cpu, (pt2) foreach: -0.17747%
- pytorch_CycleGAN_and_pix2pix, AdamW, cpu, foreach: -0.21159%
- pytorch_CycleGAN_and_pix2pix, SGD, cpu, (pt2) foreach, momentum=0.9: +0.12572%
- pytorch_CycleGAN_and_pix2pix, RAdam, cpu, (pt2) foreach: -0.18496%
- pytorch_CycleGAN_and_pix2pix, RAdam, cpu, foreach: -0.08784%
- pytorch_CycleGAN_and_pix2pix, Rprop, cpu, foreach: +0.37945%
- pytorch_CycleGAN_and_pix2pix, NAdam, cpu, (pt2) foreach: +0.23399%
- detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, (pt2) default: +0.24287%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, (pt2) amsgrad, maximize: -0.07354%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.07983%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: +0.08591%
- detectron2_fasterrcnn_r_50_fpn, Adamax, cuda, default: +0.07397%
- detectron2_fasterrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.13925%
- detectron2_fasterrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9: -0.16831%
- detectron2_fasterrcnn_r_50_fpn, Rprop, cuda, (pt2) default: +0.21609%
- phlippe_resnet, Adagrad, cuda, (pt2) default: -0.09272%
- phlippe_resnet, Adagrad, cuda, (pt2) maximize: +0.23486%
- phlippe_resnet, Adagrad, cuda, foreach: +0.42611%
- phlippe_resnet, Adam, cuda, foreach, maximize, capturable: +0.10624%
- phlippe_resnet, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.24094%
- phlippe_resnet, AdamW, cuda, (pt2) amsgrad, maximize: +0.12683%
- phlippe_resnet, AdamW, cuda, foreach: +0.09955%
- phlippe_resnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.09107%
- phlippe_resnet, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.13629%
- phlippe_resnet, AdamW, cuda, fused, amsgrad, maximize: +0.08490%
- phlippe_resnet, Adamax, cuda, (pt2) default: +0.22415%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.07613%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9: +0.15284%
- phlippe_resnet, RAdam, cuda, default: +0.28982%
- phlippe_resnet, RAdam, cuda, (pt2) foreach: +0.10391%
- speech_transformer, Adagrad, cuda, (pt2) foreach: -0.08483%
- speech_transformer, Adam, cuda, foreach, maximize, capturable: -0.08234%
- speech_transformer, AdamW, cuda, foreach, maximize, capturable: -0.07870%
- speech_transformer, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.08086%
- speech_transformer, Adamax, cuda, default: +0.13672%
- speech_transformer, RAdam, cuda, (pt2) foreach: +0.10843%
- speech_transformer, RAdam, cuda, foreach: +0.15934%
- speech_transformer, Rprop, cuda, maximize: +0.10155%
- speech_transformer, Rprop, cuda, (pt2) foreach: +0.48202%
- yolov3, Adadelta, cuda, differentiable: -0.20098%
- yolov3, AdamW, cuda, default: +0.07707%
- yolov3, AdamW, cuda, (pt2) no_foreach: +0.10727%
- yolov3, ASGD, cuda, (pt2) maximize: -0.12652%
- yolov3, ASGD, cuda, maximize: +0.20481%
- yolov3, ASGD, cuda, (pt2) no_foreach: +0.10206%
- yolov3, ASGD, cuda, (pt2) foreach: -0.07723%
- yolov3, Rprop, cuda, (pt2) maximize: -0.11187%
- yolov3, RMSprop, cuda, differentiable: +0.12719%
- Background_Matting, AdamW, cuda, foreach: -0.11130%
- Background_Matting, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.08782%
- Background_Matting, Adamax, cuda, foreach: +0.07730%
- Background_Matting, ASGD, cuda, (pt2) maximize: +0.07160%
- Background_Matting, RAdam, cuda, (pt2) default: +0.09553%
- Background_Matting, Rprop, cuda, default: +0.19095%
- Background_Matting, Rprop, cuda, (pt2) maximize: +0.23031%
- Background_Matting, Rprop, cuda, foreach: -0.18869%
- Background_Matting, NAdam, cuda, foreach: -0.10141%
- hf_BigBird, Adagrad, cuda, (pt2) default: +0.11636%
- hf_BigBird, Adamax, cuda, (pt2) maximize: +0.24474%
- hf_BigBird, ASGD, cuda, (pt2) maximize: -0.18053%
- timm_resnest, Adagrad, cuda, (pt2) maximize: +0.15992%
- timm_resnest, Adagrad, cuda, (pt2) foreach: +0.09245%
- timm_resnest, Adam, cuda, amsgrad, maximize: +0.07486%
- timm_resnest, Adam, cuda, (pt2) foreach, maximize, capturable: +0.15676%
- timm_resnest, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.11539%
- timm_resnest, ASGD, cuda, (pt2) default: +0.20210%
- timm_resnest, SGD, cuda, (pt2) foreach, momentum=0.9: -0.41660%
- timm_resnest, Rprop, cuda, (pt2) default: -0.07493%
- timm_resnest, RMSprop, cuda, maximize: +0.07946%
- shufflenet_v2_x1_0, Adadelta, cuda, (pt2) default: -0.08505%
- shufflenet_v2_x1_0, Adagrad, cuda, maximize: +0.11863%
- shufflenet_v2_x1_0, Adam, cuda, (pt2) foreach: +0.18263%
- shufflenet_v2_x1_0, Adam, cuda, foreach, maximize, capturable: -0.07981%
- shufflenet_v2_x1_0, Adam, cuda, (pt2) fused, capturable: +0.07272%
- shufflenet_v2_x1_0, Adamax, cuda, (pt2) foreach: -0.09458%
- shufflenet_v2_x1_0, RMSprop, cuda, (pt2) maximize: +0.10461%
- tacotron2, Adadelta, cuda, (pt2) default: +0.07687%
- tacotron2, Adagrad, cuda, (pt2) maximize: +0.29359%
- tacotron2, ASGD, cuda, (pt2) maximize: +0.15883%
- tacotron2, Rprop, cuda, default: +0.20433%
- timm_efficientnet, Adagrad, cuda, default: +0.33868%
- timm_efficientnet, Adagrad, cuda, (pt2) maximize: +0.14239%
- timm_efficientnet, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.21233%
- timm_efficientnet, Adamax, cuda, (pt2) default: +0.08522%
- timm_efficientnet, Adamax, cuda, (pt2) foreach: -0.18372%
- timm_efficientnet, Adamax, cuda, foreach: +0.14664%
- timm_efficientnet, ASGD, cuda, (pt2) default: +0.08722%
- timm_efficientnet, RAdam, cuda, default: +0.10263%
- timm_efficientnet, Rprop, cuda, (pt2) default: +0.48019%
- llama, Adagrad, cuda, default: -0.13341%
- llama, Adagrad, cuda, maximize: +0.15889%
- llama, Adagrad, cuda, (pt2) foreach: +0.07653%
- llama, AdamW, cuda, (pt2) default: +0.07095%
- llama, Adamax, cuda, default: -0.17859%
- llama, ASGD, cuda, (pt2) default: +0.07185%
- llama, ASGD, cuda, maximize: +0.09403%
- llama, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.98481%
- llama, SGD, cuda, foreach, momentum=0.9, nesterov: +0.10672%
- llama, RAdam, cuda, (pt2) default: +0.20214%
- llama, RAdam, cuda, default: +0.24394%
- llama, RAdam, cuda, (pt2) foreach: +0.20478%
- soft_actor_critic, Adadelta, cuda, (pt2) default: +0.13681%
- soft_actor_critic, Adam, cuda, (pt2) default: -0.08736%
- soft_actor_critic, Adam, cuda, foreach, maximize, capturable: +0.07658%
- soft_actor_critic, AdamW, cuda, (pt2) fused, amsgrad, maximize: +0.08987%
- soft_actor_critic, Adamax, cuda, maximize: +0.15035%
- soft_actor_critic, ASGD, cuda, (pt2) default: +0.11502%
- soft_actor_critic, SGD, cuda, (pt2) maximize: +0.07927%
- soft_actor_critic, RMSprop, cuda, foreach: +0.15660%
- soft_actor_critic, NAdam, cuda, foreach: +0.07383%
- detectron2_fasterrcnn_r_50_dc5, Adam, cpu, (pt2) foreach: +0.07978%
- phlippe_resnet, Adadelta, cpu, foreach: +0.09856%
- phlippe_resnet, Adam, cpu, (pt2) foreach: +0.09367%
- phlippe_resnet, ASGD, cpu, foreach: -0.07339%
- phlippe_resnet, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.09853%
- phlippe_resnet, SGD, cpu, (pt2) foreach, momentum=0.9: -0.07315%
- phlippe_resnet, RMSprop, cpu, foreach: +0.21904%
- phlippe_resnet, NAdam, cpu, (pt2) foreach: +0.20177%
- detectron2_fasterrcnn_r_50_c4, Adagrad, cpu, (pt2) foreach: +0.41810%
- detectron2_fasterrcnn_r_50_c4, Adagrad, cpu, foreach: +0.56487%
- detectron2_fasterrcnn_r_50_c4, Adam, cpu, (pt2) foreach: +0.32341%
- detectron2_fasterrcnn_r_50_c4, Adam, cpu, foreach: +0.55730%
- detectron2_fasterrcnn_r_50_c4, AdamW, cpu, (pt2) foreach: +0.45362%
- detectron2_fasterrcnn_r_50_c4, AdamW, cpu, foreach: +0.50297%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, (pt2) foreach: +0.37831%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, foreach: +0.18488%
- detectron2_fasterrcnn_r_50_c4, RAdam, cpu, (pt2) foreach: -0.08118%
- detectron2_fasterrcnn_r_50_c4, RAdam, cpu, foreach: -0.16343%
- detectron2_fasterrcnn_r_50_c4, Rprop, cpu, (pt2) foreach: -0.12218%
- detectron2_fasterrcnn_r_50_c4, Rprop, cpu, foreach: -0.11952%
- detectron2_fasterrcnn_r_50_c4, NAdam, cpu, (pt2) foreach: -0.16364%
- detectron2_fasterrcnn_r_50_c4, NAdam, cpu, foreach: -0.21956%
- lennard_jones, Adadelta, cuda, default: +0.12897%
- lennard_jones, Adadelta, cuda, (pt2) foreach: +0.08658%
- lennard_jones, Adagrad, cuda, (pt2) foreach: +0.30877%
- lennard_jones, Adam, cuda, foreach: +0.09842%
- lennard_jones, Adam, cuda, (pt2) foreach, maximize, capturable: +0.25368%
- lennard_jones, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.08093%
- lennard_jones, Adam, cuda, (pt2) fused, capturable, amsgrad: +0.08049%
- lennard_jones, Adam, cuda, fused, capturable, amsgrad: +0.07763%
- lennard_jones, Adamax, cuda, (pt2) foreach: +0.07525%
- lennard_jones, ASGD, cuda, (pt2) maximize: +0.32342%
- lennard_jones, ASGD, cuda, foreach: +0.23034%
- lennard_jones, SGD, cuda, default: +0.07738%
- lennard_jones, SGD, cuda, (pt2) maximize: -0.07863%
- lennard_jones, SGD, cuda, maximize: +0.15570%
- lennard_jones, SGD, cuda, foreach, momentum=0.9: +0.09556%
- lennard_jones, Rprop, cuda, foreach: +0.38109%
- lennard_jones, NAdam, cuda, (pt2) default: +0.12849%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) default: +0.32079%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) foreach: +0.10971%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, foreach: +0.46211%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) default: +0.14511%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, default: +0.11208%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, maximize: +0.10444%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) foreach: +0.30141%
- detectron2_fasterrcnn_r_101_c4, Rprop, cuda, default: +0.21537%
- detectron2_fasterrcnn_r_101_c4, Rprop, cuda, (pt2) maximize: +0.36255%
- detectron2_fasterrcnn_r_101_c4, Rprop, cuda, (pt2) foreach: -0.11732%
- dcgan, Adadelta, cpu, (pt2) foreach: -0.07809%
- dcgan, Adadelta, cpu, foreach: -0.09748%
- dcgan, Adagrad, cpu, (pt2) foreach: -0.23501%
- dcgan, Adagrad, cpu, foreach: -0.09343%
- dcgan, Adamax, cpu, foreach: -0.07012%
- dcgan, RMSprop, cpu, (pt2) foreach: -0.09937%
- maml, SGD, cpu, (pt2) foreach: +0.07087%
- maml, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.15898%
- maml, SGD, cpu, (pt2) foreach, momentum=0.9: +0.10177%
- maml, RAdam, cpu, foreach: +0.23692%
- resnet50, Adadelta, cuda, differentiable: +0.10036%
- resnet50, Adagrad, cuda, (pt2) default: +0.08245%
- resnet50, Adagrad, cuda, (pt2) no_foreach: -0.08943%
- resnet50, Adagrad, cuda, (pt2) differentiable: -0.07170%
- resnet50, Adagrad, cuda, differentiable: +0.44767%
- resnet50, Adagrad, cuda, foreach: -0.08627%
- resnet50, AdamW, cuda, foreach, maximize, capturable, amsgrad: +0.17595%
- resnet50, Rprop, cuda, default: +0.10953%
- resnet50, Rprop, cuda, differentiable: +0.07751%
- resnet50, Rprop, cuda, (pt2) foreach: +0.10846%
- resnet18, Adadelta, cpu, (pt2) foreach: +0.17000%
- resnet18, Adadelta, cpu, foreach: +0.08271%
- resnet18, Adagrad, cpu, foreach: -0.12927%
- resnet18, AdamW, cpu, (pt2) foreach: -0.15774%
- resnet18, SGD, cpu, foreach, momentum=0.9: +0.07016%
- resnet18, RAdam, cpu, (pt2) foreach: +0.18896%
- resnet18, Rprop, cpu, (pt2) foreach: +0.16810%
- resnet18, Rprop, cpu, foreach: +0.12645%
- functorch_dp_cifar10, Adagrad, cpu, foreach: +0.07786%
- functorch_dp_cifar10, NAdam, cpu, (pt2) foreach: +0.21745%
- vision_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +2.04393%
- vision_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9: -0.51463%
- drq, Adadelta, cuda, (pt2) foreach: +0.08491%
- drq, Adagrad, cuda, (pt2) default: +0.07195%
- drq, Adagrad, cuda, maximize: -0.10457%
- drq, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.21840%
- drq, AdamW, cuda, (pt2) fused, capturable: +0.08569%
- drq, ASGD, cuda, default: +0.08844%
- drq, RAdam, cuda, (pt2) default: +0.07008%
- mobilenet_v2, Adadelta, cuda, maximize: +0.08604%
- mobilenet_v2, Adam, cuda, (pt2) foreach, maximize, capturable: +0.13310%
- mobilenet_v2, Adamax, cuda, (pt2) default: +0.31591%
- mobilenet_v2, Adamax, cuda, maximize: +0.09901%
- mobilenet_v2, Adamax, cuda, foreach: +0.08084%
- mobilenet_v2, ASGD, cuda, foreach: +0.13008%
- mobilenet_v2, Rprop, cuda, (pt2) default: +0.11904%
- mobilenet_v2, Rprop, cuda, (pt2) maximize: -0.09474%
- mobilenet_v2, Rprop, cuda, (pt2) foreach: +0.46265%
- mobilenet_v2, Rprop, cuda, foreach: +0.11772%
- resnet50_quantized_qat, Adagrad, cuda, maximize: +0.25036%
- resnet50_quantized_qat, Adagrad, cuda, foreach: +0.28794%
- resnet50_quantized_qat, Adam, cuda, default: +0.10765%
- resnet50_quantized_qat, Adam, cuda, (pt2) foreach, maximize, capturable: -0.11088%
- resnet50_quantized_qat, Adam, cuda, foreach, maximize, capturable: -0.08394%
- resnet50_quantized_qat, AdamW, cuda, (pt2) amsgrad, maximize: +0.07956%
- hf_Albert, Adadelta, cpu, (pt2) foreach: +0.07733%
- hf_Albert, Adagrad, cpu, (pt2) foreach: +0.45493%
- hf_Albert, Adagrad, cpu, foreach: +0.38489%
- hf_Albert, Adam, cpu, (pt2) foreach: +0.42073%
- hf_Albert, Adam, cpu, foreach: +0.13335%
- hf_Albert, AdamW, cpu, (pt2) foreach: +0.41673%
- hf_Albert, AdamW, cpu, foreach: +0.36741%
- hf_Albert, Adamax, cpu, (pt2) foreach: +0.07331%
- hf_Albert, Adamax, cpu, foreach: +0.31645%
- hf_Albert, RAdam, cpu, (pt2) foreach: +0.63275%
- hf_Albert, RAdam, cpu, foreach: +0.76291%
- hf_Albert, Rprop, cpu, foreach: +0.09724%
- hf_Albert, NAdam, cpu, (pt2) foreach: +0.50675%
- hf_Albert, NAdam, cpu, foreach: +0.51893%
- lennard_jones, Adadelta, cpu, (pt2) foreach: -0.17895%
- lennard_jones, ASGD, cpu, (pt2) foreach: -0.10306%
- lennard_jones, SGD, cpu, foreach, momentum=0.9, nesterov: +0.19173%
- lennard_jones, RMSprop, cpu, (pt2) foreach: +0.16337%
- lennard_jones, RMSprop, cpu, foreach: +0.10116%
- lennard_jones, NAdam, cpu, (pt2) foreach: +0.08201%
- nvidia_deeprecommender, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.10893%
- Super_SloMo, AdamW, cuda, foreach, maximize, capturable: +0.10524%
- Super_SloMo, Adamax, cuda, maximize: +0.08122%
- Super_SloMo, ASGD, cuda, (pt2) default: -0.07610%
- Super_SloMo, ASGD, cuda, (pt2) foreach: +0.56009%
- Super_SloMo, RAdam, cuda, (pt2) foreach: +0.15665%
- Super_SloMo, Rprop, cuda, default: +0.30133%
- Super_SloMo, Rprop, cuda, foreach: +0.13322%
- timm_regnet, Adam, cuda, default: +0.08499%
- timm_regnet, AdamW, cuda, (pt2) foreach: +0.07227%
- timm_regnet, AdamW, cuda, foreach, maximize, capturable: +0.07086%
- timm_regnet, Adamax, cuda, default: +0.07068%
- timm_regnet, ASGD, cuda, foreach: +0.11481%
- timm_regnet, Rprop, cuda, maximize: +0.09787%
- timm_vision_transformer_large, Adadelta, cpu, (pt2) foreach: +0.14396%
- squeezenet1_1, Adam, cuda, (pt2) fused: +0.09184%
- squeezenet1_1, AdamW, cuda, (pt2) foreach: +0.10136%
- squeezenet1_1, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.15070%
- squeezenet1_1, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.16215%
- squeezenet1_1, Adamax, cuda, default: -0.13665%
- squeezenet1_1, ASGD, cuda, (pt2) default: +0.07929%
- squeezenet1_1, ASGD, cuda, (pt2) maximize: -0.13467%
- squeezenet1_1, ASGD, cuda, foreach: +0.14804%
- squeezenet1_1, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.09033%
- squeezenet1_1, Rprop, cuda, (pt2) foreach: +0.23056%
- hf_DistilBert, ASGD, cuda, (pt2) default: -0.07022%
- hf_DistilBert, ASGD, cuda, (pt2) maximize: -0.13213%
- detectron2_maskrcnn_r_50_c4, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.15816%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) default: +0.13870%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) foreach: +0.11319%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, foreach: +0.31831%
- detectron2_maskrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.18716%
- detectron2_maskrcnn_r_50_c4, RAdam, cuda, (pt2) default: +0.07232%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, (pt2) maximize: +0.17683%
- LearningToPaint, Adamax, cpu, foreach: -0.14847%
- LearningToPaint, RAdam, cpu, (pt2) foreach: -0.19727%
- LearningToPaint, Rprop, cpu, (pt2) foreach: +0.10949%
- LearningToPaint, Rprop, cpu, foreach: +0.08469%
- BERT_pytorch, Adadelta, cuda, no_foreach: +0.60852%
- BERT_pytorch, Adagrad, cuda, (pt2) no_foreach: -0.16863%
- BERT_pytorch, Adagrad, cuda, (pt2) foreach: +0.08524%
- BERT_pytorch, Adagrad, cuda, foreach: -0.29283%
- BERT_pytorch, Adam, cuda, (pt2) differentiable: +0.09547%
- BERT_pytorch, Adam, cuda, foreach, maximize, capturable: +0.10183%
- BERT_pytorch, Adam, cuda, (pt2) fused, amsgrad, maximize: +0.08258%
- BERT_pytorch, AdamW, cuda, foreach, maximize, capturable: -0.07535%
- BERT_pytorch, Adamax, cuda, (pt2) maximize: +0.07276%
- BERT_pytorch, ASGD, cuda, (pt2) default: -0.14115%
- BERT_pytorch, ASGD, cuda, maximize: +0.09312%
- BERT_pytorch, ASGD, cuda, (pt2) no_foreach: +0.11045%
- BERT_pytorch, ASGD, cuda, (pt2) foreach: +0.42820%
- BERT_pytorch, ASGD, cuda, foreach: -0.08985%
- BERT_pytorch, SGD, cuda, (pt2) no_foreach: -0.13119%
- BERT_pytorch, Rprop, cuda, (pt2) default: +0.53074%
- BERT_pytorch, Rprop, cuda, (pt2) maximize: -0.22458%
- BERT_pytorch, Rprop, cuda, no_foreach: +0.14982%
- BERT_pytorch, Rprop, cuda, (pt2) foreach: -0.30952%
- BERT_pytorch, RMSprop, cuda, no_foreach: -0.14912%
- BERT_pytorch, RMSprop, cuda, (pt2) differentiable: +0.12930%
- BERT_pytorch, NAdam, cuda, no_foreach: -0.26950%
- hf_GPT2, AdamW, cuda, (pt2) fused, capturable: -0.08044%
- hf_GPT2, Adamax, cuda, foreach: -0.07234%
- moco, Adagrad, cuda, (pt2) default: +0.29623%
- moco, Adagrad, cuda, (pt2) foreach: -0.07417%
- moco, Adam, cuda, (pt2) foreach: -0.08568%
- moco, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.11738%
- moco, AdamW, cuda, (pt2) foreach: -0.10483%
- moco, AdamW, cuda, foreach, maximize, capturable: -0.08335%
- moco, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.07872%
- moco, ASGD, cuda, maximize: -0.14721%
- moco, Rprop, cuda, maximize: -0.12760%
- timm_resnest, Adadelta, cpu, (pt2) foreach: +0.59279%
- timm_resnest, Adadelta, cpu, foreach: +0.47228%
- timm_resnest, Adagrad, cpu, (pt2) foreach: +0.15016%
- timm_resnest, ASGD, cpu, foreach: -0.07480%
- timm_resnest, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.44249%
- timm_resnest, RAdam, cpu, foreach: +0.07112%
- timm_resnest, Rprop, cpu, (pt2) foreach: -0.10951%
- demucs, Adagrad, cuda, (pt2) maximize: +0.07036%
- demucs, SGD, cuda, (pt2) foreach, momentum=0.9: -0.07899%
- timm_vovnet, Adam, cuda, foreach: +0.07347%
- timm_vovnet, Adamax, cuda, (pt2) foreach: -0.22068%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cuda, default: +0.35588%
- pytorch_CycleGAN_and_pix2pix, Adam, cuda, foreach, maximize, capturable, amsgrad: +0.17727%
- pytorch_CycleGAN_and_pix2pix, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.09556%
- pytorch_CycleGAN_and_pix2pix, Adamax, cuda, (pt2) default: +0.08727%
- pytorch_CycleGAN_and_pix2pix, Adamax, cuda, (pt2) maximize: +0.08886%
- pytorch_CycleGAN_and_pix2pix, Adamax, cuda, foreach: +0.09488%
- pytorch_CycleGAN_and_pix2pix, ASGD, cuda, default: +0.24309%
- pytorch_CycleGAN_and_pix2pix, Rprop, cuda, (pt2) maximize: -0.08062%
- detectron2_fasterrcnn_r_50_c4, ASGD, cuda, (pt2) foreach: +0.14652%
- mobilenet_v2_quantized_qat, Adadelta, cuda, (pt2) foreach: -0.07619%
- mobilenet_v2_quantized_qat, Adam, cuda, (pt2) foreach: +0.07290%
- mobilenet_v2_quantized_qat, Adam, cuda, (pt2) fused: +0.07845%
- mobilenet_v2_quantized_qat, AdamW, cuda, default: +0.10539%
- mobilenet_v2_quantized_qat, AdamW, cuda, (pt2) foreach: +0.07597%
- mobilenet_v2_quantized_qat, Adamax, cuda, default: +0.15787%
- mobilenet_v2_quantized_qat, ASGD, cuda, (pt2) default: -0.09750%
- mobilenet_v2_quantized_qat, RAdam, cuda, default: +0.16499%
- mobilenet_v2_quantized_qat, RAdam, cuda, (pt2) foreach: +0.21733%
- drq, Adagrad, cpu, (pt2) foreach: -0.30926%
- drq, Adagrad, cpu, foreach: -0.30678%
- drq, Adamax, cpu, (pt2) foreach: -0.40999%
- drq, Adamax, cpu, foreach: -0.33109%
- drq, ASGD, cpu, (pt2) foreach: -0.13926%
- drq, ASGD, cpu, foreach: -0.10655%
- drq, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.11335%
- drq, RAdam, cpu, foreach: +0.10636%
- drq, RMSprop, cpu, (pt2) foreach: -0.10206%
- drq, RMSprop, cpu, foreach: -0.11857%
- squeezenet1_1, Adagrad, cpu, (pt2) foreach: -0.09620%
- squeezenet1_1, Adam, cpu, (pt2) foreach: +0.08322%
- squeezenet1_1, Adam, cpu, foreach: +0.09494%
- squeezenet1_1, Adamax, cpu, (pt2) foreach: -0.07435%
- squeezenet1_1, SGD, cpu, (pt2) foreach: +0.08227%
- squeezenet1_1, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.25597%
- detectron2_fasterrcnn_r_101_dc5, Adagrad, cuda, (pt2) default: +0.13962%
- detectron2_fasterrcnn_r_101_dc5, ASGD, cuda, (pt2) default: -0.07860%
- detectron2_fasterrcnn_r_101_dc5, ASGD, cuda, (pt2) foreach: +0.13218%
- tts_angular, Adagrad, cuda, (pt2) foreach: -0.16406%
- tts_angular, Adagrad, cuda, foreach: -0.08560%
- tts_angular, Adam, cuda, (pt2) foreach: +0.08845%
- tts_angular, Adamax, cuda, maximize: -0.07265%
- tts_angular, ASGD, cuda, default: +0.07644%
- torchrec_dlrm, Adam, cpu, (pt2) foreach: +0.11453%
- torchrec_dlrm, AdamW, cpu, (pt2) foreach: -0.30257%
- torchrec_dlrm, RAdam, cpu, foreach: +0.24176%
- torchrec_dlrm, Rprop, cpu, (pt2) foreach: -0.08796%
- torchrec_dlrm, RMSprop, cpu, (pt2) foreach: +0.55010%
- torchrec_dlrm, RMSprop, cpu, foreach: +0.51542%
- torchrec_dlrm, NAdam, cpu, (pt2) foreach: +0.17252%
- timm_nfnet, Adagrad, cuda, (pt2) foreach: +0.07144%
- timm_nfnet, Adagrad, cuda, foreach: +0.15128%
- timm_nfnet, Adam, cuda, (pt2) foreach, maximize, capturable: +0.08319%
- timm_nfnet, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.08113%
- timm_nfnet, AdamW, cuda, foreach, maximize, capturable: -0.10433%
- timm_nfnet, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.07685%
- timm_nfnet, Adamax, cuda, maximize: +0.13854%
- timm_nfnet, ASGD, cuda, maximize: +0.41890%
- timm_nfnet, RAdam, cuda, foreach: +0.14358%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:


Runtime regressions found?
No runtime errors were found in the new benchmarks run--you are all good there!

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/4929351284

cc @janeyx99