Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '9a8c6559561431401ed10d063c9d28a87aaa5db2'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal or runtime regression.

Base PyTorch commit: 9a8c6559561431401ed10d063c9d28a87aaa5db2

Affected PyTorch commit: 9a8c6559561431401ed10d063c9d28a87aaa5db2

Affected Tests:
- timm_vovnet, AdamW, cuda, default: +0.10896%
- timm_vovnet, Adamax, cuda, (pt2) foreach: +0.09319%
- timm_vovnet, ASGD, cuda, (pt2) foreach: +0.13683%
- timm_vovnet, Rprop, cuda, maximize: +0.12448%
- squeezenet1_1, Adagrad, cpu, (pt2) foreach: +0.07672%
- squeezenet1_1, AdamW, cpu, (pt2) foreach: +0.12556%
- squeezenet1_1, Adamax, cpu, (pt2) foreach: +0.08145%
- squeezenet1_1, ASGD, cpu, foreach: +0.07162%
- squeezenet1_1, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.18569%
- squeezenet1_1, RAdam, cpu, foreach: +0.10239%
- hf_T5_base, Adagrad, cuda, default: -0.14367%
- hf_T5_base, AdamW, cuda, (pt2) fused, capturable: +0.07411%
- vgg16, SGD, cuda, (pt2) default: -0.08717%
- densenet121, Adam, cuda, (pt2) amsgrad, maximize: +0.13857%
- densenet121, Adam, cuda, foreach, maximize, capturable: -0.08941%
- densenet121, AdamW, cuda, (pt2) foreach: -0.07005%
- densenet121, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.08460%
- densenet121, Adamax, cuda, (pt2) default: -0.16400%
- densenet121, Adamax, cuda, maximize: -0.27771%
- densenet121, Rprop, cuda, foreach: +0.08828%
- densenet121, NAdam, cuda, default: -0.16439%
- squeezenet1_1, Adagrad, cuda, default: +0.08210%
- squeezenet1_1, Adam, cuda, (pt2) fused: -0.09006%
- squeezenet1_1, AdamW, cuda, (pt2) foreach: -0.08277%
- squeezenet1_1, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.12461%
- squeezenet1_1, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.14711%
- squeezenet1_1, ASGD, cuda, (pt2) default: -0.10530%
- squeezenet1_1, ASGD, cuda, (pt2) maximize: +0.09284%
- squeezenet1_1, ASGD, cuda, foreach: -0.11475%
- squeezenet1_1, Rprop, cuda, (pt2) foreach: -0.22165%
- squeezenet1_1, NAdam, cuda, (pt2) default: -0.07696%
- nvidia_deeprecommender, SGD, cuda, (pt2) foreach: +0.10484%
- hf_Bert_large, ASGD, cuda, (pt2) default: -0.08286%
- hf_Albert, Adadelta, cpu, (pt2) foreach: -0.11501%
- hf_Albert, Adagrad, cpu, (pt2) foreach: -0.29376%
- hf_Albert, Adagrad, cpu, foreach: -0.47261%
- hf_Albert, Adam, cpu, foreach: -0.16686%
- hf_Albert, AdamW, cpu, (pt2) foreach: -0.26449%
- hf_Albert, AdamW, cpu, foreach: -0.30287%
- hf_Albert, Adamax, cpu, (pt2) foreach: -0.25785%
- hf_Albert, Adamax, cpu, foreach: -0.26796%
- hf_Albert, RAdam, cpu, (pt2) foreach: -0.40303%
- hf_Albert, RAdam, cpu, foreach: -0.43182%
- hf_Albert, Rprop, cpu, (pt2) foreach: -0.26478%
- hf_Albert, Rprop, cpu, foreach: -0.32089%
- hf_Albert, NAdam, cpu, (pt2) foreach: -0.35367%
- hf_Albert, NAdam, cpu, foreach: -0.36370%
- phlippe_resnet, Adagrad, cuda, (pt2) maximize: -0.20322%
- phlippe_resnet, Adagrad, cuda, foreach: -0.31878%
- phlippe_resnet, Adam, cuda, foreach, maximize, capturable: -0.09129%
- phlippe_resnet, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.12853%
- phlippe_resnet, Adam, cuda, (pt2) fused, amsgrad, maximize: +0.07474%
- phlippe_resnet, AdamW, cuda, default: +0.07631%
- phlippe_resnet, AdamW, cuda, (pt2) amsgrad, maximize: -0.10956%
- phlippe_resnet, AdamW, cuda, foreach: -0.13484%
- phlippe_resnet, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.07209%
- phlippe_resnet, Adamax, cuda, (pt2) default: -0.18761%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.07751%
- phlippe_resnet, SGD, cuda, (pt2) foreach, momentum=0.9: +0.12218%
- phlippe_resnet, RAdam, cuda, default: -0.17246%
- phlippe_resnet, RAdam, cuda, (pt2) foreach: -0.10571%
- resnet18, Adagrad, cpu, (pt2) foreach: +0.08418%
- resnet18, Adam, cpu, (pt2) foreach: +0.08071%
- resnet18, RAdam, cpu, (pt2) foreach: -0.16874%
- resnet18, RAdam, cpu, foreach: -0.10565%
- resnet18, NAdam, cpu, (pt2) foreach: -0.14459%
- resnet50, Adadelta, cuda, (pt2) maximize: +0.08711%
- resnet50, Adadelta, cuda, differentiable: -0.09364%
- resnet50, Adagrad, cuda, (pt2) no_foreach: +0.12180%
- resnet50, Adagrad, cuda, (pt2) differentiable: +0.08366%
- resnet50, Adagrad, cuda, differentiable: -0.32503%
- resnet50, Adagrad, cuda, foreach: +0.25497%
- resnet50, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.14483%
- resnet50, Adamax, cuda, (pt2) differentiable: +0.07099%
- resnet50, RAdam, cuda, differentiable: +0.17907%
- resnet50, Rprop, cuda, default: -0.14503%
- resnet50, Rprop, cuda, differentiable: -0.07457%
- vision_maskrcnn, Adagrad, cuda, (pt2) maximize: +0.08644%
- vision_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.51046%
- vision_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9: -0.33312%
- vision_maskrcnn, Rprop, cuda, default: -0.09989%
- pytorch_struct, Adadelta, cpu, (pt2) foreach: +0.09405%
- pytorch_struct, Adagrad, cpu, (pt2) foreach: +0.07742%
- pytorch_struct, Adam, cpu, foreach: +0.07937%
- pytorch_struct, ASGD, cpu, (pt2) foreach: +0.20852%
- alexnet, SGD, cuda, (pt2) foreach, momentum=0.9: +0.08802%
- Background_Matting, AdamW, cuda, foreach: +0.10755%
- Background_Matting, AdamW, cuda, foreach, maximize, capturable: +0.08237%
- Background_Matting, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.09324%
- Background_Matting, ASGD, cuda, (pt2) foreach: -0.07596%
- Background_Matting, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.07272%
- Background_Matting, Rprop, cuda, default: -0.17244%
- Background_Matting, Rprop, cuda, (pt2) maximize: -0.16735%
- Background_Matting, Rprop, cuda, (pt2) foreach: +0.26881%
- torchrec_dlrm, Adadelta, cuda, (pt2) maximize: +0.07580%
- hf_Albert, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.21823%
- hf_Albert, Adamax, cuda, default: -0.08442%
- phlippe_resnet, Adadelta, cpu, foreach: -0.07715%
- phlippe_resnet, Adam, cpu, (pt2) foreach: -0.11203%
- phlippe_resnet, Rprop, cpu, (pt2) foreach: +0.08640%
- phlippe_resnet, RMSprop, cpu, foreach: -0.16189%
- phlippe_resnet, NAdam, cpu, (pt2) foreach: -0.15757%
- hf_DistilBert, Adam, cuda, (pt2) fused, amsgrad, maximize: -0.07151%
- moco, Adagrad, cuda, (pt2) default: -0.22735%
- moco, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.07103%
- moco, AdamW, cuda, (pt2) fused, capturable: +0.07733%
- tts_angular, Adam, cpu, (pt2) foreach: +0.07078%
- tts_angular, Adam, cpu, foreach: +0.07370%
- tts_angular, AdamW, cpu, foreach: -0.23496%
- tts_angular, Adamax, cpu, (pt2) foreach: -0.41856%
- tts_angular, Adamax, cpu, foreach: +0.14886%
- tts_angular, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.11763%
- tts_angular, SGD, cpu, (pt2) foreach, momentum=0.9: +0.07513%
- tts_angular, RAdam, cpu, (pt2) foreach: +0.16276%
- tts_angular, RAdam, cpu, foreach: -0.07056%
- tts_angular, NAdam, cpu, (pt2) foreach: -0.18798%
- tts_angular, NAdam, cpu, foreach: -0.24049%
- hf_GPT2, Adagrad, cuda, foreach: +0.18754%
- hf_GPT2, Rprop, cuda, (pt2) default: +0.12562%
- opacus_cifar10, Adagrad, cuda, (pt2) default: -0.10659%
- opacus_cifar10, Adagrad, cuda, foreach: +0.08775%
- opacus_cifar10, Adam, cuda, (pt2) foreach: +0.08556%
- opacus_cifar10, AdamW, cuda, default: -0.10609%
- opacus_cifar10, AdamW, cuda, foreach: +0.08070%
- opacus_cifar10, Adamax, cuda, default: -0.20244%
- opacus_cifar10, Adamax, cuda, (pt2) maximize: +0.18147%
- opacus_cifar10, Adamax, cuda, maximize: +0.09498%
- opacus_cifar10, ASGD, cuda, (pt2) default: -0.11293%
- opacus_cifar10, ASGD, cuda, foreach: +0.07207%
- opacus_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.14972%
- DALLE2_pytorch, Adagrad, cuda, (pt2) default: -0.10233%
- DALLE2_pytorch, Adagrad, cuda, maximize: -0.21223%
- DALLE2_pytorch, Adam, cuda, (pt2) foreach: -0.12740%
- DALLE2_pytorch, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.10023%
- DALLE2_pytorch, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.09661%
- DALLE2_pytorch, Adam, cuda, (pt2) fused: -0.08071%
- DALLE2_pytorch, Adam, cuda, (pt2) fused, amsgrad, maximize: -0.09773%
- DALLE2_pytorch, Adam, cuda, (pt2) fused, capturable: -0.07500%
- DALLE2_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.11987%
- DALLE2_pytorch, AdamW, cuda, foreach, maximize, capturable: -0.13373%
- DALLE2_pytorch, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.09110%
- DALLE2_pytorch, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.07057%
- DALLE2_pytorch, Adamax, cuda, (pt2) default: +0.15018%
- DALLE2_pytorch, Adamax, cuda, (pt2) foreach: -0.07732%
- DALLE2_pytorch, Adamax, cuda, foreach: -0.17566%
- DALLE2_pytorch, ASGD, cuda, (pt2) default: -0.08728%
- DALLE2_pytorch, ASGD, cuda, default: -0.08707%
- DALLE2_pytorch, ASGD, cuda, (pt2) foreach: -0.08188%
- DALLE2_pytorch, RAdam, cuda, (pt2) default: +0.23786%
- DALLE2_pytorch, RAdam, cuda, foreach: +0.11705%
- DALLE2_pytorch, Rprop, cuda, (pt2) foreach: -0.30359%
- detectron2_maskrcnn_r_101_fpn, Adamax, cuda, (pt2) maximize: -0.10041%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) default: -0.15374%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, default: -0.23652%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) maximize: -0.07504%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, maximize: -0.07350%
- detectron2_maskrcnn_r_101_fpn, ASGD, cuda, (pt2) foreach: -0.28687%
- detectron2_maskrcnn_r_101_fpn, Rprop, cuda, (pt2) default: +0.21499%
- detectron2_maskrcnn_r_101_c4, Adadelta, cuda, (pt2) maximize: +0.07767%
- detectron2_maskrcnn_r_101_c4, Adagrad, cuda, default: -0.14018%
- detectron2_maskrcnn_r_101_c4, Adamax, cuda, maximize: -0.22117%
- detectron2_maskrcnn_r_101_c4, ASGD, cuda, (pt2) default: -0.09711%
- functorch_dp_cifar10, Adagrad, cuda, maximize: -0.10891%
- functorch_dp_cifar10, Adam, cuda, (pt2) fused, capturable: +0.08966%
- functorch_dp_cifar10, AdamW, cuda, (pt2) foreach: -0.10872%
- functorch_dp_cifar10, AdamW, cuda, foreach, maximize, capturable: -0.10842%
- functorch_dp_cifar10, AdamW, cuda, (pt2) fused, capturable: +0.07684%
- functorch_dp_cifar10, ASGD, cuda, (pt2) foreach: +0.12097%
- functorch_dp_cifar10, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.17357%
- functorch_dp_cifar10, RAdam, cuda, (pt2) default: +0.08307%
- functorch_dp_cifar10, RAdam, cuda, (pt2) foreach: +0.16709%
- mnasnet1_0, Adagrad, cuda, maximize: -0.09993%
- mnasnet1_0, Adam, cuda, (pt2) foreach: -0.10091%
- mnasnet1_0, Adam, cuda, (pt2) fused: -0.07314%
- mnasnet1_0, Adamax, cuda, maximize: -0.23694%
- mnasnet1_0, ASGD, cuda, (pt2) default: -0.26784%
- mnasnet1_0, ASGD, cuda, maximize: -0.09074%
- mnasnet1_0, RAdam, cuda, (pt2) default: -0.16732%
- mnasnet1_0, Rprop, cuda, (pt2) foreach: -0.27364%
- soft_actor_critic, Adadelta, cuda, (pt2) default: -0.10790%
- soft_actor_critic, AdamW, cuda, (pt2) fused, amsgrad, maximize: -0.08616%
- soft_actor_critic, Adamax, cuda, maximize: -0.14605%
- soft_actor_critic, ASGD, cuda, (pt2) default: -0.08751%
- soft_actor_critic, Rprop, cuda, (pt2) default: +0.07886%
- soft_actor_critic, Rprop, cuda, foreach: +0.07680%
- soft_actor_critic, RMSprop, cuda, foreach: -0.13298%
- maml, Adagrad, cuda, (pt2) foreach: -0.29089%
- maml, Adam, cuda, (pt2) default: -0.12472%
- maml, Adam, cuda, (pt2) foreach: -0.13545%
- maml, AdamW, cuda, (pt2) fused, capturable: -0.08781%
- maml, AdamW, cuda, (pt2) fused, capturable, amsgrad: -0.07241%
- maml, Adamax, cuda, default: -0.09401%
- maml, ASGD, cuda, default: -0.07265%
- maml, ASGD, cuda, (pt2) maximize: -0.27688%
- maml, SGD, cuda, foreach, momentum=0.9, nesterov: -0.11349%
- maml, Rprop, cuda, (pt2) foreach: -0.25384%
- maml, NAdam, cuda, (pt2) foreach: -0.11322%
- maml, NAdam, cuda, foreach: -0.08648%
- resnet50_quantized_qat, Adagrad, cuda, maximize: -0.09688%
- resnet50_quantized_qat, Adagrad, cuda, foreach: -0.22741%
- resnet50_quantized_qat, AdamW, cuda, (pt2) amsgrad, maximize: -0.07296%
- mobilenet_v2, Adadelta, cuda, maximize: -0.09361%
- mobilenet_v2, Adagrad, cuda, (pt2) maximize: +0.15687%
- mobilenet_v2, Adam, cuda, (pt2) default: +0.14323%
- mobilenet_v2, Adam, cuda, (pt2) amsgrad, maximize: +0.08080%
- mobilenet_v2, Adam, cuda, (pt2) foreach, maximize, capturable: -0.12306%
- mobilenet_v2, AdamW, cuda, foreach, maximize, capturable: -0.09015%
- mobilenet_v2, Adamax, cuda, (pt2) default: -0.24086%
- mobilenet_v2, Adamax, cuda, maximize: -0.09205%
- mobilenet_v2, Adamax, cuda, (pt2) foreach: +0.26099%
- mobilenet_v2, ASGD, cuda, foreach: -0.10663%
- mobilenet_v2, SGD, cuda, (pt2) foreach: +0.08631%
- mobilenet_v2, Rprop, cuda, (pt2) default: -0.09502%
- mobilenet_v2, Rprop, cuda, (pt2) foreach: -0.29942%
- mobilenet_v2, Rprop, cuda, foreach: -0.14754%
- mobilenet_v2, RMSprop, cuda, (pt2) foreach: -0.08141%
- mobilenet_v2_quantized_qat, Adam, cuda, (pt2) foreach: -0.07042%
- mobilenet_v2_quantized_qat, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.10014%
- mobilenet_v2_quantized_qat, Adam, cuda, (pt2) fused: -0.08308%
- mobilenet_v2_quantized_qat, AdamW, cuda, default: -0.08893%
- mobilenet_v2_quantized_qat, AdamW, cuda, (pt2) foreach: -0.07973%
- mobilenet_v2_quantized_qat, Adamax, cuda, default: -0.20293%
- mobilenet_v2_quantized_qat, RAdam, cuda, default: -0.15488%
- mobilenet_v2_quantized_qat, RAdam, cuda, (pt2) foreach: -0.15389%
- mobilenet_v2_quantized_qat, Rprop, cuda, (pt2) default: -0.08146%
- mobilenet_v2_quantized_qat, Rprop, cuda, foreach: -0.10353%
- tts_angular, Adam, cuda, (pt2) foreach: -0.07859%
- tts_angular, ASGD, cuda, foreach: +0.11973%
- tts_angular, Rprop, cuda, (pt2) default: +0.14166%
- detectron2_maskrcnn_r_50_c4, Adagrad, cpu, (pt2) foreach: -0.22047%
- detectron2_maskrcnn_r_50_c4, Adam, cpu, foreach: +0.32053%
- detectron2_maskrcnn_r_50_c4, AdamW, cpu, foreach: +0.10075%
- detectron2_maskrcnn_r_50_c4, Adamax, cpu, (pt2) foreach: -0.09633%
- detectron2_maskrcnn_r_50_c4, Adamax, cpu, foreach: +0.48145%
- shufflenet_v2_x1_0, Adagrad, cuda, maximize: -0.15781%
- shufflenet_v2_x1_0, Adam, cuda, (pt2) foreach: -0.15098%
- shufflenet_v2_x1_0, Adam, cuda, (pt2) fused, capturable: -0.07094%
- shufflenet_v2_x1_0, Adamax, cuda, (pt2) foreach: +0.19677%
- shufflenet_v2_x1_0, ASGD, cuda, (pt2) default: +0.07680%
- shufflenet_v2_x1_0, RMSprop, cuda, (pt2) maximize: -0.11060%
- shufflenet_v2_x1_0, NAdam, cuda, (pt2) default: -0.07658%
- shufflenet_v2_x1_0, NAdam, cuda, foreach: -0.07906%
- fastNLP_Bert, Adagrad, cuda, (pt2) maximize: +0.11875%
- fastNLP_Bert, Adam, cuda, foreach, maximize, capturable: +0.07779%
- fastNLP_Bert, Adamax, cuda, (pt2) default: +0.09516%
- fastNLP_Bert, ASGD, cuda, default: -0.13482%
- fastNLP_Bert, ASGD, cuda, (pt2) maximize: +0.10824%
- fastNLP_Bert, ASGD, cuda, maximize: -0.12336%
- fastNLP_Bert, Rprop, cuda, (pt2) default: +0.18945%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cpu, (pt2) foreach: +0.12364%
- pytorch_CycleGAN_and_pix2pix, AdamW, cpu, (pt2) foreach: +0.28769%
- pytorch_CycleGAN_and_pix2pix, AdamW, cpu, foreach: +0.52162%
- pytorch_CycleGAN_and_pix2pix, NAdam, cpu, foreach: -0.18063%
- timm_resnest, Adagrad, cuda, (pt2) maximize: -0.15561%
- timm_resnest, Adagrad, cuda, (pt2) foreach: -0.12509%
- timm_resnest, Adam, cuda, (pt2) foreach, maximize, capturable: -0.14744%
- timm_resnest, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.13992%
- timm_resnest, ASGD, cuda, (pt2) default: -0.16719%
- timm_resnest, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.08576%
- timm_resnest, SGD, cuda, (pt2) foreach, momentum=0.9: -0.16221%
- timm_efficientdet, Adam, cuda, (pt2) default: +0.07967%
- timm_efficientdet, Adam, cuda, amsgrad, maximize: -0.09052%
- timm_efficientdet, Adamax, cuda, (pt2) default: -0.17048%
- timm_efficientdet, ASGD, cuda, (pt2) foreach: -0.26690%
- timm_efficientdet, RAdam, cuda, (pt2) foreach: -0.19682%
- timm_efficientdet, Rprop, cuda, (pt2) default: +0.17168%
- timm_efficientdet, Rprop, cuda, default: +0.07653%
- timm_efficientdet, Rprop, cuda, maximize: -0.17240%
- timm_efficientdet, NAdam, cuda, (pt2) foreach: -0.07088%
- torchrec_dlrm, Adagrad, cpu, (pt2) foreach: -0.32409%
- torchrec_dlrm, Adam, cpu, (pt2) foreach: +0.40895%
- torchrec_dlrm, Rprop, cpu, foreach: +0.07629%
- torchrec_dlrm, NAdam, cpu, (pt2) foreach: -0.22259%
- torchrec_dlrm, NAdam, cpu, foreach: +0.56961%
- LearningToPaint, Adagrad, cuda, (pt2) foreach: +0.33932%
- LearningToPaint, Adagrad, cuda, foreach: -0.12212%
- LearningToPaint, Adam, cuda, default: -0.07308%
- LearningToPaint, Adam, cuda, (pt2) foreach: +0.12789%
- LearningToPaint, Adam, cuda, (pt2) foreach, maximize, capturable: +0.23662%
- LearningToPaint, Adam, cuda, foreach, maximize, capturable: +0.12515%
- LearningToPaint, AdamW, cuda, foreach, maximize, capturable: +0.12668%
- LearningToPaint, SGD, cuda, foreach: +0.07308%
- LearningToPaint, Rprop, cuda, (pt2) maximize: -0.33036%
- LearningToPaint, Rprop, cuda, maximize: -0.24021%
- LearningToPaint, NAdam, cuda, foreach: +0.17526%
- timm_efficientnet, Adagrad, cuda, default: -0.27072%
- timm_efficientnet, Adagrad, cuda, (pt2) maximize: -0.11597%
- timm_efficientnet, Adam, cuda, fused, capturable: +0.07907%
- timm_efficientnet, Adam, cuda, fused, capturable, amsgrad: +0.07624%
- timm_efficientnet, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.12577%
- timm_efficientnet, Adamax, cuda, foreach: -0.13303%
- timm_efficientnet, ASGD, cuda, (pt2) default: -0.09990%
- timm_efficientnet, SGD, cuda, (pt2) default: +0.07620%
- timm_efficientnet, RAdam, cuda, default: -0.08130%
- timm_efficientnet, Rprop, cuda, (pt2) default: -0.31791%
- attention_is_all_you_need_pytorch, Adagrad, cuda, (pt2) maximize: +0.10908%
- attention_is_all_you_need_pytorch, Adagrad, cuda, maximize: -0.28001%
- attention_is_all_you_need_pytorch, AdamW, cuda, (pt2) default: +0.07565%
- attention_is_all_you_need_pytorch, Adamax, cuda, default: +0.41836%
- attention_is_all_you_need_pytorch, Adamax, cuda, (pt2) maximize: +0.13028%
- attention_is_all_you_need_pytorch, ASGD, cuda, (pt2) foreach: -0.27352%
- attention_is_all_you_need_pytorch, Rprop, cuda, (pt2) maximize: -0.11216%
- attention_is_all_you_need_pytorch, Rprop, cuda, (pt2) foreach: +0.36950%
- maml_omniglot, Adamax, cpu, foreach: -0.14672%
- maml_omniglot, SGD, cpu, (pt2) foreach: -0.08227%
- opacus_cifar10, Adadelta, cpu, (pt2) foreach: -0.34481%
- opacus_cifar10, Adadelta, cpu, foreach: -0.31620%
- opacus_cifar10, Adagrad, cpu, (pt2) foreach: -0.18762%
- opacus_cifar10, Adagrad, cpu, foreach: -0.10127%
- opacus_cifar10, AdamW, cpu, foreach: +0.16464%
- opacus_cifar10, SGD, cpu, (pt2) foreach, momentum=0.9: -0.21476%
- opacus_cifar10, SGD, cpu, foreach, momentum=0.9: +0.10559%
- opacus_cifar10, RAdam, cpu, (pt2) foreach: +0.07870%
- opacus_cifar10, RAdam, cpu, foreach: +0.17010%
- opacus_cifar10, Rprop, cpu, (pt2) foreach: +0.13422%
- tacotron2, Adagrad, cuda, (pt2) maximize: -0.22545%
- tacotron2, Adam, cuda, foreach, maximize, capturable: +0.07123%
- tacotron2, ASGD, cuda, (pt2) maximize: -0.12933%
- tacotron2, Rprop, cuda, default: -0.17871%
- tacotron2, Rprop, cuda, foreach: +0.07055%
- doctr_reco_predictor, Adam, cuda, foreach, maximize, capturable: -0.10958%
- doctr_reco_predictor, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.07493%
- doctr_reco_predictor, AdamW, cuda, default: -0.09548%
- doctr_reco_predictor, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.07545%
- doctr_reco_predictor, SGD, cuda, (pt2) foreach, momentum=0.9: +0.08292%
- doctr_reco_predictor, RAdam, cuda, (pt2) foreach: -0.07084%
- doctr_reco_predictor, Rprop, cuda, default: +0.10910%
- doctr_reco_predictor, Rprop, cuda, (pt2) maximize: +0.31232%
- doctr_reco_predictor, Rprop, cuda, maximize: -0.09868%
- doctr_reco_predictor, NAdam, cuda, default: -0.07759%
- dcgan, Adadelta, cuda, (pt2) default: -0.08888%
- dcgan, Adadelta, cuda, (pt2) foreach: -0.08450%
- dcgan, Adagrad, cuda, (pt2) maximize: -0.10577%
- dcgan, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.07569%
- dcgan, AdamW, cuda, (pt2) fused: -0.07144%
- dcgan, Adamax, cuda, maximize: +0.10892%
- dcgan, Adamax, cuda, (pt2) foreach: +0.27021%
- dcgan, Adamax, cuda, foreach: +0.16620%
- dcgan, ASGD, cuda, (pt2) foreach: +0.10328%
- dcgan, SGD, cuda, (pt2) maximize: +0.18159%
- dcgan, SGD, cuda, maximize: +0.07246%
- dcgan, Rprop, cuda, (pt2) maximize: +0.16216%
- dcgan, RMSprop, cuda, foreach: +0.17591%
- soft_actor_critic, Adagrad, cpu, (pt2) foreach: +0.08153%
- soft_actor_critic, Adagrad, cpu, foreach: +0.19205%
- soft_actor_critic, AdamW, cpu, (pt2) foreach: +0.10587%
- detectron2_fasterrcnn_r_101_dc5, Adagrad, cuda, (pt2) default: -0.13323%
- detectron2_fasterrcnn_r_101_dc5, ASGD, cuda, (pt2) foreach: -0.10117%
- drq, Adadelta, cuda, (pt2) foreach: -0.08511%
- drq, Adagrad, cuda, (pt2) default: -0.07219%
- drq, Adam, cuda, foreach, maximize, capturable: -0.08136%
- drq, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.16754%
- drq, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.07447%
- drq, AdamW, cuda, (pt2) fused, capturable: -0.09434%
- drq, RAdam, cuda, default: +0.11943%
- drq, Rprop, cuda, default: +0.11249%
- drq, Rprop, cuda, (pt2) foreach: +0.07337%
- hf_GPT2_large, SGD, cuda, (pt2) no_foreach: -0.08655%
- hf_GPT2_large, NAdam, cuda, default: +0.11794%
- timm_vovnet, Rprop, cpu, (pt2) foreach: +0.08037%
- yolov3, Adadelta, cuda, differentiable: +0.25090%
- yolov3, Adagrad, cuda, no_foreach: +0.19244%
- yolov3, Adagrad, cuda, (pt2) foreach: -0.08722%
- yolov3, Adam, cuda, differentiable: -0.07305%
- yolov3, AdamW, cuda, (pt2) no_foreach: -0.11682%
- yolov3, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.08204%
- yolov3, AdamW, cuda, (pt2) fused, capturable: -0.07661%
- yolov3, ASGD, cuda, (pt2) default: +0.08133%
- yolov3, ASGD, cuda, maximize: -0.18398%
- yolov3, ASGD, cuda, (pt2) no_foreach: -0.13650%
- yolov3, SGD, cuda, foreach: -0.07990%
- yolov3, RMSprop, cuda, differentiable: -0.15908%
- timm_nfnet, Adam, cuda, (pt2) amsgrad, maximize: +0.10025%
- timm_nfnet, Adamax, cuda, maximize: -0.10011%
- timm_nfnet, ASGD, cuda, maximize: -0.27762%
- timm_nfnet, RAdam, cuda, foreach: -0.11540%
- timm_nfnet, Rprop, cuda, (pt2) foreach: +0.08873%
- timm_nfnet, Rprop, cuda, foreach: +0.07207%
- hf_Longformer, Adagrad, cuda, default: +0.09614%
- hf_Longformer, Rprop, cuda, maximize: -0.07330%
- dcgan, NAdam, cpu, (pt2) foreach: +0.08928%
- pytorch_stargan, Adadelta, cuda, (pt2) default: -0.08133%
- pytorch_stargan, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.12876%
- pytorch_stargan, Adamax, cuda, default: -0.10091%
- pytorch_stargan, SGD, cuda, (pt2) foreach, momentum=0.9: -0.10723%
- pytorch_stargan, Rprop, cuda, (pt2) foreach: -0.11950%
- pytorch_stargan, NAdam, cuda, (pt2) default: -0.08078%
- BERT_pytorch, Adadelta, cuda, no_foreach: -0.37637%
- BERT_pytorch, Adagrad, cuda, (pt2) default: -0.07982%
- BERT_pytorch, Adagrad, cuda, (pt2) differentiable: +0.27615%
- BERT_pytorch, Adam, cuda, (pt2) differentiable: -0.07682%
- BERT_pytorch, Adam, cuda, foreach, maximize, capturable: -0.09680%
- BERT_pytorch, Adam, cuda, (pt2) fused, amsgrad, maximize: -0.07660%
- BERT_pytorch, AdamW, cuda, (pt2) no_foreach: +0.12229%
- BERT_pytorch, AdamW, cuda, differentiable: +0.15690%
- BERT_pytorch, AdamW, cuda, foreach, maximize, capturable: -0.09331%
- BERT_pytorch, AdamW, cuda, (pt2) fused: -0.07487%
- BERT_pytorch, ASGD, cuda, (pt2) maximize: +0.42760%
- BERT_pytorch, ASGD, cuda, maximize: -0.19713%
- BERT_pytorch, ASGD, cuda, no_foreach: +0.29336%
- BERT_pytorch, ASGD, cuda, differentiable: +0.34991%
- BERT_pytorch, ASGD, cuda, (pt2) foreach: -0.27569%
- BERT_pytorch, RAdam, cuda, (pt2) differentiable: -0.08109%
- BERT_pytorch, Rprop, cuda, (pt2) default: -0.35045%
- BERT_pytorch, Rprop, cuda, no_foreach: -0.13614%
- BERT_pytorch, RMSprop, cuda, (pt2) differentiable: -0.08980%
- BERT_pytorch, RMSprop, cuda, differentiable: +0.52822%
- vision_maskrcnn, Adam, cpu, (pt2) foreach: +0.16970%
- vision_maskrcnn, Adam, cpu, foreach: +0.27016%
- vision_maskrcnn, AdamW, cpu, foreach: +0.07093%
- vision_maskrcnn, RAdam, cpu, (pt2) foreach: +0.09155%
- vision_maskrcnn, NAdam, cpu, (pt2) foreach: +0.13236%
- vision_maskrcnn, NAdam, cpu, foreach: +0.24911%
- speech_transformer, Adadelta, cuda, (pt2) maximize: +0.09483%
- speech_transformer, AdamW, cuda, (pt2) amsgrad, maximize: -0.07198%
- speech_transformer, Adamax, cuda, default: -0.11519%
- speech_transformer, RAdam, cuda, (pt2) foreach: -0.11715%
- speech_transformer, RAdam, cuda, foreach: -0.13739%
- speech_transformer, Rprop, cuda, (pt2) foreach: -0.31548%
- hf_Bert, Adagrad, cuda, maximize: +0.11309%
- hf_Bert, Adamax, cuda, (pt2) default: +0.08700%
- hf_Bert, ASGD, cuda, (pt2) maximize: +0.07520%
- pytorch_struct, Adadelta, cuda, (pt2) maximize: +0.07470%
- pytorch_struct, Adagrad, cuda, (pt2) foreach: -0.28023%
- pytorch_struct, Adam, cuda, (pt2) fused, amsgrad, maximize: -0.07317%
- pytorch_struct, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.14700%
- pytorch_struct, Adamax, cuda, (pt2) default: +0.09748%
- pytorch_struct, Adamax, cuda, (pt2) maximize: -0.11621%
- pytorch_struct, ASGD, cuda, (pt2) default: -0.08689%
- pytorch_struct, ASGD, cuda, foreach: -0.22849%
- pytorch_struct, Rprop, cuda, default: +0.07284%
- detectron2_fasterrcnn_r_50_c4, Adagrad, cpu, (pt2) foreach: -0.23107%
- detectron2_fasterrcnn_r_50_c4, Adagrad, cpu, foreach: -0.32508%
- detectron2_fasterrcnn_r_50_c4, Adam, cpu, (pt2) foreach: -0.29656%
- detectron2_fasterrcnn_r_50_c4, Adam, cpu, foreach: -0.38291%
- detectron2_fasterrcnn_r_50_c4, AdamW, cpu, (pt2) foreach: -0.38275%
- detectron2_fasterrcnn_r_50_c4, AdamW, cpu, foreach: -0.37819%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, (pt2) foreach: -0.26525%
- detectron2_fasterrcnn_r_50_c4, Adamax, cpu, foreach: -0.15213%
- detectron2_fasterrcnn_r_50_c4, RAdam, cpu, (pt2) foreach: -0.13440%
- detectron2_fasterrcnn_r_50_c4, NAdam, cpu, (pt2) foreach: -0.07630%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) default: -0.24062%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, default: +0.10266%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, (pt2) foreach: -0.08263%
- detectron2_fasterrcnn_r_101_c4, Adagrad, cuda, foreach: -0.29998%
- detectron2_fasterrcnn_r_101_c4, Adam, cuda, foreach, maximize, capturable: -0.09530%
- detectron2_fasterrcnn_r_101_c4, Adam, cuda, (pt2) fused, capturable, amsgrad: +0.12073%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) default: -0.10129%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, default: -0.18977%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, maximize: -0.09886%
- detectron2_fasterrcnn_r_101_c4, Adamax, cuda, (pt2) foreach: -0.24375%
- detectron2_fasterrcnn_r_101_c4, Rprop, cuda, default: -0.20595%
- detectron2_fasterrcnn_r_101_c4, Rprop, cuda, (pt2) maximize: -0.30362%
- detectron2_fasterrcnn_r_101_c4, Rprop, cuda, (pt2) foreach: +0.17628%
- functorch_maml_omniglot, Adam, cuda, (pt2) foreach: -0.07925%
- functorch_maml_omniglot, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.16141%
- functorch_maml_omniglot, AdamW, cuda, (pt2) fused, capturable, amsgrad: -0.07387%
- functorch_maml_omniglot, ASGD, cuda, maximize: -0.07972%
- functorch_maml_omniglot, SGD, cuda, maximize: +0.09757%
- functorch_maml_omniglot, SGD, cuda, (pt2) foreach: -0.22086%
- functorch_maml_omniglot, Rprop, cuda, (pt2) default: -0.22284%
- functorch_maml_omniglot, Rprop, cuda, (pt2) maximize: -0.11831%
- demucs, SGD, cuda, (pt2) foreach, momentum=0.9: +0.09306%
- resnet18, Adadelta, cuda, (pt2) default: -0.10071%
- resnet18, Adagrad, cuda, (pt2) default: -0.11706%
- resnet18, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: -0.09362%
- resnet18, SGD, cuda, (pt2) foreach, momentum=0.9: -0.13854%
- resnet18, NAdam, cuda, default: +0.08065%
- resnet18, NAdam, cuda, (pt2) foreach: -0.13448%
- phlippe_densenet, Adamax, cuda, maximize: +0.07518%
- phlippe_densenet, NAdam, cuda, (pt2) default: -0.14765%
- hf_BigBird, Adagrad, cuda, (pt2) default: -0.10586%
- hf_BigBird, Adamax, cuda, (pt2) maximize: -0.19435%
- hf_BigBird, ASGD, cuda, (pt2) maximize: -0.13429%
- lennard_jones, Adadelta, cuda, (pt2) default: +0.07243%
- lennard_jones, Adadelta, cuda, default: -0.13089%
- lennard_jones, Adagrad, cuda, (pt2) foreach: -0.23911%
- lennard_jones, Adam, cuda, (pt2) foreach, maximize, capturable: -0.21785%
- lennard_jones, Adam, cuda, (pt2) fused, capturable, amsgrad: -0.07395%
- lennard_jones, Adam, cuda, fused, capturable, amsgrad: -0.08665%
- lennard_jones, AdamW, cuda, amsgrad, maximize: -0.07058%
- lennard_jones, Adamax, cuda, (pt2) maximize: -0.09468%
- lennard_jones, Adamax, cuda, (pt2) foreach: -0.07804%
- lennard_jones, ASGD, cuda, (pt2) maximize: -0.24284%
- lennard_jones, ASGD, cuda, foreach: -0.21660%
- lennard_jones, SGD, cuda, default: -0.08356%
- lennard_jones, SGD, cuda, (pt2) maximize: +0.15687%
- lennard_jones, SGD, cuda, maximize: -0.13344%
- lennard_jones, SGD, cuda, (pt2) foreach: +0.27803%
- lennard_jones, SGD, cuda, foreach, momentum=0.9: -0.09402%
- lennard_jones, Rprop, cuda, maximize: +0.08893%
- lennard_jones, Rprop, cuda, foreach: -0.30148%
- lennard_jones, NAdam, cuda, (pt2) default: -0.12714%
- mobilenet_v3_large, Adagrad, cuda, (pt2) foreach: +0.50834%
- mobilenet_v3_large, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.07856%
- mobilenet_v3_large, Adamax, cuda, (pt2) maximize: +0.17776%
- mobilenet_v3_large, Rprop, cuda, default: +0.07412%
- functorch_dp_cifar10, Adadelta, cpu, foreach: -0.14650%
- functorch_dp_cifar10, Adam, cpu, (pt2) foreach: -0.14254%
- functorch_dp_cifar10, AdamW, cpu, (pt2) foreach: -0.19607%
- functorch_dp_cifar10, Adamax, cpu, foreach: -0.30257%
- functorch_dp_cifar10, ASGD, cpu, foreach: +0.14242%
- functorch_dp_cifar10, NAdam, cpu, (pt2) foreach: -0.18113%
- LearningToPaint, Adagrad, cpu, foreach: -0.09663%
- LearningToPaint, SGD, cpu, (pt2) foreach, momentum=0.9: +0.32248%
- LearningToPaint, RAdam, cpu, (pt2) foreach: +0.10528%
- LearningToPaint, RAdam, cpu, foreach: +0.11546%
- LearningToPaint, Rprop, cpu, (pt2) foreach: -0.11755%
- drq, Adagrad, cpu, (pt2) foreach: +0.13011%
- drq, Adagrad, cpu, foreach: -0.28595%
- drq, SGD, cpu, (pt2) foreach: -0.08146%
- drq, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: +0.10085%
- drq, RAdam, cpu, (pt2) foreach: -0.11308%
- drq, RAdam, cpu, foreach: -0.19103%
- drq, RMSprop, cpu, (pt2) foreach: +0.12233%
- hf_T5_large, Adam, cuda, no_foreach: +0.07585%
- hf_T5_large, AdamW, cuda, differentiable: +0.08096%
- hf_T5_large, AdamW, cuda, foreach, maximize, capturable: -0.07410%
- hf_T5_large, ASGD, cuda, (pt2) maximize: -0.15252%
- hf_T5_large, ASGD, cuda, foreach: -0.20634%
- hf_T5_large, RAdam, cuda, (pt2) no_foreach: +0.40462%
- hf_T5_large, Rprop, cuda, default: -0.20632%
- detectron2_fasterrcnn_r_101_fpn, Adagrad, cuda, default: -0.09662%
- detectron2_fasterrcnn_r_101_fpn, Adagrad, cuda, foreach: -0.24363%
- detectron2_fasterrcnn_r_101_fpn, Adam, cuda, (pt2) amsgrad, maximize: +0.07187%
- detectron2_fasterrcnn_r_101_fpn, Adam, cuda, (pt2) foreach, maximize, capturable: -0.07591%
- detectron2_fasterrcnn_r_101_fpn, Adamax, cuda, (pt2) maximize: +0.07878%
- detectron2_fasterrcnn_r_101_fpn, ASGD, cuda, (pt2) default: -0.08243%
- detectron2_fasterrcnn_r_101_fpn, ASGD, cuda, default: -0.19042%
- detectron2_fasterrcnn_r_101_fpn, ASGD, cuda, maximize: +0.34598%
- detectron2_fasterrcnn_r_101_fpn, RAdam, cuda, (pt2) default: -0.09211%
- detectron2_fasterrcnn_r_101_fpn, Rprop, cuda, maximize: +0.13993%
- detectron2_maskrcnn_r_50_fpn, Adagrad, cuda, default: +0.12575%
- detectron2_maskrcnn_r_50_fpn, Adam, cuda, foreach, maximize, capturable: -0.18982%
- detectron2_maskrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: -0.10193%
- detectron2_maskrcnn_r_50_fpn, Adamax, cuda, (pt2) default: -0.10157%
- detectron2_maskrcnn_r_50_fpn, Adamax, cuda, default: +0.17817%
- detectron2_maskrcnn_r_50_fpn, Adamax, cuda, (pt2) maximize: +0.09903%
- detectron2_maskrcnn_r_50_fpn, ASGD, cuda, (pt2) maximize: +0.13398%
- detectron2_maskrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +2.06285%
- detectron2_maskrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9: -0.24748%
- detectron2_maskrcnn_r_50_fpn, Rprop, cuda, (pt2) maximize: -0.18237%
- functorch_maml_omniglot, Adagrad, cpu, (pt2) foreach: +0.07046%
- functorch_maml_omniglot, Adam, cpu, foreach: -0.12303%
- functorch_maml_omniglot, AdamW, cpu, (pt2) foreach: -0.11770%
- functorch_maml_omniglot, SGD, cpu, (pt2) foreach, momentum=0.9: +0.12543%
- functorch_maml_omniglot, NAdam, cpu, foreach: -0.11169%
- hf_T5, Adamax, cuda, (pt2) maximize: +0.08638%
- doctr_det_predictor, Adamax, cuda, maximize: +0.31354%
- doctr_det_predictor, ASGD, cuda, default: -0.32215%
- doctr_det_predictor, ASGD, cuda, (pt2) maximize: +0.46893%
- doctr_det_predictor, Rprop, cuda, maximize: +0.21048%
- resnet152, Adagrad, cuda, default: +0.10396%
- resnet152, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: +0.12891%
- resnet152, AdamW, cuda, foreach: -0.10779%
- resnet152, ASGD, cuda, foreach: +0.11280%
- resnet152, NAdam, cuda, (pt2) foreach: -0.10668%
- detectron2_fasterrcnn_r_50_fpn, Adagrad, cuda, (pt2) default: -0.20808%
- detectron2_fasterrcnn_r_50_fpn, AdamW, cuda, foreach, maximize, capturable: -0.09946%
- detectron2_fasterrcnn_r_50_fpn, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: +0.07332%
- detectron2_fasterrcnn_r_50_fpn, Rprop, cuda, (pt2) default: -0.18632%
- detectron2_fasterrcnn_r_50_fpn, Rprop, cuda, maximize: +0.17668%
- fambench_xlmr, AdamW, cuda, foreach: +0.09086%
- fambench_xlmr, Rprop, cuda, (pt2) foreach: -0.10385%
- lennard_jones, SGD, cpu, foreach, momentum=0.9, nesterov: -0.14983%
- lennard_jones, Rprop, cpu, foreach: -0.08062%
- lennard_jones, RMSprop, cpu, (pt2) foreach: -0.10897%
- lennard_jones, RMSprop, cpu, foreach: -0.11842%
- detectron2_maskrcnn, Adagrad, cuda, (pt2) maximize: +0.22335%
- detectron2_maskrcnn, Adagrad, cuda, maximize: -0.17067%
- detectron2_maskrcnn, Adagrad, cuda, (pt2) foreach: -0.11268%
- detectron2_maskrcnn, Adamax, cuda, default: -0.14802%
- detectron2_maskrcnn, Adamax, cuda, (pt2) maximize: -0.24230%
- detectron2_maskrcnn, Adamax, cuda, maximize: -0.08164%
- detectron2_maskrcnn, ASGD, cuda, default: -0.27037%
- detectron2_maskrcnn, SGD, cuda, (pt2) foreach, momentum=0.9: +2.03219%
- detectron2_maskrcnn, Rprop, cuda, maximize: -0.10130%
- timm_vision_transformer, Adagrad, cuda, (pt2) foreach: -0.08495%
- timm_vision_transformer, Adagrad, cuda, foreach: -0.25242%
- timm_vision_transformer, Adam, cuda, (pt2) default: -0.08780%
- timm_vision_transformer, Adamax, cuda, maximize: +0.07054%
- timm_vision_transformer, ASGD, cuda, differentiable: -0.09218%
- timm_vision_transformer, RAdam, cuda, no_foreach: +0.07300%
- timm_vision_transformer, RAdam, cuda, (pt2) foreach: -0.07917%
- timm_vision_transformer, Rprop, cuda, (pt2) maximize: -0.09099%
- timm_vision_transformer, Rprop, cuda, (pt2) no_foreach: -0.07329%
- timm_vision_transformer, Rprop, cuda, no_foreach: +0.13891%
- timm_vision_transformer, RMSprop, cuda, (pt2) differentiable: +0.07927%
- maml, SGD, cpu, (pt2) foreach: -0.07175%
- maml, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: -0.14321%
- maml, RAdam, cpu, foreach: -0.19814%
- timm_resnest, Adadelta, cpu, foreach: -0.19037%
- timm_resnest, Adagrad, cpu, (pt2) foreach: +0.12342%
- timm_resnest, Adagrad, cpu, foreach: +0.10901%
- detectron2_maskrcnn_r_50_c4, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.12497%
- detectron2_maskrcnn_r_50_c4, AdamW, cuda, foreach, maximize, capturable: +0.16289%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) default: -0.10427%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, (pt2) foreach: -0.11998%
- detectron2_maskrcnn_r_50_c4, Adamax, cuda, foreach: -0.24910%
- detectron2_maskrcnn_r_50_c4, ASGD, cuda, foreach: -0.07816%
- detectron2_maskrcnn_r_50_c4, SGD, cuda, (pt2) foreach, momentum=0.9: -0.10886%
- detectron2_maskrcnn_r_50_c4, RAdam, cuda, (pt2) default: -0.07042%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, (pt2) default: +0.07028%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, default: -0.09886%
- detectron2_maskrcnn_r_50_c4, Rprop, cuda, (pt2) maximize: -0.22111%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cuda, default: -0.28904%
- pytorch_CycleGAN_and_pix2pix, Adagrad, cuda, (pt2) foreach: +0.07041%
- pytorch_CycleGAN_and_pix2pix, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.15113%
- pytorch_CycleGAN_and_pix2pix, Adamax, cuda, (pt2) default: -0.08777%
- pytorch_CycleGAN_and_pix2pix, Adamax, cuda, (pt2) maximize: -0.08582%
- pytorch_CycleGAN_and_pix2pix, Adamax, cuda, foreach: -0.09408%
- pytorch_CycleGAN_and_pix2pix, ASGD, cuda, default: -0.18071%
- pytorch_CycleGAN_and_pix2pix, Rprop, cuda, (pt2) foreach: +0.30668%
- pytorch_CycleGAN_and_pix2pix, NAdam, cuda, (pt2) default: -0.07738%
- detectron2_fasterrcnn_r_50_dc5, Rprop, cuda, (pt2) default: -0.09671%
- Super_SloMo, Adagrad, cuda, (pt2) foreach: +0.08030%
- Super_SloMo, Adam, cuda, (pt2) foreach, maximize, capturable: +0.15756%
- Super_SloMo, Adamax, cuda, (pt2) maximize: +0.07019%
- Super_SloMo, ASGD, cuda, maximize: -0.11047%
- Super_SloMo, ASGD, cuda, (pt2) foreach: -0.31062%
- Super_SloMo, RAdam, cuda, (pt2) foreach: -0.12465%
- Super_SloMo, Rprop, cuda, default: -0.19846%
- Super_SloMo, Rprop, cuda, (pt2) foreach: +0.09200%
- Super_SloMo, Rprop, cuda, foreach: -0.08853%
- resnext50_32x4d, Adam, cuda, foreach, maximize, capturable, amsgrad: -0.07750%
- resnext50_32x4d, ASGD, cuda, (pt2) foreach: +0.07079%
- resnext50_32x4d, Rprop, cuda, (pt2) foreach: +0.10844%
- pytorch_unet, Adam, cuda, default: -0.09226%
- pytorch_unet, Adam, cuda, foreach, maximize, capturable: -0.11738%
- pytorch_unet, AdamW, cuda, (pt2) default: -0.11324%
- pytorch_unet, AdamW, cuda, default: -0.08699%
- pytorch_unet, AdamW, cuda, foreach: -0.08347%
- pytorch_unet, AdamW, cuda, (pt2) foreach, maximize, capturable: -0.13312%
- pytorch_unet, AdamW, cuda, foreach, maximize, capturable: -0.13652%
- pytorch_unet, AdamW, cuda, foreach, maximize, capturable, amsgrad: -0.07453%
- pytorch_unet, AdamW, cuda, (pt2) fused: -0.08968%
- pytorch_unet, Adamax, cuda, (pt2) default: -0.23320%
- pytorch_unet, Adamax, cuda, (pt2) maximize: -0.08038%
- pytorch_unet, RAdam, cuda, default: -0.08884%
- pytorch_unet, RAdam, cuda, (pt2) foreach: +0.16687%
- pytorch_unet, Rprop, cuda, foreach: -0.10774%
- pytorch_unet, NAdam, cuda, default: -0.07819%
- detectron2_fasterrcnn_r_50_c4, AdamW, cuda, (pt2) fused: +0.07577%
- detectron2_fasterrcnn_r_50_c4, ASGD, cuda, default: +0.22219%
- detectron2_fasterrcnn_r_50_c4, ASGD, cuda, (pt2) foreach: -0.13077%
- detectron2_fasterrcnn_r_50_c4, NAdam, cuda, (pt2) foreach: +0.08174%
- hf_Reformer, Adam, cuda, amsgrad, maximize: -0.07270%
- hf_Reformer, Adam, cuda, (pt2) foreach, maximize, capturable: -0.10446%
- hf_Reformer, AdamW, cuda, (pt2) amsgrad, maximize: +0.10106%
- hf_Reformer, SGD, cuda, (pt2) foreach, momentum=0.9: +0.10608%
- hf_Reformer, Rprop, cuda, (pt2) maximize: +0.12019%
- llama, Adagrad, cuda, maximize: -0.17159%
- llama, Adagrad, cuda, (pt2) foreach: -0.08134%
- llama, AdamW, cuda, (pt2) default: -0.07158%
- llama, AdamW, cuda, (pt2) foreach: +0.07999%
- llama, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.08967%
- llama, Adamax, cuda, (pt2) maximize: -0.17607%
- llama, ASGD, cuda, maximize: -0.08083%
- llama, SGD, cuda, foreach, momentum=0.9, nesterov: -0.09965%
- llama, SGD, cuda, (pt2) foreach, momentum=0.9: -0.53672%
- llama, RAdam, cuda, (pt2) default: -0.16616%
- llama, RAdam, cuda, default: -0.19170%
- llama, RAdam, cuda, (pt2) foreach: -0.17758%
- llama, Rprop, cuda, foreach: -0.07416%
- maml_omniglot, Adadelta, cuda, foreach: +0.25667%
- maml_omniglot, Adagrad, cuda, maximize: +0.07967%
- maml_omniglot, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: -0.16680%
- maml_omniglot, AdamW, cuda, (pt2) foreach: -0.13247%
- maml_omniglot, AdamW, cuda, foreach, maximize, capturable: +0.08841%
- maml_omniglot, Adamax, cuda, (pt2) default: -0.23723%
- maml_omniglot, Adamax, cuda, (pt2) maximize: -0.09528%
- maml_omniglot, ASGD, cuda, (pt2) foreach: -0.08329%
- maml_omniglot, RAdam, cuda, (pt2) foreach: -0.07476%
- maml_omniglot, RAdam, cuda, foreach: -0.12825%
- maml_omniglot, Rprop, cuda, (pt2) default: -0.29861%
- hf_Bart, Adagrad, cuda, (pt2) foreach: -0.07211%
- hf_Bart, Adamax, cuda, maximize: -0.09175%
- pytorch_stargan, Adadelta, cpu, (pt2) foreach: +0.59697%
- pytorch_stargan, Adadelta, cpu, foreach: +0.49318%
- pytorch_stargan, Adagrad, cpu, (pt2) foreach: +0.86565%
- pytorch_stargan, Adagrad, cpu, foreach: +0.54715%
- pytorch_stargan, Adam, cpu, (pt2) foreach: +0.44522%
- pytorch_stargan, Adam, cpu, foreach: +0.44521%
- pytorch_stargan, AdamW, cpu, (pt2) foreach: +0.36227%
- pytorch_stargan, AdamW, cpu, foreach: +0.40802%
- pytorch_stargan, Adamax, cpu, (pt2) foreach: +0.07389%
- pytorch_stargan, ASGD, cpu, (pt2) foreach: +0.09997%
- pytorch_stargan, SGD, cpu, foreach: +0.08970%
- pytorch_stargan, SGD, cpu, foreach, momentum=0.9, nesterov: +0.09446%
- pytorch_stargan, SGD, cpu, foreach, momentum=0.9: +0.07264%
- pytorch_stargan, RAdam, cpu, (pt2) foreach: +0.20167%
- pytorch_stargan, Rprop, cpu, foreach: -0.12774%
- timm_regnet, AdamW, cuda, (pt2) foreach, maximize, capturable: +0.09061%
- timm_regnet, ASGD, cuda, foreach: -0.08720%


Tests that were no longer run on affected commit:


Tests that were newly added on affected commit:
- hf_T5_generate, Adadelta, cuda, (pt2) default: 0.0059771603863576755
- hf_T5_generate, Adadelta, cuda, default: 0.005199751898180694
- hf_T5_generate, Adadelta, cuda, (pt2) maximize: 0.006608614067156468
- hf_T5_generate, Adadelta, cuda, maximize: 0.005600866600871086
- hf_T5_generate, Adadelta, cuda, (pt2) foreach: 0.0059821307179791965
- hf_T5_generate, Adadelta, cuda, foreach: 0.00521376928081736
- hf_T5_generate, Adagrad, cuda, (pt2) default: 0.008198375161737204
- hf_T5_generate, Adagrad, cuda, default: 0.007180200631264597
- hf_T5_generate, Adagrad, cuda, (pt2) maximize: 0.008725544923673506
- hf_T5_generate, Adagrad, cuda, maximize: 0.007560695291031152
- hf_T5_generate, Adagrad, cuda, (pt2) foreach: 0.00817479749210179
- hf_T5_generate, Adagrad, cuda, foreach: 0.0073432485316880045
- hf_T5_generate, Adam, cuda, (pt2) default: 0.0054385213156205574
- hf_T5_generate, Adam, cuda, default: 0.003907945270184427
- hf_T5_generate, Adam, cuda, (pt2) amsgrad, maximize: 0.006373957403411623
- hf_T5_generate, Adam, cuda, amsgrad, maximize: 0.004776790491305292
- hf_T5_generate, Adam, cuda, (pt2) foreach: 0.0054272341705556655
- hf_T5_generate, Adam, cuda, foreach: 0.0039033649791963398
- hf_T5_generate, Adam, cuda, (pt2) foreach, maximize, capturable: 0.0065209609531467
- hf_T5_generate, Adam, cuda, foreach, maximize, capturable: 0.0060002644802443685
- hf_T5_generate, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: 0.007092214775560745
- hf_T5_generate, Adam, cuda, foreach, maximize, capturable, amsgrad: 0.00609175181016326
- hf_T5_generate, Adam, cuda, (pt2) fused: 0.002871593926101923
- hf_T5_generate, Adam, cuda, fused: 0.001942586904624477
- hf_T5_generate, Adam, cuda, (pt2) fused, amsgrad, maximize: 0.0032519630359245403
- hf_T5_generate, Adam, cuda, fused, amsgrad, maximize: 0.002118860960472375
- hf_T5_generate, Adam, cuda, (pt2) fused, capturable: 0.0029634356854812186
- hf_T5_generate, Adam, cuda, fused, capturable: 0.0019430451444350182
- hf_T5_generate, Adam, cuda, (pt2) fused, capturable, amsgrad: 0.003094741479995159
- hf_T5_generate, Adam, cuda, fused, capturable, amsgrad: 0.0021139352302998304
- hf_T5_generate, AdamW, cuda, (pt2) default: 0.005867924960330129
- hf_T5_generate, AdamW, cuda, default: 0.004298587669618428
- hf_T5_generate, AdamW, cuda, (pt2) amsgrad, maximize: 0.006682582005547981
- hf_T5_generate, AdamW, cuda, amsgrad, maximize: 0.005168429501354694
- hf_T5_generate, AdamW, cuda, (pt2) foreach: 0.0058122594175594195
- hf_T5_generate, AdamW, cuda, foreach: 0.004293981741648167
- hf_T5_generate, AdamW, cuda, (pt2) foreach, maximize, capturable: 0.006973314323815806
- hf_T5_generate, AdamW, cuda, foreach, maximize, capturable: 0.005954519030638039
- hf_T5_generate, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: 0.0074674800745453
- hf_T5_generate, AdamW, cuda, foreach, maximize, capturable, amsgrad: 0.006262936040293425
- hf_T5_generate, AdamW, cuda, (pt2) fused: 0.0029113244170835915
- hf_T5_generate, AdamW, cuda, fused: 0.0019993494555819776
- hf_T5_generate, AdamW, cuda, (pt2) fused, amsgrad, maximize: 0.0031259044480975717
- hf_T5_generate, AdamW, cuda, fused, amsgrad, maximize: 0.0021575889899395407
- hf_T5_generate, AdamW, cuda, (pt2) fused, capturable: 0.002910820210514509
- hf_T5_generate, AdamW, cuda, fused, capturable: 0.0019980237109120938
- hf_T5_generate, AdamW, cuda, (pt2) fused, capturable, amsgrad: 0.0031329553021350875
- hf_T5_generate, AdamW, cuda, fused, capturable, amsgrad: 0.002153322980739176
- hf_T5_generate, Adamax, cuda, (pt2) default: 0.011645651629401578
- hf_T5_generate, Adamax, cuda, default: 0.010342487238813192
- hf_T5_generate, Adamax, cuda, (pt2) maximize: 0.011847498155582477
- hf_T5_generate, Adamax, cuda, maximize: 0.010742007847875358
- hf_T5_generate, Adamax, cuda, (pt2) foreach: 0.01156908104247931
- hf_T5_generate, Adamax, cuda, foreach: 0.010565630509518087
- hf_T5_generate, ASGD, cuda, (pt2) default: 0.004981139407879333
- hf_T5_generate, ASGD, cuda, default: 0.004677958958782255
- hf_T5_generate, ASGD, cuda, (pt2) maximize: 0.0058457514670278345
- hf_T5_generate, ASGD, cuda, maximize: 0.005382091980427503
- hf_T5_generate, ASGD, cuda, (pt2) foreach: 0.005028411379316822
- hf_T5_generate, ASGD, cuda, foreach: 0.004752428298816085
- hf_T5_generate, SGD, cuda, (pt2) default: 0.0012545255740405992
- hf_T5_generate, SGD, cuda, default: 0.00056339639890939
- hf_T5_generate, SGD, cuda, (pt2) maximize: 0.0018476427537004608
- hf_T5_generate, SGD, cuda, maximize: 0.0009758445294573903
- hf_T5_generate, SGD, cuda, (pt2) foreach: 0.0012346858670360145
- hf_T5_generate, SGD, cuda, foreach: 0.0005639591591898352
- hf_T5_generate, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: 0.27764953090809286
- hf_T5_generate, SGD, cuda, foreach, momentum=0.9, nesterov: 0.0020786976092495024
- hf_T5_generate, SGD, cuda, (pt2) foreach, momentum=0.9: 0.2702099420130253
- hf_T5_generate, SGD, cuda, foreach, momentum=0.9: 0.001521949424641207
- hf_T5_generate, RAdam, cuda, (pt2) default: 0.005092067250516265
- hf_T5_generate, RAdam, cuda, default: 0.005095761679112912
- hf_T5_generate, RAdam, cuda, (pt2) foreach: 0.005094433680642396
- hf_T5_generate, RAdam, cuda, foreach: 0.005232898721005768
- hf_T5_generate, Rprop, cuda, (pt2) default: 0.018801312004639345
- hf_T5_generate, Rprop, cuda, default: 0.018167121591977774
- hf_T5_generate, Rprop, cuda, (pt2) maximize: 0.019753072897649625
- hf_T5_generate, Rprop, cuda, maximize: 0.01846919379895553
- hf_T5_generate, Rprop, cuda, (pt2) foreach: 0.018780338069932026
- hf_T5_generate, Rprop, cuda, foreach: 0.01854892675764859
- hf_T5_generate, RMSprop, cuda, (pt2) default: 0.003378838715919604
- hf_T5_generate, RMSprop, cuda, default: 0.0024852001992985606
- hf_T5_generate, RMSprop, cuda, (pt2) maximize: 0.003931158660527538
- hf_T5_generate, RMSprop, cuda, maximize: 0.0028662289190106095
- hf_T5_generate, RMSprop, cuda, (pt2) foreach: 0.003353249557161083
- hf_T5_generate, RMSprop, cuda, foreach: 0.0024833995010703803
- hf_T5_generate, NAdam, cuda, (pt2) default: 0.00689453532298406
- hf_T5_generate, NAdam, cuda, default: 0.004648905110079795
- hf_T5_generate, NAdam, cuda, (pt2) foreach: 0.0068998402867723125
- hf_T5_generate, NAdam, cuda, foreach: 0.004649616191163659
- cm3leon_generate, Adadelta, cpu, (pt2) foreach: 6.70922752097249
- cm3leon_generate, Adadelta, cpu, foreach: 5.8527601030655205
- cm3leon_generate, Adagrad, cpu, (pt2) foreach: 6.097032067831606
- cm3leon_generate, Adagrad, cpu, foreach: 6.0793951149098575
- cm3leon_generate, Adam, cpu, (pt2) foreach: 4.848062047036365
- cm3leon_generate, Adam, cpu, foreach: 4.879589121090248
- cm3leon_generate, AdamW, cpu, (pt2) foreach: 5.119989216094837
- cm3leon_generate, AdamW, cpu, foreach: 5.036513285012916
- cm3leon_generate, Adamax, cpu, (pt2) foreach: 10.624467737972736
- cm3leon_generate, Adamax, cpu, foreach: 10.771854110062122
- cm3leon_generate, ASGD, cpu, (pt2) foreach: 1.2016147209797055
- cm3leon_generate, ASGD, cpu, foreach: 1.1713996059261262
- cm3leon_generate, SGD, cpu, (pt2) foreach: 0.4604991520754993
- cm3leon_generate, SGD, cpu, foreach: 0.440719025908038
- cm3leon_generate, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: 2.367620495846495
- cm3leon_generate, SGD, cpu, foreach, momentum=0.9, nesterov: 1.5918582121375948
- cm3leon_generate, SGD, cpu, (pt2) foreach, momentum=0.9: 1.9096272508613765
- cm3leon_generate, SGD, cpu, foreach, momentum=0.9: 1.1279165961313993
- cm3leon_generate, RAdam, cpu, (pt2) foreach: 6.491542018018663
- cm3leon_generate, RAdam, cpu, foreach: 6.538331978954375
- cm3leon_generate, Rprop, cpu, (pt2) foreach: 11.753072314197198
- cm3leon_generate, Rprop, cpu, foreach: 11.747695622965693
- cm3leon_generate, RMSprop, cpu, (pt2) foreach: 2.9103768481872976
- cm3leon_generate, RMSprop, cpu, foreach: 2.87562652095221
- cm3leon_generate, NAdam, cpu, (pt2) foreach: 5.510295504005626
- cm3leon_generate, NAdam, cpu, foreach: 5.4399669270496815
- hf_T5_generate, Adadelta, cpu, (pt2) foreach: 0.6139866940211505
- hf_T5_generate, Adadelta, cpu, foreach: 0.5986704328097403
- hf_T5_generate, Adagrad, cpu, (pt2) foreach: 0.41571910795755684
- hf_T5_generate, Adagrad, cpu, foreach: 0.40235046693123877
- hf_T5_generate, Adam, cpu, (pt2) foreach: 0.31662965193390846
- hf_T5_generate, Adam, cpu, foreach: 0.29548695706762373
- hf_T5_generate, AdamW, cpu, (pt2) foreach: 0.3363433030899614
- hf_T5_generate, AdamW, cpu, foreach: 0.3176669089589268
- hf_T5_generate, Adamax, cpu, (pt2) foreach: 0.574327307054773
- hf_T5_generate, Adamax, cpu, foreach: 0.5963271630462259
- hf_T5_generate, ASGD, cpu, (pt2) foreach: 0.10599827242549509
- hf_T5_generate, ASGD, cpu, foreach: 0.10817533906083554
- hf_T5_generate, SGD, cpu, (pt2) foreach: 0.04061633022502065
- hf_T5_generate, SGD, cpu, foreach: 0.04010558379814029
- hf_T5_generate, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: 0.40101737016811967
- hf_T5_generate, SGD, cpu, foreach, momentum=0.9, nesterov: 0.13916546304244548
- hf_T5_generate, SGD, cpu, (pt2) foreach, momentum=0.9: 0.3650777859147638
- hf_T5_generate, SGD, cpu, foreach, momentum=0.9: 0.10420760300476103
- hf_T5_generate, RAdam, cpu, (pt2) foreach: 0.47251928178593516
- hf_T5_generate, RAdam, cpu, foreach: 0.48083732509985566
- hf_T5_generate, Rprop, cpu, (pt2) foreach: 0.8798855508212
- hf_T5_generate, Rprop, cpu, foreach: 0.8657272502314299
- hf_T5_generate, RMSprop, cpu, (pt2) foreach: 0.18631728505715728
- hf_T5_generate, RMSprop, cpu, foreach: 0.18661551643162966
- hf_T5_generate, NAdam, cpu, (pt2) foreach: 0.3585197099018842
- hf_T5_generate, NAdam, cpu, foreach: 0.34443254000507295
- cm3leon_generate, Adadelta, cuda, (pt2) default: 0.06122355826664716
- cm3leon_generate, Adadelta, cuda, default: 0.05917098908685148
- cm3leon_generate, Adadelta, cuda, (pt2) maximize: 0.06607916316716
- cm3leon_generate, Adadelta, cuda, maximize: 0.0635202701902017
- cm3leon_generate, Adadelta, cuda, (pt2) foreach: 0.061004653573036194
- cm3leon_generate, Adadelta, cuda, foreach: 0.059138718410395086
- cm3leon_generate, Adagrad, cuda, (pt2) default: 0.046911648381501435
- cm3leon_generate, Adagrad, cuda, default: 0.03470927739981562
- cm3leon_generate, Adagrad, cuda, (pt2) maximize: 0.04773245556280017
- cm3leon_generate, Adagrad, cuda, maximize: 0.03870098490733653
- cm3leon_generate, Adagrad, cuda, (pt2) foreach: 0.04671262498013675
- cm3leon_generate, Adagrad, cuda, foreach: 0.03467015530914068
- cm3leon_generate, Adam, cuda, (pt2) default: 0.04782527475617826
- cm3leon_generate, Adam, cuda, default: 0.04395211860537529
- cm3leon_generate, Adam, cuda, (pt2) amsgrad, maximize: 0.05893437727354467
- cm3leon_generate, Adam, cuda, amsgrad, maximize: 0.05471144870389253
- cm3leon_generate, Adam, cuda, (pt2) foreach: 0.04761553104035556
- cm3leon_generate, Adam, cuda, foreach: 0.04393647271208465
- cm3leon_generate, Adam, cuda, (pt2) foreach, maximize, capturable: 0.0540198179660365
- cm3leon_generate, Adam, cuda, foreach, maximize, capturable: 0.050863431906327604
- cm3leon_generate, Adam, cuda, (pt2) foreach, maximize, capturable, amsgrad: 0.060204698238521814
- cm3leon_generate, Adam, cuda, foreach, maximize, capturable, amsgrad: 0.05741145990323275
- cm3leon_generate, Adam, cuda, (pt2) fused: 0.020244279294274747
- cm3leon_generate, Adam, cuda, fused: 0.01820675723720342
- cm3leon_generate, Adam, cuda, (pt2) fused, amsgrad, maximize: 0.02274810683189167
- cm3leon_generate, Adam, cuda, fused, amsgrad, maximize: 0.020451358007267117
- cm3leon_generate, Adam, cuda, (pt2) fused, capturable: 0.020198845025151967
- cm3leon_generate, Adam, cuda, fused, capturable: 0.018226330657489596
- cm3leon_generate, Adam, cuda, (pt2) fused, capturable, amsgrad: 0.022818813980039623
- cm3leon_generate, Adam, cuda, fused, capturable, amsgrad: 0.02051610071212053
- cm3leon_generate, AdamW, cuda, (pt2) default: 0.052261861448641866
- cm3leon_generate, AdamW, cuda, default: 0.048368645296432075
- cm3leon_generate, AdamW, cuda, (pt2) amsgrad, maximize: 0.06422440451569855
- cm3leon_generate, AdamW, cuda, amsgrad, maximize: 0.0591373325092718
- cm3leon_generate, AdamW, cuda, (pt2) foreach: 0.05248314654454589
- cm3leon_generate, AdamW, cuda, foreach: 0.048385386285372076
- cm3leon_generate, AdamW, cuda, (pt2) foreach, maximize, capturable: 0.0579173457226716
- cm3leon_generate, AdamW, cuda, foreach, maximize, capturable: 0.05530883730389178
- cm3leon_generate, AdamW, cuda, (pt2) foreach, maximize, capturable, amsgrad: 0.06471462396439165
- cm3leon_generate, AdamW, cuda, foreach, maximize, capturable, amsgrad: 0.06180030421819538
- cm3leon_generate, AdamW, cuda, (pt2) fused: 0.02080733240582049
- cm3leon_generate, AdamW, cuda, fused: 0.01873180289985612
- cm3leon_generate, AdamW, cuda, (pt2) fused, amsgrad, maximize: 0.023059069799880188
- cm3leon_generate, AdamW, cuda, fused, amsgrad, maximize: 0.02071597259491682
- cm3leon_generate, AdamW, cuda, (pt2) fused, capturable: 0.020660014497116207
- cm3leon_generate, AdamW, cuda, fused, capturable: 0.018715125089511273
- cm3leon_generate, AdamW, cuda, (pt2) fused, capturable, amsgrad: 0.02318422871434854
- cm3leon_generate, AdamW, cuda, fused, capturable, amsgrad: 0.020716638979502024
- cm3leon_generate, Adamax, cuda, (pt2) default: 0.06285892525920644
- cm3leon_generate, Adamax, cuda, default: 0.05931465290486813
- cm3leon_generate, Adamax, cuda, (pt2) maximize: 0.06624052004190162
- cm3leon_generate, Adamax, cuda, maximize: 0.0636475944891572
- cm3leon_generate, Adamax, cuda, (pt2) foreach: 0.06303046143148094
- cm3leon_generate, Adamax, cuda, foreach: 0.05938703410793096
- cm3leon_generate, ASGD, cuda, (pt2) default: 0.020086098229512572
- cm3leon_generate, ASGD, cuda, default: 0.016595750756096095
- cm3leon_generate, ASGD, cuda, (pt2) maximize: 0.023269245789075892
- cm3leon_generate, ASGD, cuda, maximize: 0.02103999729733914
- cm3leon_generate, ASGD, cuda, (pt2) foreach: 0.01989040362902663
- cm3leon_generate, ASGD, cuda, foreach: 0.01658034969586879
- cm3leon_generate, SGD, cuda, (pt2) default: 0.00796551553783222
- cm3leon_generate, SGD, cuda, default: 0.0064520816993899645
- cm3leon_generate, SGD, cuda, (pt2) maximize: 0.013015244345297106
- cm3leon_generate, SGD, cuda, maximize: 0.0110369224101305
- cm3leon_generate, SGD, cuda, (pt2) foreach: 0.007898857226022162
- cm3leon_generate, SGD, cuda, foreach: 0.006456319601275027
- cm3leon_generate, SGD, cuda, (pt2) foreach, momentum=0.9, nesterov: 0.8152376071084291
- cm3leon_generate, SGD, cuda, foreach, momentum=0.9, nesterov: 0.023940638382919134
- cm3leon_generate, SGD, cuda, (pt2) foreach, momentum=0.9: 0.8058823370374739
- cm3leon_generate, SGD, cuda, foreach, momentum=0.9: 0.017493180499877782
- cm3leon_generate, RAdam, cuda, (pt2) default: 0.05512435771524906
- cm3leon_generate, RAdam, cuda, default: 0.05512615330517292
- cm3leon_generate, RAdam, cuda, (pt2) foreach: 0.055086149484850466
- cm3leon_generate, RAdam, cuda, foreach: 0.05513808368705213
- cm3leon_generate, Rprop, cuda, (pt2) default: 0.0830960030046602
- cm3leon_generate, Rprop, cuda, default: 0.08244820800609887
- cm3leon_generate, Rprop, cuda, (pt2) maximize: 0.08726991061121225
- cm3leon_generate, Rprop, cuda, maximize: 0.08692960829163592
- cm3leon_generate, Rprop, cuda, (pt2) foreach: 0.08282202598638833
- cm3leon_generate, Rprop, cuda, foreach: 0.08205860961849491
- cm3leon_generate, RMSprop, cuda, (pt2) default: 0.03053899954206177
- cm3leon_generate, RMSprop, cuda, default: 0.028422524104826152
- cm3leon_generate, RMSprop, cuda, (pt2) maximize: 0.03547747844519714
- cm3leon_generate, RMSprop, cuda, maximize: 0.032863576686941086
- cm3leon_generate, RMSprop, cuda, (pt2) foreach: 0.030371574851285135
- cm3leon_generate, RMSprop, cuda, foreach: 0.028417335893027484
- cm3leon_generate, NAdam, cuda, (pt2) default: 0.05818292801268399
- cm3leon_generate, NAdam, cuda, default: 0.05259429581928998
- cm3leon_generate, NAdam, cuda, (pt2) foreach: 0.05986535252304748
- cm3leon_generate, NAdam, cuda, foreach: 0.05256174600217491


Runtime regressions found?
An errors log was found. Please investigate runtime errors by looking into the logs of the workflow linked.

GitHub workflow that triggered this issue: https://github.com/pytorch/benchmark/actions/runs/4943819025

cc @janeyx99


Observations:
Total: ~694
Speedups: 435 | Slowdowns: 259
(The following include new additions:)
CUDA: 735 | CPU: 193
foreach: 508 | fused: 63 | default: 186 | ...
ASGD: 96 | SGD: 96 | (N/R)Adam(W): 368 | ...
pt2: 528
within .1: 276 | .1-.2: 270 | ...