Repository: pytorch/benchmark
Issue: V2 Performance Signal Detected by TorchBench CI on '84ec5d9b1fec28fa7b3b9a3b229ee8f0f3c1da81'
Root cause: Algorithmic Error


TorchBench CI has detected a performance signal.

Base PyTorch commit: 61007ca987a316f49363186d41b86ba4d220ea48

Affected PyTorch commit: 84ec5d9b1fec28fa7b3b9a3b229ee8f0f3c1da81

Affected Tests:


Tests that were no longer run on affected commit:
- gat, Adadelta, cpu, (pt2) default: 0.0002001912271080073
- gat, Adadelta, cpu, default: 0.0003599970020004548
- gat, Adadelta, cpu, (pt2) maximize: 0.00020762342741650493
- gat, Adadelta, cpu, maximize: 0.0003898487129990826
- gat, Adadelta, cpu, (pt2) no_foreach: 0.00015793582500009507
- gat, Adadelta, cpu, no_foreach: 0.0003516968769981759
- gat, Adadelta, cpu, (pt2) differentiable: 0.00020414097959232727
- gat, Adadelta, cpu, differentiable: 0.00038919045100192307
- gat, Adadelta, cpu, (pt2) foreach: 0.00048502640999868165
- gat, Adadelta, cpu, foreach: 0.0003585724680015119
- gat, Adagrad, cpu, (pt2) default: 0.00025035087737330274
- gat, Adagrad, cpu, default: 0.00020630153449747014
- gat, Adagrad, cpu, (pt2) maximize: 0.0002489047960060911
- gat, Adagrad, cpu, maximize: 0.00023532246600007056
- gat, Adagrad, cpu, (pt2) no_foreach: 0.00020102337000207628
- gat, Adagrad, cpu, no_foreach: 0.00018720225149991164
- gat, Adagrad, cpu, (pt2) differentiable: 0.0002535354017217113
- gat, Adagrad, cpu, differentiable: 0.00020840406299976166
- gat, Adagrad, cpu, (pt2) foreach: 0.0005591161999982432
- gat, Adagrad, cpu, foreach: 0.00043596130900186837
- gat, Adam, cpu, (pt2) default: 0.0005006072539981688
- gat, Adam, cpu, default: 0.000383863544997439
- gat, Adam, cpu, (pt2) amsgrad, maximize: 0.0005743141320017457
- gat, Adam, cpu, amsgrad, maximize: 0.00046464850600023055
- gat, Adam, cpu, (pt2) no_foreach: 0.0004916761050008063
- gat, Adam, cpu, no_foreach: 0.00037836772200171254
- gat, Adam, cpu, (pt2) differentiable: 0.0007730591159997857
- gat, Adam, cpu, differentiable: 0.000643622919000336
- gat, Adam, cpu, (pt2) foreach: 0.0005190455939991807
- gat, Adam, cpu, foreach: 0.0004037317499969504
- gat, AdamW, cpu, (pt2) default: 0.0005459258810005849
- gat, AdamW, cpu, default: 0.0004153586370011908
- gat, AdamW, cpu, (pt2) amsgrad, maximize: 0.0006083095220019459
- gat, AdamW, cpu, amsgrad, maximize: 0.0004913564620001125
- gat, AdamW, cpu, (pt2) no_foreach: 0.0005356212280021282
- gat, AdamW, cpu, no_foreach: 0.000406222046000039
- gat, AdamW, cpu, (pt2) differentiable: 0.0008042562150003505
- gat, AdamW, cpu, differentiable: 0.0006717788060013845
- gat, AdamW, cpu, (pt2) foreach: 0.0005405669360006868
- gat, AdamW, cpu, foreach: 0.0004210902630002238
- gat, Adamax, cpu, (pt2) default: 0.0006660981561317173
- gat, Adamax, cpu, default: 0.00045937907199913755
- gat, Adamax, cpu, (pt2) maximize: 0.0006878888281377286
- gat, Adamax, cpu, maximize: 0.0004908344810028211
- gat, Adamax, cpu, (pt2) no_foreach: 0.0008769891490010195
- gat, Adamax, cpu, no_foreach: 0.00045345251400067353
- gat, Adamax, cpu, (pt2) differentiable: 0.0006697342543465429
- gat, Adamax, cpu, differentiable: 0.0005004572090001602
- gat, Adamax, cpu, (pt2) foreach: 0.0008993510166692432
- gat, Adamax, cpu, foreach: 0.0007537269869972078
- gat, ASGD, cpu, (pt2) default: 0.00033973650932575875
- gat, ASGD, cpu, default: 0.00021641484800056788
- gat, ASGD, cpu, (pt2) maximize: 0.000353124299869354
- gat, ASGD, cpu, maximize: 0.00027018996399783647
- gat, ASGD, cpu, (pt2) no_foreach: 0.0002757745220005745
- gat, ASGD, cpu, no_foreach: 0.00020983836600134965
- gat, ASGD, cpu, (pt2) differentiable: 0.00028461723000145866
- gat, ASGD, cpu, differentiable: 0.00021700736400089227
- gat, ASGD, cpu, (pt2) foreach: 0.00034014170299997206
- gat, ASGD, cpu, foreach: 0.00024548275000051947
- gat, SGD, cpu, (pt2) default: 0.00015042730375629398
- gat, SGD, cpu, default: 3.2336288599981346e-05
- gat, SGD, cpu, (pt2) maximize: 0.0001517058415910765
- gat, SGD, cpu, maximize: 5.7060843300132544e-05
- gat, SGD, cpu, (pt2) no_foreach: 0.00010246507500050939
- gat, SGD, cpu, no_foreach: 3.07262204001745e-05
- gat, SGD, cpu, (pt2) differentiable: 0.00010893724100060354
- gat, SGD, cpu, differentiable: 3.267036409997672e-05
- gat, SGD, cpu, (pt2) foreach: 0.00011511942549986998
- gat, SGD, cpu, foreach: 5.0658293300148215e-05
- gat, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: 0.0002142542110013892
- gat, SGD, cpu, foreach, momentum=0.9, nesterov: 0.00012083676800102694
- gat, SGD, cpu, (pt2) foreach, momentum=0.9: 0.0002001579700008733
- gat, SGD, cpu, foreach, momentum=0.9: 0.00010715531099958752
- gat, RAdam, cpu, (pt2) default: 0.0006559748520012363
- gat, RAdam, cpu, default: 0.000546862979997968
- gat, RAdam, cpu, (pt2) no_foreach: 0.0006574409299973923
- gat, RAdam, cpu, no_foreach: 0.00055180187699807
- gat, RAdam, cpu, (pt2) differentiable: 0.0006673204940016149
- gat, RAdam, cpu, differentiable: 0.0005628446059999987
- gat, RAdam, cpu, (pt2) foreach: 0.000539461445001507
- gat, RAdam, cpu, foreach: 0.0004351794649992371
- gat, Rprop, cpu, (pt2) default: 0.0002691809165530116
- gat, Rprop, cpu, default: 0.00098001518331633
- gat, Rprop, cpu, (pt2) maximize: 0.000263227422438879
- gat, Rprop, cpu, maximize: 0.0010251425050046238
- gat, Rprop, cpu, (pt2) no_foreach: 0.00021441123699696618
- gat, Rprop, cpu, no_foreach: 0.0009543001733375907
- gat, Rprop, cpu, (pt2) differentiable: 0.000250878657947337
- gat, Rprop, cpu, differentiable: 0.001002843895003025
- gat, Rprop, cpu, (pt2) foreach: 0.0011756498999966423
- gat, Rprop, cpu, foreach: 0.0009985993800122137
- gat, RMSprop, cpu, (pt2) default: 0.00017413145602219034
- gat, RMSprop, cpu, default: 0.00018349740949997795
- gat, RMSprop, cpu, (pt2) maximize: 0.0001769924955680787
- gat, RMSprop, cpu, maximize: 0.00021676672600005987
- gat, RMSprop, cpu, (pt2) no_foreach: 0.00012667303550006182
- gat, RMSprop, cpu, no_foreach: 0.0002060070849984186
- gat, RMSprop, cpu, (pt2) differentiable: 0.0001689257162125442
- gat, RMSprop, cpu, differentiable: 0.00020025029449971044
- gat, RMSprop, cpu, (pt2) foreach: 0.0003358558069994615
- gat, RMSprop, cpu, foreach: 0.0001970597320014349
- gat, NAdam, cpu, (pt2) default: 0.0012561443999857147
- gat, NAdam, cpu, default: 0.0004201779870018072
- gat, NAdam, cpu, (pt2) no_foreach: 0.0013863029050116891
- gat, NAdam, cpu, no_foreach: 0.00040540835199863066
- gat, NAdam, cpu, (pt2) differentiable: 0.0012943381225575708
- gat, NAdam, cpu, differentiable: 0.0006956145420008397
- gat, NAdam, cpu, (pt2) foreach: 0.0005494897280004807
- gat, NAdam, cpu, foreach: 0.0004322631860013644
- gcn, Adadelta, cpu, (pt2) default: 0.00013827613124009396
- gcn, Adadelta, cpu, default: 0.0001723103655003797
- gcn, Adadelta, cpu, (pt2) maximize: 0.00013875580447695758
- gcn, Adadelta, cpu, maximize: 0.00018837337849799952
- gcn, Adadelta, cpu, (pt2) no_foreach: 9.712756299995818e-05
- gcn, Adadelta, cpu, no_foreach: 0.00016397555349976757
- gcn, Adadelta, cpu, (pt2) differentiable: 0.0001380325583177069
- gcn, Adadelta, cpu, differentiable: 0.00018957107849928433
- gcn, Adadelta, cpu, (pt2) foreach: 0.0003342978780019621
- gcn, Adadelta, cpu, foreach: 0.0001967704250000679
- gcn, Adagrad, cpu, (pt2) default: 0.00017044778111144056
- gcn, Adagrad, cpu, default: 9.198890400148231e-05
- gcn, Adagrad, cpu, (pt2) maximize: 0.00017478129170124833
- gcn, Adagrad, cpu, maximize: 0.00010894861800079525
- gcn, Adagrad, cpu, (pt2) no_foreach: 0.0001243226765000145
- gcn, Adagrad, cpu, no_foreach: 8.851555833340778e-05
- gcn, Adagrad, cpu, (pt2) differentiable: 0.00017578591571728268
- gcn, Adagrad, cpu, differentiable: 0.00010207714549869705
- gcn, Adagrad, cpu, (pt2) foreach: 0.0003766397639992647
- gcn, Adagrad, cpu, foreach: 0.0002671664049994433
- gcn, Adam, cpu, (pt2) default: 0.0003189726070013421
- gcn, Adam, cpu, default: 0.00020261893999850146
- gcn, Adam, cpu, (pt2) amsgrad, maximize: 0.0003562481890003255
- gcn, Adam, cpu, amsgrad, maximize: 0.00024786506899909
- gcn, Adam, cpu, (pt2) no_foreach: 0.0003056507949986553
- gcn, Adam, cpu, no_foreach: 0.0001973901189994649
- gcn, Adam, cpu, (pt2) differentiable: 0.00045650571899750504
- gcn, Adam, cpu, differentiable: 0.0003483163759992749
- gcn, Adam, cpu, (pt2) foreach: 0.00036728975900041403
- gcn, Adam, cpu, foreach: 0.00025773326200214796
- gcn, AdamW, cpu, (pt2) default: 0.00033604211999772816
- gcn, AdamW, cpu, default: 0.00021766815799855976
- gcn, AdamW, cpu, (pt2) amsgrad, maximize: 0.0003858113960013725
- gcn, AdamW, cpu, amsgrad, maximize: 0.00026056924600197817
- gcn, AdamW, cpu, (pt2) no_foreach: 0.000326114041999972
- gcn, AdamW, cpu, no_foreach: 0.00021195392400113634
- gcn, AdamW, cpu, (pt2) differentiable: 0.00048555961799866054
- gcn, AdamW, cpu, differentiable: 0.0003476991900024586
- gcn, AdamW, cpu, (pt2) foreach: 0.0003769401149984333
- gcn, AdamW, cpu, foreach: 0.00026344578100179206
- gcn, Adamax, cpu, (pt2) default: 0.0004077372281218674
- gcn, Adamax, cpu, default: 0.00022129592999772286
- gcn, Adamax, cpu, (pt2) maximize: 0.0004250413226386984
- gcn, Adamax, cpu, maximize: 0.00026861780300168903
- gcn, Adamax, cpu, (pt2) no_foreach: 0.00047301418000279225
- gcn, Adamax, cpu, no_foreach: 0.0002151512519994867
- gcn, Adamax, cpu, (pt2) differentiable: 0.00039381187585733397
- gcn, Adamax, cpu, differentiable: 0.0002430960870005947
- gcn, Adamax, cpu, (pt2) foreach: 0.0005363689109981351
- gcn, Adamax, cpu, foreach: 0.0004442787279986078
- gcn, ASGD, cpu, (pt2) default: 0.00022682059074927185
- gcn, ASGD, cpu, default: 0.00011992927549908928
- gcn, ASGD, cpu, (pt2) maximize: 0.00021767203045875428
- gcn, ASGD, cpu, maximize: 0.00012619755000014264
- gcn, ASGD, cpu, (pt2) no_foreach: 0.0001594674460011447
- gcn, ASGD, cpu, no_foreach: 0.00010769992199857371
- gcn, ASGD, cpu, (pt2) differentiable: 0.00016449165549965982
- gcn, ASGD, cpu, differentiable: 0.00010969836500044039
- gcn, ASGD, cpu, (pt2) foreach: 0.00023319044599702467
- gcn, ASGD, cpu, foreach: 0.0001489747615014494
- gcn, SGD, cpu, (pt2) default: 0.00012613004037120845
- gcn, SGD, cpu, default: 2.248357219978061e-05
- gcn, SGD, cpu, (pt2) maximize: 0.00012393458549372818
- gcn, SGD, cpu, maximize: 3.208857259996876e-05
- gcn, SGD, cpu, (pt2) no_foreach: 8.446909339982084e-05
- gcn, SGD, cpu, no_foreach: 2.0793247199981123e-05
- gcn, SGD, cpu, (pt2) differentiable: 8.916158466672642e-05
- gcn, SGD, cpu, differentiable: 2.2558388399920658e-05
- gcn, SGD, cpu, (pt2) foreach: 9.299399833374385e-05
- gcn, SGD, cpu, foreach: 3.95321611998952e-05
- gcn, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: 0.0001539823784987675
- gcn, SGD, cpu, foreach, momentum=0.9, nesterov: 7.702860150020569e-05
- gcn, SGD, cpu, (pt2) foreach, momentum=0.9: 0.00014294944050016056
- gcn, SGD, cpu, foreach, momentum=0.9: 6.854640730016399e-05
- gcn, RAdam, cpu, (pt2) default: 0.0004019956890006142
- gcn, RAdam, cpu, default: 0.0002808347050013254
- gcn, RAdam, cpu, (pt2) no_foreach: 0.00037658931700207176
- gcn, RAdam, cpu, no_foreach: 0.00026965345099961267
- gcn, RAdam, cpu, (pt2) differentiable: 0.000403916642000695
- gcn, RAdam, cpu, differentiable: 0.00028936881299887316
- gcn, RAdam, cpu, (pt2) foreach: 0.0003601488759995846
- gcn, RAdam, cpu, foreach: 0.00025322141499782444
- gcn, Rprop, cpu, (pt2) default: 0.00018711831330258457
- gcn, Rprop, cpu, default: 0.0004671421769999142
- gcn, Rprop, cpu, (pt2) maximize: 0.00018257202916247746
- gcn, Rprop, cpu, maximize: 0.0004434198849994573
- gcn, Rprop, cpu, (pt2) no_foreach: 0.0001274805759985611
- gcn, Rprop, cpu, no_foreach: 0.0004281046699979925
- gcn, Rprop, cpu, (pt2) differentiable: 0.0001735085334344042
- gcn, Rprop, cpu, differentiable: 0.0004450318720009818
- gcn, Rprop, cpu, (pt2) foreach: 0.0006306130199991458
- gcn, Rprop, cpu, foreach: 0.0004728267570026219
- gcn, RMSprop, cpu, (pt2) default: 0.00012749421223916453
- gcn, RMSprop, cpu, default: 8.931531866498213e-05
- gcn, RMSprop, cpu, (pt2) maximize: 0.00012479934500055334
- gcn, RMSprop, cpu, maximize: 0.00010218905499823449
- gcn, RMSprop, cpu, (pt2) no_foreach: 8.852612833409996e-05
- gcn, RMSprop, cpu, no_foreach: 8.520708333283739e-05
- gcn, RMSprop, cpu, (pt2) differentiable: 0.00012203181627835957
- gcn, RMSprop, cpu, differentiable: 9.327191333189451e-05
- gcn, RMSprop, cpu, (pt2) foreach: 0.00023433145200033322
- gcn, RMSprop, cpu, foreach: 0.00010801926949898188
- gcn, NAdam, cpu, (pt2) default: 0.0007764696357446875
- gcn, NAdam, cpu, default: 0.00020015494900144405
- gcn, NAdam, cpu, (pt2) no_foreach: 0.0008774695459978829
- gcn, NAdam, cpu, no_foreach: 0.00021244522299821255
- gcn, NAdam, cpu, (pt2) differentiable: 0.0007580888786971186
- gcn, NAdam, cpu, differentiable: 0.0003462754030006181
- gcn, NAdam, cpu, (pt2) foreach: 0.00036528355599875793
- gcn, NAdam, cpu, foreach: 0.0002707745600018825
- llama, NAdam, cpu, (pt2) default: 0.14997788600157946
- llama, NAdam, cpu, (pt2) no_foreach: 0.14902233599968895
- llama, NAdam, cpu, (pt2) differentiable: 0.20075965800060658
- sage, Adadelta, cpu, (pt2) default: 0.0005768508469991502
- sage, Adadelta, cpu, default: 0.0004666956289984228
- sage, Adadelta, cpu, (pt2) maximize: 0.0006270642429990403
- sage, Adadelta, cpu, maximize: 0.0005043567669999902
- sage, Adadelta, cpu, (pt2) no_foreach: 0.0005707598899971345
- sage, Adadelta, cpu, no_foreach: 0.0004520572299988999
- sage, Adadelta, cpu, (pt2) differentiable: 0.0006345567390017095
- sage, Adadelta, cpu, differentiable: 0.0004923018039989983
- sage, Adadelta, cpu, (pt2) foreach: 0.0005287680109977373
- sage, Adadelta, cpu, foreach: 0.0004231342390012287
- sage, Adagrad, cpu, (pt2) default: 0.0004059117819997482
- sage, Adagrad, cpu, default: 0.0002807196819994715
- sage, Adagrad, cpu, (pt2) maximize: 0.000438641878998169
- sage, Adagrad, cpu, maximize: 0.00031563253200147303
- sage, Adagrad, cpu, (pt2) no_foreach: 0.0003953113099996699
- sage, Adagrad, cpu, no_foreach: 0.0002737348379996547
- sage, Adagrad, cpu, (pt2) differentiable: 0.0004323486959983711
- sage, Adagrad, cpu, differentiable: 0.00030557578400112106
- sage, Adagrad, cpu, (pt2) foreach: 0.0007614882519992534
- sage, Adagrad, cpu, foreach: 0.000605587875998026
- sage, Adam, cpu, (pt2) default: 0.0006306220160004159
- sage, Adam, cpu, default: 0.0005158872790016175
- sage, Adam, cpu, (pt2) amsgrad, maximize: 0.0007427445919965976
- sage, Adam, cpu, amsgrad, maximize: 0.0006108978659976856
- sage, Adam, cpu, (pt2) no_foreach: 0.0006239733439979318
- sage, Adam, cpu, no_foreach: 0.0005040394679999736
- sage, Adam, cpu, (pt2) differentiable: 0.0010704759800137253
- sage, Adam, cpu, differentiable: 0.0009280956866738658
- sage, Adam, cpu, (pt2) foreach: 0.0006262617960019269
- sage, Adam, cpu, foreach: 0.0005023022099994705
- sage, AdamW, cpu, (pt2) default: 0.000652339722000761
- sage, AdamW, cpu, default: 0.0005408288869984971
- sage, AdamW, cpu, (pt2) amsgrad, maximize: 0.0007855557170005341
- sage, AdamW, cpu, amsgrad, maximize: 0.0006464221030000772
- sage, AdamW, cpu, (pt2) no_foreach: 0.0006544801199997892
- sage, AdamW, cpu, no_foreach: 0.000529578057998151
- sage, AdamW, cpu, (pt2) differentiable: 0.001098357874998328
- sage, AdamW, cpu, differentiable: 0.0009557477399963925
- sage, AdamW, cpu, (pt2) foreach: 0.0006577919839983224
- sage, AdamW, cpu, foreach: 0.0005300489550027123
- sage, Adamax, cpu, (pt2) default: 0.0007436160180004662
- sage, Adamax, cpu, default: 0.0006268804909996107
- sage, Adamax, cpu, (pt2) maximize: 0.0007859496339988254
- sage, Adamax, cpu, maximize: 0.0006652005709984224
- sage, Adamax, cpu, (pt2) no_foreach: 0.0007151770700002089
- sage, Adamax, cpu, no_foreach: 0.0006103453910000098
- sage, Adamax, cpu, (pt2) differentiable: 0.0007962861609994434
- sage, Adamax, cpu, differentiable: 0.0006584906659991247
- sage, Adamax, cpu, (pt2) foreach: 0.001046835609995469
- sage, Adamax, cpu, foreach: 0.0009224195850001707
- sage, ASGD, cpu, (pt2) default: 0.00042155723299947565
- sage, ASGD, cpu, default: 0.000305791479000618
- sage, ASGD, cpu, (pt2) maximize: 0.0005093072880008549
- sage, ASGD, cpu, maximize: 0.00037892760499744324
- sage, ASGD, cpu, (pt2) no_foreach: 0.0004010132450021047
- sage, ASGD, cpu, no_foreach: 0.00029440398500082666
- sage, ASGD, cpu, (pt2) differentiable: 0.00040629445100057637
- sage, ASGD, cpu, differentiable: 0.00029463066099924616
- sage, ASGD, cpu, (pt2) foreach: 0.00044659789800061844
- sage, ASGD, cpu, foreach: 0.0003620546849997481
- sage, SGD, cpu, (pt2) default: 0.00012743672999931732
- sage, SGD, cpu, default: 6.371854960016208e-05
- sage, SGD, cpu, (pt2) maximize: 0.000198796709501039
- sage, SGD, cpu, maximize: 0.00011137914899882163
- sage, SGD, cpu, (pt2) no_foreach: 0.00012024206550086091
- sage, SGD, cpu, no_foreach: 5.844056850000925e-05
- sage, SGD, cpu, (pt2) differentiable: 0.0001278311525002209
- sage, SGD, cpu, differentiable: 6.435009039996657e-05
- sage, SGD, cpu, (pt2) foreach: 0.00017050191300040752
- sage, SGD, cpu, foreach: 9.433867379993898e-05
- sage, SGD, cpu, (pt2) foreach, momentum=0.9, nesterov: 0.00027183092399718587
- sage, SGD, cpu, foreach, momentum=0.9, nesterov: 0.00017406429699985893
- sage, SGD, cpu, (pt2) foreach, momentum=0.9: 0.0002573432299977867
- sage, SGD, cpu, foreach, momentum=0.9: 0.00015647386650016417
- sage, RAdam, cpu, (pt2) default: 0.0008828281633274553
- sage, RAdam, cpu, default: 0.0007647603100012929
- sage, RAdam, cpu, (pt2) no_foreach: 0.0008687835040000209
- sage, RAdam, cpu, no_foreach: 0.0007608631010007229
- sage, RAdam, cpu, (pt2) differentiable: 0.0008930404700125413
- sage, RAdam, cpu, differentiable: 0.0007941236340011528
- sage, RAdam, cpu, (pt2) foreach: 0.0006670491309996578
- sage, RAdam, cpu, foreach: 0.0005600116139976307
- sage, Rprop, cpu, (pt2) default: 0.0011782137599948327
- sage, Rprop, cpu, default: 0.0010464486900127667
- sage, Rprop, cpu, (pt2) maximize: 0.001230045775009785
- sage, Rprop, cpu, maximize: 0.0011006252799779757
- sage, Rprop, cpu, (pt2) no_foreach: 0.0011630521450024388
- sage, Rprop, cpu, no_foreach: 0.001034375645012915
- sage, Rprop, cpu, (pt2) differentiable: 0.001199865385005978
- sage, Rprop, cpu, differentiable: 0.0011329833550189506
- sage, Rprop, cpu, (pt2) foreach: 0.0012036167949918308
- sage, Rprop, cpu, foreach: 0.001077136905005318
- sage, RMSprop, cpu, (pt2) default: 0.00037828651900053957
- sage, RMSprop, cpu, default: 0.00027325806600129
- sage, RMSprop, cpu, (pt2) maximize: 0.00041883705299915165
- sage, RMSprop, cpu, maximize: 0.0002991924890011433
- sage, RMSprop, cpu, (pt2) no_foreach: 0.00036743251599909854
- sage, RMSprop, cpu, no_foreach: 0.00025162207299945296
- sage, RMSprop, cpu, (pt2) differentiable: 0.00039754764399913255
- sage, RMSprop, cpu, differentiable: 0.0002801561119995313
- sage, RMSprop, cpu, (pt2) foreach: 0.0003978926300005696
- sage, RMSprop, cpu, foreach: 0.00028235974699782675
- sage, NAdam, cpu, (pt2) default: 0.000645498531001067
- sage, NAdam, cpu, default: 0.0006208859889993618
- sage, NAdam, cpu, (pt2) no_foreach: 0.0006327859110024292
- sage, NAdam, cpu, no_foreach: 0.0005174445039992861
- sage, NAdam, cpu, (pt2) differentiable: 0.0010947214949919726
- sage, NAdam, cpu, differentiable: 0.0009578403866665515
- sage, NAdam, cpu, (pt2) foreach: 0.0006793211629992584
- sage, NAdam, cpu, foreach: 0.0005354838530001871


Tests that were newly added on affected commit:


cc @janeyx99


This is related to and resolved by https://github.com/pytorch/benchmark/pull/1557