Use bfloat16 for core module.
Use parallel attention and MLP in ViViT.

Epoch 001/400
Train		loss: 113924008.00	correlation: 0.0107
Validation	loss: 200107760.00	correlation: 0.0263
Elapse: 605.46s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 002/400
Train		loss: 97896768.00	correlation: 0.0340
Validation	loss: 199235856.00	correlation: 0.0378
Elapse: 556.86s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 003/400
Train		loss: 96883144.00	correlation: 0.0443
Validation	loss: 198767200.00	correlation: 0.0405
Elapse: 560.31s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 004/400
Train		loss: 96317000.00	correlation: 0.0499
Validation	loss: 198194576.00	correlation: 0.0449
Elapse: 560.78s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 005/400
Train		loss: 95592096.00	correlation: 0.0577
Validation	loss: 197382576.00	correlation: 0.0493
Elapse: 563.47s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 006/400
Train		loss: 94455696.00	correlation: 0.0692
Validation	loss: 195917728.00	correlation: 0.0602
Elapse: 565.83s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 007/400
Train		loss: 93257144.00	correlation: 0.0821
Validation	loss: 193956304.00	correlation: 0.0721
Elapse: 566.60s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 008/400
Train		loss: 92014728.00	correlation: 0.0951
Validation	loss: 192039424.00	correlation: 0.0839
Elapse: 566.05s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 009/400
Train		loss: 91067848.00	correlation: 0.1053
Validation	loss: 190701360.00	correlation: 0.0935
Elapse: 563.36s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 010/400
Train		loss: 90106464.00	correlation: 0.1149
Validation	loss: 189259712.00	correlation: 0.1026
Elapse: 562.44s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 011/400
Train		loss: 89183952.00	correlation: 0.1243
Validation	loss: 187589920.00	correlation: 0.1130
Elapse: 560.31s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 012/400
Train		loss: 88171952.00	correlation: 0.1348
Validation	loss: 186131232.00	correlation: 0.1219
Elapse: 559.08s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 013/400
Train		loss: 87220088.00	correlation: 0.1442
Validation	loss: 184294144.00	correlation: 0.1328
Elapse: 564.57s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 014/400
Train		loss: 86305888.00	correlation: 0.1535
Validation	loss: 182766432.00	correlation: 0.1425
Elapse: 564.31s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 015/400
Train		loss: 85306032.00	correlation: 0.1632
Validation	loss: 181567952.00	correlation: 0.1505
Elapse: 562.48s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 016/400
Train		loss: 84622736.00	correlation: 0.1701
Validation	loss: 180351280.00	correlation: 0.1587
Elapse: 561.75s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 017/400
Train		loss: 83839824.00	correlation: 0.1780
Validation	loss: 179434864.00	correlation: 0.1644
Elapse: 561.90s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 018/400
Train		loss: 83213000.00	correlation: 0.1840
Validation	loss: 178816208.00	correlation: 0.1695
Elapse: 562.07s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 019/400
Train		loss: 82673560.00	correlation: 0.1895
Validation	loss: 178169568.00	correlation: 0.1740
Elapse: 562.05s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 020/400
Train		loss: 82081376.00	correlation: 0.1949
Validation	loss: 177435600.00	correlation: 0.1787
Elapse: 561.87s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 021/400
Train		loss: 81644440.00	correlation: 0.1992
Validation	loss: 176873408.00	correlation: 0.1824
Elapse: 562.31s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 022/400
Train		loss: 81300944.00	correlation: 0.2027
Validation	loss: 176298464.00	correlation: 0.1855
Elapse: 562.57s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 023/400
Train		loss: 80864992.00	correlation: 0.2072
Validation	loss: 175630304.00	correlation: 0.1905
Elapse: 562.55s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 024/400
Train		loss: 80516536.00	correlation: 0.2102
Validation	loss: 175099184.00	correlation: 0.1937
Elapse: 561.74s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 025/400
Train		loss: 80141312.00	correlation: 0.2140
Validation	loss: 174953952.00	correlation: 0.1954
Elapse: 563.63s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 026/400
Train		loss: 79914320.00	correlation: 0.2162
Validation	loss: 174397456.00	correlation: 0.1975
Elapse: 563.20s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 027/400
Train		loss: 79520984.00	correlation: 0.2200
Validation	loss: 174134560.00	correlation: 0.2014
Elapse: 562.71s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 028/400
Train		loss: 79212208.00	correlation: 0.2229
Validation	loss: 173471920.00	correlation: 0.2046
Elapse: 562.79s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 029/400
Train		loss: 78973040.00	correlation: 0.2252
Validation	loss: 173183520.00	correlation: 0.2064
Elapse: 563.08s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 030/400
Train		loss: 78870784.00	correlation: 0.2264
Validation	loss: 172775088.00	correlation: 0.2096
Elapse: 562.82s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 031/400
Train		loss: 78490200.00	correlation: 0.2296
Validation	loss: 172664992.00	correlation: 0.2107
Elapse: 563.05s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 032/400
Train		loss: 78317200.00	correlation: 0.2318
Validation	loss: 172019824.00	correlation: 0.2132
Elapse: 562.89s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 033/400
Train		loss: 78137128.00	correlation: 0.2333
Validation	loss: 171897024.00	correlation: 0.2150
Elapse: 562.71s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 034/400
Train		loss: 77950032.00	correlation: 0.2354
Validation	loss: 171802656.00	correlation: 0.2157
Elapse: 562.68s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 035/400
Train		loss: 77766832.00	correlation: 0.2367
Validation	loss: 171322160.00	correlation: 0.2180
Elapse: 561.92s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 036/400
Train		loss: 77637216.00	correlation: 0.2384
Validation	loss: 171034880.00	correlation: 0.2209
Elapse: 562.15s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 037/400
Train		loss: 77482304.00	correlation: 0.2397
Validation	loss: 171174608.00	correlation: 0.2203
Elapse: 564.93s

Epoch 038/400
Train		loss: 77429408.00	correlation: 0.2402
Validation	loss: 170815888.00	correlation: 0.2218
Elapse: 564.45s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 039/400
Train		loss: 77292792.00	correlation: 0.2412
Validation	loss: 170692576.00	correlation: 0.2224
Elapse: 563.94s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 040/400
Train		loss: 77122352.00	correlation: 0.2426
Validation	loss: 170492064.00	correlation: 0.2256
Elapse: 565.03s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 041/400
Train		loss: 77124456.00	correlation: 0.2430
Validation	loss: 170532256.00	correlation: 0.2245
Elapse: 564.39s

Epoch 042/400
Train		loss: 76964592.00	correlation: 0.2445
Validation	loss: 170406688.00	correlation: 0.2257
Elapse: 565.12s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 043/400
Train		loss: 76869008.00	correlation: 0.2454
Validation	loss: 170592080.00	correlation: 0.2248
Elapse: 563.98s

Epoch 044/400
Train		loss: 76798544.00	correlation: 0.2461
Validation	loss: 170285664.00	correlation: 0.2253
Elapse: 564.88s

Epoch 045/400
Train		loss: 76706736.00	correlation: 0.2469
Validation	loss: 170137504.00	correlation: 0.2276
Elapse: 564.78s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 046/400
Train		loss: 76661312.00	correlation: 0.2474
Validation	loss: 170203360.00	correlation: 0.2258
Elapse: 563.90s

Epoch 047/400
Train		loss: 76653904.00	correlation: 0.2479
Validation	loss: 169920448.00	correlation: 0.2283
Elapse: 565.79s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 048/400
Train		loss: 76510704.00	correlation: 0.2491
Validation	loss: 169954384.00	correlation: 0.2279
Elapse: 566.41s

Epoch 049/400
Train		loss: 76493416.00	correlation: 0.2492
Validation	loss: 169840224.00	correlation: 0.2290
Elapse: 566.13s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 050/400
Train		loss: 76429016.00	correlation: 0.2495
Validation	loss: 170036560.00	correlation: 0.2272
Elapse: 567.15s

Epoch 051/400
Train		loss: 76434184.00	correlation: 0.2500
Validation	loss: 169668848.00	correlation: 0.2298
Elapse: 563.78s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 052/400
Train		loss: 76368824.00	correlation: 0.2505
Validation	loss: 169862640.00	correlation: 0.2292
Elapse: 563.52s

Epoch 053/400
Train		loss: 76235120.00	correlation: 0.2516
Validation	loss: 169524560.00	correlation: 0.2315
Elapse: 565.24s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 054/400
Train		loss: 76236176.00	correlation: 0.2518
Validation	loss: 169235696.00	correlation: 0.2324
Elapse: 564.43s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 055/400
Train		loss: 76272208.00	correlation: 0.2514
Validation	loss: 169566032.00	correlation: 0.2303
Elapse: 564.62s

Epoch 056/400
Train		loss: 76095872.00	correlation: 0.2528
Validation	loss: 169257008.00	correlation: 0.2329
Elapse: 564.52s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 057/400
Train		loss: 76150160.00	correlation: 0.2523
Validation	loss: 169306656.00	correlation: 0.2323
Elapse: 564.42s

Epoch 058/400
Train		loss: 75971168.00	correlation: 0.2540
Validation	loss: 169281824.00	correlation: 0.2326
Elapse: 565.79s

Epoch 059/400
Train		loss: 76016632.00	correlation: 0.2537
Validation	loss: 169175984.00	correlation: 0.2330
Elapse: 567.53s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 060/400
Train		loss: 75961288.00	correlation: 0.2543
Validation	loss: 169244832.00	correlation: 0.2312
Elapse: 564.74s

Epoch 061/400
Train		loss: 75918368.00	correlation: 0.2545
Validation	loss: 169277088.00	correlation: 0.2310
Elapse: 564.73s

Epoch 062/400
Train		loss: 75869072.00	correlation: 0.2550
Validation	loss: 169328944.00	correlation: 0.2320
Elapse: 564.39s

Epoch 063/400
Train		loss: 75762192.00	correlation: 0.2561
Validation	loss: 169258112.00	correlation: 0.2316
Elapse: 564.19s

Epoch 064/400
Train		loss: 75825328.00	correlation: 0.2556
Validation	loss: 169071840.00	correlation: 0.2332
Elapse: 564.93s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 065/400
Train		loss: 75713256.00	correlation: 0.2566
Validation	loss: 169214000.00	correlation: 0.2329
Elapse: 564.84s

Epoch 066/400
Train		loss: 75745160.00	correlation: 0.2561
Validation	loss: 169041248.00	correlation: 0.2327
Elapse: 564.78s

Epoch 067/400
Train		loss: 75648152.00	correlation: 0.2572
Validation	loss: 169051616.00	correlation: 0.2327
Elapse: 565.20s

Epoch 068/400
Train		loss: 75614368.00	correlation: 0.2577
Validation	loss: 169221120.00	correlation: 0.2327
Elapse: 564.71s

Epoch 069/400
Train		loss: 75625776.00	correlation: 0.2574
Validation	loss: 169122256.00	correlation: 0.2330
Elapse: 564.46s

Loaded checkpoint from epoch 64 (correlation: 0.2332).

Reduce learning rate of core to 1.4400e-03 (num. reduce: 1).
Reduce learning rate of readouts to 1.0800e-03 (num. reduce: 1).
Reduce learning rate of shifters to 1.0800e-03 (num. reduce: 1).

Epoch 070/400
Train		loss: 73759920.00	correlation: 0.2731
Validation	loss: 167411552.00	correlation: 0.2444
Elapse: 565.27s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 071/400
Train		loss: 73162624.00	correlation: 0.2785
Validation	loss: 167276400.00	correlation: 0.2454
Elapse: 565.31s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 072/400
Train		loss: 73062080.00	correlation: 0.2797
Validation	loss: 167303840.00	correlation: 0.2453
Elapse: 565.70s

Epoch 073/400
Train		loss: 72859248.00	correlation: 0.2818
Validation	loss: 167279840.00	correlation: 0.2448
Elapse: 566.21s

Epoch 074/400
Train		loss: 72867944.00	correlation: 0.2819
Validation	loss: 167328304.00	correlation: 0.2445
Elapse: 566.10s

Epoch 075/400
Train		loss: 72806064.00	correlation: 0.2823
Validation	loss: 167367392.00	correlation: 0.2443
Elapse: 566.39s

Epoch 076/400
Train		loss: 72707504.00	correlation: 0.2836
Validation	loss: 167176992.00	correlation: 0.2450
Elapse: 566.85s

Loaded checkpoint from epoch 71 (correlation: 0.2454).

Reduce learning rate of core to 4.3200e-04 (num. reduce: 1).
Reduce learning rate of readouts to 3.2400e-04 (num. reduce: 1).
Reduce learning rate of shifters to 3.2400e-04 (num. reduce: 1).

Epoch 077/400
Train		loss: 72382144.00	correlation: 0.2856
Validation	loss: 166842576.00	correlation: 0.2479
Elapse: 566.50s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 078/400
Train		loss: 72078376.00	correlation: 0.2881
Validation	loss: 166822496.00	correlation: 0.2483
Elapse: 567.42s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 079/400
Train		loss: 71958248.00	correlation: 0.2896
Validation	loss: 166738624.00	correlation: 0.2488
Elapse: 567.48s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 080/400
Train		loss: 71895880.00	correlation: 0.2900
Validation	loss: 166642640.00	correlation: 0.2490
Elapse: 567.47s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 081/400
Train		loss: 71789800.00	correlation: 0.2911
Validation	loss: 166705248.00	correlation: 0.2491
Elapse: 566.97s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 082/400
Train		loss: 71770424.00	correlation: 0.2915
Validation	loss: 166628368.00	correlation: 0.2487
Elapse: 567.08s

Epoch 083/400
Train		loss: 71742960.00	correlation: 0.2915
Validation	loss: 166623744.00	correlation: 0.2491
Elapse: 567.32s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 084/400
Train		loss: 71726080.00	correlation: 0.2915
Validation	loss: 166593008.00	correlation: 0.2489
Elapse: 566.52s

Epoch 085/400
Train		loss: 71608472.00	correlation: 0.2930
Validation	loss: 166582448.00	correlation: 0.2494
Elapse: 566.39s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 086/400
Train		loss: 71622656.00	correlation: 0.2929
Validation	loss: 166651488.00	correlation: 0.2488
Elapse: 567.27s

Epoch 087/400
Train		loss: 71550640.00	correlation: 0.2936
Validation	loss: 166630768.00	correlation: 0.2490
Elapse: 566.70s

Epoch 088/400
Train		loss: 71491864.00	correlation: 0.2941
Validation	loss: 166610080.00	correlation: 0.2493
Elapse: 567.65s

Epoch 089/400
Train		loss: 71458584.00	correlation: 0.2944
Validation	loss: 166546464.00	correlation: 0.2492
Elapse: 566.68s

Epoch 090/400
Train		loss: 71462792.00	correlation: 0.2945
Validation	loss: 166668320.00	correlation: 0.2492
Elapse: 566.84s

Loaded checkpoint from epoch 85 (correlation: 0.2494).

Reduce learning rate of core to 1.2960e-04 (num. reduce: 1).
Reduce learning rate of readouts to 9.7200e-05 (num. reduce: 1).
Reduce learning rate of shifters to 9.7200e-05 (num. reduce: 1).

Epoch 091/400
Train		loss: 71288968.00	correlation: 0.2953
Validation	loss: 166494880.00	correlation: 0.2500
Elapse: 567.12s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 092/400
Train		loss: 71275904.00	correlation: 0.2958
Validation	loss: 166504256.00	correlation: 0.2498
Elapse: 566.80s

Epoch 093/400
Train		loss: 71229392.00	correlation: 0.2962
Validation	loss: 166490080.00	correlation: 0.2502
Elapse: 567.91s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 094/400
Train		loss: 71289360.00	correlation: 0.2955
Validation	loss: 166481824.00	correlation: 0.2501
Elapse: 567.35s

Epoch 095/400
Train		loss: 71209928.00	correlation: 0.2962
Validation	loss: 166463984.00	correlation: 0.2501
Elapse: 567.29s

Epoch 096/400
Train		loss: 71155936.00	correlation: 0.2969
Validation	loss: 166447504.00	correlation: 0.2502
Elapse: 567.66s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 097/400
Train		loss: 71163080.00	correlation: 0.2967
Validation	loss: 166476448.00	correlation: 0.2501
Elapse: 567.23s

Epoch 098/400
Train		loss: 71090216.00	correlation: 0.2976
Validation	loss: 166447952.00	correlation: 0.2502
Elapse: 567.15s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 099/400
Train		loss: 71146160.00	correlation: 0.2971
Validation	loss: 166436720.00	correlation: 0.2504
Elapse: 566.92s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 100/400
Train		loss: 71064016.00	correlation: 0.2977
Validation	loss: 166422944.00	correlation: 0.2502
Elapse: 567.03s

Epoch 101/400
Train		loss: 71056432.00	correlation: 0.2979
Validation	loss: 166403840.00	correlation: 0.2505
Elapse: 567.52s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 102/400
Train		loss: 71051576.00	correlation: 0.2978
Validation	loss: 166443872.00	correlation: 0.2504
Elapse: 567.13s

Epoch 103/400
Train		loss: 71046608.00	correlation: 0.2979
Validation	loss: 166479680.00	correlation: 0.2502
Elapse: 566.67s

Epoch 104/400
Train		loss: 71038120.00	correlation: 0.2981
Validation	loss: 166454112.00	correlation: 0.2501
Elapse: 564.93s

Epoch 105/400
Train		loss: 70978192.00	correlation: 0.2988
Validation	loss: 166414432.00	correlation: 0.2503
Elapse: 564.91s

Epoch 106/400
Train		loss: 70976960.00	correlation: 0.2990
Validation	loss: 166474720.00	correlation: 0.2500
Elapse: 566.07s

Loaded checkpoint from epoch 101 (correlation: 0.2505).

Reduce learning rate of core to 3.8880e-05 (num. reduce: 1).
Reduce learning rate of readouts to 2.9160e-05 (num. reduce: 1).
Reduce learning rate of shifters to 2.9160e-05 (num. reduce: 1).

Epoch 107/400
Train		loss: 70972624.00	correlation: 0.2986
Validation	loss: 166398144.00	correlation: 0.2506
Elapse: 566.75s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 108/400
Train		loss: 70964176.00	correlation: 0.2987
Validation	loss: 166382512.00	correlation: 0.2506
Elapse: 566.44s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 109/400
Train		loss: 70976968.00	correlation: 0.2987
Validation	loss: 166404432.00	correlation: 0.2505
Elapse: 567.32s

Epoch 110/400
Train		loss: 70951736.00	correlation: 0.2985
Validation	loss: 166396896.00	correlation: 0.2506
Elapse: 567.77s

Epoch 111/400
Train		loss: 70895944.00	correlation: 0.2997
Validation	loss: 166385856.00	correlation: 0.2506
Elapse: 566.89s

Epoch 112/400
Train		loss: 70880200.00	correlation: 0.2995
Validation	loss: 166380672.00	correlation: 0.2507
Elapse: 566.94s

Checkpoint saved to /home/storage/runs/vivit_ensemble/002/ckpt/model_state.pt.

Epoch 113/400
Train		loss: 70861304.00	correlation: 0.2998
Validation	loss: 166391184.00	correlation: 0.2506
Elapse: 566.54s

Epoch 114/400
Train		loss: 70921912.00	correlation: 0.2989
Validation	loss: 166417776.00	correlation: 0.2504
Elapse: 567.26s

Epoch 115/400
Train		loss: 70914312.00	correlation: 0.2989
Validation	loss: 166403376.00	correlation: 0.2506
Elapse: 566.92s

Epoch 116/400
Train		loss: 70903784.00	correlation: 0.2992
Validation	loss: 166403360.00	correlation: 0.2506
Elapse: 567.11s

Epoch 117/400
Train		loss: 70866896.00	correlation: 0.2994
Validation	loss: 166382688.00	correlation: 0.2505
Elapse: 566.94s

Loaded checkpoint from epoch 112 (correlation: 0.2507).

Reduce learning rate of core to 1.1664e-05 (num. reduce: 1).
Reduce learning rate of readouts to 8.7480e-06 (num. reduce: 1).
Reduce learning rate of shifters to 8.7480e-06 (num. reduce: 1).

Epoch 118/400
Train		loss: 70926000.00	correlation: 0.2989
Validation	loss: 166380768.00	correlation: 0.2506
Elapse: 567.50s

Epoch 119/400
Train		loss: 70900808.00	correlation: 0.2993
Validation	loss: 166386016.00	correlation: 0.2506
Elapse: 567.35s

Epoch 120/400
Train		loss: 70881328.00	correlation: 0.2993
Validation	loss: 166387568.00	correlation: 0.2506
Elapse: 567.65s

Epoch 121/400
Train		loss: 70937256.00	correlation: 0.2984
Validation	loss: 166393568.00	correlation: 0.2506
Elapse: 566.97s

Epoch 122/400
Train		loss: 70855888.00	correlation: 0.2996
Validation	loss: 166378592.00	correlation: 0.2507
Elapse: 567.50s

Loaded checkpoint from epoch 112 (correlation: 0.2507).

Reduce learning rate of core to 3.4992e-06 (num. reduce: 2).
Reduce learning rate of readouts to 2.6244e-06 (num. reduce: 2).
Reduce learning rate of shifters to 2.6244e-06 (num. reduce: 2).

Epoch 123/400
Train		loss: 70938496.00	correlation: 0.2988
Validation	loss: 166375792.00	correlation: 0.2507
Elapse: 567.50s

Epoch 124/400
Train		loss: 70966320.00	correlation: 0.2986
Validation	loss: 166383664.00	correlation: 0.2506
Elapse: 567.77s

Epoch 125/400
Train		loss: 70872080.00	correlation: 0.2990
Validation	loss: 166383328.00	correlation: 0.2506
Elapse: 567.87s

Epoch 126/400
Train		loss: 70866264.00	correlation: 0.2997
Validation	loss: 166383136.00	correlation: 0.2506
Elapse: 567.21s

Epoch 127/400
Train		loss: 70836352.00	correlation: 0.2996
Validation	loss: 166386816.00	correlation: 0.2506
Elapse: 566.90s

Model has not improved after 2 LR reductions.

Loaded checkpoint from epoch 112 (correlation: 0.2507).


ValidationA: 0.2487	B: 0.2789	C: 0.2720	D: 0.2341	E: 0.2367	F: 0.2340	G: 0.2555	H: 0.2351	I: 0.2533	J: 0.2589	average: 0.2507

Results saved to /home/storage/runs/vivit_ensemble/002.