[2024-06-20 14:35:25,629][__main__][INFO] - Using GPU for training. Main device : cuda:3
[2024-06-20 14:35:25,630][__main__][INFO] - Using multiple GPUs : [3, 0, 1, 2]
[2024-06-20 14:35:30,491][__main__][INFO] - PSF shape : torch.Size([1, 135, 240, 3])
[2024-06-20 14:35:30,502][__main__][INFO] - PSF min : 0.0
[2024-06-20 14:35:30,512][__main__][INFO] - PSF max : 0.12337537109851837
[2024-06-20 14:35:30,512][__main__][INFO] - PSF dtype : torch.float32
[2024-06-20 14:35:30,543][__main__][INFO] - PSF norm : 1.7320507764816284
[2024-06-20 14:35:30,544][__main__][INFO] - Reconstruction a few images with ADMM...
[2024-06-20 14:35:31,164][__main__][INFO] - Cropped shape :  (135, 240, 3)
[2024-06-20 14:35:32,582][__main__][INFO] - Train test size : 24000
[2024-06-20 14:35:32,583][__main__][INFO] - Test test size : 999
[2024-06-20 14:35:32,703][__main__][INFO] - Training model with 8258449 parameters
[2024-06-20 14:35:32,704][__main__][INFO] - Setup time : 0.12098121643066406 s
[2024-06-20 14:35:32,704][__main__][INFO] - PSF shape : torch.Size([1, 135, 240, 3])
[2024-06-20 14:35:32,704][__main__][INFO] - Results saved in /root/LenslessPiCam/outputs/2024-06-20/14-35-20
[2024-06-20 14:36:40,878][__main__][INFO] - Epoch 0 with learning rate [0.0001]
[2024-06-20 15:12:35,934][__main__][INFO] - loss : 0.4297537258366742
[2024-06-20 15:13:40,148][__main__][INFO] - Epoch 1 with learning rate [0.0001]
[2024-06-20 15:49:22,226][__main__][INFO] - loss : 0.3631795203735422
[2024-06-20 15:50:26,534][__main__][INFO] - Epoch 2 with learning rate [0.0001]
[2024-06-20 16:26:21,177][__main__][INFO] - loss : 0.33756433516244205
[2024-06-20 16:27:24,611][__main__][INFO] - Epoch 3 with learning rate [0.0001]
[2024-06-20 17:03:12,686][__main__][INFO] - loss : 0.3195842023516688
[2024-06-20 17:04:17,320][__main__][INFO] - Epoch 4 with learning rate [0.0001]
[2024-06-20 17:40:07,969][__main__][INFO] - loss : 0.30713305672258157
[2024-06-20 17:41:12,001][__main__][INFO] - Epoch 5 with learning rate [0.0001]
[2024-06-20 18:17:02,850][__main__][INFO] - loss : 0.2986604493558409
[2024-06-20 18:18:05,966][__main__][INFO] - Epoch 6 with learning rate [0.0001]
[2024-06-20 18:54:00,091][__main__][INFO] - loss : 0.291894884717961
[2024-06-20 18:55:05,138][__main__][INFO] - Epoch 7 with learning rate [0.0001]
[2024-06-20 19:30:57,448][__main__][INFO] - loss : 0.28663481043775874
[2024-06-20 19:32:02,015][__main__][INFO] - Epoch 8 with learning rate [0.0001]
[2024-06-20 20:06:44,291][__main__][INFO] - loss : 0.28102396032214116
[2024-06-20 20:07:47,025][__main__][INFO] - Epoch 9 with learning rate [0.0001]
[2024-06-20 20:41:06,800][__main__][INFO] - loss : 0.27642058091114047
[2024-06-20 20:42:08,531][__main__][INFO] - Epoch 10 with learning rate [0.0001]
[2024-06-20 21:15:34,606][__main__][INFO] - loss : 0.27257460041840886
[2024-06-20 21:16:35,805][__main__][INFO] - Epoch 11 with learning rate [0.0001]
[2024-06-20 21:49:43,900][__main__][INFO] - loss : 0.2696525942360355
[2024-06-20 21:50:45,209][__main__][INFO] - Epoch 12 with learning rate [0.0001]
[2024-06-20 22:24:05,758][__main__][INFO] - loss : 0.2663207350596795
[2024-06-20 22:25:05,999][__main__][INFO] - Epoch 13 with learning rate [0.0001]
[2024-06-20 22:58:20,147][__main__][INFO] - loss : 0.2630512365425621
[2024-06-20 22:59:21,844][__main__][INFO] - Epoch 14 with learning rate [0.0001]
[2024-06-20 23:32:36,788][__main__][INFO] - loss : 0.26003734795749234
[2024-06-20 23:33:37,068][__main__][INFO] - Epoch 15 with learning rate [0.0001]
[2024-06-21 00:06:53,350][__main__][INFO] - loss : 0.2575299395297958
[2024-06-21 00:07:54,667][__main__][INFO] - Epoch 16 with learning rate [0.0001]
[2024-06-21 00:41:02,043][__main__][INFO] - loss : 0.25518408266206777
[2024-06-21 00:42:02,351][__main__][INFO] - Epoch 17 with learning rate [0.0001]
[2024-06-21 01:15:24,978][__main__][INFO] - loss : 0.2527237752377981
[2024-06-21 01:16:25,555][__main__][INFO] - Epoch 18 with learning rate [0.0001]
[2024-06-21 01:49:46,465][__main__][INFO] - loss : 0.25089738271385376
[2024-06-21 01:50:47,853][__main__][INFO] - Epoch 19 with learning rate [0.0001]
[2024-06-21 02:24:04,030][__main__][INFO] - loss : 0.24873794276763983
[2024-06-21 02:25:04,163][__main__][INFO] - Epoch 20 with learning rate [0.0001]
[2024-06-21 02:58:16,214][__main__][INFO] - loss : 0.2465690987880031
[2024-06-21 02:59:18,127][__main__][INFO] - Epoch 21 with learning rate [0.0001]
[2024-06-21 03:32:36,865][__main__][INFO] - loss : 0.24460794302821143
[2024-06-21 03:33:36,896][__main__][INFO] - Epoch 22 with learning rate [0.0001]
[2024-06-21 04:06:49,974][__main__][INFO] - loss : 0.24277881035456994
[2024-06-21 04:07:50,455][__main__][INFO] - Epoch 23 with learning rate [0.0001]
[2024-06-21 04:41:10,998][__main__][INFO] - loss : 0.2410393918504314
[2024-06-21 04:42:12,981][__main__][INFO] - Epoch 24 with learning rate [0.0001]
[2024-06-21 05:15:31,954][__main__][INFO] - loss : 0.23927416165918103
[2024-06-21 05:16:33,688][__main__][INFO] - Train time [hour] : 14.682988125218285 h
[2024-06-21 05:16:33,688][__main__][INFO] - Results saved in /root/LenslessPiCam/outputs/2024-06-20/14-35-20