diff --git "a/sf_log.txt" "b/sf_log.txt" --- "a/sf_log.txt" +++ "b/sf_log.txt" @@ -1,32 +1,39 @@ -[2023-09-25 20:18:34,219][108279] Saving configuration to ./train_atari/atari_bowling/config.json... -[2023-09-25 20:18:34,554][108279] Rollout worker 0 uses device cpu -[2023-09-25 20:18:34,555][108279] Rollout worker 1 uses device cpu -[2023-09-25 20:18:34,555][108279] Rollout worker 2 uses device cpu -[2023-09-25 20:18:34,556][108279] Rollout worker 3 uses device cpu -[2023-09-25 20:18:34,556][108279] Rollout worker 4 uses device cpu -[2023-09-25 20:18:34,556][108279] Rollout worker 5 uses device cpu -[2023-09-25 20:18:34,557][108279] Rollout worker 6 uses device cpu -[2023-09-25 20:18:34,557][108279] Rollout worker 7 uses device cpu -[2023-09-25 20:18:34,558][108279] In synchronous mode, we only accumulate one batch. Setting num_batches_to_accumulate to 1 -[2023-09-25 20:18:34,605][108279] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-09-25 20:18:34,606][108279] InferenceWorker_p0-w0: min num requests: 1 -[2023-09-25 20:18:34,609][108279] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-09-25 20:18:34,609][108279] InferenceWorker_p1-w0: min num requests: 1 -[2023-09-25 20:18:34,631][108279] Starting all processes... -[2023-09-25 20:18:34,632][108279] Starting process learner_proc0 -[2023-09-25 20:18:36,225][108279] Starting process learner_proc1 -[2023-09-25 20:18:36,230][108926] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-09-25 20:18:36,230][108926] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 -[2023-09-25 20:18:36,248][108926] Num visible devices: 1 -[2023-09-25 20:18:36,267][108926] Starting seed is not provided -[2023-09-25 20:18:36,267][108926] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-09-25 20:18:36,267][108926] Initializing actor-critic model on device cuda:0 -[2023-09-25 20:18:36,267][108926] RunningMeanStd input shape: (4, 84, 84) -[2023-09-25 20:18:36,268][108926] RunningMeanStd input shape: (1,) -[2023-09-25 20:18:36,280][108926] ConvEncoder: input_channels=4 -[2023-09-25 20:18:36,441][108926] Conv encoder output size: 512 -[2023-09-25 20:18:36,443][108926] Created Actor Critic model with architecture: -[2023-09-25 20:18:36,443][108926] ActorCriticSharedWeights( +[2023-10-09 12:13:27,274][85186] Saving configuration to ./train_atari/atari_bowling_APPO/config.json... +[2023-10-09 12:13:27,591][85186] Rollout worker 0 uses device cpu +[2023-10-09 12:13:27,592][85186] Rollout worker 1 uses device cpu +[2023-10-09 12:13:27,593][85186] Rollout worker 2 uses device cpu +[2023-10-09 12:13:27,593][85186] Rollout worker 3 uses device cpu +[2023-10-09 12:13:27,594][85186] Rollout worker 4 uses device cpu +[2023-10-09 12:13:27,594][85186] Rollout worker 5 uses device cpu +[2023-10-09 12:13:27,594][85186] Rollout worker 6 uses device cpu +[2023-10-09 12:13:27,595][85186] Rollout worker 7 uses device cpu +[2023-10-09 12:13:27,595][85186] Rollout worker 8 uses device cpu +[2023-10-09 12:13:27,596][85186] Rollout worker 9 uses device cpu +[2023-10-09 12:13:27,596][85186] Rollout worker 10 uses device cpu +[2023-10-09 12:13:27,597][85186] Rollout worker 11 uses device cpu +[2023-10-09 12:13:27,597][85186] Rollout worker 12 uses device cpu +[2023-10-09 12:13:27,597][85186] Rollout worker 13 uses device cpu +[2023-10-09 12:13:27,598][85186] Rollout worker 14 uses device cpu +[2023-10-09 12:13:27,598][85186] Rollout worker 15 uses device cpu +[2023-10-09 12:13:27,879][85186] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-10-09 12:13:27,879][85186] InferenceWorker_p0-w0: min num requests: 2 +[2023-10-09 12:13:27,882][85186] Using GPUs [1] for process 1 (actually maps to GPUs [1]) +[2023-10-09 12:13:27,883][85186] InferenceWorker_p1-w0: min num requests: 2 +[2023-10-09 12:13:27,929][85186] Starting all processes... +[2023-10-09 12:13:27,930][85186] Starting process learner_proc0 +[2023-10-09 12:13:29,647][85186] Starting process learner_proc1 +[2023-10-09 12:13:29,651][85763] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-10-09 12:13:29,651][85763] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 +[2023-10-09 12:13:29,669][85763] Num visible devices: 1 +[2023-10-09 12:13:29,684][85763] Setting fixed seed 1234 +[2023-10-09 12:13:29,685][85763] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-10-09 12:13:29,686][85763] Initializing actor-critic model on device cuda:0 +[2023-10-09 12:13:29,686][85763] RunningMeanStd input shape: (4, 84, 84) +[2023-10-09 12:13:29,687][85763] RunningMeanStd input shape: (1,) +[2023-10-09 12:13:29,697][85763] ConvEncoder: input_channels=4 +[2023-10-09 12:13:29,879][85763] Conv encoder output size: 512 +[2023-10-09 12:13:29,881][85763] Created Actor Critic model with architecture: +[2023-10-09 12:13:29,881][85763] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( @@ -67,35 +74,41 @@ (distribution_linear): Linear(in_features=512, out_features=6, bias=True) ) ) -[2023-09-25 20:18:37,021][108926] Using optimizer -[2023-09-25 20:18:37,021][108926] No checkpoints found -[2023-09-25 20:18:37,021][108926] Did not load from checkpoint, starting from scratch! -[2023-09-25 20:18:37,022][108926] Initialized policy 0 weights for model version 0 -[2023-09-25 20:18:37,023][108926] LearnerWorker_p0 finished initialization! -[2023-09-25 20:18:37,024][108926] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-09-25 20:18:37,819][108279] Starting all processes... -[2023-09-25 20:18:37,823][109025] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-09-25 20:18:37,823][109025] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 -[2023-09-25 20:18:37,826][108279] Starting process inference_proc0-0 -[2023-09-25 20:18:37,826][108279] Starting process inference_proc1-0 -[2023-09-25 20:18:37,827][108279] Starting process rollout_proc0 -[2023-09-25 20:18:37,827][108279] Starting process rollout_proc1 -[2023-09-25 20:18:37,841][109025] Num visible devices: 1 -[2023-09-25 20:18:37,827][108279] Starting process rollout_proc2 -[2023-09-25 20:18:37,828][108279] Starting process rollout_proc3 -[2023-09-25 20:18:37,866][109025] Starting seed is not provided -[2023-09-25 20:18:37,866][109025] Using GPUs [0] for process 1 (actually maps to GPUs [1]) -[2023-09-25 20:18:37,867][109025] Initializing actor-critic model on device cuda:0 -[2023-09-25 20:18:37,867][109025] RunningMeanStd input shape: (4, 84, 84) -[2023-09-25 20:18:37,868][109025] RunningMeanStd input shape: (1,) -[2023-09-25 20:18:37,835][108279] Starting process rollout_proc4 -[2023-09-25 20:18:37,835][108279] Starting process rollout_proc5 -[2023-09-25 20:18:37,837][108279] Starting process rollout_proc6 -[2023-09-25 20:18:37,838][108279] Starting process rollout_proc7 -[2023-09-25 20:18:37,881][109025] ConvEncoder: input_channels=4 -[2023-09-25 20:18:38,232][109025] Conv encoder output size: 512 -[2023-09-25 20:18:38,234][109025] Created Actor Critic model with architecture: -[2023-09-25 20:18:38,234][109025] ActorCriticSharedWeights( +[2023-10-09 12:13:30,445][85763] Using optimizer +[2023-10-09 12:13:30,446][85763] No checkpoints found +[2023-10-09 12:13:30,446][85763] Did not load from checkpoint, starting from scratch! +[2023-10-09 12:13:30,446][85763] Initialized policy 0 weights for model version 0 +[2023-10-09 12:13:30,447][85763] LearnerWorker_p0 finished initialization! +[2023-10-09 12:13:30,448][85763] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-10-09 12:13:31,400][85186] Starting all processes... +[2023-10-09 12:13:31,404][85963] Using GPUs [1] for process 1 (actually maps to GPUs [1]) +[2023-10-09 12:13:31,404][85963] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for learning process 1 +[2023-10-09 12:13:31,409][85186] Starting process inference_proc0-0 +[2023-10-09 12:13:31,409][85186] Starting process inference_proc1-0 +[2023-10-09 12:13:31,409][85186] Starting process rollout_proc0 +[2023-10-09 12:13:31,422][85963] Num visible devices: 1 +[2023-10-09 12:13:31,409][85186] Starting process rollout_proc1 +[2023-10-09 12:13:31,410][85186] Starting process rollout_proc2 +[2023-10-09 12:13:31,442][85963] Setting fixed seed 1234 +[2023-10-09 12:13:31,410][85186] Starting process rollout_proc3 +[2023-10-09 12:13:31,444][85963] Using GPUs [0] for process 1 (actually maps to GPUs [1]) +[2023-10-09 12:13:31,444][85963] Initializing actor-critic model on device cuda:0 +[2023-10-09 12:13:31,445][85963] RunningMeanStd input shape: (4, 84, 84) +[2023-10-09 12:13:31,445][85963] RunningMeanStd input shape: (1,) +[2023-10-09 12:13:31,410][85186] Starting process rollout_proc4 +[2023-10-09 12:13:31,415][85186] Starting process rollout_proc5 +[2023-10-09 12:13:31,416][85186] Starting process rollout_proc6 +[2023-10-09 12:13:31,417][85186] Starting process rollout_proc7 +[2023-10-09 12:13:31,421][85186] Starting process rollout_proc8 +[2023-10-09 12:13:31,421][85186] Starting process rollout_proc9 +[2023-10-09 12:13:31,423][85186] Starting process rollout_proc10 +[2023-10-09 12:13:31,425][85186] Starting process rollout_proc11 +[2023-10-09 12:13:31,425][85186] Starting process rollout_proc12 +[2023-10-09 12:13:31,465][85963] ConvEncoder: input_channels=4 +[2023-10-09 12:13:31,426][85186] Starting process rollout_proc13 +[2023-10-09 12:13:31,913][85963] Conv encoder output size: 512 +[2023-10-09 12:13:31,916][85963] Created Actor Critic model with architecture: +[2023-10-09 12:13:31,916][85963] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( @@ -136,2050 +149,25819 @@ (distribution_linear): Linear(in_features=512, out_features=6, bias=True) ) ) -[2023-09-25 20:18:38,816][109025] Using optimizer -[2023-09-25 20:18:38,817][109025] No checkpoints found -[2023-09-25 20:18:38,817][109025] Did not load from checkpoint, starting from scratch! -[2023-09-25 20:18:38,817][109025] Initialized policy 1 weights for model version 0 -[2023-09-25 20:18:38,819][109025] LearnerWorker_p1 finished initialization! -[2023-09-25 20:18:38,819][109025] Using GPUs [0] for process 1 (actually maps to GPUs [1]) -[2023-09-25 20:18:39,798][109261] Worker 3 uses CPU cores [12, 13, 14, 15] -[2023-09-25 20:18:39,816][109259] Worker 1 uses CPU cores [4, 5, 6, 7] -[2023-09-25 20:18:39,820][109225] Using GPUs [0] for process 0 (actually maps to GPUs [0]) -[2023-09-25 20:18:39,821][109225] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 -[2023-09-25 20:18:39,830][109264] Worker 5 uses CPU cores [20, 21, 22, 23] -[2023-09-25 20:18:39,839][109225] Num visible devices: 1 -[2023-09-25 20:18:39,906][109262] Worker 4 uses CPU cores [16, 17, 18, 19] -[2023-09-25 20:18:39,906][109265] Worker 6 uses CPU cores [24, 25, 26, 27] -[2023-09-25 20:18:39,941][109227] Worker 0 uses CPU cores [0, 1, 2, 3] -[2023-09-25 20:18:39,952][109224] Using GPUs [1] for process 1 (actually maps to GPUs [1]) -[2023-09-25 20:18:39,952][109224] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 -[2023-09-25 20:18:39,971][109224] Num visible devices: 1 -[2023-09-25 20:18:39,975][109263] Worker 2 uses CPU cores [8, 9, 10, 11] -[2023-09-25 20:18:40,009][109266] Worker 7 uses CPU cores [28, 29, 30, 31] -[2023-09-25 20:18:40,432][109225] RunningMeanStd input shape: (4, 84, 84) -[2023-09-25 20:18:40,433][109225] RunningMeanStd input shape: (1,) -[2023-09-25 20:18:40,443][109225] ConvEncoder: input_channels=4 -[2023-09-25 20:18:40,470][108279] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) -[2023-09-25 20:18:40,542][109225] Conv encoder output size: 512 -[2023-09-25 20:18:40,546][109224] RunningMeanStd input shape: (4, 84, 84) -[2023-09-25 20:18:40,546][109224] RunningMeanStd input shape: (1,) -[2023-09-25 20:18:40,547][108279] Inference worker 0-0 is ready! -[2023-09-25 20:18:40,557][109224] ConvEncoder: input_channels=4 -[2023-09-25 20:18:40,655][109224] Conv encoder output size: 512 -[2023-09-25 20:18:40,660][108279] Inference worker 1-0 is ready! -[2023-09-25 20:18:40,661][108279] All inference workers are ready! Signal rollout workers to start! -[2023-09-25 20:18:41,106][109264] Decorrelating experience for 0 frames... -[2023-09-25 20:18:41,108][109262] Decorrelating experience for 0 frames... -[2023-09-25 20:18:41,111][109227] Decorrelating experience for 0 frames... -[2023-09-25 20:18:41,111][109261] Decorrelating experience for 0 frames... -[2023-09-25 20:18:41,120][109266] Decorrelating experience for 0 frames... -[2023-09-25 20:18:41,155][109259] Decorrelating experience for 0 frames... -[2023-09-25 20:18:41,189][109265] Decorrelating experience for 0 frames... -[2023-09-25 20:18:41,196][109263] Decorrelating experience for 0 frames... -[2023-09-25 20:18:45,470][108279] Fps is (10 sec: 1638.4, 60 sec: 1638.4, 300 sec: 1638.4). Total num frames: 8192. Throughput: 0: 204.8, 1: 204.8. Samples: 2048. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:18:50,470][108279] Fps is (10 sec: 3276.9, 60 sec: 3276.9, 300 sec: 3276.9). Total num frames: 32768. Throughput: 0: 409.6, 1: 409.6. Samples: 8192. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:18:54,593][108279] Heartbeat connected on Batcher_0 -[2023-09-25 20:18:54,599][108279] Heartbeat connected on Batcher_1 -[2023-09-25 20:18:54,613][108279] Heartbeat connected on RolloutWorker_w0 -[2023-09-25 20:18:54,615][108279] Heartbeat connected on RolloutWorker_w1 -[2023-09-25 20:18:54,617][108279] Heartbeat connected on RolloutWorker_w2 -[2023-09-25 20:18:54,620][108279] Heartbeat connected on RolloutWorker_w3 -[2023-09-25 20:18:54,623][108279] Heartbeat connected on RolloutWorker_w4 -[2023-09-25 20:18:54,626][108279] Heartbeat connected on RolloutWorker_w5 -[2023-09-25 20:18:54,628][108279] Heartbeat connected on RolloutWorker_w6 -[2023-09-25 20:18:54,631][108279] Heartbeat connected on RolloutWorker_w7 -[2023-09-25 20:18:54,645][108279] Heartbeat connected on InferenceWorker_p1-w0 -[2023-09-25 20:18:54,653][108279] Heartbeat connected on InferenceWorker_p0-w0 -[2023-09-25 20:18:54,660][108279] Heartbeat connected on LearnerWorker_p0 -[2023-09-25 20:18:54,701][108279] Heartbeat connected on LearnerWorker_p1 -[2023-09-25 20:18:55,470][108279] Fps is (10 sec: 5734.5, 60 sec: 4369.1, 300 sec: 4369.1). Total num frames: 65536. Throughput: 0: 431.0, 1: 439.1. Samples: 13051. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:18:55,470][108279] Avg episode reward: [(0, '7.500'), (1, '6.250')] -[2023-09-25 20:18:57,192][109225] Updated weights for policy 0, policy_version 160 (0.0015) -[2023-09-25 20:18:57,192][109224] Updated weights for policy 1, policy_version 160 (0.0019) -[2023-09-25 20:19:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 4915.2, 300 sec: 4915.2). Total num frames: 98304. Throughput: 0: 569.4, 1: 575.3. Samples: 22893. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:00,470][108279] Avg episode reward: [(0, '7.500'), (1, '6.250')] -[2023-09-25 20:19:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 5242.9, 300 sec: 5242.9). Total num frames: 131072. Throughput: 0: 655.5, 1: 658.7. Samples: 32856. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:05,471][108279] Avg episode reward: [(0, '7.375'), (1, '6.500')] -[2023-09-25 20:19:09,581][109225] Updated weights for policy 0, policy_version 320 (0.0015) -[2023-09-25 20:19:09,582][109224] Updated weights for policy 1, policy_version 320 (0.0016) -[2023-09-25 20:19:10,470][108279] Fps is (10 sec: 6553.4, 60 sec: 5461.3, 300 sec: 5461.3). Total num frames: 163840. Throughput: 0: 629.0, 1: 632.9. Samples: 37858. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:10,471][108279] Avg episode reward: [(0, '7.375'), (1, '6.500')] -[2023-09-25 20:19:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 5617.4, 300 sec: 5617.4). Total num frames: 196608. Throughput: 0: 679.5, 1: 682.3. Samples: 47663. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:15,471][108279] Avg episode reward: [(0, '7.917'), (1, '7.000')] -[2023-09-25 20:19:20,470][108279] Fps is (10 sec: 6553.8, 60 sec: 5734.4, 300 sec: 5734.4). Total num frames: 229376. Throughput: 0: 716.8, 1: 718.3. Samples: 57404. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:19:20,470][108279] Avg episode reward: [(0, '7.917'), (1, '7.000')] -[2023-09-25 20:19:20,471][108926] Saving new best policy, reward=7.917! -[2023-09-25 20:19:20,471][109025] Saving new best policy, reward=7.000! -[2023-09-25 20:19:22,072][109224] Updated weights for policy 1, policy_version 480 (0.0017) -[2023-09-25 20:19:22,073][109225] Updated weights for policy 0, policy_version 480 (0.0016) -[2023-09-25 20:19:25,470][108279] Fps is (10 sec: 6553.8, 60 sec: 5825.5, 300 sec: 5825.5). Total num frames: 262144. Throughput: 0: 693.3, 1: 695.7. Samples: 62505. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:25,470][108279] Avg episode reward: [(0, '7.867'), (1, '7.400')] -[2023-09-25 20:19:25,471][109025] Saving new best policy, reward=7.400! -[2023-09-25 20:19:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 5898.2, 300 sec: 5898.2). Total num frames: 294912. Throughput: 0: 778.2, 1: 780.4. Samples: 72188. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-09-25 20:19:30,471][108279] Avg episode reward: [(0, '7.938'), (1, '7.562')] -[2023-09-25 20:19:30,475][108926] Saving new best policy, reward=7.938! -[2023-09-25 20:19:30,475][109025] Saving new best policy, reward=7.562! -[2023-09-25 20:19:34,655][109225] Updated weights for policy 0, policy_version 640 (0.0014) -[2023-09-25 20:19:34,659][109224] Updated weights for policy 1, policy_version 640 (0.0018) -[2023-09-25 20:19:35,470][108279] Fps is (10 sec: 6553.3, 60 sec: 5957.8, 300 sec: 5957.8). Total num frames: 327680. Throughput: 0: 819.2, 1: 819.3. Samples: 81925. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:35,471][108279] Avg episode reward: [(0, '7.938'), (1, '7.562')] -[2023-09-25 20:19:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6007.5, 300 sec: 6007.5). Total num frames: 360448. Throughput: 0: 819.6, 1: 819.5. Samples: 86810. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:40,471][108279] Avg episode reward: [(0, '8.350'), (1, '8.000')] -[2023-09-25 20:19:40,472][108926] Saving new best policy, reward=8.350! -[2023-09-25 20:19:40,472][109025] Saving new best policy, reward=8.000! -[2023-09-25 20:19:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6049.5). Total num frames: 393216. Throughput: 0: 817.6, 1: 817.6. Samples: 96478. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:45,471][108279] Avg episode reward: [(0, '8.350'), (1, '8.000')] -[2023-09-25 20:19:47,239][109224] Updated weights for policy 1, policy_version 800 (0.0016) -[2023-09-25 20:19:47,239][109225] Updated weights for policy 0, policy_version 800 (0.0017) -[2023-09-25 20:19:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6085.5). Total num frames: 425984. Throughput: 0: 819.1, 1: 817.4. Samples: 106497. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:19:50,470][108279] Avg episode reward: [(0, '8.625'), (1, '8.250')] -[2023-09-25 20:19:50,471][108926] Saving new best policy, reward=8.625! -[2023-09-25 20:19:50,471][109025] Saving new best policy, reward=8.250! -[2023-09-25 20:19:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6116.7). Total num frames: 458752. Throughput: 0: 817.4, 1: 817.5. Samples: 111426. Policy #0 lag: (min: 8.0, avg: 8.0, max: 8.0) -[2023-09-25 20:19:55,471][108279] Avg episode reward: [(0, '8.625'), (1, '8.250')] -[2023-09-25 20:20:00,051][109225] Updated weights for policy 0, policy_version 960 (0.0018) -[2023-09-25 20:20:00,051][109224] Updated weights for policy 1, policy_version 960 (0.0017) -[2023-09-25 20:20:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6144.0). Total num frames: 491520. Throughput: 0: 814.1, 1: 811.9. Samples: 120832. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:20:00,471][108279] Avg episode reward: [(0, '8.786'), (1, '8.464')] -[2023-09-25 20:20:00,475][108926] Saving new best policy, reward=8.786! -[2023-09-25 20:20:00,475][109025] Saving new best policy, reward=8.464! -[2023-09-25 20:20:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6168.1). Total num frames: 524288. Throughput: 0: 811.7, 1: 812.9. Samples: 130513. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:20:05,470][108279] Avg episode reward: [(0, '8.786'), (1, '8.464')] -[2023-09-25 20:20:10,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6189.5). Total num frames: 557056. Throughput: 0: 808.6, 1: 806.6. Samples: 135187. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:20:10,470][108279] Avg episode reward: [(0, '8.875'), (1, '8.656')] -[2023-09-25 20:20:10,471][109025] Saving new best policy, reward=8.656! -[2023-09-25 20:20:10,471][108926] Saving new best policy, reward=8.875! -[2023-09-25 20:20:12,698][109225] Updated weights for policy 0, policy_version 1120 (0.0018) -[2023-09-25 20:20:12,698][109224] Updated weights for policy 1, policy_version 1120 (0.0015) -[2023-09-25 20:20:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6208.7). Total num frames: 589824. Throughput: 0: 810.9, 1: 810.9. Samples: 145169. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:20:15,471][108279] Avg episode reward: [(0, '8.875'), (1, '8.656')] -[2023-09-25 20:20:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6225.9). Total num frames: 622592. Throughput: 0: 808.1, 1: 810.1. Samples: 154743. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:20:20,470][108279] Avg episode reward: [(0, '9.000'), (1, '8.806')] -[2023-09-25 20:20:20,471][108926] Saving new best policy, reward=9.000! -[2023-09-25 20:20:20,471][109025] Saving new best policy, reward=8.806! -[2023-09-25 20:20:25,316][109224] Updated weights for policy 1, policy_version 1280 (0.0018) -[2023-09-25 20:20:25,316][109225] Updated weights for policy 0, policy_version 1280 (0.0017) -[2023-09-25 20:20:25,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6241.5). Total num frames: 655360. Throughput: 0: 811.7, 1: 809.0. Samples: 159744. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:20:25,470][108279] Avg episode reward: [(0, '9.000'), (1, '8.806')] -[2023-09-25 20:20:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6255.7). Total num frames: 688128. Throughput: 0: 810.8, 1: 810.3. Samples: 169427. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:20:30,470][108279] Avg episode reward: [(0, '9.100'), (1, '8.925')] -[2023-09-25 20:20:30,474][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000001344_344064.pth... -[2023-09-25 20:20:30,474][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000001344_344064.pth... -[2023-09-25 20:20:30,510][109025] Saving new best policy, reward=8.925! -[2023-09-25 20:20:30,512][108926] Saving new best policy, reward=9.100! -[2023-09-25 20:20:35,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6268.7). Total num frames: 720896. Throughput: 0: 806.3, 1: 808.6. Samples: 179167. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:20:35,471][108279] Avg episode reward: [(0, '9.100'), (1, '8.925')] -[2023-09-25 20:20:37,822][109224] Updated weights for policy 1, policy_version 1440 (0.0019) -[2023-09-25 20:20:37,822][109225] Updated weights for policy 0, policy_version 1440 (0.0018) -[2023-09-25 20:20:40,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6280.5). Total num frames: 753664. Throughput: 0: 810.4, 1: 808.6. Samples: 184282. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:20:40,471][108279] Avg episode reward: [(0, '9.182'), (1, '9.023')] -[2023-09-25 20:20:40,473][108926] Saving new best policy, reward=9.182! -[2023-09-25 20:20:40,473][109025] Saving new best policy, reward=9.023! -[2023-09-25 20:20:45,470][108279] Fps is (10 sec: 6144.1, 60 sec: 6485.3, 300 sec: 6258.7). Total num frames: 782336. Throughput: 0: 811.7, 1: 812.4. Samples: 193917. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:20:45,471][108279] Avg episode reward: [(0, '9.182'), (1, '9.023')] -[2023-09-25 20:20:50,470][108279] Fps is (10 sec: 5734.6, 60 sec: 6417.1, 300 sec: 6238.5). Total num frames: 811008. Throughput: 0: 810.7, 1: 810.8. Samples: 203482. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-09-25 20:20:50,470][108279] Avg episode reward: [(0, '9.250'), (1, '9.065')] -[2023-09-25 20:20:50,480][109025] Saving new best policy, reward=9.065! -[2023-09-25 20:20:50,490][108926] Saving new best policy, reward=9.250! -[2023-09-25 20:20:50,493][109224] Updated weights for policy 1, policy_version 1600 (0.0017) -[2023-09-25 20:20:50,493][109225] Updated weights for policy 0, policy_version 1600 (0.0019) -[2023-09-25 20:20:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6485.3, 300 sec: 6280.5). Total num frames: 847872. Throughput: 0: 814.6, 1: 816.1. Samples: 208571. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:20:55,471][108279] Avg episode reward: [(0, '9.250'), (1, '9.104')] -[2023-09-25 20:20:55,472][109025] Saving new best policy, reward=9.104! -[2023-09-25 20:21:00,470][108279] Fps is (10 sec: 7372.7, 60 sec: 6553.6, 300 sec: 6319.5). Total num frames: 884736. Throughput: 0: 813.8, 1: 813.8. Samples: 218410. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:21:00,471][108279] Avg episode reward: [(0, '9.250'), (1, '9.104')] -[2023-09-25 20:21:02,949][109224] Updated weights for policy 1, policy_version 1760 (0.0016) -[2023-09-25 20:21:02,951][109225] Updated weights for policy 0, policy_version 1760 (0.0017) -[2023-09-25 20:21:05,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6553.6, 300 sec: 6327.6). Total num frames: 917504. Throughput: 0: 814.2, 1: 815.2. Samples: 228067. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:21:05,471][108279] Avg episode reward: [(0, '9.308'), (1, '9.135')] -[2023-09-25 20:21:05,472][108926] Saving new best policy, reward=9.308! -[2023-09-25 20:21:05,472][109025] Saving new best policy, reward=9.135! -[2023-09-25 20:21:10,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6335.2). Total num frames: 950272. Throughput: 0: 815.4, 1: 818.2. Samples: 233258. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:21:10,470][108279] Avg episode reward: [(0, '9.308'), (1, '9.135')] -[2023-09-25 20:21:15,449][109225] Updated weights for policy 0, policy_version 1920 (0.0017) -[2023-09-25 20:21:15,450][109224] Updated weights for policy 1, policy_version 1920 (0.0016) -[2023-09-25 20:21:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6342.2). Total num frames: 983040. Throughput: 0: 816.1, 1: 816.6. Samples: 242899. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:21:15,471][108279] Avg episode reward: [(0, '9.339'), (1, '9.179')] -[2023-09-25 20:21:15,474][108926] Saving new best policy, reward=9.339! -[2023-09-25 20:21:15,474][109025] Saving new best policy, reward=9.179! -[2023-09-25 20:21:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6348.8). Total num frames: 1015808. Throughput: 0: 816.6, 1: 817.0. Samples: 252678. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:21:20,471][108279] Avg episode reward: [(0, '9.339'), (1, '9.179')] -[2023-09-25 20:21:25,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6355.0). Total num frames: 1048576. Throughput: 0: 816.3, 1: 818.1. Samples: 257828. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:21:25,471][108279] Avg episode reward: [(0, '9.383'), (1, '9.233')] -[2023-09-25 20:21:25,472][108926] Saving new best policy, reward=9.383! -[2023-09-25 20:21:25,472][109025] Saving new best policy, reward=9.233! -[2023-09-25 20:21:27,893][109224] Updated weights for policy 1, policy_version 2080 (0.0016) -[2023-09-25 20:21:27,894][109225] Updated weights for policy 0, policy_version 2080 (0.0017) -[2023-09-25 20:21:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6360.9). Total num frames: 1081344. Throughput: 0: 817.7, 1: 820.3. Samples: 267628. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:21:30,470][108279] Avg episode reward: [(0, '9.383'), (1, '9.233')] -[2023-09-25 20:21:35,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6319.5). Total num frames: 1105920. Throughput: 0: 818.9, 1: 818.6. Samples: 277167. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:21:35,471][108279] Avg episode reward: [(0, '9.406'), (1, '9.281')] -[2023-09-25 20:21:35,472][108926] Saving new best policy, reward=9.406! -[2023-09-25 20:21:35,501][109025] Saving new best policy, reward=9.281! -[2023-09-25 20:21:40,459][109225] Updated weights for policy 0, policy_version 2240 (0.0018) -[2023-09-25 20:21:40,459][109224] Updated weights for policy 1, policy_version 2240 (0.0017) -[2023-09-25 20:21:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6371.6). Total num frames: 1146880. Throughput: 0: 819.1, 1: 819.9. Samples: 282327. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:21:40,471][108279] Avg episode reward: [(0, '9.406'), (1, '9.281')] -[2023-09-25 20:21:45,470][108279] Fps is (10 sec: 7373.0, 60 sec: 6621.9, 300 sec: 6376.5). Total num frames: 1179648. Throughput: 0: 819.5, 1: 819.9. Samples: 292184. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:21:45,470][108279] Avg episode reward: [(0, '9.441'), (1, '9.324')] -[2023-09-25 20:21:45,474][109025] Saving new best policy, reward=9.324! -[2023-09-25 20:21:45,474][108926] Saving new best policy, reward=9.441! -[2023-09-25 20:21:50,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6553.6, 300 sec: 6338.0). Total num frames: 1204224. Throughput: 0: 818.0, 1: 817.1. Samples: 301647. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:21:50,471][108279] Avg episode reward: [(0, '9.441'), (1, '9.324')] -[2023-09-25 20:21:53,080][109225] Updated weights for policy 0, policy_version 2400 (0.0019) -[2023-09-25 20:21:53,080][109224] Updated weights for policy 1, policy_version 2400 (0.0020) -[2023-09-25 20:21:55,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6485.3, 300 sec: 6343.5). Total num frames: 1236992. Throughput: 0: 816.0, 1: 815.8. Samples: 306687. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:21:55,471][108279] Avg episode reward: [(0, '9.472'), (1, '9.361')] -[2023-09-25 20:21:55,532][108926] Saving new best policy, reward=9.472! -[2023-09-25 20:21:55,543][109025] Saving new best policy, reward=9.361! -[2023-09-25 20:22:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6348.8). Total num frames: 1269760. Throughput: 0: 818.6, 1: 818.8. Samples: 316585. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:22:00,471][108279] Avg episode reward: [(0, '9.472'), (1, '9.361')] -[2023-09-25 20:22:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6353.8). Total num frames: 1302528. Throughput: 0: 817.1, 1: 817.2. Samples: 326222. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:22:05,471][108279] Avg episode reward: [(0, '9.487'), (1, '9.382')] -[2023-09-25 20:22:05,540][108926] Saving new best policy, reward=9.487! -[2023-09-25 20:22:05,544][109025] Saving new best policy, reward=9.382! -[2023-09-25 20:22:05,547][109224] Updated weights for policy 1, policy_version 2560 (0.0018) -[2023-09-25 20:22:05,547][109225] Updated weights for policy 0, policy_version 2560 (0.0017) -[2023-09-25 20:22:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6358.6). Total num frames: 1335296. Throughput: 0: 816.7, 1: 817.1. Samples: 331350. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:22:10,471][108279] Avg episode reward: [(0, '9.487'), (1, '9.382')] -[2023-09-25 20:22:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.0, 300 sec: 6363.1). Total num frames: 1368064. Throughput: 0: 815.4, 1: 815.0. Samples: 340999. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:22:15,471][108279] Avg episode reward: [(0, '9.512'), (1, '9.412')] -[2023-09-25 20:22:15,544][108926] Saving new best policy, reward=9.512! -[2023-09-25 20:22:15,596][109025] Saving new best policy, reward=9.412! -[2023-09-25 20:22:18,138][109224] Updated weights for policy 1, policy_version 2720 (0.0017) -[2023-09-25 20:22:18,139][109225] Updated weights for policy 0, policy_version 2720 (0.0016) -[2023-09-25 20:22:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6367.4). Total num frames: 1400832. Throughput: 0: 815.1, 1: 815.5. Samples: 350541. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:22:20,471][108279] Avg episode reward: [(0, '9.512'), (1, '9.412')] -[2023-09-25 20:22:25,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6417.1, 300 sec: 6371.6). Total num frames: 1433600. Throughput: 0: 813.3, 1: 812.9. Samples: 355507. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:22:25,470][108279] Avg episode reward: [(0, '9.512'), (1, '9.429')] -[2023-09-25 20:22:25,471][109025] Saving new best policy, reward=9.429! -[2023-09-25 20:22:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6375.5). Total num frames: 1466368. Throughput: 0: 810.2, 1: 810.1. Samples: 365095. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:22:30,470][108279] Avg episode reward: [(0, '9.512'), (1, '9.429')] -[2023-09-25 20:22:30,475][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000002864_733184.pth... -[2023-09-25 20:22:30,475][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000002864_733184.pth... -[2023-09-25 20:22:30,802][109225] Updated weights for policy 0, policy_version 2880 (0.0018) -[2023-09-25 20:22:30,802][109224] Updated weights for policy 1, policy_version 2880 (0.0017) -[2023-09-25 20:22:35,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6379.3). Total num frames: 1499136. Throughput: 0: 813.7, 1: 811.7. Samples: 374793. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:22:35,471][108279] Avg episode reward: [(0, '9.511'), (1, '9.437')] -[2023-09-25 20:22:35,472][109025] Saving new best policy, reward=9.437! -[2023-09-25 20:22:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6382.9). Total num frames: 1531904. Throughput: 0: 813.1, 1: 813.2. Samples: 379871. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:22:40,470][108279] Avg episode reward: [(0, '9.511'), (1, '9.432')] -[2023-09-25 20:22:43,282][109224] Updated weights for policy 1, policy_version 3040 (0.0019) -[2023-09-25 20:22:43,282][109225] Updated weights for policy 0, policy_version 3040 (0.0019) -[2023-09-25 20:22:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6386.4). Total num frames: 1564672. Throughput: 0: 813.3, 1: 812.8. Samples: 389760. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:22:45,471][108279] Avg episode reward: [(0, '9.516'), (1, '9.432')] -[2023-09-25 20:22:45,476][108926] Saving new best policy, reward=9.516! -[2023-09-25 20:22:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6389.8). Total num frames: 1597440. Throughput: 0: 814.0, 1: 811.4. Samples: 399365. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:22:50,470][108279] Avg episode reward: [(0, '9.522'), (1, '9.457')] -[2023-09-25 20:22:50,471][108926] Saving new best policy, reward=9.522! -[2023-09-25 20:22:50,471][109025] Saving new best policy, reward=9.457! -[2023-09-25 20:22:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6393.0). Total num frames: 1630208. Throughput: 0: 809.9, 1: 809.2. Samples: 404211. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:22:55,471][108279] Avg episode reward: [(0, '9.522'), (1, '9.457')] -[2023-09-25 20:22:55,966][109225] Updated weights for policy 0, policy_version 3200 (0.0015) -[2023-09-25 20:22:55,966][109224] Updated weights for policy 1, policy_version 3200 (0.0019) -[2023-09-25 20:23:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6396.1). Total num frames: 1662976. Throughput: 0: 810.6, 1: 809.8. Samples: 413917. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:23:00,470][108279] Avg episode reward: [(0, '9.531'), (1, '9.469')] -[2023-09-25 20:23:00,473][108926] Saving new best policy, reward=9.531! -[2023-09-25 20:23:00,474][109025] Saving new best policy, reward=9.469! -[2023-09-25 20:23:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6399.0). Total num frames: 1695744. Throughput: 0: 816.8, 1: 814.2. Samples: 423937. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:23:05,471][108279] Avg episode reward: [(0, '9.531'), (1, '9.469')] -[2023-09-25 20:23:08,438][109225] Updated weights for policy 0, policy_version 3360 (0.0019) -[2023-09-25 20:23:08,438][109224] Updated weights for policy 1, policy_version 3360 (0.0019) -[2023-09-25 20:23:10,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6401.9). Total num frames: 1728512. Throughput: 0: 814.4, 1: 814.8. Samples: 428821. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:23:10,471][108279] Avg episode reward: [(0, '9.550'), (1, '9.450')] -[2023-09-25 20:23:10,472][108926] Saving new best policy, reward=9.550! -[2023-09-25 20:23:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6404.7). Total num frames: 1761280. Throughput: 0: 818.2, 1: 818.5. Samples: 438747. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:23:15,470][108279] Avg episode reward: [(0, '9.550'), (1, '9.450')] -[2023-09-25 20:23:20,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6407.3). Total num frames: 1794048. Throughput: 0: 819.1, 1: 819.0. Samples: 448509. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:23:20,470][108279] Avg episode reward: [(0, '9.640'), (1, '9.550')] -[2023-09-25 20:23:20,471][109025] Saving new best policy, reward=9.550! -[2023-09-25 20:23:20,471][108926] Saving new best policy, reward=9.640! -[2023-09-25 20:23:21,059][109225] Updated weights for policy 0, policy_version 3520 (0.0018) -[2023-09-25 20:23:21,059][109224] Updated weights for policy 1, policy_version 3520 (0.0018) -[2023-09-25 20:23:25,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6409.9). Total num frames: 1826816. Throughput: 0: 814.9, 1: 815.0. Samples: 453216. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:23:25,471][108279] Avg episode reward: [(0, '9.640'), (1, '9.550')] -[2023-09-25 20:23:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6412.4). Total num frames: 1859584. Throughput: 0: 814.1, 1: 814.4. Samples: 463040. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:23:30,470][108279] Avg episode reward: [(0, '9.740'), (1, '9.680')] -[2023-09-25 20:23:30,479][108926] Saving new best policy, reward=9.740! -[2023-09-25 20:23:30,479][109025] Saving new best policy, reward=9.680! -[2023-09-25 20:23:33,436][109224] Updated weights for policy 1, policy_version 3680 (0.0014) -[2023-09-25 20:23:33,437][109225] Updated weights for policy 0, policy_version 3680 (0.0017) -[2023-09-25 20:23:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6414.8). Total num frames: 1892352. Throughput: 0: 819.2, 1: 819.1. Samples: 473089. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:23:35,471][108279] Avg episode reward: [(0, '9.740'), (1, '9.680')] -[2023-09-25 20:23:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 1925120. Throughput: 0: 819.0, 1: 819.0. Samples: 477920. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:23:40,470][108279] Avg episode reward: [(0, '9.780'), (1, '9.740')] -[2023-09-25 20:23:40,471][109025] Saving new best policy, reward=9.740! -[2023-09-25 20:23:40,471][108926] Saving new best policy, reward=9.780! -[2023-09-25 20:23:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 1957888. Throughput: 0: 819.4, 1: 819.4. Samples: 487662. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:23:45,470][108279] Avg episode reward: [(0, '9.780'), (1, '9.740')] -[2023-09-25 20:23:46,050][109225] Updated weights for policy 0, policy_version 3840 (0.0017) -[2023-09-25 20:23:46,050][109224] Updated weights for policy 1, policy_version 3840 (0.0017) -[2023-09-25 20:23:50,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 1990656. Throughput: 0: 817.9, 1: 819.2. Samples: 497607. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:23:50,471][108279] Avg episode reward: [(0, '9.860'), (1, '9.730')] -[2023-09-25 20:23:50,472][108926] Saving new best policy, reward=9.860! -[2023-09-25 20:23:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2023424. Throughput: 0: 815.3, 1: 815.1. Samples: 502191. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:23:55,471][108279] Avg episode reward: [(0, '9.860'), (1, '9.730')] -[2023-09-25 20:23:58,699][109224] Updated weights for policy 1, policy_version 4000 (0.0015) -[2023-09-25 20:23:58,699][109225] Updated weights for policy 0, policy_version 4000 (0.0016) -[2023-09-25 20:24:00,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2056192. Throughput: 0: 815.3, 1: 812.6. Samples: 512001. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:24:00,470][108279] Avg episode reward: [(0, '9.860'), (1, '9.660')] -[2023-09-25 20:24:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2088960. Throughput: 0: 814.5, 1: 816.5. Samples: 521905. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:24:05,470][108279] Avg episode reward: [(0, '9.860'), (1, '9.660')] -[2023-09-25 20:24:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2121728. Throughput: 0: 815.5, 1: 815.5. Samples: 526607. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:24:10,470][108279] Avg episode reward: [(0, '9.860'), (1, '9.640')] -[2023-09-25 20:24:11,209][109224] Updated weights for policy 1, policy_version 4160 (0.0017) -[2023-09-25 20:24:11,209][109225] Updated weights for policy 0, policy_version 4160 (0.0015) -[2023-09-25 20:24:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2154496. Throughput: 0: 818.4, 1: 815.8. Samples: 536577. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:24:15,470][108279] Avg episode reward: [(0, '9.860'), (1, '9.640')] -[2023-09-25 20:24:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2187264. Throughput: 0: 812.2, 1: 813.5. Samples: 546248. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:24:20,471][108279] Avg episode reward: [(0, '9.860'), (1, '9.610')] -[2023-09-25 20:24:23,982][109224] Updated weights for policy 1, policy_version 4320 (0.0018) -[2023-09-25 20:24:23,982][109225] Updated weights for policy 0, policy_version 4320 (0.0018) -[2023-09-25 20:24:25,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2220032. Throughput: 0: 812.2, 1: 809.9. Samples: 550912. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:24:25,472][108279] Avg episode reward: [(0, '9.860'), (1, '9.610')] -[2023-09-25 20:24:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2252800. Throughput: 0: 810.2, 1: 810.3. Samples: 560588. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:24:30,470][108279] Avg episode reward: [(0, '9.860'), (1, '9.560')] -[2023-09-25 20:24:30,478][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000004400_1126400.pth... -[2023-09-25 20:24:30,478][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000004400_1126400.pth... -[2023-09-25 20:24:30,506][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000001344_344064.pth -[2023-09-25 20:24:30,513][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000001344_344064.pth -[2023-09-25 20:24:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2285568. Throughput: 0: 805.0, 1: 806.2. Samples: 570107. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:24:35,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.560')] -[2023-09-25 20:24:35,472][108926] Saving new best policy, reward=9.880! -[2023-09-25 20:24:36,700][109224] Updated weights for policy 1, policy_version 4480 (0.0018) -[2023-09-25 20:24:36,700][109225] Updated weights for policy 0, policy_version 4480 (0.0018) -[2023-09-25 20:24:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2318336. Throughput: 0: 812.3, 1: 812.0. Samples: 575283. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-09-25 20:24:40,470][108279] Avg episode reward: [(0, '9.880'), (1, '9.560')] -[2023-09-25 20:24:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2351104. Throughput: 0: 810.3, 1: 812.9. Samples: 585046. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:24:45,470][108279] Avg episode reward: [(0, '9.880'), (1, '9.560')] -[2023-09-25 20:24:49,115][109224] Updated weights for policy 1, policy_version 4640 (0.0017) -[2023-09-25 20:24:49,115][109225] Updated weights for policy 0, policy_version 4640 (0.0013) -[2023-09-25 20:24:50,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2383872. Throughput: 0: 810.6, 1: 811.1. Samples: 594885. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:24:50,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.510')] -[2023-09-25 20:24:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 2416640. Throughput: 0: 815.4, 1: 814.7. Samples: 599963. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:24:55,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.470')] -[2023-09-25 20:25:00,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 2441216. Throughput: 0: 806.2, 1: 808.6. Samples: 609244. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:25:00,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.450')] -[2023-09-25 20:25:01,874][109225] Updated weights for policy 0, policy_version 4800 (0.0017) -[2023-09-25 20:25:01,874][109224] Updated weights for policy 1, policy_version 4800 (0.0016) -[2023-09-25 20:25:05,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 2473984. Throughput: 0: 806.7, 1: 808.0. Samples: 618911. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:25:05,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.430')] -[2023-09-25 20:25:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 2506752. Throughput: 0: 811.7, 1: 813.6. Samples: 624053. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:25:10,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.430')] -[2023-09-25 20:25:14,421][109224] Updated weights for policy 1, policy_version 4960 (0.0017) -[2023-09-25 20:25:14,421][109225] Updated weights for policy 0, policy_version 4960 (0.0019) -[2023-09-25 20:25:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 2539520. Throughput: 0: 811.8, 1: 812.4. Samples: 633679. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:25:15,470][108279] Avg episode reward: [(0, '9.880'), (1, '9.430')] -[2023-09-25 20:25:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 2572288. Throughput: 0: 813.8, 1: 813.8. Samples: 643349. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:25:20,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.430')] -[2023-09-25 20:25:25,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 2605056. Throughput: 0: 813.5, 1: 813.7. Samples: 648507. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:25:25,471][108279] Avg episode reward: [(0, '9.870'), (1, '9.400')] -[2023-09-25 20:25:26,992][109224] Updated weights for policy 1, policy_version 5120 (0.0019) -[2023-09-25 20:25:26,992][109225] Updated weights for policy 0, policy_version 5120 (0.0017) -[2023-09-25 20:25:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 2637824. Throughput: 0: 812.3, 1: 812.7. Samples: 658171. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:25:30,470][108279] Avg episode reward: [(0, '9.870'), (1, '9.400')] -[2023-09-25 20:25:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 2670592. Throughput: 0: 810.3, 1: 810.0. Samples: 667796. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:25:35,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.370')] -[2023-09-25 20:25:39,656][109224] Updated weights for policy 1, policy_version 5280 (0.0017) -[2023-09-25 20:25:39,658][109225] Updated weights for policy 0, policy_version 5280 (0.0020) -[2023-09-25 20:25:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6511.9). Total num frames: 2703360. Throughput: 0: 809.2, 1: 808.8. Samples: 672772. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-09-25 20:25:40,470][108279] Avg episode reward: [(0, '9.880'), (1, '9.370')] -[2023-09-25 20:25:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 2736128. Throughput: 0: 813.4, 1: 813.7. Samples: 682463. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:25:45,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.270')] -[2023-09-25 20:25:50,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6511.9). Total num frames: 2768896. Throughput: 0: 815.6, 1: 813.3. Samples: 692213. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:25:50,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.270')] -[2023-09-25 20:25:52,431][109225] Updated weights for policy 0, policy_version 5440 (0.0019) -[2023-09-25 20:25:52,431][109224] Updated weights for policy 1, policy_version 5440 (0.0019) -[2023-09-25 20:25:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 2801664. Throughput: 0: 806.5, 1: 806.9. Samples: 696658. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:25:55,471][108279] Avg episode reward: [(0, '9.870'), (1, '9.220')] -[2023-09-25 20:26:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 2834432. Throughput: 0: 811.2, 1: 808.4. Samples: 706561. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:26:00,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.220')] -[2023-09-25 20:26:05,164][109224] Updated weights for policy 1, policy_version 5600 (0.0018) -[2023-09-25 20:26:05,164][109225] Updated weights for policy 0, policy_version 5600 (0.0017) -[2023-09-25 20:26:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 2867200. Throughput: 0: 809.9, 1: 809.1. Samples: 716204. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:26:05,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.150')] -[2023-09-25 20:26:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 2899968. Throughput: 0: 805.5, 1: 803.2. Samples: 720900. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:26:10,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.150')] -[2023-09-25 20:26:15,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 2932736. Throughput: 0: 807.0, 1: 806.6. Samples: 730786. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:26:15,470][108279] Avg episode reward: [(0, '9.880'), (1, '9.140')] -[2023-09-25 20:26:17,776][109224] Updated weights for policy 1, policy_version 5760 (0.0017) -[2023-09-25 20:26:17,776][109225] Updated weights for policy 0, policy_version 5760 (0.0018) -[2023-09-25 20:26:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 2965504. Throughput: 0: 806.2, 1: 806.4. Samples: 740359. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:26:20,471][108279] Avg episode reward: [(0, '9.870'), (1, '9.140')] -[2023-09-25 20:26:25,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 2998272. Throughput: 0: 806.2, 1: 807.0. Samples: 745364. Policy #0 lag: (min: 8.0, avg: 8.0, max: 8.0) -[2023-09-25 20:26:25,471][108279] Avg episode reward: [(0, '9.870'), (1, '9.120')] -[2023-09-25 20:26:30,378][109224] Updated weights for policy 1, policy_version 5920 (0.0017) -[2023-09-25 20:26:30,379][109225] Updated weights for policy 0, policy_version 5920 (0.0017) -[2023-09-25 20:26:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3031040. Throughput: 0: 806.7, 1: 806.9. Samples: 755078. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:26:30,471][108279] Avg episode reward: [(0, '9.880'), (1, '9.120')] -[2023-09-25 20:26:30,482][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000005920_1515520.pth... -[2023-09-25 20:26:30,482][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000005920_1515520.pth... -[2023-09-25 20:26:30,517][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000002864_733184.pth -[2023-09-25 20:26:30,523][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000002864_733184.pth -[2023-09-25 20:26:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 3063808. Throughput: 0: 805.6, 1: 808.0. Samples: 764828. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:26:35,470][108279] Avg episode reward: [(0, '9.880'), (1, '9.040')] -[2023-09-25 20:26:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 3096576. Throughput: 0: 815.1, 1: 814.3. Samples: 769980. Policy #0 lag: (min: 8.0, avg: 8.0, max: 8.0) -[2023-09-25 20:26:40,471][108279] Avg episode reward: [(0, '9.870'), (1, '9.040')] -[2023-09-25 20:26:42,821][109225] Updated weights for policy 0, policy_version 6080 (0.0017) -[2023-09-25 20:26:42,821][109224] Updated weights for policy 1, policy_version 6080 (0.0014) -[2023-09-25 20:26:45,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3129344. Throughput: 0: 812.7, 1: 815.1. Samples: 779811. Policy #0 lag: (min: 8.0, avg: 8.0, max: 8.0) -[2023-09-25 20:26:45,471][108279] Avg episode reward: [(0, '9.870'), (1, '8.940')] -[2023-09-25 20:26:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3162112. Throughput: 0: 814.3, 1: 815.0. Samples: 789522. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:26:50,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.940')] -[2023-09-25 20:26:50,472][108926] Saving new best policy, reward=9.890! -[2023-09-25 20:26:55,293][109225] Updated weights for policy 0, policy_version 6240 (0.0015) -[2023-09-25 20:26:55,293][109224] Updated weights for policy 1, policy_version 6240 (0.0017) -[2023-09-25 20:26:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3194880. Throughput: 0: 818.6, 1: 819.1. Samples: 794599. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:26:55,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.890')] -[2023-09-25 20:27:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3227648. Throughput: 0: 818.8, 1: 818.2. Samples: 804453. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:27:00,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.860')] -[2023-09-25 20:27:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3260416. Throughput: 0: 821.7, 1: 821.5. Samples: 814302. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:27:05,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.800')] -[2023-09-25 20:27:07,778][109225] Updated weights for policy 0, policy_version 6400 (0.0017) -[2023-09-25 20:27:07,779][109224] Updated weights for policy 1, policy_version 6400 (0.0018) -[2023-09-25 20:27:10,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3293184. Throughput: 0: 821.6, 1: 819.2. Samples: 819200. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:27:10,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.780')] -[2023-09-25 20:27:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3325952. Throughput: 0: 822.7, 1: 822.5. Samples: 829113. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:27:15,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.740')] -[2023-09-25 20:27:20,161][109224] Updated weights for policy 1, policy_version 6560 (0.0016) -[2023-09-25 20:27:20,161][109225] Updated weights for policy 0, policy_version 6560 (0.0017) -[2023-09-25 20:27:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3358720. Throughput: 0: 824.1, 1: 824.7. Samples: 839022. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:27:20,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.740')] -[2023-09-25 20:27:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3391488. Throughput: 0: 820.7, 1: 819.4. Samples: 843787. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:27:25,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.740')] -[2023-09-25 20:27:30,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3424256. Throughput: 0: 824.4, 1: 823.3. Samples: 853954. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:27:30,470][108279] Avg episode reward: [(0, '9.880'), (1, '8.750')] -[2023-09-25 20:27:32,647][109225] Updated weights for policy 0, policy_version 6720 (0.0017) -[2023-09-25 20:27:32,647][109224] Updated weights for policy 1, policy_version 6720 (0.0018) -[2023-09-25 20:27:35,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3457024. Throughput: 0: 822.7, 1: 822.6. Samples: 863562. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:27:35,471][108279] Avg episode reward: [(0, '9.880'), (1, '8.750')] -[2023-09-25 20:27:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3489792. Throughput: 0: 819.8, 1: 819.3. Samples: 868357. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:27:40,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.780')] -[2023-09-25 20:27:45,276][109225] Updated weights for policy 0, policy_version 6880 (0.0019) -[2023-09-25 20:27:45,277][109224] Updated weights for policy 1, policy_version 6880 (0.0018) -[2023-09-25 20:27:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3522560. Throughput: 0: 819.8, 1: 820.1. Samples: 878246. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:27:45,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.780')] -[2023-09-25 20:27:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3555328. Throughput: 0: 816.8, 1: 817.6. Samples: 887853. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:27:50,471][108279] Avg episode reward: [(0, '9.890'), (1, '8.780')] -[2023-09-25 20:27:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3588096. Throughput: 0: 818.2, 1: 819.2. Samples: 892884. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:27:55,471][108279] Avg episode reward: [(0, '9.880'), (1, '8.780')] -[2023-09-25 20:27:57,963][109224] Updated weights for policy 1, policy_version 7040 (0.0018) -[2023-09-25 20:27:57,964][109225] Updated weights for policy 0, policy_version 7040 (0.0020) -[2023-09-25 20:28:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3620864. Throughput: 0: 814.6, 1: 814.6. Samples: 902426. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:28:00,470][108279] Avg episode reward: [(0, '9.880'), (1, '8.790')] -[2023-09-25 20:28:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3653632. Throughput: 0: 815.6, 1: 815.0. Samples: 912399. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:28:05,471][108279] Avg episode reward: [(0, '9.870'), (1, '8.790')] -[2023-09-25 20:28:10,200][109224] Updated weights for policy 1, policy_version 7200 (0.0017) -[2023-09-25 20:28:10,200][109225] Updated weights for policy 0, policy_version 7200 (0.0017) -[2023-09-25 20:28:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3686400. Throughput: 0: 819.2, 1: 819.0. Samples: 917504. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:28:10,471][108279] Avg episode reward: [(0, '9.870'), (1, '8.780')] -[2023-09-25 20:28:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3719168. Throughput: 0: 818.6, 1: 819.2. Samples: 927654. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:28:15,471][108279] Avg episode reward: [(0, '9.870'), (1, '8.780')] -[2023-09-25 20:28:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3751936. Throughput: 0: 820.8, 1: 820.6. Samples: 937425. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:28:20,470][108279] Avg episode reward: [(0, '9.870'), (1, '8.850')] -[2023-09-25 20:28:22,697][109224] Updated weights for policy 1, policy_version 7360 (0.0016) -[2023-09-25 20:28:22,697][109225] Updated weights for policy 0, policy_version 7360 (0.0014) -[2023-09-25 20:28:25,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3784704. Throughput: 0: 819.2, 1: 819.1. Samples: 942082. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:28:25,470][108279] Avg episode reward: [(0, '9.860'), (1, '8.850')] -[2023-09-25 20:28:30,470][108279] Fps is (10 sec: 5734.2, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 3809280. Throughput: 0: 814.4, 1: 816.0. Samples: 951616. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:28:30,471][108279] Avg episode reward: [(0, '9.860'), (1, '8.890')] -[2023-09-25 20:28:30,481][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000007456_1908736.pth... -[2023-09-25 20:28:30,514][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000004400_1126400.pth -[2023-09-25 20:28:30,545][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000007456_1908736.pth... -[2023-09-25 20:28:30,581][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000004400_1126400.pth -[2023-09-25 20:28:35,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 3842048. Throughput: 0: 814.7, 1: 814.1. Samples: 961151. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:28:35,471][108279] Avg episode reward: [(0, '9.860'), (1, '8.890')] -[2023-09-25 20:28:35,544][109224] Updated weights for policy 1, policy_version 7520 (0.0017) -[2023-09-25 20:28:35,544][109225] Updated weights for policy 0, policy_version 7520 (0.0016) -[2023-09-25 20:28:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 3874816. Throughput: 0: 814.8, 1: 816.2. Samples: 966281. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:28:40,471][108279] Avg episode reward: [(0, '9.860'), (1, '8.920')] -[2023-09-25 20:28:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 3907584. Throughput: 0: 816.0, 1: 816.5. Samples: 975888. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:28:45,471][108279] Avg episode reward: [(0, '9.870'), (1, '8.920')] -[2023-09-25 20:28:48,094][109225] Updated weights for policy 0, policy_version 7680 (0.0017) -[2023-09-25 20:28:48,095][109224] Updated weights for policy 1, policy_version 7680 (0.0018) -[2023-09-25 20:28:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 3940352. Throughput: 0: 814.0, 1: 813.9. Samples: 985654. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:28:50,471][108279] Avg episode reward: [(0, '9.870'), (1, '8.970')] -[2023-09-25 20:28:55,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 3981312. Throughput: 0: 814.2, 1: 817.2. Samples: 990919. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:28:55,470][108279] Avg episode reward: [(0, '9.870'), (1, '8.970')] -[2023-09-25 20:29:00,404][109224] Updated weights for policy 1, policy_version 7840 (0.0014) -[2023-09-25 20:29:00,404][109225] Updated weights for policy 0, policy_version 7840 (0.0017) -[2023-09-25 20:29:00,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 4014080. Throughput: 0: 811.6, 1: 812.6. Samples: 1000740. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:00,470][108279] Avg episode reward: [(0, '9.870'), (1, '8.960')] -[2023-09-25 20:29:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 4046848. Throughput: 0: 812.3, 1: 812.7. Samples: 1010550. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:05,471][108279] Avg episode reward: [(0, '9.870'), (1, '8.960')] -[2023-09-25 20:29:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 4079616. Throughput: 0: 816.0, 1: 819.2. Samples: 1015662. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:10,471][108279] Avg episode reward: [(0, '9.870'), (1, '9.030')] -[2023-09-25 20:29:13,049][109225] Updated weights for policy 0, policy_version 8000 (0.0017) -[2023-09-25 20:29:13,049][109224] Updated weights for policy 1, policy_version 8000 (0.0016) -[2023-09-25 20:29:15,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 4104192. Throughput: 0: 817.0, 1: 815.9. Samples: 1025095. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:15,471][108279] Avg episode reward: [(0, '9.860'), (1, '9.050')] -[2023-09-25 20:29:20,470][108279] Fps is (10 sec: 6144.1, 60 sec: 6485.3, 300 sec: 6511.9). Total num frames: 4141056. Throughput: 0: 820.3, 1: 819.9. Samples: 1034958. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:20,471][108279] Avg episode reward: [(0, '9.860'), (1, '9.080')] -[2023-09-25 20:29:25,439][109224] Updated weights for policy 1, policy_version 8160 (0.0017) -[2023-09-25 20:29:25,439][109225] Updated weights for policy 0, policy_version 8160 (0.0016) -[2023-09-25 20:29:25,470][108279] Fps is (10 sec: 7373.0, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 4177920. Throughput: 0: 820.0, 1: 820.3. Samples: 1040096. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:25,470][108279] Avg episode reward: [(0, '9.860'), (1, '9.080')] -[2023-09-25 20:29:30,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 4202496. Throughput: 0: 820.9, 1: 819.6. Samples: 1049714. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:30,470][108279] Avg episode reward: [(0, '9.860'), (1, '9.080')] -[2023-09-25 20:29:35,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 4235264. Throughput: 0: 820.3, 1: 820.2. Samples: 1059477. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:35,471][108279] Avg episode reward: [(0, '9.850'), (1, '9.080')] -[2023-09-25 20:29:38,006][109224] Updated weights for policy 1, policy_version 8320 (0.0017) -[2023-09-25 20:29:38,006][109225] Updated weights for policy 0, policy_version 8320 (0.0016) -[2023-09-25 20:29:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 4268032. Throughput: 0: 819.7, 1: 818.9. Samples: 1064656. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:40,471][108279] Avg episode reward: [(0, '9.840'), (1, '9.110')] -[2023-09-25 20:29:45,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6621.9, 300 sec: 6511.9). Total num frames: 4304896. Throughput: 0: 818.2, 1: 818.1. Samples: 1074371. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:45,471][108279] Avg episode reward: [(0, '9.830'), (1, '9.110')] -[2023-09-25 20:29:50,421][109224] Updated weights for policy 1, policy_version 8480 (0.0016) -[2023-09-25 20:29:50,422][109225] Updated weights for policy 0, policy_version 8480 (0.0016) -[2023-09-25 20:29:50,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6690.1, 300 sec: 6525.8). Total num frames: 4341760. Throughput: 0: 819.5, 1: 819.3. Samples: 1084298. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:29:50,471][108279] Avg episode reward: [(0, '9.830'), (1, '9.120')] -[2023-09-25 20:29:55,470][108279] Fps is (10 sec: 6144.1, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 4366336. Throughput: 0: 818.2, 1: 819.2. Samples: 1089346. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:29:55,470][108279] Avg episode reward: [(0, '9.820'), (1, '9.150')] -[2023-09-25 20:30:00,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 4403200. Throughput: 0: 820.4, 1: 820.2. Samples: 1098926. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:30:00,471][108279] Avg episode reward: [(0, '9.820'), (1, '9.150')] -[2023-09-25 20:30:02,966][109224] Updated weights for policy 1, policy_version 8640 (0.0013) -[2023-09-25 20:30:02,967][109225] Updated weights for policy 0, policy_version 8640 (0.0017) -[2023-09-25 20:30:05,470][108279] Fps is (10 sec: 7372.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 4440064. Throughput: 0: 819.8, 1: 820.5. Samples: 1108769. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:30:05,471][108279] Avg episode reward: [(0, '9.820'), (1, '9.250')] -[2023-09-25 20:30:10,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 4472832. Throughput: 0: 820.6, 1: 820.7. Samples: 1113956. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:30:10,471][108279] Avg episode reward: [(0, '9.820'), (1, '9.250')] -[2023-09-25 20:30:15,373][109225] Updated weights for policy 0, policy_version 8800 (0.0016) -[2023-09-25 20:30:15,374][109224] Updated weights for policy 1, policy_version 8800 (0.0017) -[2023-09-25 20:30:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 4505600. Throughput: 0: 821.8, 1: 822.8. Samples: 1123721. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:30:15,471][108279] Avg episode reward: [(0, '9.810'), (1, '9.300')] -[2023-09-25 20:30:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6621.9, 300 sec: 6553.6). Total num frames: 4538368. Throughput: 0: 821.7, 1: 821.8. Samples: 1133434. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:30:20,471][108279] Avg episode reward: [(0, '9.820'), (1, '9.300')] -[2023-09-25 20:30:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 4571136. Throughput: 0: 822.5, 1: 821.5. Samples: 1138638. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:30:25,470][108279] Avg episode reward: [(0, '9.820'), (1, '9.370')] -[2023-09-25 20:30:27,877][109224] Updated weights for policy 1, policy_version 8960 (0.0018) -[2023-09-25 20:30:27,877][109225] Updated weights for policy 0, policy_version 8960 (0.0017) -[2023-09-25 20:30:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 4603904. Throughput: 0: 822.2, 1: 821.3. Samples: 1148329. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:30:30,471][108279] Avg episode reward: [(0, '9.780'), (1, '9.370')] -[2023-09-25 20:30:30,479][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000008992_2301952.pth... -[2023-09-25 20:30:30,479][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000008992_2301952.pth... -[2023-09-25 20:30:30,515][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000005920_1515520.pth -[2023-09-25 20:30:30,518][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000005920_1515520.pth -[2023-09-25 20:30:35,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 4636672. Throughput: 0: 820.3, 1: 820.2. Samples: 1158119. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:30:35,471][108279] Avg episode reward: [(0, '9.780'), (1, '9.380')] -[2023-09-25 20:30:40,313][109224] Updated weights for policy 1, policy_version 9120 (0.0016) -[2023-09-25 20:30:40,313][109225] Updated weights for policy 0, policy_version 9120 (0.0015) -[2023-09-25 20:30:40,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6690.2, 300 sec: 6553.6). Total num frames: 4669440. Throughput: 0: 821.5, 1: 819.2. Samples: 1163177. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:30:40,470][108279] Avg episode reward: [(0, '9.770'), (1, '9.380')] -[2023-09-25 20:30:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6621.9, 300 sec: 6553.6). Total num frames: 4702208. Throughput: 0: 824.8, 1: 824.7. Samples: 1173152. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:30:45,471][108279] Avg episode reward: [(0, '9.760'), (1, '9.400')] -[2023-09-25 20:30:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 4734976. Throughput: 0: 822.1, 1: 821.7. Samples: 1182736. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:30:50,470][108279] Avg episode reward: [(0, '9.750'), (1, '9.400')] -[2023-09-25 20:30:52,795][109224] Updated weights for policy 1, policy_version 9280 (0.0018) -[2023-09-25 20:30:52,795][109225] Updated weights for policy 0, policy_version 9280 (0.0017) -[2023-09-25 20:30:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 4767744. Throughput: 0: 822.2, 1: 819.6. Samples: 1187835. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:30:55,470][108279] Avg episode reward: [(0, '9.740'), (1, '9.480')] -[2023-09-25 20:31:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6621.9, 300 sec: 6553.6). Total num frames: 4800512. Throughput: 0: 818.8, 1: 819.2. Samples: 1197429. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:31:00,470][108279] Avg episode reward: [(0, '9.720'), (1, '9.480')] -[2023-09-25 20:31:05,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 4825088. Throughput: 0: 816.2, 1: 816.3. Samples: 1206894. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:31:05,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.590')] -[2023-09-25 20:31:05,538][109224] Updated weights for policy 1, policy_version 9440 (0.0017) -[2023-09-25 20:31:05,538][109225] Updated weights for policy 0, policy_version 9440 (0.0017) -[2023-09-25 20:31:10,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 4857856. Throughput: 0: 814.0, 1: 816.4. Samples: 1212006. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:31:10,471][108279] Avg episode reward: [(0, '9.690'), (1, '9.590')] -[2023-09-25 20:31:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 4890624. Throughput: 0: 815.6, 1: 816.2. Samples: 1221762. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:31:15,471][108279] Avg episode reward: [(0, '9.690'), (1, '9.690')] -[2023-09-25 20:31:17,999][109224] Updated weights for policy 1, policy_version 9600 (0.0015) -[2023-09-25 20:31:17,999][109225] Updated weights for policy 0, policy_version 9600 (0.0016) -[2023-09-25 20:31:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 4923392. Throughput: 0: 815.5, 1: 815.8. Samples: 1231525. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:31:20,471][108279] Avg episode reward: [(0, '9.690'), (1, '9.690')] -[2023-09-25 20:31:25,470][108279] Fps is (10 sec: 7372.8, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 4964352. Throughput: 0: 816.0, 1: 817.2. Samples: 1236672. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:31:25,471][108279] Avg episode reward: [(0, '9.690'), (1, '9.760')] -[2023-09-25 20:31:25,473][109025] Saving new best policy, reward=9.760! -[2023-09-25 20:31:30,378][109224] Updated weights for policy 1, policy_version 9760 (0.0016) -[2023-09-25 20:31:30,378][109225] Updated weights for policy 0, policy_version 9760 (0.0017) -[2023-09-25 20:31:30,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 4997120. Throughput: 0: 815.7, 1: 815.9. Samples: 1246576. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:31:30,470][108279] Avg episode reward: [(0, '9.680'), (1, '9.760')] -[2023-09-25 20:31:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5029888. Throughput: 0: 817.0, 1: 817.6. Samples: 1256291. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:31:35,470][108279] Avg episode reward: [(0, '9.680'), (1, '9.810')] -[2023-09-25 20:31:35,471][109025] Saving new best policy, reward=9.810! -[2023-09-25 20:31:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5062656. Throughput: 0: 816.9, 1: 819.2. Samples: 1261458. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:31:40,471][108279] Avg episode reward: [(0, '9.650'), (1, '9.810')] -[2023-09-25 20:31:42,851][109225] Updated weights for policy 0, policy_version 9920 (0.0017) -[2023-09-25 20:31:42,851][109224] Updated weights for policy 1, policy_version 9920 (0.0016) -[2023-09-25 20:31:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5095424. Throughput: 0: 820.4, 1: 820.3. Samples: 1271264. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:31:45,470][108279] Avg episode reward: [(0, '9.650'), (1, '9.840')] -[2023-09-25 20:31:45,477][109025] Saving new best policy, reward=9.840! -[2023-09-25 20:31:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5128192. Throughput: 0: 824.2, 1: 823.3. Samples: 1281034. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:31:50,470][108279] Avg episode reward: [(0, '9.620'), (1, '9.840')] -[2023-09-25 20:31:55,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 5156864. Throughput: 0: 822.0, 1: 821.9. Samples: 1285981. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:31:55,471][108279] Avg episode reward: [(0, '9.620'), (1, '9.860')] -[2023-09-25 20:31:55,549][109025] Saving new best policy, reward=9.860! -[2023-09-25 20:31:55,552][109224] Updated weights for policy 1, policy_version 10080 (0.0018) -[2023-09-25 20:31:55,563][109225] Updated weights for policy 0, policy_version 10080 (0.0017) -[2023-09-25 20:32:00,470][108279] Fps is (10 sec: 5734.2, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 5185536. Throughput: 0: 817.4, 1: 817.2. Samples: 1295319. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:00,471][108279] Avg episode reward: [(0, '9.590'), (1, '9.860')] -[2023-09-25 20:32:05,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 5218304. Throughput: 0: 819.4, 1: 819.1. Samples: 1305258. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:32:05,471][108279] Avg episode reward: [(0, '9.590'), (1, '9.860')] -[2023-09-25 20:32:07,981][109224] Updated weights for policy 1, policy_version 10240 (0.0017) -[2023-09-25 20:32:07,982][109225] Updated weights for policy 0, policy_version 10240 (0.0019) -[2023-09-25 20:32:10,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6621.9, 300 sec: 6539.7). Total num frames: 5255168. Throughput: 0: 819.4, 1: 818.8. Samples: 1310389. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:32:10,471][108279] Avg episode reward: [(0, '9.580'), (1, '9.860')] -[2023-09-25 20:32:15,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6621.9, 300 sec: 6539.7). Total num frames: 5287936. Throughput: 0: 818.4, 1: 818.2. Samples: 1320223. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:15,471][108279] Avg episode reward: [(0, '9.580'), (1, '9.860')] -[2023-09-25 20:32:20,439][109224] Updated weights for policy 1, policy_version 10400 (0.0018) -[2023-09-25 20:32:20,439][109225] Updated weights for policy 0, policy_version 10400 (0.0018) -[2023-09-25 20:32:20,470][108279] Fps is (10 sec: 6963.4, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 5324800. Throughput: 0: 818.5, 1: 817.8. Samples: 1329926. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:20,470][108279] Avg episode reward: [(0, '9.550'), (1, '9.870')] -[2023-09-25 20:32:20,471][109025] Saving new best policy, reward=9.870! -[2023-09-25 20:32:25,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5357568. Throughput: 0: 818.4, 1: 818.3. Samples: 1335110. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:25,471][108279] Avg episode reward: [(0, '9.550'), (1, '9.870')] -[2023-09-25 20:32:30,470][108279] Fps is (10 sec: 5734.2, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 5382144. Throughput: 0: 816.2, 1: 815.2. Samples: 1344677. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:30,471][108279] Avg episode reward: [(0, '9.530'), (1, '9.910')] -[2023-09-25 20:32:30,484][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000010528_2695168.pth... -[2023-09-25 20:32:30,505][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000010528_2695168.pth... -[2023-09-25 20:32:30,511][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000007456_1908736.pth -[2023-09-25 20:32:30,514][109025] Saving new best policy, reward=9.910! -[2023-09-25 20:32:30,533][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000007456_1908736.pth -[2023-09-25 20:32:33,071][109224] Updated weights for policy 1, policy_version 10560 (0.0016) -[2023-09-25 20:32:33,072][109225] Updated weights for policy 0, policy_version 10560 (0.0015) -[2023-09-25 20:32:35,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 5414912. Throughput: 0: 814.1, 1: 815.0. Samples: 1354345. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:35,471][108279] Avg episode reward: [(0, '9.530'), (1, '9.910')] -[2023-09-25 20:32:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 5447680. Throughput: 0: 817.5, 1: 816.7. Samples: 1359519. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:40,471][108279] Avg episode reward: [(0, '9.520'), (1, '9.910')] -[2023-09-25 20:32:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 5480448. Throughput: 0: 820.6, 1: 821.9. Samples: 1369235. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:45,471][108279] Avg episode reward: [(0, '9.520'), (1, '9.910')] -[2023-09-25 20:32:45,581][109225] Updated weights for policy 0, policy_version 10720 (0.0016) -[2023-09-25 20:32:45,582][109224] Updated weights for policy 1, policy_version 10720 (0.0018) -[2023-09-25 20:32:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 5513216. Throughput: 0: 818.5, 1: 819.0. Samples: 1378946. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:32:50,471][108279] Avg episode reward: [(0, '9.500'), (1, '9.910')] -[2023-09-25 20:32:55,470][108279] Fps is (10 sec: 7372.7, 60 sec: 6621.8, 300 sec: 6553.6). Total num frames: 5554176. Throughput: 0: 820.4, 1: 820.2. Samples: 1384212. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:32:55,471][108279] Avg episode reward: [(0, '9.500'), (1, '9.910')] -[2023-09-25 20:32:57,886][109225] Updated weights for policy 0, policy_version 10880 (0.0017) -[2023-09-25 20:32:57,886][109224] Updated weights for policy 1, policy_version 10880 (0.0017) -[2023-09-25 20:33:00,470][108279] Fps is (10 sec: 7372.8, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 5586944. Throughput: 0: 820.5, 1: 821.2. Samples: 1394098. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:33:00,471][108279] Avg episode reward: [(0, '9.470'), (1, '9.920')] -[2023-09-25 20:33:00,480][109025] Saving new best policy, reward=9.920! -[2023-09-25 20:33:05,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 5611520. Throughput: 0: 818.0, 1: 818.0. Samples: 1403542. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:33:05,471][108279] Avg episode reward: [(0, '9.470'), (1, '9.920')] -[2023-09-25 20:33:10,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 5648384. Throughput: 0: 818.0, 1: 818.6. Samples: 1408756. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:33:10,471][108279] Avg episode reward: [(0, '9.450'), (1, '9.920')] -[2023-09-25 20:33:10,480][109224] Updated weights for policy 1, policy_version 11040 (0.0018) -[2023-09-25 20:33:10,480][109225] Updated weights for policy 0, policy_version 11040 (0.0016) -[2023-09-25 20:33:15,470][108279] Fps is (10 sec: 7372.8, 60 sec: 6621.9, 300 sec: 6553.6). Total num frames: 5685248. Throughput: 0: 819.5, 1: 820.3. Samples: 1418466. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:33:15,470][108279] Avg episode reward: [(0, '9.450'), (1, '9.920')] -[2023-09-25 20:33:20,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5718016. Throughput: 0: 821.9, 1: 822.1. Samples: 1428324. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:33:20,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.910')] -[2023-09-25 20:33:22,854][109225] Updated weights for policy 0, policy_version 11200 (0.0017) -[2023-09-25 20:33:22,854][109224] Updated weights for policy 1, policy_version 11200 (0.0017) -[2023-09-25 20:33:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6581.4). Total num frames: 5750784. Throughput: 0: 822.3, 1: 821.8. Samples: 1433506. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:33:25,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.910')] -[2023-09-25 20:33:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6690.2, 300 sec: 6581.4). Total num frames: 5783552. Throughput: 0: 825.4, 1: 823.8. Samples: 1443447. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:33:30,470][108279] Avg episode reward: [(0, '9.430'), (1, '9.900')] -[2023-09-25 20:33:35,361][109225] Updated weights for policy 0, policy_version 11360 (0.0018) -[2023-09-25 20:33:35,361][109224] Updated weights for policy 1, policy_version 11360 (0.0016) -[2023-09-25 20:33:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6690.2, 300 sec: 6581.4). Total num frames: 5816320. Throughput: 0: 822.7, 1: 822.9. Samples: 1452998. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:33:35,470][108279] Avg episode reward: [(0, '9.430'), (1, '9.900')] -[2023-09-25 20:33:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6581.4). Total num frames: 5849088. Throughput: 0: 821.2, 1: 820.6. Samples: 1458091. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:33:40,471][108279] Avg episode reward: [(0, '9.370'), (1, '9.900')] -[2023-09-25 20:33:45,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6690.1, 300 sec: 6581.4). Total num frames: 5881856. Throughput: 0: 820.6, 1: 820.2. Samples: 1467937. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:33:45,471][108279] Avg episode reward: [(0, '9.370'), (1, '9.900')] -[2023-09-25 20:33:47,790][109225] Updated weights for policy 0, policy_version 11520 (0.0015) -[2023-09-25 20:33:47,791][109224] Updated weights for policy 1, policy_version 11520 (0.0017) -[2023-09-25 20:33:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 5914624. Throughput: 0: 823.8, 1: 824.5. Samples: 1477716. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:33:50,471][108279] Avg episode reward: [(0, '9.380'), (1, '9.900')] -[2023-09-25 20:33:55,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5947392. Throughput: 0: 823.6, 1: 820.7. Samples: 1482752. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:33:55,470][108279] Avg episode reward: [(0, '9.380'), (1, '9.900')] -[2023-09-25 20:34:00,345][109224] Updated weights for policy 1, policy_version 11680 (0.0018) -[2023-09-25 20:34:00,345][109225] Updated weights for policy 0, policy_version 11680 (0.0017) -[2023-09-25 20:34:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 5980160. Throughput: 0: 823.8, 1: 823.7. Samples: 1492605. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:00,470][108279] Avg episode reward: [(0, '9.330'), (1, '9.910')] -[2023-09-25 20:34:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 6012928. Throughput: 0: 821.5, 1: 821.2. Samples: 1502247. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:05,471][108279] Avg episode reward: [(0, '9.330'), (1, '9.920')] -[2023-09-25 20:34:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6621.9, 300 sec: 6581.4). Total num frames: 6045696. Throughput: 0: 820.3, 1: 819.2. Samples: 1507285. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:34:10,470][108279] Avg episode reward: [(0, '9.290'), (1, '9.920')] -[2023-09-25 20:34:12,908][109225] Updated weights for policy 0, policy_version 11840 (0.0016) -[2023-09-25 20:34:12,909][109224] Updated weights for policy 1, policy_version 11840 (0.0018) -[2023-09-25 20:34:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6567.5). Total num frames: 6078464. Throughput: 0: 816.2, 1: 816.5. Samples: 1516917. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:34:15,471][108279] Avg episode reward: [(0, '9.270'), (1, '9.920')] -[2023-09-25 20:34:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 6111232. Throughput: 0: 819.9, 1: 819.0. Samples: 1526747. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:20,470][108279] Avg episode reward: [(0, '9.240'), (1, '9.920')] -[2023-09-25 20:34:25,430][109225] Updated weights for policy 0, policy_version 12000 (0.0017) -[2023-09-25 20:34:25,430][109224] Updated weights for policy 1, policy_version 12000 (0.0016) -[2023-09-25 20:34:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6581.4). Total num frames: 6144000. Throughput: 0: 818.4, 1: 819.2. Samples: 1531786. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:25,470][108279] Avg episode reward: [(0, '9.240'), (1, '9.920')] -[2023-09-25 20:34:30,470][108279] Fps is (10 sec: 5734.2, 60 sec: 6417.0, 300 sec: 6553.6). Total num frames: 6168576. Throughput: 0: 815.0, 1: 813.8. Samples: 1541231. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:30,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.920')] -[2023-09-25 20:34:30,521][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000012064_3088384.pth... -[2023-09-25 20:34:30,530][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000012064_3088384.pth... -[2023-09-25 20:34:30,550][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000008992_2301952.pth -[2023-09-25 20:34:30,558][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000008992_2301952.pth -[2023-09-25 20:34:35,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.0, 300 sec: 6553.6). Total num frames: 6201344. Throughput: 0: 813.6, 1: 813.0. Samples: 1550917. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:35,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.910')] -[2023-09-25 20:34:38,078][109225] Updated weights for policy 0, policy_version 12160 (0.0017) -[2023-09-25 20:34:38,079][109224] Updated weights for policy 1, policy_version 12160 (0.0017) -[2023-09-25 20:34:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6539.7). Total num frames: 6234112. Throughput: 0: 812.2, 1: 815.8. Samples: 1556015. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:40,470][108279] Avg episode reward: [(0, '9.240'), (1, '9.910')] -[2023-09-25 20:34:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 6266880. Throughput: 0: 812.2, 1: 812.5. Samples: 1565716. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:45,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.910')] -[2023-09-25 20:34:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6553.6). Total num frames: 6299648. Throughput: 0: 814.8, 1: 815.0. Samples: 1575588. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:50,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.890')] -[2023-09-25 20:34:50,507][109224] Updated weights for policy 1, policy_version 12320 (0.0016) -[2023-09-25 20:34:50,507][109225] Updated weights for policy 0, policy_version 12320 (0.0015) -[2023-09-25 20:34:55,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6567.5). Total num frames: 6340608. Throughput: 0: 816.4, 1: 817.8. Samples: 1580822. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:34:55,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.890')] -[2023-09-25 20:35:00,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 6369280. Throughput: 0: 818.6, 1: 818.5. Samples: 1590590. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:35:00,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.890')] -[2023-09-25 20:35:03,019][109225] Updated weights for policy 0, policy_version 12480 (0.0018) -[2023-09-25 20:35:03,019][109224] Updated weights for policy 1, policy_version 12480 (0.0015) -[2023-09-25 20:35:05,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 6397952. Throughput: 0: 814.5, 1: 815.0. Samples: 1600073. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:35:05,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.900')] -[2023-09-25 20:35:10,470][108279] Fps is (10 sec: 6144.1, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 6430720. Throughput: 0: 814.3, 1: 814.4. Samples: 1605077. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:35:10,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.900')] -[2023-09-25 20:35:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 6463488. Throughput: 0: 815.2, 1: 817.0. Samples: 1614678. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:35:15,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.900')] -[2023-09-25 20:35:15,670][109225] Updated weights for policy 0, policy_version 12640 (0.0013) -[2023-09-25 20:35:15,670][109224] Updated weights for policy 1, policy_version 12640 (0.0018) -[2023-09-25 20:35:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 6496256. Throughput: 0: 817.2, 1: 817.0. Samples: 1624459. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:35:20,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.900')] -[2023-09-25 20:35:25,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 6529024. Throughput: 0: 818.5, 1: 817.1. Samples: 1629616. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:35:25,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.900')] -[2023-09-25 20:35:28,201][109224] Updated weights for policy 1, policy_version 12800 (0.0017) -[2023-09-25 20:35:28,201][109225] Updated weights for policy 0, policy_version 12800 (0.0019) -[2023-09-25 20:35:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6561792. Throughput: 0: 816.4, 1: 815.6. Samples: 1639159. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:35:30,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.900')] -[2023-09-25 20:35:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6594560. Throughput: 0: 813.0, 1: 812.4. Samples: 1648735. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:35:35,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.890')] -[2023-09-25 20:35:40,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6627328. Throughput: 0: 810.8, 1: 811.5. Samples: 1653828. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:35:40,470][108279] Avg episode reward: [(0, '9.220'), (1, '9.890')] -[2023-09-25 20:35:40,868][109224] Updated weights for policy 1, policy_version 12960 (0.0017) -[2023-09-25 20:35:40,869][109225] Updated weights for policy 0, policy_version 12960 (0.0017) -[2023-09-25 20:35:45,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6660096. Throughput: 0: 809.2, 1: 809.4. Samples: 1663430. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:35:45,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.900')] -[2023-09-25 20:35:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6692864. Throughput: 0: 814.0, 1: 811.5. Samples: 1673220. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:35:50,470][108279] Avg episode reward: [(0, '9.250'), (1, '9.900')] -[2023-09-25 20:35:53,473][109224] Updated weights for policy 1, policy_version 13120 (0.0015) -[2023-09-25 20:35:53,473][109225] Updated weights for policy 0, policy_version 13120 (0.0017) -[2023-09-25 20:35:55,470][108279] Fps is (10 sec: 6553.9, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 6725632. Throughput: 0: 810.7, 1: 810.7. Samples: 1678040. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:35:55,470][108279] Avg episode reward: [(0, '9.250'), (1, '9.890')] -[2023-09-25 20:36:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6485.4, 300 sec: 6553.6). Total num frames: 6758400. Throughput: 0: 812.6, 1: 812.0. Samples: 1687785. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:36:00,470][108279] Avg episode reward: [(0, '9.240'), (1, '9.890')] -[2023-09-25 20:36:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 6791168. Throughput: 0: 815.9, 1: 813.7. Samples: 1697792. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:36:05,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.890')] -[2023-09-25 20:36:06,027][109224] Updated weights for policy 1, policy_version 13280 (0.0018) -[2023-09-25 20:36:06,027][109225] Updated weights for policy 0, policy_version 13280 (0.0018) -[2023-09-25 20:36:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 6823936. Throughput: 0: 809.9, 1: 809.9. Samples: 1702508. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:36:10,470][108279] Avg episode reward: [(0, '9.210'), (1, '9.870')] -[2023-09-25 20:36:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 6856704. Throughput: 0: 811.8, 1: 809.8. Samples: 1712132. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:36:15,470][108279] Avg episode reward: [(0, '9.170'), (1, '9.870')] -[2023-09-25 20:36:18,714][109224] Updated weights for policy 1, policy_version 13440 (0.0018) -[2023-09-25 20:36:18,714][109225] Updated weights for policy 0, policy_version 13440 (0.0017) -[2023-09-25 20:36:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6889472. Throughput: 0: 815.2, 1: 815.7. Samples: 1722125. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:36:20,470][108279] Avg episode reward: [(0, '9.180'), (1, '9.860')] -[2023-09-25 20:36:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6922240. Throughput: 0: 811.2, 1: 811.0. Samples: 1726829. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:36:25,470][108279] Avg episode reward: [(0, '9.180'), (1, '9.860')] -[2023-09-25 20:36:30,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6955008. Throughput: 0: 815.4, 1: 813.0. Samples: 1736709. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:36:30,471][108279] Avg episode reward: [(0, '9.170'), (1, '9.850')] -[2023-09-25 20:36:30,481][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000013584_3477504.pth... -[2023-09-25 20:36:30,481][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000013584_3477504.pth... -[2023-09-25 20:36:30,516][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000010528_2695168.pth -[2023-09-25 20:36:30,520][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000010528_2695168.pth -[2023-09-25 20:36:31,117][109224] Updated weights for policy 1, policy_version 13600 (0.0018) -[2023-09-25 20:36:31,118][109225] Updated weights for policy 0, policy_version 13600 (0.0016) -[2023-09-25 20:36:35,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 6987776. Throughput: 0: 816.1, 1: 819.1. Samples: 1746807. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:36:35,471][108279] Avg episode reward: [(0, '9.170'), (1, '9.850')] -[2023-09-25 20:36:40,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7020544. Throughput: 0: 812.7, 1: 812.5. Samples: 1751174. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:36:40,470][108279] Avg episode reward: [(0, '9.170'), (1, '9.830')] -[2023-09-25 20:36:43,783][109224] Updated weights for policy 1, policy_version 13760 (0.0015) -[2023-09-25 20:36:43,783][109225] Updated weights for policy 0, policy_version 13760 (0.0017) -[2023-09-25 20:36:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7053312. Throughput: 0: 817.9, 1: 815.3. Samples: 1761277. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:36:45,471][108279] Avg episode reward: [(0, '9.150'), (1, '9.830')] -[2023-09-25 20:36:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 7086080. Throughput: 0: 814.1, 1: 816.7. Samples: 1771176. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:36:50,470][108279] Avg episode reward: [(0, '9.130'), (1, '9.800')] -[2023-09-25 20:36:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7118848. Throughput: 0: 813.9, 1: 814.1. Samples: 1775767. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:36:55,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.800')] -[2023-09-25 20:36:56,323][109225] Updated weights for policy 0, policy_version 13920 (0.0017) -[2023-09-25 20:36:56,324][109224] Updated weights for policy 1, policy_version 13920 (0.0018) -[2023-09-25 20:37:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7151616. Throughput: 0: 817.2, 1: 819.1. Samples: 1785766. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:00,470][108279] Avg episode reward: [(0, '9.110'), (1, '9.800')] -[2023-09-25 20:37:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 7184384. Throughput: 0: 818.0, 1: 818.0. Samples: 1795743. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:05,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.800')] -[2023-09-25 20:37:08,897][109225] Updated weights for policy 0, policy_version 14080 (0.0015) -[2023-09-25 20:37:08,897][109224] Updated weights for policy 1, policy_version 14080 (0.0018) -[2023-09-25 20:37:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 7217152. Throughput: 0: 816.5, 1: 813.9. Samples: 1800200. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:10,470][108279] Avg episode reward: [(0, '9.110'), (1, '9.750')] -[2023-09-25 20:37:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7249920. Throughput: 0: 816.4, 1: 819.1. Samples: 1810307. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:15,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.750')] -[2023-09-25 20:37:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7282688. Throughput: 0: 810.5, 1: 810.0. Samples: 1819729. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:20,470][108279] Avg episode reward: [(0, '9.120'), (1, '9.680')] -[2023-09-25 20:37:21,516][109225] Updated weights for policy 0, policy_version 14240 (0.0017) -[2023-09-25 20:37:21,517][109224] Updated weights for policy 1, policy_version 14240 (0.0019) -[2023-09-25 20:37:25,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7315456. Throughput: 0: 819.0, 1: 816.5. Samples: 1824768. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:25,470][108279] Avg episode reward: [(0, '9.090'), (1, '9.680')] -[2023-09-25 20:37:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7348224. Throughput: 0: 815.2, 1: 817.3. Samples: 1834737. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:30,470][108279] Avg episode reward: [(0, '9.080'), (1, '9.630')] -[2023-09-25 20:37:33,963][109224] Updated weights for policy 1, policy_version 14400 (0.0017) -[2023-09-25 20:37:33,963][109225] Updated weights for policy 0, policy_version 14400 (0.0018) -[2023-09-25 20:37:35,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7380992. Throughput: 0: 814.5, 1: 814.1. Samples: 1844462. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:35,470][108279] Avg episode reward: [(0, '9.070'), (1, '9.630')] -[2023-09-25 20:37:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7413760. Throughput: 0: 818.4, 1: 816.3. Samples: 1849327. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:40,470][108279] Avg episode reward: [(0, '9.090'), (1, '9.590')] -[2023-09-25 20:37:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7446528. Throughput: 0: 812.8, 1: 812.9. Samples: 1858920. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:45,470][108279] Avg episode reward: [(0, '9.110'), (1, '9.600')] -[2023-09-25 20:37:46,755][109225] Updated weights for policy 0, policy_version 14560 (0.0018) -[2023-09-25 20:37:46,755][109224] Updated weights for policy 1, policy_version 14560 (0.0018) -[2023-09-25 20:37:50,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 7471104. Throughput: 0: 807.7, 1: 808.2. Samples: 1868460. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:50,471][108279] Avg episode reward: [(0, '9.100'), (1, '9.550')] -[2023-09-25 20:37:55,470][108279] Fps is (10 sec: 6143.9, 60 sec: 6485.3, 300 sec: 6511.9). Total num frames: 7507968. Throughput: 0: 814.0, 1: 817.5. Samples: 1873618. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:37:55,471][108279] Avg episode reward: [(0, '9.090'), (1, '9.560')] -[2023-09-25 20:37:59,125][109225] Updated weights for policy 0, policy_version 14720 (0.0016) -[2023-09-25 20:37:59,125][109224] Updated weights for policy 1, policy_version 14720 (0.0016) -[2023-09-25 20:38:00,470][108279] Fps is (10 sec: 7372.8, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 7544832. Throughput: 0: 813.8, 1: 813.5. Samples: 1883535. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:00,471][108279] Avg episode reward: [(0, '9.080'), (1, '9.560')] -[2023-09-25 20:38:05,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 7577600. Throughput: 0: 818.3, 1: 818.5. Samples: 1893388. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:38:05,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.560')] -[2023-09-25 20:38:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7610368. Throughput: 0: 819.2, 1: 819.2. Samples: 1898496. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:38:10,471][108279] Avg episode reward: [(0, '9.130'), (1, '9.550')] -[2023-09-25 20:38:11,542][109225] Updated weights for policy 0, policy_version 14880 (0.0016) -[2023-09-25 20:38:11,542][109224] Updated weights for policy 1, policy_version 14880 (0.0015) -[2023-09-25 20:38:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7643136. Throughput: 0: 817.6, 1: 817.7. Samples: 1908326. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:15,470][108279] Avg episode reward: [(0, '9.160'), (1, '9.550')] -[2023-09-25 20:38:20,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7675904. Throughput: 0: 818.4, 1: 818.5. Samples: 1918120. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:20,470][108279] Avg episode reward: [(0, '9.180'), (1, '9.550')] -[2023-09-25 20:38:24,121][109225] Updated weights for policy 0, policy_version 15040 (0.0016) -[2023-09-25 20:38:24,121][109224] Updated weights for policy 1, policy_version 15040 (0.0019) -[2023-09-25 20:38:25,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 7708672. Throughput: 0: 818.0, 1: 819.2. Samples: 1923002. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:25,471][108279] Avg episode reward: [(0, '9.190'), (1, '9.550')] -[2023-09-25 20:38:30,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 7733248. Throughput: 0: 816.2, 1: 817.3. Samples: 1932429. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:30,470][108279] Avg episode reward: [(0, '9.200'), (1, '9.550')] -[2023-09-25 20:38:30,489][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000015120_3870720.pth... -[2023-09-25 20:38:30,516][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000012064_3088384.pth -[2023-09-25 20:38:30,581][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000015120_3870720.pth... -[2023-09-25 20:38:30,614][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000012064_3088384.pth -[2023-09-25 20:38:35,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 7766016. Throughput: 0: 817.4, 1: 816.8. Samples: 1941999. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:35,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.550')] -[2023-09-25 20:38:36,860][109225] Updated weights for policy 0, policy_version 15200 (0.0016) -[2023-09-25 20:38:36,861][109224] Updated weights for policy 1, policy_version 15200 (0.0016) -[2023-09-25 20:38:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 7798784. Throughput: 0: 817.1, 1: 816.3. Samples: 1947121. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:40,471][108279] Avg episode reward: [(0, '9.200'), (1, '9.550')] -[2023-09-25 20:38:45,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 7831552. Throughput: 0: 813.8, 1: 813.6. Samples: 1956770. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:38:45,470][108279] Avg episode reward: [(0, '9.210'), (1, '9.550')] -[2023-09-25 20:38:49,344][109224] Updated weights for policy 1, policy_version 15360 (0.0017) -[2023-09-25 20:38:49,344][109225] Updated weights for policy 0, policy_version 15360 (0.0018) -[2023-09-25 20:38:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 7864320. Throughput: 0: 813.3, 1: 813.0. Samples: 1966570. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:38:50,471][108279] Avg episode reward: [(0, '9.200'), (1, '9.540')] -[2023-09-25 20:38:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6485.4, 300 sec: 6498.1). Total num frames: 7897088. Throughput: 0: 812.4, 1: 815.1. Samples: 1971730. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:38:55,470][108279] Avg episode reward: [(0, '9.170'), (1, '9.550')] -[2023-09-25 20:39:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 7929856. Throughput: 0: 814.2, 1: 813.2. Samples: 1981556. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:39:00,471][108279] Avg episode reward: [(0, '9.190'), (1, '9.550')] -[2023-09-25 20:39:01,917][109224] Updated weights for policy 1, policy_version 15520 (0.0014) -[2023-09-25 20:39:01,918][109225] Updated weights for policy 0, policy_version 15520 (0.0018) -[2023-09-25 20:39:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 7962624. Throughput: 0: 809.2, 1: 809.4. Samples: 1990954. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:39:05,471][108279] Avg episode reward: [(0, '9.200'), (1, '9.550')] -[2023-09-25 20:39:10,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 7995392. Throughput: 0: 812.6, 1: 813.6. Samples: 1996180. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:39:10,470][108279] Avg episode reward: [(0, '9.180'), (1, '9.550')] -[2023-09-25 20:39:14,359][109224] Updated weights for policy 1, policy_version 15680 (0.0018) -[2023-09-25 20:39:14,359][109225] Updated weights for policy 0, policy_version 15680 (0.0016) -[2023-09-25 20:39:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 8028160. Throughput: 0: 817.6, 1: 817.0. Samples: 2005987. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:39:15,471][108279] Avg episode reward: [(0, '9.150'), (1, '9.550')] -[2023-09-25 20:39:20,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 8060928. Throughput: 0: 819.7, 1: 819.9. Samples: 2015780. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:39:20,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.550')] -[2023-09-25 20:39:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 8093696. Throughput: 0: 820.7, 1: 820.0. Samples: 2020954. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:39:25,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.550')] -[2023-09-25 20:39:26,790][109225] Updated weights for policy 0, policy_version 15840 (0.0016) -[2023-09-25 20:39:26,791][109224] Updated weights for policy 1, policy_version 15840 (0.0017) -[2023-09-25 20:39:30,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6621.8, 300 sec: 6539.7). Total num frames: 8130560. Throughput: 0: 821.7, 1: 822.1. Samples: 2030742. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:39:30,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.550')] -[2023-09-25 20:39:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8159232. Throughput: 0: 819.8, 1: 819.6. Samples: 2040343. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:39:35,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.520')] -[2023-09-25 20:39:39,278][109225] Updated weights for policy 0, policy_version 16000 (0.0016) -[2023-09-25 20:39:39,278][109224] Updated weights for policy 1, policy_version 16000 (0.0015) -[2023-09-25 20:39:40,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8192000. Throughput: 0: 819.9, 1: 819.5. Samples: 2045503. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:39:40,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.520')] -[2023-09-25 20:39:45,470][108279] Fps is (10 sec: 7372.7, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 8232960. Throughput: 0: 819.4, 1: 820.7. Samples: 2055362. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:39:45,471][108279] Avg episode reward: [(0, '9.130'), (1, '9.500')] -[2023-09-25 20:39:50,470][108279] Fps is (10 sec: 7372.8, 60 sec: 6690.1, 300 sec: 6525.8). Total num frames: 8265728. Throughput: 0: 824.4, 1: 824.6. Samples: 2065159. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:39:50,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.500')] -[2023-09-25 20:39:51,693][109225] Updated weights for policy 0, policy_version 16160 (0.0016) -[2023-09-25 20:39:51,693][109224] Updated weights for policy 1, policy_version 16160 (0.0018) -[2023-09-25 20:39:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6690.1, 300 sec: 6539.7). Total num frames: 8298496. Throughput: 0: 824.2, 1: 823.2. Samples: 2070315. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:39:55,471][108279] Avg episode reward: [(0, '9.130'), (1, '9.500')] -[2023-09-25 20:40:00,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8323072. Throughput: 0: 823.0, 1: 822.9. Samples: 2080050. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:40:00,471][108279] Avg episode reward: [(0, '9.140'), (1, '9.500')] -[2023-09-25 20:40:04,432][109225] Updated weights for policy 0, policy_version 16320 (0.0017) -[2023-09-25 20:40:04,432][109224] Updated weights for policy 1, policy_version 16320 (0.0017) -[2023-09-25 20:40:05,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8355840. Throughput: 0: 816.7, 1: 816.9. Samples: 2089292. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:40:05,471][108279] Avg episode reward: [(0, '9.160'), (1, '9.500')] -[2023-09-25 20:40:10,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8388608. Throughput: 0: 816.8, 1: 817.2. Samples: 2094484. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:40:10,470][108279] Avg episode reward: [(0, '9.180'), (1, '9.500')] -[2023-09-25 20:40:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8421376. Throughput: 0: 816.8, 1: 816.3. Samples: 2104232. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:40:15,471][108279] Avg episode reward: [(0, '9.190'), (1, '9.490')] -[2023-09-25 20:40:16,835][109224] Updated weights for policy 1, policy_version 16480 (0.0015) -[2023-09-25 20:40:16,835][109225] Updated weights for policy 0, policy_version 16480 (0.0017) -[2023-09-25 20:40:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8454144. Throughput: 0: 819.1, 1: 819.2. Samples: 2114069. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:40:20,471][108279] Avg episode reward: [(0, '9.250'), (1, '9.480')] -[2023-09-25 20:40:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8486912. Throughput: 0: 819.5, 1: 819.9. Samples: 2119275. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:40:25,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.470')] -[2023-09-25 20:40:29,325][109224] Updated weights for policy 1, policy_version 16640 (0.0017) -[2023-09-25 20:40:29,325][109225] Updated weights for policy 0, policy_version 16640 (0.0017) -[2023-09-25 20:40:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6485.3, 300 sec: 6525.8). Total num frames: 8519680. Throughput: 0: 817.4, 1: 817.4. Samples: 2128926. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:40:30,471][108279] Avg episode reward: [(0, '9.200'), (1, '9.480')] -[2023-09-25 20:40:30,542][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000016656_4263936.pth... -[2023-09-25 20:40:30,549][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000016656_4263936.pth... -[2023-09-25 20:40:30,571][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000013584_3477504.pth -[2023-09-25 20:40:30,577][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000013584_3477504.pth -[2023-09-25 20:40:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8552448. Throughput: 0: 815.7, 1: 815.5. Samples: 2138563. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:40:35,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.470')] -[2023-09-25 20:40:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8585216. Throughput: 0: 816.2, 1: 816.8. Samples: 2143802. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:40:40,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.440')] -[2023-09-25 20:40:41,785][109224] Updated weights for policy 1, policy_version 16800 (0.0017) -[2023-09-25 20:40:41,785][109225] Updated weights for policy 0, policy_version 16800 (0.0017) -[2023-09-25 20:40:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 8617984. Throughput: 0: 816.6, 1: 817.8. Samples: 2153600. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:40:45,471][108279] Avg episode reward: [(0, '9.200'), (1, '9.440')] -[2023-09-25 20:40:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 8650752. Throughput: 0: 822.9, 1: 822.3. Samples: 2163325. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:40:50,471][108279] Avg episode reward: [(0, '9.200'), (1, '9.440')] -[2023-09-25 20:40:54,357][109224] Updated weights for policy 1, policy_version 16960 (0.0016) -[2023-09-25 20:40:54,357][109225] Updated weights for policy 0, policy_version 16960 (0.0018) -[2023-09-25 20:40:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 8683520. Throughput: 0: 819.9, 1: 820.0. Samples: 2168278. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:40:55,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.450')] -[2023-09-25 20:41:00,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8716288. Throughput: 0: 819.5, 1: 819.4. Samples: 2177982. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:00,470][108279] Avg episode reward: [(0, '9.220'), (1, '9.460')] -[2023-09-25 20:41:05,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8749056. Throughput: 0: 816.4, 1: 816.7. Samples: 2187556. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:05,470][108279] Avg episode reward: [(0, '9.230'), (1, '9.470')] -[2023-09-25 20:41:06,936][109224] Updated weights for policy 1, policy_version 17120 (0.0015) -[2023-09-25 20:41:06,937][109225] Updated weights for policy 0, policy_version 17120 (0.0018) -[2023-09-25 20:41:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8781824. Throughput: 0: 816.1, 1: 815.9. Samples: 2192715. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:10,470][108279] Avg episode reward: [(0, '9.220'), (1, '9.500')] -[2023-09-25 20:41:15,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8814592. Throughput: 0: 817.1, 1: 817.1. Samples: 2202464. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:15,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.500')] -[2023-09-25 20:41:19,549][109225] Updated weights for policy 0, policy_version 17280 (0.0015) -[2023-09-25 20:41:19,549][109224] Updated weights for policy 1, policy_version 17280 (0.0017) -[2023-09-25 20:41:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8847360. Throughput: 0: 815.6, 1: 815.1. Samples: 2211943. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:20,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.510')] -[2023-09-25 20:41:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8880128. Throughput: 0: 811.9, 1: 811.1. Samples: 2216838. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:25,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.510')] -[2023-09-25 20:41:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8912896. Throughput: 0: 808.3, 1: 805.3. Samples: 2226214. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:41:30,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.550')] -[2023-09-25 20:41:32,444][109224] Updated weights for policy 1, policy_version 17440 (0.0016) -[2023-09-25 20:41:32,445][109225] Updated weights for policy 0, policy_version 17440 (0.0017) -[2023-09-25 20:41:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8945664. Throughput: 0: 810.2, 1: 810.0. Samples: 2236235. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:41:35,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.550')] -[2023-09-25 20:41:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 8978432. Throughput: 0: 805.6, 1: 805.2. Samples: 2240765. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:40,470][108279] Avg episode reward: [(0, '9.280'), (1, '9.600')] -[2023-09-25 20:41:44,985][109224] Updated weights for policy 1, policy_version 17600 (0.0016) -[2023-09-25 20:41:44,985][109225] Updated weights for policy 0, policy_version 17600 (0.0016) -[2023-09-25 20:41:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9011200. Throughput: 0: 809.6, 1: 807.5. Samples: 2250752. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:45,470][108279] Avg episode reward: [(0, '9.280'), (1, '9.600')] -[2023-09-25 20:41:50,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9043968. Throughput: 0: 812.0, 1: 810.4. Samples: 2260563. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:41:50,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.640')] -[2023-09-25 20:41:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9076736. Throughput: 0: 805.6, 1: 805.1. Samples: 2265194. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:41:55,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.640')] -[2023-09-25 20:41:57,617][109224] Updated weights for policy 1, policy_version 17760 (0.0019) -[2023-09-25 20:41:57,617][109225] Updated weights for policy 0, policy_version 17760 (0.0019) -[2023-09-25 20:42:00,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.5, 300 sec: 6525.8). Total num frames: 9109504. Throughput: 0: 809.5, 1: 808.3. Samples: 2275263. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:42:00,471][108279] Avg episode reward: [(0, '9.250'), (1, '9.680')] -[2023-09-25 20:42:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9142272. Throughput: 0: 808.5, 1: 809.0. Samples: 2284730. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:42:05,471][108279] Avg episode reward: [(0, '9.250'), (1, '9.680')] -[2023-09-25 20:42:10,117][109224] Updated weights for policy 1, policy_version 17920 (0.0015) -[2023-09-25 20:42:10,118][109225] Updated weights for policy 0, policy_version 17920 (0.0019) -[2023-09-25 20:42:10,470][108279] Fps is (10 sec: 6553.9, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9175040. Throughput: 0: 809.9, 1: 808.6. Samples: 2289668. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:42:10,470][108279] Avg episode reward: [(0, '9.250'), (1, '9.690')] -[2023-09-25 20:42:15,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9207808. Throughput: 0: 818.4, 1: 818.4. Samples: 2299870. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:42:15,470][108279] Avg episode reward: [(0, '9.250'), (1, '9.720')] -[2023-09-25 20:42:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9240576. Throughput: 0: 814.4, 1: 814.9. Samples: 2309553. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:42:20,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.720')] -[2023-09-25 20:42:22,557][109224] Updated weights for policy 1, policy_version 18080 (0.0017) -[2023-09-25 20:42:22,557][109225] Updated weights for policy 0, policy_version 18080 (0.0018) -[2023-09-25 20:42:25,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9273344. Throughput: 0: 817.8, 1: 817.3. Samples: 2314342. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:42:25,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.720')] -[2023-09-25 20:42:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9306112. Throughput: 0: 819.2, 1: 819.2. Samples: 2324480. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:42:30,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.730')] -[2023-09-25 20:42:30,481][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000018176_4653056.pth... -[2023-09-25 20:42:30,482][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000018176_4653056.pth... -[2023-09-25 20:42:30,510][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000015120_3870720.pth -[2023-09-25 20:42:30,523][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000015120_3870720.pth -[2023-09-25 20:42:35,099][109225] Updated weights for policy 0, policy_version 18240 (0.0016) -[2023-09-25 20:42:35,100][109224] Updated weights for policy 1, policy_version 18240 (0.0014) -[2023-09-25 20:42:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9338880. Throughput: 0: 817.2, 1: 818.5. Samples: 2334171. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:42:35,470][108279] Avg episode reward: [(0, '9.260'), (1, '9.730')] -[2023-09-25 20:42:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9371648. Throughput: 0: 819.1, 1: 818.3. Samples: 2338879. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:42:40,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.730')] -[2023-09-25 20:42:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 9404416. Throughput: 0: 820.0, 1: 819.2. Samples: 2349026. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:42:45,471][108279] Avg episode reward: [(0, '9.220'), (1, '9.730')] -[2023-09-25 20:42:47,523][109224] Updated weights for policy 1, policy_version 18400 (0.0017) -[2023-09-25 20:42:47,523][109225] Updated weights for policy 0, policy_version 18400 (0.0016) -[2023-09-25 20:42:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 9437184. Throughput: 0: 823.3, 1: 823.9. Samples: 2358856. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:42:50,470][108279] Avg episode reward: [(0, '9.230'), (1, '9.730')] -[2023-09-25 20:42:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9469952. Throughput: 0: 819.2, 1: 819.2. Samples: 2363397. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:42:55,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.730')] -[2023-09-25 20:43:00,224][109224] Updated weights for policy 1, policy_version 18560 (0.0015) -[2023-09-25 20:43:00,224][109225] Updated weights for policy 0, policy_version 18560 (0.0018) -[2023-09-25 20:43:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9502720. Throughput: 0: 815.9, 1: 818.8. Samples: 2373432. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:00,470][108279] Avg episode reward: [(0, '9.250'), (1, '9.730')] -[2023-09-25 20:43:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9535488. Throughput: 0: 817.3, 1: 816.5. Samples: 2383074. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:05,470][108279] Avg episode reward: [(0, '9.260'), (1, '9.730')] -[2023-09-25 20:43:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9568256. Throughput: 0: 818.0, 1: 817.0. Samples: 2387919. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:10,471][108279] Avg episode reward: [(0, '9.290'), (1, '9.740')] -[2023-09-25 20:43:12,842][109224] Updated weights for policy 1, policy_version 18720 (0.0017) -[2023-09-25 20:43:12,842][109225] Updated weights for policy 0, policy_version 18720 (0.0017) -[2023-09-25 20:43:15,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9601024. Throughput: 0: 811.9, 1: 815.1. Samples: 2397695. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:15,471][108279] Avg episode reward: [(0, '9.300'), (1, '9.730')] -[2023-09-25 20:43:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 9633792. Throughput: 0: 813.1, 1: 814.0. Samples: 2407389. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:20,470][108279] Avg episode reward: [(0, '9.320'), (1, '9.730')] -[2023-09-25 20:43:25,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 9658368. Throughput: 0: 814.9, 1: 817.0. Samples: 2412317. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:25,471][108279] Avg episode reward: [(0, '9.330'), (1, '9.730')] -[2023-09-25 20:43:25,527][109224] Updated weights for policy 1, policy_version 18880 (0.0017) -[2023-09-25 20:43:25,527][109225] Updated weights for policy 0, policy_version 18880 (0.0018) -[2023-09-25 20:43:30,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 9691136. Throughput: 0: 809.1, 1: 811.2. Samples: 2421940. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:30,471][108279] Avg episode reward: [(0, '9.390'), (1, '9.740')] -[2023-09-25 20:43:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 9723904. Throughput: 0: 807.5, 1: 806.6. Samples: 2431488. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:35,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.750')] -[2023-09-25 20:43:38,042][109224] Updated weights for policy 1, policy_version 19040 (0.0015) -[2023-09-25 20:43:38,044][109225] Updated weights for policy 0, policy_version 19040 (0.0016) -[2023-09-25 20:43:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 9756672. Throughput: 0: 812.8, 1: 815.4. Samples: 2436669. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:43:40,471][108279] Avg episode reward: [(0, '9.430'), (1, '9.750')] -[2023-09-25 20:43:45,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 9797632. Throughput: 0: 814.1, 1: 812.0. Samples: 2446606. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:43:45,471][108279] Avg episode reward: [(0, '9.450'), (1, '9.750')] -[2023-09-25 20:43:50,432][109225] Updated weights for policy 0, policy_version 19200 (0.0018) -[2023-09-25 20:43:50,432][109224] Updated weights for policy 1, policy_version 19200 (0.0015) -[2023-09-25 20:43:50,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 9830400. Throughput: 0: 814.1, 1: 814.9. Samples: 2456379. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:43:50,471][108279] Avg episode reward: [(0, '9.460'), (1, '9.750')] -[2023-09-25 20:43:55,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 9854976. Throughput: 0: 816.4, 1: 818.5. Samples: 2461488. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:43:55,471][108279] Avg episode reward: [(0, '9.460'), (1, '9.780')] -[2023-09-25 20:44:00,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 9887744. Throughput: 0: 817.2, 1: 816.1. Samples: 2471196. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:44:00,470][108279] Avg episode reward: [(0, '9.460'), (1, '9.780')] -[2023-09-25 20:44:02,955][109224] Updated weights for policy 1, policy_version 19360 (0.0018) -[2023-09-25 20:44:02,956][109225] Updated weights for policy 0, policy_version 19360 (0.0017) -[2023-09-25 20:44:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 9920512. Throughput: 0: 817.1, 1: 815.3. Samples: 2480846. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:44:05,471][108279] Avg episode reward: [(0, '9.460'), (1, '9.790')] -[2023-09-25 20:44:10,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 9953280. Throughput: 0: 818.8, 1: 817.5. Samples: 2485952. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:10,471][108279] Avg episode reward: [(0, '9.460'), (1, '9.790')] -[2023-09-25 20:44:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 9986048. Throughput: 0: 819.3, 1: 818.4. Samples: 2495636. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:15,471][108279] Avg episode reward: [(0, '9.440'), (1, '9.800')] -[2023-09-25 20:44:15,609][109224] Updated weights for policy 1, policy_version 19520 (0.0016) -[2023-09-25 20:44:15,611][109225] Updated weights for policy 0, policy_version 19520 (0.0018) -[2023-09-25 20:44:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 10018816. Throughput: 0: 819.3, 1: 819.7. Samples: 2505244. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:20,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.800')] -[2023-09-25 20:44:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6512.0). Total num frames: 10051584. Throughput: 0: 819.8, 1: 819.6. Samples: 2510438. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:25,470][108279] Avg episode reward: [(0, '9.410'), (1, '9.800')] -[2023-09-25 20:44:28,293][109225] Updated weights for policy 0, policy_version 19680 (0.0018) -[2023-09-25 20:44:28,293][109224] Updated weights for policy 1, policy_version 19680 (0.0017) -[2023-09-25 20:44:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10084352. Throughput: 0: 810.8, 1: 812.1. Samples: 2519635. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:30,471][108279] Avg episode reward: [(0, '9.400'), (1, '9.800')] -[2023-09-25 20:44:30,484][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000019696_5042176.pth... -[2023-09-25 20:44:30,484][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000019696_5042176.pth... -[2023-09-25 20:44:30,519][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000016656_4263936.pth -[2023-09-25 20:44:30,519][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000016656_4263936.pth -[2023-09-25 20:44:35,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10117120. Throughput: 0: 811.3, 1: 808.8. Samples: 2529285. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:35,471][108279] Avg episode reward: [(0, '9.380'), (1, '9.820')] -[2023-09-25 20:44:40,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 10149888. Throughput: 0: 806.6, 1: 806.4. Samples: 2534076. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:40,471][108279] Avg episode reward: [(0, '9.400'), (1, '9.830')] -[2023-09-25 20:44:40,953][109224] Updated weights for policy 1, policy_version 19840 (0.0015) -[2023-09-25 20:44:40,954][109225] Updated weights for policy 0, policy_version 19840 (0.0016) -[2023-09-25 20:44:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 10182656. Throughput: 0: 807.2, 1: 807.7. Samples: 2543866. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:45,470][108279] Avg episode reward: [(0, '9.450'), (1, '9.840')] -[2023-09-25 20:44:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 10215424. Throughput: 0: 811.4, 1: 810.4. Samples: 2553826. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:50,470][108279] Avg episode reward: [(0, '9.430'), (1, '9.840')] -[2023-09-25 20:44:53,543][109224] Updated weights for policy 1, policy_version 20000 (0.0017) -[2023-09-25 20:44:53,543][109225] Updated weights for policy 0, policy_version 20000 (0.0019) -[2023-09-25 20:44:55,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10248192. Throughput: 0: 806.9, 1: 807.2. Samples: 2558588. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:44:55,471][108279] Avg episode reward: [(0, '9.430'), (1, '9.840')] -[2023-09-25 20:45:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10280960. Throughput: 0: 807.9, 1: 808.5. Samples: 2568376. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:45:00,471][108279] Avg episode reward: [(0, '9.410'), (1, '9.850')] -[2023-09-25 20:45:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10313728. Throughput: 0: 814.5, 1: 811.9. Samples: 2578432. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:45:05,471][108279] Avg episode reward: [(0, '9.410'), (1, '9.850')] -[2023-09-25 20:45:06,166][109224] Updated weights for policy 1, policy_version 20160 (0.0018) -[2023-09-25 20:45:06,166][109225] Updated weights for policy 0, policy_version 20160 (0.0016) -[2023-09-25 20:45:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10346496. Throughput: 0: 804.6, 1: 804.8. Samples: 2582861. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:45:10,470][108279] Avg episode reward: [(0, '9.380'), (1, '9.860')] -[2023-09-25 20:45:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10379264. Throughput: 0: 811.1, 1: 809.4. Samples: 2592556. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:45:15,471][108279] Avg episode reward: [(0, '9.390'), (1, '9.860')] -[2023-09-25 20:45:19,078][109224] Updated weights for policy 1, policy_version 20320 (0.0017) -[2023-09-25 20:45:19,078][109225] Updated weights for policy 0, policy_version 20320 (0.0017) -[2023-09-25 20:45:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10412032. Throughput: 0: 807.0, 1: 809.7. Samples: 2602035. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:45:20,471][108279] Avg episode reward: [(0, '9.390'), (1, '9.860')] -[2023-09-25 20:45:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10444800. Throughput: 0: 810.2, 1: 809.9. Samples: 2606983. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:45:25,471][108279] Avg episode reward: [(0, '9.380'), (1, '9.860')] -[2023-09-25 20:45:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10477568. Throughput: 0: 809.7, 1: 809.1. Samples: 2616712. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:45:30,471][108279] Avg episode reward: [(0, '9.370'), (1, '9.860')] -[2023-09-25 20:45:31,634][109224] Updated weights for policy 1, policy_version 20480 (0.0018) -[2023-09-25 20:45:31,634][109225] Updated weights for policy 0, policy_version 20480 (0.0017) -[2023-09-25 20:45:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10510336. Throughput: 0: 807.3, 1: 808.8. Samples: 2626550. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-09-25 20:45:35,470][108279] Avg episode reward: [(0, '9.350'), (1, '9.860')] -[2023-09-25 20:45:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10543104. Throughput: 0: 811.1, 1: 810.9. Samples: 2631577. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-09-25 20:45:40,470][108279] Avg episode reward: [(0, '9.320'), (1, '9.850')] -[2023-09-25 20:45:44,059][109224] Updated weights for policy 1, policy_version 20640 (0.0016) -[2023-09-25 20:45:44,060][109225] Updated weights for policy 0, policy_version 20640 (0.0017) -[2023-09-25 20:45:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10575872. Throughput: 0: 811.6, 1: 812.2. Samples: 2641444. Policy #0 lag: (min: 2.0, avg: 2.0, max: 2.0) -[2023-09-25 20:45:45,470][108279] Avg episode reward: [(0, '9.310'), (1, '9.840')] -[2023-09-25 20:45:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10608640. Throughput: 0: 809.4, 1: 811.3. Samples: 2651364. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:45:50,470][108279] Avg episode reward: [(0, '9.280'), (1, '9.850')] -[2023-09-25 20:45:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10641408. Throughput: 0: 815.8, 1: 814.1. Samples: 2656208. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:45:55,471][108279] Avg episode reward: [(0, '9.270'), (1, '9.860')] -[2023-09-25 20:45:56,531][109225] Updated weights for policy 0, policy_version 20800 (0.0015) -[2023-09-25 20:45:56,531][109224] Updated weights for policy 1, policy_version 20800 (0.0017) -[2023-09-25 20:46:00,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10674176. Throughput: 0: 816.4, 1: 817.1. Samples: 2666063. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:00,471][108279] Avg episode reward: [(0, '9.290'), (1, '9.870')] -[2023-09-25 20:46:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10706944. Throughput: 0: 819.7, 1: 819.4. Samples: 2675791. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:05,471][108279] Avg episode reward: [(0, '9.290'), (1, '9.860')] -[2023-09-25 20:46:09,006][109224] Updated weights for policy 1, policy_version 20960 (0.0017) -[2023-09-25 20:46:09,006][109225] Updated weights for policy 0, policy_version 20960 (0.0017) -[2023-09-25 20:46:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10739712. Throughput: 0: 821.9, 1: 819.2. Samples: 2680832. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:10,471][108279] Avg episode reward: [(0, '9.280'), (1, '9.860')] -[2023-09-25 20:46:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10772480. Throughput: 0: 823.8, 1: 824.6. Samples: 2690892. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:15,471][108279] Avg episode reward: [(0, '9.290'), (1, '9.870')] -[2023-09-25 20:46:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10805248. Throughput: 0: 822.6, 1: 823.1. Samples: 2700607. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:46:20,471][108279] Avg episode reward: [(0, '9.300'), (1, '9.840')] -[2023-09-25 20:46:21,455][109225] Updated weights for policy 0, policy_version 21120 (0.0018) -[2023-09-25 20:46:21,455][109224] Updated weights for policy 1, policy_version 21120 (0.0018) -[2023-09-25 20:46:25,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10838016. Throughput: 0: 821.5, 1: 819.2. Samples: 2705409. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:46:25,470][108279] Avg episode reward: [(0, '9.280'), (1, '9.850')] -[2023-09-25 20:46:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10870784. Throughput: 0: 822.8, 1: 822.4. Samples: 2715478. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:46:30,470][108279] Avg episode reward: [(0, '9.280'), (1, '9.840')] -[2023-09-25 20:46:30,480][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000021232_5435392.pth... -[2023-09-25 20:46:30,480][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000021232_5435392.pth... -[2023-09-25 20:46:30,516][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000018176_4653056.pth -[2023-09-25 20:46:30,519][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000018176_4653056.pth -[2023-09-25 20:46:33,910][109225] Updated weights for policy 0, policy_version 21280 (0.0018) -[2023-09-25 20:46:33,910][109224] Updated weights for policy 1, policy_version 21280 (0.0017) -[2023-09-25 20:46:35,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10903552. Throughput: 0: 820.7, 1: 821.3. Samples: 2725255. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:35,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.840')] -[2023-09-25 20:46:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10936320. Throughput: 0: 820.3, 1: 819.3. Samples: 2729988. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:40,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.850')] -[2023-09-25 20:46:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 10969088. Throughput: 0: 821.1, 1: 822.0. Samples: 2740003. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:45,470][108279] Avg episode reward: [(0, '9.240'), (1, '9.850')] -[2023-09-25 20:46:46,407][109224] Updated weights for policy 1, policy_version 21440 (0.0016) -[2023-09-25 20:46:46,408][109225] Updated weights for policy 0, policy_version 21440 (0.0016) -[2023-09-25 20:46:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11001856. Throughput: 0: 822.7, 1: 821.6. Samples: 2749784. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:50,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.840')] -[2023-09-25 20:46:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11034624. Throughput: 0: 819.2, 1: 819.2. Samples: 2754561. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:46:55,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.840')] -[2023-09-25 20:46:59,067][109225] Updated weights for policy 0, policy_version 21600 (0.0018) -[2023-09-25 20:46:59,067][109224] Updated weights for policy 1, policy_version 21600 (0.0017) -[2023-09-25 20:47:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11067392. Throughput: 0: 815.8, 1: 816.4. Samples: 2764340. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:47:00,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.830')] -[2023-09-25 20:47:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11100160. Throughput: 0: 813.6, 1: 813.6. Samples: 2773831. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:47:05,470][108279] Avg episode reward: [(0, '9.180'), (1, '9.830')] -[2023-09-25 20:47:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11132928. Throughput: 0: 816.8, 1: 819.2. Samples: 2779030. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:47:10,471][108279] Avg episode reward: [(0, '9.170'), (1, '9.830')] -[2023-09-25 20:47:11,621][109225] Updated weights for policy 0, policy_version 21760 (0.0016) -[2023-09-25 20:47:11,621][109224] Updated weights for policy 1, policy_version 21760 (0.0016) -[2023-09-25 20:47:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11165696. Throughput: 0: 813.8, 1: 813.8. Samples: 2788720. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:47:15,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.830')] -[2023-09-25 20:47:20,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6485.3, 300 sec: 6511.9). Total num frames: 11194368. Throughput: 0: 812.7, 1: 812.9. Samples: 2798407. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:47:20,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.830')] -[2023-09-25 20:47:24,399][109225] Updated weights for policy 0, policy_version 21920 (0.0019) -[2023-09-25 20:47:24,399][109224] Updated weights for policy 1, policy_version 21920 (0.0018) -[2023-09-25 20:47:25,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 11223040. Throughput: 0: 811.6, 1: 814.4. Samples: 2803161. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:47:25,471][108279] Avg episode reward: [(0, '9.070'), (1, '9.830')] -[2023-09-25 20:47:30,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11255808. Throughput: 0: 810.8, 1: 810.6. Samples: 2812963. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:47:30,471][108279] Avg episode reward: [(0, '9.070'), (1, '9.820')] -[2023-09-25 20:47:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11288576. Throughput: 0: 810.3, 1: 811.4. Samples: 2822759. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 20:47:35,471][108279] Avg episode reward: [(0, '9.020'), (1, '9.830')] -[2023-09-25 20:47:36,774][109225] Updated weights for policy 0, policy_version 22080 (0.0016) -[2023-09-25 20:47:36,774][109224] Updated weights for policy 1, policy_version 22080 (0.0016) -[2023-09-25 20:47:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11321344. Throughput: 0: 812.6, 1: 816.0. Samples: 2827848. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:47:40,470][108279] Avg episode reward: [(0, '8.980'), (1, '9.830')] -[2023-09-25 20:47:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 11354112. Throughput: 0: 809.4, 1: 809.1. Samples: 2837172. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:47:45,471][108279] Avg episode reward: [(0, '8.970'), (1, '9.820')] -[2023-09-25 20:47:49,600][109224] Updated weights for policy 1, policy_version 22240 (0.0017) -[2023-09-25 20:47:49,600][109225] Updated weights for policy 0, policy_version 22240 (0.0017) -[2023-09-25 20:47:50,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11386880. Throughput: 0: 811.3, 1: 809.7. Samples: 2846776. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 20:47:50,471][108279] Avg episode reward: [(0, '8.960'), (1, '9.820')] -[2023-09-25 20:47:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11419648. Throughput: 0: 808.4, 1: 808.9. Samples: 2851808. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:47:55,470][108279] Avg episode reward: [(0, '8.950'), (1, '9.810')] -[2023-09-25 20:48:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11452416. Throughput: 0: 808.8, 1: 808.5. Samples: 2861501. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:00,471][108279] Avg episode reward: [(0, '8.940'), (1, '9.810')] -[2023-09-25 20:48:02,063][109225] Updated weights for policy 0, policy_version 22400 (0.0014) -[2023-09-25 20:48:02,065][109224] Updated weights for policy 1, policy_version 22400 (0.0015) -[2023-09-25 20:48:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11485184. Throughput: 0: 811.3, 1: 809.6. Samples: 2871347. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:05,470][108279] Avg episode reward: [(0, '8.920'), (1, '9.800')] -[2023-09-25 20:48:10,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11517952. Throughput: 0: 814.5, 1: 814.2. Samples: 2876455. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:10,471][108279] Avg episode reward: [(0, '8.910'), (1, '9.800')] -[2023-09-25 20:48:14,519][109224] Updated weights for policy 1, policy_version 22560 (0.0017) -[2023-09-25 20:48:14,520][109225] Updated weights for policy 0, policy_version 22560 (0.0018) -[2023-09-25 20:48:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 11550720. Throughput: 0: 814.6, 1: 814.7. Samples: 2886282. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:15,470][108279] Avg episode reward: [(0, '8.900'), (1, '9.800')] -[2023-09-25 20:48:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6485.3, 300 sec: 6525.8). Total num frames: 11583488. Throughput: 0: 813.6, 1: 811.2. Samples: 2895877. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:20,470][108279] Avg episode reward: [(0, '8.900'), (1, '9.800')] -[2023-09-25 20:48:25,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11616256. Throughput: 0: 812.2, 1: 811.2. Samples: 2900903. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:25,471][108279] Avg episode reward: [(0, '8.910'), (1, '9.810')] -[2023-09-25 20:48:27,259][109225] Updated weights for policy 0, policy_version 22720 (0.0019) -[2023-09-25 20:48:27,259][109224] Updated weights for policy 1, policy_version 22720 (0.0018) -[2023-09-25 20:48:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11649024. Throughput: 0: 813.4, 1: 812.6. Samples: 2910346. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:30,470][108279] Avg episode reward: [(0, '8.900'), (1, '9.810')] -[2023-09-25 20:48:30,478][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000022752_5824512.pth... -[2023-09-25 20:48:30,478][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000022752_5824512.pth... -[2023-09-25 20:48:30,514][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000019696_5042176.pth -[2023-09-25 20:48:30,514][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000019696_5042176.pth -[2023-09-25 20:48:35,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11681792. Throughput: 0: 819.2, 1: 818.0. Samples: 2920448. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:35,470][108279] Avg episode reward: [(0, '8.890'), (1, '9.800')] -[2023-09-25 20:48:39,739][109224] Updated weights for policy 1, policy_version 22880 (0.0015) -[2023-09-25 20:48:39,741][109225] Updated weights for policy 0, policy_version 22880 (0.0017) -[2023-09-25 20:48:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 11714560. Throughput: 0: 815.4, 1: 815.2. Samples: 2925184. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:40,470][108279] Avg episode reward: [(0, '8.890'), (1, '9.800')] -[2023-09-25 20:48:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 11747328. Throughput: 0: 815.6, 1: 815.2. Samples: 2934887. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:45,471][108279] Avg episode reward: [(0, '8.880'), (1, '9.800')] -[2023-09-25 20:48:50,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11780096. Throughput: 0: 818.7, 1: 818.1. Samples: 2945002. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:50,471][108279] Avg episode reward: [(0, '8.880'), (1, '9.800')] -[2023-09-25 20:48:52,365][109224] Updated weights for policy 1, policy_version 23040 (0.0016) -[2023-09-25 20:48:52,365][109225] Updated weights for policy 0, policy_version 23040 (0.0017) -[2023-09-25 20:48:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11812864. Throughput: 0: 812.7, 1: 812.3. Samples: 2949579. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:48:55,471][108279] Avg episode reward: [(0, '8.880'), (1, '9.800')] -[2023-09-25 20:49:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11845632. Throughput: 0: 813.3, 1: 810.6. Samples: 2959360. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:49:00,470][108279] Avg episode reward: [(0, '8.890'), (1, '9.790')] -[2023-09-25 20:49:04,964][109225] Updated weights for policy 0, policy_version 23200 (0.0017) -[2023-09-25 20:49:04,964][109224] Updated weights for policy 1, policy_version 23200 (0.0016) -[2023-09-25 20:49:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11878400. Throughput: 0: 814.8, 1: 817.5. Samples: 2969329. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:49:05,471][108279] Avg episode reward: [(0, '8.930'), (1, '9.800')] -[2023-09-25 20:49:10,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11911168. Throughput: 0: 812.0, 1: 812.2. Samples: 2973993. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:49:10,471][108279] Avg episode reward: [(0, '8.930'), (1, '9.800')] -[2023-09-25 20:49:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11943936. Throughput: 0: 818.9, 1: 816.5. Samples: 2983938. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:49:15,471][108279] Avg episode reward: [(0, '8.960'), (1, '9.840')] -[2023-09-25 20:49:17,397][109224] Updated weights for policy 1, policy_version 23360 (0.0014) -[2023-09-25 20:49:17,398][109225] Updated weights for policy 0, policy_version 23360 (0.0018) -[2023-09-25 20:49:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 11976704. Throughput: 0: 815.7, 1: 818.9. Samples: 2994005. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:49:20,470][108279] Avg episode reward: [(0, '8.960'), (1, '9.840')] -[2023-09-25 20:49:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12009472. Throughput: 0: 816.6, 1: 816.5. Samples: 2998672. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:49:25,471][108279] Avg episode reward: [(0, '8.990'), (1, '9.830')] -[2023-09-25 20:49:30,073][109224] Updated weights for policy 1, policy_version 23520 (0.0018) -[2023-09-25 20:49:30,073][109225] Updated weights for policy 0, policy_version 23520 (0.0017) -[2023-09-25 20:49:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12042240. Throughput: 0: 819.1, 1: 817.0. Samples: 3008512. Policy #0 lag: (min: 8.0, avg: 8.0, max: 8.0) -[2023-09-25 20:49:30,471][108279] Avg episode reward: [(0, '8.980'), (1, '9.830')] -[2023-09-25 20:49:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12075008. Throughput: 0: 812.6, 1: 814.9. Samples: 3018238. Policy #0 lag: (min: 8.0, avg: 8.0, max: 8.0) -[2023-09-25 20:49:35,470][108279] Avg episode reward: [(0, '9.000'), (1, '9.820')] -[2023-09-25 20:49:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12107776. Throughput: 0: 815.2, 1: 813.5. Samples: 3022871. Policy #0 lag: (min: 8.0, avg: 8.0, max: 8.0) -[2023-09-25 20:49:40,471][108279] Avg episode reward: [(0, '9.010'), (1, '9.820')] -[2023-09-25 20:49:42,545][109224] Updated weights for policy 1, policy_version 23680 (0.0016) -[2023-09-25 20:49:42,545][109225] Updated weights for policy 0, policy_version 23680 (0.0017) -[2023-09-25 20:49:45,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12140544. Throughput: 0: 817.1, 1: 819.2. Samples: 3032995. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:49:45,471][108279] Avg episode reward: [(0, '9.060'), (1, '9.810')] -[2023-09-25 20:49:50,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12173312. Throughput: 0: 815.4, 1: 815.6. Samples: 3042723. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:49:50,470][108279] Avg episode reward: [(0, '9.080'), (1, '9.810')] -[2023-09-25 20:49:55,129][109224] Updated weights for policy 1, policy_version 23840 (0.0018) -[2023-09-25 20:49:55,129][109225] Updated weights for policy 0, policy_version 23840 (0.0020) -[2023-09-25 20:49:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12206080. Throughput: 0: 817.2, 1: 815.3. Samples: 3047455. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:49:55,471][108279] Avg episode reward: [(0, '9.090'), (1, '9.810')] -[2023-09-25 20:50:00,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12238848. Throughput: 0: 816.9, 1: 819.1. Samples: 3057558. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 20:50:00,471][108279] Avg episode reward: [(0, '9.070'), (1, '9.810')] -[2023-09-25 20:50:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12271616. Throughput: 0: 814.3, 1: 813.7. Samples: 3067268. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:50:05,471][108279] Avg episode reward: [(0, '9.070'), (1, '9.820')] -[2023-09-25 20:50:07,603][109225] Updated weights for policy 0, policy_version 24000 (0.0017) -[2023-09-25 20:50:07,603][109224] Updated weights for policy 1, policy_version 24000 (0.0016) -[2023-09-25 20:50:10,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12304384. Throughput: 0: 816.0, 1: 814.7. Samples: 3072053. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:50:10,470][108279] Avg episode reward: [(0, '9.060'), (1, '9.820')] -[2023-09-25 20:50:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12337152. Throughput: 0: 819.2, 1: 819.2. Samples: 3082240. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 20:50:15,471][108279] Avg episode reward: [(0, '9.070'), (1, '9.800')] -[2023-09-25 20:50:20,075][109224] Updated weights for policy 1, policy_version 24160 (0.0016) -[2023-09-25 20:50:20,075][109225] Updated weights for policy 0, policy_version 24160 (0.0018) -[2023-09-25 20:50:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12369920. Throughput: 0: 819.6, 1: 819.1. Samples: 3091978. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:50:20,470][108279] Avg episode reward: [(0, '9.080'), (1, '9.800')] -[2023-09-25 20:50:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12402688. Throughput: 0: 819.2, 1: 818.7. Samples: 3096577. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:50:25,471][108279] Avg episode reward: [(0, '9.050'), (1, '9.820')] -[2023-09-25 20:50:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 12435456. Throughput: 0: 814.7, 1: 816.1. Samples: 3106378. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:50:30,471][108279] Avg episode reward: [(0, '9.050'), (1, '9.820')] -[2023-09-25 20:50:30,481][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000024288_6217728.pth... -[2023-09-25 20:50:30,481][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000024288_6217728.pth... -[2023-09-25 20:50:30,517][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000021232_5435392.pth -[2023-09-25 20:50:30,521][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000021232_5435392.pth -[2023-09-25 20:50:33,004][109225] Updated weights for policy 0, policy_version 24320 (0.0017) -[2023-09-25 20:50:33,004][109224] Updated weights for policy 1, policy_version 24320 (0.0019) -[2023-09-25 20:50:35,470][108279] Fps is (10 sec: 5734.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12460032. Throughput: 0: 811.0, 1: 810.7. Samples: 3115702. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:50:35,470][108279] Avg episode reward: [(0, '9.020'), (1, '9.820')] -[2023-09-25 20:50:40,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12492800. Throughput: 0: 813.2, 1: 816.1. Samples: 3120777. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:50:40,471][108279] Avg episode reward: [(0, '9.020'), (1, '9.850')] -[2023-09-25 20:50:45,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12525568. Throughput: 0: 809.1, 1: 809.6. Samples: 3130398. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:50:45,471][108279] Avg episode reward: [(0, '9.030'), (1, '9.850')] -[2023-09-25 20:50:45,616][109225] Updated weights for policy 0, policy_version 24480 (0.0017) -[2023-09-25 20:50:45,616][109224] Updated weights for policy 1, policy_version 24480 (0.0015) -[2023-09-25 20:50:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 12558336. Throughput: 0: 808.3, 1: 808.3. Samples: 3140012. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:50:50,471][108279] Avg episode reward: [(0, '9.040'), (1, '9.860')] -[2023-09-25 20:50:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12591104. Throughput: 0: 812.0, 1: 813.2. Samples: 3145190. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:50:55,470][108279] Avg episode reward: [(0, '9.050'), (1, '9.850')] -[2023-09-25 20:50:58,129][109225] Updated weights for policy 0, policy_version 24640 (0.0017) -[2023-09-25 20:50:58,129][109224] Updated weights for policy 1, policy_version 24640 (0.0016) -[2023-09-25 20:51:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12623872. Throughput: 0: 805.5, 1: 807.1. Samples: 3154807. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:51:00,471][108279] Avg episode reward: [(0, '9.060'), (1, '9.850')] -[2023-09-25 20:51:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12656640. Throughput: 0: 804.0, 1: 804.1. Samples: 3164341. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 20:51:05,470][108279] Avg episode reward: [(0, '9.090'), (1, '9.850')] -[2023-09-25 20:51:10,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12689408. Throughput: 0: 806.8, 1: 809.1. Samples: 3169294. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:51:10,470][108279] Avg episode reward: [(0, '9.080'), (1, '9.860')] -[2023-09-25 20:51:10,869][109224] Updated weights for policy 1, policy_version 24800 (0.0016) -[2023-09-25 20:51:10,869][109225] Updated weights for policy 0, policy_version 24800 (0.0017) -[2023-09-25 20:51:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12722176. Throughput: 0: 806.8, 1: 806.4. Samples: 3178975. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:51:15,470][108279] Avg episode reward: [(0, '9.080'), (1, '9.870')] -[2023-09-25 20:51:20,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 12754944. Throughput: 0: 812.9, 1: 811.1. Samples: 3188781. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:51:20,471][108279] Avg episode reward: [(0, '9.060'), (1, '9.860')] -[2023-09-25 20:51:23,399][109225] Updated weights for policy 0, policy_version 24960 (0.0018) -[2023-09-25 20:51:23,399][109224] Updated weights for policy 1, policy_version 24960 (0.0016) -[2023-09-25 20:51:25,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12787712. Throughput: 0: 812.5, 1: 811.9. Samples: 3193873. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) -[2023-09-25 20:51:25,471][108279] Avg episode reward: [(0, '9.090'), (1, '9.850')] -[2023-09-25 20:51:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 12820480. Throughput: 0: 811.5, 1: 811.4. Samples: 3203430. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:51:30,471][108279] Avg episode reward: [(0, '9.090'), (1, '9.850')] -[2023-09-25 20:51:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 12853248. Throughput: 0: 815.7, 1: 813.2. Samples: 3213312. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:51:35,471][108279] Avg episode reward: [(0, '9.090'), (1, '9.850')] -[2023-09-25 20:51:35,963][109224] Updated weights for policy 1, policy_version 25120 (0.0020) -[2023-09-25 20:51:35,964][109225] Updated weights for policy 0, policy_version 25120 (0.0018) -[2023-09-25 20:51:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 12886016. Throughput: 0: 811.8, 1: 812.0. Samples: 3218258. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:51:40,471][108279] Avg episode reward: [(0, '9.100'), (1, '9.850')] -[2023-09-25 20:51:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 12918784. Throughput: 0: 810.5, 1: 811.1. Samples: 3227777. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:51:45,471][108279] Avg episode reward: [(0, '9.130'), (1, '9.860')] -[2023-09-25 20:51:48,523][109224] Updated weights for policy 1, policy_version 25280 (0.0017) -[2023-09-25 20:51:48,523][109225] Updated weights for policy 0, policy_version 25280 (0.0017) -[2023-09-25 20:51:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 12951552. Throughput: 0: 818.1, 1: 815.9. Samples: 3237871. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:51:50,471][108279] Avg episode reward: [(0, '9.150'), (1, '9.860')] -[2023-09-25 20:51:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 12984320. Throughput: 0: 814.6, 1: 815.0. Samples: 3242625. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:51:55,470][108279] Avg episode reward: [(0, '9.160'), (1, '9.860')] -[2023-09-25 20:52:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 13017088. Throughput: 0: 817.5, 1: 817.1. Samples: 3252534. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:00,471][108279] Avg episode reward: [(0, '9.140'), (1, '9.870')] -[2023-09-25 20:52:00,976][109225] Updated weights for policy 0, policy_version 25440 (0.0018) -[2023-09-25 20:52:00,976][109224] Updated weights for policy 1, policy_version 25440 (0.0018) -[2023-09-25 20:52:05,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 13049856. Throughput: 0: 819.2, 1: 818.3. Samples: 3262468. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:05,471][108279] Avg episode reward: [(0, '9.110'), (1, '9.870')] -[2023-09-25 20:52:10,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 13082624. Throughput: 0: 817.7, 1: 816.0. Samples: 3267386. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:10,470][108279] Avg episode reward: [(0, '9.110'), (1, '9.880')] -[2023-09-25 20:52:13,519][109224] Updated weights for policy 1, policy_version 25600 (0.0016) -[2023-09-25 20:52:13,520][109225] Updated weights for policy 0, policy_version 25600 (0.0018) -[2023-09-25 20:52:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6511.9). Total num frames: 13115392. Throughput: 0: 817.0, 1: 817.2. Samples: 3276967. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:15,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.880')] -[2023-09-25 20:52:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13148160. Throughput: 0: 819.2, 1: 819.2. Samples: 3287041. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:20,471][108279] Avg episode reward: [(0, '9.130'), (1, '9.890')] -[2023-09-25 20:52:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13180928. Throughput: 0: 818.1, 1: 818.1. Samples: 3291888. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:25,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.890')] -[2023-09-25 20:52:25,959][109224] Updated weights for policy 1, policy_version 25760 (0.0015) -[2023-09-25 20:52:25,960][109225] Updated weights for policy 0, policy_version 25760 (0.0018) -[2023-09-25 20:52:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13213696. Throughput: 0: 821.8, 1: 822.1. Samples: 3301753. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:30,471][108279] Avg episode reward: [(0, '9.130'), (1, '9.870')] -[2023-09-25 20:52:30,482][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000025808_6606848.pth... -[2023-09-25 20:52:30,482][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000025808_6606848.pth... -[2023-09-25 20:52:30,518][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000022752_5824512.pth -[2023-09-25 20:52:30,520][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000022752_5824512.pth -[2023-09-25 20:52:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13246464. Throughput: 0: 819.6, 1: 819.2. Samples: 3311617. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:35,471][108279] Avg episode reward: [(0, '9.150'), (1, '9.870')] -[2023-09-25 20:52:38,610][109224] Updated weights for policy 1, policy_version 25920 (0.0018) -[2023-09-25 20:52:38,610][109225] Updated weights for policy 0, policy_version 25920 (0.0017) -[2023-09-25 20:52:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13279232. Throughput: 0: 817.0, 1: 817.0. Samples: 3316153. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:52:40,471][108279] Avg episode reward: [(0, '9.150'), (1, '9.870')] -[2023-09-25 20:52:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13312000. Throughput: 0: 817.1, 1: 814.4. Samples: 3325953. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:52:45,471][108279] Avg episode reward: [(0, '9.160'), (1, '9.870')] -[2023-09-25 20:52:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13344768. Throughput: 0: 817.9, 1: 819.1. Samples: 3336134. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:52:50,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.880')] -[2023-09-25 20:52:51,038][109224] Updated weights for policy 1, policy_version 26080 (0.0017) -[2023-09-25 20:52:51,038][109225] Updated weights for policy 0, policy_version 26080 (0.0017) -[2023-09-25 20:52:55,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13377536. Throughput: 0: 817.4, 1: 818.7. Samples: 3341012. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:52:55,470][108279] Avg episode reward: [(0, '9.110'), (1, '9.880')] -[2023-09-25 20:53:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13410304. Throughput: 0: 821.6, 1: 821.3. Samples: 3350897. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:00,470][108279] Avg episode reward: [(0, '9.120'), (1, '9.880')] -[2023-09-25 20:53:03,469][109224] Updated weights for policy 1, policy_version 26240 (0.0019) -[2023-09-25 20:53:03,469][109225] Updated weights for policy 0, policy_version 26240 (0.0019) -[2023-09-25 20:53:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13443072. Throughput: 0: 819.2, 1: 819.2. Samples: 3360768. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:05,471][108279] Avg episode reward: [(0, '9.130'), (1, '9.870')] -[2023-09-25 20:53:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13475840. Throughput: 0: 820.1, 1: 820.2. Samples: 3365703. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:10,470][108279] Avg episode reward: [(0, '9.090'), (1, '9.870')] -[2023-09-25 20:53:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13508608. Throughput: 0: 819.8, 1: 819.6. Samples: 3375525. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:15,470][108279] Avg episode reward: [(0, '9.090'), (1, '9.870')] -[2023-09-25 20:53:15,870][109224] Updated weights for policy 1, policy_version 26400 (0.0012) -[2023-09-25 20:53:15,871][109225] Updated weights for policy 0, policy_version 26400 (0.0017) -[2023-09-25 20:53:20,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13541376. Throughput: 0: 819.2, 1: 820.1. Samples: 3385388. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:20,471][108279] Avg episode reward: [(0, '9.070'), (1, '9.880')] -[2023-09-25 20:53:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13574144. Throughput: 0: 825.6, 1: 825.6. Samples: 3390455. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:25,470][108279] Avg episode reward: [(0, '9.060'), (1, '9.880')] -[2023-09-25 20:53:28,380][109224] Updated weights for policy 1, policy_version 26560 (0.0016) -[2023-09-25 20:53:28,382][109225] Updated weights for policy 0, policy_version 26560 (0.0018) -[2023-09-25 20:53:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13606912. Throughput: 0: 822.8, 1: 825.5. Samples: 3400125. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:30,471][108279] Avg episode reward: [(0, '9.040'), (1, '9.880')] -[2023-09-25 20:53:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13639680. Throughput: 0: 819.7, 1: 819.2. Samples: 3409884. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:53:35,470][108279] Avg episode reward: [(0, '9.020'), (1, '9.880')] -[2023-09-25 20:53:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13672448. Throughput: 0: 817.7, 1: 817.2. Samples: 3414579. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:53:40,470][108279] Avg episode reward: [(0, '9.010'), (1, '9.880')] -[2023-09-25 20:53:41,049][109224] Updated weights for policy 1, policy_version 26720 (0.0017) -[2023-09-25 20:53:41,050][109225] Updated weights for policy 0, policy_version 26720 (0.0016) -[2023-09-25 20:53:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13705216. Throughput: 0: 816.6, 1: 816.6. Samples: 3424393. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:53:45,471][108279] Avg episode reward: [(0, '9.020'), (1, '9.890')] -[2023-09-25 20:53:50,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13737984. Throughput: 0: 819.2, 1: 819.2. Samples: 3434496. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:50,471][108279] Avg episode reward: [(0, '8.980'), (1, '9.890')] -[2023-09-25 20:53:53,486][109225] Updated weights for policy 0, policy_version 26880 (0.0016) -[2023-09-25 20:53:53,487][109224] Updated weights for policy 1, policy_version 26880 (0.0018) -[2023-09-25 20:53:55,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13770752. Throughput: 0: 818.0, 1: 818.4. Samples: 3439345. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:53:55,470][108279] Avg episode reward: [(0, '8.970'), (1, '9.900')] -[2023-09-25 20:54:00,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13803520. Throughput: 0: 817.0, 1: 816.9. Samples: 3449047. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:00,470][108279] Avg episode reward: [(0, '8.980'), (1, '9.900')] -[2023-09-25 20:54:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13836288. Throughput: 0: 816.6, 1: 818.3. Samples: 3458956. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:05,470][108279] Avg episode reward: [(0, '8.990'), (1, '9.910')] -[2023-09-25 20:54:06,154][109224] Updated weights for policy 1, policy_version 27040 (0.0018) -[2023-09-25 20:54:06,155][109225] Updated weights for policy 0, policy_version 27040 (0.0018) -[2023-09-25 20:54:10,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13869056. Throughput: 0: 812.9, 1: 812.8. Samples: 3463614. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:10,471][108279] Avg episode reward: [(0, '9.010'), (1, '9.920')] -[2023-09-25 20:54:15,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13901824. Throughput: 0: 815.6, 1: 813.7. Samples: 3473444. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:15,471][108279] Avg episode reward: [(0, '9.010'), (1, '9.920')] -[2023-09-25 20:54:18,632][109224] Updated weights for policy 1, policy_version 27200 (0.0017) -[2023-09-25 20:54:18,632][109225] Updated weights for policy 0, policy_version 27200 (0.0018) -[2023-09-25 20:54:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13934592. Throughput: 0: 817.8, 1: 819.2. Samples: 3483549. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:20,471][108279] Avg episode reward: [(0, '9.020'), (1, '9.920')] -[2023-09-25 20:54:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 13967360. Throughput: 0: 818.3, 1: 818.9. Samples: 3488255. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:25,470][108279] Avg episode reward: [(0, '9.020'), (1, '9.920')] -[2023-09-25 20:54:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14000128. Throughput: 0: 819.0, 1: 816.4. Samples: 3497984. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:30,471][108279] Avg episode reward: [(0, '9.020'), (1, '9.920')] -[2023-09-25 20:54:30,481][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000027344_7000064.pth... -[2023-09-25 20:54:30,481][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000027344_7000064.pth... -[2023-09-25 20:54:30,510][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000024288_6217728.pth -[2023-09-25 20:54:30,515][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000024288_6217728.pth -[2023-09-25 20:54:31,440][109224] Updated weights for policy 1, policy_version 27360 (0.0019) -[2023-09-25 20:54:31,440][109225] Updated weights for policy 0, policy_version 27360 (0.0019) -[2023-09-25 20:54:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14032896. Throughput: 0: 809.7, 1: 812.2. Samples: 3507480. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:35,470][108279] Avg episode reward: [(0, '9.030'), (1, '9.930')] -[2023-09-25 20:54:35,471][109025] Saving new best policy, reward=9.930! -[2023-09-25 20:54:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14065664. Throughput: 0: 812.3, 1: 809.3. Samples: 3512316. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:40,471][108279] Avg episode reward: [(0, '9.030'), (1, '9.930')] -[2023-09-25 20:54:44,024][109225] Updated weights for policy 0, policy_version 27520 (0.0017) -[2023-09-25 20:54:44,024][109224] Updated weights for policy 1, policy_version 27520 (0.0017) -[2023-09-25 20:54:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14098432. Throughput: 0: 812.7, 1: 812.4. Samples: 3522179. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:45,471][108279] Avg episode reward: [(0, '9.070'), (1, '9.930')] -[2023-09-25 20:54:50,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14131200. Throughput: 0: 810.0, 1: 810.2. Samples: 3531863. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:50,470][108279] Avg episode reward: [(0, '9.070'), (1, '9.930')] -[2023-09-25 20:54:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14163968. Throughput: 0: 815.2, 1: 813.0. Samples: 3536880. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:54:55,471][108279] Avg episode reward: [(0, '9.120'), (1, '9.930')] -[2023-09-25 20:54:56,598][109225] Updated weights for policy 0, policy_version 27680 (0.0017) -[2023-09-25 20:54:56,598][109224] Updated weights for policy 1, policy_version 27680 (0.0017) -[2023-09-25 20:55:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14196736. Throughput: 0: 812.3, 1: 814.2. Samples: 3546638. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:00,470][108279] Avg episode reward: [(0, '9.110'), (1, '9.930')] -[2023-09-25 20:55:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14229504. Throughput: 0: 809.3, 1: 809.8. Samples: 3556410. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:55:05,470][108279] Avg episode reward: [(0, '9.100'), (1, '9.930')] -[2023-09-25 20:55:09,053][109224] Updated weights for policy 1, policy_version 27840 (0.0014) -[2023-09-25 20:55:09,054][109225] Updated weights for policy 0, policy_version 27840 (0.0016) -[2023-09-25 20:55:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14262272. Throughput: 0: 814.8, 1: 812.2. Samples: 3561472. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:55:10,471][108279] Avg episode reward: [(0, '9.080'), (1, '9.930')] -[2023-09-25 20:55:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14295040. Throughput: 0: 813.1, 1: 816.1. Samples: 3571297. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:55:15,471][108279] Avg episode reward: [(0, '9.080'), (1, '9.940')] -[2023-09-25 20:55:15,480][109025] Saving new best policy, reward=9.940! -[2023-09-25 20:55:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14327808. Throughput: 0: 818.6, 1: 818.9. Samples: 3581170. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) -[2023-09-25 20:55:20,470][108279] Avg episode reward: [(0, '9.100'), (1, '9.940')] -[2023-09-25 20:55:21,453][109225] Updated weights for policy 0, policy_version 28000 (0.0014) -[2023-09-25 20:55:21,454][109224] Updated weights for policy 1, policy_version 28000 (0.0017) -[2023-09-25 20:55:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 14360576. Throughput: 0: 819.3, 1: 819.3. Samples: 3586053. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:25,470][108279] Avg episode reward: [(0, '9.130'), (1, '9.940')] -[2023-09-25 20:55:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14393344. Throughput: 0: 820.8, 1: 821.9. Samples: 3596098. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:30,470][108279] Avg episode reward: [(0, '9.150'), (1, '9.940')] -[2023-09-25 20:55:34,020][109225] Updated weights for policy 0, policy_version 28160 (0.0016) -[2023-09-25 20:55:34,020][109224] Updated weights for policy 1, policy_version 28160 (0.0015) -[2023-09-25 20:55:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14426112. Throughput: 0: 819.6, 1: 819.2. Samples: 3605611. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:35,470][108279] Avg episode reward: [(0, '9.180'), (1, '9.950')] -[2023-09-25 20:55:35,471][109025] Saving new best policy, reward=9.950! -[2023-09-25 20:55:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14458880. Throughput: 0: 819.6, 1: 819.2. Samples: 3610624. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:40,471][108279] Avg episode reward: [(0, '9.190'), (1, '9.950')] -[2023-09-25 20:55:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14491648. Throughput: 0: 821.0, 1: 821.0. Samples: 3620530. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:45,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.960')] -[2023-09-25 20:55:45,484][109025] Saving new best policy, reward=9.960! -[2023-09-25 20:55:46,480][109225] Updated weights for policy 0, policy_version 28320 (0.0018) -[2023-09-25 20:55:46,481][109224] Updated weights for policy 1, policy_version 28320 (0.0018) -[2023-09-25 20:55:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14524416. Throughput: 0: 821.9, 1: 821.6. Samples: 3630369. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:50,471][108279] Avg episode reward: [(0, '9.200'), (1, '9.960')] -[2023-09-25 20:55:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14557184. Throughput: 0: 819.2, 1: 819.2. Samples: 3635201. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:55:55,471][108279] Avg episode reward: [(0, '9.230'), (1, '9.960')] -[2023-09-25 20:55:58,932][109225] Updated weights for policy 0, policy_version 28480 (0.0017) -[2023-09-25 20:55:58,932][109224] Updated weights for policy 1, policy_version 28480 (0.0017) -[2023-09-25 20:56:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14589952. Throughput: 0: 821.1, 1: 821.2. Samples: 3645204. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:56:00,470][108279] Avg episode reward: [(0, '9.230'), (1, '9.960')] -[2023-09-25 20:56:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14622720. Throughput: 0: 818.6, 1: 817.9. Samples: 3654810. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:56:05,471][108279] Avg episode reward: [(0, '9.210'), (1, '9.960')] -[2023-09-25 20:56:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14655488. Throughput: 0: 818.5, 1: 819.1. Samples: 3659743. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:56:10,470][108279] Avg episode reward: [(0, '9.230'), (1, '9.960')] -[2023-09-25 20:56:11,554][109225] Updated weights for policy 0, policy_version 28640 (0.0014) -[2023-09-25 20:56:11,554][109224] Updated weights for policy 1, policy_version 28640 (0.0018) -[2023-09-25 20:56:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14688256. Throughput: 0: 817.3, 1: 816.6. Samples: 3669625. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:56:15,470][108279] Avg episode reward: [(0, '9.270'), (1, '9.960')] -[2023-09-25 20:56:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14721024. Throughput: 0: 820.0, 1: 820.2. Samples: 3679422. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 20:56:20,471][108279] Avg episode reward: [(0, '9.270'), (1, '9.960')] -[2023-09-25 20:56:24,006][109224] Updated weights for policy 1, policy_version 28800 (0.0015) -[2023-09-25 20:56:24,007][109225] Updated weights for policy 0, policy_version 28800 (0.0017) -[2023-09-25 20:56:25,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14753792. Throughput: 0: 819.2, 1: 819.2. Samples: 3684352. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:56:25,471][108279] Avg episode reward: [(0, '9.280'), (1, '9.960')] -[2023-09-25 20:56:30,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 14782464. Throughput: 0: 813.7, 1: 813.6. Samples: 3693761. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:56:30,471][108279] Avg episode reward: [(0, '9.280'), (1, '9.950')] -[2023-09-25 20:56:30,482][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000028880_7393280.pth... -[2023-09-25 20:56:30,514][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000025808_6606848.pth -[2023-09-25 20:56:30,517][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000028880_7393280.pth... -[2023-09-25 20:56:30,546][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000025808_6606848.pth -[2023-09-25 20:56:35,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 14811136. Throughput: 0: 811.7, 1: 812.0. Samples: 3703436. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 20:56:35,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.950')] -[2023-09-25 20:56:36,742][109225] Updated weights for policy 0, policy_version 28960 (0.0017) -[2023-09-25 20:56:36,742][109224] Updated weights for policy 1, policy_version 28960 (0.0017) -[2023-09-25 20:56:40,470][108279] Fps is (10 sec: 6963.2, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14852096. Throughput: 0: 814.5, 1: 817.0. Samples: 3708618. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:56:40,471][108279] Avg episode reward: [(0, '9.250'), (1, '9.950')] -[2023-09-25 20:56:45,470][108279] Fps is (10 sec: 7372.8, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14884864. Throughput: 0: 814.7, 1: 814.4. Samples: 3718516. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:56:45,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.950')] -[2023-09-25 20:56:49,194][109225] Updated weights for policy 0, policy_version 29120 (0.0016) -[2023-09-25 20:56:49,195][109224] Updated weights for policy 1, policy_version 29120 (0.0017) -[2023-09-25 20:56:50,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 14913536. Throughput: 0: 814.3, 1: 815.5. Samples: 3728151. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:56:50,471][108279] Avg episode reward: [(0, '9.240'), (1, '9.970')] -[2023-09-25 20:56:50,472][109025] Saving new best policy, reward=9.970! -[2023-09-25 20:56:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14950400. Throughput: 0: 816.2, 1: 818.0. Samples: 3733280. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:56:55,471][108279] Avg episode reward: [(0, '9.250'), (1, '9.970')] -[2023-09-25 20:57:00,470][108279] Fps is (10 sec: 6963.1, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 14983168. Throughput: 0: 815.2, 1: 815.7. Samples: 3743014. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:57:00,471][108279] Avg episode reward: [(0, '9.260'), (1, '9.970')] -[2023-09-25 20:57:01,692][109224] Updated weights for policy 1, policy_version 29280 (0.0016) -[2023-09-25 20:57:01,692][109225] Updated weights for policy 0, policy_version 29280 (0.0017) -[2023-09-25 20:57:05,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15007744. Throughput: 0: 811.9, 1: 811.3. Samples: 3752467. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:57:05,471][108279] Avg episode reward: [(0, '9.300'), (1, '9.970')] -[2023-09-25 20:57:10,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15040512. Throughput: 0: 811.6, 1: 813.8. Samples: 3757499. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:57:10,471][108279] Avg episode reward: [(0, '9.300'), (1, '9.970')] -[2023-09-25 20:57:14,397][109224] Updated weights for policy 1, policy_version 29440 (0.0017) -[2023-09-25 20:57:14,397][109225] Updated weights for policy 0, policy_version 29440 (0.0017) -[2023-09-25 20:57:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 15073280. Throughput: 0: 817.0, 1: 815.8. Samples: 3767237. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:57:15,471][108279] Avg episode reward: [(0, '9.290'), (1, '9.970')] -[2023-09-25 20:57:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15106048. Throughput: 0: 817.5, 1: 817.4. Samples: 3777009. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 20:57:20,471][108279] Avg episode reward: [(0, '9.330'), (1, '9.980')] -[2023-09-25 20:57:20,555][109025] Saving new best policy, reward=9.980! -[2023-09-25 20:57:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15138816. Throughput: 0: 816.7, 1: 816.8. Samples: 3782127. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:57:25,471][108279] Avg episode reward: [(0, '9.320'), (1, '9.980')] -[2023-09-25 20:57:26,808][109224] Updated weights for policy 1, policy_version 29600 (0.0017) -[2023-09-25 20:57:26,808][109225] Updated weights for policy 0, policy_version 29600 (0.0018) -[2023-09-25 20:57:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6485.3, 300 sec: 6525.8). Total num frames: 15171584. Throughput: 0: 816.1, 1: 815.5. Samples: 3791939. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:57:30,470][108279] Avg episode reward: [(0, '9.300'), (1, '9.980')] -[2023-09-25 20:57:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15204352. Throughput: 0: 814.7, 1: 813.5. Samples: 3801422. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:57:35,471][108279] Avg episode reward: [(0, '9.350'), (1, '9.980')] -[2023-09-25 20:57:39,541][109224] Updated weights for policy 1, policy_version 29760 (0.0016) -[2023-09-25 20:57:39,542][109225] Updated weights for policy 0, policy_version 29760 (0.0017) -[2023-09-25 20:57:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15237120. Throughput: 0: 812.2, 1: 812.2. Samples: 3806378. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:57:40,471][108279] Avg episode reward: [(0, '9.360'), (1, '9.970')] -[2023-09-25 20:57:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15269888. Throughput: 0: 807.7, 1: 807.4. Samples: 3815691. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:57:45,471][108279] Avg episode reward: [(0, '9.360'), (1, '9.970')] -[2023-09-25 20:57:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6485.3, 300 sec: 6525.8). Total num frames: 15302656. Throughput: 0: 814.2, 1: 812.4. Samples: 3825668. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:57:50,471][108279] Avg episode reward: [(0, '9.400'), (1, '9.970')] -[2023-09-25 20:57:52,133][109224] Updated weights for policy 1, policy_version 29920 (0.0016) -[2023-09-25 20:57:52,133][109225] Updated weights for policy 0, policy_version 29920 (0.0017) -[2023-09-25 20:57:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15335424. Throughput: 0: 813.3, 1: 813.8. Samples: 3830720. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:57:55,470][108279] Avg episode reward: [(0, '9.400'), (1, '9.970')] -[2023-09-25 20:58:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 15368192. Throughput: 0: 811.4, 1: 812.4. Samples: 3840308. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:58:00,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.970')] -[2023-09-25 20:58:04,703][109224] Updated weights for policy 1, policy_version 30080 (0.0018) -[2023-09-25 20:58:04,704][109225] Updated weights for policy 0, policy_version 30080 (0.0018) -[2023-09-25 20:58:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15400960. Throughput: 0: 815.0, 1: 812.5. Samples: 3850244. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:58:05,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.970')] -[2023-09-25 20:58:10,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15433728. Throughput: 0: 811.4, 1: 811.4. Samples: 3855156. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:58:10,470][108279] Avg episode reward: [(0, '9.420'), (1, '9.970')] -[2023-09-25 20:58:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15466496. Throughput: 0: 809.7, 1: 810.2. Samples: 3864837. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:58:15,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.970')] -[2023-09-25 20:58:17,144][109224] Updated weights for policy 1, policy_version 30240 (0.0014) -[2023-09-25 20:58:17,144][109225] Updated weights for policy 0, policy_version 30240 (0.0017) -[2023-09-25 20:58:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15499264. Throughput: 0: 816.6, 1: 815.7. Samples: 3874874. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) -[2023-09-25 20:58:20,471][108279] Avg episode reward: [(0, '9.390'), (1, '9.970')] -[2023-09-25 20:58:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15532032. Throughput: 0: 815.9, 1: 816.4. Samples: 3879834. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:58:25,471][108279] Avg episode reward: [(0, '9.380'), (1, '9.970')] -[2023-09-25 20:58:29,748][109224] Updated weights for policy 1, policy_version 30400 (0.0018) -[2023-09-25 20:58:29,748][109225] Updated weights for policy 0, policy_version 30400 (0.0019) -[2023-09-25 20:58:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15564800. Throughput: 0: 818.4, 1: 818.4. Samples: 3889347. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:58:30,471][108279] Avg episode reward: [(0, '9.390'), (1, '9.970')] -[2023-09-25 20:58:30,480][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000030400_7782400.pth... -[2023-09-25 20:58:30,480][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000030400_7782400.pth... -[2023-09-25 20:58:30,509][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000027344_7000064.pth -[2023-09-25 20:58:30,516][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000027344_7000064.pth -[2023-09-25 20:58:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15597568. Throughput: 0: 816.3, 1: 819.0. Samples: 3899257. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:58:35,470][108279] Avg episode reward: [(0, '9.410'), (1, '9.970')] -[2023-09-25 20:58:40,470][108279] Fps is (10 sec: 6553.9, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15630336. Throughput: 0: 813.0, 1: 812.6. Samples: 3903872. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) -[2023-09-25 20:58:40,470][108279] Avg episode reward: [(0, '9.400'), (1, '9.970')] -[2023-09-25 20:58:42,422][109224] Updated weights for policy 1, policy_version 30560 (0.0017) -[2023-09-25 20:58:42,422][109225] Updated weights for policy 0, policy_version 30560 (0.0018) -[2023-09-25 20:58:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15663104. Throughput: 0: 817.0, 1: 814.7. Samples: 3913736. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:58:45,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.970')] -[2023-09-25 20:58:50,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15695872. Throughput: 0: 817.9, 1: 819.1. Samples: 3923909. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:58:50,471][108279] Avg episode reward: [(0, '9.410'), (1, '9.970')] -[2023-09-25 20:58:54,816][109225] Updated weights for policy 0, policy_version 30720 (0.0017) -[2023-09-25 20:58:54,817][109224] Updated weights for policy 1, policy_version 30720 (0.0015) -[2023-09-25 20:58:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15728640. Throughput: 0: 816.6, 1: 816.1. Samples: 3928627. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:58:55,471][108279] Avg episode reward: [(0, '9.420'), (1, '9.970')] -[2023-09-25 20:59:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15761408. Throughput: 0: 817.7, 1: 817.1. Samples: 3938405. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) -[2023-09-25 20:59:00,470][108279] Avg episode reward: [(0, '9.430'), (1, '9.970')] -[2023-09-25 20:59:05,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15794176. Throughput: 0: 818.5, 1: 818.0. Samples: 3948515. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:05,470][108279] Avg episode reward: [(0, '9.430'), (1, '9.970')] -[2023-09-25 20:59:07,339][109225] Updated weights for policy 0, policy_version 30880 (0.0017) -[2023-09-25 20:59:07,339][109224] Updated weights for policy 1, policy_version 30880 (0.0016) -[2023-09-25 20:59:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15826944. Throughput: 0: 814.8, 1: 814.6. Samples: 3953159. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:10,470][108279] Avg episode reward: [(0, '9.430'), (1, '9.970')] -[2023-09-25 20:59:15,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15859712. Throughput: 0: 818.4, 1: 817.3. Samples: 3962953. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:15,471][108279] Avg episode reward: [(0, '9.450'), (1, '9.970')] -[2023-09-25 20:59:19,922][109224] Updated weights for policy 1, policy_version 31040 (0.0015) -[2023-09-25 20:59:19,922][109225] Updated weights for policy 0, policy_version 31040 (0.0017) -[2023-09-25 20:59:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15892480. Throughput: 0: 819.7, 1: 818.5. Samples: 3972978. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:20,471][108279] Avg episode reward: [(0, '9.470'), (1, '9.970')] -[2023-09-25 20:59:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15925248. Throughput: 0: 818.1, 1: 818.4. Samples: 3977514. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:25,471][108279] Avg episode reward: [(0, '9.490'), (1, '9.970')] -[2023-09-25 20:59:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15958016. Throughput: 0: 818.8, 1: 819.0. Samples: 3987437. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:30,470][108279] Avg episode reward: [(0, '9.500'), (1, '9.970')] -[2023-09-25 20:59:32,695][109225] Updated weights for policy 0, policy_version 31200 (0.0017) -[2023-09-25 20:59:32,695][109224] Updated weights for policy 1, policy_version 31200 (0.0017) -[2023-09-25 20:59:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 15990784. Throughput: 0: 809.2, 1: 812.0. Samples: 3996861. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:35,471][108279] Avg episode reward: [(0, '9.500'), (1, '9.970')] -[2023-09-25 20:59:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16023552. Throughput: 0: 814.0, 1: 811.9. Samples: 4001792. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:40,471][108279] Avg episode reward: [(0, '9.510'), (1, '9.970')] -[2023-09-25 20:59:45,200][109225] Updated weights for policy 0, policy_version 31360 (0.0014) -[2023-09-25 20:59:45,200][109224] Updated weights for policy 1, policy_version 31360 (0.0017) -[2023-09-25 20:59:45,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16056320. Throughput: 0: 815.0, 1: 815.9. Samples: 4011797. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:45,471][108279] Avg episode reward: [(0, '9.510'), (1, '9.970')] -[2023-09-25 20:59:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16089088. Throughput: 0: 810.2, 1: 812.3. Samples: 4021531. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:50,471][108279] Avg episode reward: [(0, '9.510'), (1, '9.970')] -[2023-09-25 20:59:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16121856. Throughput: 0: 814.9, 1: 812.0. Samples: 4026368. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 20:59:55,471][108279] Avg episode reward: [(0, '9.510'), (1, '9.970')] -[2023-09-25 20:59:57,706][109224] Updated weights for policy 1, policy_version 31520 (0.0017) -[2023-09-25 20:59:57,707][109225] Updated weights for policy 0, policy_version 31520 (0.0018) -[2023-09-25 21:00:00,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16154624. Throughput: 0: 815.3, 1: 816.6. Samples: 4036387. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:00:00,471][108279] Avg episode reward: [(0, '9.530'), (1, '9.970')] -[2023-09-25 21:00:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16187392. Throughput: 0: 812.2, 1: 813.6. Samples: 4046139. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 21:00:05,471][108279] Avg episode reward: [(0, '9.520'), (1, '9.970')] -[2023-09-25 21:00:10,320][109225] Updated weights for policy 0, policy_version 31680 (0.0017) -[2023-09-25 21:00:10,321][109224] Updated weights for policy 1, policy_version 31680 (0.0017) -[2023-09-25 21:00:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16220160. Throughput: 0: 817.2, 1: 814.6. Samples: 4050944. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 21:00:10,471][108279] Avg episode reward: [(0, '9.510'), (1, '9.970')] -[2023-09-25 21:00:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16252928. Throughput: 0: 813.4, 1: 815.5. Samples: 4060738. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 21:00:15,471][108279] Avg episode reward: [(0, '9.540'), (1, '9.970')] -[2023-09-25 21:00:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16285696. Throughput: 0: 818.1, 1: 815.9. Samples: 4070390. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 21:00:20,470][108279] Avg episode reward: [(0, '9.530'), (1, '9.970')] -[2023-09-25 21:00:22,864][109225] Updated weights for policy 0, policy_version 31840 (0.0018) -[2023-09-25 21:00:22,864][109224] Updated weights for policy 1, policy_version 31840 (0.0018) -[2023-09-25 21:00:25,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16318464. Throughput: 0: 817.5, 1: 819.2. Samples: 4075444. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) -[2023-09-25 21:00:25,470][108279] Avg episode reward: [(0, '9.530'), (1, '9.970')] -[2023-09-25 21:00:30,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16351232. Throughput: 0: 815.2, 1: 814.2. Samples: 4085120. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:00:30,471][108279] Avg episode reward: [(0, '9.520'), (1, '9.970')] -[2023-09-25 21:00:30,480][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000031936_8175616.pth... -[2023-09-25 21:00:30,481][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000031936_8175616.pth... -[2023-09-25 21:00:30,519][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000028880_7393280.pth -[2023-09-25 21:00:30,519][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000028880_7393280.pth -[2023-09-25 21:00:35,470][108279] Fps is (10 sec: 5734.3, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 16375808. Throughput: 0: 813.3, 1: 812.8. Samples: 4094705. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:00:35,471][108279] Avg episode reward: [(0, '9.520'), (1, '9.970')] -[2023-09-25 21:00:35,497][109224] Updated weights for policy 1, policy_version 32000 (0.0017) -[2023-09-25 21:00:35,497][109225] Updated weights for policy 0, policy_version 32000 (0.0016) -[2023-09-25 21:00:40,470][108279] Fps is (10 sec: 6144.2, 60 sec: 6485.3, 300 sec: 6511.9). Total num frames: 16412672. Throughput: 0: 814.4, 1: 816.8. Samples: 4099773. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:00:40,471][108279] Avg episode reward: [(0, '9.540'), (1, '9.970')] -[2023-09-25 21:00:45,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16449536. Throughput: 0: 813.1, 1: 813.1. Samples: 4109563. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:00:45,470][108279] Avg episode reward: [(0, '9.540'), (1, '9.980')] -[2023-09-25 21:00:47,990][109225] Updated weights for policy 0, policy_version 32160 (0.0017) -[2023-09-25 21:00:47,990][109224] Updated weights for policy 1, policy_version 32160 (0.0017) -[2023-09-25 21:00:50,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 16474112. Throughput: 0: 812.6, 1: 811.9. Samples: 4119244. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:00:50,471][108279] Avg episode reward: [(0, '9.580'), (1, '9.980')] -[2023-09-25 21:00:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16515072. Throughput: 0: 815.0, 1: 817.6. Samples: 4124409. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:00:55,471][108279] Avg episode reward: [(0, '9.600'), (1, '9.980')] -[2023-09-25 21:01:00,357][109225] Updated weights for policy 0, policy_version 32320 (0.0016) -[2023-09-25 21:01:00,357][109224] Updated weights for policy 1, policy_version 32320 (0.0017) -[2023-09-25 21:01:00,470][108279] Fps is (10 sec: 7372.9, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16547840. Throughput: 0: 817.6, 1: 817.4. Samples: 4134316. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:00,470][108279] Avg episode reward: [(0, '9.620'), (1, '9.980')] -[2023-09-25 21:01:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16580608. Throughput: 0: 818.2, 1: 818.0. Samples: 4144021. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:05,471][108279] Avg episode reward: [(0, '9.650'), (1, '9.980')] -[2023-09-25 21:01:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 16613376. Throughput: 0: 817.2, 1: 818.1. Samples: 4149034. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:10,471][108279] Avg episode reward: [(0, '9.650'), (1, '9.980')] -[2023-09-25 21:01:12,965][109224] Updated weights for policy 1, policy_version 32480 (0.0017) -[2023-09-25 21:01:12,965][109225] Updated weights for policy 0, policy_version 32480 (0.0016) -[2023-09-25 21:01:15,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6498.1). Total num frames: 16637952. Throughput: 0: 817.6, 1: 817.1. Samples: 4158681. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:15,471][108279] Avg episode reward: [(0, '9.670'), (1, '9.980')] -[2023-09-25 21:01:20,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.0, 300 sec: 6498.1). Total num frames: 16670720. Throughput: 0: 817.1, 1: 817.4. Samples: 4168255. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:20,471][108279] Avg episode reward: [(0, '9.660'), (1, '9.950')] -[2023-09-25 21:01:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6511.9). Total num frames: 16703488. Throughput: 0: 819.1, 1: 818.7. Samples: 4173474. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:25,471][108279] Avg episode reward: [(0, '9.670'), (1, '9.950')] -[2023-09-25 21:01:25,566][109224] Updated weights for policy 1, policy_version 32640 (0.0018) -[2023-09-25 21:01:25,566][109225] Updated weights for policy 0, policy_version 32640 (0.0018) -[2023-09-25 21:01:30,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 16736256. Throughput: 0: 817.7, 1: 817.8. Samples: 4183161. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:30,470][108279] Avg episode reward: [(0, '9.670'), (1, '9.950')] -[2023-09-25 21:01:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6498.1). Total num frames: 16769024. Throughput: 0: 819.0, 1: 819.1. Samples: 4192961. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:35,471][108279] Avg episode reward: [(0, '9.660'), (1, '9.950')] -[2023-09-25 21:01:37,988][109225] Updated weights for policy 0, policy_version 32800 (0.0016) -[2023-09-25 21:01:37,988][109224] Updated weights for policy 1, policy_version 32800 (0.0017) -[2023-09-25 21:01:40,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6485.3, 300 sec: 6498.1). Total num frames: 16801792. Throughput: 0: 818.4, 1: 819.1. Samples: 4198095. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:40,471][108279] Avg episode reward: [(0, '9.660'), (1, '9.950')] -[2023-09-25 21:01:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6511.9). Total num frames: 16834560. Throughput: 0: 815.2, 1: 815.3. Samples: 4207691. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:45,470][108279] Avg episode reward: [(0, '9.710'), (1, '9.950')] -[2023-09-25 21:01:50,459][109225] Updated weights for policy 0, policy_version 32960 (0.0016) -[2023-09-25 21:01:50,460][109224] Updated weights for policy 1, policy_version 32960 (0.0017) -[2023-09-25 21:01:50,470][108279] Fps is (10 sec: 7372.8, 60 sec: 6690.1, 300 sec: 6525.8). Total num frames: 16875520. Throughput: 0: 816.9, 1: 817.6. Samples: 4217577. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:50,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.950')] -[2023-09-25 21:01:55,470][108279] Fps is (10 sec: 6963.1, 60 sec: 6485.3, 300 sec: 6511.9). Total num frames: 16904192. Throughput: 0: 819.9, 1: 819.3. Samples: 4222800. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:01:55,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.960')] -[2023-09-25 21:02:00,470][108279] Fps is (10 sec: 6144.0, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 16936960. Throughput: 0: 818.3, 1: 819.1. Samples: 4232364. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:00,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.960')] -[2023-09-25 21:02:03,000][109224] Updated weights for policy 1, policy_version 33120 (0.0017) -[2023-09-25 21:02:03,000][109225] Updated weights for policy 0, policy_version 33120 (0.0018) -[2023-09-25 21:02:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 16969728. Throughput: 0: 821.0, 1: 820.6. Samples: 4242124. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:05,471][108279] Avg episode reward: [(0, '9.740'), (1, '9.960')] -[2023-09-25 21:02:10,470][108279] Fps is (10 sec: 6963.3, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 17006592. Throughput: 0: 820.6, 1: 821.1. Samples: 4247352. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:10,470][108279] Avg episode reward: [(0, '9.740'), (1, '9.960')] -[2023-09-25 21:02:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6621.9, 300 sec: 6539.7). Total num frames: 17035264. Throughput: 0: 821.6, 1: 821.0. Samples: 4257081. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:15,471][108279] Avg episode reward: [(0, '9.750'), (1, '9.960')] -[2023-09-25 21:02:15,476][109224] Updated weights for policy 1, policy_version 33280 (0.0017) -[2023-09-25 21:02:15,477][109225] Updated weights for policy 0, policy_version 33280 (0.0017) -[2023-09-25 21:02:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6690.2, 300 sec: 6553.6). Total num frames: 17072128. Throughput: 0: 820.6, 1: 820.5. Samples: 4266812. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:20,470][108279] Avg episode reward: [(0, '9.760'), (1, '9.960')] -[2023-09-25 21:02:25,470][108279] Fps is (10 sec: 6963.4, 60 sec: 6690.2, 300 sec: 6553.6). Total num frames: 17104896. Throughput: 0: 821.1, 1: 820.7. Samples: 4271976. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:25,470][108279] Avg episode reward: [(0, '9.770'), (1, '9.960')] -[2023-09-25 21:02:27,858][109225] Updated weights for policy 0, policy_version 33440 (0.0018) -[2023-09-25 21:02:27,858][109224] Updated weights for policy 1, policy_version 33440 (0.0017) -[2023-09-25 21:02:30,470][108279] Fps is (10 sec: 6553.3, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 17137664. Throughput: 0: 823.7, 1: 824.3. Samples: 4281851. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:30,471][108279] Avg episode reward: [(0, '9.780'), (1, '9.960')] -[2023-09-25 21:02:30,482][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000033472_8568832.pth... -[2023-09-25 21:02:30,482][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000033472_8568832.pth... -[2023-09-25 21:02:30,516][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000030400_7782400.pth -[2023-09-25 21:02:30,520][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000030400_7782400.pth -[2023-09-25 21:02:35,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 17170432. Throughput: 0: 823.5, 1: 823.2. Samples: 4291676. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:35,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.960')] -[2023-09-25 21:02:40,396][109224] Updated weights for policy 1, policy_version 33600 (0.0016) -[2023-09-25 21:02:40,396][109225] Updated weights for policy 0, policy_version 33600 (0.0017) -[2023-09-25 21:02:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6690.1, 300 sec: 6553.6). Total num frames: 17203200. Throughput: 0: 821.4, 1: 820.2. Samples: 4296669. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:40,471][108279] Avg episode reward: [(0, '9.820'), (1, '9.960')] -[2023-09-25 21:02:45,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17227776. Throughput: 0: 817.3, 1: 818.1. Samples: 4305954. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:45,473][108279] Avg episode reward: [(0, '9.820'), (1, '9.960')] -[2023-09-25 21:02:50,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 17260544. Throughput: 0: 815.0, 1: 815.5. Samples: 4315494. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:50,470][108279] Avg episode reward: [(0, '9.820'), (1, '9.960')] -[2023-09-25 21:02:53,143][109225] Updated weights for policy 0, policy_version 33760 (0.0017) -[2023-09-25 21:02:53,143][109224] Updated weights for policy 1, policy_version 33760 (0.0017) -[2023-09-25 21:02:55,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6485.4, 300 sec: 6525.8). Total num frames: 17293312. Throughput: 0: 814.6, 1: 814.3. Samples: 4320650. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:02:55,470][108279] Avg episode reward: [(0, '9.820'), (1, '9.960')] -[2023-09-25 21:03:00,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6485.3, 300 sec: 6525.8). Total num frames: 17326080. Throughput: 0: 815.0, 1: 815.1. Samples: 4330436. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:03:00,471][108279] Avg episode reward: [(0, '9.830'), (1, '9.960')] -[2023-09-25 21:03:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6485.3, 300 sec: 6525.8). Total num frames: 17358848. Throughput: 0: 814.4, 1: 814.9. Samples: 4340134. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:03:05,470][108279] Avg episode reward: [(0, '9.840'), (1, '9.970')] -[2023-09-25 21:03:05,662][109225] Updated weights for policy 0, policy_version 33920 (0.0017) -[2023-09-25 21:03:05,662][109224] Updated weights for policy 1, policy_version 33920 (0.0015) -[2023-09-25 21:03:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 17391616. Throughput: 0: 814.2, 1: 813.9. Samples: 4345240. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:03:10,471][108279] Avg episode reward: [(0, '9.840'), (1, '9.970')] -[2023-09-25 21:03:15,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6485.3, 300 sec: 6525.8). Total num frames: 17424384. Throughput: 0: 813.0, 1: 812.5. Samples: 4355001. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:03:15,471][108279] Avg episode reward: [(0, '9.850'), (1, '9.970')] -[2023-09-25 21:03:18,085][109224] Updated weights for policy 1, policy_version 34080 (0.0017) -[2023-09-25 21:03:18,086][109225] Updated weights for policy 0, policy_version 34080 (0.0016) -[2023-09-25 21:03:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 17457152. Throughput: 0: 811.6, 1: 812.6. Samples: 4364764. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:03:20,471][108279] Avg episode reward: [(0, '9.850'), (1, '9.970')] -[2023-09-25 21:03:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 17489920. Throughput: 0: 810.1, 1: 811.8. Samples: 4369655. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 21:03:25,471][108279] Avg episode reward: [(0, '9.850'), (1, '9.970')] -[2023-09-25 21:03:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 17522688. Throughput: 0: 815.6, 1: 814.7. Samples: 4379318. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 21:03:30,471][108279] Avg episode reward: [(0, '9.840'), (1, '9.970')] -[2023-09-25 21:03:30,756][109225] Updated weights for policy 0, policy_version 34240 (0.0017) -[2023-09-25 21:03:30,757][109224] Updated weights for policy 1, policy_version 34240 (0.0016) -[2023-09-25 21:03:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 17555456. Throughput: 0: 817.7, 1: 817.3. Samples: 4389070. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 21:03:35,471][108279] Avg episode reward: [(0, '9.840'), (1, '9.970')] -[2023-09-25 21:03:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 17588224. Throughput: 0: 816.8, 1: 816.8. Samples: 4394162. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 21:03:40,471][108279] Avg episode reward: [(0, '9.840'), (1, '9.970')] -[2023-09-25 21:03:43,276][109224] Updated weights for policy 1, policy_version 34400 (0.0017) -[2023-09-25 21:03:43,276][109225] Updated weights for policy 0, policy_version 34400 (0.0018) -[2023-09-25 21:03:45,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17620992. Throughput: 0: 816.0, 1: 816.1. Samples: 4403880. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) -[2023-09-25 21:03:45,470][108279] Avg episode reward: [(0, '9.840'), (1, '9.970')] -[2023-09-25 21:03:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17653760. Throughput: 0: 816.0, 1: 813.8. Samples: 4413476. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:03:50,471][108279] Avg episode reward: [(0, '9.830'), (1, '9.960')] -[2023-09-25 21:03:55,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17686528. Throughput: 0: 813.3, 1: 814.0. Samples: 4418469. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:03:55,471][108279] Avg episode reward: [(0, '9.830'), (1, '9.960')] -[2023-09-25 21:03:55,879][109225] Updated weights for policy 0, policy_version 34560 (0.0017) -[2023-09-25 21:03:55,879][109224] Updated weights for policy 1, policy_version 34560 (0.0017) -[2023-09-25 21:04:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17719296. Throughput: 0: 813.8, 1: 813.9. Samples: 4428251. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:04:00,470][108279] Avg episode reward: [(0, '9.840'), (1, '9.960')] -[2023-09-25 21:04:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17752064. Throughput: 0: 815.4, 1: 812.4. Samples: 4438016. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:04:05,471][108279] Avg episode reward: [(0, '9.830'), (1, '9.960')] -[2023-09-25 21:04:08,474][109225] Updated weights for policy 0, policy_version 34720 (0.0017) -[2023-09-25 21:04:08,474][109224] Updated weights for policy 1, policy_version 34720 (0.0017) -[2023-09-25 21:04:10,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17784832. Throughput: 0: 813.1, 1: 813.2. Samples: 4442840. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:04:10,471][108279] Avg episode reward: [(0, '9.820'), (1, '9.960')] -[2023-09-25 21:04:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17817600. Throughput: 0: 814.9, 1: 815.5. Samples: 4452687. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:04:15,471][108279] Avg episode reward: [(0, '9.820'), (1, '9.960')] -[2023-09-25 21:04:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17850368. Throughput: 0: 818.0, 1: 815.8. Samples: 4462592. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:04:20,471][108279] Avg episode reward: [(0, '9.800'), (1, '9.960')] -[2023-09-25 21:04:20,999][109225] Updated weights for policy 0, policy_version 34880 (0.0014) -[2023-09-25 21:04:20,999][109224] Updated weights for policy 1, policy_version 34880 (0.0017) -[2023-09-25 21:04:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17883136. Throughput: 0: 814.6, 1: 815.0. Samples: 4467492. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:04:25,471][108279] Avg episode reward: [(0, '9.800'), (1, '9.960')] -[2023-09-25 21:04:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17915904. Throughput: 0: 816.4, 1: 816.5. Samples: 4477360. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) -[2023-09-25 21:04:30,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.960')] -[2023-09-25 21:04:30,481][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000034992_8957952.pth... -[2023-09-25 21:04:30,481][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000034992_8957952.pth... -[2023-09-25 21:04:30,516][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000031936_8175616.pth -[2023-09-25 21:04:30,519][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000031936_8175616.pth -[2023-09-25 21:04:33,434][109224] Updated weights for policy 1, policy_version 35040 (0.0017) -[2023-09-25 21:04:33,434][109225] Updated weights for policy 0, policy_version 35040 (0.0018) -[2023-09-25 21:04:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17948672. Throughput: 0: 819.2, 1: 818.4. Samples: 4487168. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 21:04:35,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.960')] -[2023-09-25 21:04:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 17981440. Throughput: 0: 815.8, 1: 814.6. Samples: 4491839. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 21:04:40,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.960')] -[2023-09-25 21:04:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18014208. Throughput: 0: 815.2, 1: 812.8. Samples: 4501513. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 21:04:45,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.960')] -[2023-09-25 21:04:46,220][109225] Updated weights for policy 0, policy_version 35200 (0.0017) -[2023-09-25 21:04:46,220][109224] Updated weights for policy 1, policy_version 35200 (0.0017) -[2023-09-25 21:04:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18046976. Throughput: 0: 813.1, 1: 816.2. Samples: 4511335. Policy #0 lag: (min: 9.0, avg: 9.0, max: 9.0) -[2023-09-25 21:04:50,471][108279] Avg episode reward: [(0, '9.800'), (1, '9.960')] -[2023-09-25 21:04:55,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18079744. Throughput: 0: 813.2, 1: 813.2. Samples: 4516030. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:04:55,471][108279] Avg episode reward: [(0, '9.800'), (1, '9.960')] -[2023-09-25 21:04:58,782][109224] Updated weights for policy 1, policy_version 35360 (0.0014) -[2023-09-25 21:04:58,783][109225] Updated weights for policy 0, policy_version 35360 (0.0017) -[2023-09-25 21:05:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18112512. Throughput: 0: 816.8, 1: 814.1. Samples: 4526080. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:05:00,471][108279] Avg episode reward: [(0, '9.810'), (1, '9.960')] -[2023-09-25 21:05:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18145280. Throughput: 0: 809.3, 1: 811.9. Samples: 4535544. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:05:05,471][108279] Avg episode reward: [(0, '9.810'), (1, '9.960')] -[2023-09-25 21:05:10,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18178048. Throughput: 0: 811.6, 1: 809.1. Samples: 4540421. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:05:10,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.960')] -[2023-09-25 21:05:11,334][109224] Updated weights for policy 1, policy_version 35520 (0.0017) -[2023-09-25 21:05:11,334][109225] Updated weights for policy 0, policy_version 35520 (0.0017) -[2023-09-25 21:05:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18210816. Throughput: 0: 815.7, 1: 813.1. Samples: 4550653. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:05:15,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.960')] -[2023-09-25 21:05:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18243584. Throughput: 0: 813.4, 1: 815.3. Samples: 4560457. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:20,471][108279] Avg episode reward: [(0, '9.770'), (1, '9.960')] -[2023-09-25 21:05:23,807][109224] Updated weights for policy 1, policy_version 35680 (0.0018) -[2023-09-25 21:05:23,807][109225] Updated weights for policy 0, policy_version 35680 (0.0013) -[2023-09-25 21:05:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18276352. Throughput: 0: 813.9, 1: 814.4. Samples: 4565113. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:25,470][108279] Avg episode reward: [(0, '9.770'), (1, '9.960')] -[2023-09-25 21:05:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18309120. Throughput: 0: 817.7, 1: 819.0. Samples: 4575165. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:30,471][108279] Avg episode reward: [(0, '9.760'), (1, '9.960')] -[2023-09-25 21:05:35,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 18341888. Throughput: 0: 818.5, 1: 817.8. Samples: 4584967. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:35,471][108279] Avg episode reward: [(0, '9.770'), (1, '9.990')] -[2023-09-25 21:05:35,472][109025] Saving new best policy, reward=9.990! -[2023-09-25 21:05:36,357][109224] Updated weights for policy 1, policy_version 35840 (0.0015) -[2023-09-25 21:05:36,357][109225] Updated weights for policy 0, policy_version 35840 (0.0018) -[2023-09-25 21:05:40,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18374656. Throughput: 0: 818.5, 1: 817.4. Samples: 4589643. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:40,471][108279] Avg episode reward: [(0, '9.770'), (1, '9.990')] -[2023-09-25 21:05:45,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18407424. Throughput: 0: 817.4, 1: 818.7. Samples: 4599703. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:45,471][108279] Avg episode reward: [(0, '9.770'), (1, '9.990')] -[2023-09-25 21:05:48,933][109224] Updated weights for policy 1, policy_version 36000 (0.0017) -[2023-09-25 21:05:48,933][109225] Updated weights for policy 0, policy_version 36000 (0.0016) -[2023-09-25 21:05:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18440192. Throughput: 0: 820.1, 1: 820.5. Samples: 4609373. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:50,471][108279] Avg episode reward: [(0, '9.780'), (1, '9.990')] -[2023-09-25 21:05:55,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18472960. Throughput: 0: 819.2, 1: 820.6. Samples: 4614211. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:05:55,470][108279] Avg episode reward: [(0, '9.780'), (1, '9.990')] -[2023-09-25 21:06:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18505728. Throughput: 0: 817.8, 1: 819.2. Samples: 4624320. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:00,471][108279] Avg episode reward: [(0, '9.770'), (1, '9.980')] -[2023-09-25 21:06:01,317][109225] Updated weights for policy 0, policy_version 36160 (0.0016) -[2023-09-25 21:06:01,317][109224] Updated weights for policy 1, policy_version 36160 (0.0018) -[2023-09-25 21:06:05,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18538496. Throughput: 0: 818.3, 1: 819.1. Samples: 4634137. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:05,471][108279] Avg episode reward: [(0, '9.760'), (1, '9.980')] -[2023-09-25 21:06:10,470][108279] Fps is (10 sec: 6553.8, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18571264. Throughput: 0: 819.2, 1: 819.3. Samples: 4638846. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:10,470][108279] Avg episode reward: [(0, '9.770'), (1, '9.980')] -[2023-09-25 21:06:13,776][109224] Updated weights for policy 1, policy_version 36320 (0.0015) -[2023-09-25 21:06:13,776][109225] Updated weights for policy 0, policy_version 36320 (0.0017) -[2023-09-25 21:06:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18604032. Throughput: 0: 820.6, 1: 819.2. Samples: 4648958. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:15,471][108279] Avg episode reward: [(0, '9.770'), (1, '9.980')] -[2023-09-25 21:06:20,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18636800. Throughput: 0: 818.6, 1: 818.5. Samples: 4658635. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:20,471][108279] Avg episode reward: [(0, '9.740'), (1, '9.980')] -[2023-09-25 21:06:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18669568. Throughput: 0: 819.1, 1: 818.9. Samples: 4663356. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:25,471][108279] Avg episode reward: [(0, '9.740'), (1, '9.980')] -[2023-09-25 21:06:26,465][109224] Updated weights for policy 1, policy_version 36480 (0.0017) -[2023-09-25 21:06:26,465][109225] Updated weights for policy 0, policy_version 36480 (0.0016) -[2023-09-25 21:06:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18702336. Throughput: 0: 815.0, 1: 816.5. Samples: 4673122. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:30,471][108279] Avg episode reward: [(0, '9.730'), (1, '9.980')] -[2023-09-25 21:06:30,479][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000036528_9351168.pth... -[2023-09-25 21:06:30,479][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000036528_9351168.pth... -[2023-09-25 21:06:30,508][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000033472_8568832.pth -[2023-09-25 21:06:30,514][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000033472_8568832.pth -[2023-09-25 21:06:35,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18735104. Throughput: 0: 816.6, 1: 816.3. Samples: 4682857. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:35,471][108279] Avg episode reward: [(0, '9.730'), (1, '9.980')] -[2023-09-25 21:06:39,008][109224] Updated weights for policy 1, policy_version 36640 (0.0017) -[2023-09-25 21:06:39,008][109225] Updated weights for policy 0, policy_version 36640 (0.0017) -[2023-09-25 21:06:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 18767872. Throughput: 0: 819.1, 1: 817.7. Samples: 4687867. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:40,470][108279] Avg episode reward: [(0, '9.730'), (1, '9.980')] -[2023-09-25 21:06:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18800640. Throughput: 0: 816.6, 1: 817.1. Samples: 4697837. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:45,470][108279] Avg episode reward: [(0, '9.730'), (1, '9.980')] -[2023-09-25 21:06:50,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 18833408. Throughput: 0: 814.8, 1: 814.5. Samples: 4707452. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:06:50,470][108279] Avg episode reward: [(0, '9.730'), (1, '9.980')] -[2023-09-25 21:06:51,549][109224] Updated weights for policy 1, policy_version 36800 (0.0017) -[2023-09-25 21:06:51,550][109225] Updated weights for policy 0, policy_version 36800 (0.0014) -[2023-09-25 21:06:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 18866176. Throughput: 0: 818.7, 1: 816.6. Samples: 4712435. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 21:06:55,471][108279] Avg episode reward: [(0, '9.730'), (1, '9.980')] -[2023-09-25 21:07:00,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 18898944. Throughput: 0: 813.0, 1: 815.5. Samples: 4722238. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 21:07:00,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.980')] -[2023-09-25 21:07:04,024][109225] Updated weights for policy 0, policy_version 36960 (0.0014) -[2023-09-25 21:07:04,024][109224] Updated weights for policy 1, policy_version 36960 (0.0015) -[2023-09-25 21:07:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18931712. Throughput: 0: 815.5, 1: 816.2. Samples: 4732060. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 21:07:05,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.980')] -[2023-09-25 21:07:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6539.7). Total num frames: 18964480. Throughput: 0: 819.2, 1: 817.9. Samples: 4737024. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 21:07:10,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.980')] -[2023-09-25 21:07:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 18997248. Throughput: 0: 820.7, 1: 820.5. Samples: 4746974. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) -[2023-09-25 21:07:15,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.980')] -[2023-09-25 21:07:16,421][109225] Updated weights for policy 0, policy_version 37120 (0.0016) -[2023-09-25 21:07:16,421][109224] Updated weights for policy 1, policy_version 37120 (0.0015) -[2023-09-25 21:07:20,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 19030016. Throughput: 0: 822.1, 1: 820.9. Samples: 4756792. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:20,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.970')] -[2023-09-25 21:07:25,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 19062784. Throughput: 0: 819.3, 1: 819.2. Samples: 4761600. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:25,470][108279] Avg episode reward: [(0, '9.720'), (1, '9.970')] -[2023-09-25 21:07:29,116][109224] Updated weights for policy 1, policy_version 37280 (0.0017) -[2023-09-25 21:07:29,116][109225] Updated weights for policy 0, policy_version 37280 (0.0019) -[2023-09-25 21:07:30,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 19095552. Throughput: 0: 816.4, 1: 817.1. Samples: 4771342. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:30,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.970')] -[2023-09-25 21:07:35,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6525.8). Total num frames: 19128320. Throughput: 0: 814.6, 1: 815.3. Samples: 4780797. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:35,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.970')] -[2023-09-25 21:07:40,470][108279] Fps is (10 sec: 6144.2, 60 sec: 6485.3, 300 sec: 6539.7). Total num frames: 19156992. Throughput: 0: 814.7, 1: 817.2. Samples: 4785872. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:40,470][108279] Avg episode reward: [(0, '9.710'), (1, '9.970')] -[2023-09-25 21:07:41,731][109225] Updated weights for policy 0, policy_version 37440 (0.0017) -[2023-09-25 21:07:41,731][109224] Updated weights for policy 1, policy_version 37440 (0.0017) -[2023-09-25 21:07:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19193856. Throughput: 0: 815.3, 1: 815.7. Samples: 4795632. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:45,470][108279] Avg episode reward: [(0, '9.720'), (1, '9.970')] -[2023-09-25 21:07:50,470][108279] Fps is (10 sec: 6963.1, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19226624. Throughput: 0: 816.7, 1: 815.6. Samples: 4805514. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:50,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.970')] -[2023-09-25 21:07:54,147][109224] Updated weights for policy 1, policy_version 37600 (0.0019) -[2023-09-25 21:07:54,148][109225] Updated weights for policy 0, policy_version 37600 (0.0019) -[2023-09-25 21:07:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19259392. Throughput: 0: 816.0, 1: 819.0. Samples: 4810598. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:07:55,471][108279] Avg episode reward: [(0, '9.720'), (1, '9.960')] -[2023-09-25 21:08:00,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19292160. Throughput: 0: 816.8, 1: 816.6. Samples: 4820473. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:08:00,470][108279] Avg episode reward: [(0, '9.720'), (1, '9.960')] -[2023-09-25 21:08:05,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19324928. Throughput: 0: 813.6, 1: 815.1. Samples: 4830086. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 21:08:05,471][108279] Avg episode reward: [(0, '9.730'), (1, '9.970')] -[2023-09-25 21:08:06,684][109225] Updated weights for policy 0, policy_version 37760 (0.0018) -[2023-09-25 21:08:06,684][109224] Updated weights for policy 1, policy_version 37760 (0.0017) -[2023-09-25 21:08:10,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19357696. Throughput: 0: 816.0, 1: 818.7. Samples: 4835164. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 21:08:10,471][108279] Avg episode reward: [(0, '9.730'), (1, '9.970')] -[2023-09-25 21:08:15,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19390464. Throughput: 0: 817.6, 1: 818.5. Samples: 4844963. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 21:08:15,471][108279] Avg episode reward: [(0, '9.730'), (1, '9.970')] -[2023-09-25 21:08:19,117][109224] Updated weights for policy 1, policy_version 37920 (0.0015) -[2023-09-25 21:08:19,117][109225] Updated weights for policy 0, policy_version 37920 (0.0017) -[2023-09-25 21:08:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19423232. Throughput: 0: 821.9, 1: 821.3. Samples: 4854740. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 21:08:20,470][108279] Avg episode reward: [(0, '9.730'), (1, '9.970')] -[2023-09-25 21:08:25,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19456000. Throughput: 0: 821.6, 1: 821.2. Samples: 4859798. Policy #0 lag: (min: 1.0, avg: 1.0, max: 1.0) -[2023-09-25 21:08:25,471][108279] Avg episode reward: [(0, '9.740'), (1, '9.970')] -[2023-09-25 21:08:30,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19488768. Throughput: 0: 824.6, 1: 823.7. Samples: 4869803. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:08:30,471][108279] Avg episode reward: [(0, '9.770'), (1, '9.970')] -[2023-09-25 21:08:30,484][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000038064_9744384.pth... -[2023-09-25 21:08:30,485][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000038064_9744384.pth... -[2023-09-25 21:08:30,518][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000034992_8957952.pth -[2023-09-25 21:08:30,522][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000034992_8957952.pth -[2023-09-25 21:08:31,498][109224] Updated weights for policy 1, policy_version 38080 (0.0017) -[2023-09-25 21:08:31,499][109225] Updated weights for policy 0, policy_version 38080 (0.0019) -[2023-09-25 21:08:35,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19521536. Throughput: 0: 821.1, 1: 821.6. Samples: 4879434. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:08:35,470][108279] Avg episode reward: [(0, '9.770'), (1, '9.970')] -[2023-09-25 21:08:40,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6621.8, 300 sec: 6553.6). Total num frames: 19554304. Throughput: 0: 821.9, 1: 819.4. Samples: 4884457. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:08:40,471][108279] Avg episode reward: [(0, '9.780'), (1, '9.970')] -[2023-09-25 21:08:43,976][109225] Updated weights for policy 0, policy_version 38240 (0.0016) -[2023-09-25 21:08:43,976][109224] Updated weights for policy 1, policy_version 38240 (0.0016) -[2023-09-25 21:08:45,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19587072. Throughput: 0: 821.4, 1: 821.9. Samples: 4894419. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:08:45,471][108279] Avg episode reward: [(0, '9.780'), (1, '9.970')] -[2023-09-25 21:08:50,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19619840. Throughput: 0: 823.3, 1: 823.8. Samples: 4904203. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:08:50,470][108279] Avg episode reward: [(0, '9.790'), (1, '9.970')] -[2023-09-25 21:08:55,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19652608. Throughput: 0: 822.4, 1: 819.8. Samples: 4909060. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:08:55,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.970')] -[2023-09-25 21:08:56,459][109224] Updated weights for policy 1, policy_version 38400 (0.0016) -[2023-09-25 21:08:56,459][109225] Updated weights for policy 0, policy_version 38400 (0.0017) -[2023-09-25 21:09:00,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19685376. Throughput: 0: 824.3, 1: 823.0. Samples: 4919090. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:09:00,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.970')] -[2023-09-25 21:09:05,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19718144. Throughput: 0: 823.7, 1: 823.2. Samples: 4928850. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:09:05,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.970')] -[2023-09-25 21:09:08,915][109224] Updated weights for policy 1, policy_version 38560 (0.0016) -[2023-09-25 21:09:08,916][109225] Updated weights for policy 0, policy_version 38560 (0.0018) -[2023-09-25 21:09:10,470][108279] Fps is (10 sec: 6553.6, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19750912. Throughput: 0: 821.5, 1: 819.6. Samples: 4933649. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:09:10,471][108279] Avg episode reward: [(0, '9.790'), (1, '9.970')] -[2023-09-25 21:09:15,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19783680. Throughput: 0: 817.8, 1: 818.4. Samples: 4943430. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) -[2023-09-25 21:09:15,470][108279] Avg episode reward: [(0, '9.760'), (1, '9.970')] -[2023-09-25 21:09:20,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19816448. Throughput: 0: 819.8, 1: 820.4. Samples: 4953245. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 21:09:20,471][108279] Avg episode reward: [(0, '9.760'), (1, '9.970')] -[2023-09-25 21:09:21,484][109224] Updated weights for policy 1, policy_version 38720 (0.0013) -[2023-09-25 21:09:21,485][109225] Updated weights for policy 0, policy_version 38720 (0.0018) -[2023-09-25 21:09:25,470][108279] Fps is (10 sec: 6553.4, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19849216. Throughput: 0: 819.7, 1: 819.2. Samples: 4958208. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 21:09:25,471][108279] Avg episode reward: [(0, '9.750'), (1, '9.970')] -[2023-09-25 21:09:30,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 19881984. Throughput: 0: 819.2, 1: 818.6. Samples: 4968118. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 21:09:30,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.970')] -[2023-09-25 21:09:34,236][109224] Updated weights for policy 1, policy_version 38880 (0.0017) -[2023-09-25 21:09:34,236][109225] Updated weights for policy 0, policy_version 38880 (0.0015) -[2023-09-25 21:09:35,470][108279] Fps is (10 sec: 5734.5, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 19906560. Throughput: 0: 812.9, 1: 812.3. Samples: 4977335. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 21:09:35,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.970')] -[2023-09-25 21:09:40,470][108279] Fps is (10 sec: 5734.4, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 19939328. Throughput: 0: 810.6, 1: 813.0. Samples: 4982120. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 21:09:40,471][108279] Avg episode reward: [(0, '9.710'), (1, '9.970')] -[2023-09-25 21:09:45,470][108279] Fps is (10 sec: 6553.7, 60 sec: 6417.1, 300 sec: 6525.8). Total num frames: 19972096. Throughput: 0: 808.3, 1: 807.7. Samples: 4991813. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) -[2023-09-25 21:09:45,470][108279] Avg episode reward: [(0, '9.730'), (1, '9.960')] -[2023-09-25 21:09:47,148][109224] Updated weights for policy 1, policy_version 39040 (0.0016) -[2023-09-25 21:09:47,148][109225] Updated weights for policy 0, policy_version 39040 (0.0017) -[2023-09-25 21:09:50,470][108279] Fps is (10 sec: 6553.5, 60 sec: 6417.0, 300 sec: 6525.8). Total num frames: 20004864. Throughput: 0: 805.1, 1: 803.6. Samples: 5001241. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) -[2023-09-25 21:09:50,471][108279] Avg episode reward: [(0, '9.730'), (1, '9.960')] -[2023-09-25 21:09:50,856][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000039088_10006528.pth... -[2023-09-25 21:09:50,856][109227] Stopping RolloutWorker_w0... -[2023-09-25 21:09:50,856][109259] Stopping RolloutWorker_w1... -[2023-09-25 21:09:50,856][109262] Stopping RolloutWorker_w4... -[2023-09-25 21:09:50,857][109227] Loop rollout_proc0_evt_loop terminating... -[2023-09-25 21:09:50,856][109266] Stopping RolloutWorker_w7... -[2023-09-25 21:09:50,856][109265] Stopping RolloutWorker_w6... -[2023-09-25 21:09:50,857][109262] Loop rollout_proc4_evt_loop terminating... -[2023-09-25 21:09:50,857][109261] Stopping RolloutWorker_w3... -[2023-09-25 21:09:50,856][109264] Stopping RolloutWorker_w5... -[2023-09-25 21:09:50,857][109263] Stopping RolloutWorker_w2... -[2023-09-25 21:09:50,856][108279] Component RolloutWorker_w1 stopped! -[2023-09-25 21:09:50,857][109259] Loop rollout_proc1_evt_loop terminating... -[2023-09-25 21:09:50,857][109266] Loop rollout_proc7_evt_loop terminating... -[2023-09-25 21:09:50,857][108926] Stopping Batcher_0... -[2023-09-25 21:09:50,857][109264] Loop rollout_proc5_evt_loop terminating... -[2023-09-25 21:09:50,857][109261] Loop rollout_proc3_evt_loop terminating... -[2023-09-25 21:09:50,857][109265] Loop rollout_proc6_evt_loop terminating... -[2023-09-25 21:09:50,857][109263] Loop rollout_proc2_evt_loop terminating... -[2023-09-25 21:09:50,858][108279] Component RolloutWorker_w0 stopped! -[2023-09-25 21:09:50,858][108926] Loop batcher_evt_loop terminating... -[2023-09-25 21:09:50,858][108279] Component RolloutWorker_w4 stopped! -[2023-09-25 21:09:50,859][108279] Component RolloutWorker_w6 stopped! -[2023-09-25 21:09:50,859][108279] Component Batcher_1 stopped! -[2023-09-25 21:09:50,860][108279] Component RolloutWorker_w7 stopped! -[2023-09-25 21:09:50,860][108279] Component RolloutWorker_w5 stopped! -[2023-09-25 21:09:50,861][108279] Component RolloutWorker_w3 stopped! -[2023-09-25 21:09:50,861][108279] Component RolloutWorker_w2 stopped! -[2023-09-25 21:09:50,861][108279] Component Batcher_0 stopped! -[2023-09-25 21:09:50,856][109025] Stopping Batcher_1... -[2023-09-25 21:09:50,875][109025] Loop batcher_evt_loop terminating... -[2023-09-25 21:09:50,893][109025] Removing ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000036528_9351168.pth -[2023-09-25 21:09:50,897][109025] Saving ./train_atari/atari_bowling/checkpoint_p1/checkpoint_000039088_10006528.pth... -[2023-09-25 21:09:50,913][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000039088_10006528.pth... -[2023-09-25 21:09:50,919][109225] Weights refcount: 2 0 -[2023-09-25 21:09:50,921][109225] Stopping InferenceWorker_p0-w0... -[2023-09-25 21:09:50,921][109225] Loop inference_proc0-0_evt_loop terminating... -[2023-09-25 21:09:50,921][108279] Component InferenceWorker_p0-w0 stopped! -[2023-09-25 21:09:50,925][109224] Weights refcount: 2 0 -[2023-09-25 21:09:50,926][109224] Stopping InferenceWorker_p1-w0... -[2023-09-25 21:09:50,926][109224] Loop inference_proc1-0_evt_loop terminating... -[2023-09-25 21:09:50,926][108279] Component InferenceWorker_p1-w0 stopped! -[2023-09-25 21:09:50,936][109025] Stopping LearnerWorker_p1... -[2023-09-25 21:09:50,936][109025] Loop learner_proc1_evt_loop terminating... -[2023-09-25 21:09:50,938][108279] Component LearnerWorker_p1 stopped! -[2023-09-25 21:09:50,942][108926] Removing ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000036528_9351168.pth -[2023-09-25 21:09:50,946][108926] Saving ./train_atari/atari_bowling/checkpoint_p0/checkpoint_000039088_10006528.pth... -[2023-09-25 21:09:50,982][108926] Stopping LearnerWorker_p0... -[2023-09-25 21:09:50,982][108926] Loop learner_proc0_evt_loop terminating... -[2023-09-25 21:09:50,982][108279] Component LearnerWorker_p0 stopped! -[2023-09-25 21:09:50,983][108279] Waiting for process learner_proc0 to stop... -[2023-09-25 21:09:51,643][108279] Waiting for process learner_proc1 to stop... -[2023-09-25 21:09:51,644][108279] Waiting for process inference_proc0-0 to join... -[2023-09-25 21:09:51,673][108279] Waiting for process inference_proc1-0 to join... -[2023-09-25 21:09:51,674][108279] Waiting for process rollout_proc0 to join... -[2023-09-25 21:09:51,675][108279] Waiting for process rollout_proc1 to join... -[2023-09-25 21:09:51,676][108279] Waiting for process rollout_proc2 to join... -[2023-09-25 21:09:51,676][108279] Waiting for process rollout_proc3 to join... -[2023-09-25 21:09:51,677][108279] Waiting for process rollout_proc4 to join... -[2023-09-25 21:09:51,678][108279] Waiting for process rollout_proc5 to join... -[2023-09-25 21:09:51,679][108279] Waiting for process rollout_proc6 to join... -[2023-09-25 21:09:51,680][108279] Waiting for process rollout_proc7 to join... -[2023-09-25 21:09:51,680][108279] Batcher 0 profile tree view: -batching: 20.6225, releasing_batches: 1.7891 -[2023-09-25 21:09:51,681][108279] Batcher 1 profile tree view: -batching: 20.8886, releasing_batches: 1.8818 -[2023-09-25 21:09:51,681][108279] InferenceWorker_p0-w0 profile tree view: -wait_policy: 0.0051 - wait_policy_total: 616.9296 -update_model: 36.3037 - weight_update: 0.0017 -one_step: 0.0011 - handle_policy_step: 2218.6380 - deserialize: 66.2649, stack: 15.9493, obs_to_device_normalize: 540.3936, forward: 1066.9566, send_messages: 92.6701 - prepare_outputs: 294.9488 - to_cpu: 147.0230 -[2023-09-25 21:09:51,681][108279] InferenceWorker_p1-w0 profile tree view: -wait_policy: 0.0052 - wait_policy_total: 619.4736 -update_model: 36.8075 - weight_update: 0.0015 -one_step: 0.0012 - handle_policy_step: 2212.1271 - deserialize: 67.7604, stack: 16.3694, obs_to_device_normalize: 537.5653, forward: 1059.3219, send_messages: 93.4417 - prepare_outputs: 292.5068 - to_cpu: 146.2553 -[2023-09-25 21:09:51,682][108279] Learner 0 profile tree view: -misc: 0.0152, prepare_batch: 31.9294 -train: 457.3288 - epoch_init: 0.1032, minibatch_init: 3.0735, losses_postprocess: 62.6364, kl_divergence: 5.3809, after_optimizer: 23.2850 - calculate_losses: 44.6980 - losses_init: 0.0995, forward_head: 14.1978, bptt_initial: 0.4313, bptt: 0.4746, tail: 10.3070, advantages_returns: 3.0465, losses: 12.6316 - update: 314.1444 - clip: 162.4632 -[2023-09-25 21:09:51,682][108279] Learner 1 profile tree view: -misc: 0.0152, prepare_batch: 32.1193 -train: 456.3154 - epoch_init: 0.1005, minibatch_init: 3.2161, losses_postprocess: 61.2132, kl_divergence: 5.5294, after_optimizer: 23.1887 - calculate_losses: 44.6887 - losses_init: 0.1012, forward_head: 13.5405, bptt_initial: 0.4526, bptt: 0.4558, tail: 10.5134, advantages_returns: 3.1060, losses: 12.8752 - update: 314.2029 - clip: 161.0696 -[2023-09-25 21:09:51,683][108279] RolloutWorker_w0 profile tree view: -wait_for_trajectories: 0.3973, enqueue_policy_requests: 43.3810, env_step: 969.9135, overhead: 29.1769, complete_rollouts: 1.0872 -save_policy_outputs: 54.0324 - split_output_tensors: 18.8523 -[2023-09-25 21:09:51,683][108279] RolloutWorker_w7 profile tree view: -wait_for_trajectories: 0.3966, enqueue_policy_requests: 42.2082, env_step: 1022.0259, overhead: 29.4626, complete_rollouts: 1.0551 -save_policy_outputs: 52.9515 - split_output_tensors: 18.1506 -[2023-09-25 21:09:51,684][108279] Loop Runner_EvtLoop terminating... -[2023-09-25 21:09:51,684][108279] Runner profile tree view: -main_loop: 3077.0532 -[2023-09-25 21:09:51,685][108279] Collected {0: 10006528, 1: 10006528}, FPS: 6504.0 +[2023-10-09 12:13:32,721][85963] Using optimizer +[2023-10-09 12:13:32,722][85963] No checkpoints found +[2023-10-09 12:13:32,722][85963] Did not load from checkpoint, starting from scratch! +[2023-10-09 12:13:32,722][85963] Initialized policy 1 weights for model version 0 +[2023-10-09 12:13:32,724][85963] LearnerWorker_p1 finished initialization! +[2023-10-09 12:13:32,724][85963] Using GPUs [0] for process 1 (actually maps to GPUs [1]) +[2023-10-09 12:13:33,610][85186] Starting process rollout_proc14 +[2023-10-09 12:13:33,620][86159] Worker 4 uses CPU cores [8, 9] +[2023-10-09 12:13:33,661][85186] Starting process rollout_proc15 +[2023-10-09 12:13:33,668][86167] Worker 8 uses CPU cores [16, 17] +[2023-10-09 12:13:33,684][86161] Worker 7 uses CPU cores [14, 15] +[2023-10-09 12:13:33,698][86155] Worker 0 uses CPU cores [0, 1] +[2023-10-09 12:13:33,752][86158] Worker 3 uses CPU cores [6, 7] +[2023-10-09 12:13:33,823][86154] Worker 1 uses CPU cores [2, 3] +[2023-10-09 12:13:33,900][86163] Worker 9 uses CPU cores [18, 19] +[2023-10-09 12:13:33,906][86166] Worker 12 uses CPU cores [24, 25] +[2023-10-09 12:13:33,907][86165] Worker 10 uses CPU cores [20, 21] +[2023-10-09 12:13:33,967][86122] Using GPUs [1] for process 1 (actually maps to GPUs [1]) +[2023-10-09 12:13:33,967][86122] Set environment var CUDA_VISIBLE_DEVICES to '1' (GPU indices [1]) for inference process 1 +[2023-10-09 12:13:33,971][86162] Worker 5 uses CPU cores [10, 11] +[2023-10-09 12:13:33,986][86122] Num visible devices: 1 +[2023-10-09 12:13:34,030][86168] Worker 13 uses CPU cores [26, 27] +[2023-10-09 12:13:34,093][86121] Using GPUs [0] for process 0 (actually maps to GPUs [0]) +[2023-10-09 12:13:34,094][86121] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 +[2023-10-09 12:13:34,112][86121] Num visible devices: 1 +[2023-10-09 12:13:34,168][86164] Worker 11 uses CPU cores [22, 23] +[2023-10-09 12:13:34,251][86160] Worker 6 uses CPU cores [12, 13] +[2023-10-09 12:13:34,306][86157] Worker 2 uses CPU cores [4, 5] +[2023-10-09 12:13:34,623][86122] RunningMeanStd input shape: (4, 84, 84) +[2023-10-09 12:13:34,624][86122] RunningMeanStd input shape: (1,) +[2023-10-09 12:13:34,636][86122] ConvEncoder: input_channels=4 +[2023-10-09 12:13:34,721][86121] RunningMeanStd input shape: (4, 84, 84) +[2023-10-09 12:13:34,722][86121] RunningMeanStd input shape: (1,) +[2023-10-09 12:13:34,733][86121] ConvEncoder: input_channels=4 +[2023-10-09 12:13:34,742][86122] Conv encoder output size: 512 +[2023-10-09 12:13:34,835][86121] Conv encoder output size: 512 +[2023-10-09 12:13:35,547][86745] Worker 15 uses CPU cores [30, 31] +[2023-10-09 12:13:35,655][85186] Inference worker 1-0 is ready! +[2023-10-09 12:13:35,655][85186] Inference worker 0-0 is ready! +[2023-10-09 12:13:35,656][85186] All inference workers are ready! Signal rollout workers to start! +[2023-10-09 12:13:35,657][86168] EnvRunner 13-0 uses policy 1 +[2023-10-09 12:13:35,657][86161] EnvRunner 7-0 uses policy 1 +[2023-10-09 12:13:35,657][86166] EnvRunner 12-0 uses policy 0 +[2023-10-09 12:13:35,657][86159] EnvRunner 4-0 uses policy 0 +[2023-10-09 12:13:35,657][85186] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan, 1: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-10-09 12:13:35,657][86160] EnvRunner 6-0 uses policy 0 +[2023-10-09 12:13:35,657][86164] EnvRunner 11-0 uses policy 1 +[2023-10-09 12:13:35,657][86158] EnvRunner 3-0 uses policy 1 +[2023-10-09 12:13:35,657][86162] EnvRunner 5-0 uses policy 1 +[2023-10-09 12:13:35,657][86157] EnvRunner 2-0 uses policy 0 +[2023-10-09 12:13:35,657][86165] EnvRunner 10-0 uses policy 0 +[2023-10-09 12:13:35,657][86154] EnvRunner 1-0 uses policy 1 +[2023-10-09 12:13:35,657][86163] EnvRunner 9-0 uses policy 1 +[2023-10-09 12:13:35,657][86155] EnvRunner 0-0 uses policy 0 +[2023-10-09 12:13:35,657][86167] EnvRunner 8-0 uses policy 0 +[2023-10-09 12:13:35,658][86713] Worker 14 uses CPU cores [28, 29] +[2023-10-09 12:13:35,756][86745] EnvRunner 15-0 uses policy 1 +[2023-10-09 12:13:35,777][86713] EnvRunner 14-0 uses policy 0 +[2023-10-09 12:13:37,866][85186] Heartbeat connected on Batcher_0 +[2023-10-09 12:13:37,869][85186] Heartbeat connected on LearnerWorker_p0 +[2023-10-09 12:13:37,872][85186] Heartbeat connected on Batcher_1 +[2023-10-09 12:13:37,875][85186] Heartbeat connected on LearnerWorker_p1 +[2023-10-09 12:13:37,882][85186] Heartbeat connected on InferenceWorker_p0-w0 +[2023-10-09 12:13:37,888][85186] Heartbeat connected on InferenceWorker_p1-w0 +[2023-10-09 12:13:37,890][85186] Heartbeat connected on RolloutWorker_w0 +[2023-10-09 12:13:37,891][85186] Heartbeat connected on RolloutWorker_w2 +[2023-10-09 12:13:37,893][85186] Heartbeat connected on RolloutWorker_w1 +[2023-10-09 12:13:37,897][85186] Heartbeat connected on RolloutWorker_w3 +[2023-10-09 12:13:37,898][85186] Heartbeat connected on RolloutWorker_w4 +[2023-10-09 12:13:37,901][85186] Heartbeat connected on RolloutWorker_w5 +[2023-10-09 12:13:37,906][85186] Heartbeat connected on RolloutWorker_w7 +[2023-10-09 12:13:37,906][85186] Heartbeat connected on RolloutWorker_w6 +[2023-10-09 12:13:37,909][85186] Heartbeat connected on RolloutWorker_w8 +[2023-10-09 12:13:37,912][85186] Heartbeat connected on RolloutWorker_w9 +[2023-10-09 12:13:37,916][85186] Heartbeat connected on RolloutWorker_w11 +[2023-10-09 12:13:37,918][85186] Heartbeat connected on RolloutWorker_w10 +[2023-10-09 12:13:37,920][85186] Heartbeat connected on RolloutWorker_w12 +[2023-10-09 12:13:37,923][85186] Heartbeat connected on RolloutWorker_w13 +[2023-10-09 12:13:37,925][85186] Heartbeat connected on RolloutWorker_w14 +[2023-10-09 12:13:37,933][85186] Heartbeat connected on RolloutWorker_w15 +[2023-10-09 12:13:38,397][85186] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 613.1, 1: 481.0. Samples: 2998. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-10-09 12:13:43,397][85186] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 1033.5, 1: 980.6. Samples: 15590. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) +[2023-10-09 12:13:45,302][86121] Updated weights for policy 0, policy_version 10 (0.0010) +[2023-10-09 12:13:45,441][86122] Updated weights for policy 1, policy_version 10 (0.0010) +[2023-10-09 12:13:45,668][86121] Updated weights for policy 0, policy_version 20 (0.0011) +[2023-10-09 12:13:45,793][86122] Updated weights for policy 1, policy_version 20 (0.0010) +[2023-10-09 12:13:46,040][86121] Updated weights for policy 0, policy_version 30 (0.0008) +[2023-10-09 12:13:46,153][86122] Updated weights for policy 1, policy_version 30 (0.0007) +[2023-10-09 12:13:48,397][85186] Fps is (10 sec: 6553.6, 60 sec: 5144.0, 300 sec: 5144.0). Total num frames: 65536. Throughput: 0: 1306.9, 1: 1292.1. Samples: 33112. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) +[2023-10-09 12:13:48,399][86122] Updated weights for policy 1, policy_version 40 (0.0009) +[2023-10-09 12:13:48,470][86121] Updated weights for policy 0, policy_version 40 (0.0008) +[2023-10-09 12:13:48,757][86122] Updated weights for policy 1, policy_version 50 (0.0008) +[2023-10-09 12:13:48,836][86121] Updated weights for policy 0, policy_version 50 (0.0009) +[2023-10-09 12:13:49,121][86122] Updated weights for policy 1, policy_version 60 (0.0009) +[2023-10-09 12:13:49,198][86121] Updated weights for policy 0, policy_version 60 (0.0008) +[2023-10-09 12:13:52,402][86122] Updated weights for policy 1, policy_version 70 (0.0007) +[2023-10-09 12:13:52,436][86121] Updated weights for policy 0, policy_version 70 (0.0008) +[2023-10-09 12:13:52,764][86122] Updated weights for policy 1, policy_version 80 (0.0008) +[2023-10-09 12:13:52,796][86121] Updated weights for policy 0, policy_version 80 (0.0007) +[2023-10-09 12:13:53,135][86122] Updated weights for policy 1, policy_version 90 (0.0009) +[2023-10-09 12:13:53,169][86121] Updated weights for policy 0, policy_version 90 (0.0007) +[2023-10-09 12:13:53,397][85186] Fps is (10 sec: 19660.6, 60 sec: 11082.5, 300 sec: 11082.5). Total num frames: 196608. Throughput: 0: 1512.8, 1: 1509.9. Samples: 53624. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 12:13:56,393][86122] Updated weights for policy 1, policy_version 100 (0.0008) +[2023-10-09 12:13:56,570][86121] Updated weights for policy 0, policy_version 100 (0.0009) +[2023-10-09 12:13:56,753][86122] Updated weights for policy 1, policy_version 110 (0.0007) +[2023-10-09 12:13:56,932][86121] Updated weights for policy 0, policy_version 110 (0.0010) +[2023-10-09 12:13:57,117][86122] Updated weights for policy 1, policy_version 120 (0.0008) +[2023-10-09 12:13:57,301][86121] Updated weights for policy 0, policy_version 120 (0.0007) +[2023-10-09 12:13:58,397][85186] Fps is (10 sec: 19660.8, 60 sec: 11527.8, 300 sec: 11527.8). Total num frames: 262144. Throughput: 0: 1428.3, 1: 1432.5. Samples: 65056. Policy #0 lag: (min: 22.0, avg: 24.9, max: 54.0) +[2023-10-09 12:13:58,398][85186] Avg episode reward: [(0, '7.562'), (1, '7.750')] +[2023-10-09 12:13:58,398][85763] Saving new best policy, reward=7.562! +[2023-10-09 12:13:58,398][85963] Saving new best policy, reward=7.750! +[2023-10-09 12:14:00,922][86122] Updated weights for policy 1, policy_version 130 (0.0009) +[2023-10-09 12:14:01,126][86121] Updated weights for policy 0, policy_version 130 (0.0009) +[2023-10-09 12:14:01,282][86122] Updated weights for policy 1, policy_version 140 (0.0009) +[2023-10-09 12:14:01,489][86121] Updated weights for policy 0, policy_version 140 (0.0007) +[2023-10-09 12:14:01,641][86122] Updated weights for policy 1, policy_version 150 (0.0009) +[2023-10-09 12:14:01,859][86121] Updated weights for policy 0, policy_version 150 (0.0008) +[2023-10-09 12:14:02,008][86122] Updated weights for policy 1, policy_version 160 (0.0009) +[2023-10-09 12:14:02,221][86121] Updated weights for policy 0, policy_version 160 (0.0008) +[2023-10-09 12:14:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 11812.4, 300 sec: 11812.4). Total num frames: 327680. Throughput: 0: 1536.9, 1: 1534.7. Samples: 85206. Policy #0 lag: (min: 31.0, avg: 33.8, max: 63.0) +[2023-10-09 12:14:03,398][85186] Avg episode reward: [(0, '7.562'), (1, '7.750')] +[2023-10-09 12:14:05,894][86122] Updated weights for policy 1, policy_version 170 (0.0008) +[2023-10-09 12:14:06,058][86121] Updated weights for policy 0, policy_version 170 (0.0007) +[2023-10-09 12:14:06,261][86122] Updated weights for policy 1, policy_version 180 (0.0008) +[2023-10-09 12:14:06,425][86121] Updated weights for policy 0, policy_version 180 (0.0007) +[2023-10-09 12:14:06,615][86122] Updated weights for policy 1, policy_version 190 (0.0007) +[2023-10-09 12:14:06,794][86121] Updated weights for policy 0, policy_version 190 (0.0007) +[2023-10-09 12:14:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 12010.1, 300 sec: 12010.1). Total num frames: 393216. Throughput: 0: 1635.7, 1: 1641.6. Samples: 107300. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:14:08,399][85186] Avg episode reward: [(0, '7.562'), (1, '7.750')] +[2023-10-09 12:14:10,284][86122] Updated weights for policy 1, policy_version 200 (0.0009) +[2023-10-09 12:14:10,545][86121] Updated weights for policy 0, policy_version 200 (0.0008) +[2023-10-09 12:14:10,643][86122] Updated weights for policy 1, policy_version 210 (0.0008) +[2023-10-09 12:14:10,908][86121] Updated weights for policy 0, policy_version 210 (0.0008) +[2023-10-09 12:14:11,010][86122] Updated weights for policy 1, policy_version 220 (0.0008) +[2023-10-09 12:14:11,282][86121] Updated weights for policy 0, policy_version 220 (0.0009) +[2023-10-09 12:14:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 12155.5, 300 sec: 12155.5). Total num frames: 458752. Throughput: 0: 1567.9, 1: 1566.8. Samples: 118304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:14:13,398][85186] Avg episode reward: [(0, '7.562'), (1, '7.750')] +[2023-10-09 12:14:14,801][86122] Updated weights for policy 1, policy_version 230 (0.0009) +[2023-10-09 12:14:15,075][86121] Updated weights for policy 0, policy_version 230 (0.0008) +[2023-10-09 12:14:15,167][86122] Updated weights for policy 1, policy_version 240 (0.0007) +[2023-10-09 12:14:15,442][86121] Updated weights for policy 0, policy_version 240 (0.0009) +[2023-10-09 12:14:15,525][86122] Updated weights for policy 1, policy_version 250 (0.0008) +[2023-10-09 12:14:15,812][86121] Updated weights for policy 0, policy_version 250 (0.0011) +[2023-10-09 12:14:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 12266.8, 300 sec: 12266.8). Total num frames: 524288. Throughput: 0: 1626.8, 1: 1639.6. Samples: 139608. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) +[2023-10-09 12:14:18,398][85186] Avg episode reward: [(0, '7.469'), (1, '7.406')] +[2023-10-09 12:14:19,184][86122] Updated weights for policy 1, policy_version 260 (0.0007) +[2023-10-09 12:14:19,544][86122] Updated weights for policy 1, policy_version 270 (0.0009) +[2023-10-09 12:14:19,556][86121] Updated weights for policy 0, policy_version 260 (0.0009) +[2023-10-09 12:14:19,909][86122] Updated weights for policy 1, policy_version 280 (0.0007) +[2023-10-09 12:14:19,922][86121] Updated weights for policy 0, policy_version 270 (0.0007) +[2023-10-09 12:14:20,293][86121] Updated weights for policy 0, policy_version 280 (0.0008) +[2023-10-09 12:14:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 12354.9, 300 sec: 12354.9). Total num frames: 589824. Throughput: 0: 1756.4, 1: 1782.4. Samples: 162246. Policy #0 lag: (min: 31.0, avg: 41.8, max: 63.0) +[2023-10-09 12:14:23,398][85186] Avg episode reward: [(0, '7.469'), (1, '7.406')] +[2023-10-09 12:14:23,640][86122] Updated weights for policy 1, policy_version 290 (0.0009) +[2023-10-09 12:14:24,010][86122] Updated weights for policy 1, policy_version 300 (0.0007) +[2023-10-09 12:14:24,048][86121] Updated weights for policy 0, policy_version 290 (0.0010) +[2023-10-09 12:14:24,374][86122] Updated weights for policy 1, policy_version 310 (0.0008) +[2023-10-09 12:14:24,418][86121] Updated weights for policy 0, policy_version 300 (0.0010) +[2023-10-09 12:14:24,736][86122] Updated weights for policy 1, policy_version 320 (0.0007) +[2023-10-09 12:14:24,782][86121] Updated weights for policy 0, policy_version 310 (0.0008) +[2023-10-09 12:14:25,150][86121] Updated weights for policy 0, policy_version 320 (0.0008) +[2023-10-09 12:14:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 12426.2, 300 sec: 12426.2). Total num frames: 655360. Throughput: 0: 1725.5, 1: 1754.0. Samples: 172164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:14:28,398][85186] Avg episode reward: [(0, '7.469'), (1, '7.406')] +[2023-10-09 12:14:28,543][86122] Updated weights for policy 1, policy_version 330 (0.0008) +[2023-10-09 12:14:28,810][86121] Updated weights for policy 0, policy_version 330 (0.0009) +[2023-10-09 12:14:28,909][86122] Updated weights for policy 1, policy_version 340 (0.0007) +[2023-10-09 12:14:29,179][86121] Updated weights for policy 0, policy_version 340 (0.0008) +[2023-10-09 12:14:29,271][86122] Updated weights for policy 1, policy_version 350 (0.0009) +[2023-10-09 12:14:29,547][86121] Updated weights for policy 0, policy_version 350 (0.0008) +[2023-10-09 12:14:32,834][86122] Updated weights for policy 1, policy_version 360 (0.0007) +[2023-10-09 12:14:33,209][86122] Updated weights for policy 1, policy_version 370 (0.0007) +[2023-10-09 12:14:33,332][86121] Updated weights for policy 0, policy_version 360 (0.0009) +[2023-10-09 12:14:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 12485.1, 300 sec: 12485.1). Total num frames: 720896. Throughput: 0: 1785.5, 1: 1806.2. Samples: 194736. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 12:14:33,398][85186] Avg episode reward: [(0, '7.457'), (1, '7.400')] +[2023-10-09 12:14:33,571][86122] Updated weights for policy 1, policy_version 380 (0.0008) +[2023-10-09 12:14:33,712][86121] Updated weights for policy 0, policy_version 370 (0.0008) +[2023-10-09 12:14:34,074][86121] Updated weights for policy 0, policy_version 380 (0.0009) +[2023-10-09 12:14:37,244][86122] Updated weights for policy 1, policy_version 390 (0.0008) +[2023-10-09 12:14:37,608][86122] Updated weights for policy 1, policy_version 400 (0.0009) +[2023-10-09 12:14:37,816][86121] Updated weights for policy 0, policy_version 390 (0.0008) +[2023-10-09 12:14:37,976][86122] Updated weights for policy 1, policy_version 410 (0.0010) +[2023-10-09 12:14:38,187][86121] Updated weights for policy 0, policy_version 400 (0.0008) +[2023-10-09 12:14:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 13653.3, 300 sec: 13057.0). Total num frames: 819200. Throughput: 0: 1801.5, 1: 1807.1. Samples: 216010. Policy #0 lag: (min: 31.0, avg: 35.5, max: 63.0) +[2023-10-09 12:14:38,398][85186] Avg episode reward: [(0, '7.396'), (1, '7.729')] +[2023-10-09 12:14:38,558][86121] Updated weights for policy 0, policy_version 410 (0.0010) +[2023-10-09 12:14:41,723][86122] Updated weights for policy 1, policy_version 420 (0.0010) +[2023-10-09 12:14:42,080][86122] Updated weights for policy 1, policy_version 430 (0.0010) +[2023-10-09 12:14:42,309][86121] Updated weights for policy 0, policy_version 420 (0.0007) +[2023-10-09 12:14:42,437][86122] Updated weights for policy 1, policy_version 440 (0.0009) +[2023-10-09 12:14:42,677][86121] Updated weights for policy 0, policy_version 430 (0.0008) +[2023-10-09 12:14:43,040][86121] Updated weights for policy 0, policy_version 440 (0.0008) +[2023-10-09 12:14:43,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 13544.4). Total num frames: 917504. Throughput: 0: 1791.7, 1: 1806.6. Samples: 226978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:14:43,398][85186] Avg episode reward: [(0, '7.396'), (1, '7.729')] +[2023-10-09 12:14:46,245][86122] Updated weights for policy 1, policy_version 450 (0.0010) +[2023-10-09 12:14:46,602][86122] Updated weights for policy 1, policy_version 460 (0.0008) +[2023-10-09 12:14:46,729][86121] Updated weights for policy 0, policy_version 450 (0.0009) +[2023-10-09 12:14:46,970][86122] Updated weights for policy 1, policy_version 470 (0.0007) +[2023-10-09 12:14:47,099][86121] Updated weights for policy 0, policy_version 460 (0.0009) +[2023-10-09 12:14:47,328][86122] Updated weights for policy 1, policy_version 480 (0.0007) +[2023-10-09 12:14:47,473][86121] Updated weights for policy 0, policy_version 470 (0.0007) +[2023-10-09 12:14:47,837][86121] Updated weights for policy 0, policy_version 480 (0.0008) +[2023-10-09 12:14:48,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 13514.4). Total num frames: 983040. Throughput: 0: 1809.7, 1: 1822.1. Samples: 248632. Policy #0 lag: (min: 1.0, avg: 8.8, max: 33.0) +[2023-10-09 12:14:48,398][85186] Avg episode reward: [(0, '7.396'), (1, '7.729')] +[2023-10-09 12:14:51,166][86122] Updated weights for policy 1, policy_version 490 (0.0008) +[2023-10-09 12:14:51,525][86122] Updated weights for policy 1, policy_version 500 (0.0008) +[2023-10-09 12:14:51,562][86121] Updated weights for policy 0, policy_version 490 (0.0010) +[2023-10-09 12:14:51,895][86122] Updated weights for policy 1, policy_version 510 (0.0008) +[2023-10-09 12:14:51,929][86121] Updated weights for policy 0, policy_version 500 (0.0008) +[2023-10-09 12:14:52,308][86121] Updated weights for policy 0, policy_version 510 (0.0008) +[2023-10-09 12:14:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13488.2). Total num frames: 1048576. Throughput: 0: 1786.9, 1: 1810.0. Samples: 269162. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 12:14:53,398][85186] Avg episode reward: [(0, '7.643'), (1, '7.966')] +[2023-10-09 12:14:53,401][85963] Saving new best policy, reward=7.966! +[2023-10-09 12:14:53,401][85763] Saving new best policy, reward=7.643! +[2023-10-09 12:14:55,504][86122] Updated weights for policy 1, policy_version 520 (0.0007) +[2023-10-09 12:14:55,869][86122] Updated weights for policy 1, policy_version 530 (0.0008) +[2023-10-09 12:14:55,992][86121] Updated weights for policy 0, policy_version 520 (0.0008) +[2023-10-09 12:14:56,229][86122] Updated weights for policy 1, policy_version 540 (0.0008) +[2023-10-09 12:14:56,353][86121] Updated weights for policy 0, policy_version 530 (0.0008) +[2023-10-09 12:14:56,709][86121] Updated weights for policy 0, policy_version 540 (0.0010) +[2023-10-09 12:14:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13465.2). Total num frames: 1114112. Throughput: 0: 1799.8, 1: 1815.8. Samples: 281008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:14:58,398][85186] Avg episode reward: [(0, '7.719'), (1, '8.016')] +[2023-10-09 12:14:58,398][85763] Saving new best policy, reward=7.719! +[2023-10-09 12:14:58,398][85963] Saving new best policy, reward=8.016! +[2023-10-09 12:15:00,030][86122] Updated weights for policy 1, policy_version 550 (0.0008) +[2023-10-09 12:15:00,393][86122] Updated weights for policy 1, policy_version 560 (0.0009) +[2023-10-09 12:15:00,596][86121] Updated weights for policy 0, policy_version 550 (0.0009) +[2023-10-09 12:15:00,759][86122] Updated weights for policy 1, policy_version 570 (0.0009) +[2023-10-09 12:15:00,956][86121] Updated weights for policy 0, policy_version 560 (0.0010) +[2023-10-09 12:15:01,329][86121] Updated weights for policy 0, policy_version 570 (0.0010) +[2023-10-09 12:15:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13444.8). Total num frames: 1179648. Throughput: 0: 1793.7, 1: 1809.4. Samples: 301748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:15:03,398][85186] Avg episode reward: [(0, '7.719'), (1, '8.016')] +[2023-10-09 12:15:04,567][86122] Updated weights for policy 1, policy_version 580 (0.0007) +[2023-10-09 12:15:04,933][86122] Updated weights for policy 1, policy_version 590 (0.0008) +[2023-10-09 12:15:04,971][86121] Updated weights for policy 0, policy_version 580 (0.0007) +[2023-10-09 12:15:05,295][86122] Updated weights for policy 1, policy_version 600 (0.0008) +[2023-10-09 12:15:05,340][86121] Updated weights for policy 0, policy_version 590 (0.0008) +[2023-10-09 12:15:05,708][86121] Updated weights for policy 0, policy_version 600 (0.0009) +[2023-10-09 12:15:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13426.6). Total num frames: 1245184. Throughput: 0: 1804.8, 1: 1811.4. Samples: 324972. Policy #0 lag: (min: 17.0, avg: 30.8, max: 49.0) +[2023-10-09 12:15:08,398][85186] Avg episode reward: [(0, '7.719'), (1, '8.016')] +[2023-10-09 12:15:08,970][86122] Updated weights for policy 1, policy_version 610 (0.0008) +[2023-10-09 12:15:09,332][86122] Updated weights for policy 1, policy_version 620 (0.0007) +[2023-10-09 12:15:09,427][86121] Updated weights for policy 0, policy_version 610 (0.0007) +[2023-10-09 12:15:09,697][86122] Updated weights for policy 1, policy_version 630 (0.0008) +[2023-10-09 12:15:09,795][86121] Updated weights for policy 0, policy_version 620 (0.0010) +[2023-10-09 12:15:10,062][86122] Updated weights for policy 1, policy_version 640 (0.0009) +[2023-10-09 12:15:10,170][86121] Updated weights for policy 0, policy_version 630 (0.0010) +[2023-10-09 12:15:10,535][86121] Updated weights for policy 0, policy_version 640 (0.0010) +[2023-10-09 12:15:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13410.2). Total num frames: 1310720. Throughput: 0: 1796.7, 1: 1807.1. Samples: 334332. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) +[2023-10-09 12:15:13,398][85186] Avg episode reward: [(0, '8.039'), (1, '8.295')] +[2023-10-09 12:15:13,398][85763] Saving new best policy, reward=8.039! +[2023-10-09 12:15:13,399][85963] Saving new best policy, reward=8.295! +[2023-10-09 12:15:14,064][86122] Updated weights for policy 1, policy_version 650 (0.0011) +[2023-10-09 12:15:14,427][86122] Updated weights for policy 1, policy_version 660 (0.0011) +[2023-10-09 12:15:14,735][86121] Updated weights for policy 0, policy_version 650 (0.0008) +[2023-10-09 12:15:14,790][86122] Updated weights for policy 1, policy_version 670 (0.0008) +[2023-10-09 12:15:15,103][86121] Updated weights for policy 0, policy_version 660 (0.0009) +[2023-10-09 12:15:15,476][86121] Updated weights for policy 0, policy_version 670 (0.0009) +[2023-10-09 12:15:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 13395.5). Total num frames: 1376256. Throughput: 0: 1779.8, 1: 1795.8. Samples: 355640. Policy #0 lag: (min: 31.0, avg: 32.9, max: 61.0) +[2023-10-09 12:15:18,398][85186] Avg episode reward: [(0, '8.087'), (1, '8.325')] +[2023-10-09 12:15:18,398][85763] Saving new best policy, reward=8.087! +[2023-10-09 12:15:18,592][86122] Updated weights for policy 1, policy_version 680 (0.0008) +[2023-10-09 12:15:18,965][86122] Updated weights for policy 1, policy_version 690 (0.0010) +[2023-10-09 12:15:19,323][86122] Updated weights for policy 1, policy_version 700 (0.0010) +[2023-10-09 12:15:19,383][86121] Updated weights for policy 0, policy_version 680 (0.0009) +[2023-10-09 12:15:19,472][85963] Saving new best policy, reward=8.325! +[2023-10-09 12:15:19,751][86121] Updated weights for policy 0, policy_version 690 (0.0009) +[2023-10-09 12:15:20,121][86121] Updated weights for policy 0, policy_version 700 (0.0010) +[2023-10-09 12:15:23,315][86122] Updated weights for policy 1, policy_version 710 (0.0010) +[2023-10-09 12:15:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13382.1). Total num frames: 1441792. Throughput: 0: 1772.5, 1: 1800.4. Samples: 376788. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 12:15:23,398][85186] Avg episode reward: [(0, '8.087'), (1, '8.325')] +[2023-10-09 12:15:23,400][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000000704_720896.pth... +[2023-10-09 12:15:23,678][86122] Updated weights for policy 1, policy_version 720 (0.0008) +[2023-10-09 12:15:24,042][86122] Updated weights for policy 1, policy_version 730 (0.0010) +[2023-10-09 12:15:24,101][86121] Updated weights for policy 0, policy_version 710 (0.0010) +[2023-10-09 12:15:24,258][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000000736_753664.pth... +[2023-10-09 12:15:24,464][86121] Updated weights for policy 0, policy_version 720 (0.0010) +[2023-10-09 12:15:24,833][86121] Updated weights for policy 0, policy_version 730 (0.0011) +[2023-10-09 12:15:27,655][86122] Updated weights for policy 1, policy_version 740 (0.0009) +[2023-10-09 12:15:28,019][86122] Updated weights for policy 1, policy_version 750 (0.0009) +[2023-10-09 12:15:28,380][86122] Updated weights for policy 1, policy_version 760 (0.0009) +[2023-10-09 12:15:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 13369.9). Total num frames: 1507328. Throughput: 0: 1762.7, 1: 1779.5. Samples: 386376. Policy #0 lag: (min: 4.0, avg: 12.0, max: 36.0) +[2023-10-09 12:15:28,398][85186] Avg episode reward: [(0, '8.087'), (1, '8.325')] +[2023-10-09 12:15:28,579][86121] Updated weights for policy 0, policy_version 740 (0.0009) +[2023-10-09 12:15:28,948][86121] Updated weights for policy 0, policy_version 750 (0.0010) +[2023-10-09 12:15:29,322][86121] Updated weights for policy 0, policy_version 760 (0.0008) +[2023-10-09 12:15:32,187][86122] Updated weights for policy 1, policy_version 770 (0.0009) +[2023-10-09 12:15:32,550][86122] Updated weights for policy 1, policy_version 780 (0.0009) +[2023-10-09 12:15:32,923][86122] Updated weights for policy 1, policy_version 790 (0.0010) +[2023-10-09 12:15:33,190][86121] Updated weights for policy 0, policy_version 770 (0.0008) +[2023-10-09 12:15:33,296][86122] Updated weights for policy 1, policy_version 800 (0.0009) +[2023-10-09 12:15:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 13637.1). Total num frames: 1605632. Throughput: 0: 1768.9, 1: 1794.8. Samples: 409000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:15:33,398][85186] Avg episode reward: [(0, '8.362'), (1, '8.532')] +[2023-10-09 12:15:33,399][85963] Saving new best policy, reward=8.532! +[2023-10-09 12:15:33,561][86121] Updated weights for policy 0, policy_version 780 (0.0008) +[2023-10-09 12:15:33,935][86121] Updated weights for policy 0, policy_version 790 (0.0010) +[2023-10-09 12:15:34,303][85763] Saving new best policy, reward=8.362! +[2023-10-09 12:15:34,305][86121] Updated weights for policy 0, policy_version 800 (0.0010) +[2023-10-09 12:15:36,880][86122] Updated weights for policy 1, policy_version 810 (0.0008) +[2023-10-09 12:15:37,235][86122] Updated weights for policy 1, policy_version 820 (0.0008) +[2023-10-09 12:15:37,598][86122] Updated weights for policy 1, policy_version 830 (0.0008) +[2023-10-09 12:15:37,784][86121] Updated weights for policy 0, policy_version 810 (0.0009) +[2023-10-09 12:15:38,143][86121] Updated weights for policy 0, policy_version 820 (0.0008) +[2023-10-09 12:15:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 13615.5). Total num frames: 1671168. Throughput: 0: 1791.8, 1: 1778.9. Samples: 429844. Policy #0 lag: (min: 15.0, avg: 22.7, max: 47.0) +[2023-10-09 12:15:38,398][85186] Avg episode reward: [(0, '8.396'), (1, '8.562')] +[2023-10-09 12:15:38,403][85963] Saving new best policy, reward=8.562! +[2023-10-09 12:15:38,523][86121] Updated weights for policy 0, policy_version 830 (0.0007) +[2023-10-09 12:15:38,588][85763] Saving new best policy, reward=8.396! +[2023-10-09 12:15:41,367][86122] Updated weights for policy 1, policy_version 840 (0.0007) +[2023-10-09 12:15:41,730][86122] Updated weights for policy 1, policy_version 850 (0.0009) +[2023-10-09 12:15:42,094][86122] Updated weights for policy 1, policy_version 860 (0.0008) +[2023-10-09 12:15:42,112][86121] Updated weights for policy 0, policy_version 840 (0.0008) +[2023-10-09 12:15:42,478][86121] Updated weights for policy 0, policy_version 850 (0.0008) +[2023-10-09 12:15:42,853][86121] Updated weights for policy 0, policy_version 860 (0.0009) +[2023-10-09 12:15:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 13852.1). Total num frames: 1769472. Throughput: 0: 1780.8, 1: 1796.3. Samples: 441982. Policy #0 lag: (min: 14.0, avg: 38.1, max: 40.0) +[2023-10-09 12:15:43,398][85186] Avg episode reward: [(0, '8.396'), (1, '8.562')] +[2023-10-09 12:15:45,866][86122] Updated weights for policy 1, policy_version 870 (0.0009) +[2023-10-09 12:15:46,233][86122] Updated weights for policy 1, policy_version 880 (0.0008) +[2023-10-09 12:15:46,399][86121] Updated weights for policy 0, policy_version 870 (0.0008) +[2023-10-09 12:15:46,600][86122] Updated weights for policy 1, policy_version 890 (0.0008) +[2023-10-09 12:15:46,767][86121] Updated weights for policy 0, policy_version 880 (0.0007) +[2023-10-09 12:15:47,131][86121] Updated weights for policy 0, policy_version 890 (0.0007) +[2023-10-09 12:15:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14199.5, 300 sec: 13824.1). Total num frames: 1835008. Throughput: 0: 1795.3, 1: 1778.6. Samples: 462576. Policy #0 lag: (min: 16.0, avg: 39.3, max: 48.0) +[2023-10-09 12:15:48,398][85186] Avg episode reward: [(0, '8.412'), (1, '8.567')] +[2023-10-09 12:15:48,399][85763] Saving new best policy, reward=8.412! +[2023-10-09 12:15:48,399][85963] Saving new best policy, reward=8.567! +[2023-10-09 12:15:50,220][86122] Updated weights for policy 1, policy_version 900 (0.0008) +[2023-10-09 12:15:50,591][86122] Updated weights for policy 1, policy_version 910 (0.0008) +[2023-10-09 12:15:50,836][86121] Updated weights for policy 0, policy_version 900 (0.0009) +[2023-10-09 12:15:50,950][86122] Updated weights for policy 1, policy_version 920 (0.0007) +[2023-10-09 12:15:51,202][86121] Updated weights for policy 0, policy_version 910 (0.0007) +[2023-10-09 12:15:51,580][86121] Updated weights for policy 0, policy_version 920 (0.0010) +[2023-10-09 12:15:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13798.0). Total num frames: 1900544. Throughput: 0: 1776.1, 1: 1776.8. Samples: 484854. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 12:15:53,398][85186] Avg episode reward: [(0, '8.680'), (1, '8.890')] +[2023-10-09 12:15:53,408][85763] Saving new best policy, reward=8.680! +[2023-10-09 12:15:53,408][85963] Saving new best policy, reward=8.890! +[2023-10-09 12:15:54,705][86122] Updated weights for policy 1, policy_version 930 (0.0008) +[2023-10-09 12:15:55,064][86122] Updated weights for policy 1, policy_version 940 (0.0010) +[2023-10-09 12:15:55,243][86121] Updated weights for policy 0, policy_version 930 (0.0008) +[2023-10-09 12:15:55,432][86122] Updated weights for policy 1, policy_version 950 (0.0008) +[2023-10-09 12:15:55,611][86121] Updated weights for policy 0, policy_version 940 (0.0009) +[2023-10-09 12:15:55,793][86122] Updated weights for policy 1, policy_version 960 (0.0009) +[2023-10-09 12:15:55,985][86121] Updated weights for policy 0, policy_version 950 (0.0007) +[2023-10-09 12:15:56,355][86121] Updated weights for policy 0, policy_version 960 (0.0008) +[2023-10-09 12:15:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13773.8). Total num frames: 1966080. Throughput: 0: 1798.9, 1: 1781.4. Samples: 495446. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) +[2023-10-09 12:15:58,398][85186] Avg episode reward: [(0, '8.730'), (1, '8.890')] +[2023-10-09 12:15:58,398][85763] Saving new best policy, reward=8.730! +[2023-10-09 12:15:59,580][86122] Updated weights for policy 1, policy_version 970 (0.0007) +[2023-10-09 12:15:59,948][86122] Updated weights for policy 1, policy_version 980 (0.0007) +[2023-10-09 12:16:00,077][86121] Updated weights for policy 0, policy_version 970 (0.0009) +[2023-10-09 12:16:00,311][86122] Updated weights for policy 1, policy_version 990 (0.0009) +[2023-10-09 12:16:00,436][86121] Updated weights for policy 0, policy_version 980 (0.0010) +[2023-10-09 12:16:00,808][86121] Updated weights for policy 0, policy_version 990 (0.0010) +[2023-10-09 12:16:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 13751.3). Total num frames: 2031616. Throughput: 0: 1804.5, 1: 1793.6. Samples: 517556. Policy #0 lag: (min: 21.0, avg: 27.4, max: 53.0) +[2023-10-09 12:16:03,398][85186] Avg episode reward: [(0, '8.730'), (1, '8.890')] +[2023-10-09 12:16:04,071][86122] Updated weights for policy 1, policy_version 1000 (0.0009) +[2023-10-09 12:16:04,451][86122] Updated weights for policy 1, policy_version 1010 (0.0009) +[2023-10-09 12:16:04,463][86121] Updated weights for policy 0, policy_version 1000 (0.0009) +[2023-10-09 12:16:04,822][86122] Updated weights for policy 1, policy_version 1020 (0.0007) +[2023-10-09 12:16:04,846][86121] Updated weights for policy 0, policy_version 1010 (0.0008) +[2023-10-09 12:16:05,212][86121] Updated weights for policy 0, policy_version 1020 (0.0007) +[2023-10-09 12:16:08,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 13730.2). Total num frames: 2097152. Throughput: 0: 1826.6, 1: 1812.2. Samples: 540534. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 12:16:08,398][85186] Avg episode reward: [(0, '8.910'), (1, '9.130')] +[2023-10-09 12:16:08,408][85763] Saving new best policy, reward=8.910! +[2023-10-09 12:16:08,454][86122] Updated weights for policy 1, policy_version 1030 (0.0009) +[2023-10-09 12:16:08,819][86122] Updated weights for policy 1, policy_version 1040 (0.0008) +[2023-10-09 12:16:08,917][86121] Updated weights for policy 0, policy_version 1030 (0.0009) +[2023-10-09 12:16:09,181][86122] Updated weights for policy 1, policy_version 1050 (0.0008) +[2023-10-09 12:16:09,278][86121] Updated weights for policy 0, policy_version 1040 (0.0008) +[2023-10-09 12:16:09,400][85963] Saving new best policy, reward=9.130! +[2023-10-09 12:16:09,658][86121] Updated weights for policy 0, policy_version 1050 (0.0009) +[2023-10-09 12:16:13,001][86122] Updated weights for policy 1, policy_version 1060 (0.0008) +[2023-10-09 12:16:13,333][86121] Updated weights for policy 0, policy_version 1060 (0.0008) +[2023-10-09 12:16:13,365][86122] Updated weights for policy 1, policy_version 1070 (0.0008) +[2023-10-09 12:16:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13710.4). Total num frames: 2162688. Throughput: 0: 1829.8, 1: 1809.3. Samples: 550136. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 12:16:13,398][85186] Avg episode reward: [(0, '9.120'), (1, '9.310')] +[2023-10-09 12:16:13,700][86121] Updated weights for policy 0, policy_version 1070 (0.0007) +[2023-10-09 12:16:13,731][86122] Updated weights for policy 1, policy_version 1080 (0.0008) +[2023-10-09 12:16:14,022][85963] Saving new best policy, reward=9.310! +[2023-10-09 12:16:14,063][86121] Updated weights for policy 0, policy_version 1080 (0.0008) +[2023-10-09 12:16:14,358][85763] Saving new best policy, reward=9.120! +[2023-10-09 12:16:17,351][86122] Updated weights for policy 1, policy_version 1090 (0.0007) +[2023-10-09 12:16:17,717][86122] Updated weights for policy 1, policy_version 1100 (0.0007) +[2023-10-09 12:16:17,768][86121] Updated weights for policy 0, policy_version 1090 (0.0010) +[2023-10-09 12:16:18,082][86122] Updated weights for policy 1, policy_version 1110 (0.0008) +[2023-10-09 12:16:18,133][86121] Updated weights for policy 0, policy_version 1100 (0.0007) +[2023-10-09 12:16:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 13691.9). Total num frames: 2228224. Throughput: 0: 1830.9, 1: 1805.2. Samples: 572624. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:16:18,398][85186] Avg episode reward: [(0, '9.120'), (1, '9.310')] +[2023-10-09 12:16:18,449][86122] Updated weights for policy 1, policy_version 1120 (0.0008) +[2023-10-09 12:16:18,503][86121] Updated weights for policy 0, policy_version 1110 (0.0011) +[2023-10-09 12:16:18,868][86121] Updated weights for policy 0, policy_version 1120 (0.0008) +[2023-10-09 12:16:22,224][86122] Updated weights for policy 1, policy_version 1130 (0.0008) +[2023-10-09 12:16:22,563][86121] Updated weights for policy 0, policy_version 1130 (0.0007) +[2023-10-09 12:16:22,590][86122] Updated weights for policy 1, policy_version 1140 (0.0008) +[2023-10-09 12:16:22,926][86121] Updated weights for policy 0, policy_version 1140 (0.0008) +[2023-10-09 12:16:22,946][86122] Updated weights for policy 1, policy_version 1150 (0.0007) +[2023-10-09 12:16:23,305][86121] Updated weights for policy 0, policy_version 1150 (0.0009) +[2023-10-09 12:16:23,397][85186] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14065.2). Total num frames: 2359296. Throughput: 0: 1823.0, 1: 1812.5. Samples: 593444. Policy #0 lag: (min: 31.0, avg: 35.9, max: 63.0) +[2023-10-09 12:16:23,398][85186] Avg episode reward: [(0, '9.120'), (1, '9.310')] +[2023-10-09 12:16:26,641][86122] Updated weights for policy 1, policy_version 1160 (0.0007) +[2023-10-09 12:16:27,018][86122] Updated weights for policy 1, policy_version 1170 (0.0008) +[2023-10-09 12:16:27,061][86121] Updated weights for policy 0, policy_version 1160 (0.0009) +[2023-10-09 12:16:27,380][86122] Updated weights for policy 1, policy_version 1180 (0.0008) +[2023-10-09 12:16:27,422][86121] Updated weights for policy 0, policy_version 1170 (0.0009) +[2023-10-09 12:16:27,786][86121] Updated weights for policy 0, policy_version 1180 (0.0011) +[2023-10-09 12:16:28,397][85186] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14037.5). Total num frames: 2424832. Throughput: 0: 1821.0, 1: 1806.9. Samples: 605236. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 12:16:28,398][85186] Avg episode reward: [(0, '9.330'), (1, '9.560')] +[2023-10-09 12:16:28,398][85763] Saving new best policy, reward=9.330! +[2023-10-09 12:16:28,398][85963] Saving new best policy, reward=9.560! +[2023-10-09 12:16:31,255][86122] Updated weights for policy 1, policy_version 1190 (0.0010) +[2023-10-09 12:16:31,631][86122] Updated weights for policy 1, policy_version 1200 (0.0010) +[2023-10-09 12:16:31,777][86121] Updated weights for policy 0, policy_version 1190 (0.0010) +[2023-10-09 12:16:31,983][86122] Updated weights for policy 1, policy_version 1210 (0.0009) +[2023-10-09 12:16:32,144][86121] Updated weights for policy 0, policy_version 1200 (0.0009) +[2023-10-09 12:16:32,506][86121] Updated weights for policy 0, policy_version 1210 (0.0009) +[2023-10-09 12:16:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14011.3). Total num frames: 2490368. Throughput: 0: 1816.0, 1: 1813.1. Samples: 625888. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:16:33,398][85186] Avg episode reward: [(0, '9.550'), (1, '9.600')] +[2023-10-09 12:16:33,398][85763] Saving new best policy, reward=9.550! +[2023-10-09 12:16:33,399][85963] Saving new best policy, reward=9.600! +[2023-10-09 12:16:35,761][86122] Updated weights for policy 1, policy_version 1220 (0.0009) +[2023-10-09 12:16:36,118][86122] Updated weights for policy 1, policy_version 1230 (0.0008) +[2023-10-09 12:16:36,273][86121] Updated weights for policy 0, policy_version 1220 (0.0009) +[2023-10-09 12:16:36,493][86122] Updated weights for policy 1, policy_version 1240 (0.0009) +[2023-10-09 12:16:36,650][86121] Updated weights for policy 0, policy_version 1230 (0.0007) +[2023-10-09 12:16:37,010][86121] Updated weights for policy 0, policy_version 1240 (0.0010) +[2023-10-09 12:16:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 13986.5). Total num frames: 2555904. Throughput: 0: 1802.2, 1: 1797.5. Samples: 646840. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) +[2023-10-09 12:16:38,398][85186] Avg episode reward: [(0, '9.550'), (1, '9.600')] +[2023-10-09 12:16:40,239][86122] Updated weights for policy 1, policy_version 1250 (0.0008) +[2023-10-09 12:16:40,605][86122] Updated weights for policy 1, policy_version 1260 (0.0009) +[2023-10-09 12:16:40,717][86121] Updated weights for policy 0, policy_version 1250 (0.0009) +[2023-10-09 12:16:40,958][86122] Updated weights for policy 1, policy_version 1270 (0.0010) +[2023-10-09 12:16:41,073][86121] Updated weights for policy 0, policy_version 1260 (0.0008) +[2023-10-09 12:16:41,323][86122] Updated weights for policy 1, policy_version 1280 (0.0008) +[2023-10-09 12:16:41,444][86121] Updated weights for policy 0, policy_version 1270 (0.0007) +[2023-10-09 12:16:41,809][86121] Updated weights for policy 0, policy_version 1280 (0.0008) +[2023-10-09 12:16:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13963.1). Total num frames: 2621440. Throughput: 0: 1814.3, 1: 1808.3. Samples: 658462. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 12:16:43,398][85186] Avg episode reward: [(0, '9.550'), (1, '9.600')] +[2023-10-09 12:16:44,954][86122] Updated weights for policy 1, policy_version 1290 (0.0009) +[2023-10-09 12:16:45,323][86122] Updated weights for policy 1, policy_version 1300 (0.0009) +[2023-10-09 12:16:45,486][86121] Updated weights for policy 0, policy_version 1290 (0.0010) +[2023-10-09 12:16:45,697][86122] Updated weights for policy 1, policy_version 1310 (0.0010) +[2023-10-09 12:16:45,853][86121] Updated weights for policy 0, policy_version 1300 (0.0009) +[2023-10-09 12:16:46,224][86121] Updated weights for policy 0, policy_version 1310 (0.0007) +[2023-10-09 12:16:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13940.9). Total num frames: 2686976. Throughput: 0: 1800.1, 1: 1796.1. Samples: 679380. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 12:16:48,398][85186] Avg episode reward: [(0, '9.700'), (1, '9.730')] +[2023-10-09 12:16:48,399][85763] Saving new best policy, reward=9.700! +[2023-10-09 12:16:48,399][85963] Saving new best policy, reward=9.730! +[2023-10-09 12:16:49,547][86122] Updated weights for policy 1, policy_version 1320 (0.0007) +[2023-10-09 12:16:49,931][86122] Updated weights for policy 1, policy_version 1330 (0.0008) +[2023-10-09 12:16:50,177][86121] Updated weights for policy 0, policy_version 1320 (0.0008) +[2023-10-09 12:16:50,287][86122] Updated weights for policy 1, policy_version 1340 (0.0008) +[2023-10-09 12:16:50,550][86121] Updated weights for policy 0, policy_version 1330 (0.0010) +[2023-10-09 12:16:50,919][86121] Updated weights for policy 0, policy_version 1340 (0.0007) +[2023-10-09 12:16:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13919.8). Total num frames: 2752512. Throughput: 0: 1790.0, 1: 1789.5. Samples: 701610. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) +[2023-10-09 12:16:53,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.780')] +[2023-10-09 12:16:53,408][85763] Saving new best policy, reward=9.830! +[2023-10-09 12:16:53,409][85963] Saving new best policy, reward=9.780! +[2023-10-09 12:16:53,906][86122] Updated weights for policy 1, policy_version 1350 (0.0010) +[2023-10-09 12:16:54,277][86122] Updated weights for policy 1, policy_version 1360 (0.0010) +[2023-10-09 12:16:54,648][86122] Updated weights for policy 1, policy_version 1370 (0.0010) +[2023-10-09 12:16:54,724][86121] Updated weights for policy 0, policy_version 1350 (0.0009) +[2023-10-09 12:16:55,099][86121] Updated weights for policy 0, policy_version 1360 (0.0009) +[2023-10-09 12:16:55,463][86121] Updated weights for policy 0, policy_version 1370 (0.0009) +[2023-10-09 12:16:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 13899.8). Total num frames: 2818048. Throughput: 0: 1788.6, 1: 1792.9. Samples: 711306. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) +[2023-10-09 12:16:58,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.780')] +[2023-10-09 12:16:58,578][86122] Updated weights for policy 1, policy_version 1380 (0.0009) +[2023-10-09 12:16:58,942][86122] Updated weights for policy 1, policy_version 1390 (0.0009) +[2023-10-09 12:16:59,245][86121] Updated weights for policy 0, policy_version 1380 (0.0009) +[2023-10-09 12:16:59,303][86122] Updated weights for policy 1, policy_version 1400 (0.0009) +[2023-10-09 12:16:59,613][86121] Updated weights for policy 0, policy_version 1390 (0.0009) +[2023-10-09 12:16:59,985][86121] Updated weights for policy 0, policy_version 1400 (0.0008) +[2023-10-09 12:17:03,275][86122] Updated weights for policy 1, policy_version 1410 (0.0009) +[2023-10-09 12:17:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13880.7). Total num frames: 2883584. Throughput: 0: 1780.7, 1: 1784.5. Samples: 733058. Policy #0 lag: (min: 10.0, avg: 19.2, max: 42.0) +[2023-10-09 12:17:03,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.780')] +[2023-10-09 12:17:03,634][86122] Updated weights for policy 1, policy_version 1420 (0.0010) +[2023-10-09 12:17:03,870][86121] Updated weights for policy 0, policy_version 1410 (0.0009) +[2023-10-09 12:17:04,007][86122] Updated weights for policy 1, policy_version 1430 (0.0009) +[2023-10-09 12:17:04,245][86121] Updated weights for policy 0, policy_version 1420 (0.0010) +[2023-10-09 12:17:04,368][86122] Updated weights for policy 1, policy_version 1440 (0.0009) +[2023-10-09 12:17:04,602][86121] Updated weights for policy 0, policy_version 1430 (0.0009) +[2023-10-09 12:17:04,969][86121] Updated weights for policy 0, policy_version 1440 (0.0011) +[2023-10-09 12:17:08,275][86122] Updated weights for policy 1, policy_version 1450 (0.0007) +[2023-10-09 12:17:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13862.5). Total num frames: 2949120. Throughput: 0: 1792.1, 1: 1798.8. Samples: 755032. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 12:17:08,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.890')] +[2023-10-09 12:17:08,645][86122] Updated weights for policy 1, policy_version 1460 (0.0008) +[2023-10-09 12:17:08,702][86121] Updated weights for policy 0, policy_version 1450 (0.0007) +[2023-10-09 12:17:09,013][86122] Updated weights for policy 1, policy_version 1470 (0.0010) +[2023-10-09 12:17:09,074][86121] Updated weights for policy 0, policy_version 1460 (0.0008) +[2023-10-09 12:17:09,086][85963] Saving new best policy, reward=9.890! +[2023-10-09 12:17:09,444][86121] Updated weights for policy 0, policy_version 1470 (0.0011) +[2023-10-09 12:17:09,518][85763] Saving new best policy, reward=9.890! +[2023-10-09 12:17:12,835][86122] Updated weights for policy 1, policy_version 1480 (0.0009) +[2023-10-09 12:17:13,146][86121] Updated weights for policy 0, policy_version 1480 (0.0009) +[2023-10-09 12:17:13,198][86122] Updated weights for policy 1, policy_version 1490 (0.0007) +[2023-10-09 12:17:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13845.2). Total num frames: 3014656. Throughput: 0: 1774.8, 1: 1770.0. Samples: 764754. Policy #0 lag: (min: 31.0, avg: 33.7, max: 63.0) +[2023-10-09 12:17:13,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.900')] +[2023-10-09 12:17:13,510][86121] Updated weights for policy 0, policy_version 1490 (0.0008) +[2023-10-09 12:17:13,556][86122] Updated weights for policy 1, policy_version 1500 (0.0008) +[2023-10-09 12:17:13,702][85963] Saving new best policy, reward=9.900! +[2023-10-09 12:17:13,879][86121] Updated weights for policy 0, policy_version 1500 (0.0009) +[2023-10-09 12:17:17,349][86122] Updated weights for policy 1, policy_version 1510 (0.0007) +[2023-10-09 12:17:17,606][86121] Updated weights for policy 0, policy_version 1510 (0.0009) +[2023-10-09 12:17:17,707][86122] Updated weights for policy 1, policy_version 1520 (0.0008) +[2023-10-09 12:17:17,973][86121] Updated weights for policy 0, policy_version 1520 (0.0009) +[2023-10-09 12:17:18,077][86122] Updated weights for policy 1, policy_version 1530 (0.0009) +[2023-10-09 12:17:18,345][86121] Updated weights for policy 0, policy_version 1530 (0.0008) +[2023-10-09 12:17:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 13975.7). Total num frames: 3112960. Throughput: 0: 1792.4, 1: 1791.0. Samples: 787142. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 12:17:18,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.900')] +[2023-10-09 12:17:21,788][86122] Updated weights for policy 1, policy_version 1540 (0.0008) +[2023-10-09 12:17:21,883][86121] Updated weights for policy 0, policy_version 1540 (0.0009) +[2023-10-09 12:17:22,160][86122] Updated weights for policy 1, policy_version 1550 (0.0007) +[2023-10-09 12:17:22,253][86121] Updated weights for policy 0, policy_version 1550 (0.0007) +[2023-10-09 12:17:22,523][86122] Updated weights for policy 1, policy_version 1560 (0.0007) +[2023-10-09 12:17:22,618][86121] Updated weights for policy 0, policy_version 1560 (0.0008) +[2023-10-09 12:17:23,397][85186] Fps is (10 sec: 19660.4, 60 sec: 14199.4, 300 sec: 14100.5). Total num frames: 3211264. Throughput: 0: 1791.0, 1: 1775.5. Samples: 807334. Policy #0 lag: (min: 17.0, avg: 29.1, max: 49.0) +[2023-10-09 12:17:23,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.900')] +[2023-10-09 12:17:23,405][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth... +[2023-10-09 12:17:23,405][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth... +[2023-10-09 12:17:23,447][85763] Saving new best policy, reward=9.920! +[2023-10-09 12:17:26,203][86122] Updated weights for policy 1, policy_version 1570 (0.0008) +[2023-10-09 12:17:26,266][86121] Updated weights for policy 0, policy_version 1570 (0.0009) +[2023-10-09 12:17:26,568][86122] Updated weights for policy 1, policy_version 1580 (0.0007) +[2023-10-09 12:17:26,638][86121] Updated weights for policy 0, policy_version 1580 (0.0007) +[2023-10-09 12:17:26,930][86122] Updated weights for policy 1, policy_version 1590 (0.0007) +[2023-10-09 12:17:27,002][86121] Updated weights for policy 0, policy_version 1590 (0.0008) +[2023-10-09 12:17:27,291][86122] Updated weights for policy 1, policy_version 1600 (0.0009) +[2023-10-09 12:17:27,375][86121] Updated weights for policy 0, policy_version 1600 (0.0008) +[2023-10-09 12:17:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14079.2). Total num frames: 3276800. Throughput: 0: 1795.8, 1: 1794.2. Samples: 820010. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 12:17:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:17:28,398][85763] Saving new best policy, reward=9.940! +[2023-10-09 12:17:28,398][85963] Saving new best policy, reward=9.960! +[2023-10-09 12:17:31,072][86122] Updated weights for policy 1, policy_version 1610 (0.0008) +[2023-10-09 12:17:31,141][86121] Updated weights for policy 0, policy_version 1610 (0.0007) +[2023-10-09 12:17:31,436][86122] Updated weights for policy 1, policy_version 1620 (0.0007) +[2023-10-09 12:17:31,504][86121] Updated weights for policy 0, policy_version 1620 (0.0009) +[2023-10-09 12:17:31,800][86122] Updated weights for policy 1, policy_version 1630 (0.0008) +[2023-10-09 12:17:31,869][86121] Updated weights for policy 0, policy_version 1630 (0.0009) +[2023-10-09 12:17:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14058.8). Total num frames: 3342336. Throughput: 0: 1789.6, 1: 1773.8. Samples: 839732. Policy #0 lag: (min: 13.0, avg: 19.4, max: 45.0) +[2023-10-09 12:17:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:17:35,487][86122] Updated weights for policy 1, policy_version 1640 (0.0008) +[2023-10-09 12:17:35,600][86121] Updated weights for policy 0, policy_version 1640 (0.0009) +[2023-10-09 12:17:35,851][86122] Updated weights for policy 1, policy_version 1650 (0.0008) +[2023-10-09 12:17:35,974][86121] Updated weights for policy 0, policy_version 1650 (0.0008) +[2023-10-09 12:17:36,223][86122] Updated weights for policy 1, policy_version 1660 (0.0008) +[2023-10-09 12:17:36,340][86121] Updated weights for policy 0, policy_version 1660 (0.0008) +[2023-10-09 12:17:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14039.2). Total num frames: 3407872. Throughput: 0: 1791.3, 1: 1781.2. Samples: 862370. Policy #0 lag: (min: 4.0, avg: 9.7, max: 36.0) +[2023-10-09 12:17:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:17:39,921][86121] Updated weights for policy 0, policy_version 1670 (0.0007) +[2023-10-09 12:17:39,930][86122] Updated weights for policy 1, policy_version 1670 (0.0008) +[2023-10-09 12:17:40,290][86122] Updated weights for policy 1, policy_version 1680 (0.0009) +[2023-10-09 12:17:40,296][86121] Updated weights for policy 0, policy_version 1680 (0.0008) +[2023-10-09 12:17:40,662][86121] Updated weights for policy 0, policy_version 1690 (0.0009) +[2023-10-09 12:17:40,663][86122] Updated weights for policy 1, policy_version 1690 (0.0008) +[2023-10-09 12:17:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14020.4). Total num frames: 3473408. Throughput: 0: 1793.3, 1: 1782.4. Samples: 872214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:17:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.960')] +[2023-10-09 12:17:43,399][85763] Saving new best policy, reward=9.950! +[2023-10-09 12:17:44,388][86122] Updated weights for policy 1, policy_version 1700 (0.0008) +[2023-10-09 12:17:44,508][86121] Updated weights for policy 0, policy_version 1700 (0.0007) +[2023-10-09 12:17:44,742][86122] Updated weights for policy 1, policy_version 1710 (0.0008) +[2023-10-09 12:17:44,881][86121] Updated weights for policy 0, policy_version 1710 (0.0008) +[2023-10-09 12:17:45,113][86122] Updated weights for policy 1, policy_version 1720 (0.0008) +[2023-10-09 12:17:45,243][86121] Updated weights for policy 0, policy_version 1720 (0.0007) +[2023-10-09 12:17:48,398][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14002.3). Total num frames: 3538944. Throughput: 0: 1806.1, 1: 1789.4. Samples: 894856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:17:48,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 12:17:48,400][85763] Saving new best policy, reward=9.960! +[2023-10-09 12:17:48,400][85963] Saving new best policy, reward=9.970! +[2023-10-09 12:17:48,889][86121] Updated weights for policy 0, policy_version 1730 (0.0008) +[2023-10-09 12:17:48,923][86122] Updated weights for policy 1, policy_version 1730 (0.0008) +[2023-10-09 12:17:49,262][86121] Updated weights for policy 0, policy_version 1740 (0.0009) +[2023-10-09 12:17:49,286][86122] Updated weights for policy 1, policy_version 1740 (0.0010) +[2023-10-09 12:17:49,632][86121] Updated weights for policy 0, policy_version 1750 (0.0007) +[2023-10-09 12:17:49,661][86122] Updated weights for policy 1, policy_version 1750 (0.0008) +[2023-10-09 12:17:49,996][86121] Updated weights for policy 0, policy_version 1760 (0.0007) +[2023-10-09 12:17:50,021][86122] Updated weights for policy 1, policy_version 1760 (0.0007) +[2023-10-09 12:17:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 13984.9). Total num frames: 3604480. Throughput: 0: 1806.3, 1: 1785.4. Samples: 916658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:17:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 12:17:53,410][85963] Saving new best policy, reward=9.980! +[2023-10-09 12:17:53,930][86121] Updated weights for policy 0, policy_version 1770 (0.0010) +[2023-10-09 12:17:54,012][86122] Updated weights for policy 1, policy_version 1770 (0.0009) +[2023-10-09 12:17:54,288][86121] Updated weights for policy 0, policy_version 1780 (0.0008) +[2023-10-09 12:17:54,376][86122] Updated weights for policy 1, policy_version 1780 (0.0009) +[2023-10-09 12:17:54,663][86121] Updated weights for policy 0, policy_version 1790 (0.0007) +[2023-10-09 12:17:54,732][86122] Updated weights for policy 1, policy_version 1790 (0.0008) +[2023-10-09 12:17:58,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 13968.2). Total num frames: 3670016. Throughput: 0: 1803.5, 1: 1782.7. Samples: 926132. Policy #0 lag: (min: 19.0, avg: 38.4, max: 40.0) +[2023-10-09 12:17:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 12:17:58,492][86121] Updated weights for policy 0, policy_version 1800 (0.0008) +[2023-10-09 12:17:58,579][86122] Updated weights for policy 1, policy_version 1800 (0.0008) +[2023-10-09 12:17:58,851][86121] Updated weights for policy 0, policy_version 1810 (0.0008) +[2023-10-09 12:17:58,940][86122] Updated weights for policy 1, policy_version 1810 (0.0008) +[2023-10-09 12:17:59,215][86121] Updated weights for policy 0, policy_version 1820 (0.0008) +[2023-10-09 12:17:59,305][86122] Updated weights for policy 1, policy_version 1820 (0.0011) +[2023-10-09 12:18:03,339][86121] Updated weights for policy 0, policy_version 1830 (0.0008) +[2023-10-09 12:18:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 13952.2). Total num frames: 3735552. Throughput: 0: 1788.6, 1: 1771.4. Samples: 947342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:18:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:18:03,483][86122] Updated weights for policy 1, policy_version 1830 (0.0010) +[2023-10-09 12:18:03,694][86121] Updated weights for policy 0, policy_version 1840 (0.0008) +[2023-10-09 12:18:03,852][86122] Updated weights for policy 1, policy_version 1840 (0.0008) +[2023-10-09 12:18:04,061][86121] Updated weights for policy 0, policy_version 1850 (0.0009) +[2023-10-09 12:18:04,210][86122] Updated weights for policy 1, policy_version 1850 (0.0009) +[2023-10-09 12:18:04,283][85763] Saving new best policy, reward=9.970! +[2023-10-09 12:18:07,951][86122] Updated weights for policy 1, policy_version 1860 (0.0008) +[2023-10-09 12:18:08,040][86121] Updated weights for policy 0, policy_version 1860 (0.0008) +[2023-10-09 12:18:08,307][86122] Updated weights for policy 1, policy_version 1870 (0.0007) +[2023-10-09 12:18:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13936.7). Total num frames: 3801088. Throughput: 0: 1795.9, 1: 1788.8. Samples: 968648. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 12:18:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:18:08,401][86121] Updated weights for policy 0, policy_version 1870 (0.0008) +[2023-10-09 12:18:08,672][86122] Updated weights for policy 1, policy_version 1880 (0.0009) +[2023-10-09 12:18:08,765][86121] Updated weights for policy 0, policy_version 1880 (0.0008) +[2023-10-09 12:18:12,556][86122] Updated weights for policy 1, policy_version 1890 (0.0009) +[2023-10-09 12:18:12,639][86121] Updated weights for policy 0, policy_version 1890 (0.0007) +[2023-10-09 12:18:12,923][86122] Updated weights for policy 1, policy_version 1900 (0.0007) +[2023-10-09 12:18:13,007][86121] Updated weights for policy 0, policy_version 1900 (0.0007) +[2023-10-09 12:18:13,288][86122] Updated weights for policy 1, policy_version 1910 (0.0008) +[2023-10-09 12:18:13,370][86121] Updated weights for policy 0, policy_version 1910 (0.0008) +[2023-10-09 12:18:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 13921.7). Total num frames: 3866624. Throughput: 0: 1761.3, 1: 1754.4. Samples: 978214. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 12:18:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:18:13,650][85963] Saving new best policy, reward=9.990! +[2023-10-09 12:18:13,651][86122] Updated weights for policy 1, policy_version 1920 (0.0009) +[2023-10-09 12:18:13,748][85763] Saving new best policy, reward=9.980! +[2023-10-09 12:18:13,751][86121] Updated weights for policy 0, policy_version 1920 (0.0009) +[2023-10-09 12:18:17,488][86122] Updated weights for policy 1, policy_version 1930 (0.0007) +[2023-10-09 12:18:17,607][86121] Updated weights for policy 0, policy_version 1930 (0.0007) +[2023-10-09 12:18:17,858][86122] Updated weights for policy 1, policy_version 1940 (0.0007) +[2023-10-09 12:18:17,976][86121] Updated weights for policy 0, policy_version 1940 (0.0009) +[2023-10-09 12:18:18,213][86122] Updated weights for policy 1, policy_version 1950 (0.0008) +[2023-10-09 12:18:18,337][86121] Updated weights for policy 0, policy_version 1950 (0.0008) +[2023-10-09 12:18:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14023.2). Total num frames: 3964928. Throughput: 0: 1787.0, 1: 1779.5. Samples: 1000222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:18:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:18:22,001][86122] Updated weights for policy 1, policy_version 1960 (0.0007) +[2023-10-09 12:18:22,171][86121] Updated weights for policy 0, policy_version 1960 (0.0009) +[2023-10-09 12:18:22,380][86122] Updated weights for policy 1, policy_version 1970 (0.0007) +[2023-10-09 12:18:22,537][86121] Updated weights for policy 0, policy_version 1970 (0.0009) +[2023-10-09 12:18:22,736][86122] Updated weights for policy 1, policy_version 1980 (0.0009) +[2023-10-09 12:18:22,904][86121] Updated weights for policy 0, policy_version 1980 (0.0008) +[2023-10-09 12:18:23,397][85186] Fps is (10 sec: 19660.3, 60 sec: 14199.4, 300 sec: 14121.2). Total num frames: 4063232. Throughput: 0: 1757.3, 1: 1742.4. Samples: 1019854. Policy #0 lag: (min: 31.0, avg: 39.1, max: 63.0) +[2023-10-09 12:18:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:18:26,447][86122] Updated weights for policy 1, policy_version 1990 (0.0007) +[2023-10-09 12:18:26,559][86121] Updated weights for policy 0, policy_version 1990 (0.0007) +[2023-10-09 12:18:26,806][86122] Updated weights for policy 1, policy_version 2000 (0.0009) +[2023-10-09 12:18:26,924][86121] Updated weights for policy 0, policy_version 2000 (0.0013) +[2023-10-09 12:18:27,174][86122] Updated weights for policy 1, policy_version 2010 (0.0010) +[2023-10-09 12:18:27,303][86121] Updated weights for policy 0, policy_version 2010 (0.0010) +[2023-10-09 12:18:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14103.9). Total num frames: 4128768. Throughput: 0: 1784.9, 1: 1769.5. Samples: 1032164. Policy #0 lag: (min: 1.0, avg: 9.1, max: 33.0) +[2023-10-09 12:18:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:18:31,178][86121] Updated weights for policy 0, policy_version 2020 (0.0011) +[2023-10-09 12:18:31,304][86122] Updated weights for policy 1, policy_version 2020 (0.0010) +[2023-10-09 12:18:31,552][86121] Updated weights for policy 0, policy_version 2030 (0.0011) +[2023-10-09 12:18:31,664][86122] Updated weights for policy 1, policy_version 2030 (0.0010) +[2023-10-09 12:18:31,921][86121] Updated weights for policy 0, policy_version 2040 (0.0008) +[2023-10-09 12:18:32,031][86122] Updated weights for policy 1, policy_version 2040 (0.0009) +[2023-10-09 12:18:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4194304. Throughput: 0: 1747.9, 1: 1741.8. Samples: 1051892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:18:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:18:35,846][86121] Updated weights for policy 0, policy_version 2050 (0.0009) +[2023-10-09 12:18:36,034][86122] Updated weights for policy 1, policy_version 2050 (0.0008) +[2023-10-09 12:18:36,208][86121] Updated weights for policy 0, policy_version 2060 (0.0008) +[2023-10-09 12:18:36,405][86122] Updated weights for policy 1, policy_version 2060 (0.0008) +[2023-10-09 12:18:36,575][86121] Updated weights for policy 0, policy_version 2070 (0.0010) +[2023-10-09 12:18:36,760][86122] Updated weights for policy 1, policy_version 2070 (0.0008) +[2023-10-09 12:18:36,944][86121] Updated weights for policy 0, policy_version 2080 (0.0008) +[2023-10-09 12:18:37,127][86122] Updated weights for policy 1, policy_version 2080 (0.0007) +[2023-10-09 12:18:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4259840. Throughput: 0: 1732.8, 1: 1729.6. Samples: 1072464. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 12:18:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:18:40,757][86121] Updated weights for policy 0, policy_version 2090 (0.0009) +[2023-10-09 12:18:40,993][86122] Updated weights for policy 1, policy_version 2090 (0.0010) +[2023-10-09 12:18:41,122][86121] Updated weights for policy 0, policy_version 2100 (0.0007) +[2023-10-09 12:18:41,356][86122] Updated weights for policy 1, policy_version 2100 (0.0008) +[2023-10-09 12:18:41,501][86121] Updated weights for policy 0, policy_version 2110 (0.0008) +[2023-10-09 12:18:41,717][86122] Updated weights for policy 1, policy_version 2110 (0.0011) +[2023-10-09 12:18:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4325376. Throughput: 0: 1751.1, 1: 1757.4. Samples: 1084016. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 12:18:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:18:43,399][85963] Saving new best policy, reward=10.000! +[2023-10-09 12:18:45,091][86121] Updated weights for policy 0, policy_version 2120 (0.0010) +[2023-10-09 12:18:45,469][86121] Updated weights for policy 0, policy_version 2130 (0.0009) +[2023-10-09 12:18:45,551][86122] Updated weights for policy 1, policy_version 2120 (0.0008) +[2023-10-09 12:18:45,821][86121] Updated weights for policy 0, policy_version 2140 (0.0009) +[2023-10-09 12:18:45,917][86122] Updated weights for policy 1, policy_version 2130 (0.0009) +[2023-10-09 12:18:46,284][86122] Updated weights for policy 1, policy_version 2140 (0.0007) +[2023-10-09 12:18:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4390912. Throughput: 0: 1753.5, 1: 1748.8. Samples: 1104948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:18:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:18:49,769][86121] Updated weights for policy 0, policy_version 2150 (0.0009) +[2023-10-09 12:18:50,036][86122] Updated weights for policy 1, policy_version 2150 (0.0008) +[2023-10-09 12:18:50,143][86121] Updated weights for policy 0, policy_version 2160 (0.0009) +[2023-10-09 12:18:50,397][86122] Updated weights for policy 1, policy_version 2160 (0.0009) +[2023-10-09 12:18:50,509][86121] Updated weights for policy 0, policy_version 2170 (0.0007) +[2023-10-09 12:18:50,767][86122] Updated weights for policy 1, policy_version 2170 (0.0009) +[2023-10-09 12:18:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4456448. Throughput: 0: 1770.6, 1: 1759.6. Samples: 1127504. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 12:18:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:18:54,185][86121] Updated weights for policy 0, policy_version 2180 (0.0009) +[2023-10-09 12:18:54,343][86122] Updated weights for policy 1, policy_version 2180 (0.0008) +[2023-10-09 12:18:54,558][86121] Updated weights for policy 0, policy_version 2190 (0.0007) +[2023-10-09 12:18:54,706][86122] Updated weights for policy 1, policy_version 2190 (0.0008) +[2023-10-09 12:18:54,920][86121] Updated weights for policy 0, policy_version 2200 (0.0007) +[2023-10-09 12:18:55,076][86122] Updated weights for policy 1, policy_version 2200 (0.0008) +[2023-10-09 12:18:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4521984. Throughput: 0: 1771.9, 1: 1768.1. Samples: 1137516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:18:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:18:58,513][86121] Updated weights for policy 0, policy_version 2210 (0.0007) +[2023-10-09 12:18:58,712][86122] Updated weights for policy 1, policy_version 2210 (0.0007) +[2023-10-09 12:18:58,880][86121] Updated weights for policy 0, policy_version 2220 (0.0007) +[2023-10-09 12:18:59,085][86122] Updated weights for policy 1, policy_version 2220 (0.0008) +[2023-10-09 12:18:59,247][86121] Updated weights for policy 0, policy_version 2230 (0.0008) +[2023-10-09 12:18:59,455][86122] Updated weights for policy 1, policy_version 2230 (0.0007) +[2023-10-09 12:18:59,615][86121] Updated weights for policy 0, policy_version 2240 (0.0007) +[2023-10-09 12:18:59,817][86122] Updated weights for policy 1, policy_version 2240 (0.0009) +[2023-10-09 12:19:03,253][86121] Updated weights for policy 0, policy_version 2250 (0.0007) +[2023-10-09 12:19:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4587520. Throughput: 0: 1784.6, 1: 1773.2. Samples: 1160320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:19:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:19:03,511][86122] Updated weights for policy 1, policy_version 2250 (0.0007) +[2023-10-09 12:19:03,625][86121] Updated weights for policy 0, policy_version 2260 (0.0008) +[2023-10-09 12:19:03,875][86122] Updated weights for policy 1, policy_version 2260 (0.0010) +[2023-10-09 12:19:03,987][86121] Updated weights for policy 0, policy_version 2270 (0.0009) +[2023-10-09 12:19:04,247][86122] Updated weights for policy 1, policy_version 2270 (0.0008) +[2023-10-09 12:19:07,729][86121] Updated weights for policy 0, policy_version 2280 (0.0008) +[2023-10-09 12:19:07,943][86122] Updated weights for policy 1, policy_version 2280 (0.0008) +[2023-10-09 12:19:08,107][86121] Updated weights for policy 0, policy_version 2290 (0.0010) +[2023-10-09 12:19:08,306][86122] Updated weights for policy 1, policy_version 2290 (0.0007) +[2023-10-09 12:19:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 4653056. Throughput: 0: 1807.1, 1: 1805.0. Samples: 1182398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:19:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:19:08,471][86121] Updated weights for policy 0, policy_version 2300 (0.0009) +[2023-10-09 12:19:08,670][86122] Updated weights for policy 1, policy_version 2300 (0.0008) +[2023-10-09 12:19:12,287][86122] Updated weights for policy 1, policy_version 2310 (0.0007) +[2023-10-09 12:19:12,370][86121] Updated weights for policy 0, policy_version 2310 (0.0008) +[2023-10-09 12:19:12,649][86122] Updated weights for policy 1, policy_version 2320 (0.0007) +[2023-10-09 12:19:12,737][86121] Updated weights for policy 0, policy_version 2320 (0.0007) +[2023-10-09 12:19:13,018][86122] Updated weights for policy 1, policy_version 2330 (0.0007) +[2023-10-09 12:19:13,100][86121] Updated weights for policy 0, policy_version 2330 (0.0007) +[2023-10-09 12:19:13,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 4784128. Throughput: 0: 1785.5, 1: 1783.3. Samples: 1192760. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 12:19:13,399][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 12:19:16,631][86121] Updated weights for policy 0, policy_version 2340 (0.0009) +[2023-10-09 12:19:16,746][86122] Updated weights for policy 1, policy_version 2340 (0.0008) +[2023-10-09 12:19:16,987][86121] Updated weights for policy 0, policy_version 2350 (0.0008) +[2023-10-09 12:19:17,112][86122] Updated weights for policy 1, policy_version 2350 (0.0007) +[2023-10-09 12:19:17,359][86121] Updated weights for policy 0, policy_version 2360 (0.0007) +[2023-10-09 12:19:17,467][86122] Updated weights for policy 1, policy_version 2360 (0.0009) +[2023-10-09 12:19:18,397][85186] Fps is (10 sec: 19660.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 4849664. Throughput: 0: 1802.6, 1: 1814.8. Samples: 1214676. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 12:19:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:19:21,139][86121] Updated weights for policy 0, policy_version 2370 (0.0008) +[2023-10-09 12:19:21,357][86122] Updated weights for policy 1, policy_version 2370 (0.0010) +[2023-10-09 12:19:21,506][86121] Updated weights for policy 0, policy_version 2380 (0.0009) +[2023-10-09 12:19:21,716][86122] Updated weights for policy 1, policy_version 2380 (0.0010) +[2023-10-09 12:19:21,869][86121] Updated weights for policy 0, policy_version 2390 (0.0009) +[2023-10-09 12:19:22,086][86122] Updated weights for policy 1, policy_version 2390 (0.0009) +[2023-10-09 12:19:22,245][86121] Updated weights for policy 0, policy_version 2400 (0.0009) +[2023-10-09 12:19:22,461][86122] Updated weights for policy 1, policy_version 2400 (0.0009) +[2023-10-09 12:19:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 4915200. Throughput: 0: 1805.0, 1: 1807.0. Samples: 1235002. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) +[2023-10-09 12:19:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:19:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000002400_2457600.pth... +[2023-10-09 12:19:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000002400_2457600.pth... +[2023-10-09 12:19:23,439][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000000736_753664.pth +[2023-10-09 12:19:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000000704_720896.pth +[2023-10-09 12:19:25,881][86121] Updated weights for policy 0, policy_version 2410 (0.0011) +[2023-10-09 12:19:26,246][86121] Updated weights for policy 0, policy_version 2420 (0.0009) +[2023-10-09 12:19:26,399][86122] Updated weights for policy 1, policy_version 2410 (0.0008) +[2023-10-09 12:19:26,613][86121] Updated weights for policy 0, policy_version 2430 (0.0009) +[2023-10-09 12:19:26,761][86122] Updated weights for policy 1, policy_version 2420 (0.0008) +[2023-10-09 12:19:27,120][86122] Updated weights for policy 1, policy_version 2430 (0.0007) +[2023-10-09 12:19:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 4980736. Throughput: 0: 1814.4, 1: 1812.6. Samples: 1247232. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) +[2023-10-09 12:19:28,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:19:30,289][86121] Updated weights for policy 0, policy_version 2440 (0.0008) +[2023-10-09 12:19:30,660][86121] Updated weights for policy 0, policy_version 2450 (0.0007) +[2023-10-09 12:19:30,981][86122] Updated weights for policy 1, policy_version 2440 (0.0008) +[2023-10-09 12:19:31,022][86121] Updated weights for policy 0, policy_version 2460 (0.0008) +[2023-10-09 12:19:31,345][86122] Updated weights for policy 1, policy_version 2450 (0.0009) +[2023-10-09 12:19:31,712][86122] Updated weights for policy 1, policy_version 2460 (0.0008) +[2023-10-09 12:19:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5046272. Throughput: 0: 1807.5, 1: 1802.5. Samples: 1267398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:19:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:19:34,664][86121] Updated weights for policy 0, policy_version 2470 (0.0010) +[2023-10-09 12:19:35,030][86121] Updated weights for policy 0, policy_version 2480 (0.0011) +[2023-10-09 12:19:35,356][86122] Updated weights for policy 1, policy_version 2470 (0.0008) +[2023-10-09 12:19:35,396][86121] Updated weights for policy 0, policy_version 2490 (0.0008) +[2023-10-09 12:19:35,725][86122] Updated weights for policy 1, policy_version 2480 (0.0009) +[2023-10-09 12:19:36,099][86122] Updated weights for policy 1, policy_version 2490 (0.0008) +[2023-10-09 12:19:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 5111808. Throughput: 0: 1812.9, 1: 1801.1. Samples: 1290132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:19:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:19:39,237][86121] Updated weights for policy 0, policy_version 2500 (0.0010) +[2023-10-09 12:19:39,614][86121] Updated weights for policy 0, policy_version 2510 (0.0008) +[2023-10-09 12:19:39,857][86122] Updated weights for policy 1, policy_version 2500 (0.0009) +[2023-10-09 12:19:39,995][86121] Updated weights for policy 0, policy_version 2520 (0.0008) +[2023-10-09 12:19:40,217][86122] Updated weights for policy 1, policy_version 2510 (0.0008) +[2023-10-09 12:19:40,577][86122] Updated weights for policy 1, policy_version 2520 (0.0009) +[2023-10-09 12:19:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 5177344. Throughput: 0: 1815.9, 1: 1795.9. Samples: 1300046. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 12:19:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:19:43,517][86121] Updated weights for policy 0, policy_version 2530 (0.0008) +[2023-10-09 12:19:43,891][86121] Updated weights for policy 0, policy_version 2540 (0.0009) +[2023-10-09 12:19:44,009][86122] Updated weights for policy 1, policy_version 2530 (0.0007) +[2023-10-09 12:19:44,263][86121] Updated weights for policy 0, policy_version 2550 (0.0007) +[2023-10-09 12:19:44,382][86122] Updated weights for policy 1, policy_version 2540 (0.0007) +[2023-10-09 12:19:44,624][86121] Updated weights for policy 0, policy_version 2560 (0.0007) +[2023-10-09 12:19:44,739][86122] Updated weights for policy 1, policy_version 2550 (0.0010) +[2023-10-09 12:19:45,113][86122] Updated weights for policy 1, policy_version 2560 (0.0009) +[2023-10-09 12:19:48,177][86121] Updated weights for policy 0, policy_version 2570 (0.0009) +[2023-10-09 12:19:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 5242880. Throughput: 0: 1821.0, 1: 1803.1. Samples: 1323404. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) +[2023-10-09 12:19:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:19:48,546][86121] Updated weights for policy 0, policy_version 2580 (0.0008) +[2023-10-09 12:19:48,643][86122] Updated weights for policy 1, policy_version 2570 (0.0008) +[2023-10-09 12:19:48,912][86121] Updated weights for policy 0, policy_version 2590 (0.0009) +[2023-10-09 12:19:49,012][86122] Updated weights for policy 1, policy_version 2580 (0.0010) +[2023-10-09 12:19:49,383][86122] Updated weights for policy 1, policy_version 2590 (0.0009) +[2023-10-09 12:19:52,742][86121] Updated weights for policy 0, policy_version 2600 (0.0010) +[2023-10-09 12:19:53,117][86121] Updated weights for policy 0, policy_version 2610 (0.0008) +[2023-10-09 12:19:53,134][86122] Updated weights for policy 1, policy_version 2600 (0.0007) +[2023-10-09 12:19:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 5308416. Throughput: 0: 1817.4, 1: 1809.7. Samples: 1345618. Policy #0 lag: (min: 31.0, avg: 32.3, max: 56.0) +[2023-10-09 12:19:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:19:53,479][86121] Updated weights for policy 0, policy_version 2620 (0.0007) +[2023-10-09 12:19:53,507][86122] Updated weights for policy 1, policy_version 2610 (0.0007) +[2023-10-09 12:19:53,875][86122] Updated weights for policy 1, policy_version 2620 (0.0008) +[2023-10-09 12:19:57,239][86121] Updated weights for policy 0, policy_version 2630 (0.0009) +[2023-10-09 12:19:57,615][86121] Updated weights for policy 0, policy_version 2640 (0.0009) +[2023-10-09 12:19:57,638][86122] Updated weights for policy 1, policy_version 2630 (0.0008) +[2023-10-09 12:19:57,986][86121] Updated weights for policy 0, policy_version 2650 (0.0008) +[2023-10-09 12:19:58,000][86122] Updated weights for policy 1, policy_version 2640 (0.0009) +[2023-10-09 12:19:58,356][86122] Updated weights for policy 1, policy_version 2650 (0.0009) +[2023-10-09 12:19:58,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14329.0). Total num frames: 5406720. Throughput: 0: 1820.7, 1: 1800.8. Samples: 1355724. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 12:19:58,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 12:20:01,717][86121] Updated weights for policy 0, policy_version 2660 (0.0009) +[2023-10-09 12:20:02,081][86121] Updated weights for policy 0, policy_version 2670 (0.0010) +[2023-10-09 12:20:02,223][86122] Updated weights for policy 1, policy_version 2660 (0.0007) +[2023-10-09 12:20:02,445][86121] Updated weights for policy 0, policy_version 2680 (0.0008) +[2023-10-09 12:20:02,592][86122] Updated weights for policy 1, policy_version 2670 (0.0008) +[2023-10-09 12:20:02,961][86122] Updated weights for policy 1, policy_version 2680 (0.0007) +[2023-10-09 12:20:03,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 5505024. Throughput: 0: 1827.8, 1: 1800.2. Samples: 1377938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:20:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 12:20:06,246][86121] Updated weights for policy 0, policy_version 2690 (0.0008) +[2023-10-09 12:20:06,606][86121] Updated weights for policy 0, policy_version 2700 (0.0009) +[2023-10-09 12:20:06,851][86122] Updated weights for policy 1, policy_version 2690 (0.0007) +[2023-10-09 12:20:06,972][86121] Updated weights for policy 0, policy_version 2710 (0.0009) +[2023-10-09 12:20:07,214][86122] Updated weights for policy 1, policy_version 2700 (0.0008) +[2023-10-09 12:20:07,338][86121] Updated weights for policy 0, policy_version 2720 (0.0008) +[2023-10-09 12:20:07,579][86122] Updated weights for policy 1, policy_version 2710 (0.0008) +[2023-10-09 12:20:07,936][86122] Updated weights for policy 1, policy_version 2720 (0.0010) +[2023-10-09 12:20:08,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 5570560. Throughput: 0: 1817.8, 1: 1797.1. Samples: 1397670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:20:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:20:11,373][86121] Updated weights for policy 0, policy_version 2730 (0.0009) +[2023-10-09 12:20:11,752][86121] Updated weights for policy 0, policy_version 2740 (0.0009) +[2023-10-09 12:20:11,919][86122] Updated weights for policy 1, policy_version 2730 (0.0007) +[2023-10-09 12:20:12,114][86121] Updated weights for policy 0, policy_version 2750 (0.0009) +[2023-10-09 12:20:12,288][86122] Updated weights for policy 1, policy_version 2740 (0.0008) +[2023-10-09 12:20:12,654][86122] Updated weights for policy 1, policy_version 2750 (0.0007) +[2023-10-09 12:20:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5636096. Throughput: 0: 1817.6, 1: 1786.5. Samples: 1409416. Policy #0 lag: (min: 15.0, avg: 15.4, max: 28.0) +[2023-10-09 12:20:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:20:15,710][86121] Updated weights for policy 0, policy_version 2760 (0.0008) +[2023-10-09 12:20:16,081][86121] Updated weights for policy 0, policy_version 2770 (0.0011) +[2023-10-09 12:20:16,447][86121] Updated weights for policy 0, policy_version 2780 (0.0010) +[2023-10-09 12:20:16,496][86122] Updated weights for policy 1, policy_version 2760 (0.0008) +[2023-10-09 12:20:16,866][86122] Updated weights for policy 1, policy_version 2770 (0.0009) +[2023-10-09 12:20:17,233][86122] Updated weights for policy 1, policy_version 2780 (0.0010) +[2023-10-09 12:20:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5701632. Throughput: 0: 1804.1, 1: 1796.1. Samples: 1429408. Policy #0 lag: (min: 25.0, avg: 32.7, max: 57.0) +[2023-10-09 12:20:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:20:20,734][86121] Updated weights for policy 0, policy_version 2790 (0.0008) +[2023-10-09 12:20:21,111][86121] Updated weights for policy 0, policy_version 2800 (0.0009) +[2023-10-09 12:20:21,368][86122] Updated weights for policy 1, policy_version 2790 (0.0009) +[2023-10-09 12:20:21,478][86121] Updated weights for policy 0, policy_version 2810 (0.0007) +[2023-10-09 12:20:21,735][86122] Updated weights for policy 1, policy_version 2800 (0.0008) +[2023-10-09 12:20:22,102][86122] Updated weights for policy 1, policy_version 2810 (0.0008) +[2023-10-09 12:20:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 5767168. Throughput: 0: 1776.5, 1: 1761.5. Samples: 1449340. Policy #0 lag: (min: 19.0, avg: 20.4, max: 45.0) +[2023-10-09 12:20:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 12:20:25,142][86121] Updated weights for policy 0, policy_version 2820 (0.0008) +[2023-10-09 12:20:25,511][86121] Updated weights for policy 0, policy_version 2830 (0.0010) +[2023-10-09 12:20:25,882][86121] Updated weights for policy 0, policy_version 2840 (0.0009) +[2023-10-09 12:20:25,932][86122] Updated weights for policy 1, policy_version 2820 (0.0010) +[2023-10-09 12:20:26,298][86122] Updated weights for policy 1, policy_version 2830 (0.0009) +[2023-10-09 12:20:26,673][86122] Updated weights for policy 1, policy_version 2840 (0.0009) +[2023-10-09 12:20:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 5832704. Throughput: 0: 1786.4, 1: 1788.0. Samples: 1460896. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:20:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 12:20:29,545][86121] Updated weights for policy 0, policy_version 2850 (0.0007) +[2023-10-09 12:20:29,919][86121] Updated weights for policy 0, policy_version 2860 (0.0008) +[2023-10-09 12:20:30,254][86122] Updated weights for policy 1, policy_version 2850 (0.0008) +[2023-10-09 12:20:30,288][86121] Updated weights for policy 0, policy_version 2870 (0.0009) +[2023-10-09 12:20:30,621][86122] Updated weights for policy 1, policy_version 2860 (0.0009) +[2023-10-09 12:20:30,650][86121] Updated weights for policy 0, policy_version 2880 (0.0009) +[2023-10-09 12:20:30,986][86122] Updated weights for policy 1, policy_version 2870 (0.0009) +[2023-10-09 12:20:31,353][86122] Updated weights for policy 1, policy_version 2880 (0.0007) +[2023-10-09 12:20:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14329.1). Total num frames: 5898240. Throughput: 0: 1768.7, 1: 1754.4. Samples: 1481946. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:20:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:20:34,395][86121] Updated weights for policy 0, policy_version 2890 (0.0007) +[2023-10-09 12:20:34,767][86121] Updated weights for policy 0, policy_version 2900 (0.0008) +[2023-10-09 12:20:34,997][86122] Updated weights for policy 1, policy_version 2890 (0.0007) +[2023-10-09 12:20:35,128][86121] Updated weights for policy 0, policy_version 2910 (0.0009) +[2023-10-09 12:20:35,363][86122] Updated weights for policy 1, policy_version 2900 (0.0009) +[2023-10-09 12:20:35,728][86122] Updated weights for policy 1, policy_version 2910 (0.0010) +[2023-10-09 12:20:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 5963776. Throughput: 0: 1781.6, 1: 1755.2. Samples: 1504778. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 12:20:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:20:38,884][86121] Updated weights for policy 0, policy_version 2920 (0.0008) +[2023-10-09 12:20:39,269][86121] Updated weights for policy 0, policy_version 2930 (0.0008) +[2023-10-09 12:20:39,632][86121] Updated weights for policy 0, policy_version 2940 (0.0009) +[2023-10-09 12:20:39,659][86122] Updated weights for policy 1, policy_version 2920 (0.0008) +[2023-10-09 12:20:40,027][86122] Updated weights for policy 1, policy_version 2930 (0.0009) +[2023-10-09 12:20:40,391][86122] Updated weights for policy 1, policy_version 2940 (0.0008) +[2023-10-09 12:20:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 6029312. Throughput: 0: 1769.2, 1: 1754.8. Samples: 1514304. Policy #0 lag: (min: 3.0, avg: 4.6, max: 29.0) +[2023-10-09 12:20:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:20:43,541][86121] Updated weights for policy 0, policy_version 2950 (0.0010) +[2023-10-09 12:20:43,907][86121] Updated weights for policy 0, policy_version 2960 (0.0010) +[2023-10-09 12:20:44,281][86121] Updated weights for policy 0, policy_version 2970 (0.0010) +[2023-10-09 12:20:44,469][86122] Updated weights for policy 1, policy_version 2950 (0.0009) +[2023-10-09 12:20:44,826][86122] Updated weights for policy 1, policy_version 2960 (0.0010) +[2023-10-09 12:20:45,189][86122] Updated weights for policy 1, policy_version 2970 (0.0010) +[2023-10-09 12:20:48,126][86121] Updated weights for policy 0, policy_version 2980 (0.0009) +[2023-10-09 12:20:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 6094848. Throughput: 0: 1761.1, 1: 1739.8. Samples: 1535478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:20:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:20:48,497][86121] Updated weights for policy 0, policy_version 2990 (0.0008) +[2023-10-09 12:20:48,865][86121] Updated weights for policy 0, policy_version 3000 (0.0009) +[2023-10-09 12:20:49,132][86122] Updated weights for policy 1, policy_version 2980 (0.0008) +[2023-10-09 12:20:49,505][86122] Updated weights for policy 1, policy_version 2990 (0.0007) +[2023-10-09 12:20:49,872][86122] Updated weights for policy 1, policy_version 3000 (0.0008) +[2023-10-09 12:20:52,601][86121] Updated weights for policy 0, policy_version 3010 (0.0009) +[2023-10-09 12:20:52,964][86121] Updated weights for policy 0, policy_version 3020 (0.0008) +[2023-10-09 12:20:53,347][86121] Updated weights for policy 0, policy_version 3030 (0.0008) +[2023-10-09 12:20:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 6160384. Throughput: 0: 1775.8, 1: 1777.1. Samples: 1557552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:20:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:20:53,468][86122] Updated weights for policy 1, policy_version 3010 (0.0008) +[2023-10-09 12:20:53,717][86121] Updated weights for policy 0, policy_version 3040 (0.0008) +[2023-10-09 12:20:53,836][86122] Updated weights for policy 1, policy_version 3020 (0.0008) +[2023-10-09 12:20:54,199][86122] Updated weights for policy 1, policy_version 3030 (0.0010) +[2023-10-09 12:20:54,561][86122] Updated weights for policy 1, policy_version 3040 (0.0010) +[2023-10-09 12:20:57,541][86121] Updated weights for policy 0, policy_version 3050 (0.0007) +[2023-10-09 12:20:57,910][86121] Updated weights for policy 0, policy_version 3060 (0.0007) +[2023-10-09 12:20:58,273][86122] Updated weights for policy 1, policy_version 3050 (0.0009) +[2023-10-09 12:20:58,273][86121] Updated weights for policy 0, policy_version 3070 (0.0007) +[2023-10-09 12:20:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 6258688. Throughput: 0: 1759.9, 1: 1760.3. Samples: 1567826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:20:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:20:58,639][86122] Updated weights for policy 1, policy_version 3060 (0.0010) +[2023-10-09 12:20:58,998][86122] Updated weights for policy 1, policy_version 3070 (0.0009) +[2023-10-09 12:21:02,040][86121] Updated weights for policy 0, policy_version 3080 (0.0007) +[2023-10-09 12:21:02,413][86121] Updated weights for policy 0, policy_version 3090 (0.0008) +[2023-10-09 12:21:02,600][86122] Updated weights for policy 1, policy_version 3080 (0.0009) +[2023-10-09 12:21:02,785][86121] Updated weights for policy 0, policy_version 3100 (0.0008) +[2023-10-09 12:21:02,958][86122] Updated weights for policy 1, policy_version 3090 (0.0007) +[2023-10-09 12:21:03,323][86122] Updated weights for policy 1, policy_version 3100 (0.0008) +[2023-10-09 12:21:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 13653.4, 300 sec: 14329.1). Total num frames: 6324224. Throughput: 0: 1783.1, 1: 1791.0. Samples: 1590242. Policy #0 lag: (min: 3.0, avg: 12.5, max: 35.0) +[2023-10-09 12:21:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:21:06,520][86121] Updated weights for policy 0, policy_version 3110 (0.0007) +[2023-10-09 12:21:06,892][86121] Updated weights for policy 0, policy_version 3120 (0.0009) +[2023-10-09 12:21:06,939][86122] Updated weights for policy 1, policy_version 3110 (0.0007) +[2023-10-09 12:21:07,268][86121] Updated weights for policy 0, policy_version 3130 (0.0008) +[2023-10-09 12:21:07,309][86122] Updated weights for policy 1, policy_version 3120 (0.0009) +[2023-10-09 12:21:07,676][86122] Updated weights for policy 1, policy_version 3130 (0.0008) +[2023-10-09 12:21:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 6422528. Throughput: 0: 1777.5, 1: 1802.1. Samples: 1610422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:21:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:21:11,083][86121] Updated weights for policy 0, policy_version 3140 (0.0008) +[2023-10-09 12:21:11,308][86122] Updated weights for policy 1, policy_version 3140 (0.0009) +[2023-10-09 12:21:11,446][86121] Updated weights for policy 0, policy_version 3150 (0.0007) +[2023-10-09 12:21:11,676][86122] Updated weights for policy 1, policy_version 3150 (0.0009) +[2023-10-09 12:21:11,820][86121] Updated weights for policy 0, policy_version 3160 (0.0007) +[2023-10-09 12:21:12,045][86122] Updated weights for policy 1, policy_version 3160 (0.0007) +[2023-10-09 12:21:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 6488064. Throughput: 0: 1799.2, 1: 1807.2. Samples: 1623184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:21:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:21:15,653][86121] Updated weights for policy 0, policy_version 3170 (0.0009) +[2023-10-09 12:21:15,785][86122] Updated weights for policy 1, policy_version 3170 (0.0008) +[2023-10-09 12:21:16,016][86121] Updated weights for policy 0, policy_version 3180 (0.0008) +[2023-10-09 12:21:16,145][86122] Updated weights for policy 1, policy_version 3180 (0.0008) +[2023-10-09 12:21:16,386][86121] Updated weights for policy 0, policy_version 3190 (0.0009) +[2023-10-09 12:21:16,507][86122] Updated weights for policy 1, policy_version 3190 (0.0008) +[2023-10-09 12:21:16,746][86121] Updated weights for policy 0, policy_version 3200 (0.0007) +[2023-10-09 12:21:16,870][86122] Updated weights for policy 1, policy_version 3200 (0.0008) +[2023-10-09 12:21:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 6553600. Throughput: 0: 1772.1, 1: 1804.0. Samples: 1642872. Policy #0 lag: (min: 29.0, avg: 29.5, max: 40.0) +[2023-10-09 12:21:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:20,327][86121] Updated weights for policy 0, policy_version 3210 (0.0008) +[2023-10-09 12:21:20,442][86122] Updated weights for policy 1, policy_version 3210 (0.0009) +[2023-10-09 12:21:20,691][86121] Updated weights for policy 0, policy_version 3220 (0.0008) +[2023-10-09 12:21:20,807][86122] Updated weights for policy 1, policy_version 3220 (0.0007) +[2023-10-09 12:21:21,047][86121] Updated weights for policy 0, policy_version 3230 (0.0010) +[2023-10-09 12:21:21,170][86122] Updated weights for policy 1, policy_version 3230 (0.0008) +[2023-10-09 12:21:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 6619136. Throughput: 0: 1774.8, 1: 1801.4. Samples: 1665706. Policy #0 lag: (min: 29.0, avg: 29.5, max: 40.0) +[2023-10-09 12:21:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000003232_3309568.pth... +[2023-10-09 12:21:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000003232_3309568.pth... +[2023-10-09 12:21:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000001568_1605632.pth +[2023-10-09 12:21:23,448][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000001568_1605632.pth +[2023-10-09 12:21:24,889][86121] Updated weights for policy 0, policy_version 3240 (0.0008) +[2023-10-09 12:21:25,046][86122] Updated weights for policy 1, policy_version 3240 (0.0009) +[2023-10-09 12:21:25,255][86121] Updated weights for policy 0, policy_version 3250 (0.0007) +[2023-10-09 12:21:25,418][86122] Updated weights for policy 1, policy_version 3250 (0.0009) +[2023-10-09 12:21:25,624][86121] Updated weights for policy 0, policy_version 3260 (0.0008) +[2023-10-09 12:21:25,787][86122] Updated weights for policy 1, policy_version 3260 (0.0008) +[2023-10-09 12:21:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 6684672. Throughput: 0: 1776.3, 1: 1809.0. Samples: 1675642. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) +[2023-10-09 12:21:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:29,231][86121] Updated weights for policy 0, policy_version 3270 (0.0009) +[2023-10-09 12:21:29,491][86122] Updated weights for policy 1, policy_version 3270 (0.0009) +[2023-10-09 12:21:29,610][86121] Updated weights for policy 0, policy_version 3280 (0.0007) +[2023-10-09 12:21:29,857][86122] Updated weights for policy 1, policy_version 3280 (0.0010) +[2023-10-09 12:21:29,967][86121] Updated weights for policy 0, policy_version 3290 (0.0010) +[2023-10-09 12:21:30,217][86122] Updated weights for policy 1, policy_version 3290 (0.0009) +[2023-10-09 12:21:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 6750208. Throughput: 0: 1792.5, 1: 1822.2. Samples: 1698138. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) +[2023-10-09 12:21:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:33,619][86121] Updated weights for policy 0, policy_version 3300 (0.0010) +[2023-10-09 12:21:33,945][86122] Updated weights for policy 1, policy_version 3300 (0.0009) +[2023-10-09 12:21:33,986][86121] Updated weights for policy 0, policy_version 3310 (0.0008) +[2023-10-09 12:21:34,307][86122] Updated weights for policy 1, policy_version 3310 (0.0008) +[2023-10-09 12:21:34,349][86121] Updated weights for policy 0, policy_version 3320 (0.0007) +[2023-10-09 12:21:34,671][86122] Updated weights for policy 1, policy_version 3320 (0.0007) +[2023-10-09 12:21:38,035][86121] Updated weights for policy 0, policy_version 3330 (0.0009) +[2023-10-09 12:21:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 6815744. Throughput: 0: 1811.7, 1: 1824.5. Samples: 1721184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:21:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:38,404][86121] Updated weights for policy 0, policy_version 3340 (0.0009) +[2023-10-09 12:21:38,483][86122] Updated weights for policy 1, policy_version 3330 (0.0008) +[2023-10-09 12:21:38,774][86121] Updated weights for policy 0, policy_version 3350 (0.0008) +[2023-10-09 12:21:38,847][86122] Updated weights for policy 1, policy_version 3340 (0.0007) +[2023-10-09 12:21:39,142][86121] Updated weights for policy 0, policy_version 3360 (0.0009) +[2023-10-09 12:21:39,208][86122] Updated weights for policy 1, policy_version 3350 (0.0008) +[2023-10-09 12:21:39,578][86122] Updated weights for policy 1, policy_version 3360 (0.0008) +[2023-10-09 12:21:42,912][86121] Updated weights for policy 0, policy_version 3370 (0.0007) +[2023-10-09 12:21:43,174][86122] Updated weights for policy 1, policy_version 3370 (0.0008) +[2023-10-09 12:21:43,276][86121] Updated weights for policy 0, policy_version 3380 (0.0007) +[2023-10-09 12:21:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 6881280. Throughput: 0: 1804.3, 1: 1821.3. Samples: 1730982. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:21:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:43,541][86122] Updated weights for policy 1, policy_version 3380 (0.0008) +[2023-10-09 12:21:43,639][86121] Updated weights for policy 0, policy_version 3390 (0.0007) +[2023-10-09 12:21:43,900][86122] Updated weights for policy 1, policy_version 3390 (0.0010) +[2023-10-09 12:21:47,358][86121] Updated weights for policy 0, policy_version 3400 (0.0008) +[2023-10-09 12:21:47,449][86122] Updated weights for policy 1, policy_version 3400 (0.0007) +[2023-10-09 12:21:47,725][86121] Updated weights for policy 0, policy_version 3410 (0.0007) +[2023-10-09 12:21:47,804][86122] Updated weights for policy 1, policy_version 3410 (0.0007) +[2023-10-09 12:21:48,086][86121] Updated weights for policy 0, policy_version 3420 (0.0007) +[2023-10-09 12:21:48,171][86122] Updated weights for policy 1, policy_version 3420 (0.0009) +[2023-10-09 12:21:48,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 7012352. Throughput: 0: 1812.8, 1: 1819.8. Samples: 1753710. Policy #0 lag: (min: 31.0, avg: 37.1, max: 63.0) +[2023-10-09 12:21:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:51,809][86121] Updated weights for policy 0, policy_version 3430 (0.0009) +[2023-10-09 12:21:51,976][86122] Updated weights for policy 1, policy_version 3430 (0.0008) +[2023-10-09 12:21:52,173][86121] Updated weights for policy 0, policy_version 3440 (0.0007) +[2023-10-09 12:21:52,330][86122] Updated weights for policy 1, policy_version 3440 (0.0007) +[2023-10-09 12:21:52,540][86121] Updated weights for policy 0, policy_version 3450 (0.0007) +[2023-10-09 12:21:52,703][86122] Updated weights for policy 1, policy_version 3450 (0.0009) +[2023-10-09 12:21:53,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 7077888. Throughput: 0: 1807.7, 1: 1817.1. Samples: 1773536. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:21:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:21:56,406][86121] Updated weights for policy 0, policy_version 3460 (0.0008) +[2023-10-09 12:21:56,511][86122] Updated weights for policy 1, policy_version 3460 (0.0009) +[2023-10-09 12:21:56,771][86121] Updated weights for policy 0, policy_version 3470 (0.0009) +[2023-10-09 12:21:56,885][86122] Updated weights for policy 1, policy_version 3470 (0.0008) +[2023-10-09 12:21:57,147][86121] Updated weights for policy 0, policy_version 3480 (0.0008) +[2023-10-09 12:21:57,251][86122] Updated weights for policy 1, policy_version 3480 (0.0008) +[2023-10-09 12:21:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7143424. Throughput: 0: 1806.6, 1: 1814.9. Samples: 1786150. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:21:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:21:58,399][85763] Saving new best policy, reward=10.000! +[2023-10-09 12:22:00,955][86122] Updated weights for policy 1, policy_version 3490 (0.0008) +[2023-10-09 12:22:01,105][86121] Updated weights for policy 0, policy_version 3490 (0.0008) +[2023-10-09 12:22:01,324][86122] Updated weights for policy 1, policy_version 3500 (0.0007) +[2023-10-09 12:22:01,464][86121] Updated weights for policy 0, policy_version 3500 (0.0010) +[2023-10-09 12:22:01,689][86122] Updated weights for policy 1, policy_version 3510 (0.0009) +[2023-10-09 12:22:01,831][86121] Updated weights for policy 0, policy_version 3510 (0.0009) +[2023-10-09 12:22:02,059][86122] Updated weights for policy 1, policy_version 3520 (0.0008) +[2023-10-09 12:22:02,199][86121] Updated weights for policy 0, policy_version 3520 (0.0008) +[2023-10-09 12:22:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 7208960. Throughput: 0: 1809.1, 1: 1820.4. Samples: 1806200. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) +[2023-10-09 12:22:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:22:05,685][86121] Updated weights for policy 0, policy_version 3530 (0.0008) +[2023-10-09 12:22:05,693][86122] Updated weights for policy 1, policy_version 3530 (0.0008) +[2023-10-09 12:22:06,054][86122] Updated weights for policy 1, policy_version 3540 (0.0008) +[2023-10-09 12:22:06,066][86121] Updated weights for policy 0, policy_version 3540 (0.0008) +[2023-10-09 12:22:06,411][86122] Updated weights for policy 1, policy_version 3550 (0.0008) +[2023-10-09 12:22:06,437][86121] Updated weights for policy 0, policy_version 3550 (0.0008) +[2023-10-09 12:22:08,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 7274496. Throughput: 0: 1803.8, 1: 1817.5. Samples: 1828666. Policy #0 lag: (min: 31.0, avg: 31.3, max: 43.0) +[2023-10-09 12:22:08,399][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:22:10,263][86121] Updated weights for policy 0, policy_version 3560 (0.0009) +[2023-10-09 12:22:10,358][86122] Updated weights for policy 1, policy_version 3560 (0.0008) +[2023-10-09 12:22:10,633][86121] Updated weights for policy 0, policy_version 3570 (0.0010) +[2023-10-09 12:22:10,730][86122] Updated weights for policy 1, policy_version 3570 (0.0009) +[2023-10-09 12:22:10,999][86121] Updated weights for policy 0, policy_version 3580 (0.0009) +[2023-10-09 12:22:11,093][86122] Updated weights for policy 1, policy_version 3580 (0.0009) +[2023-10-09 12:22:13,398][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14329.0). Total num frames: 7340032. Throughput: 0: 1811.5, 1: 1818.6. Samples: 1838996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:22:13,399][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 12:22:14,656][86121] Updated weights for policy 0, policy_version 3590 (0.0008) +[2023-10-09 12:22:14,697][86122] Updated weights for policy 1, policy_version 3590 (0.0009) +[2023-10-09 12:22:15,025][86121] Updated weights for policy 0, policy_version 3600 (0.0009) +[2023-10-09 12:22:15,063][86122] Updated weights for policy 1, policy_version 3600 (0.0007) +[2023-10-09 12:22:15,381][86121] Updated weights for policy 0, policy_version 3610 (0.0008) +[2023-10-09 12:22:15,429][86122] Updated weights for policy 1, policy_version 3610 (0.0008) +[2023-10-09 12:22:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 7405568. Throughput: 0: 1808.3, 1: 1808.8. Samples: 1860908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:22:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:22:19,096][86121] Updated weights for policy 0, policy_version 3620 (0.0008) +[2023-10-09 12:22:19,159][86122] Updated weights for policy 1, policy_version 3620 (0.0009) +[2023-10-09 12:22:19,462][86121] Updated weights for policy 0, policy_version 3630 (0.0007) +[2023-10-09 12:22:19,526][86122] Updated weights for policy 1, policy_version 3630 (0.0008) +[2023-10-09 12:22:19,829][86121] Updated weights for policy 0, policy_version 3640 (0.0007) +[2023-10-09 12:22:19,887][86122] Updated weights for policy 1, policy_version 3640 (0.0007) +[2023-10-09 12:22:23,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14218.0). Total num frames: 7471104. Throughput: 0: 1800.1, 1: 1805.0. Samples: 1883412. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 12:22:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:22:23,438][86121] Updated weights for policy 0, policy_version 3650 (0.0008) +[2023-10-09 12:22:23,625][86122] Updated weights for policy 1, policy_version 3650 (0.0009) +[2023-10-09 12:22:23,795][86121] Updated weights for policy 0, policy_version 3660 (0.0009) +[2023-10-09 12:22:23,990][86122] Updated weights for policy 1, policy_version 3660 (0.0008) +[2023-10-09 12:22:24,172][86121] Updated weights for policy 0, policy_version 3670 (0.0008) +[2023-10-09 12:22:24,355][86122] Updated weights for policy 1, policy_version 3670 (0.0008) +[2023-10-09 12:22:24,530][86121] Updated weights for policy 0, policy_version 3680 (0.0007) +[2023-10-09 12:22:24,713][86122] Updated weights for policy 1, policy_version 3680 (0.0010) +[2023-10-09 12:22:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 7536640. Throughput: 0: 1799.7, 1: 1805.4. Samples: 1893214. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 12:22:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:22:28,433][86121] Updated weights for policy 0, policy_version 3690 (0.0009) +[2023-10-09 12:22:28,453][86122] Updated weights for policy 1, policy_version 3690 (0.0008) +[2023-10-09 12:22:28,813][86121] Updated weights for policy 0, policy_version 3700 (0.0008) +[2023-10-09 12:22:28,815][86122] Updated weights for policy 1, policy_version 3700 (0.0010) +[2023-10-09 12:22:29,170][86121] Updated weights for policy 0, policy_version 3710 (0.0008) +[2023-10-09 12:22:29,176][86122] Updated weights for policy 1, policy_version 3710 (0.0008) +[2023-10-09 12:22:32,739][86121] Updated weights for policy 0, policy_version 3720 (0.0007) +[2023-10-09 12:22:32,971][86122] Updated weights for policy 1, policy_version 3720 (0.0009) +[2023-10-09 12:22:33,103][86121] Updated weights for policy 0, policy_version 3730 (0.0008) +[2023-10-09 12:22:33,332][86122] Updated weights for policy 1, policy_version 3730 (0.0008) +[2023-10-09 12:22:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 7602176. Throughput: 0: 1801.4, 1: 1798.4. Samples: 1915702. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) +[2023-10-09 12:22:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 12:22:33,467][86121] Updated weights for policy 0, policy_version 3740 (0.0008) +[2023-10-09 12:22:33,685][86122] Updated weights for policy 1, policy_version 3740 (0.0008) +[2023-10-09 12:22:37,277][86121] Updated weights for policy 0, policy_version 3750 (0.0007) +[2023-10-09 12:22:37,561][86122] Updated weights for policy 1, policy_version 3750 (0.0007) +[2023-10-09 12:22:37,656][86121] Updated weights for policy 0, policy_version 3760 (0.0008) +[2023-10-09 12:22:37,927][86122] Updated weights for policy 1, policy_version 3760 (0.0007) +[2023-10-09 12:22:38,020][86121] Updated weights for policy 0, policy_version 3770 (0.0008) +[2023-10-09 12:22:38,292][86122] Updated weights for policy 1, policy_version 3770 (0.0008) +[2023-10-09 12:22:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 7700480. Throughput: 0: 1814.3, 1: 1811.8. Samples: 1936708. Policy #0 lag: (min: 16.0, avg: 38.1, max: 48.0) +[2023-10-09 12:22:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 12:22:41,777][86121] Updated weights for policy 0, policy_version 3780 (0.0009) +[2023-10-09 12:22:42,069][86122] Updated weights for policy 1, policy_version 3780 (0.0010) +[2023-10-09 12:22:42,154][86121] Updated weights for policy 0, policy_version 3790 (0.0008) +[2023-10-09 12:22:42,436][86122] Updated weights for policy 1, policy_version 3790 (0.0009) +[2023-10-09 12:22:42,520][86121] Updated weights for policy 0, policy_version 3800 (0.0007) +[2023-10-09 12:22:42,809][86122] Updated weights for policy 1, policy_version 3800 (0.0009) +[2023-10-09 12:22:43,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14440.2). Total num frames: 7798784. Throughput: 0: 1804.7, 1: 1790.5. Samples: 1947934. Policy #0 lag: (min: 16.0, avg: 38.1, max: 48.0) +[2023-10-09 12:22:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 12:22:46,288][86121] Updated weights for policy 0, policy_version 3810 (0.0007) +[2023-10-09 12:22:46,457][86122] Updated weights for policy 1, policy_version 3810 (0.0009) +[2023-10-09 12:22:46,647][86121] Updated weights for policy 0, policy_version 3820 (0.0008) +[2023-10-09 12:22:46,814][86122] Updated weights for policy 1, policy_version 3820 (0.0008) +[2023-10-09 12:22:47,012][86121] Updated weights for policy 0, policy_version 3830 (0.0009) +[2023-10-09 12:22:47,176][86122] Updated weights for policy 1, policy_version 3830 (0.0008) +[2023-10-09 12:22:47,382][86121] Updated weights for policy 0, policy_version 3840 (0.0009) +[2023-10-09 12:22:47,544][86122] Updated weights for policy 1, policy_version 3840 (0.0009) +[2023-10-09 12:22:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7864320. Throughput: 0: 1810.3, 1: 1809.6. Samples: 1969096. Policy #0 lag: (min: 29.0, avg: 36.9, max: 61.0) +[2023-10-09 12:22:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 12:22:51,062][86121] Updated weights for policy 0, policy_version 3850 (0.0011) +[2023-10-09 12:22:51,391][86122] Updated weights for policy 1, policy_version 3850 (0.0010) +[2023-10-09 12:22:51,425][86121] Updated weights for policy 0, policy_version 3860 (0.0007) +[2023-10-09 12:22:51,758][86122] Updated weights for policy 1, policy_version 3860 (0.0009) +[2023-10-09 12:22:51,796][86121] Updated weights for policy 0, policy_version 3870 (0.0007) +[2023-10-09 12:22:52,118][86122] Updated weights for policy 1, policy_version 3870 (0.0008) +[2023-10-09 12:22:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 7929856. Throughput: 0: 1801.7, 1: 1788.8. Samples: 1990234. Policy #0 lag: (min: 29.0, avg: 36.9, max: 61.0) +[2023-10-09 12:22:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 12:22:55,730][86121] Updated weights for policy 0, policy_version 3880 (0.0008) +[2023-10-09 12:22:55,890][86122] Updated weights for policy 1, policy_version 3880 (0.0010) +[2023-10-09 12:22:56,093][86121] Updated weights for policy 0, policy_version 3890 (0.0009) +[2023-10-09 12:22:56,263][86122] Updated weights for policy 1, policy_version 3890 (0.0008) +[2023-10-09 12:22:56,459][86121] Updated weights for policy 0, policy_version 3900 (0.0008) +[2023-10-09 12:22:56,626][86122] Updated weights for policy 1, policy_version 3900 (0.0008) +[2023-10-09 12:22:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 7995392. Throughput: 0: 1808.9, 1: 1807.9. Samples: 2001752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:22:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 12:23:00,111][86121] Updated weights for policy 0, policy_version 3910 (0.0008) +[2023-10-09 12:23:00,289][86122] Updated weights for policy 1, policy_version 3910 (0.0010) +[2023-10-09 12:23:00,477][86121] Updated weights for policy 0, policy_version 3920 (0.0010) +[2023-10-09 12:23:00,653][86122] Updated weights for policy 1, policy_version 3920 (0.0007) +[2023-10-09 12:23:00,851][86121] Updated weights for policy 0, policy_version 3930 (0.0008) +[2023-10-09 12:23:01,016][86122] Updated weights for policy 1, policy_version 3930 (0.0007) +[2023-10-09 12:23:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8060928. Throughput: 0: 1796.0, 1: 1793.2. Samples: 2022420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 12:23:04,526][86121] Updated weights for policy 0, policy_version 3940 (0.0008) +[2023-10-09 12:23:04,901][86121] Updated weights for policy 0, policy_version 3950 (0.0007) +[2023-10-09 12:23:04,917][86122] Updated weights for policy 1, policy_version 3940 (0.0010) +[2023-10-09 12:23:05,271][86121] Updated weights for policy 0, policy_version 3960 (0.0008) +[2023-10-09 12:23:05,277][86122] Updated weights for policy 1, policy_version 3950 (0.0008) +[2023-10-09 12:23:05,645][86122] Updated weights for policy 1, policy_version 3960 (0.0009) +[2023-10-09 12:23:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8126464. Throughput: 0: 1803.0, 1: 1793.8. Samples: 2045268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:23:08,756][86121] Updated weights for policy 0, policy_version 3970 (0.0009) +[2023-10-09 12:23:09,127][86121] Updated weights for policy 0, policy_version 3980 (0.0007) +[2023-10-09 12:23:09,307][86122] Updated weights for policy 1, policy_version 3970 (0.0011) +[2023-10-09 12:23:09,505][86121] Updated weights for policy 0, policy_version 3990 (0.0007) +[2023-10-09 12:23:09,676][86122] Updated weights for policy 1, policy_version 3980 (0.0007) +[2023-10-09 12:23:09,874][86121] Updated weights for policy 0, policy_version 4000 (0.0010) +[2023-10-09 12:23:10,048][86122] Updated weights for policy 1, policy_version 3990 (0.0009) +[2023-10-09 12:23:10,417][86122] Updated weights for policy 1, policy_version 4000 (0.0008) +[2023-10-09 12:23:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 8192000. Throughput: 0: 1802.5, 1: 1791.7. Samples: 2054956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:23:13,642][86121] Updated weights for policy 0, policy_version 4010 (0.0007) +[2023-10-09 12:23:14,014][86121] Updated weights for policy 0, policy_version 4020 (0.0008) +[2023-10-09 12:23:14,043][86122] Updated weights for policy 1, policy_version 4010 (0.0009) +[2023-10-09 12:23:14,381][86121] Updated weights for policy 0, policy_version 4030 (0.0008) +[2023-10-09 12:23:14,415][86122] Updated weights for policy 1, policy_version 4020 (0.0008) +[2023-10-09 12:23:14,772][86122] Updated weights for policy 1, policy_version 4030 (0.0009) +[2023-10-09 12:23:18,231][86121] Updated weights for policy 0, policy_version 4040 (0.0007) +[2023-10-09 12:23:18,259][86122] Updated weights for policy 1, policy_version 4040 (0.0008) +[2023-10-09 12:23:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 8257536. Throughput: 0: 1808.2, 1: 1808.5. Samples: 2078452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 12:23:18,590][86121] Updated weights for policy 0, policy_version 4050 (0.0008) +[2023-10-09 12:23:18,636][86122] Updated weights for policy 1, policy_version 4050 (0.0008) +[2023-10-09 12:23:18,954][86121] Updated weights for policy 0, policy_version 4060 (0.0009) +[2023-10-09 12:23:18,994][86122] Updated weights for policy 1, policy_version 4060 (0.0008) +[2023-10-09 12:23:22,730][86122] Updated weights for policy 1, policy_version 4070 (0.0007) +[2023-10-09 12:23:22,832][86121] Updated weights for policy 0, policy_version 4070 (0.0008) +[2023-10-09 12:23:23,089][86122] Updated weights for policy 1, policy_version 4080 (0.0008) +[2023-10-09 12:23:23,198][86121] Updated weights for policy 0, policy_version 4080 (0.0008) +[2023-10-09 12:23:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 8323072. Throughput: 0: 1817.2, 1: 1814.0. Samples: 2100112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:23:23,442][86122] Updated weights for policy 1, policy_version 4090 (0.0009) +[2023-10-09 12:23:23,559][86121] Updated weights for policy 0, policy_version 4090 (0.0008) +[2023-10-09 12:23:23,663][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000004096_4194304.pth... +[2023-10-09 12:23:23,694][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000002400_2457600.pth +[2023-10-09 12:23:23,780][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000004096_4194304.pth... +[2023-10-09 12:23:23,815][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000002400_2457600.pth +[2023-10-09 12:23:27,128][86121] Updated weights for policy 0, policy_version 4100 (0.0009) +[2023-10-09 12:23:27,149][86122] Updated weights for policy 1, policy_version 4100 (0.0007) +[2023-10-09 12:23:27,500][86121] Updated weights for policy 0, policy_version 4110 (0.0008) +[2023-10-09 12:23:27,509][86122] Updated weights for policy 1, policy_version 4110 (0.0007) +[2023-10-09 12:23:27,863][86121] Updated weights for policy 0, policy_version 4120 (0.0007) +[2023-10-09 12:23:27,877][86122] Updated weights for policy 1, policy_version 4120 (0.0010) +[2023-10-09 12:23:28,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 8454144. Throughput: 0: 1800.5, 1: 1812.4. Samples: 2110518. Policy #0 lag: (min: 2.0, avg: 3.8, max: 22.0) +[2023-10-09 12:23:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:23:31,578][86122] Updated weights for policy 1, policy_version 4130 (0.0009) +[2023-10-09 12:23:31,636][86121] Updated weights for policy 0, policy_version 4130 (0.0008) +[2023-10-09 12:23:31,937][86122] Updated weights for policy 1, policy_version 4140 (0.0007) +[2023-10-09 12:23:32,009][86121] Updated weights for policy 0, policy_version 4140 (0.0009) +[2023-10-09 12:23:32,295][86122] Updated weights for policy 1, policy_version 4150 (0.0009) +[2023-10-09 12:23:32,366][86121] Updated weights for policy 0, policy_version 4150 (0.0008) +[2023-10-09 12:23:32,650][86122] Updated weights for policy 1, policy_version 4160 (0.0007) +[2023-10-09 12:23:32,737][86121] Updated weights for policy 0, policy_version 4160 (0.0008) +[2023-10-09 12:23:33,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 8519680. Throughput: 0: 1813.4, 1: 1812.2. Samples: 2132248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) +[2023-10-09 12:23:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:23:36,355][86122] Updated weights for policy 1, policy_version 4170 (0.0009) +[2023-10-09 12:23:36,583][86121] Updated weights for policy 0, policy_version 4170 (0.0007) +[2023-10-09 12:23:36,713][86122] Updated weights for policy 1, policy_version 4180 (0.0010) +[2023-10-09 12:23:36,948][86121] Updated weights for policy 0, policy_version 4180 (0.0008) +[2023-10-09 12:23:37,081][86122] Updated weights for policy 1, policy_version 4190 (0.0009) +[2023-10-09 12:23:37,322][86121] Updated weights for policy 0, policy_version 4190 (0.0010) +[2023-10-09 12:23:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 8585216. Throughput: 0: 1802.6, 1: 1816.1. Samples: 2153076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 34.0) +[2023-10-09 12:23:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:23:40,840][86122] Updated weights for policy 1, policy_version 4200 (0.0009) +[2023-10-09 12:23:41,085][86121] Updated weights for policy 0, policy_version 4200 (0.0009) +[2023-10-09 12:23:41,199][86122] Updated weights for policy 1, policy_version 4210 (0.0009) +[2023-10-09 12:23:41,449][86121] Updated weights for policy 0, policy_version 4210 (0.0007) +[2023-10-09 12:23:41,562][86122] Updated weights for policy 1, policy_version 4220 (0.0007) +[2023-10-09 12:23:41,815][86121] Updated weights for policy 0, policy_version 4220 (0.0010) +[2023-10-09 12:23:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8650752. Throughput: 0: 1817.3, 1: 1813.0. Samples: 2165114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:23:45,222][86122] Updated weights for policy 1, policy_version 4230 (0.0009) +[2023-10-09 12:23:45,411][86121] Updated weights for policy 0, policy_version 4230 (0.0009) +[2023-10-09 12:23:45,589][86122] Updated weights for policy 1, policy_version 4240 (0.0009) +[2023-10-09 12:23:45,774][86121] Updated weights for policy 0, policy_version 4240 (0.0009) +[2023-10-09 12:23:45,953][86122] Updated weights for policy 1, policy_version 4250 (0.0008) +[2023-10-09 12:23:46,138][86121] Updated weights for policy 0, policy_version 4250 (0.0008) +[2023-10-09 12:23:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8716288. Throughput: 0: 1802.8, 1: 1813.5. Samples: 2185152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:23:49,872][86122] Updated weights for policy 1, policy_version 4260 (0.0009) +[2023-10-09 12:23:49,947][86121] Updated weights for policy 0, policy_version 4260 (0.0009) +[2023-10-09 12:23:50,236][86122] Updated weights for policy 1, policy_version 4270 (0.0007) +[2023-10-09 12:23:50,319][86121] Updated weights for policy 0, policy_version 4270 (0.0008) +[2023-10-09 12:23:50,601][86122] Updated weights for policy 1, policy_version 4280 (0.0009) +[2023-10-09 12:23:50,687][86121] Updated weights for policy 0, policy_version 4280 (0.0007) +[2023-10-09 12:23:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 8781824. Throughput: 0: 1796.8, 1: 1810.4. Samples: 2207596. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:23:54,457][86122] Updated weights for policy 1, policy_version 4290 (0.0009) +[2023-10-09 12:23:54,522][86121] Updated weights for policy 0, policy_version 4290 (0.0008) +[2023-10-09 12:23:54,815][86122] Updated weights for policy 1, policy_version 4300 (0.0008) +[2023-10-09 12:23:54,883][86121] Updated weights for policy 0, policy_version 4300 (0.0008) +[2023-10-09 12:23:55,184][86122] Updated weights for policy 1, policy_version 4310 (0.0007) +[2023-10-09 12:23:55,254][86121] Updated weights for policy 0, policy_version 4310 (0.0009) +[2023-10-09 12:23:55,544][86122] Updated weights for policy 1, policy_version 4320 (0.0009) +[2023-10-09 12:23:55,619][86121] Updated weights for policy 0, policy_version 4320 (0.0008) +[2023-10-09 12:23:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 8847360. Throughput: 0: 1796.7, 1: 1815.5. Samples: 2217504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:23:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:23:59,353][86122] Updated weights for policy 1, policy_version 4330 (0.0008) +[2023-10-09 12:23:59,371][86121] Updated weights for policy 0, policy_version 4330 (0.0008) +[2023-10-09 12:23:59,723][86122] Updated weights for policy 1, policy_version 4340 (0.0009) +[2023-10-09 12:23:59,743][86121] Updated weights for policy 0, policy_version 4340 (0.0008) +[2023-10-09 12:24:00,080][86122] Updated weights for policy 1, policy_version 4350 (0.0008) +[2023-10-09 12:24:00,110][86121] Updated weights for policy 0, policy_version 4350 (0.0008) +[2023-10-09 12:24:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 8912896. Throughput: 0: 1786.9, 1: 1805.5. Samples: 2240110. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 12:24:03,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:24:03,777][86122] Updated weights for policy 1, policy_version 4360 (0.0007) +[2023-10-09 12:24:03,827][86121] Updated weights for policy 0, policy_version 4360 (0.0008) +[2023-10-09 12:24:04,141][86122] Updated weights for policy 1, policy_version 4370 (0.0008) +[2023-10-09 12:24:04,190][86121] Updated weights for policy 0, policy_version 4370 (0.0008) +[2023-10-09 12:24:04,510][86122] Updated weights for policy 1, policy_version 4380 (0.0008) +[2023-10-09 12:24:04,566][86121] Updated weights for policy 0, policy_version 4380 (0.0009) +[2023-10-09 12:24:08,036][86122] Updated weights for policy 1, policy_version 4390 (0.0008) +[2023-10-09 12:24:08,360][86121] Updated weights for policy 0, policy_version 4390 (0.0008) +[2023-10-09 12:24:08,394][86122] Updated weights for policy 1, policy_version 4400 (0.0008) +[2023-10-09 12:24:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 8978432. Throughput: 0: 1802.0, 1: 1820.0. Samples: 2263100. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 12:24:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:24:08,744][86121] Updated weights for policy 0, policy_version 4400 (0.0008) +[2023-10-09 12:24:08,757][86122] Updated weights for policy 1, policy_version 4410 (0.0008) +[2023-10-09 12:24:09,105][86121] Updated weights for policy 0, policy_version 4410 (0.0008) +[2023-10-09 12:24:12,501][86122] Updated weights for policy 1, policy_version 4420 (0.0007) +[2023-10-09 12:24:12,778][86121] Updated weights for policy 0, policy_version 4420 (0.0010) +[2023-10-09 12:24:12,874][86122] Updated weights for policy 1, policy_version 4430 (0.0007) +[2023-10-09 12:24:13,145][86121] Updated weights for policy 0, policy_version 4430 (0.0009) +[2023-10-09 12:24:13,243][86122] Updated weights for policy 1, policy_version 4440 (0.0007) +[2023-10-09 12:24:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 9043968. Throughput: 0: 1791.1, 1: 1815.6. Samples: 2272818. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) +[2023-10-09 12:24:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:24:13,524][86121] Updated weights for policy 0, policy_version 4440 (0.0008) +[2023-10-09 12:24:16,966][86122] Updated weights for policy 1, policy_version 4450 (0.0008) +[2023-10-09 12:24:17,311][86121] Updated weights for policy 0, policy_version 4450 (0.0009) +[2023-10-09 12:24:17,323][86122] Updated weights for policy 1, policy_version 4460 (0.0008) +[2023-10-09 12:24:17,687][86122] Updated weights for policy 1, policy_version 4470 (0.0008) +[2023-10-09 12:24:17,687][86121] Updated weights for policy 0, policy_version 4460 (0.0007) +[2023-10-09 12:24:18,047][86121] Updated weights for policy 0, policy_version 4470 (0.0009) +[2023-10-09 12:24:18,057][86122] Updated weights for policy 1, policy_version 4480 (0.0008) +[2023-10-09 12:24:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 9142272. Throughput: 0: 1796.8, 1: 1822.6. Samples: 2295122. Policy #0 lag: (min: 31.0, avg: 32.0, max: 53.0) +[2023-10-09 12:24:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:24:18,418][86121] Updated weights for policy 0, policy_version 4480 (0.0010) +[2023-10-09 12:24:21,832][86122] Updated weights for policy 1, policy_version 4490 (0.0011) +[2023-10-09 12:24:22,180][86121] Updated weights for policy 0, policy_version 4490 (0.0007) +[2023-10-09 12:24:22,201][86122] Updated weights for policy 1, policy_version 4500 (0.0009) +[2023-10-09 12:24:22,546][86121] Updated weights for policy 0, policy_version 4500 (0.0008) +[2023-10-09 12:24:22,566][86122] Updated weights for policy 1, policy_version 4510 (0.0009) +[2023-10-09 12:24:22,913][86121] Updated weights for policy 0, policy_version 4510 (0.0007) +[2023-10-09 12:24:23,397][85186] Fps is (10 sec: 19660.2, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 9240576. Throughput: 0: 1792.2, 1: 1804.0. Samples: 2314906. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 12:24:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:24:26,327][86122] Updated weights for policy 1, policy_version 4520 (0.0009) +[2023-10-09 12:24:26,706][86122] Updated weights for policy 1, policy_version 4530 (0.0008) +[2023-10-09 12:24:26,762][86121] Updated weights for policy 0, policy_version 4520 (0.0007) +[2023-10-09 12:24:27,069][86122] Updated weights for policy 1, policy_version 4540 (0.0008) +[2023-10-09 12:24:27,137][86121] Updated weights for policy 0, policy_version 4530 (0.0007) +[2023-10-09 12:24:27,513][86121] Updated weights for policy 0, policy_version 4540 (0.0008) +[2023-10-09 12:24:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9306112. Throughput: 0: 1790.5, 1: 1819.4. Samples: 2327558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:24:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:24:30,761][86122] Updated weights for policy 1, policy_version 4550 (0.0009) +[2023-10-09 12:24:31,117][86122] Updated weights for policy 1, policy_version 4560 (0.0009) +[2023-10-09 12:24:31,252][86121] Updated weights for policy 0, policy_version 4550 (0.0008) +[2023-10-09 12:24:31,479][86122] Updated weights for policy 1, policy_version 4570 (0.0008) +[2023-10-09 12:24:31,620][86121] Updated weights for policy 0, policy_version 4560 (0.0008) +[2023-10-09 12:24:31,990][86121] Updated weights for policy 0, policy_version 4570 (0.0010) +[2023-10-09 12:24:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9371648. Throughput: 0: 1791.7, 1: 1811.6. Samples: 2347300. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:24:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:24:35,180][86122] Updated weights for policy 1, policy_version 4580 (0.0008) +[2023-10-09 12:24:35,555][86122] Updated weights for policy 1, policy_version 4590 (0.0008) +[2023-10-09 12:24:35,607][86121] Updated weights for policy 0, policy_version 4580 (0.0012) +[2023-10-09 12:24:35,922][86122] Updated weights for policy 1, policy_version 4600 (0.0009) +[2023-10-09 12:24:35,983][86121] Updated weights for policy 0, policy_version 4590 (0.0007) +[2023-10-09 12:24:36,347][86121] Updated weights for policy 0, policy_version 4600 (0.0008) +[2023-10-09 12:24:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 9437184. Throughput: 0: 1790.7, 1: 1816.9. Samples: 2369936. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) +[2023-10-09 12:24:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:24:39,666][86122] Updated weights for policy 1, policy_version 4610 (0.0008) +[2023-10-09 12:24:40,021][86121] Updated weights for policy 0, policy_version 4610 (0.0007) +[2023-10-09 12:24:40,041][86122] Updated weights for policy 1, policy_version 4620 (0.0008) +[2023-10-09 12:24:40,382][86121] Updated weights for policy 0, policy_version 4620 (0.0008) +[2023-10-09 12:24:40,395][86122] Updated weights for policy 1, policy_version 4630 (0.0008) +[2023-10-09 12:24:40,743][86121] Updated weights for policy 0, policy_version 4630 (0.0008) +[2023-10-09 12:24:40,759][86122] Updated weights for policy 1, policy_version 4640 (0.0009) +[2023-10-09 12:24:41,113][86121] Updated weights for policy 0, policy_version 4640 (0.0008) +[2023-10-09 12:24:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9502720. Throughput: 0: 1797.7, 1: 1813.1. Samples: 2379994. Policy #0 lag: (min: 10.0, avg: 16.8, max: 42.0) +[2023-10-09 12:24:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:24:44,544][86122] Updated weights for policy 1, policy_version 4650 (0.0008) +[2023-10-09 12:24:44,919][86122] Updated weights for policy 1, policy_version 4660 (0.0008) +[2023-10-09 12:24:44,929][86121] Updated weights for policy 0, policy_version 4650 (0.0007) +[2023-10-09 12:24:45,275][86122] Updated weights for policy 1, policy_version 4670 (0.0009) +[2023-10-09 12:24:45,298][86121] Updated weights for policy 0, policy_version 4660 (0.0008) +[2023-10-09 12:24:45,667][86121] Updated weights for policy 0, policy_version 4670 (0.0009) +[2023-10-09 12:24:48,398][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 9568256. Throughput: 0: 1790.4, 1: 1809.8. Samples: 2402120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:24:48,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:24:48,968][86122] Updated weights for policy 1, policy_version 4680 (0.0009) +[2023-10-09 12:24:49,342][86122] Updated weights for policy 1, policy_version 4690 (0.0008) +[2023-10-09 12:24:49,387][86121] Updated weights for policy 0, policy_version 4680 (0.0007) +[2023-10-09 12:24:49,712][86122] Updated weights for policy 1, policy_version 4700 (0.0008) +[2023-10-09 12:24:49,755][86121] Updated weights for policy 0, policy_version 4690 (0.0009) +[2023-10-09 12:24:50,131][86121] Updated weights for policy 0, policy_version 4700 (0.0008) +[2023-10-09 12:24:53,374][86122] Updated weights for policy 1, policy_version 4710 (0.0008) +[2023-10-09 12:24:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14329.1). Total num frames: 9633792. Throughput: 0: 1786.0, 1: 1810.0. Samples: 2424920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:24:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:24:53,740][86122] Updated weights for policy 1, policy_version 4720 (0.0008) +[2023-10-09 12:24:53,812][86121] Updated weights for policy 0, policy_version 4710 (0.0008) +[2023-10-09 12:24:54,098][86122] Updated weights for policy 1, policy_version 4730 (0.0007) +[2023-10-09 12:24:54,174][86121] Updated weights for policy 0, policy_version 4720 (0.0007) +[2023-10-09 12:24:54,545][86121] Updated weights for policy 0, policy_version 4730 (0.0009) +[2023-10-09 12:24:57,738][86122] Updated weights for policy 1, policy_version 4740 (0.0007) +[2023-10-09 12:24:58,102][86122] Updated weights for policy 1, policy_version 4750 (0.0008) +[2023-10-09 12:24:58,330][86121] Updated weights for policy 0, policy_version 4740 (0.0010) +[2023-10-09 12:24:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14218.0). Total num frames: 9699328. Throughput: 0: 1788.7, 1: 1809.8. Samples: 2434750. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) +[2023-10-09 12:24:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:24:58,465][86122] Updated weights for policy 1, policy_version 4760 (0.0008) +[2023-10-09 12:24:58,697][86121] Updated weights for policy 0, policy_version 4750 (0.0007) +[2023-10-09 12:24:59,069][86121] Updated weights for policy 0, policy_version 4760 (0.0008) +[2023-10-09 12:25:02,180][86122] Updated weights for policy 1, policy_version 4770 (0.0009) +[2023-10-09 12:25:02,550][86122] Updated weights for policy 1, policy_version 4780 (0.0007) +[2023-10-09 12:25:02,784][86121] Updated weights for policy 0, policy_version 4770 (0.0007) +[2023-10-09 12:25:02,916][86122] Updated weights for policy 1, policy_version 4790 (0.0007) +[2023-10-09 12:25:03,153][86121] Updated weights for policy 0, policy_version 4780 (0.0009) +[2023-10-09 12:25:03,293][86122] Updated weights for policy 1, policy_version 4800 (0.0008) +[2023-10-09 12:25:03,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 9797632. Throughput: 0: 1796.0, 1: 1813.3. Samples: 2457542. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) +[2023-10-09 12:25:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:25:03,515][86121] Updated weights for policy 0, policy_version 4790 (0.0008) +[2023-10-09 12:25:03,880][86121] Updated weights for policy 0, policy_version 4800 (0.0009) +[2023-10-09 12:25:07,055][86122] Updated weights for policy 1, policy_version 4810 (0.0008) +[2023-10-09 12:25:07,414][86122] Updated weights for policy 1, policy_version 4820 (0.0007) +[2023-10-09 12:25:07,503][86121] Updated weights for policy 0, policy_version 4810 (0.0007) +[2023-10-09 12:25:07,773][86122] Updated weights for policy 1, policy_version 4830 (0.0010) +[2023-10-09 12:25:07,870][86121] Updated weights for policy 0, policy_version 4820 (0.0008) +[2023-10-09 12:25:08,243][86121] Updated weights for policy 0, policy_version 4830 (0.0008) +[2023-10-09 12:25:08,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 9895936. Throughput: 0: 1803.8, 1: 1819.2. Samples: 2477940. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 12:25:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:25:11,753][86122] Updated weights for policy 1, policy_version 4840 (0.0008) +[2023-10-09 12:25:12,115][86121] Updated weights for policy 0, policy_version 4840 (0.0007) +[2023-10-09 12:25:12,125][86122] Updated weights for policy 1, policy_version 4850 (0.0007) +[2023-10-09 12:25:12,489][86122] Updated weights for policy 1, policy_version 4860 (0.0007) +[2023-10-09 12:25:12,491][86121] Updated weights for policy 0, policy_version 4850 (0.0008) +[2023-10-09 12:25:12,852][86121] Updated weights for policy 0, policy_version 4860 (0.0008) +[2023-10-09 12:25:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14440.1). Total num frames: 9961472. Throughput: 0: 1794.8, 1: 1809.0. Samples: 2489728. Policy #0 lag: (min: 12.0, avg: 15.1, max: 44.0) +[2023-10-09 12:25:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:25:16,002][86122] Updated weights for policy 1, policy_version 4870 (0.0008) +[2023-10-09 12:25:16,376][86122] Updated weights for policy 1, policy_version 4880 (0.0010) +[2023-10-09 12:25:16,668][86121] Updated weights for policy 0, policy_version 4870 (0.0009) +[2023-10-09 12:25:16,737][86122] Updated weights for policy 1, policy_version 4890 (0.0007) +[2023-10-09 12:25:17,041][86121] Updated weights for policy 0, policy_version 4880 (0.0009) +[2023-10-09 12:25:17,417][86121] Updated weights for policy 0, policy_version 4890 (0.0008) +[2023-10-09 12:25:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 10027008. Throughput: 0: 1806.8, 1: 1815.9. Samples: 2510320. Policy #0 lag: (min: 12.0, avg: 15.1, max: 44.0) +[2023-10-09 12:25:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:25:20,314][86122] Updated weights for policy 1, policy_version 4900 (0.0008) +[2023-10-09 12:25:20,684][86122] Updated weights for policy 1, policy_version 4910 (0.0009) +[2023-10-09 12:25:21,044][86122] Updated weights for policy 1, policy_version 4920 (0.0009) +[2023-10-09 12:25:21,135][86121] Updated weights for policy 0, policy_version 4900 (0.0007) +[2023-10-09 12:25:21,503][86121] Updated weights for policy 0, policy_version 4910 (0.0008) +[2023-10-09 12:25:21,863][86121] Updated weights for policy 0, policy_version 4920 (0.0008) +[2023-10-09 12:25:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10092544. Throughput: 0: 1789.4, 1: 1810.4. Samples: 2531926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:25:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:25:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000004928_5046272.pth... +[2023-10-09 12:25:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000004928_5046272.pth... +[2023-10-09 12:25:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000003232_3309568.pth +[2023-10-09 12:25:23,445][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000003232_3309568.pth +[2023-10-09 12:25:24,681][86122] Updated weights for policy 1, policy_version 4930 (0.0009) +[2023-10-09 12:25:25,052][86122] Updated weights for policy 1, policy_version 4940 (0.0009) +[2023-10-09 12:25:25,393][86121] Updated weights for policy 0, policy_version 4930 (0.0011) +[2023-10-09 12:25:25,407][86122] Updated weights for policy 1, policy_version 4950 (0.0008) +[2023-10-09 12:25:25,752][86121] Updated weights for policy 0, policy_version 4940 (0.0008) +[2023-10-09 12:25:25,769][86122] Updated weights for policy 1, policy_version 4960 (0.0008) +[2023-10-09 12:25:26,118][86121] Updated weights for policy 0, policy_version 4950 (0.0007) +[2023-10-09 12:25:26,490][86121] Updated weights for policy 0, policy_version 4960 (0.0008) +[2023-10-09 12:25:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10158080. Throughput: 0: 1807.2, 1: 1809.6. Samples: 2542750. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:25:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:25:29,473][86122] Updated weights for policy 1, policy_version 4970 (0.0007) +[2023-10-09 12:25:29,845][86122] Updated weights for policy 1, policy_version 4980 (0.0008) +[2023-10-09 12:25:30,208][86122] Updated weights for policy 1, policy_version 4990 (0.0008) +[2023-10-09 12:25:30,225][86121] Updated weights for policy 0, policy_version 4970 (0.0008) +[2023-10-09 12:25:30,589][86121] Updated weights for policy 0, policy_version 4980 (0.0008) +[2023-10-09 12:25:30,958][86121] Updated weights for policy 0, policy_version 4990 (0.0011) +[2023-10-09 12:25:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10223616. Throughput: 0: 1800.4, 1: 1814.5. Samples: 2564792. Policy #0 lag: (min: 8.0, avg: 13.9, max: 40.0) +[2023-10-09 12:25:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:25:33,857][86122] Updated weights for policy 1, policy_version 5000 (0.0007) +[2023-10-09 12:25:34,227][86122] Updated weights for policy 1, policy_version 5010 (0.0007) +[2023-10-09 12:25:34,597][86122] Updated weights for policy 1, policy_version 5020 (0.0008) +[2023-10-09 12:25:34,650][86121] Updated weights for policy 0, policy_version 5000 (0.0009) +[2023-10-09 12:25:35,016][86121] Updated weights for policy 0, policy_version 5010 (0.0007) +[2023-10-09 12:25:35,388][86121] Updated weights for policy 0, policy_version 5020 (0.0008) +[2023-10-09 12:25:38,297][86122] Updated weights for policy 1, policy_version 5030 (0.0009) +[2023-10-09 12:25:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10289152. Throughput: 0: 1809.3, 1: 1815.3. Samples: 2588024. Policy #0 lag: (min: 8.0, avg: 13.9, max: 40.0) +[2023-10-09 12:25:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:25:38,665][86122] Updated weights for policy 1, policy_version 5040 (0.0007) +[2023-10-09 12:25:39,037][86122] Updated weights for policy 1, policy_version 5050 (0.0009) +[2023-10-09 12:25:39,111][86121] Updated weights for policy 0, policy_version 5030 (0.0009) +[2023-10-09 12:25:39,481][86121] Updated weights for policy 0, policy_version 5040 (0.0007) +[2023-10-09 12:25:39,845][86121] Updated weights for policy 0, policy_version 5050 (0.0009) +[2023-10-09 12:25:42,781][86122] Updated weights for policy 1, policy_version 5060 (0.0009) +[2023-10-09 12:25:43,149][86122] Updated weights for policy 1, policy_version 5070 (0.0007) +[2023-10-09 12:25:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10354688. Throughput: 0: 1809.5, 1: 1812.0. Samples: 2597716. Policy #0 lag: (min: 27.0, avg: 38.1, max: 59.0) +[2023-10-09 12:25:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:25:43,522][86122] Updated weights for policy 1, policy_version 5080 (0.0010) +[2023-10-09 12:25:43,652][86121] Updated weights for policy 0, policy_version 5060 (0.0010) +[2023-10-09 12:25:44,021][86121] Updated weights for policy 0, policy_version 5070 (0.0009) +[2023-10-09 12:25:44,403][86121] Updated weights for policy 0, policy_version 5080 (0.0009) +[2023-10-09 12:25:47,260][86122] Updated weights for policy 1, policy_version 5090 (0.0009) +[2023-10-09 12:25:47,622][86122] Updated weights for policy 1, policy_version 5100 (0.0008) +[2023-10-09 12:25:47,996][86122] Updated weights for policy 1, policy_version 5110 (0.0008) +[2023-10-09 12:25:48,085][86121] Updated weights for policy 0, policy_version 5090 (0.0008) +[2023-10-09 12:25:48,353][86122] Updated weights for policy 1, policy_version 5120 (0.0008) +[2023-10-09 12:25:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 10452992. Throughput: 0: 1812.0, 1: 1811.1. Samples: 2620580. Policy #0 lag: (min: 27.0, avg: 38.1, max: 59.0) +[2023-10-09 12:25:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:25:48,449][86121] Updated weights for policy 0, policy_version 5100 (0.0007) +[2023-10-09 12:25:48,825][86121] Updated weights for policy 0, policy_version 5110 (0.0008) +[2023-10-09 12:25:49,196][86121] Updated weights for policy 0, policy_version 5120 (0.0007) +[2023-10-09 12:25:51,957][86122] Updated weights for policy 1, policy_version 5130 (0.0007) +[2023-10-09 12:25:52,324][86122] Updated weights for policy 1, policy_version 5140 (0.0008) +[2023-10-09 12:25:52,694][86122] Updated weights for policy 1, policy_version 5150 (0.0009) +[2023-10-09 12:25:52,960][86121] Updated weights for policy 0, policy_version 5130 (0.0009) +[2023-10-09 12:25:53,328][86121] Updated weights for policy 0, policy_version 5140 (0.0008) +[2023-10-09 12:25:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 10518528. Throughput: 0: 1824.1, 1: 1814.2. Samples: 2641664. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 12:25:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:25:53,696][86121] Updated weights for policy 0, policy_version 5150 (0.0008) +[2023-10-09 12:25:56,358][86122] Updated weights for policy 1, policy_version 5160 (0.0008) +[2023-10-09 12:25:56,723][86122] Updated weights for policy 1, policy_version 5170 (0.0009) +[2023-10-09 12:25:57,090][86122] Updated weights for policy 1, policy_version 5180 (0.0007) +[2023-10-09 12:25:57,481][86121] Updated weights for policy 0, policy_version 5160 (0.0007) +[2023-10-09 12:25:57,860][86121] Updated weights for policy 0, policy_version 5170 (0.0008) +[2023-10-09 12:25:58,229][86121] Updated weights for policy 0, policy_version 5180 (0.0009) +[2023-10-09 12:25:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 10616832. Throughput: 0: 1812.7, 1: 1823.1. Samples: 2653340. Policy #0 lag: (min: 26.0, avg: 27.1, max: 43.0) +[2023-10-09 12:25:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:26:00,893][86122] Updated weights for policy 1, policy_version 5190 (0.0010) +[2023-10-09 12:26:01,262][86122] Updated weights for policy 1, policy_version 5200 (0.0008) +[2023-10-09 12:26:01,632][86122] Updated weights for policy 1, policy_version 5210 (0.0007) +[2023-10-09 12:26:01,890][86121] Updated weights for policy 0, policy_version 5190 (0.0010) +[2023-10-09 12:26:02,256][86121] Updated weights for policy 0, policy_version 5200 (0.0009) +[2023-10-09 12:26:02,628][86121] Updated weights for policy 0, policy_version 5210 (0.0008) +[2023-10-09 12:26:03,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 10682368. Throughput: 0: 1823.4, 1: 1815.7. Samples: 2674082. Policy #0 lag: (min: 26.0, avg: 27.1, max: 43.0) +[2023-10-09 12:26:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:26:05,079][86122] Updated weights for policy 1, policy_version 5220 (0.0008) +[2023-10-09 12:26:05,449][86122] Updated weights for policy 1, policy_version 5230 (0.0007) +[2023-10-09 12:26:05,816][86122] Updated weights for policy 1, policy_version 5240 (0.0009) +[2023-10-09 12:26:06,356][86121] Updated weights for policy 0, policy_version 5220 (0.0008) +[2023-10-09 12:26:06,726][86121] Updated weights for policy 0, policy_version 5230 (0.0007) +[2023-10-09 12:26:07,095][86121] Updated weights for policy 0, policy_version 5240 (0.0007) +[2023-10-09 12:26:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10747904. Throughput: 0: 1819.1, 1: 1828.7. Samples: 2696076. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) +[2023-10-09 12:26:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 12:26:09,617][86122] Updated weights for policy 1, policy_version 5250 (0.0009) +[2023-10-09 12:26:09,979][86122] Updated weights for policy 1, policy_version 5260 (0.0008) +[2023-10-09 12:26:10,340][86122] Updated weights for policy 1, policy_version 5270 (0.0008) +[2023-10-09 12:26:10,697][86121] Updated weights for policy 0, policy_version 5250 (0.0007) +[2023-10-09 12:26:10,700][86122] Updated weights for policy 1, policy_version 5280 (0.0009) +[2023-10-09 12:26:11,069][86121] Updated weights for policy 0, policy_version 5260 (0.0009) +[2023-10-09 12:26:11,439][86121] Updated weights for policy 0, policy_version 5270 (0.0008) +[2023-10-09 12:26:11,804][86121] Updated weights for policy 0, policy_version 5280 (0.0008) +[2023-10-09 12:26:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10813440. Throughput: 0: 1824.2, 1: 1826.4. Samples: 2707028. Policy #0 lag: (min: 1.0, avg: 14.2, max: 33.0) +[2023-10-09 12:26:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 12:26:14,392][86122] Updated weights for policy 1, policy_version 5290 (0.0010) +[2023-10-09 12:26:14,760][86122] Updated weights for policy 1, policy_version 5300 (0.0010) +[2023-10-09 12:26:15,128][86122] Updated weights for policy 1, policy_version 5310 (0.0008) +[2023-10-09 12:26:15,454][86121] Updated weights for policy 0, policy_version 5290 (0.0009) +[2023-10-09 12:26:15,817][86121] Updated weights for policy 0, policy_version 5300 (0.0008) +[2023-10-09 12:26:16,186][86121] Updated weights for policy 0, policy_version 5310 (0.0007) +[2023-10-09 12:26:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 10878976. Throughput: 0: 1819.8, 1: 1824.0. Samples: 2728762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:26:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.930')] +[2023-10-09 12:26:18,982][86122] Updated weights for policy 1, policy_version 5320 (0.0007) +[2023-10-09 12:26:19,353][86122] Updated weights for policy 1, policy_version 5330 (0.0007) +[2023-10-09 12:26:19,717][86122] Updated weights for policy 1, policy_version 5340 (0.0007) +[2023-10-09 12:26:19,822][86121] Updated weights for policy 0, policy_version 5320 (0.0009) +[2023-10-09 12:26:20,190][86121] Updated weights for policy 0, policy_version 5330 (0.0007) +[2023-10-09 12:26:20,564][86121] Updated weights for policy 0, policy_version 5340 (0.0007) +[2023-10-09 12:26:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 10944512. Throughput: 0: 1814.6, 1: 1815.9. Samples: 2751398. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:26:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.930')] +[2023-10-09 12:26:23,421][86122] Updated weights for policy 1, policy_version 5350 (0.0009) +[2023-10-09 12:26:23,794][86122] Updated weights for policy 1, policy_version 5360 (0.0008) +[2023-10-09 12:26:24,155][86122] Updated weights for policy 1, policy_version 5370 (0.0008) +[2023-10-09 12:26:24,373][86121] Updated weights for policy 0, policy_version 5350 (0.0007) +[2023-10-09 12:26:24,740][86121] Updated weights for policy 0, policy_version 5360 (0.0008) +[2023-10-09 12:26:25,108][86121] Updated weights for policy 0, policy_version 5370 (0.0009) +[2023-10-09 12:26:27,881][86122] Updated weights for policy 1, policy_version 5380 (0.0008) +[2023-10-09 12:26:28,242][86122] Updated weights for policy 1, policy_version 5390 (0.0008) +[2023-10-09 12:26:28,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11010048. Throughput: 0: 1813.6, 1: 1818.8. Samples: 2761172. Policy #0 lag: (min: 4.0, avg: 12.1, max: 36.0) +[2023-10-09 12:26:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.930')] +[2023-10-09 12:26:28,607][86122] Updated weights for policy 1, policy_version 5400 (0.0008) +[2023-10-09 12:26:28,849][86121] Updated weights for policy 0, policy_version 5380 (0.0009) +[2023-10-09 12:26:29,224][86121] Updated weights for policy 0, policy_version 5390 (0.0009) +[2023-10-09 12:26:29,597][86121] Updated weights for policy 0, policy_version 5400 (0.0007) +[2023-10-09 12:26:32,141][86122] Updated weights for policy 1, policy_version 5410 (0.0009) +[2023-10-09 12:26:32,514][86122] Updated weights for policy 1, policy_version 5420 (0.0008) +[2023-10-09 12:26:32,872][86122] Updated weights for policy 1, policy_version 5430 (0.0009) +[2023-10-09 12:26:33,247][86122] Updated weights for policy 1, policy_version 5440 (0.0007) +[2023-10-09 12:26:33,381][86121] Updated weights for policy 0, policy_version 5410 (0.0009) +[2023-10-09 12:26:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11108352. Throughput: 0: 1805.1, 1: 1823.7. Samples: 2783876. Policy #0 lag: (min: 4.0, avg: 12.1, max: 36.0) +[2023-10-09 12:26:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.920')] +[2023-10-09 12:26:33,750][86121] Updated weights for policy 0, policy_version 5420 (0.0008) +[2023-10-09 12:26:34,117][86121] Updated weights for policy 0, policy_version 5430 (0.0008) +[2023-10-09 12:26:34,487][86121] Updated weights for policy 0, policy_version 5440 (0.0009) +[2023-10-09 12:26:37,126][86122] Updated weights for policy 1, policy_version 5450 (0.0007) +[2023-10-09 12:26:37,491][86122] Updated weights for policy 1, policy_version 5460 (0.0009) +[2023-10-09 12:26:37,851][86122] Updated weights for policy 1, policy_version 5470 (0.0008) +[2023-10-09 12:26:38,135][86121] Updated weights for policy 0, policy_version 5450 (0.0007) +[2023-10-09 12:26:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 11173888. Throughput: 0: 1814.6, 1: 1821.7. Samples: 2805296. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) +[2023-10-09 12:26:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.900')] +[2023-10-09 12:26:38,509][86121] Updated weights for policy 0, policy_version 5460 (0.0009) +[2023-10-09 12:26:38,873][86121] Updated weights for policy 0, policy_version 5470 (0.0009) +[2023-10-09 12:26:41,524][86122] Updated weights for policy 1, policy_version 5480 (0.0008) +[2023-10-09 12:26:41,882][86122] Updated weights for policy 1, policy_version 5490 (0.0011) +[2023-10-09 12:26:42,254][86122] Updated weights for policy 1, policy_version 5500 (0.0010) +[2023-10-09 12:26:42,592][86121] Updated weights for policy 0, policy_version 5480 (0.0011) +[2023-10-09 12:26:42,966][86121] Updated weights for policy 0, policy_version 5490 (0.0009) +[2023-10-09 12:26:43,341][86121] Updated weights for policy 0, policy_version 5500 (0.0010) +[2023-10-09 12:26:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14329.1). Total num frames: 11239424. Throughput: 0: 1810.3, 1: 1818.6. Samples: 2816640. Policy #0 lag: (min: 2.0, avg: 9.7, max: 34.0) +[2023-10-09 12:26:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.900')] +[2023-10-09 12:26:45,871][86122] Updated weights for policy 1, policy_version 5510 (0.0010) +[2023-10-09 12:26:46,230][86122] Updated weights for policy 1, policy_version 5520 (0.0010) +[2023-10-09 12:26:46,599][86122] Updated weights for policy 1, policy_version 5530 (0.0009) +[2023-10-09 12:26:47,016][86121] Updated weights for policy 0, policy_version 5510 (0.0008) +[2023-10-09 12:26:47,379][86121] Updated weights for policy 0, policy_version 5520 (0.0007) +[2023-10-09 12:26:47,749][86121] Updated weights for policy 0, policy_version 5530 (0.0009) +[2023-10-09 12:26:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 11337728. Throughput: 0: 1814.1, 1: 1821.7. Samples: 2837694. Policy #0 lag: (min: 17.0, avg: 27.1, max: 49.0) +[2023-10-09 12:26:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.900')] +[2023-10-09 12:26:50,276][86122] Updated weights for policy 1, policy_version 5540 (0.0007) +[2023-10-09 12:26:50,646][86122] Updated weights for policy 1, policy_version 5550 (0.0007) +[2023-10-09 12:26:51,018][86122] Updated weights for policy 1, policy_version 5560 (0.0007) +[2023-10-09 12:26:51,419][86121] Updated weights for policy 0, policy_version 5540 (0.0009) +[2023-10-09 12:26:51,796][86121] Updated weights for policy 0, policy_version 5550 (0.0009) +[2023-10-09 12:26:52,167][86121] Updated weights for policy 0, policy_version 5560 (0.0008) +[2023-10-09 12:26:53,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 11403264. Throughput: 0: 1808.4, 1: 1814.0. Samples: 2859086. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 12:26:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.890')] +[2023-10-09 12:26:54,504][86122] Updated weights for policy 1, policy_version 5570 (0.0010) +[2023-10-09 12:26:54,866][86122] Updated weights for policy 1, policy_version 5580 (0.0009) +[2023-10-09 12:26:55,234][86122] Updated weights for policy 1, policy_version 5590 (0.0008) +[2023-10-09 12:26:55,585][86122] Updated weights for policy 1, policy_version 5600 (0.0010) +[2023-10-09 12:26:55,945][86121] Updated weights for policy 0, policy_version 5570 (0.0009) +[2023-10-09 12:26:56,318][86121] Updated weights for policy 0, policy_version 5580 (0.0010) +[2023-10-09 12:26:56,680][86121] Updated weights for policy 0, policy_version 5590 (0.0011) +[2023-10-09 12:26:57,049][86121] Updated weights for policy 0, policy_version 5600 (0.0010) +[2023-10-09 12:26:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11468800. Throughput: 0: 1813.7, 1: 1820.3. Samples: 2870558. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 12:26:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.890')] +[2023-10-09 12:26:59,226][86122] Updated weights for policy 1, policy_version 5610 (0.0010) +[2023-10-09 12:26:59,589][86122] Updated weights for policy 1, policy_version 5620 (0.0008) +[2023-10-09 12:26:59,958][86122] Updated weights for policy 1, policy_version 5630 (0.0010) +[2023-10-09 12:27:00,821][86121] Updated weights for policy 0, policy_version 5610 (0.0009) +[2023-10-09 12:27:01,191][86121] Updated weights for policy 0, policy_version 5620 (0.0009) +[2023-10-09 12:27:01,556][86121] Updated weights for policy 0, policy_version 5630 (0.0008) +[2023-10-09 12:27:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 11534336. Throughput: 0: 1795.2, 1: 1828.0. Samples: 2891804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:27:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.890')] +[2023-10-09 12:27:03,670][86122] Updated weights for policy 1, policy_version 5640 (0.0007) +[2023-10-09 12:27:04,045][86122] Updated weights for policy 1, policy_version 5650 (0.0010) +[2023-10-09 12:27:04,401][86122] Updated weights for policy 1, policy_version 5660 (0.0008) +[2023-10-09 12:27:05,236][86121] Updated weights for policy 0, policy_version 5640 (0.0011) +[2023-10-09 12:27:05,609][86121] Updated weights for policy 0, policy_version 5650 (0.0010) +[2023-10-09 12:27:05,970][86121] Updated weights for policy 0, policy_version 5660 (0.0009) +[2023-10-09 12:27:08,150][86122] Updated weights for policy 1, policy_version 5670 (0.0008) +[2023-10-09 12:27:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11599872. Throughput: 0: 1798.1, 1: 1831.2. Samples: 2914716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:27:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.880')] +[2023-10-09 12:27:08,510][86122] Updated weights for policy 1, policy_version 5680 (0.0007) +[2023-10-09 12:27:08,874][86122] Updated weights for policy 1, policy_version 5690 (0.0007) +[2023-10-09 12:27:09,760][86121] Updated weights for policy 0, policy_version 5670 (0.0009) +[2023-10-09 12:27:10,135][86121] Updated weights for policy 0, policy_version 5680 (0.0007) +[2023-10-09 12:27:10,512][86121] Updated weights for policy 0, policy_version 5690 (0.0008) +[2023-10-09 12:27:12,628][86122] Updated weights for policy 1, policy_version 5700 (0.0008) +[2023-10-09 12:27:12,999][86122] Updated weights for policy 1, policy_version 5710 (0.0010) +[2023-10-09 12:27:13,363][86122] Updated weights for policy 1, policy_version 5720 (0.0007) +[2023-10-09 12:27:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11665408. Throughput: 0: 1798.5, 1: 1830.6. Samples: 2924480. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 12:27:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.870')] +[2023-10-09 12:27:14,369][86121] Updated weights for policy 0, policy_version 5700 (0.0009) +[2023-10-09 12:27:14,730][86121] Updated weights for policy 0, policy_version 5710 (0.0008) +[2023-10-09 12:27:15,100][86121] Updated weights for policy 0, policy_version 5720 (0.0007) +[2023-10-09 12:27:16,966][86122] Updated weights for policy 1, policy_version 5730 (0.0008) +[2023-10-09 12:27:17,324][86122] Updated weights for policy 1, policy_version 5740 (0.0008) +[2023-10-09 12:27:17,695][86122] Updated weights for policy 1, policy_version 5750 (0.0008) +[2023-10-09 12:27:18,059][86122] Updated weights for policy 1, policy_version 5760 (0.0009) +[2023-10-09 12:27:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11763712. Throughput: 0: 1797.6, 1: 1826.6. Samples: 2946964. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 12:27:18,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.870')] +[2023-10-09 12:27:18,699][86121] Updated weights for policy 0, policy_version 5730 (0.0008) +[2023-10-09 12:27:19,064][86121] Updated weights for policy 0, policy_version 5740 (0.0009) +[2023-10-09 12:27:19,428][86121] Updated weights for policy 0, policy_version 5750 (0.0008) +[2023-10-09 12:27:19,795][86121] Updated weights for policy 0, policy_version 5760 (0.0008) +[2023-10-09 12:27:21,825][86122] Updated weights for policy 1, policy_version 5770 (0.0008) +[2023-10-09 12:27:22,195][86122] Updated weights for policy 1, policy_version 5780 (0.0007) +[2023-10-09 12:27:22,568][86122] Updated weights for policy 1, policy_version 5790 (0.0008) +[2023-10-09 12:27:23,397][85186] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11829248. Throughput: 0: 1800.1, 1: 1819.2. Samples: 2968164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:27:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.850')] +[2023-10-09 12:27:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000005792_5931008.pth... +[2023-10-09 12:27:23,441][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000004096_4194304.pth +[2023-10-09 12:27:23,558][86121] Updated weights for policy 0, policy_version 5770 (0.0008) +[2023-10-09 12:27:23,928][86121] Updated weights for policy 0, policy_version 5780 (0.0007) +[2023-10-09 12:27:24,296][86121] Updated weights for policy 0, policy_version 5790 (0.0008) +[2023-10-09 12:27:24,364][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000005792_5931008.pth... +[2023-10-09 12:27:24,403][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000004096_4194304.pth +[2023-10-09 12:27:26,365][86122] Updated weights for policy 1, policy_version 5800 (0.0009) +[2023-10-09 12:27:26,734][86122] Updated weights for policy 1, policy_version 5810 (0.0007) +[2023-10-09 12:27:27,094][86122] Updated weights for policy 1, policy_version 5820 (0.0007) +[2023-10-09 12:27:28,118][86121] Updated weights for policy 0, policy_version 5800 (0.0009) +[2023-10-09 12:27:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 11894784. Throughput: 0: 1795.7, 1: 1824.2. Samples: 2979532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:27:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.830')] +[2023-10-09 12:27:28,490][86121] Updated weights for policy 0, policy_version 5810 (0.0010) +[2023-10-09 12:27:28,863][86121] Updated weights for policy 0, policy_version 5820 (0.0010) +[2023-10-09 12:27:30,766][86122] Updated weights for policy 1, policy_version 5830 (0.0007) +[2023-10-09 12:27:31,131][86122] Updated weights for policy 1, policy_version 5840 (0.0008) +[2023-10-09 12:27:31,506][86122] Updated weights for policy 1, policy_version 5850 (0.0009) +[2023-10-09 12:27:32,781][86121] Updated weights for policy 0, policy_version 5830 (0.0008) +[2023-10-09 12:27:33,154][86121] Updated weights for policy 0, policy_version 5840 (0.0009) +[2023-10-09 12:27:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 11960320. Throughput: 0: 1793.3, 1: 1823.6. Samples: 3000452. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) +[2023-10-09 12:27:33,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.820')] +[2023-10-09 12:27:33,527][86121] Updated weights for policy 0, policy_version 5850 (0.0008) +[2023-10-09 12:27:35,066][86122] Updated weights for policy 1, policy_version 5860 (0.0008) +[2023-10-09 12:27:35,432][86122] Updated weights for policy 1, policy_version 5870 (0.0009) +[2023-10-09 12:27:35,805][86122] Updated weights for policy 1, policy_version 5880 (0.0010) +[2023-10-09 12:27:37,149][86121] Updated weights for policy 0, policy_version 5860 (0.0009) +[2023-10-09 12:27:37,518][86121] Updated weights for policy 0, policy_version 5870 (0.0011) +[2023-10-09 12:27:37,895][86121] Updated weights for policy 0, policy_version 5880 (0.0010) +[2023-10-09 12:27:38,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12058624. Throughput: 0: 1799.9, 1: 1822.2. Samples: 3022078. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) +[2023-10-09 12:27:38,399][85186] Avg episode reward: [(0, '9.920'), (1, '9.810')] +[2023-10-09 12:27:39,603][86122] Updated weights for policy 1, policy_version 5890 (0.0009) +[2023-10-09 12:27:39,984][86122] Updated weights for policy 1, policy_version 5900 (0.0008) +[2023-10-09 12:27:40,347][86122] Updated weights for policy 1, policy_version 5910 (0.0008) +[2023-10-09 12:27:40,710][86122] Updated weights for policy 1, policy_version 5920 (0.0007) +[2023-10-09 12:27:41,590][86121] Updated weights for policy 0, policy_version 5890 (0.0007) +[2023-10-09 12:27:41,967][86121] Updated weights for policy 0, policy_version 5900 (0.0009) +[2023-10-09 12:27:42,322][86121] Updated weights for policy 0, policy_version 5910 (0.0010) +[2023-10-09 12:27:42,693][86121] Updated weights for policy 0, policy_version 5920 (0.0008) +[2023-10-09 12:27:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12124160. Throughput: 0: 1792.2, 1: 1817.6. Samples: 3033002. Policy #0 lag: (min: 31.0, avg: 39.4, max: 63.0) +[2023-10-09 12:27:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.790')] +[2023-10-09 12:27:44,586][86122] Updated weights for policy 1, policy_version 5930 (0.0008) +[2023-10-09 12:27:44,949][86122] Updated weights for policy 1, policy_version 5940 (0.0007) +[2023-10-09 12:27:45,315][86122] Updated weights for policy 1, policy_version 5950 (0.0009) +[2023-10-09 12:27:46,335][86121] Updated weights for policy 0, policy_version 5930 (0.0009) +[2023-10-09 12:27:46,705][86121] Updated weights for policy 0, policy_version 5940 (0.0011) +[2023-10-09 12:27:47,075][86121] Updated weights for policy 0, policy_version 5950 (0.0011) +[2023-10-09 12:27:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 12189696. Throughput: 0: 1808.7, 1: 1810.9. Samples: 3054688. Policy #0 lag: (min: 17.0, avg: 33.0, max: 49.0) +[2023-10-09 12:27:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.790')] +[2023-10-09 12:27:48,903][86122] Updated weights for policy 1, policy_version 5960 (0.0007) +[2023-10-09 12:27:49,277][86122] Updated weights for policy 1, policy_version 5970 (0.0007) +[2023-10-09 12:27:49,636][86122] Updated weights for policy 1, policy_version 5980 (0.0007) +[2023-10-09 12:27:50,883][86121] Updated weights for policy 0, policy_version 5960 (0.0010) +[2023-10-09 12:27:51,257][86121] Updated weights for policy 0, policy_version 5970 (0.0008) +[2023-10-09 12:27:51,629][86121] Updated weights for policy 0, policy_version 5980 (0.0010) +[2023-10-09 12:27:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12255232. Throughput: 0: 1797.9, 1: 1807.5. Samples: 3076960. Policy #0 lag: (min: 17.0, avg: 33.0, max: 49.0) +[2023-10-09 12:27:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.790')] +[2023-10-09 12:27:53,522][86122] Updated weights for policy 1, policy_version 5990 (0.0009) +[2023-10-09 12:27:53,892][86122] Updated weights for policy 1, policy_version 6000 (0.0009) +[2023-10-09 12:27:54,264][86122] Updated weights for policy 1, policy_version 6010 (0.0009) +[2023-10-09 12:27:55,121][86121] Updated weights for policy 0, policy_version 5990 (0.0008) +[2023-10-09 12:27:55,489][86121] Updated weights for policy 0, policy_version 6000 (0.0010) +[2023-10-09 12:27:55,854][86121] Updated weights for policy 0, policy_version 6010 (0.0007) +[2023-10-09 12:27:57,960][86122] Updated weights for policy 1, policy_version 6020 (0.0008) +[2023-10-09 12:27:58,321][86122] Updated weights for policy 1, policy_version 6030 (0.0008) +[2023-10-09 12:27:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12320768. Throughput: 0: 1809.4, 1: 1805.8. Samples: 3087162. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:27:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.790')] +[2023-10-09 12:27:58,684][86122] Updated weights for policy 1, policy_version 6040 (0.0009) +[2023-10-09 12:27:59,739][86121] Updated weights for policy 0, policy_version 6020 (0.0009) +[2023-10-09 12:28:00,103][86121] Updated weights for policy 0, policy_version 6030 (0.0007) +[2023-10-09 12:28:00,469][86121] Updated weights for policy 0, policy_version 6040 (0.0007) +[2023-10-09 12:28:02,417][86122] Updated weights for policy 1, policy_version 6050 (0.0008) +[2023-10-09 12:28:02,792][86122] Updated weights for policy 1, policy_version 6060 (0.0011) +[2023-10-09 12:28:03,166][86122] Updated weights for policy 1, policy_version 6070 (0.0010) +[2023-10-09 12:28:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12386304. Throughput: 0: 1804.2, 1: 1808.6. Samples: 3109538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:28:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.790')] +[2023-10-09 12:28:03,525][86122] Updated weights for policy 1, policy_version 6080 (0.0010) +[2023-10-09 12:28:04,146][86121] Updated weights for policy 0, policy_version 6050 (0.0010) +[2023-10-09 12:28:04,508][86121] Updated weights for policy 0, policy_version 6060 (0.0009) +[2023-10-09 12:28:04,882][86121] Updated weights for policy 0, policy_version 6070 (0.0010) +[2023-10-09 12:28:05,256][86121] Updated weights for policy 0, policy_version 6080 (0.0007) +[2023-10-09 12:28:07,138][86122] Updated weights for policy 1, policy_version 6090 (0.0011) +[2023-10-09 12:28:07,494][86122] Updated weights for policy 1, policy_version 6100 (0.0008) +[2023-10-09 12:28:07,864][86122] Updated weights for policy 1, policy_version 6110 (0.0007) +[2023-10-09 12:28:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12484608. Throughput: 0: 1807.0, 1: 1820.8. Samples: 3131412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:28:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.810')] +[2023-10-09 12:28:08,968][86121] Updated weights for policy 0, policy_version 6090 (0.0008) +[2023-10-09 12:28:09,327][86121] Updated weights for policy 0, policy_version 6100 (0.0007) +[2023-10-09 12:28:09,702][86121] Updated weights for policy 0, policy_version 6110 (0.0007) +[2023-10-09 12:28:11,560][86122] Updated weights for policy 1, policy_version 6120 (0.0010) +[2023-10-09 12:28:11,944][86122] Updated weights for policy 1, policy_version 6130 (0.0008) +[2023-10-09 12:28:12,317][86122] Updated weights for policy 1, policy_version 6140 (0.0008) +[2023-10-09 12:28:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 12550144. Throughput: 0: 1810.0, 1: 1815.0. Samples: 3142660. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:28:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.840')] +[2023-10-09 12:28:13,609][86121] Updated weights for policy 0, policy_version 6120 (0.0009) +[2023-10-09 12:28:13,994][86121] Updated weights for policy 0, policy_version 6130 (0.0008) +[2023-10-09 12:28:14,361][86121] Updated weights for policy 0, policy_version 6140 (0.0007) +[2023-10-09 12:28:15,964][86122] Updated weights for policy 1, policy_version 6150 (0.0008) +[2023-10-09 12:28:16,331][86122] Updated weights for policy 1, policy_version 6160 (0.0007) +[2023-10-09 12:28:16,696][86122] Updated weights for policy 1, policy_version 6170 (0.0007) +[2023-10-09 12:28:18,024][86121] Updated weights for policy 0, policy_version 6150 (0.0009) +[2023-10-09 12:28:18,395][86121] Updated weights for policy 0, policy_version 6160 (0.0010) +[2023-10-09 12:28:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 12615680. Throughput: 0: 1810.1, 1: 1816.0. Samples: 3163630. Policy #0 lag: (min: 10.0, avg: 10.0, max: 14.0) +[2023-10-09 12:28:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.840')] +[2023-10-09 12:28:18,755][86121] Updated weights for policy 0, policy_version 6170 (0.0010) +[2023-10-09 12:28:20,330][86122] Updated weights for policy 1, policy_version 6180 (0.0007) +[2023-10-09 12:28:20,693][86122] Updated weights for policy 1, policy_version 6190 (0.0007) +[2023-10-09 12:28:21,053][86122] Updated weights for policy 1, policy_version 6200 (0.0008) +[2023-10-09 12:28:22,281][86121] Updated weights for policy 0, policy_version 6180 (0.0009) +[2023-10-09 12:28:22,654][86121] Updated weights for policy 0, policy_version 6190 (0.0009) +[2023-10-09 12:28:23,027][86121] Updated weights for policy 0, policy_version 6200 (0.0008) +[2023-10-09 12:28:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 12713984. Throughput: 0: 1812.6, 1: 1818.0. Samples: 3185454. Policy #0 lag: (min: 30.0, avg: 35.6, max: 62.0) +[2023-10-09 12:28:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.840')] +[2023-10-09 12:28:24,759][86122] Updated weights for policy 1, policy_version 6210 (0.0008) +[2023-10-09 12:28:25,123][86122] Updated weights for policy 1, policy_version 6220 (0.0007) +[2023-10-09 12:28:25,483][86122] Updated weights for policy 1, policy_version 6230 (0.0007) +[2023-10-09 12:28:25,849][86122] Updated weights for policy 1, policy_version 6240 (0.0008) +[2023-10-09 12:28:26,815][86121] Updated weights for policy 0, policy_version 6210 (0.0010) +[2023-10-09 12:28:27,193][86121] Updated weights for policy 0, policy_version 6220 (0.0008) +[2023-10-09 12:28:27,556][86121] Updated weights for policy 0, policy_version 6230 (0.0010) +[2023-10-09 12:28:27,933][86121] Updated weights for policy 0, policy_version 6240 (0.0009) +[2023-10-09 12:28:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12779520. Throughput: 0: 1809.6, 1: 1824.3. Samples: 3196528. Policy #0 lag: (min: 30.0, avg: 35.6, max: 62.0) +[2023-10-09 12:28:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.850')] +[2023-10-09 12:28:29,357][86122] Updated weights for policy 1, policy_version 6250 (0.0011) +[2023-10-09 12:28:29,716][86122] Updated weights for policy 1, policy_version 6260 (0.0011) +[2023-10-09 12:28:30,087][86122] Updated weights for policy 1, policy_version 6270 (0.0010) +[2023-10-09 12:28:31,556][86121] Updated weights for policy 0, policy_version 6250 (0.0008) +[2023-10-09 12:28:31,931][86121] Updated weights for policy 0, policy_version 6260 (0.0007) +[2023-10-09 12:28:32,295][86121] Updated weights for policy 0, policy_version 6270 (0.0008) +[2023-10-09 12:28:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 12845056. Throughput: 0: 1813.7, 1: 1821.9. Samples: 3218292. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) +[2023-10-09 12:28:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.860')] +[2023-10-09 12:28:33,991][86122] Updated weights for policy 1, policy_version 6280 (0.0009) +[2023-10-09 12:28:34,359][86122] Updated weights for policy 1, policy_version 6290 (0.0009) +[2023-10-09 12:28:34,728][86122] Updated weights for policy 1, policy_version 6300 (0.0009) +[2023-10-09 12:28:35,982][86121] Updated weights for policy 0, policy_version 6280 (0.0012) +[2023-10-09 12:28:36,353][86121] Updated weights for policy 0, policy_version 6290 (0.0009) +[2023-10-09 12:28:36,725][86121] Updated weights for policy 0, policy_version 6300 (0.0008) +[2023-10-09 12:28:38,350][86122] Updated weights for policy 1, policy_version 6310 (0.0009) +[2023-10-09 12:28:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12910592. Throughput: 0: 1807.7, 1: 1829.7. Samples: 3240644. Policy #0 lag: (min: 31.0, avg: 41.7, max: 63.0) +[2023-10-09 12:28:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.870')] +[2023-10-09 12:28:38,718][86122] Updated weights for policy 1, policy_version 6320 (0.0009) +[2023-10-09 12:28:39,086][86122] Updated weights for policy 1, policy_version 6330 (0.0010) +[2023-10-09 12:28:40,395][86121] Updated weights for policy 0, policy_version 6310 (0.0010) +[2023-10-09 12:28:40,776][86121] Updated weights for policy 0, policy_version 6320 (0.0009) +[2023-10-09 12:28:41,146][86121] Updated weights for policy 0, policy_version 6330 (0.0008) +[2023-10-09 12:28:42,765][86122] Updated weights for policy 1, policy_version 6340 (0.0009) +[2023-10-09 12:28:43,126][86122] Updated weights for policy 1, policy_version 6350 (0.0007) +[2023-10-09 12:28:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 12976128. Throughput: 0: 1814.0, 1: 1831.1. Samples: 3251192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:28:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.860')] +[2023-10-09 12:28:43,484][86122] Updated weights for policy 1, policy_version 6360 (0.0007) +[2023-10-09 12:28:44,808][86121] Updated weights for policy 0, policy_version 6340 (0.0007) +[2023-10-09 12:28:45,181][86121] Updated weights for policy 0, policy_version 6350 (0.0007) +[2023-10-09 12:28:45,549][86121] Updated weights for policy 0, policy_version 6360 (0.0008) +[2023-10-09 12:28:47,155][86122] Updated weights for policy 1, policy_version 6370 (0.0009) +[2023-10-09 12:28:47,523][86122] Updated weights for policy 1, policy_version 6380 (0.0008) +[2023-10-09 12:28:47,891][86122] Updated weights for policy 1, policy_version 6390 (0.0008) +[2023-10-09 12:28:48,264][86122] Updated weights for policy 1, policy_version 6400 (0.0009) +[2023-10-09 12:28:48,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 13074432. Throughput: 0: 1818.1, 1: 1828.9. Samples: 3273652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:28:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.870')] +[2023-10-09 12:28:49,207][86121] Updated weights for policy 0, policy_version 6370 (0.0009) +[2023-10-09 12:28:49,574][86121] Updated weights for policy 0, policy_version 6380 (0.0007) +[2023-10-09 12:28:49,937][86121] Updated weights for policy 0, policy_version 6390 (0.0009) +[2023-10-09 12:28:50,306][86121] Updated weights for policy 0, policy_version 6400 (0.0009) +[2023-10-09 12:28:52,032][86122] Updated weights for policy 1, policy_version 6410 (0.0008) +[2023-10-09 12:28:52,399][86122] Updated weights for policy 1, policy_version 6420 (0.0007) +[2023-10-09 12:28:52,774][86122] Updated weights for policy 1, policy_version 6430 (0.0010) +[2023-10-09 12:28:53,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13139968. Throughput: 0: 1814.2, 1: 1821.3. Samples: 3295012. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 12:28:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.860')] +[2023-10-09 12:28:53,968][86121] Updated weights for policy 0, policy_version 6410 (0.0010) +[2023-10-09 12:28:54,337][86121] Updated weights for policy 0, policy_version 6420 (0.0009) +[2023-10-09 12:28:54,702][86121] Updated weights for policy 0, policy_version 6430 (0.0009) +[2023-10-09 12:28:56,527][86122] Updated weights for policy 1, policy_version 6440 (0.0009) +[2023-10-09 12:28:56,911][86122] Updated weights for policy 1, policy_version 6450 (0.0007) +[2023-10-09 12:28:57,273][86122] Updated weights for policy 1, policy_version 6460 (0.0007) +[2023-10-09 12:28:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13205504. Throughput: 0: 1811.7, 1: 1824.8. Samples: 3306304. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 12:28:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.870')] +[2023-10-09 12:28:58,461][86121] Updated weights for policy 0, policy_version 6440 (0.0009) +[2023-10-09 12:28:58,820][86121] Updated weights for policy 0, policy_version 6450 (0.0008) +[2023-10-09 12:28:59,192][86121] Updated weights for policy 0, policy_version 6460 (0.0007) +[2023-10-09 12:29:00,968][86122] Updated weights for policy 1, policy_version 6470 (0.0008) +[2023-10-09 12:29:01,334][86122] Updated weights for policy 1, policy_version 6480 (0.0008) +[2023-10-09 12:29:01,695][86122] Updated weights for policy 1, policy_version 6490 (0.0008) +[2023-10-09 12:29:02,787][86121] Updated weights for policy 0, policy_version 6470 (0.0009) +[2023-10-09 12:29:03,159][86121] Updated weights for policy 0, policy_version 6480 (0.0008) +[2023-10-09 12:29:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 13271040. Throughput: 0: 1827.2, 1: 1825.0. Samples: 3327980. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:29:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.880')] +[2023-10-09 12:29:03,534][86121] Updated weights for policy 0, policy_version 6490 (0.0008) +[2023-10-09 12:29:05,350][86122] Updated weights for policy 1, policy_version 6500 (0.0009) +[2023-10-09 12:29:05,713][86122] Updated weights for policy 1, policy_version 6510 (0.0009) +[2023-10-09 12:29:06,075][86122] Updated weights for policy 1, policy_version 6520 (0.0009) +[2023-10-09 12:29:07,355][86121] Updated weights for policy 0, policy_version 6500 (0.0008) +[2023-10-09 12:29:07,740][86121] Updated weights for policy 0, policy_version 6510 (0.0008) +[2023-10-09 12:29:08,098][86121] Updated weights for policy 0, policy_version 6520 (0.0007) +[2023-10-09 12:29:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 13369344. Throughput: 0: 1823.6, 1: 1829.5. Samples: 3349844. Policy #0 lag: (min: 0.0, avg: 26.4, max: 32.0) +[2023-10-09 12:29:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.890')] +[2023-10-09 12:29:09,640][86122] Updated weights for policy 1, policy_version 6530 (0.0007) +[2023-10-09 12:29:10,000][86122] Updated weights for policy 1, policy_version 6540 (0.0007) +[2023-10-09 12:29:10,355][86122] Updated weights for policy 1, policy_version 6550 (0.0008) +[2023-10-09 12:29:10,720][86122] Updated weights for policy 1, policy_version 6560 (0.0008) +[2023-10-09 12:29:11,774][86121] Updated weights for policy 0, policy_version 6530 (0.0008) +[2023-10-09 12:29:12,150][86121] Updated weights for policy 0, policy_version 6540 (0.0008) +[2023-10-09 12:29:12,517][86121] Updated weights for policy 0, policy_version 6550 (0.0008) +[2023-10-09 12:29:12,890][86121] Updated weights for policy 0, policy_version 6560 (0.0007) +[2023-10-09 12:29:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13434880. Throughput: 0: 1824.4, 1: 1827.0. Samples: 3360844. Policy #0 lag: (min: 0.0, avg: 26.4, max: 32.0) +[2023-10-09 12:29:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.890')] +[2023-10-09 12:29:14,496][86122] Updated weights for policy 1, policy_version 6570 (0.0008) +[2023-10-09 12:29:14,853][86122] Updated weights for policy 1, policy_version 6580 (0.0010) +[2023-10-09 12:29:15,219][86122] Updated weights for policy 1, policy_version 6590 (0.0010) +[2023-10-09 12:29:16,388][86121] Updated weights for policy 0, policy_version 6570 (0.0009) +[2023-10-09 12:29:16,765][86121] Updated weights for policy 0, policy_version 6580 (0.0007) +[2023-10-09 12:29:17,125][86121] Updated weights for policy 0, policy_version 6590 (0.0007) +[2023-10-09 12:29:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 13500416. Throughput: 0: 1824.5, 1: 1827.5. Samples: 3382632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:29:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.900')] +[2023-10-09 12:29:18,908][86122] Updated weights for policy 1, policy_version 6600 (0.0009) +[2023-10-09 12:29:19,281][86122] Updated weights for policy 1, policy_version 6610 (0.0008) +[2023-10-09 12:29:19,648][86122] Updated weights for policy 1, policy_version 6620 (0.0009) +[2023-10-09 12:29:20,849][86121] Updated weights for policy 0, policy_version 6600 (0.0009) +[2023-10-09 12:29:21,218][86121] Updated weights for policy 0, policy_version 6610 (0.0008) +[2023-10-09 12:29:21,589][86121] Updated weights for policy 0, policy_version 6620 (0.0007) +[2023-10-09 12:29:23,181][86122] Updated weights for policy 1, policy_version 6630 (0.0009) +[2023-10-09 12:29:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 13565952. Throughput: 0: 1830.4, 1: 1826.2. Samples: 3405192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:29:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:29:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000006624_6782976.pth... +[2023-10-09 12:29:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000004928_5046272.pth +[2023-10-09 12:29:23,548][86122] Updated weights for policy 1, policy_version 6640 (0.0008) +[2023-10-09 12:29:23,914][86122] Updated weights for policy 1, policy_version 6650 (0.0008) +[2023-10-09 12:29:24,131][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth... +[2023-10-09 12:29:24,171][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000004928_5046272.pth +[2023-10-09 12:29:25,184][86121] Updated weights for policy 0, policy_version 6630 (0.0008) +[2023-10-09 12:29:25,559][86121] Updated weights for policy 0, policy_version 6640 (0.0008) +[2023-10-09 12:29:25,931][86121] Updated weights for policy 0, policy_version 6650 (0.0009) +[2023-10-09 12:29:27,460][86122] Updated weights for policy 1, policy_version 6660 (0.0008) +[2023-10-09 12:29:27,823][86122] Updated weights for policy 1, policy_version 6670 (0.0007) +[2023-10-09 12:29:28,187][86122] Updated weights for policy 1, policy_version 6680 (0.0007) +[2023-10-09 12:29:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 13631488. Throughput: 0: 1826.4, 1: 1830.2. Samples: 3415740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:29:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 12:29:29,631][86121] Updated weights for policy 0, policy_version 6660 (0.0010) +[2023-10-09 12:29:30,002][86121] Updated weights for policy 0, policy_version 6670 (0.0010) +[2023-10-09 12:29:30,368][86121] Updated weights for policy 0, policy_version 6680 (0.0010) +[2023-10-09 12:29:32,078][86122] Updated weights for policy 1, policy_version 6690 (0.0007) +[2023-10-09 12:29:32,439][86122] Updated weights for policy 1, policy_version 6700 (0.0008) +[2023-10-09 12:29:32,801][86122] Updated weights for policy 1, policy_version 6710 (0.0009) +[2023-10-09 12:29:33,174][86122] Updated weights for policy 1, policy_version 6720 (0.0008) +[2023-10-09 12:29:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13729792. Throughput: 0: 1821.6, 1: 1829.7. Samples: 3437960. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:29:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 12:29:34,262][86121] Updated weights for policy 0, policy_version 6690 (0.0009) +[2023-10-09 12:29:34,630][86121] Updated weights for policy 0, policy_version 6700 (0.0007) +[2023-10-09 12:29:34,998][86121] Updated weights for policy 0, policy_version 6710 (0.0008) +[2023-10-09 12:29:35,368][86121] Updated weights for policy 0, policy_version 6720 (0.0008) +[2023-10-09 12:29:36,841][86122] Updated weights for policy 1, policy_version 6730 (0.0008) +[2023-10-09 12:29:37,214][86122] Updated weights for policy 1, policy_version 6740 (0.0010) +[2023-10-09 12:29:37,576][86122] Updated weights for policy 1, policy_version 6750 (0.0008) +[2023-10-09 12:29:38,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 13795328. Throughput: 0: 1824.8, 1: 1829.1. Samples: 3459434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:29:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 12:29:39,013][86121] Updated weights for policy 0, policy_version 6730 (0.0007) +[2023-10-09 12:29:39,377][86121] Updated weights for policy 0, policy_version 6740 (0.0007) +[2023-10-09 12:29:39,747][86121] Updated weights for policy 0, policy_version 6750 (0.0007) +[2023-10-09 12:29:41,466][86122] Updated weights for policy 1, policy_version 6760 (0.0009) +[2023-10-09 12:29:41,832][86122] Updated weights for policy 1, policy_version 6770 (0.0008) +[2023-10-09 12:29:42,201][86122] Updated weights for policy 1, policy_version 6780 (0.0007) +[2023-10-09 12:29:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 13860864. Throughput: 0: 1827.4, 1: 1827.8. Samples: 3470788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:29:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 12:29:43,526][86121] Updated weights for policy 0, policy_version 6760 (0.0008) +[2023-10-09 12:29:43,908][86121] Updated weights for policy 0, policy_version 6770 (0.0008) +[2023-10-09 12:29:44,277][86121] Updated weights for policy 0, policy_version 6780 (0.0009) +[2023-10-09 12:29:45,803][86122] Updated weights for policy 1, policy_version 6790 (0.0008) +[2023-10-09 12:29:46,159][86122] Updated weights for policy 1, policy_version 6800 (0.0007) +[2023-10-09 12:29:46,520][86122] Updated weights for policy 1, policy_version 6810 (0.0008) +[2023-10-09 12:29:47,862][86121] Updated weights for policy 0, policy_version 6790 (0.0009) +[2023-10-09 12:29:48,228][86121] Updated weights for policy 0, policy_version 6800 (0.0010) +[2023-10-09 12:29:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 13926400. Throughput: 0: 1814.3, 1: 1829.1. Samples: 3491932. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) +[2023-10-09 12:29:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 12:29:48,604][86121] Updated weights for policy 0, policy_version 6810 (0.0008) +[2023-10-09 12:29:49,954][86122] Updated weights for policy 1, policy_version 6820 (0.0010) +[2023-10-09 12:29:50,325][86122] Updated weights for policy 1, policy_version 6830 (0.0009) +[2023-10-09 12:29:50,693][86122] Updated weights for policy 1, policy_version 6840 (0.0008) +[2023-10-09 12:29:52,348][86121] Updated weights for policy 0, policy_version 6820 (0.0008) +[2023-10-09 12:29:52,719][86121] Updated weights for policy 0, policy_version 6830 (0.0009) +[2023-10-09 12:29:53,082][86121] Updated weights for policy 0, policy_version 6840 (0.0007) +[2023-10-09 12:29:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 14024704. Throughput: 0: 1820.6, 1: 1829.3. Samples: 3514088. Policy #0 lag: (min: 14.0, avg: 14.7, max: 31.0) +[2023-10-09 12:29:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 12:29:54,169][86122] Updated weights for policy 1, policy_version 6850 (0.0007) +[2023-10-09 12:29:54,545][86122] Updated weights for policy 1, policy_version 6860 (0.0008) +[2023-10-09 12:29:54,915][86122] Updated weights for policy 1, policy_version 6870 (0.0008) +[2023-10-09 12:29:55,274][86122] Updated weights for policy 1, policy_version 6880 (0.0010) +[2023-10-09 12:29:56,858][86121] Updated weights for policy 0, policy_version 6850 (0.0008) +[2023-10-09 12:29:57,221][86121] Updated weights for policy 0, policy_version 6860 (0.0008) +[2023-10-09 12:29:57,602][86121] Updated weights for policy 0, policy_version 6870 (0.0010) +[2023-10-09 12:29:57,980][86121] Updated weights for policy 0, policy_version 6880 (0.0008) +[2023-10-09 12:29:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14090240. Throughput: 0: 1813.6, 1: 1830.1. Samples: 3524812. Policy #0 lag: (min: 14.0, avg: 14.7, max: 31.0) +[2023-10-09 12:29:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.950')] +[2023-10-09 12:29:58,897][86122] Updated weights for policy 1, policy_version 6890 (0.0009) +[2023-10-09 12:29:59,270][86122] Updated weights for policy 1, policy_version 6900 (0.0009) +[2023-10-09 12:29:59,640][86122] Updated weights for policy 1, policy_version 6910 (0.0008) +[2023-10-09 12:30:01,744][86121] Updated weights for policy 0, policy_version 6890 (0.0008) +[2023-10-09 12:30:02,114][86121] Updated weights for policy 0, policy_version 6900 (0.0007) +[2023-10-09 12:30:02,495][86121] Updated weights for policy 0, policy_version 6910 (0.0009) +[2023-10-09 12:30:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 14155776. Throughput: 0: 1815.4, 1: 1830.2. Samples: 3546684. Policy #0 lag: (min: 24.0, avg: 51.3, max: 56.0) +[2023-10-09 12:30:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:30:03,431][86122] Updated weights for policy 1, policy_version 6920 (0.0007) +[2023-10-09 12:30:03,807][86122] Updated weights for policy 1, policy_version 6930 (0.0007) +[2023-10-09 12:30:04,166][86122] Updated weights for policy 1, policy_version 6940 (0.0009) +[2023-10-09 12:30:06,368][86121] Updated weights for policy 0, policy_version 6920 (0.0008) +[2023-10-09 12:30:06,743][86121] Updated weights for policy 0, policy_version 6930 (0.0009) +[2023-10-09 12:30:07,115][86121] Updated weights for policy 0, policy_version 6940 (0.0009) +[2023-10-09 12:30:07,885][86122] Updated weights for policy 1, policy_version 6950 (0.0008) +[2023-10-09 12:30:08,241][86122] Updated weights for policy 1, policy_version 6960 (0.0008) +[2023-10-09 12:30:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 14221312. Throughput: 0: 1804.0, 1: 1828.1. Samples: 3568634. Policy #0 lag: (min: 24.0, avg: 51.3, max: 56.0) +[2023-10-09 12:30:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:30:08,609][86122] Updated weights for policy 1, policy_version 6970 (0.0008) +[2023-10-09 12:30:10,783][86121] Updated weights for policy 0, policy_version 6950 (0.0009) +[2023-10-09 12:30:11,139][86121] Updated weights for policy 0, policy_version 6960 (0.0008) +[2023-10-09 12:30:11,505][86121] Updated weights for policy 0, policy_version 6970 (0.0007) +[2023-10-09 12:30:12,407][86122] Updated weights for policy 1, policy_version 6980 (0.0009) +[2023-10-09 12:30:12,779][86122] Updated weights for policy 1, policy_version 6990 (0.0008) +[2023-10-09 12:30:13,136][86122] Updated weights for policy 1, policy_version 7000 (0.0007) +[2023-10-09 12:30:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 14286848. Throughput: 0: 1814.8, 1: 1829.4. Samples: 3579728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:30:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:30:15,053][86121] Updated weights for policy 0, policy_version 6980 (0.0008) +[2023-10-09 12:30:15,422][86121] Updated weights for policy 0, policy_version 6990 (0.0009) +[2023-10-09 12:30:15,796][86121] Updated weights for policy 0, policy_version 7000 (0.0008) +[2023-10-09 12:30:16,737][86122] Updated weights for policy 1, policy_version 7010 (0.0009) +[2023-10-09 12:30:17,095][86122] Updated weights for policy 1, policy_version 7020 (0.0007) +[2023-10-09 12:30:17,470][86122] Updated weights for policy 1, policy_version 7030 (0.0007) +[2023-10-09 12:30:17,840][86122] Updated weights for policy 1, policy_version 7040 (0.0008) +[2023-10-09 12:30:18,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14385152. Throughput: 0: 1809.2, 1: 1822.4. Samples: 3601384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:30:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:30:19,354][86121] Updated weights for policy 0, policy_version 7010 (0.0008) +[2023-10-09 12:30:19,718][86121] Updated weights for policy 0, policy_version 7020 (0.0007) +[2023-10-09 12:30:20,089][86121] Updated weights for policy 0, policy_version 7030 (0.0007) +[2023-10-09 12:30:20,464][86121] Updated weights for policy 0, policy_version 7040 (0.0008) +[2023-10-09 12:30:21,782][86122] Updated weights for policy 1, policy_version 7050 (0.0008) +[2023-10-09 12:30:22,147][86122] Updated weights for policy 1, policy_version 7060 (0.0009) +[2023-10-09 12:30:22,501][86122] Updated weights for policy 1, policy_version 7070 (0.0009) +[2023-10-09 12:30:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 14450688. Throughput: 0: 1805.4, 1: 1828.2. Samples: 3622946. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:30:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:30:24,154][86121] Updated weights for policy 0, policy_version 7050 (0.0009) +[2023-10-09 12:30:24,520][86121] Updated weights for policy 0, policy_version 7060 (0.0008) +[2023-10-09 12:30:24,889][86121] Updated weights for policy 0, policy_version 7070 (0.0009) +[2023-10-09 12:30:26,288][86122] Updated weights for policy 1, policy_version 7080 (0.0009) +[2023-10-09 12:30:26,649][86122] Updated weights for policy 1, policy_version 7090 (0.0008) +[2023-10-09 12:30:27,017][86122] Updated weights for policy 1, policy_version 7100 (0.0008) +[2023-10-09 12:30:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14516224. Throughput: 0: 1801.8, 1: 1825.6. Samples: 3634022. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:30:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:30:28,707][86121] Updated weights for policy 0, policy_version 7080 (0.0008) +[2023-10-09 12:30:29,081][86121] Updated weights for policy 0, policy_version 7090 (0.0008) +[2023-10-09 12:30:29,451][86121] Updated weights for policy 0, policy_version 7100 (0.0007) +[2023-10-09 12:30:30,716][86122] Updated weights for policy 1, policy_version 7110 (0.0008) +[2023-10-09 12:30:31,079][86122] Updated weights for policy 1, policy_version 7120 (0.0010) +[2023-10-09 12:30:31,444][86122] Updated weights for policy 1, policy_version 7130 (0.0009) +[2023-10-09 12:30:33,202][86121] Updated weights for policy 0, policy_version 7110 (0.0007) +[2023-10-09 12:30:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 14581760. Throughput: 0: 1803.1, 1: 1819.2. Samples: 3654936. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 12:30:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:30:33,570][86121] Updated weights for policy 0, policy_version 7120 (0.0007) +[2023-10-09 12:30:33,939][86121] Updated weights for policy 0, policy_version 7130 (0.0007) +[2023-10-09 12:30:35,077][86122] Updated weights for policy 1, policy_version 7140 (0.0009) +[2023-10-09 12:30:35,453][86122] Updated weights for policy 1, policy_version 7150 (0.0010) +[2023-10-09 12:30:35,815][86122] Updated weights for policy 1, policy_version 7160 (0.0008) +[2023-10-09 12:30:37,551][86121] Updated weights for policy 0, policy_version 7140 (0.0007) +[2023-10-09 12:30:37,922][86121] Updated weights for policy 0, policy_version 7150 (0.0007) +[2023-10-09 12:30:38,291][86121] Updated weights for policy 0, policy_version 7160 (0.0009) +[2023-10-09 12:30:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 14647296. Throughput: 0: 1813.2, 1: 1819.0. Samples: 3677538. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 12:30:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.930')] +[2023-10-09 12:30:39,397][86122] Updated weights for policy 1, policy_version 7170 (0.0009) +[2023-10-09 12:30:39,756][86122] Updated weights for policy 1, policy_version 7180 (0.0008) +[2023-10-09 12:30:40,117][86122] Updated weights for policy 1, policy_version 7190 (0.0010) +[2023-10-09 12:30:40,480][86122] Updated weights for policy 1, policy_version 7200 (0.0009) +[2023-10-09 12:30:42,003][86121] Updated weights for policy 0, policy_version 7170 (0.0008) +[2023-10-09 12:30:42,364][86121] Updated weights for policy 0, policy_version 7180 (0.0008) +[2023-10-09 12:30:42,739][86121] Updated weights for policy 0, policy_version 7190 (0.0008) +[2023-10-09 12:30:43,103][86121] Updated weights for policy 0, policy_version 7200 (0.0008) +[2023-10-09 12:30:43,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14745600. Throughput: 0: 1811.1, 1: 1816.9. Samples: 3688074. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:30:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.930')] +[2023-10-09 12:30:44,086][86122] Updated weights for policy 1, policy_version 7210 (0.0011) +[2023-10-09 12:30:44,444][86122] Updated weights for policy 1, policy_version 7220 (0.0009) +[2023-10-09 12:30:44,816][86122] Updated weights for policy 1, policy_version 7230 (0.0009) +[2023-10-09 12:30:46,823][86121] Updated weights for policy 0, policy_version 7210 (0.0007) +[2023-10-09 12:30:47,187][86121] Updated weights for policy 0, policy_version 7220 (0.0010) +[2023-10-09 12:30:47,557][86121] Updated weights for policy 0, policy_version 7230 (0.0009) +[2023-10-09 12:30:48,354][86122] Updated weights for policy 1, policy_version 7240 (0.0011) +[2023-10-09 12:30:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 14811136. Throughput: 0: 1815.7, 1: 1822.3. Samples: 3710392. Policy #0 lag: (min: 29.0, avg: 29.1, max: 37.0) +[2023-10-09 12:30:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.920')] +[2023-10-09 12:30:48,718][86122] Updated weights for policy 1, policy_version 7250 (0.0008) +[2023-10-09 12:30:49,079][86122] Updated weights for policy 1, policy_version 7260 (0.0010) +[2023-10-09 12:30:51,366][86121] Updated weights for policy 0, policy_version 7240 (0.0008) +[2023-10-09 12:30:51,730][86121] Updated weights for policy 0, policy_version 7250 (0.0009) +[2023-10-09 12:30:52,091][86121] Updated weights for policy 0, policy_version 7260 (0.0008) +[2023-10-09 12:30:52,684][86122] Updated weights for policy 1, policy_version 7270 (0.0008) +[2023-10-09 12:30:53,051][86122] Updated weights for policy 1, policy_version 7280 (0.0007) +[2023-10-09 12:30:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 14876672. Throughput: 0: 1808.7, 1: 1817.4. Samples: 3731808. Policy #0 lag: (min: 29.0, avg: 29.1, max: 37.0) +[2023-10-09 12:30:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.920')] +[2023-10-09 12:30:53,414][86122] Updated weights for policy 1, policy_version 7290 (0.0008) +[2023-10-09 12:30:55,873][86121] Updated weights for policy 0, policy_version 7270 (0.0009) +[2023-10-09 12:30:56,233][86121] Updated weights for policy 0, policy_version 7280 (0.0007) +[2023-10-09 12:30:56,602][86121] Updated weights for policy 0, policy_version 7290 (0.0008) +[2023-10-09 12:30:57,160][86122] Updated weights for policy 1, policy_version 7300 (0.0008) +[2023-10-09 12:30:57,526][86122] Updated weights for policy 1, policy_version 7310 (0.0009) +[2023-10-09 12:30:57,889][86122] Updated weights for policy 1, policy_version 7320 (0.0009) +[2023-10-09 12:30:58,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 14974976. Throughput: 0: 1810.7, 1: 1818.2. Samples: 3743030. Policy #0 lag: (min: 1.0, avg: 2.6, max: 26.0) +[2023-10-09 12:30:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.920')] +[2023-10-09 12:31:00,356][86121] Updated weights for policy 0, policy_version 7300 (0.0009) +[2023-10-09 12:31:00,724][86121] Updated weights for policy 0, policy_version 7310 (0.0008) +[2023-10-09 12:31:01,097][86121] Updated weights for policy 0, policy_version 7320 (0.0009) +[2023-10-09 12:31:01,733][86122] Updated weights for policy 1, policy_version 7330 (0.0010) +[2023-10-09 12:31:02,095][86122] Updated weights for policy 1, policy_version 7340 (0.0008) +[2023-10-09 12:31:02,470][86122] Updated weights for policy 1, policy_version 7350 (0.0010) +[2023-10-09 12:31:02,830][86122] Updated weights for policy 1, policy_version 7360 (0.0007) +[2023-10-09 12:31:03,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 15040512. Throughput: 0: 1801.4, 1: 1818.0. Samples: 3764258. Policy #0 lag: (min: 1.0, avg: 2.6, max: 26.0) +[2023-10-09 12:31:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:31:04,621][86121] Updated weights for policy 0, policy_version 7330 (0.0007) +[2023-10-09 12:31:04,977][86121] Updated weights for policy 0, policy_version 7340 (0.0007) +[2023-10-09 12:31:05,346][86121] Updated weights for policy 0, policy_version 7350 (0.0009) +[2023-10-09 12:31:05,714][86121] Updated weights for policy 0, policy_version 7360 (0.0012) +[2023-10-09 12:31:06,645][86122] Updated weights for policy 1, policy_version 7370 (0.0008) +[2023-10-09 12:31:07,008][86122] Updated weights for policy 1, policy_version 7380 (0.0007) +[2023-10-09 12:31:07,373][86122] Updated weights for policy 1, policy_version 7390 (0.0007) +[2023-10-09 12:31:08,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 15106048. Throughput: 0: 1806.6, 1: 1815.9. Samples: 3785958. Policy #0 lag: (min: 13.0, avg: 24.0, max: 45.0) +[2023-10-09 12:31:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 12:31:09,460][86121] Updated weights for policy 0, policy_version 7370 (0.0007) +[2023-10-09 12:31:09,825][86121] Updated weights for policy 0, policy_version 7380 (0.0007) +[2023-10-09 12:31:10,198][86121] Updated weights for policy 0, policy_version 7390 (0.0007) +[2023-10-09 12:31:11,049][86122] Updated weights for policy 1, policy_version 7400 (0.0009) +[2023-10-09 12:31:11,424][86122] Updated weights for policy 1, policy_version 7410 (0.0009) +[2023-10-09 12:31:11,794][86122] Updated weights for policy 1, policy_version 7420 (0.0010) +[2023-10-09 12:31:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15171584. Throughput: 0: 1807.7, 1: 1816.5. Samples: 3797110. Policy #0 lag: (min: 13.0, avg: 24.0, max: 45.0) +[2023-10-09 12:31:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.910')] +[2023-10-09 12:31:14,012][86121] Updated weights for policy 0, policy_version 7400 (0.0009) +[2023-10-09 12:31:14,382][86121] Updated weights for policy 0, policy_version 7410 (0.0007) +[2023-10-09 12:31:14,750][86121] Updated weights for policy 0, policy_version 7420 (0.0007) +[2023-10-09 12:31:15,461][86122] Updated weights for policy 1, policy_version 7430 (0.0011) +[2023-10-09 12:31:15,828][86122] Updated weights for policy 1, policy_version 7440 (0.0010) +[2023-10-09 12:31:16,203][86122] Updated weights for policy 1, policy_version 7450 (0.0008) +[2023-10-09 12:31:18,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 15237120. Throughput: 0: 1808.1, 1: 1824.0. Samples: 3818380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:31:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.910')] +[2023-10-09 12:31:18,400][86121] Updated weights for policy 0, policy_version 7430 (0.0008) +[2023-10-09 12:31:18,773][86121] Updated weights for policy 0, policy_version 7440 (0.0010) +[2023-10-09 12:31:19,136][86121] Updated weights for policy 0, policy_version 7450 (0.0007) +[2023-10-09 12:31:20,034][86122] Updated weights for policy 1, policy_version 7460 (0.0010) +[2023-10-09 12:31:20,400][86122] Updated weights for policy 1, policy_version 7470 (0.0008) +[2023-10-09 12:31:20,762][86122] Updated weights for policy 1, policy_version 7480 (0.0009) +[2023-10-09 12:31:22,867][86121] Updated weights for policy 0, policy_version 7460 (0.0009) +[2023-10-09 12:31:23,230][86121] Updated weights for policy 0, policy_version 7470 (0.0010) +[2023-10-09 12:31:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 15302656. Throughput: 0: 1809.0, 1: 1811.8. Samples: 3840474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:31:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.910')] +[2023-10-09 12:31:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000007488_7667712.pth... +[2023-10-09 12:31:23,441][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000005792_5931008.pth +[2023-10-09 12:31:23,593][86121] Updated weights for policy 0, policy_version 7480 (0.0009) +[2023-10-09 12:31:23,889][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000007488_7667712.pth... +[2023-10-09 12:31:23,918][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000005792_5931008.pth +[2023-10-09 12:31:24,447][86122] Updated weights for policy 1, policy_version 7490 (0.0009) +[2023-10-09 12:31:24,803][86122] Updated weights for policy 1, policy_version 7500 (0.0007) +[2023-10-09 12:31:25,170][86122] Updated weights for policy 1, policy_version 7510 (0.0010) +[2023-10-09 12:31:25,538][86122] Updated weights for policy 1, policy_version 7520 (0.0009) +[2023-10-09 12:31:27,332][86121] Updated weights for policy 0, policy_version 7490 (0.0008) +[2023-10-09 12:31:27,703][86121] Updated weights for policy 0, policy_version 7500 (0.0007) +[2023-10-09 12:31:28,070][86121] Updated weights for policy 0, policy_version 7510 (0.0007) +[2023-10-09 12:31:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15368192. Throughput: 0: 1800.6, 1: 1816.1. Samples: 3850828. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:31:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.900')] +[2023-10-09 12:31:28,443][86121] Updated weights for policy 0, policy_version 7520 (0.0009) +[2023-10-09 12:31:29,053][86122] Updated weights for policy 1, policy_version 7530 (0.0012) +[2023-10-09 12:31:29,423][86122] Updated weights for policy 1, policy_version 7540 (0.0009) +[2023-10-09 12:31:29,794][86122] Updated weights for policy 1, policy_version 7550 (0.0008) +[2023-10-09 12:31:32,311][86121] Updated weights for policy 0, policy_version 7530 (0.0008) +[2023-10-09 12:31:32,682][86121] Updated weights for policy 0, policy_version 7540 (0.0007) +[2023-10-09 12:31:33,048][86121] Updated weights for policy 0, policy_version 7550 (0.0008) +[2023-10-09 12:31:33,341][86122] Updated weights for policy 1, policy_version 7560 (0.0008) +[2023-10-09 12:31:33,397][85186] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15466496. Throughput: 0: 1810.6, 1: 1818.2. Samples: 3873688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:31:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.900')] +[2023-10-09 12:31:33,711][86122] Updated weights for policy 1, policy_version 7570 (0.0008) +[2023-10-09 12:31:34,070][86122] Updated weights for policy 1, policy_version 7580 (0.0009) +[2023-10-09 12:31:36,753][86121] Updated weights for policy 0, policy_version 7560 (0.0008) +[2023-10-09 12:31:37,120][86121] Updated weights for policy 0, policy_version 7570 (0.0008) +[2023-10-09 12:31:37,495][86121] Updated weights for policy 0, policy_version 7580 (0.0009) +[2023-10-09 12:31:37,876][86122] Updated weights for policy 1, policy_version 7590 (0.0008) +[2023-10-09 12:31:38,240][86122] Updated weights for policy 1, policy_version 7600 (0.0010) +[2023-10-09 12:31:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15532032. Throughput: 0: 1801.1, 1: 1820.5. Samples: 3894780. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:31:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.910')] +[2023-10-09 12:31:38,616][86122] Updated weights for policy 1, policy_version 7610 (0.0008) +[2023-10-09 12:31:41,354][86121] Updated weights for policy 0, policy_version 7590 (0.0010) +[2023-10-09 12:31:41,723][86121] Updated weights for policy 0, policy_version 7600 (0.0007) +[2023-10-09 12:31:42,102][86121] Updated weights for policy 0, policy_version 7610 (0.0007) +[2023-10-09 12:31:42,381][86122] Updated weights for policy 1, policy_version 7620 (0.0010) +[2023-10-09 12:31:42,754][86122] Updated weights for policy 1, policy_version 7630 (0.0007) +[2023-10-09 12:31:43,135][86122] Updated weights for policy 1, policy_version 7640 (0.0007) +[2023-10-09 12:31:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 15597568. Throughput: 0: 1808.5, 1: 1818.4. Samples: 3906240. Policy #0 lag: (min: 28.0, avg: 33.3, max: 60.0) +[2023-10-09 12:31:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 12:31:45,865][86121] Updated weights for policy 0, policy_version 7620 (0.0008) +[2023-10-09 12:31:46,244][86121] Updated weights for policy 0, policy_version 7630 (0.0009) +[2023-10-09 12:31:46,612][86121] Updated weights for policy 0, policy_version 7640 (0.0008) +[2023-10-09 12:31:46,624][86122] Updated weights for policy 1, policy_version 7650 (0.0007) +[2023-10-09 12:31:46,979][86122] Updated weights for policy 1, policy_version 7660 (0.0010) +[2023-10-09 12:31:47,343][86122] Updated weights for policy 1, policy_version 7670 (0.0010) +[2023-10-09 12:31:47,710][86122] Updated weights for policy 1, policy_version 7680 (0.0010) +[2023-10-09 12:31:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 15695872. Throughput: 0: 1800.6, 1: 1820.7. Samples: 3927218. Policy #0 lag: (min: 28.0, avg: 33.3, max: 60.0) +[2023-10-09 12:31:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 12:31:50,258][86121] Updated weights for policy 0, policy_version 7650 (0.0011) +[2023-10-09 12:31:50,640][86121] Updated weights for policy 0, policy_version 7660 (0.0012) +[2023-10-09 12:31:51,005][86121] Updated weights for policy 0, policy_version 7670 (0.0011) +[2023-10-09 12:31:51,366][86121] Updated weights for policy 0, policy_version 7680 (0.0007) +[2023-10-09 12:31:51,473][86122] Updated weights for policy 1, policy_version 7690 (0.0008) +[2023-10-09 12:31:51,838][86122] Updated weights for policy 1, policy_version 7700 (0.0009) +[2023-10-09 12:31:52,208][86122] Updated weights for policy 1, policy_version 7710 (0.0008) +[2023-10-09 12:31:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 15761408. Throughput: 0: 1791.3, 1: 1825.0. Samples: 3948690. Policy #0 lag: (min: 25.0, avg: 31.9, max: 57.0) +[2023-10-09 12:31:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.930')] +[2023-10-09 12:31:55,056][86121] Updated weights for policy 0, policy_version 7690 (0.0007) +[2023-10-09 12:31:55,432][86121] Updated weights for policy 0, policy_version 7700 (0.0010) +[2023-10-09 12:31:55,794][86121] Updated weights for policy 0, policy_version 7710 (0.0008) +[2023-10-09 12:31:56,067][86122] Updated weights for policy 1, policy_version 7720 (0.0008) +[2023-10-09 12:31:56,427][86122] Updated weights for policy 1, policy_version 7730 (0.0008) +[2023-10-09 12:31:56,800][86122] Updated weights for policy 1, policy_version 7740 (0.0009) +[2023-10-09 12:31:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15826944. Throughput: 0: 1791.8, 1: 1826.0. Samples: 3959908. Policy #0 lag: (min: 25.0, avg: 31.9, max: 57.0) +[2023-10-09 12:31:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:31:59,590][86121] Updated weights for policy 0, policy_version 7720 (0.0008) +[2023-10-09 12:31:59,967][86121] Updated weights for policy 0, policy_version 7730 (0.0010) +[2023-10-09 12:32:00,338][86121] Updated weights for policy 0, policy_version 7740 (0.0010) +[2023-10-09 12:32:00,543][86122] Updated weights for policy 1, policy_version 7750 (0.0008) +[2023-10-09 12:32:00,913][86122] Updated weights for policy 1, policy_version 7760 (0.0008) +[2023-10-09 12:32:01,272][86122] Updated weights for policy 1, policy_version 7770 (0.0009) +[2023-10-09 12:32:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 15892480. Throughput: 0: 1796.1, 1: 1822.4. Samples: 3981214. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 12:32:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 12:32:04,166][86121] Updated weights for policy 0, policy_version 7750 (0.0009) +[2023-10-09 12:32:04,523][86121] Updated weights for policy 0, policy_version 7760 (0.0009) +[2023-10-09 12:32:04,880][86122] Updated weights for policy 1, policy_version 7780 (0.0007) +[2023-10-09 12:32:04,884][86121] Updated weights for policy 0, policy_version 7770 (0.0009) +[2023-10-09 12:32:05,247][86122] Updated weights for policy 1, policy_version 7790 (0.0009) +[2023-10-09 12:32:05,606][86122] Updated weights for policy 1, policy_version 7800 (0.0010) +[2023-10-09 12:32:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 15958016. Throughput: 0: 1795.7, 1: 1834.3. Samples: 4003822. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 12:32:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 12:32:08,737][86121] Updated weights for policy 0, policy_version 7780 (0.0008) +[2023-10-09 12:32:09,093][86121] Updated weights for policy 0, policy_version 7790 (0.0008) +[2023-10-09 12:32:09,342][86122] Updated weights for policy 1, policy_version 7810 (0.0011) +[2023-10-09 12:32:09,461][86121] Updated weights for policy 0, policy_version 7800 (0.0010) +[2023-10-09 12:32:09,703][86122] Updated weights for policy 1, policy_version 7820 (0.0008) +[2023-10-09 12:32:10,064][86122] Updated weights for policy 1, policy_version 7830 (0.0010) +[2023-10-09 12:32:10,431][86122] Updated weights for policy 1, policy_version 7840 (0.0008) +[2023-10-09 12:32:13,129][86121] Updated weights for policy 0, policy_version 7810 (0.0007) +[2023-10-09 12:32:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 16023552. Throughput: 0: 1785.1, 1: 1828.8. Samples: 4013456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:32:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 12:32:13,498][86121] Updated weights for policy 0, policy_version 7820 (0.0008) +[2023-10-09 12:32:13,867][86121] Updated weights for policy 0, policy_version 7830 (0.0011) +[2023-10-09 12:32:14,208][86122] Updated weights for policy 1, policy_version 7850 (0.0010) +[2023-10-09 12:32:14,235][86121] Updated weights for policy 0, policy_version 7840 (0.0009) +[2023-10-09 12:32:14,573][86122] Updated weights for policy 1, policy_version 7860 (0.0009) +[2023-10-09 12:32:14,943][86122] Updated weights for policy 1, policy_version 7870 (0.0011) +[2023-10-09 12:32:17,946][86121] Updated weights for policy 0, policy_version 7850 (0.0008) +[2023-10-09 12:32:18,318][86121] Updated weights for policy 0, policy_version 7860 (0.0009) +[2023-10-09 12:32:18,395][86122] Updated weights for policy 1, policy_version 7880 (0.0008) +[2023-10-09 12:32:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 16089088. Throughput: 0: 1792.4, 1: 1819.6. Samples: 4036232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:32:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 12:32:18,694][86121] Updated weights for policy 0, policy_version 7870 (0.0009) +[2023-10-09 12:32:18,759][86122] Updated weights for policy 1, policy_version 7890 (0.0008) +[2023-10-09 12:32:19,122][86122] Updated weights for policy 1, policy_version 7900 (0.0007) +[2023-10-09 12:32:22,500][86121] Updated weights for policy 0, policy_version 7880 (0.0010) +[2023-10-09 12:32:22,865][86121] Updated weights for policy 0, policy_version 7890 (0.0007) +[2023-10-09 12:32:22,902][86122] Updated weights for policy 1, policy_version 7910 (0.0007) +[2023-10-09 12:32:23,228][86121] Updated weights for policy 0, policy_version 7900 (0.0009) +[2023-10-09 12:32:23,264][86122] Updated weights for policy 1, policy_version 7920 (0.0009) +[2023-10-09 12:32:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 16187392. Throughput: 0: 1802.4, 1: 1818.3. Samples: 4057714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:32:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:32:23,640][86122] Updated weights for policy 1, policy_version 7930 (0.0007) +[2023-10-09 12:32:26,934][86121] Updated weights for policy 0, policy_version 7910 (0.0008) +[2023-10-09 12:32:27,313][86121] Updated weights for policy 0, policy_version 7920 (0.0008) +[2023-10-09 12:32:27,498][86122] Updated weights for policy 1, policy_version 7940 (0.0009) +[2023-10-09 12:32:27,676][86121] Updated weights for policy 0, policy_version 7930 (0.0008) +[2023-10-09 12:32:27,861][86122] Updated weights for policy 1, policy_version 7950 (0.0007) +[2023-10-09 12:32:28,217][86122] Updated weights for policy 1, policy_version 7960 (0.0010) +[2023-10-09 12:32:28,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16252928. Throughput: 0: 1787.8, 1: 1818.1. Samples: 4068504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:32:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:32:31,450][86121] Updated weights for policy 0, policy_version 7940 (0.0010) +[2023-10-09 12:32:31,823][86121] Updated weights for policy 0, policy_version 7950 (0.0010) +[2023-10-09 12:32:31,947][86122] Updated weights for policy 1, policy_version 7970 (0.0010) +[2023-10-09 12:32:32,189][86121] Updated weights for policy 0, policy_version 7960 (0.0008) +[2023-10-09 12:32:32,307][86122] Updated weights for policy 1, policy_version 7980 (0.0009) +[2023-10-09 12:32:32,672][86122] Updated weights for policy 1, policy_version 7990 (0.0010) +[2023-10-09 12:32:33,035][86122] Updated weights for policy 1, policy_version 8000 (0.0008) +[2023-10-09 12:32:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16351232. Throughput: 0: 1802.1, 1: 1821.8. Samples: 4090292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:32:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.920')] +[2023-10-09 12:32:35,889][86121] Updated weights for policy 0, policy_version 7970 (0.0008) +[2023-10-09 12:32:36,259][86121] Updated weights for policy 0, policy_version 7980 (0.0009) +[2023-10-09 12:32:36,635][86121] Updated weights for policy 0, policy_version 7990 (0.0009) +[2023-10-09 12:32:36,792][86122] Updated weights for policy 1, policy_version 8010 (0.0007) +[2023-10-09 12:32:37,006][86121] Updated weights for policy 0, policy_version 8000 (0.0008) +[2023-10-09 12:32:37,153][86122] Updated weights for policy 1, policy_version 8020 (0.0007) +[2023-10-09 12:32:37,522][86122] Updated weights for policy 1, policy_version 8030 (0.0008) +[2023-10-09 12:32:38,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16416768. Throughput: 0: 1786.6, 1: 1812.7. Samples: 4110656. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) +[2023-10-09 12:32:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.910')] +[2023-10-09 12:32:40,824][86121] Updated weights for policy 0, policy_version 8010 (0.0009) +[2023-10-09 12:32:41,154][86122] Updated weights for policy 1, policy_version 8040 (0.0009) +[2023-10-09 12:32:41,197][86121] Updated weights for policy 0, policy_version 8020 (0.0009) +[2023-10-09 12:32:41,517][86122] Updated weights for policy 1, policy_version 8050 (0.0009) +[2023-10-09 12:32:41,575][86121] Updated weights for policy 0, policy_version 8030 (0.0007) +[2023-10-09 12:32:41,892][86122] Updated weights for policy 1, policy_version 8060 (0.0010) +[2023-10-09 12:32:43,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 16482304. Throughput: 0: 1808.1, 1: 1816.3. Samples: 4123006. Policy #0 lag: (min: 1.0, avg: 16.7, max: 33.0) +[2023-10-09 12:32:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:32:45,204][86121] Updated weights for policy 0, policy_version 8040 (0.0010) +[2023-10-09 12:32:45,515][86122] Updated weights for policy 1, policy_version 8070 (0.0009) +[2023-10-09 12:32:45,567][86121] Updated weights for policy 0, policy_version 8050 (0.0008) +[2023-10-09 12:32:45,885][86122] Updated weights for policy 1, policy_version 8080 (0.0007) +[2023-10-09 12:32:45,944][86121] Updated weights for policy 0, policy_version 8060 (0.0008) +[2023-10-09 12:32:46,248][86122] Updated weights for policy 1, policy_version 8090 (0.0009) +[2023-10-09 12:32:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 16547840. Throughput: 0: 1788.1, 1: 1818.6. Samples: 4143514. Policy #0 lag: (min: 21.0, avg: 27.4, max: 53.0) +[2023-10-09 12:32:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:32:49,657][86121] Updated weights for policy 0, policy_version 8070 (0.0007) +[2023-10-09 12:32:50,035][86121] Updated weights for policy 0, policy_version 8080 (0.0007) +[2023-10-09 12:32:50,080][86122] Updated weights for policy 1, policy_version 8100 (0.0008) +[2023-10-09 12:32:50,399][86121] Updated weights for policy 0, policy_version 8090 (0.0009) +[2023-10-09 12:32:50,442][86122] Updated weights for policy 1, policy_version 8110 (0.0007) +[2023-10-09 12:32:50,804][86122] Updated weights for policy 1, policy_version 8120 (0.0010) +[2023-10-09 12:32:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16613376. Throughput: 0: 1796.1, 1: 1809.9. Samples: 4166092. Policy #0 lag: (min: 21.0, avg: 27.4, max: 53.0) +[2023-10-09 12:32:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:32:54,166][86121] Updated weights for policy 0, policy_version 8100 (0.0008) +[2023-10-09 12:32:54,367][86122] Updated weights for policy 1, policy_version 8130 (0.0010) +[2023-10-09 12:32:54,528][86121] Updated weights for policy 0, policy_version 8110 (0.0008) +[2023-10-09 12:32:54,736][86122] Updated weights for policy 1, policy_version 8140 (0.0007) +[2023-10-09 12:32:54,895][86121] Updated weights for policy 0, policy_version 8120 (0.0008) +[2023-10-09 12:32:55,106][86122] Updated weights for policy 1, policy_version 8150 (0.0008) +[2023-10-09 12:32:55,476][86122] Updated weights for policy 1, policy_version 8160 (0.0009) +[2023-10-09 12:32:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 16678912. Throughput: 0: 1798.3, 1: 1812.4. Samples: 4175938. Policy #0 lag: (min: 2.0, avg: 8.9, max: 34.0) +[2023-10-09 12:32:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:32:58,572][86121] Updated weights for policy 0, policy_version 8130 (0.0008) +[2023-10-09 12:32:58,943][86121] Updated weights for policy 0, policy_version 8140 (0.0008) +[2023-10-09 12:32:59,190][86122] Updated weights for policy 1, policy_version 8170 (0.0009) +[2023-10-09 12:32:59,308][86121] Updated weights for policy 0, policy_version 8150 (0.0008) +[2023-10-09 12:32:59,556][86122] Updated weights for policy 1, policy_version 8180 (0.0007) +[2023-10-09 12:32:59,675][86121] Updated weights for policy 0, policy_version 8160 (0.0007) +[2023-10-09 12:32:59,919][86122] Updated weights for policy 1, policy_version 8190 (0.0010) +[2023-10-09 12:33:03,394][86121] Updated weights for policy 0, policy_version 8170 (0.0008) +[2023-10-09 12:33:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 16744448. Throughput: 0: 1796.4, 1: 1811.8. Samples: 4198598. Policy #0 lag: (min: 2.0, avg: 8.9, max: 34.0) +[2023-10-09 12:33:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 12:33:03,588][86122] Updated weights for policy 1, policy_version 8200 (0.0007) +[2023-10-09 12:33:03,753][86121] Updated weights for policy 0, policy_version 8180 (0.0008) +[2023-10-09 12:33:03,955][86122] Updated weights for policy 1, policy_version 8210 (0.0009) +[2023-10-09 12:33:04,126][86121] Updated weights for policy 0, policy_version 8190 (0.0007) +[2023-10-09 12:33:04,317][86122] Updated weights for policy 1, policy_version 8220 (0.0008) +[2023-10-09 12:33:07,734][86121] Updated weights for policy 0, policy_version 8200 (0.0009) +[2023-10-09 12:33:08,022][86122] Updated weights for policy 1, policy_version 8230 (0.0009) +[2023-10-09 12:33:08,109][86121] Updated weights for policy 0, policy_version 8210 (0.0008) +[2023-10-09 12:33:08,377][86122] Updated weights for policy 1, policy_version 8240 (0.0010) +[2023-10-09 12:33:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 16809984. Throughput: 0: 1811.9, 1: 1812.6. Samples: 4220820. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 12:33:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 12:33:08,475][86121] Updated weights for policy 0, policy_version 8220 (0.0010) +[2023-10-09 12:33:08,742][86122] Updated weights for policy 1, policy_version 8250 (0.0008) +[2023-10-09 12:33:12,318][86121] Updated weights for policy 0, policy_version 8230 (0.0007) +[2023-10-09 12:33:12,585][86122] Updated weights for policy 1, policy_version 8260 (0.0009) +[2023-10-09 12:33:12,685][86121] Updated weights for policy 0, policy_version 8240 (0.0007) +[2023-10-09 12:33:12,955][86122] Updated weights for policy 1, policy_version 8270 (0.0009) +[2023-10-09 12:33:13,053][86121] Updated weights for policy 0, policy_version 8250 (0.0007) +[2023-10-09 12:33:13,317][86122] Updated weights for policy 1, policy_version 8280 (0.0009) +[2023-10-09 12:33:13,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 16908288. Throughput: 0: 1805.7, 1: 1809.3. Samples: 4231180. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 12:33:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 12:33:16,677][86121] Updated weights for policy 0, policy_version 8260 (0.0008) +[2023-10-09 12:33:17,046][86121] Updated weights for policy 0, policy_version 8270 (0.0010) +[2023-10-09 12:33:17,146][86122] Updated weights for policy 1, policy_version 8290 (0.0009) +[2023-10-09 12:33:17,411][86121] Updated weights for policy 0, policy_version 8280 (0.0008) +[2023-10-09 12:33:17,511][86122] Updated weights for policy 1, policy_version 8300 (0.0007) +[2023-10-09 12:33:17,882][86122] Updated weights for policy 1, policy_version 8310 (0.0009) +[2023-10-09 12:33:18,256][86122] Updated weights for policy 1, policy_version 8320 (0.0008) +[2023-10-09 12:33:18,397][85186] Fps is (10 sec: 19661.3, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 17006592. Throughput: 0: 1814.9, 1: 1805.7. Samples: 4253220. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 12:33:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.930')] +[2023-10-09 12:33:21,123][86121] Updated weights for policy 0, policy_version 8290 (0.0009) +[2023-10-09 12:33:21,498][86121] Updated weights for policy 0, policy_version 8300 (0.0009) +[2023-10-09 12:33:21,866][86121] Updated weights for policy 0, policy_version 8310 (0.0007) +[2023-10-09 12:33:21,914][86122] Updated weights for policy 1, policy_version 8330 (0.0008) +[2023-10-09 12:33:22,227][86121] Updated weights for policy 0, policy_version 8320 (0.0008) +[2023-10-09 12:33:22,289][86122] Updated weights for policy 1, policy_version 8340 (0.0008) +[2023-10-09 12:33:22,654][86122] Updated weights for policy 1, policy_version 8350 (0.0008) +[2023-10-09 12:33:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 17072128. Throughput: 0: 1812.3, 1: 1810.4. Samples: 4273682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:33:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.930')] +[2023-10-09 12:33:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000008320_8519680.pth... +[2023-10-09 12:33:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000008352_8552448.pth... +[2023-10-09 12:33:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000006624_6782976.pth +[2023-10-09 12:33:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000006656_6815744.pth +[2023-10-09 12:33:23,448][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000008320_8519680.pth +[2023-10-09 12:33:23,449][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000008352_8552448.pth +[2023-10-09 12:33:25,911][86121] Updated weights for policy 0, policy_version 8330 (0.0007) +[2023-10-09 12:33:26,285][86121] Updated weights for policy 0, policy_version 8340 (0.0009) +[2023-10-09 12:33:26,408][86122] Updated weights for policy 1, policy_version 8360 (0.0009) +[2023-10-09 12:33:26,648][86121] Updated weights for policy 0, policy_version 8350 (0.0007) +[2023-10-09 12:33:26,787][86122] Updated weights for policy 1, policy_version 8370 (0.0008) +[2023-10-09 12:33:27,155][86122] Updated weights for policy 1, policy_version 8380 (0.0009) +[2023-10-09 12:33:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 17137664. Throughput: 0: 1816.8, 1: 1807.7. Samples: 4286104. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:33:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 12:33:30,267][86121] Updated weights for policy 0, policy_version 8360 (0.0009) +[2023-10-09 12:33:30,626][86121] Updated weights for policy 0, policy_version 8370 (0.0009) +[2023-10-09 12:33:30,745][86122] Updated weights for policy 1, policy_version 8390 (0.0008) +[2023-10-09 12:33:30,993][86121] Updated weights for policy 0, policy_version 8380 (0.0008) +[2023-10-09 12:33:31,112][86122] Updated weights for policy 1, policy_version 8400 (0.0008) +[2023-10-09 12:33:31,485][86122] Updated weights for policy 1, policy_version 8410 (0.0010) +[2023-10-09 12:33:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 17203200. Throughput: 0: 1816.4, 1: 1805.6. Samples: 4306506. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 12:33:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 12:33:34,832][86121] Updated weights for policy 0, policy_version 8390 (0.0008) +[2023-10-09 12:33:35,224][86121] Updated weights for policy 0, policy_version 8400 (0.0008) +[2023-10-09 12:33:35,272][86122] Updated weights for policy 1, policy_version 8420 (0.0010) +[2023-10-09 12:33:35,598][86121] Updated weights for policy 0, policy_version 8410 (0.0008) +[2023-10-09 12:33:35,625][86122] Updated weights for policy 1, policy_version 8430 (0.0009) +[2023-10-09 12:33:36,002][86122] Updated weights for policy 1, policy_version 8440 (0.0009) +[2023-10-09 12:33:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 17268736. Throughput: 0: 1818.7, 1: 1806.1. Samples: 4329208. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 12:33:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 12:33:39,217][86121] Updated weights for policy 0, policy_version 8420 (0.0008) +[2023-10-09 12:33:39,590][86121] Updated weights for policy 0, policy_version 8430 (0.0008) +[2023-10-09 12:33:39,691][86122] Updated weights for policy 1, policy_version 8450 (0.0008) +[2023-10-09 12:33:39,962][86121] Updated weights for policy 0, policy_version 8440 (0.0009) +[2023-10-09 12:33:40,055][86122] Updated weights for policy 1, policy_version 8460 (0.0009) +[2023-10-09 12:33:40,417][86122] Updated weights for policy 1, policy_version 8470 (0.0012) +[2023-10-09 12:33:40,782][86122] Updated weights for policy 1, policy_version 8480 (0.0008) +[2023-10-09 12:33:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17334272. Throughput: 0: 1818.1, 1: 1804.8. Samples: 4338972. Policy #0 lag: (min: 21.0, avg: 38.4, max: 53.0) +[2023-10-09 12:33:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 12:33:43,711][86121] Updated weights for policy 0, policy_version 8450 (0.0008) +[2023-10-09 12:33:44,080][86121] Updated weights for policy 0, policy_version 8460 (0.0010) +[2023-10-09 12:33:44,455][86121] Updated weights for policy 0, policy_version 8470 (0.0008) +[2023-10-09 12:33:44,515][86122] Updated weights for policy 1, policy_version 8490 (0.0009) +[2023-10-09 12:33:44,812][86121] Updated weights for policy 0, policy_version 8480 (0.0009) +[2023-10-09 12:33:44,883][86122] Updated weights for policy 1, policy_version 8500 (0.0008) +[2023-10-09 12:33:45,249][86122] Updated weights for policy 1, policy_version 8510 (0.0010) +[2023-10-09 12:33:48,396][86121] Updated weights for policy 0, policy_version 8490 (0.0012) +[2023-10-09 12:33:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17399808. Throughput: 0: 1821.5, 1: 1805.6. Samples: 4361816. Policy #0 lag: (min: 21.0, avg: 38.4, max: 53.0) +[2023-10-09 12:33:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.940')] +[2023-10-09 12:33:48,754][86121] Updated weights for policy 0, policy_version 8500 (0.0009) +[2023-10-09 12:33:49,041][86122] Updated weights for policy 1, policy_version 8520 (0.0008) +[2023-10-09 12:33:49,122][86121] Updated weights for policy 0, policy_version 8510 (0.0008) +[2023-10-09 12:33:49,407][86122] Updated weights for policy 1, policy_version 8530 (0.0008) +[2023-10-09 12:33:49,777][86122] Updated weights for policy 1, policy_version 8540 (0.0007) +[2023-10-09 12:33:52,879][86121] Updated weights for policy 0, policy_version 8520 (0.0008) +[2023-10-09 12:33:53,230][86121] Updated weights for policy 0, policy_version 8530 (0.0007) +[2023-10-09 12:33:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17465344. Throughput: 0: 1821.3, 1: 1802.3. Samples: 4383878. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 12:33:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.950')] +[2023-10-09 12:33:53,527][86122] Updated weights for policy 1, policy_version 8550 (0.0008) +[2023-10-09 12:33:53,592][86121] Updated weights for policy 0, policy_version 8540 (0.0008) +[2023-10-09 12:33:53,888][86122] Updated weights for policy 1, policy_version 8560 (0.0008) +[2023-10-09 12:33:54,259][86122] Updated weights for policy 1, policy_version 8570 (0.0009) +[2023-10-09 12:33:57,330][86121] Updated weights for policy 0, policy_version 8550 (0.0007) +[2023-10-09 12:33:57,693][86121] Updated weights for policy 0, policy_version 8560 (0.0007) +[2023-10-09 12:33:57,821][86122] Updated weights for policy 1, policy_version 8580 (0.0009) +[2023-10-09 12:33:58,057][86121] Updated weights for policy 0, policy_version 8570 (0.0009) +[2023-10-09 12:33:58,181][86122] Updated weights for policy 1, policy_version 8590 (0.0008) +[2023-10-09 12:33:58,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17563648. Throughput: 0: 1820.6, 1: 1802.8. Samples: 4394232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:33:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.950')] +[2023-10-09 12:33:58,555][86122] Updated weights for policy 1, policy_version 8600 (0.0010) +[2023-10-09 12:34:01,845][86121] Updated weights for policy 0, policy_version 8580 (0.0010) +[2023-10-09 12:34:02,207][86121] Updated weights for policy 0, policy_version 8590 (0.0009) +[2023-10-09 12:34:02,318][86122] Updated weights for policy 1, policy_version 8610 (0.0007) +[2023-10-09 12:34:02,582][86121] Updated weights for policy 0, policy_version 8600 (0.0008) +[2023-10-09 12:34:02,683][86122] Updated weights for policy 1, policy_version 8620 (0.0008) +[2023-10-09 12:34:03,051][86122] Updated weights for policy 1, policy_version 8630 (0.0008) +[2023-10-09 12:34:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 17629184. Throughput: 0: 1824.2, 1: 1815.0. Samples: 4416984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:34:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.950')] +[2023-10-09 12:34:03,414][86122] Updated weights for policy 1, policy_version 8640 (0.0008) +[2023-10-09 12:34:06,104][86121] Updated weights for policy 0, policy_version 8610 (0.0010) +[2023-10-09 12:34:06,482][86121] Updated weights for policy 0, policy_version 8620 (0.0011) +[2023-10-09 12:34:06,844][86121] Updated weights for policy 0, policy_version 8630 (0.0008) +[2023-10-09 12:34:07,043][86122] Updated weights for policy 1, policy_version 8650 (0.0009) +[2023-10-09 12:34:07,218][86121] Updated weights for policy 0, policy_version 8640 (0.0008) +[2023-10-09 12:34:07,413][86122] Updated weights for policy 1, policy_version 8660 (0.0009) +[2023-10-09 12:34:07,779][86122] Updated weights for policy 1, policy_version 8670 (0.0008) +[2023-10-09 12:34:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 17727488. Throughput: 0: 1825.5, 1: 1815.7. Samples: 4437536. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) +[2023-10-09 12:34:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.950')] +[2023-10-09 12:34:11,016][86121] Updated weights for policy 0, policy_version 8650 (0.0009) +[2023-10-09 12:34:11,383][86121] Updated weights for policy 0, policy_version 8660 (0.0008) +[2023-10-09 12:34:11,570][86122] Updated weights for policy 1, policy_version 8680 (0.0009) +[2023-10-09 12:34:11,753][86121] Updated weights for policy 0, policy_version 8670 (0.0007) +[2023-10-09 12:34:11,935][86122] Updated weights for policy 1, policy_version 8690 (0.0008) +[2023-10-09 12:34:12,298][86122] Updated weights for policy 1, policy_version 8700 (0.0008) +[2023-10-09 12:34:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 17793024. Throughput: 0: 1822.3, 1: 1812.0. Samples: 4449644. Policy #0 lag: (min: 31.0, avg: 36.1, max: 63.0) +[2023-10-09 12:34:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.950')] +[2023-10-09 12:34:15,411][86121] Updated weights for policy 0, policy_version 8680 (0.0010) +[2023-10-09 12:34:15,786][86121] Updated weights for policy 0, policy_version 8690 (0.0007) +[2023-10-09 12:34:15,921][86122] Updated weights for policy 1, policy_version 8710 (0.0008) +[2023-10-09 12:34:16,149][86121] Updated weights for policy 0, policy_version 8700 (0.0007) +[2023-10-09 12:34:16,294][86122] Updated weights for policy 1, policy_version 8720 (0.0008) +[2023-10-09 12:34:16,666][86122] Updated weights for policy 1, policy_version 8730 (0.0010) +[2023-10-09 12:34:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 17858560. Throughput: 0: 1815.1, 1: 1815.4. Samples: 4469876. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 12:34:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:34:19,912][86121] Updated weights for policy 0, policy_version 8710 (0.0008) +[2023-10-09 12:34:20,288][86121] Updated weights for policy 0, policy_version 8720 (0.0008) +[2023-10-09 12:34:20,338][86122] Updated weights for policy 1, policy_version 8740 (0.0009) +[2023-10-09 12:34:20,655][86121] Updated weights for policy 0, policy_version 8730 (0.0007) +[2023-10-09 12:34:20,716][86122] Updated weights for policy 1, policy_version 8750 (0.0011) +[2023-10-09 12:34:21,075][86122] Updated weights for policy 1, policy_version 8760 (0.0009) +[2023-10-09 12:34:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 17924096. Throughput: 0: 1814.8, 1: 1814.2. Samples: 4492512. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 12:34:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.950')] +[2023-10-09 12:34:24,211][86121] Updated weights for policy 0, policy_version 8740 (0.0009) +[2023-10-09 12:34:24,577][86121] Updated weights for policy 0, policy_version 8750 (0.0008) +[2023-10-09 12:34:24,743][86122] Updated weights for policy 1, policy_version 8770 (0.0008) +[2023-10-09 12:34:24,946][86121] Updated weights for policy 0, policy_version 8760 (0.0009) +[2023-10-09 12:34:25,106][86122] Updated weights for policy 1, policy_version 8780 (0.0007) +[2023-10-09 12:34:25,478][86122] Updated weights for policy 1, policy_version 8790 (0.0008) +[2023-10-09 12:34:25,839][86122] Updated weights for policy 1, policy_version 8800 (0.0010) +[2023-10-09 12:34:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 17989632. Throughput: 0: 1819.6, 1: 1819.9. Samples: 4502748. Policy #0 lag: (min: 9.0, avg: 21.3, max: 41.0) +[2023-10-09 12:34:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:34:28,621][86121] Updated weights for policy 0, policy_version 8770 (0.0008) +[2023-10-09 12:34:28,982][86121] Updated weights for policy 0, policy_version 8780 (0.0009) +[2023-10-09 12:34:29,344][86121] Updated weights for policy 0, policy_version 8790 (0.0009) +[2023-10-09 12:34:29,560][86122] Updated weights for policy 1, policy_version 8810 (0.0007) +[2023-10-09 12:34:29,710][86121] Updated weights for policy 0, policy_version 8800 (0.0007) +[2023-10-09 12:34:29,918][86122] Updated weights for policy 1, policy_version 8820 (0.0010) +[2023-10-09 12:34:30,285][86122] Updated weights for policy 1, policy_version 8830 (0.0009) +[2023-10-09 12:34:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18055168. Throughput: 0: 1814.9, 1: 1817.2. Samples: 4525262. Policy #0 lag: (min: 9.0, avg: 21.3, max: 41.0) +[2023-10-09 12:34:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:34:33,484][86121] Updated weights for policy 0, policy_version 8810 (0.0009) +[2023-10-09 12:34:33,842][86121] Updated weights for policy 0, policy_version 8820 (0.0007) +[2023-10-09 12:34:33,987][86122] Updated weights for policy 1, policy_version 8840 (0.0008) +[2023-10-09 12:34:34,211][86121] Updated weights for policy 0, policy_version 8830 (0.0009) +[2023-10-09 12:34:34,352][86122] Updated weights for policy 1, policy_version 8850 (0.0009) +[2023-10-09 12:34:34,711][86122] Updated weights for policy 1, policy_version 8860 (0.0009) +[2023-10-09 12:34:38,101][86121] Updated weights for policy 0, policy_version 8840 (0.0011) +[2023-10-09 12:34:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18120704. Throughput: 0: 1814.7, 1: 1823.1. Samples: 4547580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:34:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.970')] +[2023-10-09 12:34:38,411][86122] Updated weights for policy 1, policy_version 8870 (0.0008) +[2023-10-09 12:34:38,466][86121] Updated weights for policy 0, policy_version 8850 (0.0009) +[2023-10-09 12:34:38,780][86122] Updated weights for policy 1, policy_version 8880 (0.0008) +[2023-10-09 12:34:38,835][86121] Updated weights for policy 0, policy_version 8860 (0.0009) +[2023-10-09 12:34:39,150][86122] Updated weights for policy 1, policy_version 8890 (0.0009) +[2023-10-09 12:34:42,426][86121] Updated weights for policy 0, policy_version 8870 (0.0009) +[2023-10-09 12:34:42,804][86121] Updated weights for policy 0, policy_version 8880 (0.0007) +[2023-10-09 12:34:42,978][86122] Updated weights for policy 1, policy_version 8900 (0.0010) +[2023-10-09 12:34:43,170][86121] Updated weights for policy 0, policy_version 8890 (0.0008) +[2023-10-09 12:34:43,346][86122] Updated weights for policy 1, policy_version 8910 (0.0007) +[2023-10-09 12:34:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18219008. Throughput: 0: 1808.3, 1: 1820.8. Samples: 4557544. Policy #0 lag: (min: 17.0, avg: 21.3, max: 49.0) +[2023-10-09 12:34:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:34:43,705][86122] Updated weights for policy 1, policy_version 8920 (0.0008) +[2023-10-09 12:34:47,031][86121] Updated weights for policy 0, policy_version 8900 (0.0009) +[2023-10-09 12:34:47,401][86121] Updated weights for policy 0, policy_version 8910 (0.0009) +[2023-10-09 12:34:47,557][86122] Updated weights for policy 1, policy_version 8930 (0.0009) +[2023-10-09 12:34:47,766][86121] Updated weights for policy 0, policy_version 8920 (0.0009) +[2023-10-09 12:34:47,919][86122] Updated weights for policy 1, policy_version 8940 (0.0008) +[2023-10-09 12:34:48,291][86122] Updated weights for policy 1, policy_version 8950 (0.0009) +[2023-10-09 12:34:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 18284544. Throughput: 0: 1811.7, 1: 1809.6. Samples: 4579946. Policy #0 lag: (min: 17.0, avg: 21.3, max: 49.0) +[2023-10-09 12:34:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:34:48,646][86122] Updated weights for policy 1, policy_version 8960 (0.0009) +[2023-10-09 12:34:51,506][86121] Updated weights for policy 0, policy_version 8930 (0.0009) +[2023-10-09 12:34:51,870][86121] Updated weights for policy 0, policy_version 8940 (0.0009) +[2023-10-09 12:34:52,243][86121] Updated weights for policy 0, policy_version 8950 (0.0009) +[2023-10-09 12:34:52,316][86122] Updated weights for policy 1, policy_version 8970 (0.0010) +[2023-10-09 12:34:52,610][86121] Updated weights for policy 0, policy_version 8960 (0.0008) +[2023-10-09 12:34:52,680][86122] Updated weights for policy 1, policy_version 8980 (0.0009) +[2023-10-09 12:34:53,045][86122] Updated weights for policy 1, policy_version 8990 (0.0009) +[2023-10-09 12:34:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 18382848. Throughput: 0: 1799.3, 1: 1819.4. Samples: 4600380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:34:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:34:56,336][86121] Updated weights for policy 0, policy_version 8970 (0.0009) +[2023-10-09 12:34:56,710][86121] Updated weights for policy 0, policy_version 8980 (0.0008) +[2023-10-09 12:34:56,784][86122] Updated weights for policy 1, policy_version 9000 (0.0008) +[2023-10-09 12:34:57,071][86121] Updated weights for policy 0, policy_version 8990 (0.0010) +[2023-10-09 12:34:57,142][86122] Updated weights for policy 1, policy_version 9010 (0.0007) +[2023-10-09 12:34:57,505][86122] Updated weights for policy 1, policy_version 9020 (0.0007) +[2023-10-09 12:34:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18448384. Throughput: 0: 1810.4, 1: 1814.8. Samples: 4612778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:34:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:35:00,739][86121] Updated weights for policy 0, policy_version 9000 (0.0008) +[2023-10-09 12:35:01,111][86121] Updated weights for policy 0, policy_version 9010 (0.0007) +[2023-10-09 12:35:01,230][86122] Updated weights for policy 1, policy_version 9030 (0.0008) +[2023-10-09 12:35:01,472][86121] Updated weights for policy 0, policy_version 9020 (0.0007) +[2023-10-09 12:35:01,595][86122] Updated weights for policy 1, policy_version 9040 (0.0008) +[2023-10-09 12:35:01,972][86122] Updated weights for policy 1, policy_version 9050 (0.0008) +[2023-10-09 12:35:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 18513920. Throughput: 0: 1803.9, 1: 1821.2. Samples: 4633004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:35:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:35:05,233][86121] Updated weights for policy 0, policy_version 9030 (0.0008) +[2023-10-09 12:35:05,612][86121] Updated weights for policy 0, policy_version 9040 (0.0009) +[2023-10-09 12:35:05,612][86122] Updated weights for policy 1, policy_version 9060 (0.0008) +[2023-10-09 12:35:05,969][86121] Updated weights for policy 0, policy_version 9050 (0.0008) +[2023-10-09 12:35:05,971][86122] Updated weights for policy 1, policy_version 9070 (0.0009) +[2023-10-09 12:35:06,336][86122] Updated weights for policy 1, policy_version 9080 (0.0008) +[2023-10-09 12:35:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 18579456. Throughput: 0: 1803.2, 1: 1816.6. Samples: 4655402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:35:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:35:09,705][86121] Updated weights for policy 0, policy_version 9060 (0.0008) +[2023-10-09 12:35:10,071][86121] Updated weights for policy 0, policy_version 9070 (0.0008) +[2023-10-09 12:35:10,149][86122] Updated weights for policy 1, policy_version 9090 (0.0011) +[2023-10-09 12:35:10,434][86121] Updated weights for policy 0, policy_version 9080 (0.0008) +[2023-10-09 12:35:10,511][86122] Updated weights for policy 1, policy_version 9100 (0.0007) +[2023-10-09 12:35:10,886][86122] Updated weights for policy 1, policy_version 9110 (0.0008) +[2023-10-09 12:35:11,244][86122] Updated weights for policy 1, policy_version 9120 (0.0007) +[2023-10-09 12:35:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 18644992. Throughput: 0: 1800.0, 1: 1821.1. Samples: 4665698. Policy #0 lag: (min: 26.0, avg: 29.9, max: 58.0) +[2023-10-09 12:35:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:35:14,152][86121] Updated weights for policy 0, policy_version 9090 (0.0009) +[2023-10-09 12:35:14,523][86121] Updated weights for policy 0, policy_version 9100 (0.0008) +[2023-10-09 12:35:14,893][86121] Updated weights for policy 0, policy_version 9110 (0.0008) +[2023-10-09 12:35:14,983][86122] Updated weights for policy 1, policy_version 9130 (0.0008) +[2023-10-09 12:35:15,251][86121] Updated weights for policy 0, policy_version 9120 (0.0009) +[2023-10-09 12:35:15,347][86122] Updated weights for policy 1, policy_version 9140 (0.0008) +[2023-10-09 12:35:15,724][86122] Updated weights for policy 1, policy_version 9150 (0.0008) +[2023-10-09 12:35:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18710528. Throughput: 0: 1802.0, 1: 1810.0. Samples: 4687804. Policy #0 lag: (min: 26.0, avg: 29.9, max: 58.0) +[2023-10-09 12:35:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:35:19,061][86121] Updated weights for policy 0, policy_version 9130 (0.0010) +[2023-10-09 12:35:19,427][86121] Updated weights for policy 0, policy_version 9140 (0.0008) +[2023-10-09 12:35:19,456][86122] Updated weights for policy 1, policy_version 9160 (0.0009) +[2023-10-09 12:35:19,792][86121] Updated weights for policy 0, policy_version 9150 (0.0007) +[2023-10-09 12:35:19,831][86122] Updated weights for policy 1, policy_version 9170 (0.0008) +[2023-10-09 12:35:20,197][86122] Updated weights for policy 1, policy_version 9180 (0.0009) +[2023-10-09 12:35:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 18776064. Throughput: 0: 1807.1, 1: 1810.3. Samples: 4710360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:35:23,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:35:23,412][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000009184_9404416.pth... +[2023-10-09 12:35:23,442][86121] Updated weights for policy 0, policy_version 9160 (0.0007) +[2023-10-09 12:35:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000007488_7667712.pth +[2023-10-09 12:35:23,806][86121] Updated weights for policy 0, policy_version 9170 (0.0007) +[2023-10-09 12:35:23,894][86122] Updated weights for policy 1, policy_version 9190 (0.0008) +[2023-10-09 12:35:24,188][86121] Updated weights for policy 0, policy_version 9180 (0.0008) +[2023-10-09 12:35:24,261][86122] Updated weights for policy 1, policy_version 9200 (0.0008) +[2023-10-09 12:35:24,322][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000009184_9404416.pth... +[2023-10-09 12:35:24,350][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000007488_7667712.pth +[2023-10-09 12:35:24,632][86122] Updated weights for policy 1, policy_version 9210 (0.0009) +[2023-10-09 12:35:27,891][86121] Updated weights for policy 0, policy_version 9190 (0.0007) +[2023-10-09 12:35:28,221][86122] Updated weights for policy 1, policy_version 9220 (0.0009) +[2023-10-09 12:35:28,259][86121] Updated weights for policy 0, policy_version 9200 (0.0007) +[2023-10-09 12:35:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 18841600. Throughput: 0: 1803.5, 1: 1807.9. Samples: 4720054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:35:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:35:28,595][86122] Updated weights for policy 1, policy_version 9230 (0.0008) +[2023-10-09 12:35:28,631][86121] Updated weights for policy 0, policy_version 9210 (0.0008) +[2023-10-09 12:35:28,950][86122] Updated weights for policy 1, policy_version 9240 (0.0009) +[2023-10-09 12:35:32,490][86121] Updated weights for policy 0, policy_version 9220 (0.0009) +[2023-10-09 12:35:32,764][86122] Updated weights for policy 1, policy_version 9250 (0.0010) +[2023-10-09 12:35:32,862][86121] Updated weights for policy 0, policy_version 9230 (0.0009) +[2023-10-09 12:35:33,121][86122] Updated weights for policy 1, policy_version 9260 (0.0009) +[2023-10-09 12:35:33,224][86121] Updated weights for policy 0, policy_version 9240 (0.0007) +[2023-10-09 12:35:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 18907136. Throughput: 0: 1806.4, 1: 1812.2. Samples: 4742782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:35:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:35:33,486][86122] Updated weights for policy 1, policy_version 9270 (0.0009) +[2023-10-09 12:35:33,852][86122] Updated weights for policy 1, policy_version 9280 (0.0010) +[2023-10-09 12:35:36,873][86121] Updated weights for policy 0, policy_version 9250 (0.0008) +[2023-10-09 12:35:37,243][86121] Updated weights for policy 0, policy_version 9260 (0.0008) +[2023-10-09 12:35:37,479][86122] Updated weights for policy 1, policy_version 9290 (0.0007) +[2023-10-09 12:35:37,610][86121] Updated weights for policy 0, policy_version 9270 (0.0009) +[2023-10-09 12:35:37,854][86122] Updated weights for policy 1, policy_version 9300 (0.0008) +[2023-10-09 12:35:37,970][86121] Updated weights for policy 0, policy_version 9280 (0.0009) +[2023-10-09 12:35:38,220][86122] Updated weights for policy 1, policy_version 9310 (0.0008) +[2023-10-09 12:35:38,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 19038208. Throughput: 0: 1809.2, 1: 1814.6. Samples: 4763452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:35:38,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:35:41,746][86121] Updated weights for policy 0, policy_version 9290 (0.0008) +[2023-10-09 12:35:42,092][86122] Updated weights for policy 1, policy_version 9320 (0.0009) +[2023-10-09 12:35:42,114][86121] Updated weights for policy 0, policy_version 9300 (0.0007) +[2023-10-09 12:35:42,463][86122] Updated weights for policy 1, policy_version 9330 (0.0007) +[2023-10-09 12:35:42,492][86121] Updated weights for policy 0, policy_version 9310 (0.0008) +[2023-10-09 12:35:42,831][86122] Updated weights for policy 1, policy_version 9340 (0.0008) +[2023-10-09 12:35:43,397][85186] Fps is (10 sec: 19660.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19103744. Throughput: 0: 1802.2, 1: 1806.2. Samples: 4775158. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:35:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:35:46,234][86121] Updated weights for policy 0, policy_version 9320 (0.0008) +[2023-10-09 12:35:46,484][86122] Updated weights for policy 1, policy_version 9350 (0.0008) +[2023-10-09 12:35:46,606][86121] Updated weights for policy 0, policy_version 9330 (0.0008) +[2023-10-09 12:35:46,848][86122] Updated weights for policy 1, policy_version 9360 (0.0007) +[2023-10-09 12:35:46,971][86121] Updated weights for policy 0, policy_version 9340 (0.0007) +[2023-10-09 12:35:47,209][86122] Updated weights for policy 1, policy_version 9370 (0.0009) +[2023-10-09 12:35:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 19169280. Throughput: 0: 1807.8, 1: 1810.4. Samples: 4795820. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) +[2023-10-09 12:35:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:35:50,694][86121] Updated weights for policy 0, policy_version 9350 (0.0009) +[2023-10-09 12:35:50,897][86122] Updated weights for policy 1, policy_version 9380 (0.0009) +[2023-10-09 12:35:51,067][86121] Updated weights for policy 0, policy_version 9360 (0.0009) +[2023-10-09 12:35:51,272][86122] Updated weights for policy 1, policy_version 9390 (0.0008) +[2023-10-09 12:35:51,433][86121] Updated weights for policy 0, policy_version 9370 (0.0007) +[2023-10-09 12:35:51,637][86122] Updated weights for policy 1, policy_version 9400 (0.0007) +[2023-10-09 12:35:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 19234816. Throughput: 0: 1797.3, 1: 1802.1. Samples: 4817376. Policy #0 lag: (min: 31.0, avg: 37.8, max: 63.0) +[2023-10-09 12:35:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 12:35:55,250][86121] Updated weights for policy 0, policy_version 9380 (0.0009) +[2023-10-09 12:35:55,377][86122] Updated weights for policy 1, policy_version 9410 (0.0008) +[2023-10-09 12:35:55,616][86121] Updated weights for policy 0, policy_version 9390 (0.0008) +[2023-10-09 12:35:55,743][86122] Updated weights for policy 1, policy_version 9420 (0.0007) +[2023-10-09 12:35:55,972][86121] Updated weights for policy 0, policy_version 9400 (0.0007) +[2023-10-09 12:35:56,106][86122] Updated weights for policy 1, policy_version 9430 (0.0007) +[2023-10-09 12:35:56,473][86122] Updated weights for policy 1, policy_version 9440 (0.0007) +[2023-10-09 12:35:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 19300352. Throughput: 0: 1804.0, 1: 1809.5. Samples: 4828306. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 12:35:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:35:59,735][86121] Updated weights for policy 0, policy_version 9410 (0.0007) +[2023-10-09 12:36:00,105][86121] Updated weights for policy 0, policy_version 9420 (0.0008) +[2023-10-09 12:36:00,150][86122] Updated weights for policy 1, policy_version 9450 (0.0008) +[2023-10-09 12:36:00,472][86121] Updated weights for policy 0, policy_version 9430 (0.0009) +[2023-10-09 12:36:00,518][86122] Updated weights for policy 1, policy_version 9460 (0.0008) +[2023-10-09 12:36:00,832][86121] Updated weights for policy 0, policy_version 9440 (0.0009) +[2023-10-09 12:36:00,878][86122] Updated weights for policy 1, policy_version 9470 (0.0008) +[2023-10-09 12:36:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19365888. Throughput: 0: 1789.6, 1: 1809.4. Samples: 4849760. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 12:36:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:36:04,490][86122] Updated weights for policy 1, policy_version 9480 (0.0009) +[2023-10-09 12:36:04,778][86121] Updated weights for policy 0, policy_version 9450 (0.0009) +[2023-10-09 12:36:04,850][86122] Updated weights for policy 1, policy_version 9490 (0.0008) +[2023-10-09 12:36:05,138][86121] Updated weights for policy 0, policy_version 9460 (0.0008) +[2023-10-09 12:36:05,227][86122] Updated weights for policy 1, policy_version 9500 (0.0008) +[2023-10-09 12:36:05,504][86121] Updated weights for policy 0, policy_version 9470 (0.0008) +[2023-10-09 12:36:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19431424. Throughput: 0: 1792.9, 1: 1817.9. Samples: 4872846. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:36:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:36:08,885][86122] Updated weights for policy 1, policy_version 9510 (0.0007) +[2023-10-09 12:36:09,181][86121] Updated weights for policy 0, policy_version 9480 (0.0009) +[2023-10-09 12:36:09,239][86122] Updated weights for policy 1, policy_version 9520 (0.0008) +[2023-10-09 12:36:09,549][86121] Updated weights for policy 0, policy_version 9490 (0.0007) +[2023-10-09 12:36:09,609][86122] Updated weights for policy 1, policy_version 9530 (0.0008) +[2023-10-09 12:36:09,921][86121] Updated weights for policy 0, policy_version 9500 (0.0008) +[2023-10-09 12:36:13,346][86122] Updated weights for policy 1, policy_version 9540 (0.0007) +[2023-10-09 12:36:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19496960. Throughput: 0: 1790.9, 1: 1820.1. Samples: 4882546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:36:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:36:13,488][86121] Updated weights for policy 0, policy_version 9510 (0.0008) +[2023-10-09 12:36:13,711][86122] Updated weights for policy 1, policy_version 9550 (0.0008) +[2023-10-09 12:36:13,854][86121] Updated weights for policy 0, policy_version 9520 (0.0008) +[2023-10-09 12:36:14,077][86122] Updated weights for policy 1, policy_version 9560 (0.0009) +[2023-10-09 12:36:14,222][86121] Updated weights for policy 0, policy_version 9530 (0.0009) +[2023-10-09 12:36:17,920][86122] Updated weights for policy 1, policy_version 9570 (0.0009) +[2023-10-09 12:36:17,969][86121] Updated weights for policy 0, policy_version 9540 (0.0009) +[2023-10-09 12:36:18,282][86122] Updated weights for policy 1, policy_version 9580 (0.0008) +[2023-10-09 12:36:18,336][86121] Updated weights for policy 0, policy_version 9550 (0.0009) +[2023-10-09 12:36:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 19562496. Throughput: 0: 1793.0, 1: 1814.9. Samples: 4905138. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) +[2023-10-09 12:36:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:36:18,642][86122] Updated weights for policy 1, policy_version 9590 (0.0009) +[2023-10-09 12:36:18,708][86121] Updated weights for policy 0, policy_version 9560 (0.0007) +[2023-10-09 12:36:19,016][86122] Updated weights for policy 1, policy_version 9600 (0.0008) +[2023-10-09 12:36:22,319][86121] Updated weights for policy 0, policy_version 9570 (0.0008) +[2023-10-09 12:36:22,547][86122] Updated weights for policy 1, policy_version 9610 (0.0007) +[2023-10-09 12:36:22,684][86121] Updated weights for policy 0, policy_version 9580 (0.0007) +[2023-10-09 12:36:22,906][86122] Updated weights for policy 1, policy_version 9620 (0.0007) +[2023-10-09 12:36:23,047][86121] Updated weights for policy 0, policy_version 9590 (0.0008) +[2023-10-09 12:36:23,273][86122] Updated weights for policy 1, policy_version 9630 (0.0008) +[2023-10-09 12:36:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 19660800. Throughput: 0: 1803.0, 1: 1818.4. Samples: 4926416. Policy #0 lag: (min: 9.0, avg: 20.2, max: 41.0) +[2023-10-09 12:36:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:36:23,422][86121] Updated weights for policy 0, policy_version 9600 (0.0009) +[2023-10-09 12:36:27,016][86122] Updated weights for policy 1, policy_version 9640 (0.0009) +[2023-10-09 12:36:27,259][86121] Updated weights for policy 0, policy_version 9610 (0.0008) +[2023-10-09 12:36:27,391][86122] Updated weights for policy 1, policy_version 9650 (0.0008) +[2023-10-09 12:36:27,626][86121] Updated weights for policy 0, policy_version 9620 (0.0007) +[2023-10-09 12:36:27,761][86122] Updated weights for policy 1, policy_version 9660 (0.0007) +[2023-10-09 12:36:27,989][86121] Updated weights for policy 0, policy_version 9630 (0.0008) +[2023-10-09 12:36:28,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 19759104. Throughput: 0: 1792.6, 1: 1819.2. Samples: 4937688. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 12:36:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:36:31,346][86122] Updated weights for policy 1, policy_version 9670 (0.0009) +[2023-10-09 12:36:31,717][86122] Updated weights for policy 1, policy_version 9680 (0.0007) +[2023-10-09 12:36:31,861][86121] Updated weights for policy 0, policy_version 9640 (0.0008) +[2023-10-09 12:36:32,077][86122] Updated weights for policy 1, policy_version 9690 (0.0007) +[2023-10-09 12:36:32,225][86121] Updated weights for policy 0, policy_version 9650 (0.0008) +[2023-10-09 12:36:32,591][86121] Updated weights for policy 0, policy_version 9660 (0.0007) +[2023-10-09 12:36:33,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 19824640. Throughput: 0: 1804.0, 1: 1823.2. Samples: 4959044. Policy #0 lag: (min: 25.0, avg: 39.3, max: 57.0) +[2023-10-09 12:36:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:36:35,778][86122] Updated weights for policy 1, policy_version 9700 (0.0007) +[2023-10-09 12:36:36,136][86122] Updated weights for policy 1, policy_version 9710 (0.0008) +[2023-10-09 12:36:36,453][86121] Updated weights for policy 0, policy_version 9670 (0.0010) +[2023-10-09 12:36:36,498][86122] Updated weights for policy 1, policy_version 9720 (0.0008) +[2023-10-09 12:36:36,841][86121] Updated weights for policy 0, policy_version 9680 (0.0008) +[2023-10-09 12:36:37,208][86121] Updated weights for policy 0, policy_version 9690 (0.0008) +[2023-10-09 12:36:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 19890176. Throughput: 0: 1786.4, 1: 1828.6. Samples: 4980052. Policy #0 lag: (min: 25.0, avg: 39.3, max: 57.0) +[2023-10-09 12:36:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:36:40,112][86122] Updated weights for policy 1, policy_version 9730 (0.0010) +[2023-10-09 12:36:40,476][86122] Updated weights for policy 1, policy_version 9740 (0.0008) +[2023-10-09 12:36:40,811][86121] Updated weights for policy 0, policy_version 9700 (0.0008) +[2023-10-09 12:36:40,848][86122] Updated weights for policy 1, policy_version 9750 (0.0008) +[2023-10-09 12:36:41,176][86121] Updated weights for policy 0, policy_version 9710 (0.0009) +[2023-10-09 12:36:41,211][86122] Updated weights for policy 1, policy_version 9760 (0.0009) +[2023-10-09 12:36:41,548][86121] Updated weights for policy 0, policy_version 9720 (0.0007) +[2023-10-09 12:36:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 19955712. Throughput: 0: 1808.6, 1: 1821.0. Samples: 4991640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:36:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:36:45,034][86122] Updated weights for policy 1, policy_version 9770 (0.0009) +[2023-10-09 12:36:45,284][86121] Updated weights for policy 0, policy_version 9730 (0.0008) +[2023-10-09 12:36:45,405][86122] Updated weights for policy 1, policy_version 9780 (0.0008) +[2023-10-09 12:36:45,657][86121] Updated weights for policy 0, policy_version 9740 (0.0010) +[2023-10-09 12:36:45,763][86122] Updated weights for policy 1, policy_version 9790 (0.0008) +[2023-10-09 12:36:46,029][86121] Updated weights for policy 0, policy_version 9750 (0.0008) +[2023-10-09 12:36:46,399][86121] Updated weights for policy 0, policy_version 9760 (0.0007) +[2023-10-09 12:36:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20021248. Throughput: 0: 1798.1, 1: 1826.4. Samples: 5012862. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:36:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:36:49,373][86122] Updated weights for policy 1, policy_version 9800 (0.0008) +[2023-10-09 12:36:49,731][86122] Updated weights for policy 1, policy_version 9810 (0.0008) +[2023-10-09 12:36:50,010][86121] Updated weights for policy 0, policy_version 9770 (0.0009) +[2023-10-09 12:36:50,104][86122] Updated weights for policy 1, policy_version 9820 (0.0008) +[2023-10-09 12:36:50,386][86121] Updated weights for policy 0, policy_version 9780 (0.0009) +[2023-10-09 12:36:50,755][86121] Updated weights for policy 0, policy_version 9790 (0.0009) +[2023-10-09 12:36:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 20086784. Throughput: 0: 1803.2, 1: 1820.5. Samples: 5035914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:36:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:36:53,675][86122] Updated weights for policy 1, policy_version 9830 (0.0011) +[2023-10-09 12:36:54,037][86122] Updated weights for policy 1, policy_version 9840 (0.0010) +[2023-10-09 12:36:54,383][86121] Updated weights for policy 0, policy_version 9800 (0.0007) +[2023-10-09 12:36:54,404][86122] Updated weights for policy 1, policy_version 9850 (0.0007) +[2023-10-09 12:36:54,741][86121] Updated weights for policy 0, policy_version 9810 (0.0007) +[2023-10-09 12:36:55,119][86121] Updated weights for policy 0, policy_version 9820 (0.0007) +[2023-10-09 12:36:57,972][86122] Updated weights for policy 1, policy_version 9860 (0.0008) +[2023-10-09 12:36:58,329][86122] Updated weights for policy 1, policy_version 9870 (0.0010) +[2023-10-09 12:36:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20152320. Throughput: 0: 1804.1, 1: 1823.9. Samples: 5045804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:36:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:36:58,695][86122] Updated weights for policy 1, policy_version 9880 (0.0008) +[2023-10-09 12:36:58,901][86121] Updated weights for policy 0, policy_version 9830 (0.0010) +[2023-10-09 12:36:59,267][86121] Updated weights for policy 0, policy_version 9840 (0.0011) +[2023-10-09 12:36:59,638][86121] Updated weights for policy 0, policy_version 9850 (0.0011) +[2023-10-09 12:37:02,576][86122] Updated weights for policy 1, policy_version 9890 (0.0008) +[2023-10-09 12:37:02,954][86122] Updated weights for policy 1, policy_version 9900 (0.0007) +[2023-10-09 12:37:03,322][86122] Updated weights for policy 1, policy_version 9910 (0.0008) +[2023-10-09 12:37:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20217856. Throughput: 0: 1798.5, 1: 1836.9. Samples: 5068730. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 12:37:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:37:03,408][86121] Updated weights for policy 0, policy_version 9860 (0.0009) +[2023-10-09 12:37:03,692][86122] Updated weights for policy 1, policy_version 9920 (0.0007) +[2023-10-09 12:37:03,778][86121] Updated weights for policy 0, policy_version 9870 (0.0007) +[2023-10-09 12:37:04,137][86121] Updated weights for policy 0, policy_version 9880 (0.0008) +[2023-10-09 12:37:07,438][86122] Updated weights for policy 1, policy_version 9930 (0.0010) +[2023-10-09 12:37:07,805][86122] Updated weights for policy 1, policy_version 9940 (0.0007) +[2023-10-09 12:37:07,936][86121] Updated weights for policy 0, policy_version 9890 (0.0007) +[2023-10-09 12:37:08,171][86122] Updated weights for policy 1, policy_version 9950 (0.0009) +[2023-10-09 12:37:08,304][86121] Updated weights for policy 0, policy_version 9900 (0.0008) +[2023-10-09 12:37:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 20316160. Throughput: 0: 1818.2, 1: 1826.5. Samples: 5090428. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 12:37:08,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:37:08,668][86121] Updated weights for policy 0, policy_version 9910 (0.0008) +[2023-10-09 12:37:09,040][86121] Updated weights for policy 0, policy_version 9920 (0.0009) +[2023-10-09 12:37:11,877][86122] Updated weights for policy 1, policy_version 9960 (0.0010) +[2023-10-09 12:37:12,257][86122] Updated weights for policy 1, policy_version 9970 (0.0010) +[2023-10-09 12:37:12,625][86122] Updated weights for policy 1, policy_version 9980 (0.0007) +[2023-10-09 12:37:12,736][86121] Updated weights for policy 0, policy_version 9930 (0.0010) +[2023-10-09 12:37:13,099][86121] Updated weights for policy 0, policy_version 9940 (0.0009) +[2023-10-09 12:37:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 20381696. Throughput: 0: 1800.8, 1: 1831.0. Samples: 5101118. Policy #0 lag: (min: 24.0, avg: 50.7, max: 56.0) +[2023-10-09 12:37:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:37:13,458][86121] Updated weights for policy 0, policy_version 9950 (0.0010) +[2023-10-09 12:37:16,282][86122] Updated weights for policy 1, policy_version 9990 (0.0009) +[2023-10-09 12:37:16,642][86122] Updated weights for policy 1, policy_version 10000 (0.0008) +[2023-10-09 12:37:17,008][86122] Updated weights for policy 1, policy_version 10010 (0.0009) +[2023-10-09 12:37:17,134][86121] Updated weights for policy 0, policy_version 9960 (0.0009) +[2023-10-09 12:37:17,503][86121] Updated weights for policy 0, policy_version 9970 (0.0008) +[2023-10-09 12:37:17,868][86121] Updated weights for policy 0, policy_version 9980 (0.0007) +[2023-10-09 12:37:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 20480000. Throughput: 0: 1814.8, 1: 1823.0. Samples: 5122744. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 12:37:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:37:20,582][86122] Updated weights for policy 1, policy_version 10020 (0.0008) +[2023-10-09 12:37:20,942][86122] Updated weights for policy 1, policy_version 10030 (0.0010) +[2023-10-09 12:37:21,313][86122] Updated weights for policy 1, policy_version 10040 (0.0009) +[2023-10-09 12:37:21,599][86121] Updated weights for policy 0, policy_version 9990 (0.0009) +[2023-10-09 12:37:21,968][86121] Updated weights for policy 0, policy_version 10000 (0.0009) +[2023-10-09 12:37:22,338][86121] Updated weights for policy 0, policy_version 10010 (0.0010) +[2023-10-09 12:37:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 20545536. Throughput: 0: 1811.8, 1: 1825.6. Samples: 5143736. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 12:37:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:37:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000010048_10289152.pth... +[2023-10-09 12:37:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000010016_10256384.pth... +[2023-10-09 12:37:23,444][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000008320_8519680.pth +[2023-10-09 12:37:23,449][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000008352_8552448.pth +[2023-10-09 12:37:24,997][86122] Updated weights for policy 1, policy_version 10050 (0.0007) +[2023-10-09 12:37:25,358][86122] Updated weights for policy 1, policy_version 10060 (0.0008) +[2023-10-09 12:37:25,721][86122] Updated weights for policy 1, policy_version 10070 (0.0011) +[2023-10-09 12:37:26,078][86121] Updated weights for policy 0, policy_version 10020 (0.0008) +[2023-10-09 12:37:26,091][86122] Updated weights for policy 1, policy_version 10080 (0.0008) +[2023-10-09 12:37:26,445][86121] Updated weights for policy 0, policy_version 10030 (0.0007) +[2023-10-09 12:37:26,817][86121] Updated weights for policy 0, policy_version 10040 (0.0008) +[2023-10-09 12:37:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20611072. Throughput: 0: 1814.7, 1: 1822.9. Samples: 5155334. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) +[2023-10-09 12:37:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:37:29,835][86122] Updated weights for policy 1, policy_version 10090 (0.0007) +[2023-10-09 12:37:30,202][86122] Updated weights for policy 1, policy_version 10100 (0.0008) +[2023-10-09 12:37:30,493][86121] Updated weights for policy 0, policy_version 10050 (0.0007) +[2023-10-09 12:37:30,578][86122] Updated weights for policy 1, policy_version 10110 (0.0009) +[2023-10-09 12:37:30,864][86121] Updated weights for policy 0, policy_version 10060 (0.0007) +[2023-10-09 12:37:31,227][86121] Updated weights for policy 0, policy_version 10070 (0.0009) +[2023-10-09 12:37:31,596][86121] Updated weights for policy 0, policy_version 10080 (0.0010) +[2023-10-09 12:37:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 20676608. Throughput: 0: 1808.7, 1: 1820.1. Samples: 5176158. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) +[2023-10-09 12:37:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:37:34,404][86122] Updated weights for policy 1, policy_version 10120 (0.0009) +[2023-10-09 12:37:34,764][86122] Updated weights for policy 1, policy_version 10130 (0.0008) +[2023-10-09 12:37:35,128][86122] Updated weights for policy 1, policy_version 10140 (0.0008) +[2023-10-09 12:37:35,214][86121] Updated weights for policy 0, policy_version 10090 (0.0007) +[2023-10-09 12:37:35,584][86121] Updated weights for policy 0, policy_version 10100 (0.0007) +[2023-10-09 12:37:35,954][86121] Updated weights for policy 0, policy_version 10110 (0.0009) +[2023-10-09 12:37:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20742144. Throughput: 0: 1802.0, 1: 1814.3. Samples: 5198646. Policy #0 lag: (min: 31.0, avg: 39.2, max: 63.0) +[2023-10-09 12:37:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 12:37:38,871][86122] Updated weights for policy 1, policy_version 10150 (0.0008) +[2023-10-09 12:37:39,246][86122] Updated weights for policy 1, policy_version 10160 (0.0008) +[2023-10-09 12:37:39,618][86122] Updated weights for policy 1, policy_version 10170 (0.0009) +[2023-10-09 12:37:39,797][86121] Updated weights for policy 0, policy_version 10120 (0.0008) +[2023-10-09 12:37:40,169][86121] Updated weights for policy 0, policy_version 10130 (0.0008) +[2023-10-09 12:37:40,537][86121] Updated weights for policy 0, policy_version 10140 (0.0008) +[2023-10-09 12:37:43,360][86122] Updated weights for policy 1, policy_version 10180 (0.0008) +[2023-10-09 12:37:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 20807680. Throughput: 0: 1801.6, 1: 1813.4. Samples: 5208480. Policy #0 lag: (min: 9.0, avg: 18.5, max: 41.0) +[2023-10-09 12:37:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 12:37:43,724][86122] Updated weights for policy 1, policy_version 10190 (0.0008) +[2023-10-09 12:37:44,088][86122] Updated weights for policy 1, policy_version 10200 (0.0008) +[2023-10-09 12:37:44,360][86121] Updated weights for policy 0, policy_version 10150 (0.0008) +[2023-10-09 12:37:44,730][86121] Updated weights for policy 0, policy_version 10160 (0.0008) +[2023-10-09 12:37:45,107][86121] Updated weights for policy 0, policy_version 10170 (0.0008) +[2023-10-09 12:37:47,885][86122] Updated weights for policy 1, policy_version 10210 (0.0007) +[2023-10-09 12:37:48,246][86122] Updated weights for policy 1, policy_version 10220 (0.0010) +[2023-10-09 12:37:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 20873216. Throughput: 0: 1797.7, 1: 1801.6. Samples: 5230702. Policy #0 lag: (min: 9.0, avg: 18.5, max: 41.0) +[2023-10-09 12:37:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:37:48,617][86122] Updated weights for policy 1, policy_version 10230 (0.0008) +[2023-10-09 12:37:48,979][86122] Updated weights for policy 1, policy_version 10240 (0.0009) +[2023-10-09 12:37:48,985][86121] Updated weights for policy 0, policy_version 10180 (0.0008) +[2023-10-09 12:37:49,350][86121] Updated weights for policy 0, policy_version 10190 (0.0007) +[2023-10-09 12:37:49,731][86121] Updated weights for policy 0, policy_version 10200 (0.0007) +[2023-10-09 12:37:52,629][86122] Updated weights for policy 1, policy_version 10250 (0.0010) +[2023-10-09 12:37:52,989][86122] Updated weights for policy 1, policy_version 10260 (0.0010) +[2023-10-09 12:37:53,356][86122] Updated weights for policy 1, policy_version 10270 (0.0011) +[2023-10-09 12:37:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 20938752. Throughput: 0: 1795.7, 1: 1808.3. Samples: 5252606. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:37:53,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:37:53,503][86121] Updated weights for policy 0, policy_version 10210 (0.0008) +[2023-10-09 12:37:53,867][86121] Updated weights for policy 0, policy_version 10220 (0.0009) +[2023-10-09 12:37:54,230][86121] Updated weights for policy 0, policy_version 10230 (0.0010) +[2023-10-09 12:37:54,598][86121] Updated weights for policy 0, policy_version 10240 (0.0010) +[2023-10-09 12:37:57,189][86122] Updated weights for policy 1, policy_version 10280 (0.0008) +[2023-10-09 12:37:57,566][86122] Updated weights for policy 1, policy_version 10290 (0.0007) +[2023-10-09 12:37:57,934][86122] Updated weights for policy 1, policy_version 10300 (0.0007) +[2023-10-09 12:37:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21037056. Throughput: 0: 1795.1, 1: 1801.2. Samples: 5262952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:37:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:37:58,401][86121] Updated weights for policy 0, policy_version 10250 (0.0009) +[2023-10-09 12:37:58,768][86121] Updated weights for policy 0, policy_version 10260 (0.0008) +[2023-10-09 12:37:59,142][86121] Updated weights for policy 0, policy_version 10270 (0.0008) +[2023-10-09 12:38:01,663][86122] Updated weights for policy 1, policy_version 10310 (0.0010) +[2023-10-09 12:38:02,033][86122] Updated weights for policy 1, policy_version 10320 (0.0011) +[2023-10-09 12:38:02,398][86122] Updated weights for policy 1, policy_version 10330 (0.0010) +[2023-10-09 12:38:02,749][86121] Updated weights for policy 0, policy_version 10280 (0.0009) +[2023-10-09 12:38:03,118][86121] Updated weights for policy 0, policy_version 10290 (0.0009) +[2023-10-09 12:38:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 21102592. Throughput: 0: 1790.2, 1: 1806.6. Samples: 5284598. Policy #0 lag: (min: 10.0, avg: 10.9, max: 24.0) +[2023-10-09 12:38:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:38:03,481][86121] Updated weights for policy 0, policy_version 10300 (0.0008) +[2023-10-09 12:38:06,176][86122] Updated weights for policy 1, policy_version 10340 (0.0008) +[2023-10-09 12:38:06,551][86122] Updated weights for policy 1, policy_version 10350 (0.0007) +[2023-10-09 12:38:06,917][86122] Updated weights for policy 1, policy_version 10360 (0.0008) +[2023-10-09 12:38:07,431][86121] Updated weights for policy 0, policy_version 10310 (0.0008) +[2023-10-09 12:38:07,816][86121] Updated weights for policy 0, policy_version 10320 (0.0008) +[2023-10-09 12:38:08,185][86121] Updated weights for policy 0, policy_version 10330 (0.0011) +[2023-10-09 12:38:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21168128. Throughput: 0: 1797.7, 1: 1793.7. Samples: 5305350. Policy #0 lag: (min: 10.0, avg: 10.9, max: 24.0) +[2023-10-09 12:38:08,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:38:10,726][86122] Updated weights for policy 1, policy_version 10370 (0.0008) +[2023-10-09 12:38:11,099][86122] Updated weights for policy 1, policy_version 10380 (0.0007) +[2023-10-09 12:38:11,463][86122] Updated weights for policy 1, policy_version 10390 (0.0007) +[2023-10-09 12:38:11,822][86122] Updated weights for policy 1, policy_version 10400 (0.0008) +[2023-10-09 12:38:11,945][86121] Updated weights for policy 0, policy_version 10340 (0.0008) +[2023-10-09 12:38:12,314][86121] Updated weights for policy 0, policy_version 10350 (0.0007) +[2023-10-09 12:38:12,686][86121] Updated weights for policy 0, policy_version 10360 (0.0008) +[2023-10-09 12:38:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 21266432. Throughput: 0: 1782.6, 1: 1813.1. Samples: 5317140. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) +[2023-10-09 12:38:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:38:15,767][86122] Updated weights for policy 1, policy_version 10410 (0.0007) +[2023-10-09 12:38:16,129][86122] Updated weights for policy 1, policy_version 10420 (0.0008) +[2023-10-09 12:38:16,385][86121] Updated weights for policy 0, policy_version 10370 (0.0007) +[2023-10-09 12:38:16,496][86122] Updated weights for policy 1, policy_version 10430 (0.0009) +[2023-10-09 12:38:16,745][86121] Updated weights for policy 0, policy_version 10380 (0.0008) +[2023-10-09 12:38:17,117][86121] Updated weights for policy 0, policy_version 10390 (0.0008) +[2023-10-09 12:38:17,482][86121] Updated weights for policy 0, policy_version 10400 (0.0009) +[2023-10-09 12:38:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21331968. Throughput: 0: 1796.7, 1: 1790.1. Samples: 5337564. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) +[2023-10-09 12:38:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:38:20,323][86122] Updated weights for policy 1, policy_version 10440 (0.0009) +[2023-10-09 12:38:20,681][86122] Updated weights for policy 1, policy_version 10450 (0.0009) +[2023-10-09 12:38:21,045][86122] Updated weights for policy 1, policy_version 10460 (0.0008) +[2023-10-09 12:38:21,142][86121] Updated weights for policy 0, policy_version 10410 (0.0008) +[2023-10-09 12:38:21,513][86121] Updated weights for policy 0, policy_version 10420 (0.0009) +[2023-10-09 12:38:21,892][86121] Updated weights for policy 0, policy_version 10430 (0.0008) +[2023-10-09 12:38:23,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 21397504. Throughput: 0: 1783.5, 1: 1786.7. Samples: 5359306. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 12:38:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:38:24,872][86122] Updated weights for policy 1, policy_version 10470 (0.0008) +[2023-10-09 12:38:25,235][86122] Updated weights for policy 1, policy_version 10480 (0.0007) +[2023-10-09 12:38:25,555][86121] Updated weights for policy 0, policy_version 10440 (0.0010) +[2023-10-09 12:38:25,594][86122] Updated weights for policy 1, policy_version 10490 (0.0009) +[2023-10-09 12:38:25,922][86121] Updated weights for policy 0, policy_version 10450 (0.0007) +[2023-10-09 12:38:26,289][86121] Updated weights for policy 0, policy_version 10460 (0.0007) +[2023-10-09 12:38:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 21463040. Throughput: 0: 1799.6, 1: 1784.0. Samples: 5369740. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 12:38:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 12:38:29,220][86122] Updated weights for policy 1, policy_version 10500 (0.0008) +[2023-10-09 12:38:29,588][86122] Updated weights for policy 1, policy_version 10510 (0.0010) +[2023-10-09 12:38:29,958][86122] Updated weights for policy 1, policy_version 10520 (0.0007) +[2023-10-09 12:38:29,965][86121] Updated weights for policy 0, policy_version 10470 (0.0007) +[2023-10-09 12:38:30,333][86121] Updated weights for policy 0, policy_version 10480 (0.0007) +[2023-10-09 12:38:30,704][86121] Updated weights for policy 0, policy_version 10490 (0.0010) +[2023-10-09 12:38:33,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21528576. Throughput: 0: 1788.8, 1: 1783.0. Samples: 5391434. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 12:38:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 12:38:33,738][86122] Updated weights for policy 1, policy_version 10530 (0.0008) +[2023-10-09 12:38:34,103][86122] Updated weights for policy 1, policy_version 10540 (0.0009) +[2023-10-09 12:38:34,471][86122] Updated weights for policy 1, policy_version 10550 (0.0007) +[2023-10-09 12:38:34,486][86121] Updated weights for policy 0, policy_version 10500 (0.0009) +[2023-10-09 12:38:34,841][86122] Updated weights for policy 1, policy_version 10560 (0.0007) +[2023-10-09 12:38:34,854][86121] Updated weights for policy 0, policy_version 10510 (0.0007) +[2023-10-09 12:38:35,217][86121] Updated weights for policy 0, policy_version 10520 (0.0008) +[2023-10-09 12:38:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21594112. Throughput: 0: 1789.7, 1: 1794.7. Samples: 5413906. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 12:38:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 12:38:38,634][86122] Updated weights for policy 1, policy_version 10570 (0.0008) +[2023-10-09 12:38:38,988][86122] Updated weights for policy 1, policy_version 10580 (0.0007) +[2023-10-09 12:38:38,999][86121] Updated weights for policy 0, policy_version 10530 (0.0008) +[2023-10-09 12:38:39,355][86122] Updated weights for policy 1, policy_version 10590 (0.0009) +[2023-10-09 12:38:39,374][86121] Updated weights for policy 0, policy_version 10540 (0.0009) +[2023-10-09 12:38:39,734][86121] Updated weights for policy 0, policy_version 10550 (0.0010) +[2023-10-09 12:38:40,105][86121] Updated weights for policy 0, policy_version 10560 (0.0010) +[2023-10-09 12:38:43,105][86122] Updated weights for policy 1, policy_version 10600 (0.0009) +[2023-10-09 12:38:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 21659648. Throughput: 0: 1789.2, 1: 1782.4. Samples: 5423678. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 12:38:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:38:43,464][86122] Updated weights for policy 1, policy_version 10610 (0.0007) +[2023-10-09 12:38:43,800][86121] Updated weights for policy 0, policy_version 10570 (0.0010) +[2023-10-09 12:38:43,824][86122] Updated weights for policy 1, policy_version 10620 (0.0008) +[2023-10-09 12:38:44,171][86121] Updated weights for policy 0, policy_version 10580 (0.0009) +[2023-10-09 12:38:44,538][86121] Updated weights for policy 0, policy_version 10590 (0.0007) +[2023-10-09 12:38:47,429][86122] Updated weights for policy 1, policy_version 10630 (0.0008) +[2023-10-09 12:38:47,792][86122] Updated weights for policy 1, policy_version 10640 (0.0010) +[2023-10-09 12:38:48,160][86122] Updated weights for policy 1, policy_version 10650 (0.0010) +[2023-10-09 12:38:48,227][86121] Updated weights for policy 0, policy_version 10600 (0.0009) +[2023-10-09 12:38:48,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21757952. Throughput: 0: 1798.1, 1: 1800.2. Samples: 5446520. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) +[2023-10-09 12:38:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:38:48,596][86121] Updated weights for policy 0, policy_version 10610 (0.0008) +[2023-10-09 12:38:48,959][86121] Updated weights for policy 0, policy_version 10620 (0.0009) +[2023-10-09 12:38:51,879][86122] Updated weights for policy 1, policy_version 10660 (0.0009) +[2023-10-09 12:38:52,244][86122] Updated weights for policy 1, policy_version 10670 (0.0008) +[2023-10-09 12:38:52,601][86122] Updated weights for policy 1, policy_version 10680 (0.0008) +[2023-10-09 12:38:52,790][86121] Updated weights for policy 0, policy_version 10630 (0.0007) +[2023-10-09 12:38:53,178][86121] Updated weights for policy 0, policy_version 10640 (0.0007) +[2023-10-09 12:38:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 21823488. Throughput: 0: 1807.6, 1: 1794.9. Samples: 5467462. Policy #0 lag: (min: 25.0, avg: 33.2, max: 57.0) +[2023-10-09 12:38:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:38:53,544][86121] Updated weights for policy 0, policy_version 10650 (0.0007) +[2023-10-09 12:38:56,280][86122] Updated weights for policy 1, policy_version 10690 (0.0008) +[2023-10-09 12:38:56,656][86122] Updated weights for policy 1, policy_version 10700 (0.0008) +[2023-10-09 12:38:57,019][86122] Updated weights for policy 1, policy_version 10710 (0.0009) +[2023-10-09 12:38:57,196][86121] Updated weights for policy 0, policy_version 10660 (0.0010) +[2023-10-09 12:38:57,377][86122] Updated weights for policy 1, policy_version 10720 (0.0008) +[2023-10-09 12:38:57,558][86121] Updated weights for policy 0, policy_version 10670 (0.0009) +[2023-10-09 12:38:57,925][86121] Updated weights for policy 0, policy_version 10680 (0.0010) +[2023-10-09 12:38:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 21921792. Throughput: 0: 1802.1, 1: 1799.2. Samples: 5479200. Policy #0 lag: (min: 27.0, avg: 30.8, max: 59.0) +[2023-10-09 12:38:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:39:01,099][86122] Updated weights for policy 1, policy_version 10730 (0.0007) +[2023-10-09 12:39:01,465][86122] Updated weights for policy 1, policy_version 10740 (0.0007) +[2023-10-09 12:39:01,695][86121] Updated weights for policy 0, policy_version 10690 (0.0007) +[2023-10-09 12:39:01,826][86122] Updated weights for policy 1, policy_version 10750 (0.0007) +[2023-10-09 12:39:02,064][86121] Updated weights for policy 0, policy_version 10700 (0.0009) +[2023-10-09 12:39:02,432][86121] Updated weights for policy 0, policy_version 10710 (0.0009) +[2023-10-09 12:39:02,801][86121] Updated weights for policy 0, policy_version 10720 (0.0007) +[2023-10-09 12:39:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 21987328. Throughput: 0: 1809.2, 1: 1804.8. Samples: 5500196. Policy #0 lag: (min: 27.0, avg: 30.8, max: 59.0) +[2023-10-09 12:39:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:39:05,480][86122] Updated weights for policy 1, policy_version 10760 (0.0008) +[2023-10-09 12:39:05,857][86122] Updated weights for policy 1, policy_version 10770 (0.0008) +[2023-10-09 12:39:06,220][86122] Updated weights for policy 1, policy_version 10780 (0.0009) +[2023-10-09 12:39:06,358][86121] Updated weights for policy 0, policy_version 10730 (0.0008) +[2023-10-09 12:39:06,737][86121] Updated weights for policy 0, policy_version 10740 (0.0009) +[2023-10-09 12:39:07,100][86121] Updated weights for policy 0, policy_version 10750 (0.0010) +[2023-10-09 12:39:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 22052864. Throughput: 0: 1804.5, 1: 1804.2. Samples: 5521696. Policy #0 lag: (min: 27.0, avg: 30.8, max: 59.0) +[2023-10-09 12:39:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:39:09,938][86122] Updated weights for policy 1, policy_version 10790 (0.0008) +[2023-10-09 12:39:10,306][86122] Updated weights for policy 1, policy_version 10800 (0.0009) +[2023-10-09 12:39:10,670][86122] Updated weights for policy 1, policy_version 10810 (0.0008) +[2023-10-09 12:39:10,823][86121] Updated weights for policy 0, policy_version 10760 (0.0008) +[2023-10-09 12:39:11,185][86121] Updated weights for policy 0, policy_version 10770 (0.0008) +[2023-10-09 12:39:11,565][86121] Updated weights for policy 0, policy_version 10780 (0.0007) +[2023-10-09 12:39:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22118400. Throughput: 0: 1816.0, 1: 1810.8. Samples: 5532944. Policy #0 lag: (min: 5.0, avg: 8.2, max: 37.0) +[2023-10-09 12:39:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:39:14,348][86122] Updated weights for policy 1, policy_version 10820 (0.0010) +[2023-10-09 12:39:14,722][86122] Updated weights for policy 1, policy_version 10830 (0.0009) +[2023-10-09 12:39:15,089][86122] Updated weights for policy 1, policy_version 10840 (0.0009) +[2023-10-09 12:39:15,245][86121] Updated weights for policy 0, policy_version 10790 (0.0008) +[2023-10-09 12:39:15,613][86121] Updated weights for policy 0, policy_version 10800 (0.0010) +[2023-10-09 12:39:15,997][86121] Updated weights for policy 0, policy_version 10810 (0.0009) +[2023-10-09 12:39:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 22183936. Throughput: 0: 1811.9, 1: 1818.2. Samples: 5554790. Policy #0 lag: (min: 5.0, avg: 8.2, max: 37.0) +[2023-10-09 12:39:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:39:18,847][86122] Updated weights for policy 1, policy_version 10850 (0.0008) +[2023-10-09 12:39:19,216][86122] Updated weights for policy 1, policy_version 10860 (0.0008) +[2023-10-09 12:39:19,581][86122] Updated weights for policy 1, policy_version 10870 (0.0007) +[2023-10-09 12:39:19,615][86121] Updated weights for policy 0, policy_version 10820 (0.0009) +[2023-10-09 12:39:19,939][86122] Updated weights for policy 1, policy_version 10880 (0.0007) +[2023-10-09 12:39:19,987][86121] Updated weights for policy 0, policy_version 10830 (0.0008) +[2023-10-09 12:39:20,353][86121] Updated weights for policy 0, policy_version 10840 (0.0007) +[2023-10-09 12:39:23,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22249472. Throughput: 0: 1813.1, 1: 1820.1. Samples: 5577402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:39:23,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 12:39:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000010848_11108352.pth... +[2023-10-09 12:39:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000009184_9404416.pth +[2023-10-09 12:39:23,625][86122] Updated weights for policy 1, policy_version 10890 (0.0009) +[2023-10-09 12:39:23,994][86122] Updated weights for policy 1, policy_version 10900 (0.0008) +[2023-10-09 12:39:24,092][86121] Updated weights for policy 0, policy_version 10850 (0.0008) +[2023-10-09 12:39:24,360][86122] Updated weights for policy 1, policy_version 10910 (0.0008) +[2023-10-09 12:39:24,434][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000010912_11173888.pth... +[2023-10-09 12:39:24,459][86121] Updated weights for policy 0, policy_version 10860 (0.0007) +[2023-10-09 12:39:24,462][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000009184_9404416.pth +[2023-10-09 12:39:24,828][86121] Updated weights for policy 0, policy_version 10870 (0.0007) +[2023-10-09 12:39:25,204][86121] Updated weights for policy 0, policy_version 10880 (0.0008) +[2023-10-09 12:39:28,132][86122] Updated weights for policy 1, policy_version 10920 (0.0009) +[2023-10-09 12:39:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22315008. Throughput: 0: 1816.0, 1: 1819.9. Samples: 5587292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:39:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 12:39:28,502][86122] Updated weights for policy 1, policy_version 10930 (0.0008) +[2023-10-09 12:39:28,873][86122] Updated weights for policy 1, policy_version 10940 (0.0007) +[2023-10-09 12:39:29,027][86121] Updated weights for policy 0, policy_version 10890 (0.0008) +[2023-10-09 12:39:29,404][86121] Updated weights for policy 0, policy_version 10900 (0.0009) +[2023-10-09 12:39:29,770][86121] Updated weights for policy 0, policy_version 10910 (0.0008) +[2023-10-09 12:39:32,449][86122] Updated weights for policy 1, policy_version 10950 (0.0008) +[2023-10-09 12:39:32,812][86122] Updated weights for policy 1, policy_version 10960 (0.0008) +[2023-10-09 12:39:33,183][86122] Updated weights for policy 1, policy_version 10970 (0.0008) +[2023-10-09 12:39:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 22380544. Throughput: 0: 1807.0, 1: 1818.6. Samples: 5609672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:39:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:39:33,585][86121] Updated weights for policy 0, policy_version 10920 (0.0009) +[2023-10-09 12:39:33,951][86121] Updated weights for policy 0, policy_version 10930 (0.0007) +[2023-10-09 12:39:34,329][86121] Updated weights for policy 0, policy_version 10940 (0.0009) +[2023-10-09 12:39:36,779][86122] Updated weights for policy 1, policy_version 10980 (0.0007) +[2023-10-09 12:39:37,147][86122] Updated weights for policy 1, policy_version 10990 (0.0007) +[2023-10-09 12:39:37,514][86122] Updated weights for policy 1, policy_version 11000 (0.0008) +[2023-10-09 12:39:38,176][86121] Updated weights for policy 0, policy_version 10950 (0.0010) +[2023-10-09 12:39:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 22478848. Throughput: 0: 1818.2, 1: 1816.8. Samples: 5631038. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 12:39:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:39:38,549][86121] Updated weights for policy 0, policy_version 10960 (0.0008) +[2023-10-09 12:39:38,921][86121] Updated weights for policy 0, policy_version 10970 (0.0009) +[2023-10-09 12:39:41,196][86122] Updated weights for policy 1, policy_version 11010 (0.0008) +[2023-10-09 12:39:41,558][86122] Updated weights for policy 1, policy_version 11020 (0.0010) +[2023-10-09 12:39:41,925][86122] Updated weights for policy 1, policy_version 11030 (0.0008) +[2023-10-09 12:39:42,295][86122] Updated weights for policy 1, policy_version 11040 (0.0007) +[2023-10-09 12:39:42,644][86121] Updated weights for policy 0, policy_version 10980 (0.0008) +[2023-10-09 12:39:43,011][86121] Updated weights for policy 0, policy_version 10990 (0.0008) +[2023-10-09 12:39:43,381][86121] Updated weights for policy 0, policy_version 11000 (0.0007) +[2023-10-09 12:39:43,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 22544384. Throughput: 0: 1804.9, 1: 1820.1. Samples: 5642328. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 12:39:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:39:45,850][86122] Updated weights for policy 1, policy_version 11050 (0.0009) +[2023-10-09 12:39:46,219][86122] Updated weights for policy 1, policy_version 11060 (0.0008) +[2023-10-09 12:39:46,578][86122] Updated weights for policy 1, policy_version 11070 (0.0008) +[2023-10-09 12:39:46,988][86121] Updated weights for policy 0, policy_version 11010 (0.0008) +[2023-10-09 12:39:47,356][86121] Updated weights for policy 0, policy_version 11020 (0.0009) +[2023-10-09 12:39:47,730][86121] Updated weights for policy 0, policy_version 11030 (0.0010) +[2023-10-09 12:39:48,101][86121] Updated weights for policy 0, policy_version 11040 (0.0011) +[2023-10-09 12:39:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 22642688. Throughput: 0: 1812.1, 1: 1823.0. Samples: 5663778. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) +[2023-10-09 12:39:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:39:50,329][86122] Updated weights for policy 1, policy_version 11080 (0.0009) +[2023-10-09 12:39:50,694][86122] Updated weights for policy 1, policy_version 11090 (0.0009) +[2023-10-09 12:39:51,065][86122] Updated weights for policy 1, policy_version 11100 (0.0010) +[2023-10-09 12:39:51,867][86121] Updated weights for policy 0, policy_version 11050 (0.0009) +[2023-10-09 12:39:52,240][86121] Updated weights for policy 0, policy_version 11060 (0.0010) +[2023-10-09 12:39:52,617][86121] Updated weights for policy 0, policy_version 11070 (0.0007) +[2023-10-09 12:39:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 22708224. Throughput: 0: 1795.5, 1: 1828.1. Samples: 5684756. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) +[2023-10-09 12:39:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:39:54,717][86122] Updated weights for policy 1, policy_version 11110 (0.0011) +[2023-10-09 12:39:55,090][86122] Updated weights for policy 1, policy_version 11120 (0.0008) +[2023-10-09 12:39:55,450][86122] Updated weights for policy 1, policy_version 11130 (0.0010) +[2023-10-09 12:39:56,292][86121] Updated weights for policy 0, policy_version 11080 (0.0007) +[2023-10-09 12:39:56,668][86121] Updated weights for policy 0, policy_version 11090 (0.0009) +[2023-10-09 12:39:57,041][86121] Updated weights for policy 0, policy_version 11100 (0.0007) +[2023-10-09 12:39:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 22773760. Throughput: 0: 1809.7, 1: 1824.9. Samples: 5696500. Policy #0 lag: (min: 26.0, avg: 28.7, max: 58.0) +[2023-10-09 12:39:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:39:59,220][86122] Updated weights for policy 1, policy_version 11140 (0.0008) +[2023-10-09 12:39:59,587][86122] Updated weights for policy 1, policy_version 11150 (0.0007) +[2023-10-09 12:39:59,949][86122] Updated weights for policy 1, policy_version 11160 (0.0007) +[2023-10-09 12:40:00,778][86121] Updated weights for policy 0, policy_version 11110 (0.0009) +[2023-10-09 12:40:01,155][86121] Updated weights for policy 0, policy_version 11120 (0.0010) +[2023-10-09 12:40:01,522][86121] Updated weights for policy 0, policy_version 11130 (0.0010) +[2023-10-09 12:40:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22839296. Throughput: 0: 1796.2, 1: 1822.6. Samples: 5717638. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 12:40:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:40:03,520][86122] Updated weights for policy 1, policy_version 11170 (0.0010) +[2023-10-09 12:40:03,875][86122] Updated weights for policy 1, policy_version 11180 (0.0009) +[2023-10-09 12:40:04,242][86122] Updated weights for policy 1, policy_version 11190 (0.0009) +[2023-10-09 12:40:04,610][86122] Updated weights for policy 1, policy_version 11200 (0.0008) +[2023-10-09 12:40:05,028][86121] Updated weights for policy 0, policy_version 11140 (0.0009) +[2023-10-09 12:40:05,394][86121] Updated weights for policy 0, policy_version 11150 (0.0010) +[2023-10-09 12:40:05,754][86121] Updated weights for policy 0, policy_version 11160 (0.0012) +[2023-10-09 12:40:08,343][86122] Updated weights for policy 1, policy_version 11210 (0.0008) +[2023-10-09 12:40:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 22904832. Throughput: 0: 1806.6, 1: 1827.1. Samples: 5740918. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 12:40:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:40:08,704][86122] Updated weights for policy 1, policy_version 11220 (0.0008) +[2023-10-09 12:40:09,071][86122] Updated weights for policy 1, policy_version 11230 (0.0008) +[2023-10-09 12:40:09,313][86121] Updated weights for policy 0, policy_version 11170 (0.0009) +[2023-10-09 12:40:09,695][86121] Updated weights for policy 0, policy_version 11180 (0.0011) +[2023-10-09 12:40:10,058][86121] Updated weights for policy 0, policy_version 11190 (0.0009) +[2023-10-09 12:40:10,427][86121] Updated weights for policy 0, policy_version 11200 (0.0010) +[2023-10-09 12:40:12,807][86122] Updated weights for policy 1, policy_version 11240 (0.0008) +[2023-10-09 12:40:13,166][86122] Updated weights for policy 1, policy_version 11250 (0.0008) +[2023-10-09 12:40:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 22970368. Throughput: 0: 1806.9, 1: 1826.2. Samples: 5750782. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 12:40:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:40:13,530][86122] Updated weights for policy 1, policy_version 11260 (0.0010) +[2023-10-09 12:40:14,158][86121] Updated weights for policy 0, policy_version 11210 (0.0010) +[2023-10-09 12:40:14,535][86121] Updated weights for policy 0, policy_version 11220 (0.0011) +[2023-10-09 12:40:14,905][86121] Updated weights for policy 0, policy_version 11230 (0.0010) +[2023-10-09 12:40:17,346][86122] Updated weights for policy 1, policy_version 11270 (0.0008) +[2023-10-09 12:40:17,715][86122] Updated weights for policy 1, policy_version 11280 (0.0009) +[2023-10-09 12:40:18,093][86122] Updated weights for policy 1, policy_version 11290 (0.0009) +[2023-10-09 12:40:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23068672. Throughput: 0: 1811.2, 1: 1830.4. Samples: 5773542. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) +[2023-10-09 12:40:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 12:40:18,664][86121] Updated weights for policy 0, policy_version 11240 (0.0008) +[2023-10-09 12:40:19,028][86121] Updated weights for policy 0, policy_version 11250 (0.0008) +[2023-10-09 12:40:19,401][86121] Updated weights for policy 0, policy_version 11260 (0.0010) +[2023-10-09 12:40:21,781][86122] Updated weights for policy 1, policy_version 11300 (0.0009) +[2023-10-09 12:40:22,140][86122] Updated weights for policy 1, policy_version 11310 (0.0007) +[2023-10-09 12:40:22,512][86122] Updated weights for policy 1, policy_version 11320 (0.0007) +[2023-10-09 12:40:23,252][86121] Updated weights for policy 0, policy_version 11270 (0.0010) +[2023-10-09 12:40:23,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 23134208. Throughput: 0: 1810.1, 1: 1821.0. Samples: 5794436. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) +[2023-10-09 12:40:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 12:40:23,620][86121] Updated weights for policy 0, policy_version 11280 (0.0009) +[2023-10-09 12:40:23,991][86121] Updated weights for policy 0, policy_version 11290 (0.0010) +[2023-10-09 12:40:26,235][86122] Updated weights for policy 1, policy_version 11330 (0.0007) +[2023-10-09 12:40:26,595][86122] Updated weights for policy 1, policy_version 11340 (0.0008) +[2023-10-09 12:40:26,971][86122] Updated weights for policy 1, policy_version 11350 (0.0010) +[2023-10-09 12:40:27,333][86122] Updated weights for policy 1, policy_version 11360 (0.0010) +[2023-10-09 12:40:27,606][86121] Updated weights for policy 0, policy_version 11300 (0.0010) +[2023-10-09 12:40:27,980][86121] Updated weights for policy 0, policy_version 11310 (0.0010) +[2023-10-09 12:40:28,351][86121] Updated weights for policy 0, policy_version 11320 (0.0009) +[2023-10-09 12:40:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23199744. Throughput: 0: 1812.0, 1: 1822.3. Samples: 5805870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:40:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:40:31,028][86122] Updated weights for policy 1, policy_version 11370 (0.0008) +[2023-10-09 12:40:31,403][86122] Updated weights for policy 1, policy_version 11380 (0.0007) +[2023-10-09 12:40:31,765][86122] Updated weights for policy 1, policy_version 11390 (0.0008) +[2023-10-09 12:40:32,129][86121] Updated weights for policy 0, policy_version 11330 (0.0008) +[2023-10-09 12:40:32,497][86121] Updated weights for policy 0, policy_version 11340 (0.0007) +[2023-10-09 12:40:32,873][86121] Updated weights for policy 0, policy_version 11350 (0.0007) +[2023-10-09 12:40:33,247][86121] Updated weights for policy 0, policy_version 11360 (0.0007) +[2023-10-09 12:40:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14440.1). Total num frames: 23298048. Throughput: 0: 1816.0, 1: 1814.7. Samples: 5827156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:40:33,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:40:35,529][86122] Updated weights for policy 1, policy_version 11400 (0.0009) +[2023-10-09 12:40:35,895][86122] Updated weights for policy 1, policy_version 11410 (0.0007) +[2023-10-09 12:40:36,267][86122] Updated weights for policy 1, policy_version 11420 (0.0009) +[2023-10-09 12:40:36,904][86121] Updated weights for policy 0, policy_version 11370 (0.0007) +[2023-10-09 12:40:37,274][86121] Updated weights for policy 0, policy_version 11380 (0.0008) +[2023-10-09 12:40:37,638][86121] Updated weights for policy 0, policy_version 11390 (0.0008) +[2023-10-09 12:40:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 23363584. Throughput: 0: 1822.3, 1: 1816.4. Samples: 5848498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:40:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:40:39,937][86122] Updated weights for policy 1, policy_version 11430 (0.0008) +[2023-10-09 12:40:40,297][86122] Updated weights for policy 1, policy_version 11440 (0.0008) +[2023-10-09 12:40:40,661][86122] Updated weights for policy 1, policy_version 11450 (0.0010) +[2023-10-09 12:40:41,500][86121] Updated weights for policy 0, policy_version 11400 (0.0008) +[2023-10-09 12:40:41,867][86121] Updated weights for policy 0, policy_version 11410 (0.0007) +[2023-10-09 12:40:42,241][86121] Updated weights for policy 0, policy_version 11420 (0.0007) +[2023-10-09 12:40:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 23429120. Throughput: 0: 1812.9, 1: 1816.5. Samples: 5859826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:40:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:40:44,333][86122] Updated weights for policy 1, policy_version 11460 (0.0009) +[2023-10-09 12:40:44,701][86122] Updated weights for policy 1, policy_version 11470 (0.0009) +[2023-10-09 12:40:45,067][86122] Updated weights for policy 1, policy_version 11480 (0.0008) +[2023-10-09 12:40:45,881][86121] Updated weights for policy 0, policy_version 11430 (0.0008) +[2023-10-09 12:40:46,262][86121] Updated weights for policy 0, policy_version 11440 (0.0010) +[2023-10-09 12:40:46,628][86121] Updated weights for policy 0, policy_version 11450 (0.0007) +[2023-10-09 12:40:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 23494656. Throughput: 0: 1819.0, 1: 1813.2. Samples: 5881086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:40:48,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:40:48,728][86122] Updated weights for policy 1, policy_version 11490 (0.0010) +[2023-10-09 12:40:49,102][86122] Updated weights for policy 1, policy_version 11500 (0.0009) +[2023-10-09 12:40:49,478][86122] Updated weights for policy 1, policy_version 11510 (0.0008) +[2023-10-09 12:40:49,840][86122] Updated weights for policy 1, policy_version 11520 (0.0007) +[2023-10-09 12:40:50,605][86121] Updated weights for policy 0, policy_version 11460 (0.0010) +[2023-10-09 12:40:50,980][86121] Updated weights for policy 0, policy_version 11470 (0.0009) +[2023-10-09 12:40:51,354][86121] Updated weights for policy 0, policy_version 11480 (0.0007) +[2023-10-09 12:40:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23560192. Throughput: 0: 1795.9, 1: 1812.3. Samples: 5903286. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 12:40:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:40:53,487][86122] Updated weights for policy 1, policy_version 11530 (0.0008) +[2023-10-09 12:40:53,850][86122] Updated weights for policy 1, policy_version 11540 (0.0007) +[2023-10-09 12:40:54,224][86122] Updated weights for policy 1, policy_version 11550 (0.0007) +[2023-10-09 12:40:55,131][86121] Updated weights for policy 0, policy_version 11490 (0.0008) +[2023-10-09 12:40:55,497][86121] Updated weights for policy 0, policy_version 11500 (0.0008) +[2023-10-09 12:40:55,866][86121] Updated weights for policy 0, policy_version 11510 (0.0010) +[2023-10-09 12:40:56,224][86121] Updated weights for policy 0, policy_version 11520 (0.0009) +[2023-10-09 12:40:57,823][86122] Updated weights for policy 1, policy_version 11560 (0.0007) +[2023-10-09 12:40:58,195][86122] Updated weights for policy 1, policy_version 11570 (0.0007) +[2023-10-09 12:40:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23625728. Throughput: 0: 1805.9, 1: 1813.3. Samples: 5913644. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 12:40:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:40:58,561][86122] Updated weights for policy 1, policy_version 11580 (0.0009) +[2023-10-09 12:40:59,922][86121] Updated weights for policy 0, policy_version 11530 (0.0007) +[2023-10-09 12:41:00,286][86121] Updated weights for policy 0, policy_version 11540 (0.0008) +[2023-10-09 12:41:00,666][86121] Updated weights for policy 0, policy_version 11550 (0.0009) +[2023-10-09 12:41:02,320][86122] Updated weights for policy 1, policy_version 11590 (0.0009) +[2023-10-09 12:41:02,707][86122] Updated weights for policy 1, policy_version 11600 (0.0008) +[2023-10-09 12:41:03,074][86122] Updated weights for policy 1, policy_version 11610 (0.0009) +[2023-10-09 12:41:03,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23724032. Throughput: 0: 1796.8, 1: 1814.2. Samples: 5936036. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 12:41:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:41:04,452][86121] Updated weights for policy 0, policy_version 11560 (0.0011) +[2023-10-09 12:41:04,819][86121] Updated weights for policy 0, policy_version 11570 (0.0011) +[2023-10-09 12:41:05,192][86121] Updated weights for policy 0, policy_version 11580 (0.0010) +[2023-10-09 12:41:06,631][86122] Updated weights for policy 1, policy_version 11620 (0.0007) +[2023-10-09 12:41:07,006][86122] Updated weights for policy 1, policy_version 11630 (0.0008) +[2023-10-09 12:41:07,368][86122] Updated weights for policy 1, policy_version 11640 (0.0008) +[2023-10-09 12:41:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 23789568. Throughput: 0: 1800.5, 1: 1817.3. Samples: 5957238. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 12:41:08,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 12:41:08,907][86121] Updated weights for policy 0, policy_version 11590 (0.0009) +[2023-10-09 12:41:09,293][86121] Updated weights for policy 0, policy_version 11600 (0.0007) +[2023-10-09 12:41:09,655][86121] Updated weights for policy 0, policy_version 11610 (0.0009) +[2023-10-09 12:41:11,122][86122] Updated weights for policy 1, policy_version 11650 (0.0007) +[2023-10-09 12:41:11,479][86122] Updated weights for policy 1, policy_version 11660 (0.0007) +[2023-10-09 12:41:11,856][86122] Updated weights for policy 1, policy_version 11670 (0.0009) +[2023-10-09 12:41:12,228][86122] Updated weights for policy 1, policy_version 11680 (0.0008) +[2023-10-09 12:41:13,190][86121] Updated weights for policy 0, policy_version 11620 (0.0007) +[2023-10-09 12:41:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 23855104. Throughput: 0: 1796.0, 1: 1820.0. Samples: 5968592. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 12:41:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 12:41:13,554][86121] Updated weights for policy 0, policy_version 11630 (0.0008) +[2023-10-09 12:41:13,932][86121] Updated weights for policy 0, policy_version 11640 (0.0009) +[2023-10-09 12:41:15,913][86122] Updated weights for policy 1, policy_version 11690 (0.0007) +[2023-10-09 12:41:16,279][86122] Updated weights for policy 1, policy_version 11700 (0.0007) +[2023-10-09 12:41:16,650][86122] Updated weights for policy 1, policy_version 11710 (0.0009) +[2023-10-09 12:41:17,630][86121] Updated weights for policy 0, policy_version 11650 (0.0010) +[2023-10-09 12:41:17,995][86121] Updated weights for policy 0, policy_version 11660 (0.0008) +[2023-10-09 12:41:18,363][86121] Updated weights for policy 0, policy_version 11670 (0.0010) +[2023-10-09 12:41:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 23920640. Throughput: 0: 1798.3, 1: 1827.8. Samples: 5990330. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 12:41:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 12:41:18,735][86121] Updated weights for policy 0, policy_version 11680 (0.0007) +[2023-10-09 12:41:20,569][86122] Updated weights for policy 1, policy_version 11720 (0.0011) +[2023-10-09 12:41:20,938][86122] Updated weights for policy 1, policy_version 11730 (0.0010) +[2023-10-09 12:41:21,310][86122] Updated weights for policy 1, policy_version 11740 (0.0007) +[2023-10-09 12:41:22,487][86121] Updated weights for policy 0, policy_version 11690 (0.0007) +[2023-10-09 12:41:22,848][86121] Updated weights for policy 0, policy_version 11700 (0.0008) +[2023-10-09 12:41:23,221][86121] Updated weights for policy 0, policy_version 11710 (0.0008) +[2023-10-09 12:41:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 24018944. Throughput: 0: 1809.6, 1: 1827.6. Samples: 6012172. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 12:41:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 12:41:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000011744_12025856.pth... +[2023-10-09 12:41:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000011712_11993088.pth... +[2023-10-09 12:41:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000010016_10256384.pth +[2023-10-09 12:41:23,448][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000010048_10289152.pth +[2023-10-09 12:41:24,789][86122] Updated weights for policy 1, policy_version 11750 (0.0009) +[2023-10-09 12:41:25,165][86122] Updated weights for policy 1, policy_version 11760 (0.0009) +[2023-10-09 12:41:25,524][86122] Updated weights for policy 1, policy_version 11770 (0.0009) +[2023-10-09 12:41:26,887][86121] Updated weights for policy 0, policy_version 11720 (0.0009) +[2023-10-09 12:41:27,258][86121] Updated weights for policy 0, policy_version 11730 (0.0010) +[2023-10-09 12:41:27,635][86121] Updated weights for policy 0, policy_version 11740 (0.0011) +[2023-10-09 12:41:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 24084480. Throughput: 0: 1798.4, 1: 1827.5. Samples: 6022990. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 12:41:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 12:41:29,104][86122] Updated weights for policy 1, policy_version 11780 (0.0010) +[2023-10-09 12:41:29,466][86122] Updated weights for policy 1, policy_version 11790 (0.0010) +[2023-10-09 12:41:29,845][86122] Updated weights for policy 1, policy_version 11800 (0.0007) +[2023-10-09 12:41:31,376][86121] Updated weights for policy 0, policy_version 11750 (0.0008) +[2023-10-09 12:41:31,752][86121] Updated weights for policy 0, policy_version 11760 (0.0008) +[2023-10-09 12:41:32,122][86121] Updated weights for policy 0, policy_version 11770 (0.0009) +[2023-10-09 12:41:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24150016. Throughput: 0: 1806.0, 1: 1833.2. Samples: 6044848. Policy #0 lag: (min: 9.0, avg: 24.4, max: 41.0) +[2023-10-09 12:41:33,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:41:33,637][86122] Updated weights for policy 1, policy_version 11810 (0.0007) +[2023-10-09 12:41:33,997][86122] Updated weights for policy 1, policy_version 11820 (0.0007) +[2023-10-09 12:41:34,366][86122] Updated weights for policy 1, policy_version 11830 (0.0007) +[2023-10-09 12:41:34,726][86122] Updated weights for policy 1, policy_version 11840 (0.0007) +[2023-10-09 12:41:35,734][86121] Updated weights for policy 0, policy_version 11780 (0.0008) +[2023-10-09 12:41:36,101][86121] Updated weights for policy 0, policy_version 11790 (0.0008) +[2023-10-09 12:41:36,466][86121] Updated weights for policy 0, policy_version 11800 (0.0007) +[2023-10-09 12:41:38,292][86122] Updated weights for policy 1, policy_version 11850 (0.0008) +[2023-10-09 12:41:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24215552. Throughput: 0: 1809.4, 1: 1836.2. Samples: 6067338. Policy #0 lag: (min: 9.0, avg: 24.4, max: 41.0) +[2023-10-09 12:41:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:41:38,665][86122] Updated weights for policy 1, policy_version 11860 (0.0007) +[2023-10-09 12:41:39,035][86122] Updated weights for policy 1, policy_version 11870 (0.0008) +[2023-10-09 12:41:40,240][86121] Updated weights for policy 0, policy_version 11810 (0.0008) +[2023-10-09 12:41:40,606][86121] Updated weights for policy 0, policy_version 11820 (0.0010) +[2023-10-09 12:41:40,970][86121] Updated weights for policy 0, policy_version 11830 (0.0009) +[2023-10-09 12:41:41,345][86121] Updated weights for policy 0, policy_version 11840 (0.0009) +[2023-10-09 12:41:42,784][86122] Updated weights for policy 1, policy_version 11880 (0.0009) +[2023-10-09 12:41:43,143][86122] Updated weights for policy 1, policy_version 11890 (0.0007) +[2023-10-09 12:41:43,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 24281088. Throughput: 0: 1814.2, 1: 1836.7. Samples: 6077934. Policy #0 lag: (min: 9.0, avg: 24.4, max: 41.0) +[2023-10-09 12:41:43,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:41:43,503][86122] Updated weights for policy 1, policy_version 11900 (0.0007) +[2023-10-09 12:41:45,024][86121] Updated weights for policy 0, policy_version 11850 (0.0007) +[2023-10-09 12:41:45,390][86121] Updated weights for policy 0, policy_version 11860 (0.0008) +[2023-10-09 12:41:45,759][86121] Updated weights for policy 0, policy_version 11870 (0.0009) +[2023-10-09 12:41:47,166][86122] Updated weights for policy 1, policy_version 11910 (0.0009) +[2023-10-09 12:41:47,555][86122] Updated weights for policy 1, policy_version 11920 (0.0009) +[2023-10-09 12:41:47,927][86122] Updated weights for policy 1, policy_version 11930 (0.0008) +[2023-10-09 12:41:48,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 24379392. Throughput: 0: 1813.5, 1: 1835.4. Samples: 6100238. Policy #0 lag: (min: 26.0, avg: 32.8, max: 58.0) +[2023-10-09 12:41:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 12:41:49,349][86121] Updated weights for policy 0, policy_version 11880 (0.0009) +[2023-10-09 12:41:49,716][86121] Updated weights for policy 0, policy_version 11890 (0.0009) +[2023-10-09 12:41:50,085][86121] Updated weights for policy 0, policy_version 11900 (0.0007) +[2023-10-09 12:41:51,625][86122] Updated weights for policy 1, policy_version 11940 (0.0008) +[2023-10-09 12:41:51,994][86122] Updated weights for policy 1, policy_version 11950 (0.0008) +[2023-10-09 12:41:52,347][86122] Updated weights for policy 1, policy_version 11960 (0.0010) +[2023-10-09 12:41:53,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24444928. Throughput: 0: 1815.3, 1: 1836.1. Samples: 6121548. Policy #0 lag: (min: 26.0, avg: 32.8, max: 58.0) +[2023-10-09 12:41:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 12:41:53,852][86121] Updated weights for policy 0, policy_version 11910 (0.0009) +[2023-10-09 12:41:54,223][86121] Updated weights for policy 0, policy_version 11920 (0.0008) +[2023-10-09 12:41:54,590][86121] Updated weights for policy 0, policy_version 11930 (0.0010) +[2023-10-09 12:41:56,088][86122] Updated weights for policy 1, policy_version 11970 (0.0009) +[2023-10-09 12:41:56,446][86122] Updated weights for policy 1, policy_version 11980 (0.0009) +[2023-10-09 12:41:56,810][86122] Updated weights for policy 1, policy_version 11990 (0.0009) +[2023-10-09 12:41:57,176][86122] Updated weights for policy 1, policy_version 12000 (0.0007) +[2023-10-09 12:41:58,291][86121] Updated weights for policy 0, policy_version 11940 (0.0007) +[2023-10-09 12:41:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24510464. Throughput: 0: 1819.3, 1: 1828.3. Samples: 6132734. Policy #0 lag: (min: 26.0, avg: 32.8, max: 58.0) +[2023-10-09 12:41:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 12:41:58,674][86121] Updated weights for policy 0, policy_version 11950 (0.0008) +[2023-10-09 12:41:59,038][86121] Updated weights for policy 0, policy_version 11960 (0.0008) +[2023-10-09 12:42:00,821][86122] Updated weights for policy 1, policy_version 12010 (0.0010) +[2023-10-09 12:42:01,182][86122] Updated weights for policy 1, policy_version 12020 (0.0007) +[2023-10-09 12:42:01,547][86122] Updated weights for policy 1, policy_version 12030 (0.0007) +[2023-10-09 12:42:02,720][86121] Updated weights for policy 0, policy_version 11970 (0.0007) +[2023-10-09 12:42:03,089][86121] Updated weights for policy 0, policy_version 11980 (0.0009) +[2023-10-09 12:42:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24576000. Throughput: 0: 1817.4, 1: 1827.6. Samples: 6154354. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 12:42:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:42:03,447][86121] Updated weights for policy 0, policy_version 11990 (0.0008) +[2023-10-09 12:42:03,813][86121] Updated weights for policy 0, policy_version 12000 (0.0007) +[2023-10-09 12:42:05,120][86122] Updated weights for policy 1, policy_version 12040 (0.0008) +[2023-10-09 12:42:05,489][86122] Updated weights for policy 1, policy_version 12050 (0.0009) +[2023-10-09 12:42:05,856][86122] Updated weights for policy 1, policy_version 12060 (0.0008) +[2023-10-09 12:42:07,435][86121] Updated weights for policy 0, policy_version 12010 (0.0008) +[2023-10-09 12:42:07,810][86121] Updated weights for policy 0, policy_version 12020 (0.0007) +[2023-10-09 12:42:08,169][86121] Updated weights for policy 0, policy_version 12030 (0.0007) +[2023-10-09 12:42:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 24674304. Throughput: 0: 1816.9, 1: 1832.7. Samples: 6176408. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 12:42:08,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:42:09,646][86122] Updated weights for policy 1, policy_version 12070 (0.0008) +[2023-10-09 12:42:10,003][86122] Updated weights for policy 1, policy_version 12080 (0.0007) +[2023-10-09 12:42:10,372][86122] Updated weights for policy 1, policy_version 12090 (0.0007) +[2023-10-09 12:42:11,721][86121] Updated weights for policy 0, policy_version 12040 (0.0008) +[2023-10-09 12:42:12,091][86121] Updated weights for policy 0, policy_version 12050 (0.0007) +[2023-10-09 12:42:12,458][86121] Updated weights for policy 0, policy_version 12060 (0.0007) +[2023-10-09 12:42:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 24739840. Throughput: 0: 1827.1, 1: 1828.0. Samples: 6187466. Policy #0 lag: (min: 13.0, avg: 18.5, max: 45.0) +[2023-10-09 12:42:13,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:42:14,168][86122] Updated weights for policy 1, policy_version 12100 (0.0009) +[2023-10-09 12:42:14,534][86122] Updated weights for policy 1, policy_version 12110 (0.0007) +[2023-10-09 12:42:14,900][86122] Updated weights for policy 1, policy_version 12120 (0.0010) +[2023-10-09 12:42:16,097][86121] Updated weights for policy 0, policy_version 12070 (0.0008) +[2023-10-09 12:42:16,474][86121] Updated weights for policy 0, policy_version 12080 (0.0007) +[2023-10-09 12:42:16,836][86121] Updated weights for policy 0, policy_version 12090 (0.0007) +[2023-10-09 12:42:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 24805376. Throughput: 0: 1828.1, 1: 1818.0. Samples: 6208922. Policy #0 lag: (min: 13.0, avg: 18.5, max: 45.0) +[2023-10-09 12:42:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 12:42:18,853][86122] Updated weights for policy 1, policy_version 12130 (0.0009) +[2023-10-09 12:42:19,226][86122] Updated weights for policy 1, policy_version 12140 (0.0009) +[2023-10-09 12:42:19,594][86122] Updated weights for policy 1, policy_version 12150 (0.0008) +[2023-10-09 12:42:19,951][86122] Updated weights for policy 1, policy_version 12160 (0.0008) +[2023-10-09 12:42:20,478][86121] Updated weights for policy 0, policy_version 12100 (0.0008) +[2023-10-09 12:42:20,850][86121] Updated weights for policy 0, policy_version 12110 (0.0007) +[2023-10-09 12:42:21,220][86121] Updated weights for policy 0, policy_version 12120 (0.0007) +[2023-10-09 12:42:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 24870912. Throughput: 0: 1836.3, 1: 1814.5. Samples: 6231624. Policy #0 lag: (min: 13.0, avg: 18.5, max: 45.0) +[2023-10-09 12:42:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:42:23,531][86122] Updated weights for policy 1, policy_version 12170 (0.0009) +[2023-10-09 12:42:23,904][86122] Updated weights for policy 1, policy_version 12180 (0.0008) +[2023-10-09 12:42:24,264][86122] Updated weights for policy 1, policy_version 12190 (0.0008) +[2023-10-09 12:42:24,843][86121] Updated weights for policy 0, policy_version 12130 (0.0009) +[2023-10-09 12:42:25,218][86121] Updated weights for policy 0, policy_version 12140 (0.0007) +[2023-10-09 12:42:25,576][86121] Updated weights for policy 0, policy_version 12150 (0.0009) +[2023-10-09 12:42:25,945][86121] Updated weights for policy 0, policy_version 12160 (0.0009) +[2023-10-09 12:42:27,887][86122] Updated weights for policy 1, policy_version 12200 (0.0008) +[2023-10-09 12:42:28,250][86122] Updated weights for policy 1, policy_version 12210 (0.0009) +[2023-10-09 12:42:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 24936448. Throughput: 0: 1827.4, 1: 1811.3. Samples: 6241674. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 12:42:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:42:28,619][86122] Updated weights for policy 1, policy_version 12220 (0.0009) +[2023-10-09 12:42:29,536][86121] Updated weights for policy 0, policy_version 12170 (0.0007) +[2023-10-09 12:42:29,910][86121] Updated weights for policy 0, policy_version 12180 (0.0008) +[2023-10-09 12:42:30,271][86121] Updated weights for policy 0, policy_version 12190 (0.0008) +[2023-10-09 12:42:32,355][86122] Updated weights for policy 1, policy_version 12230 (0.0008) +[2023-10-09 12:42:32,736][86122] Updated weights for policy 1, policy_version 12240 (0.0008) +[2023-10-09 12:42:33,105][86122] Updated weights for policy 1, policy_version 12250 (0.0009) +[2023-10-09 12:42:33,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 25034752. Throughput: 0: 1837.6, 1: 1808.3. Samples: 6264306. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 12:42:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:42:34,127][86121] Updated weights for policy 0, policy_version 12200 (0.0008) +[2023-10-09 12:42:34,497][86121] Updated weights for policy 0, policy_version 12210 (0.0011) +[2023-10-09 12:42:34,862][86121] Updated weights for policy 0, policy_version 12220 (0.0008) +[2023-10-09 12:42:36,786][86122] Updated weights for policy 1, policy_version 12260 (0.0010) +[2023-10-09 12:42:37,149][86122] Updated weights for policy 1, policy_version 12270 (0.0011) +[2023-10-09 12:42:37,516][86122] Updated weights for policy 1, policy_version 12280 (0.0009) +[2023-10-09 12:42:38,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 25100288. Throughput: 0: 1838.0, 1: 1813.3. Samples: 6285858. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 12:42:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:42:38,702][86121] Updated weights for policy 0, policy_version 12230 (0.0008) +[2023-10-09 12:42:39,093][86121] Updated weights for policy 0, policy_version 12240 (0.0009) +[2023-10-09 12:42:39,452][86121] Updated weights for policy 0, policy_version 12250 (0.0010) +[2023-10-09 12:42:41,274][86122] Updated weights for policy 1, policy_version 12290 (0.0008) +[2023-10-09 12:42:41,641][86122] Updated weights for policy 1, policy_version 12300 (0.0010) +[2023-10-09 12:42:42,010][86122] Updated weights for policy 1, policy_version 12310 (0.0010) +[2023-10-09 12:42:42,375][86122] Updated weights for policy 1, policy_version 12320 (0.0008) +[2023-10-09 12:42:43,212][86121] Updated weights for policy 0, policy_version 12260 (0.0009) +[2023-10-09 12:42:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 25165824. Throughput: 0: 1833.9, 1: 1812.6. Samples: 6296828. Policy #0 lag: (min: 13.0, avg: 38.1, max: 40.0) +[2023-10-09 12:42:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:42:43,593][86121] Updated weights for policy 0, policy_version 12270 (0.0008) +[2023-10-09 12:42:43,958][86121] Updated weights for policy 0, policy_version 12280 (0.0008) +[2023-10-09 12:42:45,939][86122] Updated weights for policy 1, policy_version 12330 (0.0007) +[2023-10-09 12:42:46,296][86122] Updated weights for policy 1, policy_version 12340 (0.0009) +[2023-10-09 12:42:46,669][86122] Updated weights for policy 1, policy_version 12350 (0.0010) +[2023-10-09 12:42:47,728][86121] Updated weights for policy 0, policy_version 12290 (0.0008) +[2023-10-09 12:42:48,081][86121] Updated weights for policy 0, policy_version 12300 (0.0009) +[2023-10-09 12:42:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 25231360. Throughput: 0: 1830.6, 1: 1808.6. Samples: 6318120. Policy #0 lag: (min: 13.0, avg: 38.1, max: 40.0) +[2023-10-09 12:42:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:42:48,449][86121] Updated weights for policy 0, policy_version 12310 (0.0008) +[2023-10-09 12:42:48,813][86121] Updated weights for policy 0, policy_version 12320 (0.0007) +[2023-10-09 12:42:50,423][86122] Updated weights for policy 1, policy_version 12360 (0.0009) +[2023-10-09 12:42:50,789][86122] Updated weights for policy 1, policy_version 12370 (0.0008) +[2023-10-09 12:42:51,161][86122] Updated weights for policy 1, policy_version 12380 (0.0008) +[2023-10-09 12:42:52,367][86121] Updated weights for policy 0, policy_version 12330 (0.0008) +[2023-10-09 12:42:52,739][86121] Updated weights for policy 0, policy_version 12340 (0.0008) +[2023-10-09 12:42:53,102][86121] Updated weights for policy 0, policy_version 12350 (0.0007) +[2023-10-09 12:42:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25329664. Throughput: 0: 1826.1, 1: 1804.9. Samples: 6339802. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) +[2023-10-09 12:42:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 12:42:54,870][86122] Updated weights for policy 1, policy_version 12390 (0.0007) +[2023-10-09 12:42:55,244][86122] Updated weights for policy 1, policy_version 12400 (0.0008) +[2023-10-09 12:42:55,608][86122] Updated weights for policy 1, policy_version 12410 (0.0010) +[2023-10-09 12:42:56,800][86121] Updated weights for policy 0, policy_version 12360 (0.0008) +[2023-10-09 12:42:57,170][86121] Updated weights for policy 0, policy_version 12370 (0.0009) +[2023-10-09 12:42:57,545][86121] Updated weights for policy 0, policy_version 12380 (0.0007) +[2023-10-09 12:42:58,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25395200. Throughput: 0: 1820.6, 1: 1810.1. Samples: 6350848. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) +[2023-10-09 12:42:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 12:42:59,252][86122] Updated weights for policy 1, policy_version 12420 (0.0008) +[2023-10-09 12:42:59,623][86122] Updated weights for policy 1, policy_version 12430 (0.0007) +[2023-10-09 12:42:59,986][86122] Updated weights for policy 1, policy_version 12440 (0.0010) +[2023-10-09 12:43:01,181][86121] Updated weights for policy 0, policy_version 12390 (0.0009) +[2023-10-09 12:43:01,546][86121] Updated weights for policy 0, policy_version 12400 (0.0007) +[2023-10-09 12:43:01,919][86121] Updated weights for policy 0, policy_version 12410 (0.0008) +[2023-10-09 12:43:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25460736. Throughput: 0: 1822.1, 1: 1821.9. Samples: 6372902. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) +[2023-10-09 12:43:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:43:03,605][86122] Updated weights for policy 1, policy_version 12450 (0.0009) +[2023-10-09 12:43:03,972][86122] Updated weights for policy 1, policy_version 12460 (0.0009) +[2023-10-09 12:43:04,335][86122] Updated weights for policy 1, policy_version 12470 (0.0010) +[2023-10-09 12:43:04,701][86122] Updated weights for policy 1, policy_version 12480 (0.0011) +[2023-10-09 12:43:05,463][86121] Updated weights for policy 0, policy_version 12420 (0.0009) +[2023-10-09 12:43:05,835][86121] Updated weights for policy 0, policy_version 12430 (0.0008) +[2023-10-09 12:43:06,207][86121] Updated weights for policy 0, policy_version 12440 (0.0008) +[2023-10-09 12:43:08,327][86122] Updated weights for policy 1, policy_version 12490 (0.0008) +[2023-10-09 12:43:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25526272. Throughput: 0: 1818.0, 1: 1827.5. Samples: 6395676. Policy #0 lag: (min: 22.0, avg: 38.3, max: 40.0) +[2023-10-09 12:43:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:43:08,706][86122] Updated weights for policy 1, policy_version 12500 (0.0008) +[2023-10-09 12:43:09,064][86122] Updated weights for policy 1, policy_version 12510 (0.0009) +[2023-10-09 12:43:09,908][86121] Updated weights for policy 0, policy_version 12450 (0.0008) +[2023-10-09 12:43:10,272][86121] Updated weights for policy 0, policy_version 12460 (0.0008) +[2023-10-09 12:43:10,641][86121] Updated weights for policy 0, policy_version 12470 (0.0007) +[2023-10-09 12:43:11,008][86121] Updated weights for policy 0, policy_version 12480 (0.0011) +[2023-10-09 12:43:12,838][86122] Updated weights for policy 1, policy_version 12520 (0.0008) +[2023-10-09 12:43:13,211][86122] Updated weights for policy 1, policy_version 12530 (0.0009) +[2023-10-09 12:43:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 25591808. Throughput: 0: 1818.7, 1: 1826.9. Samples: 6405726. Policy #0 lag: (min: 22.0, avg: 38.3, max: 40.0) +[2023-10-09 12:43:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 12:43:13,581][86122] Updated weights for policy 1, policy_version 12540 (0.0008) +[2023-10-09 12:43:14,863][86121] Updated weights for policy 0, policy_version 12490 (0.0008) +[2023-10-09 12:43:15,227][86121] Updated weights for policy 0, policy_version 12500 (0.0009) +[2023-10-09 12:43:15,595][86121] Updated weights for policy 0, policy_version 12510 (0.0011) +[2023-10-09 12:43:17,303][86122] Updated weights for policy 1, policy_version 12550 (0.0008) +[2023-10-09 12:43:17,683][86122] Updated weights for policy 1, policy_version 12560 (0.0007) +[2023-10-09 12:43:18,044][86122] Updated weights for policy 1, policy_version 12570 (0.0007) +[2023-10-09 12:43:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 25690112. Throughput: 0: 1814.3, 1: 1829.6. Samples: 6428282. Policy #0 lag: (min: 22.0, avg: 38.3, max: 40.0) +[2023-10-09 12:43:18,399][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 12:43:19,153][86121] Updated weights for policy 0, policy_version 12520 (0.0009) +[2023-10-09 12:43:19,528][86121] Updated weights for policy 0, policy_version 12530 (0.0007) +[2023-10-09 12:43:19,888][86121] Updated weights for policy 0, policy_version 12540 (0.0008) +[2023-10-09 12:43:21,710][86122] Updated weights for policy 1, policy_version 12580 (0.0008) +[2023-10-09 12:43:22,071][86122] Updated weights for policy 1, policy_version 12590 (0.0008) +[2023-10-09 12:43:22,438][86122] Updated weights for policy 1, policy_version 12600 (0.0007) +[2023-10-09 12:43:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25755648. Throughput: 0: 1815.9, 1: 1823.6. Samples: 6449634. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) +[2023-10-09 12:43:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 12:43:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000012608_12910592.pth... +[2023-10-09 12:43:23,435][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000010912_11173888.pth +[2023-10-09 12:43:23,583][86121] Updated weights for policy 0, policy_version 12550 (0.0008) +[2023-10-09 12:43:23,955][86121] Updated weights for policy 0, policy_version 12560 (0.0009) +[2023-10-09 12:43:24,323][86121] Updated weights for policy 0, policy_version 12570 (0.0007) +[2023-10-09 12:43:24,544][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000012576_12877824.pth... +[2023-10-09 12:43:24,583][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000010848_11108352.pth +[2023-10-09 12:43:26,193][86122] Updated weights for policy 1, policy_version 12610 (0.0007) +[2023-10-09 12:43:26,562][86122] Updated weights for policy 1, policy_version 12620 (0.0008) +[2023-10-09 12:43:26,937][86122] Updated weights for policy 1, policy_version 12630 (0.0011) +[2023-10-09 12:43:27,298][86122] Updated weights for policy 1, policy_version 12640 (0.0008) +[2023-10-09 12:43:27,897][86121] Updated weights for policy 0, policy_version 12580 (0.0007) +[2023-10-09 12:43:28,269][86121] Updated weights for policy 0, policy_version 12590 (0.0009) +[2023-10-09 12:43:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 25821184. Throughput: 0: 1821.7, 1: 1828.0. Samples: 6461066. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) +[2023-10-09 12:43:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:43:28,626][86121] Updated weights for policy 0, policy_version 12600 (0.0011) +[2023-10-09 12:43:30,894][86122] Updated weights for policy 1, policy_version 12650 (0.0011) +[2023-10-09 12:43:31,261][86122] Updated weights for policy 1, policy_version 12660 (0.0009) +[2023-10-09 12:43:31,635][86122] Updated weights for policy 1, policy_version 12670 (0.0011) +[2023-10-09 12:43:32,527][86121] Updated weights for policy 0, policy_version 12610 (0.0009) +[2023-10-09 12:43:32,896][86121] Updated weights for policy 0, policy_version 12620 (0.0009) +[2023-10-09 12:43:33,279][86121] Updated weights for policy 0, policy_version 12630 (0.0009) +[2023-10-09 12:43:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 25886720. Throughput: 0: 1821.5, 1: 1827.3. Samples: 6482314. Policy #0 lag: (min: 17.0, avg: 31.6, max: 49.0) +[2023-10-09 12:43:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:43:33,650][86121] Updated weights for policy 0, policy_version 12640 (0.0009) +[2023-10-09 12:43:35,332][86122] Updated weights for policy 1, policy_version 12680 (0.0009) +[2023-10-09 12:43:35,696][86122] Updated weights for policy 1, policy_version 12690 (0.0011) +[2023-10-09 12:43:36,055][86122] Updated weights for policy 1, policy_version 12700 (0.0011) +[2023-10-09 12:43:37,356][86121] Updated weights for policy 0, policy_version 12650 (0.0008) +[2023-10-09 12:43:37,737][86121] Updated weights for policy 0, policy_version 12660 (0.0009) +[2023-10-09 12:43:38,111][86121] Updated weights for policy 0, policy_version 12670 (0.0009) +[2023-10-09 12:43:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 25985024. Throughput: 0: 1823.4, 1: 1829.3. Samples: 6504174. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) +[2023-10-09 12:43:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:43:39,549][86122] Updated weights for policy 1, policy_version 12710 (0.0008) +[2023-10-09 12:43:39,917][86122] Updated weights for policy 1, policy_version 12720 (0.0008) +[2023-10-09 12:43:40,287][86122] Updated weights for policy 1, policy_version 12730 (0.0009) +[2023-10-09 12:43:41,755][86121] Updated weights for policy 0, policy_version 12680 (0.0008) +[2023-10-09 12:43:42,115][86121] Updated weights for policy 0, policy_version 12690 (0.0007) +[2023-10-09 12:43:42,492][86121] Updated weights for policy 0, policy_version 12700 (0.0007) +[2023-10-09 12:43:43,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26050560. Throughput: 0: 1823.6, 1: 1829.3. Samples: 6515228. Policy #0 lag: (min: 19.0, avg: 19.0, max: 19.0) +[2023-10-09 12:43:43,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 12:43:43,974][86122] Updated weights for policy 1, policy_version 12740 (0.0010) +[2023-10-09 12:43:44,325][86122] Updated weights for policy 1, policy_version 12750 (0.0008) +[2023-10-09 12:43:44,694][86122] Updated weights for policy 1, policy_version 12760 (0.0010) +[2023-10-09 12:43:46,158][86121] Updated weights for policy 0, policy_version 12710 (0.0009) +[2023-10-09 12:43:46,529][86121] Updated weights for policy 0, policy_version 12720 (0.0009) +[2023-10-09 12:43:46,892][86121] Updated weights for policy 0, policy_version 12730 (0.0008) +[2023-10-09 12:43:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26116096. Throughput: 0: 1817.0, 1: 1827.0. Samples: 6536884. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 12:43:48,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 12:43:48,400][86122] Updated weights for policy 1, policy_version 12770 (0.0011) +[2023-10-09 12:43:48,772][86122] Updated weights for policy 1, policy_version 12780 (0.0008) +[2023-10-09 12:43:49,140][86122] Updated weights for policy 1, policy_version 12790 (0.0009) +[2023-10-09 12:43:49,492][86122] Updated weights for policy 1, policy_version 12800 (0.0008) +[2023-10-09 12:43:50,666][86121] Updated weights for policy 0, policy_version 12740 (0.0010) +[2023-10-09 12:43:51,033][86121] Updated weights for policy 0, policy_version 12750 (0.0009) +[2023-10-09 12:43:51,407][86121] Updated weights for policy 0, policy_version 12760 (0.0008) +[2023-10-09 12:43:53,249][86122] Updated weights for policy 1, policy_version 12810 (0.0008) +[2023-10-09 12:43:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26181632. Throughput: 0: 1812.0, 1: 1822.5. Samples: 6559224. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 12:43:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:43:53,611][86122] Updated weights for policy 1, policy_version 12820 (0.0007) +[2023-10-09 12:43:53,981][86122] Updated weights for policy 1, policy_version 12830 (0.0008) +[2023-10-09 12:43:55,195][86121] Updated weights for policy 0, policy_version 12770 (0.0010) +[2023-10-09 12:43:55,569][86121] Updated weights for policy 0, policy_version 12780 (0.0009) +[2023-10-09 12:43:55,934][86121] Updated weights for policy 0, policy_version 12790 (0.0010) +[2023-10-09 12:43:56,300][86121] Updated weights for policy 0, policy_version 12800 (0.0009) +[2023-10-09 12:43:57,671][86122] Updated weights for policy 1, policy_version 12840 (0.0009) +[2023-10-09 12:43:58,041][86122] Updated weights for policy 1, policy_version 12850 (0.0008) +[2023-10-09 12:43:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 26247168. Throughput: 0: 1815.6, 1: 1822.8. Samples: 6569454. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 12:43:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:43:58,401][86122] Updated weights for policy 1, policy_version 12860 (0.0007) +[2023-10-09 12:43:59,983][86121] Updated weights for policy 0, policy_version 12810 (0.0007) +[2023-10-09 12:44:00,351][86121] Updated weights for policy 0, policy_version 12820 (0.0008) +[2023-10-09 12:44:00,718][86121] Updated weights for policy 0, policy_version 12830 (0.0007) +[2023-10-09 12:44:02,115][86122] Updated weights for policy 1, policy_version 12870 (0.0008) +[2023-10-09 12:44:02,505][86122] Updated weights for policy 1, policy_version 12880 (0.0008) +[2023-10-09 12:44:02,872][86122] Updated weights for policy 1, policy_version 12890 (0.0009) +[2023-10-09 12:44:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 26345472. Throughput: 0: 1813.1, 1: 1826.6. Samples: 6592068. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 12:44:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:44:04,217][86121] Updated weights for policy 0, policy_version 12840 (0.0008) +[2023-10-09 12:44:04,590][86121] Updated weights for policy 0, policy_version 12850 (0.0008) +[2023-10-09 12:44:04,954][86121] Updated weights for policy 0, policy_version 12860 (0.0009) +[2023-10-09 12:44:06,570][86122] Updated weights for policy 1, policy_version 12900 (0.0010) +[2023-10-09 12:44:06,933][86122] Updated weights for policy 1, policy_version 12910 (0.0010) +[2023-10-09 12:44:07,299][86122] Updated weights for policy 1, policy_version 12920 (0.0011) +[2023-10-09 12:44:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 26411008. Throughput: 0: 1813.9, 1: 1822.0. Samples: 6613252. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 12:44:08,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 12:44:08,623][86121] Updated weights for policy 0, policy_version 12870 (0.0010) +[2023-10-09 12:44:09,001][86121] Updated weights for policy 0, policy_version 12880 (0.0010) +[2023-10-09 12:44:09,364][86121] Updated weights for policy 0, policy_version 12890 (0.0008) +[2023-10-09 12:44:10,979][86122] Updated weights for policy 1, policy_version 12930 (0.0008) +[2023-10-09 12:44:11,341][86122] Updated weights for policy 1, policy_version 12940 (0.0010) +[2023-10-09 12:44:11,705][86122] Updated weights for policy 1, policy_version 12950 (0.0010) +[2023-10-09 12:44:12,072][86122] Updated weights for policy 1, policy_version 12960 (0.0010) +[2023-10-09 12:44:13,156][86121] Updated weights for policy 0, policy_version 12900 (0.0008) +[2023-10-09 12:44:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26476544. Throughput: 0: 1808.5, 1: 1819.3. Samples: 6624314. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 12:44:13,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 12:44:13,534][86121] Updated weights for policy 0, policy_version 12910 (0.0008) +[2023-10-09 12:44:13,905][86121] Updated weights for policy 0, policy_version 12920 (0.0009) +[2023-10-09 12:44:15,729][86122] Updated weights for policy 1, policy_version 12970 (0.0009) +[2023-10-09 12:44:16,100][86122] Updated weights for policy 1, policy_version 12980 (0.0008) +[2023-10-09 12:44:16,480][86122] Updated weights for policy 1, policy_version 12990 (0.0010) +[2023-10-09 12:44:17,595][86121] Updated weights for policy 0, policy_version 12930 (0.0009) +[2023-10-09 12:44:17,953][86121] Updated weights for policy 0, policy_version 12940 (0.0007) +[2023-10-09 12:44:18,327][86121] Updated weights for policy 0, policy_version 12950 (0.0009) +[2023-10-09 12:44:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 26542080. Throughput: 0: 1808.3, 1: 1821.7. Samples: 6645666. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) +[2023-10-09 12:44:18,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 12:44:18,696][86121] Updated weights for policy 0, policy_version 12960 (0.0007) +[2023-10-09 12:44:20,334][86122] Updated weights for policy 1, policy_version 13000 (0.0009) +[2023-10-09 12:44:20,697][86122] Updated weights for policy 1, policy_version 13010 (0.0010) +[2023-10-09 12:44:21,057][86122] Updated weights for policy 1, policy_version 13020 (0.0009) +[2023-10-09 12:44:22,577][86121] Updated weights for policy 0, policy_version 12970 (0.0007) +[2023-10-09 12:44:22,938][86121] Updated weights for policy 0, policy_version 12980 (0.0007) +[2023-10-09 12:44:23,309][86121] Updated weights for policy 0, policy_version 12990 (0.0007) +[2023-10-09 12:44:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26640384. Throughput: 0: 1809.6, 1: 1816.4. Samples: 6667342. Policy #0 lag: (min: 31.0, avg: 32.5, max: 58.0) +[2023-10-09 12:44:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 12:44:24,800][86122] Updated weights for policy 1, policy_version 13030 (0.0011) +[2023-10-09 12:44:25,175][86122] Updated weights for policy 1, policy_version 13040 (0.0008) +[2023-10-09 12:44:25,543][86122] Updated weights for policy 1, policy_version 13050 (0.0008) +[2023-10-09 12:44:27,000][86121] Updated weights for policy 0, policy_version 13000 (0.0010) +[2023-10-09 12:44:27,378][86121] Updated weights for policy 0, policy_version 13010 (0.0009) +[2023-10-09 12:44:27,739][86121] Updated weights for policy 0, policy_version 13020 (0.0009) +[2023-10-09 12:44:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 26705920. Throughput: 0: 1803.5, 1: 1813.3. Samples: 6677982. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) +[2023-10-09 12:44:28,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 12:44:29,244][86122] Updated weights for policy 1, policy_version 13060 (0.0010) +[2023-10-09 12:44:29,610][86122] Updated weights for policy 1, policy_version 13070 (0.0007) +[2023-10-09 12:44:29,981][86122] Updated weights for policy 1, policy_version 13080 (0.0007) +[2023-10-09 12:44:31,695][86121] Updated weights for policy 0, policy_version 13030 (0.0008) +[2023-10-09 12:44:32,065][86121] Updated weights for policy 0, policy_version 13040 (0.0009) +[2023-10-09 12:44:32,436][86121] Updated weights for policy 0, policy_version 13050 (0.0007) +[2023-10-09 12:44:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 26771456. Throughput: 0: 1810.1, 1: 1809.6. Samples: 6699770. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) +[2023-10-09 12:44:33,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 12:44:33,915][86122] Updated weights for policy 1, policy_version 13090 (0.0008) +[2023-10-09 12:44:34,276][86122] Updated weights for policy 1, policy_version 13100 (0.0011) +[2023-10-09 12:44:34,639][86122] Updated weights for policy 1, policy_version 13110 (0.0011) +[2023-10-09 12:44:35,004][86122] Updated weights for policy 1, policy_version 13120 (0.0010) +[2023-10-09 12:44:36,032][86121] Updated weights for policy 0, policy_version 13060 (0.0009) +[2023-10-09 12:44:36,397][86121] Updated weights for policy 0, policy_version 13070 (0.0009) +[2023-10-09 12:44:36,766][86121] Updated weights for policy 0, policy_version 13080 (0.0009) +[2023-10-09 12:44:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 26836992. Throughput: 0: 1799.7, 1: 1808.9. Samples: 6721612. Policy #0 lag: (min: 31.0, avg: 31.4, max: 44.0) +[2023-10-09 12:44:38,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 12:44:38,596][86122] Updated weights for policy 1, policy_version 13130 (0.0010) +[2023-10-09 12:44:38,959][86122] Updated weights for policy 1, policy_version 13140 (0.0007) +[2023-10-09 12:44:39,326][86122] Updated weights for policy 1, policy_version 13150 (0.0007) +[2023-10-09 12:44:40,461][86121] Updated weights for policy 0, policy_version 13090 (0.0010) +[2023-10-09 12:44:40,828][86121] Updated weights for policy 0, policy_version 13100 (0.0009) +[2023-10-09 12:44:41,200][86121] Updated weights for policy 0, policy_version 13110 (0.0009) +[2023-10-09 12:44:41,575][86121] Updated weights for policy 0, policy_version 13120 (0.0008) +[2023-10-09 12:44:42,977][86122] Updated weights for policy 1, policy_version 13160 (0.0007) +[2023-10-09 12:44:43,344][86122] Updated weights for policy 1, policy_version 13170 (0.0007) +[2023-10-09 12:44:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 26902528. Throughput: 0: 1808.0, 1: 1812.9. Samples: 6732390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:44:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:44:43,711][86122] Updated weights for policy 1, policy_version 13180 (0.0008) +[2023-10-09 12:44:45,230][86121] Updated weights for policy 0, policy_version 13130 (0.0007) +[2023-10-09 12:44:45,596][86121] Updated weights for policy 0, policy_version 13140 (0.0008) +[2023-10-09 12:44:45,963][86121] Updated weights for policy 0, policy_version 13150 (0.0008) +[2023-10-09 12:44:47,377][86122] Updated weights for policy 1, policy_version 13190 (0.0009) +[2023-10-09 12:44:47,749][86122] Updated weights for policy 1, policy_version 13200 (0.0008) +[2023-10-09 12:44:48,120][86122] Updated weights for policy 1, policy_version 13210 (0.0011) +[2023-10-09 12:44:48,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27000832. Throughput: 0: 1801.1, 1: 1808.0. Samples: 6754480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:44:48,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 12:44:49,651][86121] Updated weights for policy 0, policy_version 13160 (0.0007) +[2023-10-09 12:44:50,021][86121] Updated weights for policy 0, policy_version 13170 (0.0009) +[2023-10-09 12:44:50,389][86121] Updated weights for policy 0, policy_version 13180 (0.0009) +[2023-10-09 12:44:51,867][86122] Updated weights for policy 1, policy_version 13220 (0.0010) +[2023-10-09 12:44:52,235][86122] Updated weights for policy 1, policy_version 13230 (0.0008) +[2023-10-09 12:44:52,600][86122] Updated weights for policy 1, policy_version 13240 (0.0008) +[2023-10-09 12:44:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27066368. Throughput: 0: 1795.4, 1: 1815.8. Samples: 6775756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:44:53,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 12:44:54,275][86121] Updated weights for policy 0, policy_version 13190 (0.0008) +[2023-10-09 12:44:54,656][86121] Updated weights for policy 0, policy_version 13200 (0.0007) +[2023-10-09 12:44:55,032][86121] Updated weights for policy 0, policy_version 13210 (0.0010) +[2023-10-09 12:44:56,353][86122] Updated weights for policy 1, policy_version 13250 (0.0008) +[2023-10-09 12:44:56,727][86122] Updated weights for policy 1, policy_version 13260 (0.0010) +[2023-10-09 12:44:57,080][86122] Updated weights for policy 1, policy_version 13270 (0.0011) +[2023-10-09 12:44:57,443][86122] Updated weights for policy 1, policy_version 13280 (0.0008) +[2023-10-09 12:44:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27131904. Throughput: 0: 1793.4, 1: 1818.8. Samples: 6786864. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 12:44:58,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 12:44:58,761][86121] Updated weights for policy 0, policy_version 13220 (0.0010) +[2023-10-09 12:44:59,120][86121] Updated weights for policy 0, policy_version 13230 (0.0007) +[2023-10-09 12:44:59,488][86121] Updated weights for policy 0, policy_version 13240 (0.0007) +[2023-10-09 12:45:01,078][86122] Updated weights for policy 1, policy_version 13290 (0.0011) +[2023-10-09 12:45:01,441][86122] Updated weights for policy 1, policy_version 13300 (0.0010) +[2023-10-09 12:45:01,806][86122] Updated weights for policy 1, policy_version 13310 (0.0009) +[2023-10-09 12:45:03,255][86121] Updated weights for policy 0, policy_version 13250 (0.0008) +[2023-10-09 12:45:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 27197440. Throughput: 0: 1798.3, 1: 1818.3. Samples: 6808414. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 12:45:03,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 12:45:03,627][86121] Updated weights for policy 0, policy_version 13260 (0.0010) +[2023-10-09 12:45:03,996][86121] Updated weights for policy 0, policy_version 13270 (0.0010) +[2023-10-09 12:45:04,361][86121] Updated weights for policy 0, policy_version 13280 (0.0007) +[2023-10-09 12:45:05,412][86122] Updated weights for policy 1, policy_version 13320 (0.0009) +[2023-10-09 12:45:05,779][86122] Updated weights for policy 1, policy_version 13330 (0.0008) +[2023-10-09 12:45:06,136][86122] Updated weights for policy 1, policy_version 13340 (0.0008) +[2023-10-09 12:45:08,093][86121] Updated weights for policy 0, policy_version 13290 (0.0009) +[2023-10-09 12:45:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 27262976. Throughput: 0: 1812.1, 1: 1819.9. Samples: 6830782. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 12:45:08,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:45:08,465][86121] Updated weights for policy 0, policy_version 13300 (0.0010) +[2023-10-09 12:45:08,841][86121] Updated weights for policy 0, policy_version 13310 (0.0009) +[2023-10-09 12:45:09,693][86122] Updated weights for policy 1, policy_version 13350 (0.0008) +[2023-10-09 12:45:10,049][86122] Updated weights for policy 1, policy_version 13360 (0.0008) +[2023-10-09 12:45:10,416][86122] Updated weights for policy 1, policy_version 13370 (0.0010) +[2023-10-09 12:45:12,551][86121] Updated weights for policy 0, policy_version 13320 (0.0007) +[2023-10-09 12:45:12,913][86121] Updated weights for policy 0, policy_version 13330 (0.0008) +[2023-10-09 12:45:13,289][86121] Updated weights for policy 0, policy_version 13340 (0.0009) +[2023-10-09 12:45:13,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 27328512. Throughput: 0: 1797.8, 1: 1822.5. Samples: 6840898. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) +[2023-10-09 12:45:13,399][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 12:45:14,039][86122] Updated weights for policy 1, policy_version 13380 (0.0011) +[2023-10-09 12:45:14,408][86122] Updated weights for policy 1, policy_version 13390 (0.0010) +[2023-10-09 12:45:14,767][86122] Updated weights for policy 1, policy_version 13400 (0.0008) +[2023-10-09 12:45:17,135][86121] Updated weights for policy 0, policy_version 13350 (0.0010) +[2023-10-09 12:45:17,507][86121] Updated weights for policy 0, policy_version 13360 (0.0010) +[2023-10-09 12:45:17,871][86121] Updated weights for policy 0, policy_version 13370 (0.0011) +[2023-10-09 12:45:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27426816. Throughput: 0: 1819.4, 1: 1825.3. Samples: 6863784. Policy #0 lag: (min: 31.0, avg: 38.5, max: 63.0) +[2023-10-09 12:45:18,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 12:45:18,530][86122] Updated weights for policy 1, policy_version 13410 (0.0010) +[2023-10-09 12:45:18,893][86122] Updated weights for policy 1, policy_version 13420 (0.0008) +[2023-10-09 12:45:19,267][86122] Updated weights for policy 1, policy_version 13430 (0.0008) +[2023-10-09 12:45:19,632][86122] Updated weights for policy 1, policy_version 13440 (0.0010) +[2023-10-09 12:45:21,471][86121] Updated weights for policy 0, policy_version 13380 (0.0008) +[2023-10-09 12:45:21,843][86121] Updated weights for policy 0, policy_version 13390 (0.0009) +[2023-10-09 12:45:22,207][86121] Updated weights for policy 0, policy_version 13400 (0.0007) +[2023-10-09 12:45:23,248][86122] Updated weights for policy 1, policy_version 13450 (0.0010) +[2023-10-09 12:45:23,398][85186] Fps is (10 sec: 16384.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 27492352. Throughput: 0: 1806.1, 1: 1828.1. Samples: 6885154. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 12:45:23,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 12:45:23,413][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000013408_13729792.pth... +[2023-10-09 12:45:23,452][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000011712_11993088.pth +[2023-10-09 12:45:23,609][86122] Updated weights for policy 1, policy_version 13460 (0.0009) +[2023-10-09 12:45:23,976][86122] Updated weights for policy 1, policy_version 13470 (0.0009) +[2023-10-09 12:45:24,042][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000013472_13795328.pth... +[2023-10-09 12:45:24,077][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000011744_12025856.pth +[2023-10-09 12:45:25,888][86121] Updated weights for policy 0, policy_version 13410 (0.0007) +[2023-10-09 12:45:26,257][86121] Updated weights for policy 0, policy_version 13420 (0.0009) +[2023-10-09 12:45:26,619][86121] Updated weights for policy 0, policy_version 13430 (0.0011) +[2023-10-09 12:45:26,982][86121] Updated weights for policy 0, policy_version 13440 (0.0011) +[2023-10-09 12:45:27,819][86122] Updated weights for policy 1, policy_version 13480 (0.0008) +[2023-10-09 12:45:28,176][86122] Updated weights for policy 1, policy_version 13490 (0.0008) +[2023-10-09 12:45:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27557888. Throughput: 0: 1816.4, 1: 1821.5. Samples: 6896098. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 12:45:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 12:45:28,545][86122] Updated weights for policy 1, policy_version 13500 (0.0008) +[2023-10-09 12:45:30,817][86121] Updated weights for policy 0, policy_version 13450 (0.0009) +[2023-10-09 12:45:31,180][86121] Updated weights for policy 0, policy_version 13460 (0.0009) +[2023-10-09 12:45:31,549][86121] Updated weights for policy 0, policy_version 13470 (0.0009) +[2023-10-09 12:45:32,169][86122] Updated weights for policy 1, policy_version 13510 (0.0009) +[2023-10-09 12:45:32,560][86122] Updated weights for policy 1, policy_version 13520 (0.0008) +[2023-10-09 12:45:32,932][86122] Updated weights for policy 1, policy_version 13530 (0.0008) +[2023-10-09 12:45:33,397][85186] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 27656192. Throughput: 0: 1798.1, 1: 1825.3. Samples: 6917534. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 12:45:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 12:45:35,261][86121] Updated weights for policy 0, policy_version 13480 (0.0008) +[2023-10-09 12:45:35,626][86121] Updated weights for policy 0, policy_version 13490 (0.0011) +[2023-10-09 12:45:35,989][86121] Updated weights for policy 0, policy_version 13500 (0.0007) +[2023-10-09 12:45:36,539][86122] Updated weights for policy 1, policy_version 13540 (0.0008) +[2023-10-09 12:45:36,902][86122] Updated weights for policy 1, policy_version 13550 (0.0011) +[2023-10-09 12:45:37,264][86122] Updated weights for policy 1, policy_version 13560 (0.0011) +[2023-10-09 12:45:38,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27721728. Throughput: 0: 1804.3, 1: 1823.3. Samples: 6939002. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:45:38,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:45:39,705][86121] Updated weights for policy 0, policy_version 13510 (0.0008) +[2023-10-09 12:45:40,085][86121] Updated weights for policy 0, policy_version 13520 (0.0008) +[2023-10-09 12:45:40,444][86121] Updated weights for policy 0, policy_version 13530 (0.0008) +[2023-10-09 12:45:40,846][86122] Updated weights for policy 1, policy_version 13570 (0.0009) +[2023-10-09 12:45:41,209][86122] Updated weights for policy 1, policy_version 13580 (0.0007) +[2023-10-09 12:45:41,578][86122] Updated weights for policy 1, policy_version 13590 (0.0010) +[2023-10-09 12:45:41,939][86122] Updated weights for policy 1, policy_version 13600 (0.0009) +[2023-10-09 12:45:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 27787264. Throughput: 0: 1805.6, 1: 1824.5. Samples: 6950218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:45:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 12:45:44,025][86121] Updated weights for policy 0, policy_version 13540 (0.0010) +[2023-10-09 12:45:44,387][86121] Updated weights for policy 0, policy_version 13550 (0.0010) +[2023-10-09 12:45:44,757][86121] Updated weights for policy 0, policy_version 13560 (0.0007) +[2023-10-09 12:45:45,577][86122] Updated weights for policy 1, policy_version 13610 (0.0011) +[2023-10-09 12:45:45,939][86122] Updated weights for policy 1, policy_version 13620 (0.0009) +[2023-10-09 12:45:46,298][86122] Updated weights for policy 1, policy_version 13630 (0.0008) +[2023-10-09 12:45:48,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 27852800. Throughput: 0: 1804.1, 1: 1827.1. Samples: 6971820. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:45:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:45:48,515][86121] Updated weights for policy 0, policy_version 13570 (0.0009) +[2023-10-09 12:45:48,879][86121] Updated weights for policy 0, policy_version 13580 (0.0009) +[2023-10-09 12:45:49,249][86121] Updated weights for policy 0, policy_version 13590 (0.0007) +[2023-10-09 12:45:49,618][86121] Updated weights for policy 0, policy_version 13600 (0.0008) +[2023-10-09 12:45:50,097][86122] Updated weights for policy 1, policy_version 13640 (0.0011) +[2023-10-09 12:45:50,467][86122] Updated weights for policy 1, policy_version 13650 (0.0011) +[2023-10-09 12:45:50,834][86122] Updated weights for policy 1, policy_version 13660 (0.0012) +[2023-10-09 12:45:53,347][86121] Updated weights for policy 0, policy_version 13610 (0.0007) +[2023-10-09 12:45:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 27918336. Throughput: 0: 1811.7, 1: 1831.1. Samples: 6994706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:45:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 12:45:53,713][86121] Updated weights for policy 0, policy_version 13620 (0.0009) +[2023-10-09 12:45:54,088][86121] Updated weights for policy 0, policy_version 13630 (0.0009) +[2023-10-09 12:45:54,403][86122] Updated weights for policy 1, policy_version 13670 (0.0008) +[2023-10-09 12:45:54,758][86122] Updated weights for policy 1, policy_version 13680 (0.0009) +[2023-10-09 12:45:55,126][86122] Updated weights for policy 1, policy_version 13690 (0.0010) +[2023-10-09 12:45:57,874][86121] Updated weights for policy 0, policy_version 13640 (0.0008) +[2023-10-09 12:45:58,237][86121] Updated weights for policy 0, policy_version 13650 (0.0010) +[2023-10-09 12:45:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 27983872. Throughput: 0: 1808.8, 1: 1830.0. Samples: 7004640. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:45:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:45:58,605][86121] Updated weights for policy 0, policy_version 13660 (0.0009) +[2023-10-09 12:45:58,767][86122] Updated weights for policy 1, policy_version 13700 (0.0010) +[2023-10-09 12:45:59,129][86122] Updated weights for policy 1, policy_version 13710 (0.0008) +[2023-10-09 12:45:59,493][86122] Updated weights for policy 1, policy_version 13720 (0.0008) +[2023-10-09 12:46:02,264][86121] Updated weights for policy 0, policy_version 13670 (0.0009) +[2023-10-09 12:46:02,640][86121] Updated weights for policy 0, policy_version 13680 (0.0007) +[2023-10-09 12:46:03,019][86121] Updated weights for policy 0, policy_version 13690 (0.0007) +[2023-10-09 12:46:03,320][86122] Updated weights for policy 1, policy_version 13730 (0.0008) +[2023-10-09 12:46:03,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28082176. Throughput: 0: 1804.8, 1: 1828.8. Samples: 7027294. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 12:46:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.960')] +[2023-10-09 12:46:03,674][86122] Updated weights for policy 1, policy_version 13740 (0.0009) +[2023-10-09 12:46:04,044][86122] Updated weights for policy 1, policy_version 13750 (0.0010) +[2023-10-09 12:46:04,399][86122] Updated weights for policy 1, policy_version 13760 (0.0010) +[2023-10-09 12:46:06,660][86121] Updated weights for policy 0, policy_version 13700 (0.0007) +[2023-10-09 12:46:07,028][86121] Updated weights for policy 0, policy_version 13710 (0.0007) +[2023-10-09 12:46:07,396][86121] Updated weights for policy 0, policy_version 13720 (0.0007) +[2023-10-09 12:46:08,161][86122] Updated weights for policy 1, policy_version 13770 (0.0009) +[2023-10-09 12:46:08,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28147712. Throughput: 0: 1803.9, 1: 1830.1. Samples: 7048682. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 12:46:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.970')] +[2023-10-09 12:46:08,529][86122] Updated weights for policy 1, policy_version 13780 (0.0009) +[2023-10-09 12:46:08,889][86122] Updated weights for policy 1, policy_version 13790 (0.0007) +[2023-10-09 12:46:10,983][86121] Updated weights for policy 0, policy_version 13730 (0.0007) +[2023-10-09 12:46:11,347][86121] Updated weights for policy 0, policy_version 13740 (0.0007) +[2023-10-09 12:46:11,718][86121] Updated weights for policy 0, policy_version 13750 (0.0008) +[2023-10-09 12:46:12,080][86121] Updated weights for policy 0, policy_version 13760 (0.0010) +[2023-10-09 12:46:12,552][86122] Updated weights for policy 1, policy_version 13800 (0.0007) +[2023-10-09 12:46:12,916][86122] Updated weights for policy 1, policy_version 13810 (0.0011) +[2023-10-09 12:46:13,286][86122] Updated weights for policy 1, policy_version 13820 (0.0008) +[2023-10-09 12:46:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 28213248. Throughput: 0: 1812.1, 1: 1834.4. Samples: 7060190. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 12:46:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.970')] +[2023-10-09 12:46:15,884][86121] Updated weights for policy 0, policy_version 13770 (0.0009) +[2023-10-09 12:46:16,250][86121] Updated weights for policy 0, policy_version 13780 (0.0010) +[2023-10-09 12:46:16,618][86121] Updated weights for policy 0, policy_version 13790 (0.0009) +[2023-10-09 12:46:17,147][86122] Updated weights for policy 1, policy_version 13830 (0.0008) +[2023-10-09 12:46:17,524][86122] Updated weights for policy 1, policy_version 13840 (0.0009) +[2023-10-09 12:46:17,878][86122] Updated weights for policy 1, policy_version 13850 (0.0007) +[2023-10-09 12:46:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28311552. Throughput: 0: 1809.9, 1: 1832.1. Samples: 7081424. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) +[2023-10-09 12:46:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.970')] +[2023-10-09 12:46:20,382][86121] Updated weights for policy 0, policy_version 13800 (0.0011) +[2023-10-09 12:46:20,758][86121] Updated weights for policy 0, policy_version 13810 (0.0010) +[2023-10-09 12:46:21,112][86121] Updated weights for policy 0, policy_version 13820 (0.0010) +[2023-10-09 12:46:21,379][86122] Updated weights for policy 1, policy_version 13860 (0.0008) +[2023-10-09 12:46:21,745][86122] Updated weights for policy 1, policy_version 13870 (0.0007) +[2023-10-09 12:46:22,110][86122] Updated weights for policy 1, policy_version 13880 (0.0010) +[2023-10-09 12:46:23,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28377088. Throughput: 0: 1805.6, 1: 1836.4. Samples: 7102894. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) +[2023-10-09 12:46:23,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 12:46:24,961][86121] Updated weights for policy 0, policy_version 13830 (0.0011) +[2023-10-09 12:46:25,332][86121] Updated weights for policy 0, policy_version 13840 (0.0010) +[2023-10-09 12:46:25,644][86122] Updated weights for policy 1, policy_version 13890 (0.0010) +[2023-10-09 12:46:25,695][86121] Updated weights for policy 0, policy_version 13850 (0.0010) +[2023-10-09 12:46:26,014][86122] Updated weights for policy 1, policy_version 13900 (0.0008) +[2023-10-09 12:46:26,381][86122] Updated weights for policy 1, policy_version 13910 (0.0008) +[2023-10-09 12:46:26,746][86122] Updated weights for policy 1, policy_version 13920 (0.0009) +[2023-10-09 12:46:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28442624. Throughput: 0: 1807.0, 1: 1831.3. Samples: 7113940. Policy #0 lag: (min: 31.0, avg: 31.1, max: 39.0) +[2023-10-09 12:46:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 12:46:29,496][86121] Updated weights for policy 0, policy_version 13860 (0.0009) +[2023-10-09 12:46:29,857][86121] Updated weights for policy 0, policy_version 13870 (0.0009) +[2023-10-09 12:46:30,229][86121] Updated weights for policy 0, policy_version 13880 (0.0010) +[2023-10-09 12:46:30,558][86122] Updated weights for policy 1, policy_version 13930 (0.0009) +[2023-10-09 12:46:30,918][86122] Updated weights for policy 1, policy_version 13940 (0.0008) +[2023-10-09 12:46:31,282][86122] Updated weights for policy 1, policy_version 13950 (0.0009) +[2023-10-09 12:46:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 28508160. Throughput: 0: 1800.1, 1: 1834.4. Samples: 7135372. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) +[2023-10-09 12:46:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:46:33,968][86121] Updated weights for policy 0, policy_version 13890 (0.0008) +[2023-10-09 12:46:34,331][86121] Updated weights for policy 0, policy_version 13900 (0.0009) +[2023-10-09 12:46:34,701][86121] Updated weights for policy 0, policy_version 13910 (0.0008) +[2023-10-09 12:46:34,903][86122] Updated weights for policy 1, policy_version 13960 (0.0007) +[2023-10-09 12:46:35,065][86121] Updated weights for policy 0, policy_version 13920 (0.0008) +[2023-10-09 12:46:35,267][86122] Updated weights for policy 1, policy_version 13970 (0.0008) +[2023-10-09 12:46:35,625][86122] Updated weights for policy 1, policy_version 13980 (0.0008) +[2023-10-09 12:46:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 28573696. Throughput: 0: 1798.0, 1: 1838.0. Samples: 7158322. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) +[2023-10-09 12:46:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 12:46:38,927][86121] Updated weights for policy 0, policy_version 13930 (0.0009) +[2023-10-09 12:46:39,301][86121] Updated weights for policy 0, policy_version 13940 (0.0009) +[2023-10-09 12:46:39,423][86122] Updated weights for policy 1, policy_version 13990 (0.0008) +[2023-10-09 12:46:39,672][86121] Updated weights for policy 0, policy_version 13950 (0.0009) +[2023-10-09 12:46:39,777][86122] Updated weights for policy 1, policy_version 14000 (0.0009) +[2023-10-09 12:46:40,151][86122] Updated weights for policy 1, policy_version 14010 (0.0008) +[2023-10-09 12:46:43,325][86121] Updated weights for policy 0, policy_version 13960 (0.0008) +[2023-10-09 12:46:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 28639232. Throughput: 0: 1792.4, 1: 1834.0. Samples: 7167830. Policy #0 lag: (min: 18.0, avg: 25.7, max: 50.0) +[2023-10-09 12:46:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:46:43,688][86121] Updated weights for policy 0, policy_version 13970 (0.0009) +[2023-10-09 12:46:43,827][86122] Updated weights for policy 1, policy_version 14020 (0.0009) +[2023-10-09 12:46:44,057][86121] Updated weights for policy 0, policy_version 13980 (0.0009) +[2023-10-09 12:46:44,189][86122] Updated weights for policy 1, policy_version 14030 (0.0008) +[2023-10-09 12:46:44,549][86122] Updated weights for policy 1, policy_version 14040 (0.0011) +[2023-10-09 12:46:47,787][86121] Updated weights for policy 0, policy_version 13990 (0.0008) +[2023-10-09 12:46:48,153][86121] Updated weights for policy 0, policy_version 14000 (0.0009) +[2023-10-09 12:46:48,210][86122] Updated weights for policy 1, policy_version 14050 (0.0009) +[2023-10-09 12:46:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 28704768. Throughput: 0: 1794.8, 1: 1831.2. Samples: 7190468. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 12:46:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:46:48,526][86121] Updated weights for policy 0, policy_version 14010 (0.0007) +[2023-10-09 12:46:48,570][86122] Updated weights for policy 1, policy_version 14060 (0.0008) +[2023-10-09 12:46:48,942][86122] Updated weights for policy 1, policy_version 14070 (0.0009) +[2023-10-09 12:46:49,305][86122] Updated weights for policy 1, policy_version 14080 (0.0010) +[2023-10-09 12:46:52,322][86121] Updated weights for policy 0, policy_version 14020 (0.0008) +[2023-10-09 12:46:52,694][86121] Updated weights for policy 0, policy_version 14030 (0.0008) +[2023-10-09 12:46:52,961][86122] Updated weights for policy 1, policy_version 14090 (0.0008) +[2023-10-09 12:46:53,065][86121] Updated weights for policy 0, policy_version 14040 (0.0010) +[2023-10-09 12:46:53,333][86122] Updated weights for policy 1, policy_version 14100 (0.0008) +[2023-10-09 12:46:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28803072. Throughput: 0: 1801.8, 1: 1824.4. Samples: 7211860. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 12:46:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 12:46:53,693][86122] Updated weights for policy 1, policy_version 14110 (0.0009) +[2023-10-09 12:46:56,822][86121] Updated weights for policy 0, policy_version 14050 (0.0010) +[2023-10-09 12:46:57,190][86121] Updated weights for policy 0, policy_version 14060 (0.0011) +[2023-10-09 12:46:57,487][86122] Updated weights for policy 1, policy_version 14120 (0.0009) +[2023-10-09 12:46:57,553][86121] Updated weights for policy 0, policy_version 14070 (0.0008) +[2023-10-09 12:46:57,840][86122] Updated weights for policy 1, policy_version 14130 (0.0009) +[2023-10-09 12:46:57,926][86121] Updated weights for policy 0, policy_version 14080 (0.0007) +[2023-10-09 12:46:58,204][86122] Updated weights for policy 1, policy_version 14140 (0.0008) +[2023-10-09 12:46:58,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 28901376. Throughput: 0: 1781.6, 1: 1829.8. Samples: 7222704. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 12:46:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:47:01,745][86121] Updated weights for policy 0, policy_version 14090 (0.0008) +[2023-10-09 12:47:02,082][86122] Updated weights for policy 1, policy_version 14150 (0.0008) +[2023-10-09 12:47:02,114][86121] Updated weights for policy 0, policy_version 14100 (0.0007) +[2023-10-09 12:47:02,468][86122] Updated weights for policy 1, policy_version 14160 (0.0007) +[2023-10-09 12:47:02,479][86121] Updated weights for policy 0, policy_version 14110 (0.0008) +[2023-10-09 12:47:02,838][86122] Updated weights for policy 1, policy_version 14170 (0.0009) +[2023-10-09 12:47:03,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 28966912. Throughput: 0: 1804.5, 1: 1822.2. Samples: 7244624. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 12:47:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 12:47:06,364][86121] Updated weights for policy 0, policy_version 14120 (0.0010) +[2023-10-09 12:47:06,548][86122] Updated weights for policy 1, policy_version 14180 (0.0009) +[2023-10-09 12:47:06,723][86121] Updated weights for policy 0, policy_version 14130 (0.0007) +[2023-10-09 12:47:06,910][86122] Updated weights for policy 1, policy_version 14190 (0.0008) +[2023-10-09 12:47:07,092][86121] Updated weights for policy 0, policy_version 14140 (0.0008) +[2023-10-09 12:47:07,279][86122] Updated weights for policy 1, policy_version 14200 (0.0009) +[2023-10-09 12:47:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29032448. Throughput: 0: 1783.2, 1: 1816.4. Samples: 7264878. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 12:47:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:47:10,847][86122] Updated weights for policy 1, policy_version 14210 (0.0009) +[2023-10-09 12:47:10,938][86121] Updated weights for policy 0, policy_version 14150 (0.0008) +[2023-10-09 12:47:11,209][86122] Updated weights for policy 1, policy_version 14220 (0.0007) +[2023-10-09 12:47:11,320][86121] Updated weights for policy 0, policy_version 14160 (0.0011) +[2023-10-09 12:47:11,578][86122] Updated weights for policy 1, policy_version 14230 (0.0008) +[2023-10-09 12:47:11,687][86121] Updated weights for policy 0, policy_version 14170 (0.0008) +[2023-10-09 12:47:11,939][86122] Updated weights for policy 1, policy_version 14240 (0.0010) +[2023-10-09 12:47:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29097984. Throughput: 0: 1805.5, 1: 1820.2. Samples: 7277094. Policy #0 lag: (min: 30.0, avg: 42.5, max: 62.0) +[2023-10-09 12:47:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:47:15,401][86121] Updated weights for policy 0, policy_version 14180 (0.0010) +[2023-10-09 12:47:15,713][86122] Updated weights for policy 1, policy_version 14250 (0.0010) +[2023-10-09 12:47:15,771][86121] Updated weights for policy 0, policy_version 14190 (0.0007) +[2023-10-09 12:47:16,072][86122] Updated weights for policy 1, policy_version 14260 (0.0009) +[2023-10-09 12:47:16,138][86121] Updated weights for policy 0, policy_version 14200 (0.0007) +[2023-10-09 12:47:16,441][86122] Updated weights for policy 1, policy_version 14270 (0.0008) +[2023-10-09 12:47:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29163520. Throughput: 0: 1776.0, 1: 1812.4. Samples: 7296852. Policy #0 lag: (min: 30.0, avg: 42.5, max: 62.0) +[2023-10-09 12:47:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:47:19,865][86121] Updated weights for policy 0, policy_version 14210 (0.0007) +[2023-10-09 12:47:20,143][86122] Updated weights for policy 1, policy_version 14280 (0.0007) +[2023-10-09 12:47:20,233][86121] Updated weights for policy 0, policy_version 14220 (0.0008) +[2023-10-09 12:47:20,507][86122] Updated weights for policy 1, policy_version 14290 (0.0008) +[2023-10-09 12:47:20,608][86121] Updated weights for policy 0, policy_version 14230 (0.0009) +[2023-10-09 12:47:20,867][86122] Updated weights for policy 1, policy_version 14300 (0.0008) +[2023-10-09 12:47:20,972][86121] Updated weights for policy 0, policy_version 14240 (0.0008) +[2023-10-09 12:47:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29229056. Throughput: 0: 1775.5, 1: 1806.8. Samples: 7319526. Policy #0 lag: (min: 30.0, avg: 42.5, max: 62.0) +[2023-10-09 12:47:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:47:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000014304_14647296.pth... +[2023-10-09 12:47:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000014240_14581760.pth... +[2023-10-09 12:47:23,440][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000012608_12910592.pth +[2023-10-09 12:47:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000012576_12877824.pth +[2023-10-09 12:47:24,594][86122] Updated weights for policy 1, policy_version 14310 (0.0008) +[2023-10-09 12:47:24,850][86121] Updated weights for policy 0, policy_version 14250 (0.0008) +[2023-10-09 12:47:24,960][86122] Updated weights for policy 1, policy_version 14320 (0.0009) +[2023-10-09 12:47:25,215][86121] Updated weights for policy 0, policy_version 14260 (0.0009) +[2023-10-09 12:47:25,323][86122] Updated weights for policy 1, policy_version 14330 (0.0009) +[2023-10-09 12:47:25,591][86121] Updated weights for policy 0, policy_version 14270 (0.0009) +[2023-10-09 12:47:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29294592. Throughput: 0: 1775.8, 1: 1810.1. Samples: 7329194. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) +[2023-10-09 12:47:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:47:29,123][86122] Updated weights for policy 1, policy_version 14340 (0.0008) +[2023-10-09 12:47:29,328][86121] Updated weights for policy 0, policy_version 14280 (0.0008) +[2023-10-09 12:47:29,496][86122] Updated weights for policy 1, policy_version 14350 (0.0007) +[2023-10-09 12:47:29,690][86121] Updated weights for policy 0, policy_version 14290 (0.0008) +[2023-10-09 12:47:29,871][86122] Updated weights for policy 1, policy_version 14360 (0.0007) +[2023-10-09 12:47:30,054][86121] Updated weights for policy 0, policy_version 14300 (0.0008) +[2023-10-09 12:47:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29360128. Throughput: 0: 1778.8, 1: 1802.6. Samples: 7351628. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) +[2023-10-09 12:47:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:47:33,622][86122] Updated weights for policy 1, policy_version 14370 (0.0009) +[2023-10-09 12:47:33,819][86121] Updated weights for policy 0, policy_version 14310 (0.0008) +[2023-10-09 12:47:33,975][86122] Updated weights for policy 1, policy_version 14380 (0.0009) +[2023-10-09 12:47:34,187][86121] Updated weights for policy 0, policy_version 14320 (0.0009) +[2023-10-09 12:47:34,344][86122] Updated weights for policy 1, policy_version 14390 (0.0009) +[2023-10-09 12:47:34,551][86121] Updated weights for policy 0, policy_version 14330 (0.0008) +[2023-10-09 12:47:34,704][86122] Updated weights for policy 1, policy_version 14400 (0.0009) +[2023-10-09 12:47:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29425664. Throughput: 0: 1803.9, 1: 1809.4. Samples: 7374456. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) +[2023-10-09 12:47:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 12:47:38,401][86121] Updated weights for policy 0, policy_version 14340 (0.0008) +[2023-10-09 12:47:38,431][86122] Updated weights for policy 1, policy_version 14410 (0.0009) +[2023-10-09 12:47:38,772][86121] Updated weights for policy 0, policy_version 14350 (0.0008) +[2023-10-09 12:47:38,796][86122] Updated weights for policy 1, policy_version 14420 (0.0008) +[2023-10-09 12:47:39,129][86121] Updated weights for policy 0, policy_version 14360 (0.0007) +[2023-10-09 12:47:39,163][86122] Updated weights for policy 1, policy_version 14430 (0.0008) +[2023-10-09 12:47:42,765][86122] Updated weights for policy 1, policy_version 14440 (0.0008) +[2023-10-09 12:47:42,851][86121] Updated weights for policy 0, policy_version 14370 (0.0010) +[2023-10-09 12:47:43,126][86122] Updated weights for policy 1, policy_version 14450 (0.0007) +[2023-10-09 12:47:43,217][86121] Updated weights for policy 0, policy_version 14380 (0.0010) +[2023-10-09 12:47:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 29491200. Throughput: 0: 1781.7, 1: 1807.9. Samples: 7384236. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 12:47:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:47:43,497][86122] Updated weights for policy 1, policy_version 14460 (0.0007) +[2023-10-09 12:47:43,576][86121] Updated weights for policy 0, policy_version 14390 (0.0008) +[2023-10-09 12:47:43,947][86121] Updated weights for policy 0, policy_version 14400 (0.0008) +[2023-10-09 12:47:47,189][86122] Updated weights for policy 1, policy_version 14470 (0.0009) +[2023-10-09 12:47:47,567][86122] Updated weights for policy 1, policy_version 14480 (0.0008) +[2023-10-09 12:47:47,696][86121] Updated weights for policy 0, policy_version 14410 (0.0009) +[2023-10-09 12:47:47,935][86122] Updated weights for policy 1, policy_version 14490 (0.0008) +[2023-10-09 12:47:48,066][86121] Updated weights for policy 0, policy_version 14420 (0.0007) +[2023-10-09 12:47:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 29589504. Throughput: 0: 1796.1, 1: 1814.7. Samples: 7407108. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 12:47:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:47:48,439][86121] Updated weights for policy 0, policy_version 14430 (0.0010) +[2023-10-09 12:47:51,587][86122] Updated weights for policy 1, policy_version 14500 (0.0008) +[2023-10-09 12:47:51,950][86122] Updated weights for policy 1, policy_version 14510 (0.0007) +[2023-10-09 12:47:52,039][86121] Updated weights for policy 0, policy_version 14440 (0.0009) +[2023-10-09 12:47:52,317][86122] Updated weights for policy 1, policy_version 14520 (0.0008) +[2023-10-09 12:47:52,407][86121] Updated weights for policy 0, policy_version 14450 (0.0008) +[2023-10-09 12:47:52,766][86121] Updated weights for policy 0, policy_version 14460 (0.0009) +[2023-10-09 12:47:53,397][85186] Fps is (10 sec: 19660.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 29687808. Throughput: 0: 1784.0, 1: 1813.6. Samples: 7426774. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) +[2023-10-09 12:47:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:47:55,890][86122] Updated weights for policy 1, policy_version 14530 (0.0007) +[2023-10-09 12:47:56,257][86122] Updated weights for policy 1, policy_version 14540 (0.0008) +[2023-10-09 12:47:56,618][86122] Updated weights for policy 1, policy_version 14550 (0.0008) +[2023-10-09 12:47:56,623][86121] Updated weights for policy 0, policy_version 14470 (0.0007) +[2023-10-09 12:47:56,988][86122] Updated weights for policy 1, policy_version 14560 (0.0009) +[2023-10-09 12:47:57,007][86121] Updated weights for policy 0, policy_version 14480 (0.0008) +[2023-10-09 12:47:57,372][86121] Updated weights for policy 0, policy_version 14490 (0.0007) +[2023-10-09 12:47:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29753344. Throughput: 0: 1798.4, 1: 1814.4. Samples: 7439674. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) +[2023-10-09 12:47:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:00,546][86122] Updated weights for policy 1, policy_version 14570 (0.0010) +[2023-10-09 12:48:00,906][86122] Updated weights for policy 1, policy_version 14580 (0.0010) +[2023-10-09 12:48:01,073][86121] Updated weights for policy 0, policy_version 14500 (0.0008) +[2023-10-09 12:48:01,276][86122] Updated weights for policy 1, policy_version 14590 (0.0008) +[2023-10-09 12:48:01,439][86121] Updated weights for policy 0, policy_version 14510 (0.0008) +[2023-10-09 12:48:01,806][86121] Updated weights for policy 0, policy_version 14520 (0.0008) +[2023-10-09 12:48:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 29818880. Throughput: 0: 1805.4, 1: 1817.6. Samples: 7459890. Policy #0 lag: (min: 31.0, avg: 38.8, max: 63.0) +[2023-10-09 12:48:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:04,941][86122] Updated weights for policy 1, policy_version 14600 (0.0007) +[2023-10-09 12:48:05,304][86122] Updated weights for policy 1, policy_version 14610 (0.0008) +[2023-10-09 12:48:05,433][86121] Updated weights for policy 0, policy_version 14530 (0.0009) +[2023-10-09 12:48:05,668][86122] Updated weights for policy 1, policy_version 14620 (0.0008) +[2023-10-09 12:48:05,806][86121] Updated weights for policy 0, policy_version 14540 (0.0009) +[2023-10-09 12:48:06,171][86121] Updated weights for policy 0, policy_version 14550 (0.0009) +[2023-10-09 12:48:06,536][86121] Updated weights for policy 0, policy_version 14560 (0.0012) +[2023-10-09 12:48:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 29884416. Throughput: 0: 1799.5, 1: 1823.0. Samples: 7482536. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 12:48:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:09,391][86122] Updated weights for policy 1, policy_version 14630 (0.0011) +[2023-10-09 12:48:09,762][86122] Updated weights for policy 1, policy_version 14640 (0.0010) +[2023-10-09 12:48:10,118][86122] Updated weights for policy 1, policy_version 14650 (0.0010) +[2023-10-09 12:48:10,309][86121] Updated weights for policy 0, policy_version 14570 (0.0009) +[2023-10-09 12:48:10,671][86121] Updated weights for policy 0, policy_version 14580 (0.0009) +[2023-10-09 12:48:11,037][86121] Updated weights for policy 0, policy_version 14590 (0.0008) +[2023-10-09 12:48:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 29949952. Throughput: 0: 1811.9, 1: 1821.8. Samples: 7492710. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 12:48:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:14,008][86122] Updated weights for policy 1, policy_version 14660 (0.0008) +[2023-10-09 12:48:14,389][86122] Updated weights for policy 1, policy_version 14670 (0.0009) +[2023-10-09 12:48:14,714][86121] Updated weights for policy 0, policy_version 14600 (0.0007) +[2023-10-09 12:48:14,743][86122] Updated weights for policy 1, policy_version 14680 (0.0008) +[2023-10-09 12:48:15,085][86121] Updated weights for policy 0, policy_version 14610 (0.0009) +[2023-10-09 12:48:15,463][86121] Updated weights for policy 0, policy_version 14620 (0.0010) +[2023-10-09 12:48:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30015488. Throughput: 0: 1799.7, 1: 1832.3. Samples: 7515068. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 12:48:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:18,398][86122] Updated weights for policy 1, policy_version 14690 (0.0008) +[2023-10-09 12:48:18,754][86122] Updated weights for policy 1, policy_version 14700 (0.0008) +[2023-10-09 12:48:19,104][86121] Updated weights for policy 0, policy_version 14630 (0.0008) +[2023-10-09 12:48:19,120][86122] Updated weights for policy 1, policy_version 14710 (0.0008) +[2023-10-09 12:48:19,472][86121] Updated weights for policy 0, policy_version 14640 (0.0007) +[2023-10-09 12:48:19,479][86122] Updated weights for policy 1, policy_version 14720 (0.0007) +[2023-10-09 12:48:19,834][86121] Updated weights for policy 0, policy_version 14650 (0.0010) +[2023-10-09 12:48:23,313][86122] Updated weights for policy 1, policy_version 14730 (0.0008) +[2023-10-09 12:48:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30081024. Throughput: 0: 1796.3, 1: 1830.5. Samples: 7537664. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:48:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:23,554][86121] Updated weights for policy 0, policy_version 14660 (0.0009) +[2023-10-09 12:48:23,682][86122] Updated weights for policy 1, policy_version 14740 (0.0008) +[2023-10-09 12:48:23,921][86121] Updated weights for policy 0, policy_version 14670 (0.0009) +[2023-10-09 12:48:24,037][86122] Updated weights for policy 1, policy_version 14750 (0.0008) +[2023-10-09 12:48:24,291][86121] Updated weights for policy 0, policy_version 14680 (0.0007) +[2023-10-09 12:48:27,673][86122] Updated weights for policy 1, policy_version 14760 (0.0009) +[2023-10-09 12:48:27,872][86121] Updated weights for policy 0, policy_version 14690 (0.0008) +[2023-10-09 12:48:28,043][86122] Updated weights for policy 1, policy_version 14770 (0.0008) +[2023-10-09 12:48:28,235][86121] Updated weights for policy 0, policy_version 14700 (0.0008) +[2023-10-09 12:48:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30146560. Throughput: 0: 1802.8, 1: 1823.8. Samples: 7547434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:48:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:48:28,398][86122] Updated weights for policy 1, policy_version 14780 (0.0008) +[2023-10-09 12:48:28,591][86121] Updated weights for policy 0, policy_version 14710 (0.0007) +[2023-10-09 12:48:28,958][86121] Updated weights for policy 0, policy_version 14720 (0.0010) +[2023-10-09 12:48:32,202][86122] Updated weights for policy 1, policy_version 14790 (0.0008) +[2023-10-09 12:48:32,580][86122] Updated weights for policy 1, policy_version 14800 (0.0008) +[2023-10-09 12:48:32,805][86121] Updated weights for policy 0, policy_version 14730 (0.0007) +[2023-10-09 12:48:32,947][86122] Updated weights for policy 1, policy_version 14810 (0.0008) +[2023-10-09 12:48:33,163][86121] Updated weights for policy 0, policy_version 14740 (0.0008) +[2023-10-09 12:48:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 30244864. Throughput: 0: 1805.7, 1: 1823.3. Samples: 7570408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:48:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:48:33,531][86121] Updated weights for policy 0, policy_version 14750 (0.0007) +[2023-10-09 12:48:36,606][86122] Updated weights for policy 1, policy_version 14820 (0.0008) +[2023-10-09 12:48:36,970][86122] Updated weights for policy 1, policy_version 14830 (0.0008) +[2023-10-09 12:48:37,100][86121] Updated weights for policy 0, policy_version 14760 (0.0009) +[2023-10-09 12:48:37,323][86122] Updated weights for policy 1, policy_version 14840 (0.0007) +[2023-10-09 12:48:37,473][86121] Updated weights for policy 0, policy_version 14770 (0.0008) +[2023-10-09 12:48:37,835][86121] Updated weights for policy 0, policy_version 14780 (0.0009) +[2023-10-09 12:48:38,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 30343168. Throughput: 0: 1810.2, 1: 1823.6. Samples: 7590292. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 12:48:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:48:40,970][86122] Updated weights for policy 1, policy_version 14850 (0.0008) +[2023-10-09 12:48:41,329][86122] Updated weights for policy 1, policy_version 14860 (0.0007) +[2023-10-09 12:48:41,696][86122] Updated weights for policy 1, policy_version 14870 (0.0008) +[2023-10-09 12:48:41,745][86121] Updated weights for policy 0, policy_version 14790 (0.0008) +[2023-10-09 12:48:42,061][86122] Updated weights for policy 1, policy_version 14880 (0.0007) +[2023-10-09 12:48:42,112][86121] Updated weights for policy 0, policy_version 14800 (0.0009) +[2023-10-09 12:48:42,474][86121] Updated weights for policy 0, policy_version 14810 (0.0009) +[2023-10-09 12:48:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 30408704. Throughput: 0: 1804.4, 1: 1825.3. Samples: 7603008. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 12:48:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:48:45,678][86122] Updated weights for policy 1, policy_version 14890 (0.0009) +[2023-10-09 12:48:46,051][86122] Updated weights for policy 1, policy_version 14900 (0.0010) +[2023-10-09 12:48:46,109][86121] Updated weights for policy 0, policy_version 14820 (0.0008) +[2023-10-09 12:48:46,409][86122] Updated weights for policy 1, policy_version 14910 (0.0007) +[2023-10-09 12:48:46,483][86121] Updated weights for policy 0, policy_version 14830 (0.0009) +[2023-10-09 12:48:46,847][86121] Updated weights for policy 0, policy_version 14840 (0.0010) +[2023-10-09 12:48:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 30474240. Throughput: 0: 1802.6, 1: 1824.9. Samples: 7623130. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) +[2023-10-09 12:48:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:48:50,048][86122] Updated weights for policy 1, policy_version 14920 (0.0007) +[2023-10-09 12:48:50,422][86122] Updated weights for policy 1, policy_version 14930 (0.0009) +[2023-10-09 12:48:50,698][86121] Updated weights for policy 0, policy_version 14850 (0.0010) +[2023-10-09 12:48:50,788][86122] Updated weights for policy 1, policy_version 14940 (0.0007) +[2023-10-09 12:48:51,066][86121] Updated weights for policy 0, policy_version 14860 (0.0009) +[2023-10-09 12:48:51,430][86121] Updated weights for policy 0, policy_version 14870 (0.0009) +[2023-10-09 12:48:51,794][86121] Updated weights for policy 0, policy_version 14880 (0.0008) +[2023-10-09 12:48:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 30539776. Throughput: 0: 1803.6, 1: 1821.5. Samples: 7645666. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) +[2023-10-09 12:48:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:54,429][86122] Updated weights for policy 1, policy_version 14950 (0.0007) +[2023-10-09 12:48:54,796][86122] Updated weights for policy 1, policy_version 14960 (0.0007) +[2023-10-09 12:48:55,164][86122] Updated weights for policy 1, policy_version 14970 (0.0008) +[2023-10-09 12:48:55,345][86121] Updated weights for policy 0, policy_version 14890 (0.0008) +[2023-10-09 12:48:55,714][86121] Updated weights for policy 0, policy_version 14900 (0.0011) +[2023-10-09 12:48:56,077][86121] Updated weights for policy 0, policy_version 14910 (0.0009) +[2023-10-09 12:48:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30605312. Throughput: 0: 1806.9, 1: 1821.3. Samples: 7655980. Policy #0 lag: (min: 31.0, avg: 40.6, max: 63.0) +[2023-10-09 12:48:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:48:58,839][86122] Updated weights for policy 1, policy_version 14980 (0.0010) +[2023-10-09 12:48:59,210][86122] Updated weights for policy 1, policy_version 14990 (0.0010) +[2023-10-09 12:48:59,563][86122] Updated weights for policy 1, policy_version 15000 (0.0008) +[2023-10-09 12:48:59,779][86121] Updated weights for policy 0, policy_version 14920 (0.0009) +[2023-10-09 12:49:00,151][86121] Updated weights for policy 0, policy_version 14930 (0.0008) +[2023-10-09 12:49:00,521][86121] Updated weights for policy 0, policy_version 14940 (0.0011) +[2023-10-09 12:49:03,279][86122] Updated weights for policy 1, policy_version 15010 (0.0007) +[2023-10-09 12:49:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30670848. Throughput: 0: 1812.0, 1: 1822.7. Samples: 7678632. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 12:49:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:49:03,642][86122] Updated weights for policy 1, policy_version 15020 (0.0009) +[2023-10-09 12:49:04,010][86122] Updated weights for policy 1, policy_version 15030 (0.0009) +[2023-10-09 12:49:04,319][86121] Updated weights for policy 0, policy_version 14950 (0.0008) +[2023-10-09 12:49:04,368][86122] Updated weights for policy 1, policy_version 15040 (0.0008) +[2023-10-09 12:49:04,687][86121] Updated weights for policy 0, policy_version 14960 (0.0007) +[2023-10-09 12:49:05,058][86121] Updated weights for policy 0, policy_version 14970 (0.0007) +[2023-10-09 12:49:08,038][86122] Updated weights for policy 1, policy_version 15050 (0.0010) +[2023-10-09 12:49:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 30736384. Throughput: 0: 1815.6, 1: 1818.3. Samples: 7701192. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 12:49:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:49:08,404][86122] Updated weights for policy 1, policy_version 15060 (0.0008) +[2023-10-09 12:49:08,632][86121] Updated weights for policy 0, policy_version 14980 (0.0008) +[2023-10-09 12:49:08,767][86122] Updated weights for policy 1, policy_version 15070 (0.0008) +[2023-10-09 12:49:09,002][86121] Updated weights for policy 0, policy_version 14990 (0.0009) +[2023-10-09 12:49:09,380][86121] Updated weights for policy 0, policy_version 15000 (0.0009) +[2023-10-09 12:49:12,470][86122] Updated weights for policy 1, policy_version 15080 (0.0008) +[2023-10-09 12:49:12,846][86122] Updated weights for policy 1, policy_version 15090 (0.0009) +[2023-10-09 12:49:13,080][86121] Updated weights for policy 0, policy_version 15010 (0.0007) +[2023-10-09 12:49:13,209][86122] Updated weights for policy 1, policy_version 15100 (0.0007) +[2023-10-09 12:49:13,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 30834688. Throughput: 0: 1813.6, 1: 1825.0. Samples: 7711170. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 12:49:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:49:13,445][86121] Updated weights for policy 0, policy_version 15020 (0.0007) +[2023-10-09 12:49:13,810][86121] Updated weights for policy 0, policy_version 15030 (0.0009) +[2023-10-09 12:49:14,172][86121] Updated weights for policy 0, policy_version 15040 (0.0008) +[2023-10-09 12:49:16,849][86122] Updated weights for policy 1, policy_version 15110 (0.0008) +[2023-10-09 12:49:17,209][86122] Updated weights for policy 1, policy_version 15120 (0.0007) +[2023-10-09 12:49:17,578][86122] Updated weights for policy 1, policy_version 15130 (0.0007) +[2023-10-09 12:49:17,917][86121] Updated weights for policy 0, policy_version 15050 (0.0009) +[2023-10-09 12:49:18,279][86121] Updated weights for policy 0, policy_version 15060 (0.0008) +[2023-10-09 12:49:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 30900224. Throughput: 0: 1808.4, 1: 1819.9. Samples: 7733680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:49:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:49:18,637][86121] Updated weights for policy 0, policy_version 15070 (0.0008) +[2023-10-09 12:49:21,323][86122] Updated weights for policy 1, policy_version 15140 (0.0008) +[2023-10-09 12:49:21,691][86122] Updated weights for policy 1, policy_version 15150 (0.0008) +[2023-10-09 12:49:22,063][86122] Updated weights for policy 1, policy_version 15160 (0.0008) +[2023-10-09 12:49:22,432][86121] Updated weights for policy 0, policy_version 15080 (0.0009) +[2023-10-09 12:49:22,797][86121] Updated weights for policy 0, policy_version 15090 (0.0007) +[2023-10-09 12:49:23,171][86121] Updated weights for policy 0, policy_version 15100 (0.0007) +[2023-10-09 12:49:23,398][85186] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 30998528. Throughput: 0: 1815.4, 1: 1828.9. Samples: 7754288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:49:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:49:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000015104_15466496.pth... +[2023-10-09 12:49:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000015168_15532032.pth... +[2023-10-09 12:49:23,440][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000013472_13795328.pth +[2023-10-09 12:49:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000013408_13729792.pth +[2023-10-09 12:49:25,806][86122] Updated weights for policy 1, policy_version 15170 (0.0008) +[2023-10-09 12:49:26,166][86122] Updated weights for policy 1, policy_version 15180 (0.0008) +[2023-10-09 12:49:26,534][86122] Updated weights for policy 1, policy_version 15190 (0.0011) +[2023-10-09 12:49:26,890][86122] Updated weights for policy 1, policy_version 15200 (0.0009) +[2023-10-09 12:49:26,935][86121] Updated weights for policy 0, policy_version 15110 (0.0007) +[2023-10-09 12:49:27,316][86121] Updated weights for policy 0, policy_version 15120 (0.0008) +[2023-10-09 12:49:27,689][86121] Updated weights for policy 0, policy_version 15130 (0.0008) +[2023-10-09 12:49:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 31064064. Throughput: 0: 1813.1, 1: 1823.0. Samples: 7766634. Policy #0 lag: (min: 24.0, avg: 45.3, max: 56.0) +[2023-10-09 12:49:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:49:30,742][86122] Updated weights for policy 1, policy_version 15210 (0.0009) +[2023-10-09 12:49:31,102][86122] Updated weights for policy 1, policy_version 15220 (0.0010) +[2023-10-09 12:49:31,444][86121] Updated weights for policy 0, policy_version 15140 (0.0008) +[2023-10-09 12:49:31,467][86122] Updated weights for policy 1, policy_version 15230 (0.0009) +[2023-10-09 12:49:31,815][86121] Updated weights for policy 0, policy_version 15150 (0.0009) +[2023-10-09 12:49:32,184][86121] Updated weights for policy 0, policy_version 15160 (0.0010) +[2023-10-09 12:49:33,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31129600. Throughput: 0: 1821.6, 1: 1820.7. Samples: 7787034. Policy #0 lag: (min: 24.0, avg: 45.3, max: 56.0) +[2023-10-09 12:49:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:49:35,083][86122] Updated weights for policy 1, policy_version 15240 (0.0008) +[2023-10-09 12:49:35,446][86122] Updated weights for policy 1, policy_version 15250 (0.0008) +[2023-10-09 12:49:35,814][86122] Updated weights for policy 1, policy_version 15260 (0.0010) +[2023-10-09 12:49:35,981][86121] Updated weights for policy 0, policy_version 15170 (0.0009) +[2023-10-09 12:49:36,352][86121] Updated weights for policy 0, policy_version 15180 (0.0009) +[2023-10-09 12:49:36,727][86121] Updated weights for policy 0, policy_version 15190 (0.0010) +[2023-10-09 12:49:37,097][86121] Updated weights for policy 0, policy_version 15200 (0.0008) +[2023-10-09 12:49:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 31195136. Throughput: 0: 1811.0, 1: 1818.0. Samples: 7808972. Policy #0 lag: (min: 24.0, avg: 45.3, max: 56.0) +[2023-10-09 12:49:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 12:49:39,407][86122] Updated weights for policy 1, policy_version 15270 (0.0010) +[2023-10-09 12:49:39,761][86122] Updated weights for policy 1, policy_version 15280 (0.0008) +[2023-10-09 12:49:40,120][86122] Updated weights for policy 1, policy_version 15290 (0.0008) +[2023-10-09 12:49:40,850][86121] Updated weights for policy 0, policy_version 15210 (0.0008) +[2023-10-09 12:49:41,214][86121] Updated weights for policy 0, policy_version 15220 (0.0007) +[2023-10-09 12:49:41,584][86121] Updated weights for policy 0, policy_version 15230 (0.0010) +[2023-10-09 12:49:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31260672. Throughput: 0: 1819.7, 1: 1819.8. Samples: 7819760. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:49:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:49:43,702][86122] Updated weights for policy 1, policy_version 15300 (0.0007) +[2023-10-09 12:49:44,076][86122] Updated weights for policy 1, policy_version 15310 (0.0010) +[2023-10-09 12:49:44,433][86122] Updated weights for policy 1, policy_version 15320 (0.0009) +[2023-10-09 12:49:45,366][86121] Updated weights for policy 0, policy_version 15240 (0.0010) +[2023-10-09 12:49:45,741][86121] Updated weights for policy 0, policy_version 15250 (0.0010) +[2023-10-09 12:49:46,099][86121] Updated weights for policy 0, policy_version 15260 (0.0008) +[2023-10-09 12:49:48,192][86122] Updated weights for policy 1, policy_version 15330 (0.0008) +[2023-10-09 12:49:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 31326208. Throughput: 0: 1801.6, 1: 1821.1. Samples: 7841656. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:49:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:49:48,562][86122] Updated weights for policy 1, policy_version 15340 (0.0009) +[2023-10-09 12:49:48,924][86122] Updated weights for policy 1, policy_version 15350 (0.0009) +[2023-10-09 12:49:49,298][86122] Updated weights for policy 1, policy_version 15360 (0.0011) +[2023-10-09 12:49:49,751][86121] Updated weights for policy 0, policy_version 15270 (0.0008) +[2023-10-09 12:49:50,116][86121] Updated weights for policy 0, policy_version 15280 (0.0008) +[2023-10-09 12:49:50,489][86121] Updated weights for policy 0, policy_version 15290 (0.0010) +[2023-10-09 12:49:52,860][86122] Updated weights for policy 1, policy_version 15370 (0.0009) +[2023-10-09 12:49:53,223][86122] Updated weights for policy 1, policy_version 15380 (0.0008) +[2023-10-09 12:49:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31391744. Throughput: 0: 1802.5, 1: 1819.5. Samples: 7864184. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:49:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:49:53,589][86122] Updated weights for policy 1, policy_version 15390 (0.0010) +[2023-10-09 12:49:54,364][86121] Updated weights for policy 0, policy_version 15300 (0.0009) +[2023-10-09 12:49:54,734][86121] Updated weights for policy 0, policy_version 15310 (0.0007) +[2023-10-09 12:49:55,098][86121] Updated weights for policy 0, policy_version 15320 (0.0008) +[2023-10-09 12:49:57,293][86122] Updated weights for policy 1, policy_version 15400 (0.0008) +[2023-10-09 12:49:57,661][86122] Updated weights for policy 1, policy_version 15410 (0.0009) +[2023-10-09 12:49:58,025][86122] Updated weights for policy 1, policy_version 15420 (0.0011) +[2023-10-09 12:49:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31490048. Throughput: 0: 1799.9, 1: 1827.3. Samples: 7874396. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) +[2023-10-09 12:49:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:49:58,872][86121] Updated weights for policy 0, policy_version 15330 (0.0008) +[2023-10-09 12:49:59,236][86121] Updated weights for policy 0, policy_version 15340 (0.0008) +[2023-10-09 12:49:59,606][86121] Updated weights for policy 0, policy_version 15350 (0.0009) +[2023-10-09 12:49:59,968][86121] Updated weights for policy 0, policy_version 15360 (0.0010) +[2023-10-09 12:50:01,857][86122] Updated weights for policy 1, policy_version 15430 (0.0011) +[2023-10-09 12:50:02,222][86122] Updated weights for policy 1, policy_version 15440 (0.0008) +[2023-10-09 12:50:02,588][86122] Updated weights for policy 1, policy_version 15450 (0.0008) +[2023-10-09 12:50:03,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31555584. Throughput: 0: 1797.0, 1: 1818.7. Samples: 7896388. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) +[2023-10-09 12:50:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:50:03,636][86121] Updated weights for policy 0, policy_version 15370 (0.0007) +[2023-10-09 12:50:03,998][86121] Updated weights for policy 0, policy_version 15380 (0.0008) +[2023-10-09 12:50:04,360][86121] Updated weights for policy 0, policy_version 15390 (0.0008) +[2023-10-09 12:50:06,206][86122] Updated weights for policy 1, policy_version 15460 (0.0007) +[2023-10-09 12:50:06,577][86122] Updated weights for policy 1, policy_version 15470 (0.0008) +[2023-10-09 12:50:06,945][86122] Updated weights for policy 1, policy_version 15480 (0.0008) +[2023-10-09 12:50:07,988][86121] Updated weights for policy 0, policy_version 15400 (0.0009) +[2023-10-09 12:50:08,348][86121] Updated weights for policy 0, policy_version 15410 (0.0009) +[2023-10-09 12:50:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31621120. Throughput: 0: 1815.4, 1: 1819.9. Samples: 7917878. Policy #0 lag: (min: 31.0, avg: 31.4, max: 45.0) +[2023-10-09 12:50:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:50:08,720][86121] Updated weights for policy 0, policy_version 15420 (0.0011) +[2023-10-09 12:50:10,653][86122] Updated weights for policy 1, policy_version 15490 (0.0010) +[2023-10-09 12:50:11,004][86122] Updated weights for policy 1, policy_version 15500 (0.0010) +[2023-10-09 12:50:11,374][86122] Updated weights for policy 1, policy_version 15510 (0.0007) +[2023-10-09 12:50:11,733][86122] Updated weights for policy 1, policy_version 15520 (0.0010) +[2023-10-09 12:50:12,372][86121] Updated weights for policy 0, policy_version 15430 (0.0008) +[2023-10-09 12:50:12,764][86121] Updated weights for policy 0, policy_version 15440 (0.0009) +[2023-10-09 12:50:13,128][86121] Updated weights for policy 0, policy_version 15450 (0.0007) +[2023-10-09 12:50:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 31719424. Throughput: 0: 1798.4, 1: 1816.6. Samples: 7929312. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:50:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:50:15,346][86122] Updated weights for policy 1, policy_version 15530 (0.0008) +[2023-10-09 12:50:15,710][86122] Updated weights for policy 1, policy_version 15540 (0.0010) +[2023-10-09 12:50:16,069][86122] Updated weights for policy 1, policy_version 15550 (0.0010) +[2023-10-09 12:50:16,857][86121] Updated weights for policy 0, policy_version 15460 (0.0007) +[2023-10-09 12:50:17,220][86121] Updated weights for policy 0, policy_version 15470 (0.0008) +[2023-10-09 12:50:17,595][86121] Updated weights for policy 0, policy_version 15480 (0.0010) +[2023-10-09 12:50:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 31784960. Throughput: 0: 1809.7, 1: 1827.0. Samples: 7950688. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:50:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:50:19,843][86122] Updated weights for policy 1, policy_version 15560 (0.0009) +[2023-10-09 12:50:20,203][86122] Updated weights for policy 1, policy_version 15570 (0.0008) +[2023-10-09 12:50:20,565][86122] Updated weights for policy 1, policy_version 15580 (0.0007) +[2023-10-09 12:50:21,269][86121] Updated weights for policy 0, policy_version 15490 (0.0008) +[2023-10-09 12:50:21,628][86121] Updated weights for policy 0, policy_version 15500 (0.0007) +[2023-10-09 12:50:21,997][86121] Updated weights for policy 0, policy_version 15510 (0.0008) +[2023-10-09 12:50:22,374][86121] Updated weights for policy 0, policy_version 15520 (0.0008) +[2023-10-09 12:50:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 31850496. Throughput: 0: 1801.0, 1: 1828.4. Samples: 7972294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:50:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:24,322][86122] Updated weights for policy 1, policy_version 15590 (0.0009) +[2023-10-09 12:50:24,683][86122] Updated weights for policy 1, policy_version 15600 (0.0009) +[2023-10-09 12:50:25,051][86122] Updated weights for policy 1, policy_version 15610 (0.0007) +[2023-10-09 12:50:26,041][86121] Updated weights for policy 0, policy_version 15530 (0.0009) +[2023-10-09 12:50:26,400][86121] Updated weights for policy 0, policy_version 15540 (0.0011) +[2023-10-09 12:50:26,766][86121] Updated weights for policy 0, policy_version 15550 (0.0011) +[2023-10-09 12:50:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 31916032. Throughput: 0: 1809.9, 1: 1827.8. Samples: 7983458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:50:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:28,750][86122] Updated weights for policy 1, policy_version 15620 (0.0008) +[2023-10-09 12:50:29,108][86122] Updated weights for policy 1, policy_version 15630 (0.0009) +[2023-10-09 12:50:29,470][86122] Updated weights for policy 1, policy_version 15640 (0.0009) +[2023-10-09 12:50:30,550][86121] Updated weights for policy 0, policy_version 15560 (0.0010) +[2023-10-09 12:50:30,915][86121] Updated weights for policy 0, policy_version 15570 (0.0009) +[2023-10-09 12:50:31,287][86121] Updated weights for policy 0, policy_version 15580 (0.0009) +[2023-10-09 12:50:33,097][86122] Updated weights for policy 1, policy_version 15650 (0.0009) +[2023-10-09 12:50:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 31981568. Throughput: 0: 1805.2, 1: 1825.2. Samples: 8005022. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:50:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:33,454][86122] Updated weights for policy 1, policy_version 15660 (0.0010) +[2023-10-09 12:50:33,815][86122] Updated weights for policy 1, policy_version 15670 (0.0011) +[2023-10-09 12:50:34,176][86122] Updated weights for policy 1, policy_version 15680 (0.0010) +[2023-10-09 12:50:34,942][86121] Updated weights for policy 0, policy_version 15590 (0.0008) +[2023-10-09 12:50:35,310][86121] Updated weights for policy 0, policy_version 15600 (0.0009) +[2023-10-09 12:50:35,683][86121] Updated weights for policy 0, policy_version 15610 (0.0010) +[2023-10-09 12:50:37,961][86122] Updated weights for policy 1, policy_version 15690 (0.0011) +[2023-10-09 12:50:38,321][86122] Updated weights for policy 1, policy_version 15700 (0.0011) +[2023-10-09 12:50:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32047104. Throughput: 0: 1808.4, 1: 1823.9. Samples: 8027638. Policy #0 lag: (min: 21.0, avg: 21.1, max: 27.0) +[2023-10-09 12:50:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:38,686][86122] Updated weights for policy 1, policy_version 15710 (0.0010) +[2023-10-09 12:50:39,461][86121] Updated weights for policy 0, policy_version 15620 (0.0008) +[2023-10-09 12:50:39,819][86121] Updated weights for policy 0, policy_version 15630 (0.0008) +[2023-10-09 12:50:40,188][86121] Updated weights for policy 0, policy_version 15640 (0.0008) +[2023-10-09 12:50:42,468][86122] Updated weights for policy 1, policy_version 15720 (0.0009) +[2023-10-09 12:50:42,839][86122] Updated weights for policy 1, policy_version 15730 (0.0007) +[2023-10-09 12:50:43,199][86122] Updated weights for policy 1, policy_version 15740 (0.0008) +[2023-10-09 12:50:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32145408. Throughput: 0: 1810.8, 1: 1816.7. Samples: 8037634. Policy #0 lag: (min: 21.0, avg: 21.1, max: 27.0) +[2023-10-09 12:50:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:44,082][86121] Updated weights for policy 0, policy_version 15650 (0.0008) +[2023-10-09 12:50:44,446][86121] Updated weights for policy 0, policy_version 15660 (0.0010) +[2023-10-09 12:50:44,816][86121] Updated weights for policy 0, policy_version 15670 (0.0010) +[2023-10-09 12:50:45,187][86121] Updated weights for policy 0, policy_version 15680 (0.0008) +[2023-10-09 12:50:46,823][86122] Updated weights for policy 1, policy_version 15750 (0.0008) +[2023-10-09 12:50:47,208][86122] Updated weights for policy 1, policy_version 15760 (0.0008) +[2023-10-09 12:50:47,584][86122] Updated weights for policy 1, policy_version 15770 (0.0009) +[2023-10-09 12:50:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32210944. Throughput: 0: 1809.5, 1: 1820.8. Samples: 8059750. Policy #0 lag: (min: 21.0, avg: 21.1, max: 27.0) +[2023-10-09 12:50:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:48,911][86121] Updated weights for policy 0, policy_version 15690 (0.0007) +[2023-10-09 12:50:49,273][86121] Updated weights for policy 0, policy_version 15700 (0.0008) +[2023-10-09 12:50:49,641][86121] Updated weights for policy 0, policy_version 15710 (0.0008) +[2023-10-09 12:50:51,202][86122] Updated weights for policy 1, policy_version 15780 (0.0009) +[2023-10-09 12:50:51,570][86122] Updated weights for policy 1, policy_version 15790 (0.0007) +[2023-10-09 12:50:51,931][86122] Updated weights for policy 1, policy_version 15800 (0.0010) +[2023-10-09 12:50:53,307][86121] Updated weights for policy 0, policy_version 15720 (0.0011) +[2023-10-09 12:50:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 32276480. Throughput: 0: 1811.2, 1: 1825.6. Samples: 8081534. Policy #0 lag: (min: 1.0, avg: 13.7, max: 33.0) +[2023-10-09 12:50:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:53,668][86121] Updated weights for policy 0, policy_version 15730 (0.0010) +[2023-10-09 12:50:54,035][86121] Updated weights for policy 0, policy_version 15740 (0.0010) +[2023-10-09 12:50:55,527][86122] Updated weights for policy 1, policy_version 15810 (0.0010) +[2023-10-09 12:50:55,887][86122] Updated weights for policy 1, policy_version 15820 (0.0011) +[2023-10-09 12:50:56,248][86122] Updated weights for policy 1, policy_version 15830 (0.0010) +[2023-10-09 12:50:56,610][86122] Updated weights for policy 1, policy_version 15840 (0.0008) +[2023-10-09 12:50:57,885][86121] Updated weights for policy 0, policy_version 15750 (0.0009) +[2023-10-09 12:50:58,270][86121] Updated weights for policy 0, policy_version 15760 (0.0011) +[2023-10-09 12:50:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32342016. Throughput: 0: 1798.7, 1: 1825.2. Samples: 8092388. Policy #0 lag: (min: 1.0, avg: 13.7, max: 33.0) +[2023-10-09 12:50:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:50:58,633][86121] Updated weights for policy 0, policy_version 15770 (0.0010) +[2023-10-09 12:51:00,318][86122] Updated weights for policy 1, policy_version 15850 (0.0008) +[2023-10-09 12:51:00,680][86122] Updated weights for policy 1, policy_version 15860 (0.0010) +[2023-10-09 12:51:01,043][86122] Updated weights for policy 1, policy_version 15870 (0.0010) +[2023-10-09 12:51:02,383][86121] Updated weights for policy 0, policy_version 15780 (0.0008) +[2023-10-09 12:51:02,743][86121] Updated weights for policy 0, policy_version 15790 (0.0007) +[2023-10-09 12:51:03,109][86121] Updated weights for policy 0, policy_version 15800 (0.0008) +[2023-10-09 12:51:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32407552. Throughput: 0: 1805.4, 1: 1827.8. Samples: 8114182. Policy #0 lag: (min: 1.0, avg: 13.7, max: 33.0) +[2023-10-09 12:51:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:51:04,674][86122] Updated weights for policy 1, policy_version 15880 (0.0009) +[2023-10-09 12:51:05,040][86122] Updated weights for policy 1, policy_version 15890 (0.0008) +[2023-10-09 12:51:05,403][86122] Updated weights for policy 1, policy_version 15900 (0.0010) +[2023-10-09 12:51:06,819][86121] Updated weights for policy 0, policy_version 15810 (0.0009) +[2023-10-09 12:51:07,187][86121] Updated weights for policy 0, policy_version 15820 (0.0010) +[2023-10-09 12:51:07,560][86121] Updated weights for policy 0, policy_version 15830 (0.0007) +[2023-10-09 12:51:07,928][86121] Updated weights for policy 0, policy_version 15840 (0.0008) +[2023-10-09 12:51:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32505856. Throughput: 0: 1802.8, 1: 1826.9. Samples: 8135630. Policy #0 lag: (min: 2.0, avg: 9.5, max: 34.0) +[2023-10-09 12:51:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:51:09,133][86122] Updated weights for policy 1, policy_version 15910 (0.0009) +[2023-10-09 12:51:09,489][86122] Updated weights for policy 1, policy_version 15920 (0.0008) +[2023-10-09 12:51:09,860][86122] Updated weights for policy 1, policy_version 15930 (0.0007) +[2023-10-09 12:51:11,656][86121] Updated weights for policy 0, policy_version 15850 (0.0008) +[2023-10-09 12:51:12,020][86121] Updated weights for policy 0, policy_version 15860 (0.0010) +[2023-10-09 12:51:12,385][86121] Updated weights for policy 0, policy_version 15870 (0.0010) +[2023-10-09 12:51:13,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32571392. Throughput: 0: 1807.1, 1: 1824.6. Samples: 8146882. Policy #0 lag: (min: 2.0, avg: 9.5, max: 34.0) +[2023-10-09 12:51:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:51:13,569][86122] Updated weights for policy 1, policy_version 15940 (0.0010) +[2023-10-09 12:51:13,932][86122] Updated weights for policy 1, policy_version 15950 (0.0011) +[2023-10-09 12:51:14,306][86122] Updated weights for policy 1, policy_version 15960 (0.0010) +[2023-10-09 12:51:15,863][86121] Updated weights for policy 0, policy_version 15880 (0.0009) +[2023-10-09 12:51:16,236][86121] Updated weights for policy 0, policy_version 15890 (0.0009) +[2023-10-09 12:51:16,602][86121] Updated weights for policy 0, policy_version 15900 (0.0010) +[2023-10-09 12:51:18,021][86122] Updated weights for policy 1, policy_version 15970 (0.0008) +[2023-10-09 12:51:18,383][86122] Updated weights for policy 1, policy_version 15980 (0.0009) +[2023-10-09 12:51:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32636928. Throughput: 0: 1805.0, 1: 1826.6. Samples: 8168444. Policy #0 lag: (min: 0.0, avg: 22.1, max: 32.0) +[2023-10-09 12:51:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 12:51:18,742][86122] Updated weights for policy 1, policy_version 15990 (0.0010) +[2023-10-09 12:51:19,109][86122] Updated weights for policy 1, policy_version 16000 (0.0009) +[2023-10-09 12:51:20,298][86121] Updated weights for policy 0, policy_version 15910 (0.0008) +[2023-10-09 12:51:20,663][86121] Updated weights for policy 0, policy_version 15920 (0.0009) +[2023-10-09 12:51:21,027][86121] Updated weights for policy 0, policy_version 15930 (0.0010) +[2023-10-09 12:51:22,895][86122] Updated weights for policy 1, policy_version 16010 (0.0008) +[2023-10-09 12:51:23,257][86122] Updated weights for policy 1, policy_version 16020 (0.0009) +[2023-10-09 12:51:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 32702464. Throughput: 0: 1802.6, 1: 1821.3. Samples: 8190712. Policy #0 lag: (min: 0.0, avg: 22.1, max: 32.0) +[2023-10-09 12:51:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 12:51:23,405][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000015936_16318464.pth... +[2023-10-09 12:51:23,434][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000014240_14581760.pth +[2023-10-09 12:51:23,624][86122] Updated weights for policy 1, policy_version 16030 (0.0008) +[2023-10-09 12:51:23,685][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000016032_16416768.pth... +[2023-10-09 12:51:23,719][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000014304_14647296.pth +[2023-10-09 12:51:24,747][86121] Updated weights for policy 0, policy_version 15940 (0.0008) +[2023-10-09 12:51:25,121][86121] Updated weights for policy 0, policy_version 15950 (0.0008) +[2023-10-09 12:51:25,495][86121] Updated weights for policy 0, policy_version 15960 (0.0010) +[2023-10-09 12:51:27,353][86122] Updated weights for policy 1, policy_version 16040 (0.0008) +[2023-10-09 12:51:27,730][86122] Updated weights for policy 1, policy_version 16050 (0.0008) +[2023-10-09 12:51:28,096][86122] Updated weights for policy 1, policy_version 16060 (0.0009) +[2023-10-09 12:51:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32800768. Throughput: 0: 1807.0, 1: 1823.2. Samples: 8200994. Policy #0 lag: (min: 0.0, avg: 22.1, max: 32.0) +[2023-10-09 12:51:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 12:51:29,347][86121] Updated weights for policy 0, policy_version 15970 (0.0009) +[2023-10-09 12:51:29,713][86121] Updated weights for policy 0, policy_version 15980 (0.0007) +[2023-10-09 12:51:30,081][86121] Updated weights for policy 0, policy_version 15990 (0.0007) +[2023-10-09 12:51:30,443][86121] Updated weights for policy 0, policy_version 16000 (0.0008) +[2023-10-09 12:51:31,927][86122] Updated weights for policy 1, policy_version 16070 (0.0009) +[2023-10-09 12:51:32,295][86122] Updated weights for policy 1, policy_version 16080 (0.0009) +[2023-10-09 12:51:32,662][86122] Updated weights for policy 1, policy_version 16090 (0.0008) +[2023-10-09 12:51:33,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 32866304. Throughput: 0: 1810.5, 1: 1827.8. Samples: 8223474. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) +[2023-10-09 12:51:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:51:34,057][86121] Updated weights for policy 0, policy_version 16010 (0.0007) +[2023-10-09 12:51:34,424][86121] Updated weights for policy 0, policy_version 16020 (0.0007) +[2023-10-09 12:51:34,792][86121] Updated weights for policy 0, policy_version 16030 (0.0007) +[2023-10-09 12:51:36,236][86122] Updated weights for policy 1, policy_version 16100 (0.0007) +[2023-10-09 12:51:36,608][86122] Updated weights for policy 1, policy_version 16110 (0.0009) +[2023-10-09 12:51:36,977][86122] Updated weights for policy 1, policy_version 16120 (0.0010) +[2023-10-09 12:51:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 32931840. Throughput: 0: 1815.6, 1: 1821.9. Samples: 8245222. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) +[2023-10-09 12:51:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:51:38,483][86121] Updated weights for policy 0, policy_version 16040 (0.0010) +[2023-10-09 12:51:38,841][86121] Updated weights for policy 0, policy_version 16050 (0.0008) +[2023-10-09 12:51:39,210][86121] Updated weights for policy 0, policy_version 16060 (0.0010) +[2023-10-09 12:51:40,681][86122] Updated weights for policy 1, policy_version 16130 (0.0008) +[2023-10-09 12:51:41,040][86122] Updated weights for policy 1, policy_version 16140 (0.0007) +[2023-10-09 12:51:41,407][86122] Updated weights for policy 1, policy_version 16150 (0.0008) +[2023-10-09 12:51:41,777][86122] Updated weights for policy 1, policy_version 16160 (0.0008) +[2023-10-09 12:51:42,697][86121] Updated weights for policy 0, policy_version 16070 (0.0007) +[2023-10-09 12:51:43,077][86121] Updated weights for policy 0, policy_version 16080 (0.0009) +[2023-10-09 12:51:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 32997376. Throughput: 0: 1818.6, 1: 1820.7. Samples: 8256158. Policy #0 lag: (min: 31.0, avg: 36.5, max: 63.0) +[2023-10-09 12:51:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:51:43,446][86121] Updated weights for policy 0, policy_version 16090 (0.0009) +[2023-10-09 12:51:45,349][86122] Updated weights for policy 1, policy_version 16170 (0.0010) +[2023-10-09 12:51:45,711][86122] Updated weights for policy 1, policy_version 16180 (0.0009) +[2023-10-09 12:51:46,074][86122] Updated weights for policy 1, policy_version 16190 (0.0010) +[2023-10-09 12:51:46,945][86121] Updated weights for policy 0, policy_version 16100 (0.0009) +[2023-10-09 12:51:47,318][86121] Updated weights for policy 0, policy_version 16110 (0.0011) +[2023-10-09 12:51:47,690][86121] Updated weights for policy 0, policy_version 16120 (0.0007) +[2023-10-09 12:51:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33095680. Throughput: 0: 1825.6, 1: 1818.5. Samples: 8278166. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:51:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:51:49,694][86122] Updated weights for policy 1, policy_version 16200 (0.0008) +[2023-10-09 12:51:50,052][86122] Updated weights for policy 1, policy_version 16210 (0.0008) +[2023-10-09 12:51:50,414][86122] Updated weights for policy 1, policy_version 16220 (0.0009) +[2023-10-09 12:51:51,715][86121] Updated weights for policy 0, policy_version 16130 (0.0009) +[2023-10-09 12:51:52,082][86121] Updated weights for policy 0, policy_version 16140 (0.0007) +[2023-10-09 12:51:52,453][86121] Updated weights for policy 0, policy_version 16150 (0.0008) +[2023-10-09 12:51:52,815][86121] Updated weights for policy 0, policy_version 16160 (0.0010) +[2023-10-09 12:51:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 33161216. Throughput: 0: 1819.6, 1: 1821.0. Samples: 8299454. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:51:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:51:54,004][86122] Updated weights for policy 1, policy_version 16230 (0.0007) +[2023-10-09 12:51:54,365][86122] Updated weights for policy 1, policy_version 16240 (0.0007) +[2023-10-09 12:51:54,723][86122] Updated weights for policy 1, policy_version 16250 (0.0007) +[2023-10-09 12:51:56,516][86121] Updated weights for policy 0, policy_version 16170 (0.0008) +[2023-10-09 12:51:56,884][86121] Updated weights for policy 0, policy_version 16180 (0.0010) +[2023-10-09 12:51:57,250][86121] Updated weights for policy 0, policy_version 16190 (0.0009) +[2023-10-09 12:51:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 33226752. Throughput: 0: 1819.1, 1: 1821.4. Samples: 8310706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:51:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:51:58,512][86122] Updated weights for policy 1, policy_version 16260 (0.0008) +[2023-10-09 12:51:58,876][86122] Updated weights for policy 1, policy_version 16270 (0.0009) +[2023-10-09 12:51:59,238][86122] Updated weights for policy 1, policy_version 16280 (0.0009) +[2023-10-09 12:52:00,925][86121] Updated weights for policy 0, policy_version 16200 (0.0007) +[2023-10-09 12:52:01,296][86121] Updated weights for policy 0, policy_version 16210 (0.0009) +[2023-10-09 12:52:01,664][86121] Updated weights for policy 0, policy_version 16220 (0.0010) +[2023-10-09 12:52:02,996][86122] Updated weights for policy 1, policy_version 16290 (0.0008) +[2023-10-09 12:52:03,364][86122] Updated weights for policy 1, policy_version 16300 (0.0008) +[2023-10-09 12:52:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 33292288. Throughput: 0: 1816.7, 1: 1820.0. Samples: 8332096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:52:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:52:03,721][86122] Updated weights for policy 1, policy_version 16310 (0.0009) +[2023-10-09 12:52:04,085][86122] Updated weights for policy 1, policy_version 16320 (0.0011) +[2023-10-09 12:52:05,379][86121] Updated weights for policy 0, policy_version 16230 (0.0009) +[2023-10-09 12:52:05,733][86121] Updated weights for policy 0, policy_version 16240 (0.0010) +[2023-10-09 12:52:06,112][86121] Updated weights for policy 0, policy_version 16250 (0.0009) +[2023-10-09 12:52:07,832][86122] Updated weights for policy 1, policy_version 16330 (0.0010) +[2023-10-09 12:52:08,195][86122] Updated weights for policy 1, policy_version 16340 (0.0009) +[2023-10-09 12:52:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 33357824. Throughput: 0: 1824.7, 1: 1817.1. Samples: 8354590. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:52:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:52:08,554][86122] Updated weights for policy 1, policy_version 16350 (0.0007) +[2023-10-09 12:52:09,762][86121] Updated weights for policy 0, policy_version 16260 (0.0008) +[2023-10-09 12:52:10,137][86121] Updated weights for policy 0, policy_version 16270 (0.0008) +[2023-10-09 12:52:10,499][86121] Updated weights for policy 0, policy_version 16280 (0.0008) +[2023-10-09 12:52:12,375][86122] Updated weights for policy 1, policy_version 16360 (0.0008) +[2023-10-09 12:52:12,743][86122] Updated weights for policy 1, policy_version 16370 (0.0007) +[2023-10-09 12:52:13,107][86122] Updated weights for policy 1, policy_version 16380 (0.0007) +[2023-10-09 12:52:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33456128. Throughput: 0: 1819.6, 1: 1817.3. Samples: 8364654. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 12:52:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:52:14,322][86121] Updated weights for policy 0, policy_version 16290 (0.0009) +[2023-10-09 12:52:14,682][86121] Updated weights for policy 0, policy_version 16300 (0.0008) +[2023-10-09 12:52:15,054][86121] Updated weights for policy 0, policy_version 16310 (0.0008) +[2023-10-09 12:52:15,426][86121] Updated weights for policy 0, policy_version 16320 (0.0009) +[2023-10-09 12:52:16,857][86122] Updated weights for policy 1, policy_version 16390 (0.0009) +[2023-10-09 12:52:17,225][86122] Updated weights for policy 1, policy_version 16400 (0.0011) +[2023-10-09 12:52:17,584][86122] Updated weights for policy 1, policy_version 16410 (0.0010) +[2023-10-09 12:52:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33521664. Throughput: 0: 1821.1, 1: 1813.4. Samples: 8387024. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 12:52:18,400][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:52:19,030][86121] Updated weights for policy 0, policy_version 16330 (0.0008) +[2023-10-09 12:52:19,400][86121] Updated weights for policy 0, policy_version 16340 (0.0007) +[2023-10-09 12:52:19,775][86121] Updated weights for policy 0, policy_version 16350 (0.0007) +[2023-10-09 12:52:21,521][86122] Updated weights for policy 1, policy_version 16420 (0.0010) +[2023-10-09 12:52:21,904][86122] Updated weights for policy 1, policy_version 16430 (0.0009) +[2023-10-09 12:52:22,275][86122] Updated weights for policy 1, policy_version 16440 (0.0007) +[2023-10-09 12:52:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33587200. Throughput: 0: 1819.2, 1: 1805.5. Samples: 8408330. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 12:52:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:52:23,463][86121] Updated weights for policy 0, policy_version 16360 (0.0009) +[2023-10-09 12:52:23,834][86121] Updated weights for policy 0, policy_version 16370 (0.0010) +[2023-10-09 12:52:24,192][86121] Updated weights for policy 0, policy_version 16380 (0.0007) +[2023-10-09 12:52:25,925][86122] Updated weights for policy 1, policy_version 16450 (0.0008) +[2023-10-09 12:52:26,285][86122] Updated weights for policy 1, policy_version 16460 (0.0007) +[2023-10-09 12:52:26,652][86122] Updated weights for policy 1, policy_version 16470 (0.0007) +[2023-10-09 12:52:27,016][86122] Updated weights for policy 1, policy_version 16480 (0.0008) +[2023-10-09 12:52:27,903][86121] Updated weights for policy 0, policy_version 16390 (0.0008) +[2023-10-09 12:52:28,276][86121] Updated weights for policy 0, policy_version 16400 (0.0010) +[2023-10-09 12:52:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 33652736. Throughput: 0: 1823.2, 1: 1817.7. Samples: 8419996. Policy #0 lag: (min: 8.0, avg: 30.9, max: 40.0) +[2023-10-09 12:52:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:52:28,641][86121] Updated weights for policy 0, policy_version 16410 (0.0011) +[2023-10-09 12:52:30,658][86122] Updated weights for policy 1, policy_version 16490 (0.0007) +[2023-10-09 12:52:31,033][86122] Updated weights for policy 1, policy_version 16500 (0.0008) +[2023-10-09 12:52:31,394][86122] Updated weights for policy 1, policy_version 16510 (0.0009) +[2023-10-09 12:52:32,410][86121] Updated weights for policy 0, policy_version 16420 (0.0010) +[2023-10-09 12:52:32,790][86121] Updated weights for policy 0, policy_version 16430 (0.0007) +[2023-10-09 12:52:33,148][86121] Updated weights for policy 0, policy_version 16440 (0.0007) +[2023-10-09 12:52:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 33718272. Throughput: 0: 1813.7, 1: 1807.3. Samples: 8441110. Policy #0 lag: (min: 8.0, avg: 30.9, max: 40.0) +[2023-10-09 12:52:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:52:34,981][86122] Updated weights for policy 1, policy_version 16520 (0.0009) +[2023-10-09 12:52:35,354][86122] Updated weights for policy 1, policy_version 16530 (0.0010) +[2023-10-09 12:52:35,706][86122] Updated weights for policy 1, policy_version 16540 (0.0010) +[2023-10-09 12:52:36,787][86121] Updated weights for policy 0, policy_version 16450 (0.0007) +[2023-10-09 12:52:37,153][86121] Updated weights for policy 0, policy_version 16460 (0.0007) +[2023-10-09 12:52:37,516][86121] Updated weights for policy 0, policy_version 16470 (0.0007) +[2023-10-09 12:52:37,884][86121] Updated weights for policy 0, policy_version 16480 (0.0007) +[2023-10-09 12:52:38,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 33816576. Throughput: 0: 1816.7, 1: 1806.1. Samples: 8462480. Policy #0 lag: (min: 6.0, avg: 14.6, max: 38.0) +[2023-10-09 12:52:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:52:39,592][86122] Updated weights for policy 1, policy_version 16550 (0.0008) +[2023-10-09 12:52:39,955][86122] Updated weights for policy 1, policy_version 16560 (0.0011) +[2023-10-09 12:52:40,319][86122] Updated weights for policy 1, policy_version 16570 (0.0009) +[2023-10-09 12:52:41,685][86121] Updated weights for policy 0, policy_version 16490 (0.0008) +[2023-10-09 12:52:42,045][86121] Updated weights for policy 0, policy_version 16500 (0.0007) +[2023-10-09 12:52:42,406][86121] Updated weights for policy 0, policy_version 16510 (0.0007) +[2023-10-09 12:52:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 33882112. Throughput: 0: 1817.5, 1: 1802.8. Samples: 8473620. Policy #0 lag: (min: 6.0, avg: 14.6, max: 38.0) +[2023-10-09 12:52:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:52:43,889][86122] Updated weights for policy 1, policy_version 16580 (0.0010) +[2023-10-09 12:52:44,252][86122] Updated weights for policy 1, policy_version 16590 (0.0007) +[2023-10-09 12:52:44,611][86122] Updated weights for policy 1, policy_version 16600 (0.0008) +[2023-10-09 12:52:46,110][86121] Updated weights for policy 0, policy_version 16520 (0.0008) +[2023-10-09 12:52:46,469][86121] Updated weights for policy 0, policy_version 16530 (0.0007) +[2023-10-09 12:52:46,832][86121] Updated weights for policy 0, policy_version 16540 (0.0007) +[2023-10-09 12:52:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 33947648. Throughput: 0: 1820.1, 1: 1804.8. Samples: 8495218. Policy #0 lag: (min: 6.0, avg: 14.6, max: 38.0) +[2023-10-09 12:52:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:52:48,524][86122] Updated weights for policy 1, policy_version 16610 (0.0008) +[2023-10-09 12:52:48,882][86122] Updated weights for policy 1, policy_version 16620 (0.0010) +[2023-10-09 12:52:49,254][86122] Updated weights for policy 1, policy_version 16630 (0.0008) +[2023-10-09 12:52:49,620][86122] Updated weights for policy 1, policy_version 16640 (0.0008) +[2023-10-09 12:52:50,482][86121] Updated weights for policy 0, policy_version 16550 (0.0007) +[2023-10-09 12:52:50,850][86121] Updated weights for policy 0, policy_version 16560 (0.0008) +[2023-10-09 12:52:51,221][86121] Updated weights for policy 0, policy_version 16570 (0.0009) +[2023-10-09 12:52:53,267][86122] Updated weights for policy 1, policy_version 16650 (0.0008) +[2023-10-09 12:52:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 34013184. Throughput: 0: 1808.3, 1: 1814.7. Samples: 8517624. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) +[2023-10-09 12:52:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:52:53,639][86122] Updated weights for policy 1, policy_version 16660 (0.0008) +[2023-10-09 12:52:53,999][86122] Updated weights for policy 1, policy_version 16670 (0.0010) +[2023-10-09 12:52:55,020][86121] Updated weights for policy 0, policy_version 16580 (0.0010) +[2023-10-09 12:52:55,393][86121] Updated weights for policy 0, policy_version 16590 (0.0007) +[2023-10-09 12:52:55,751][86121] Updated weights for policy 0, policy_version 16600 (0.0010) +[2023-10-09 12:52:57,713][86122] Updated weights for policy 1, policy_version 16680 (0.0008) +[2023-10-09 12:52:58,078][86122] Updated weights for policy 1, policy_version 16690 (0.0007) +[2023-10-09 12:52:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34078720. Throughput: 0: 1818.2, 1: 1806.4. Samples: 8527760. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) +[2023-10-09 12:52:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:52:58,444][86122] Updated weights for policy 1, policy_version 16700 (0.0007) +[2023-10-09 12:52:59,277][86121] Updated weights for policy 0, policy_version 16610 (0.0007) +[2023-10-09 12:52:59,649][86121] Updated weights for policy 0, policy_version 16620 (0.0010) +[2023-10-09 12:53:00,011][86121] Updated weights for policy 0, policy_version 16630 (0.0010) +[2023-10-09 12:53:00,378][86121] Updated weights for policy 0, policy_version 16640 (0.0008) +[2023-10-09 12:53:02,390][86122] Updated weights for policy 1, policy_version 16710 (0.0008) +[2023-10-09 12:53:02,763][86122] Updated weights for policy 1, policy_version 16720 (0.0009) +[2023-10-09 12:53:03,129][86122] Updated weights for policy 1, policy_version 16730 (0.0009) +[2023-10-09 12:53:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34177024. Throughput: 0: 1816.6, 1: 1806.7. Samples: 8550076. Policy #0 lag: (min: 9.0, avg: 19.9, max: 41.0) +[2023-10-09 12:53:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:53:04,122][86121] Updated weights for policy 0, policy_version 16650 (0.0009) +[2023-10-09 12:53:04,493][86121] Updated weights for policy 0, policy_version 16660 (0.0008) +[2023-10-09 12:53:04,863][86121] Updated weights for policy 0, policy_version 16670 (0.0007) +[2023-10-09 12:53:06,764][86122] Updated weights for policy 1, policy_version 16740 (0.0009) +[2023-10-09 12:53:07,147][86122] Updated weights for policy 1, policy_version 16750 (0.0009) +[2023-10-09 12:53:07,517][86122] Updated weights for policy 1, policy_version 16760 (0.0008) +[2023-10-09 12:53:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34242560. Throughput: 0: 1813.7, 1: 1810.4. Samples: 8571414. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 12:53:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:53:08,671][86121] Updated weights for policy 0, policy_version 16680 (0.0008) +[2023-10-09 12:53:09,044][86121] Updated weights for policy 0, policy_version 16690 (0.0007) +[2023-10-09 12:53:09,419][86121] Updated weights for policy 0, policy_version 16700 (0.0008) +[2023-10-09 12:53:11,337][86122] Updated weights for policy 1, policy_version 16770 (0.0009) +[2023-10-09 12:53:11,696][86122] Updated weights for policy 1, policy_version 16780 (0.0008) +[2023-10-09 12:53:12,065][86122] Updated weights for policy 1, policy_version 16790 (0.0009) +[2023-10-09 12:53:12,420][86122] Updated weights for policy 1, policy_version 16800 (0.0008) +[2023-10-09 12:53:13,205][86121] Updated weights for policy 0, policy_version 16710 (0.0010) +[2023-10-09 12:53:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 34308096. Throughput: 0: 1805.7, 1: 1804.0. Samples: 8582428. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 12:53:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:53:13,589][86121] Updated weights for policy 0, policy_version 16720 (0.0010) +[2023-10-09 12:53:13,954][86121] Updated weights for policy 0, policy_version 16730 (0.0008) +[2023-10-09 12:53:15,977][86122] Updated weights for policy 1, policy_version 16810 (0.0007) +[2023-10-09 12:53:16,336][86122] Updated weights for policy 1, policy_version 16820 (0.0010) +[2023-10-09 12:53:16,698][86122] Updated weights for policy 1, policy_version 16830 (0.0008) +[2023-10-09 12:53:17,623][86121] Updated weights for policy 0, policy_version 16740 (0.0009) +[2023-10-09 12:53:18,003][86121] Updated weights for policy 0, policy_version 16750 (0.0010) +[2023-10-09 12:53:18,366][86121] Updated weights for policy 0, policy_version 16760 (0.0009) +[2023-10-09 12:53:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 34373632. Throughput: 0: 1809.3, 1: 1806.6. Samples: 8603826. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 12:53:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:53:20,452][86122] Updated weights for policy 1, policy_version 16840 (0.0009) +[2023-10-09 12:53:20,814][86122] Updated weights for policy 1, policy_version 16850 (0.0009) +[2023-10-09 12:53:21,172][86122] Updated weights for policy 1, policy_version 16860 (0.0010) +[2023-10-09 12:53:22,009][86121] Updated weights for policy 0, policy_version 16770 (0.0008) +[2023-10-09 12:53:22,368][86121] Updated weights for policy 0, policy_version 16780 (0.0010) +[2023-10-09 12:53:22,727][86121] Updated weights for policy 0, policy_version 16790 (0.0009) +[2023-10-09 12:53:23,093][86121] Updated weights for policy 0, policy_version 16800 (0.0009) +[2023-10-09 12:53:23,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 34471936. Throughput: 0: 1814.7, 1: 1805.2. Samples: 8625376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:53:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:53:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000016800_17203200.pth... +[2023-10-09 12:53:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000016864_17268736.pth... +[2023-10-09 12:53:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000015168_15532032.pth +[2023-10-09 12:53:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000015104_15466496.pth +[2023-10-09 12:53:23,448][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000016864_17268736.pth +[2023-10-09 12:53:23,451][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000016800_17203200.pth +[2023-10-09 12:53:24,712][86122] Updated weights for policy 1, policy_version 16870 (0.0011) +[2023-10-09 12:53:25,074][86122] Updated weights for policy 1, policy_version 16880 (0.0007) +[2023-10-09 12:53:25,441][86122] Updated weights for policy 1, policy_version 16890 (0.0010) +[2023-10-09 12:53:26,981][86121] Updated weights for policy 0, policy_version 16810 (0.0011) +[2023-10-09 12:53:27,352][86121] Updated weights for policy 0, policy_version 16820 (0.0008) +[2023-10-09 12:53:27,726][86121] Updated weights for policy 0, policy_version 16830 (0.0007) +[2023-10-09 12:53:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34537472. Throughput: 0: 1808.1, 1: 1811.3. Samples: 8636492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:53:28,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:53:29,135][86122] Updated weights for policy 1, policy_version 16900 (0.0008) +[2023-10-09 12:53:29,501][86122] Updated weights for policy 1, policy_version 16910 (0.0009) +[2023-10-09 12:53:29,851][86122] Updated weights for policy 1, policy_version 16920 (0.0008) +[2023-10-09 12:53:31,252][86121] Updated weights for policy 0, policy_version 16840 (0.0008) +[2023-10-09 12:53:31,625][86121] Updated weights for policy 0, policy_version 16850 (0.0008) +[2023-10-09 12:53:32,000][86121] Updated weights for policy 0, policy_version 16860 (0.0011) +[2023-10-09 12:53:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 34603008. Throughput: 0: 1812.8, 1: 1807.7. Samples: 8658142. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) +[2023-10-09 12:53:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:53:33,686][86122] Updated weights for policy 1, policy_version 16930 (0.0008) +[2023-10-09 12:53:34,042][86122] Updated weights for policy 1, policy_version 16940 (0.0009) +[2023-10-09 12:53:34,407][86122] Updated weights for policy 1, policy_version 16950 (0.0008) +[2023-10-09 12:53:34,770][86122] Updated weights for policy 1, policy_version 16960 (0.0008) +[2023-10-09 12:53:35,770][86121] Updated weights for policy 0, policy_version 16870 (0.0008) +[2023-10-09 12:53:36,137][86121] Updated weights for policy 0, policy_version 16880 (0.0007) +[2023-10-09 12:53:36,496][86121] Updated weights for policy 0, policy_version 16890 (0.0008) +[2023-10-09 12:53:38,397][86122] Updated weights for policy 1, policy_version 16970 (0.0009) +[2023-10-09 12:53:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 34668544. Throughput: 0: 1810.8, 1: 1813.4. Samples: 8680710. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) +[2023-10-09 12:53:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:53:38,763][86122] Updated weights for policy 1, policy_version 16980 (0.0008) +[2023-10-09 12:53:39,120][86122] Updated weights for policy 1, policy_version 16990 (0.0007) +[2023-10-09 12:53:40,291][86121] Updated weights for policy 0, policy_version 16900 (0.0007) +[2023-10-09 12:53:40,661][86121] Updated weights for policy 0, policy_version 16910 (0.0007) +[2023-10-09 12:53:41,017][86121] Updated weights for policy 0, policy_version 16920 (0.0007) +[2023-10-09 12:53:42,867][86122] Updated weights for policy 1, policy_version 17000 (0.0008) +[2023-10-09 12:53:43,224][86122] Updated weights for policy 1, policy_version 17010 (0.0008) +[2023-10-09 12:53:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 34734080. Throughput: 0: 1815.9, 1: 1813.4. Samples: 8691080. Policy #0 lag: (min: 31.0, avg: 47.0, max: 63.0) +[2023-10-09 12:53:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:53:43,593][86122] Updated weights for policy 1, policy_version 17020 (0.0007) +[2023-10-09 12:53:44,598][86121] Updated weights for policy 0, policy_version 16930 (0.0007) +[2023-10-09 12:53:44,968][86121] Updated weights for policy 0, policy_version 16940 (0.0007) +[2023-10-09 12:53:45,341][86121] Updated weights for policy 0, policy_version 16950 (0.0008) +[2023-10-09 12:53:45,702][86121] Updated weights for policy 0, policy_version 16960 (0.0009) +[2023-10-09 12:53:47,215][86122] Updated weights for policy 1, policy_version 17030 (0.0007) +[2023-10-09 12:53:47,574][86122] Updated weights for policy 1, policy_version 17040 (0.0008) +[2023-10-09 12:53:47,934][86122] Updated weights for policy 1, policy_version 17050 (0.0007) +[2023-10-09 12:53:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34832384. Throughput: 0: 1806.8, 1: 1823.0. Samples: 8713418. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) +[2023-10-09 12:53:48,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:53:49,340][86121] Updated weights for policy 0, policy_version 16970 (0.0011) +[2023-10-09 12:53:49,708][86121] Updated weights for policy 0, policy_version 16980 (0.0009) +[2023-10-09 12:53:50,077][86121] Updated weights for policy 0, policy_version 16990 (0.0008) +[2023-10-09 12:53:51,825][86122] Updated weights for policy 1, policy_version 17060 (0.0010) +[2023-10-09 12:53:52,198][86122] Updated weights for policy 1, policy_version 17070 (0.0011) +[2023-10-09 12:53:52,560][86122] Updated weights for policy 1, policy_version 17080 (0.0007) +[2023-10-09 12:53:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 34897920. Throughput: 0: 1808.1, 1: 1820.1. Samples: 8734682. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) +[2023-10-09 12:53:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:53:53,828][86121] Updated weights for policy 0, policy_version 17000 (0.0008) +[2023-10-09 12:53:54,191][86121] Updated weights for policy 0, policy_version 17010 (0.0009) +[2023-10-09 12:53:54,559][86121] Updated weights for policy 0, policy_version 17020 (0.0009) +[2023-10-09 12:53:56,314][86122] Updated weights for policy 1, policy_version 17090 (0.0008) +[2023-10-09 12:53:56,725][86122] Updated weights for policy 1, policy_version 17100 (0.0007) +[2023-10-09 12:53:57,087][86122] Updated weights for policy 1, policy_version 17110 (0.0008) +[2023-10-09 12:53:57,456][86122] Updated weights for policy 1, policy_version 17120 (0.0007) +[2023-10-09 12:53:58,367][86121] Updated weights for policy 0, policy_version 17030 (0.0009) +[2023-10-09 12:53:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 34963456. Throughput: 0: 1812.5, 1: 1819.9. Samples: 8745888. Policy #0 lag: (min: 12.0, avg: 20.0, max: 44.0) +[2023-10-09 12:53:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:53:58,734][86121] Updated weights for policy 0, policy_version 17040 (0.0010) +[2023-10-09 12:53:59,098][86121] Updated weights for policy 0, policy_version 17050 (0.0008) +[2023-10-09 12:54:01,111][86122] Updated weights for policy 1, policy_version 17130 (0.0009) +[2023-10-09 12:54:01,472][86122] Updated weights for policy 1, policy_version 17140 (0.0008) +[2023-10-09 12:54:01,834][86122] Updated weights for policy 1, policy_version 17150 (0.0008) +[2023-10-09 12:54:02,842][86121] Updated weights for policy 0, policy_version 17060 (0.0008) +[2023-10-09 12:54:03,204][86121] Updated weights for policy 0, policy_version 17070 (0.0010) +[2023-10-09 12:54:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35028992. Throughput: 0: 1810.3, 1: 1815.0. Samples: 8766962. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) +[2023-10-09 12:54:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:54:03,575][86121] Updated weights for policy 0, policy_version 17080 (0.0008) +[2023-10-09 12:54:05,493][86122] Updated weights for policy 1, policy_version 17160 (0.0010) +[2023-10-09 12:54:05,853][86122] Updated weights for policy 1, policy_version 17170 (0.0008) +[2023-10-09 12:54:06,210][86122] Updated weights for policy 1, policy_version 17180 (0.0008) +[2023-10-09 12:54:07,248][86121] Updated weights for policy 0, policy_version 17090 (0.0009) +[2023-10-09 12:54:07,606][86121] Updated weights for policy 0, policy_version 17100 (0.0011) +[2023-10-09 12:54:07,972][86121] Updated weights for policy 0, policy_version 17110 (0.0011) +[2023-10-09 12:54:08,354][86121] Updated weights for policy 0, policy_version 17120 (0.0010) +[2023-10-09 12:54:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35127296. Throughput: 0: 1819.6, 1: 1818.2. Samples: 8789076. Policy #0 lag: (min: 23.0, avg: 25.7, max: 55.0) +[2023-10-09 12:54:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:54:09,836][86122] Updated weights for policy 1, policy_version 17190 (0.0010) +[2023-10-09 12:54:10,209][86122] Updated weights for policy 1, policy_version 17200 (0.0008) +[2023-10-09 12:54:10,575][86122] Updated weights for policy 1, policy_version 17210 (0.0010) +[2023-10-09 12:54:11,824][86121] Updated weights for policy 0, policy_version 17130 (0.0011) +[2023-10-09 12:54:12,187][86121] Updated weights for policy 0, policy_version 17140 (0.0008) +[2023-10-09 12:54:12,556][86121] Updated weights for policy 0, policy_version 17150 (0.0008) +[2023-10-09 12:54:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35192832. Throughput: 0: 1814.8, 1: 1817.1. Samples: 8799928. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:54:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:54:14,186][86122] Updated weights for policy 1, policy_version 17220 (0.0007) +[2023-10-09 12:54:14,551][86122] Updated weights for policy 1, policy_version 17230 (0.0010) +[2023-10-09 12:54:14,918][86122] Updated weights for policy 1, policy_version 17240 (0.0010) +[2023-10-09 12:54:16,286][86121] Updated weights for policy 0, policy_version 17160 (0.0008) +[2023-10-09 12:54:16,654][86121] Updated weights for policy 0, policy_version 17170 (0.0007) +[2023-10-09 12:54:17,018][86121] Updated weights for policy 0, policy_version 17180 (0.0008) +[2023-10-09 12:54:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 35258368. Throughput: 0: 1819.5, 1: 1820.4. Samples: 8821938. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:54:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:54:18,709][86122] Updated weights for policy 1, policy_version 17250 (0.0009) +[2023-10-09 12:54:19,070][86122] Updated weights for policy 1, policy_version 17260 (0.0009) +[2023-10-09 12:54:19,435][86122] Updated weights for policy 1, policy_version 17270 (0.0008) +[2023-10-09 12:54:19,793][86122] Updated weights for policy 1, policy_version 17280 (0.0011) +[2023-10-09 12:54:20,721][86121] Updated weights for policy 0, policy_version 17190 (0.0010) +[2023-10-09 12:54:21,091][86121] Updated weights for policy 0, policy_version 17200 (0.0011) +[2023-10-09 12:54:21,466][86121] Updated weights for policy 0, policy_version 17210 (0.0007) +[2023-10-09 12:54:23,342][86122] Updated weights for policy 1, policy_version 17290 (0.0010) +[2023-10-09 12:54:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35323904. Throughput: 0: 1816.9, 1: 1819.3. Samples: 8844338. Policy #0 lag: (min: 31.0, avg: 38.0, max: 63.0) +[2023-10-09 12:54:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:54:23,710][86122] Updated weights for policy 1, policy_version 17300 (0.0010) +[2023-10-09 12:54:24,074][86122] Updated weights for policy 1, policy_version 17310 (0.0009) +[2023-10-09 12:54:25,028][86121] Updated weights for policy 0, policy_version 17220 (0.0009) +[2023-10-09 12:54:25,392][86121] Updated weights for policy 0, policy_version 17230 (0.0008) +[2023-10-09 12:54:25,750][86121] Updated weights for policy 0, policy_version 17240 (0.0008) +[2023-10-09 12:54:27,913][86122] Updated weights for policy 1, policy_version 17320 (0.0008) +[2023-10-09 12:54:28,276][86122] Updated weights for policy 1, policy_version 17330 (0.0011) +[2023-10-09 12:54:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 35389440. Throughput: 0: 1815.3, 1: 1818.4. Samples: 8854598. Policy #0 lag: (min: 26.0, avg: 33.0, max: 58.0) +[2023-10-09 12:54:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:54:28,641][86122] Updated weights for policy 1, policy_version 17340 (0.0010) +[2023-10-09 12:54:29,442][86121] Updated weights for policy 0, policy_version 17250 (0.0008) +[2023-10-09 12:54:29,818][86121] Updated weights for policy 0, policy_version 17260 (0.0008) +[2023-10-09 12:54:30,179][86121] Updated weights for policy 0, policy_version 17270 (0.0008) +[2023-10-09 12:54:30,541][86121] Updated weights for policy 0, policy_version 17280 (0.0010) +[2023-10-09 12:54:32,429][86122] Updated weights for policy 1, policy_version 17350 (0.0008) +[2023-10-09 12:54:32,788][86122] Updated weights for policy 1, policy_version 17360 (0.0007) +[2023-10-09 12:54:33,152][86122] Updated weights for policy 1, policy_version 17370 (0.0008) +[2023-10-09 12:54:33,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 35487744. Throughput: 0: 1820.3, 1: 1813.9. Samples: 8876956. Policy #0 lag: (min: 26.0, avg: 33.0, max: 58.0) +[2023-10-09 12:54:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 12:54:34,274][86121] Updated weights for policy 0, policy_version 17290 (0.0010) +[2023-10-09 12:54:34,641][86121] Updated weights for policy 0, policy_version 17300 (0.0009) +[2023-10-09 12:54:35,008][86121] Updated weights for policy 0, policy_version 17310 (0.0011) +[2023-10-09 12:54:36,787][86122] Updated weights for policy 1, policy_version 17380 (0.0008) +[2023-10-09 12:54:37,143][86122] Updated weights for policy 1, policy_version 17390 (0.0008) +[2023-10-09 12:54:37,513][86122] Updated weights for policy 1, policy_version 17400 (0.0008) +[2023-10-09 12:54:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 35553280. Throughput: 0: 1818.1, 1: 1819.0. Samples: 8898352. Policy #0 lag: (min: 26.0, avg: 33.0, max: 58.0) +[2023-10-09 12:54:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:54:38,768][86121] Updated weights for policy 0, policy_version 17320 (0.0009) +[2023-10-09 12:54:39,139][86121] Updated weights for policy 0, policy_version 17330 (0.0010) +[2023-10-09 12:54:39,509][86121] Updated weights for policy 0, policy_version 17340 (0.0008) +[2023-10-09 12:54:41,210][86122] Updated weights for policy 1, policy_version 17410 (0.0007) +[2023-10-09 12:54:41,615][86122] Updated weights for policy 1, policy_version 17420 (0.0007) +[2023-10-09 12:54:41,977][86122] Updated weights for policy 1, policy_version 17430 (0.0009) +[2023-10-09 12:54:42,347][86122] Updated weights for policy 1, policy_version 17440 (0.0008) +[2023-10-09 12:54:43,167][86121] Updated weights for policy 0, policy_version 17350 (0.0009) +[2023-10-09 12:54:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35618816. Throughput: 0: 1818.9, 1: 1821.7. Samples: 8909716. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) +[2023-10-09 12:54:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:54:43,536][86121] Updated weights for policy 0, policy_version 17360 (0.0009) +[2023-10-09 12:54:43,903][86121] Updated weights for policy 0, policy_version 17370 (0.0008) +[2023-10-09 12:54:45,894][86122] Updated weights for policy 1, policy_version 17450 (0.0008) +[2023-10-09 12:54:46,249][86122] Updated weights for policy 1, policy_version 17460 (0.0007) +[2023-10-09 12:54:46,616][86122] Updated weights for policy 1, policy_version 17470 (0.0007) +[2023-10-09 12:54:47,581][86121] Updated weights for policy 0, policy_version 17380 (0.0007) +[2023-10-09 12:54:47,950][86121] Updated weights for policy 0, policy_version 17390 (0.0009) +[2023-10-09 12:54:48,319][86121] Updated weights for policy 0, policy_version 17400 (0.0009) +[2023-10-09 12:54:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 35684352. Throughput: 0: 1828.1, 1: 1823.5. Samples: 8931282. Policy #0 lag: (min: 15.0, avg: 19.7, max: 47.0) +[2023-10-09 12:54:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:54:50,346][86122] Updated weights for policy 1, policy_version 17480 (0.0008) +[2023-10-09 12:54:50,717][86122] Updated weights for policy 1, policy_version 17490 (0.0010) +[2023-10-09 12:54:51,079][86122] Updated weights for policy 1, policy_version 17500 (0.0008) +[2023-10-09 12:54:52,027][86121] Updated weights for policy 0, policy_version 17410 (0.0009) +[2023-10-09 12:54:52,396][86121] Updated weights for policy 0, policy_version 17420 (0.0009) +[2023-10-09 12:54:52,757][86121] Updated weights for policy 0, policy_version 17430 (0.0007) +[2023-10-09 12:54:53,122][86121] Updated weights for policy 0, policy_version 17440 (0.0009) +[2023-10-09 12:54:53,398][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 35782656. Throughput: 0: 1823.4, 1: 1822.0. Samples: 8953120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:54:53,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:54:54,680][86122] Updated weights for policy 1, policy_version 17510 (0.0009) +[2023-10-09 12:54:55,051][86122] Updated weights for policy 1, policy_version 17520 (0.0009) +[2023-10-09 12:54:55,419][86122] Updated weights for policy 1, policy_version 17530 (0.0010) +[2023-10-09 12:54:56,708][86121] Updated weights for policy 0, policy_version 17450 (0.0010) +[2023-10-09 12:54:57,079][86121] Updated weights for policy 0, policy_version 17460 (0.0009) +[2023-10-09 12:54:57,450][86121] Updated weights for policy 0, policy_version 17470 (0.0008) +[2023-10-09 12:54:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35848192. Throughput: 0: 1828.1, 1: 1823.5. Samples: 8964252. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:54:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:54:59,172][86122] Updated weights for policy 1, policy_version 17540 (0.0010) +[2023-10-09 12:54:59,540][86122] Updated weights for policy 1, policy_version 17550 (0.0011) +[2023-10-09 12:54:59,907][86122] Updated weights for policy 1, policy_version 17560 (0.0007) +[2023-10-09 12:55:01,428][86121] Updated weights for policy 0, policy_version 17480 (0.0007) +[2023-10-09 12:55:01,796][86121] Updated weights for policy 0, policy_version 17490 (0.0009) +[2023-10-09 12:55:02,161][86121] Updated weights for policy 0, policy_version 17500 (0.0007) +[2023-10-09 12:55:03,397][85186] Fps is (10 sec: 13107.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 35913728. Throughput: 0: 1822.3, 1: 1820.9. Samples: 8985880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:55:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:55:03,511][86122] Updated weights for policy 1, policy_version 17570 (0.0010) +[2023-10-09 12:55:03,866][86122] Updated weights for policy 1, policy_version 17580 (0.0008) +[2023-10-09 12:55:04,234][86122] Updated weights for policy 1, policy_version 17590 (0.0008) +[2023-10-09 12:55:04,588][86122] Updated weights for policy 1, policy_version 17600 (0.0008) +[2023-10-09 12:55:05,865][86121] Updated weights for policy 0, policy_version 17510 (0.0008) +[2023-10-09 12:55:06,226][86121] Updated weights for policy 0, policy_version 17520 (0.0008) +[2023-10-09 12:55:06,590][86121] Updated weights for policy 0, policy_version 17530 (0.0007) +[2023-10-09 12:55:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 35979264. Throughput: 0: 1824.5, 1: 1821.5. Samples: 9008408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:55:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:55:08,447][86122] Updated weights for policy 1, policy_version 17610 (0.0008) +[2023-10-09 12:55:08,813][86122] Updated weights for policy 1, policy_version 17620 (0.0008) +[2023-10-09 12:55:09,183][86122] Updated weights for policy 1, policy_version 17630 (0.0008) +[2023-10-09 12:55:10,198][86121] Updated weights for policy 0, policy_version 17540 (0.0008) +[2023-10-09 12:55:10,551][86121] Updated weights for policy 0, policy_version 17550 (0.0010) +[2023-10-09 12:55:10,922][86121] Updated weights for policy 0, policy_version 17560 (0.0011) +[2023-10-09 12:55:12,900][86122] Updated weights for policy 1, policy_version 17640 (0.0009) +[2023-10-09 12:55:13,258][86122] Updated weights for policy 1, policy_version 17650 (0.0007) +[2023-10-09 12:55:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 36044800. Throughput: 0: 1823.8, 1: 1824.4. Samples: 9018766. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:55:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:55:13,623][86122] Updated weights for policy 1, policy_version 17660 (0.0008) +[2023-10-09 12:55:14,743][86121] Updated weights for policy 0, policy_version 17570 (0.0011) +[2023-10-09 12:55:15,105][86121] Updated weights for policy 0, policy_version 17580 (0.0009) +[2023-10-09 12:55:15,468][86121] Updated weights for policy 0, policy_version 17590 (0.0008) +[2023-10-09 12:55:15,835][86121] Updated weights for policy 0, policy_version 17600 (0.0009) +[2023-10-09 12:55:17,305][86122] Updated weights for policy 1, policy_version 17670 (0.0009) +[2023-10-09 12:55:17,664][86122] Updated weights for policy 1, policy_version 17680 (0.0010) +[2023-10-09 12:55:18,032][86122] Updated weights for policy 1, policy_version 17690 (0.0008) +[2023-10-09 12:55:18,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36143104. Throughput: 0: 1815.2, 1: 1827.3. Samples: 9040870. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:55:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 12:55:19,617][86121] Updated weights for policy 0, policy_version 17610 (0.0008) +[2023-10-09 12:55:19,983][86121] Updated weights for policy 0, policy_version 17620 (0.0008) +[2023-10-09 12:55:20,351][86121] Updated weights for policy 0, policy_version 17630 (0.0008) +[2023-10-09 12:55:21,787][86122] Updated weights for policy 1, policy_version 17700 (0.0008) +[2023-10-09 12:55:22,147][86122] Updated weights for policy 1, policy_version 17710 (0.0008) +[2023-10-09 12:55:22,511][86122] Updated weights for policy 1, policy_version 17720 (0.0009) +[2023-10-09 12:55:23,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36208640. Throughput: 0: 1816.9, 1: 1823.6. Samples: 9062172. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) +[2023-10-09 12:55:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:55:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000017728_18153472.pth... +[2023-10-09 12:55:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000017632_18055168.pth... +[2023-10-09 12:55:23,444][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000015936_16318464.pth +[2023-10-09 12:55:23,448][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000016032_16416768.pth +[2023-10-09 12:55:24,013][86121] Updated weights for policy 0, policy_version 17640 (0.0009) +[2023-10-09 12:55:24,383][86121] Updated weights for policy 0, policy_version 17650 (0.0009) +[2023-10-09 12:55:24,742][86121] Updated weights for policy 0, policy_version 17660 (0.0007) +[2023-10-09 12:55:26,428][86122] Updated weights for policy 1, policy_version 17730 (0.0008) +[2023-10-09 12:55:26,838][86122] Updated weights for policy 1, policy_version 17740 (0.0010) +[2023-10-09 12:55:27,198][86122] Updated weights for policy 1, policy_version 17750 (0.0008) +[2023-10-09 12:55:27,564][86122] Updated weights for policy 1, policy_version 17760 (0.0008) +[2023-10-09 12:55:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36274176. Throughput: 0: 1820.1, 1: 1817.6. Samples: 9073416. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) +[2023-10-09 12:55:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:55:28,456][86121] Updated weights for policy 0, policy_version 17670 (0.0007) +[2023-10-09 12:55:28,822][86121] Updated weights for policy 0, policy_version 17680 (0.0007) +[2023-10-09 12:55:29,197][86121] Updated weights for policy 0, policy_version 17690 (0.0009) +[2023-10-09 12:55:31,142][86122] Updated weights for policy 1, policy_version 17770 (0.0007) +[2023-10-09 12:55:31,509][86122] Updated weights for policy 1, policy_version 17780 (0.0008) +[2023-10-09 12:55:31,871][86122] Updated weights for policy 1, policy_version 17790 (0.0008) +[2023-10-09 12:55:32,749][86121] Updated weights for policy 0, policy_version 17700 (0.0008) +[2023-10-09 12:55:33,110][86121] Updated weights for policy 0, policy_version 17710 (0.0009) +[2023-10-09 12:55:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36339712. Throughput: 0: 1814.7, 1: 1821.9. Samples: 9094934. Policy #0 lag: (min: 24.0, avg: 50.5, max: 56.0) +[2023-10-09 12:55:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 12:55:33,476][86121] Updated weights for policy 0, policy_version 17720 (0.0007) +[2023-10-09 12:55:35,439][86122] Updated weights for policy 1, policy_version 17800 (0.0010) +[2023-10-09 12:55:35,806][86122] Updated weights for policy 1, policy_version 17810 (0.0009) +[2023-10-09 12:55:36,165][86122] Updated weights for policy 1, policy_version 17820 (0.0008) +[2023-10-09 12:55:37,156][86121] Updated weights for policy 0, policy_version 17730 (0.0008) +[2023-10-09 12:55:37,529][86121] Updated weights for policy 0, policy_version 17740 (0.0009) +[2023-10-09 12:55:37,898][86121] Updated weights for policy 0, policy_version 17750 (0.0008) +[2023-10-09 12:55:38,270][86121] Updated weights for policy 0, policy_version 17760 (0.0008) +[2023-10-09 12:55:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36438016. Throughput: 0: 1816.5, 1: 1817.8. Samples: 9116664. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) +[2023-10-09 12:55:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:55:39,755][86122] Updated weights for policy 1, policy_version 17830 (0.0008) +[2023-10-09 12:55:40,128][86122] Updated weights for policy 1, policy_version 17840 (0.0007) +[2023-10-09 12:55:40,486][86122] Updated weights for policy 1, policy_version 17850 (0.0008) +[2023-10-09 12:55:42,013][86121] Updated weights for policy 0, policy_version 17770 (0.0007) +[2023-10-09 12:55:42,391][86121] Updated weights for policy 0, policy_version 17780 (0.0007) +[2023-10-09 12:55:42,761][86121] Updated weights for policy 0, policy_version 17790 (0.0007) +[2023-10-09 12:55:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36503552. Throughput: 0: 1810.9, 1: 1816.2. Samples: 9127470. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) +[2023-10-09 12:55:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 12:55:44,157][86122] Updated weights for policy 1, policy_version 17860 (0.0008) +[2023-10-09 12:55:44,528][86122] Updated weights for policy 1, policy_version 17870 (0.0009) +[2023-10-09 12:55:44,895][86122] Updated weights for policy 1, policy_version 17880 (0.0008) +[2023-10-09 12:55:46,557][86121] Updated weights for policy 0, policy_version 17800 (0.0008) +[2023-10-09 12:55:46,931][86121] Updated weights for policy 0, policy_version 17810 (0.0010) +[2023-10-09 12:55:47,317][86121] Updated weights for policy 0, policy_version 17820 (0.0007) +[2023-10-09 12:55:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36569088. Throughput: 0: 1817.3, 1: 1818.1. Samples: 9149476. Policy #0 lag: (min: 18.0, avg: 25.4, max: 50.0) +[2023-10-09 12:55:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:55:48,416][86122] Updated weights for policy 1, policy_version 17890 (0.0008) +[2023-10-09 12:55:48,782][86122] Updated weights for policy 1, policy_version 17900 (0.0008) +[2023-10-09 12:55:49,151][86122] Updated weights for policy 1, policy_version 17910 (0.0010) +[2023-10-09 12:55:49,507][86122] Updated weights for policy 1, policy_version 17920 (0.0010) +[2023-10-09 12:55:51,066][86121] Updated weights for policy 0, policy_version 17830 (0.0010) +[2023-10-09 12:55:51,442][86121] Updated weights for policy 0, policy_version 17840 (0.0010) +[2023-10-09 12:55:51,805][86121] Updated weights for policy 0, policy_version 17850 (0.0009) +[2023-10-09 12:55:53,370][86122] Updated weights for policy 1, policy_version 17930 (0.0009) +[2023-10-09 12:55:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 36634624. Throughput: 0: 1801.9, 1: 1819.5. Samples: 9171368. Policy #0 lag: (min: 18.0, avg: 25.4, max: 50.0) +[2023-10-09 12:55:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:55:53,731][86122] Updated weights for policy 1, policy_version 17940 (0.0011) +[2023-10-09 12:55:54,089][86122] Updated weights for policy 1, policy_version 17950 (0.0010) +[2023-10-09 12:55:55,629][86121] Updated weights for policy 0, policy_version 17860 (0.0010) +[2023-10-09 12:55:56,015][86121] Updated weights for policy 0, policy_version 17870 (0.0009) +[2023-10-09 12:55:56,386][86121] Updated weights for policy 0, policy_version 17880 (0.0009) +[2023-10-09 12:55:57,795][86122] Updated weights for policy 1, policy_version 17960 (0.0008) +[2023-10-09 12:55:58,154][86122] Updated weights for policy 1, policy_version 17970 (0.0009) +[2023-10-09 12:55:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36700160. Throughput: 0: 1812.5, 1: 1818.3. Samples: 9182148. Policy #0 lag: (min: 18.0, avg: 25.4, max: 50.0) +[2023-10-09 12:55:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:55:58,523][86122] Updated weights for policy 1, policy_version 17980 (0.0008) +[2023-10-09 12:56:00,192][86121] Updated weights for policy 0, policy_version 17890 (0.0008) +[2023-10-09 12:56:00,566][86121] Updated weights for policy 0, policy_version 17900 (0.0011) +[2023-10-09 12:56:00,935][86121] Updated weights for policy 0, policy_version 17910 (0.0008) +[2023-10-09 12:56:01,292][86121] Updated weights for policy 0, policy_version 17920 (0.0008) +[2023-10-09 12:56:02,277][86122] Updated weights for policy 1, policy_version 17990 (0.0008) +[2023-10-09 12:56:02,641][86122] Updated weights for policy 1, policy_version 18000 (0.0009) +[2023-10-09 12:56:03,011][86122] Updated weights for policy 1, policy_version 18010 (0.0007) +[2023-10-09 12:56:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 36798464. Throughput: 0: 1801.6, 1: 1821.2. Samples: 9203894. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 12:56:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:04,896][86121] Updated weights for policy 0, policy_version 17930 (0.0008) +[2023-10-09 12:56:05,271][86121] Updated weights for policy 0, policy_version 17940 (0.0007) +[2023-10-09 12:56:05,633][86121] Updated weights for policy 0, policy_version 17950 (0.0009) +[2023-10-09 12:56:06,612][86122] Updated weights for policy 1, policy_version 18020 (0.0007) +[2023-10-09 12:56:06,974][86122] Updated weights for policy 1, policy_version 18030 (0.0007) +[2023-10-09 12:56:07,334][86122] Updated weights for policy 1, policy_version 18040 (0.0007) +[2023-10-09 12:56:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36864000. Throughput: 0: 1799.2, 1: 1826.3. Samples: 9225320. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 12:56:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:09,349][86121] Updated weights for policy 0, policy_version 17960 (0.0008) +[2023-10-09 12:56:09,719][86121] Updated weights for policy 0, policy_version 17970 (0.0007) +[2023-10-09 12:56:10,075][86121] Updated weights for policy 0, policy_version 17980 (0.0008) +[2023-10-09 12:56:11,051][86122] Updated weights for policy 1, policy_version 18050 (0.0009) +[2023-10-09 12:56:11,448][86122] Updated weights for policy 1, policy_version 18060 (0.0008) +[2023-10-09 12:56:11,809][86122] Updated weights for policy 1, policy_version 18070 (0.0008) +[2023-10-09 12:56:12,185][86122] Updated weights for policy 1, policy_version 18080 (0.0007) +[2023-10-09 12:56:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 36929536. Throughput: 0: 1793.3, 1: 1832.4. Samples: 9236576. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 12:56:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:13,902][86121] Updated weights for policy 0, policy_version 17990 (0.0010) +[2023-10-09 12:56:14,286][86121] Updated weights for policy 0, policy_version 18000 (0.0008) +[2023-10-09 12:56:14,648][86121] Updated weights for policy 0, policy_version 18010 (0.0008) +[2023-10-09 12:56:15,666][86122] Updated weights for policy 1, policy_version 18090 (0.0009) +[2023-10-09 12:56:16,010][86122] Updated weights for policy 1, policy_version 18100 (0.0011) +[2023-10-09 12:56:16,381][86122] Updated weights for policy 1, policy_version 18110 (0.0009) +[2023-10-09 12:56:18,178][86121] Updated weights for policy 0, policy_version 18020 (0.0009) +[2023-10-09 12:56:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 36995072. Throughput: 0: 1799.0, 1: 1826.0. Samples: 9258058. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) +[2023-10-09 12:56:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:18,539][86121] Updated weights for policy 0, policy_version 18030 (0.0011) +[2023-10-09 12:56:18,900][86121] Updated weights for policy 0, policy_version 18040 (0.0010) +[2023-10-09 12:56:20,179][86122] Updated weights for policy 1, policy_version 18120 (0.0009) +[2023-10-09 12:56:20,547][86122] Updated weights for policy 1, policy_version 18130 (0.0009) +[2023-10-09 12:56:20,913][86122] Updated weights for policy 1, policy_version 18140 (0.0010) +[2023-10-09 12:56:22,606][86121] Updated weights for policy 0, policy_version 18050 (0.0010) +[2023-10-09 12:56:22,965][86121] Updated weights for policy 0, policy_version 18060 (0.0008) +[2023-10-09 12:56:23,325][86121] Updated weights for policy 0, policy_version 18070 (0.0010) +[2023-10-09 12:56:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37060608. Throughput: 0: 1814.6, 1: 1827.6. Samples: 9280562. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) +[2023-10-09 12:56:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:23,687][86121] Updated weights for policy 0, policy_version 18080 (0.0008) +[2023-10-09 12:56:24,470][86122] Updated weights for policy 1, policy_version 18150 (0.0008) +[2023-10-09 12:56:24,838][86122] Updated weights for policy 1, policy_version 18160 (0.0007) +[2023-10-09 12:56:25,198][86122] Updated weights for policy 1, policy_version 18170 (0.0008) +[2023-10-09 12:56:27,293][86121] Updated weights for policy 0, policy_version 18090 (0.0008) +[2023-10-09 12:56:27,661][86121] Updated weights for policy 0, policy_version 18100 (0.0009) +[2023-10-09 12:56:28,037][86121] Updated weights for policy 0, policy_version 18110 (0.0008) +[2023-10-09 12:56:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37158912. Throughput: 0: 1808.4, 1: 1827.2. Samples: 9291072. Policy #0 lag: (min: 24.0, avg: 51.6, max: 56.0) +[2023-10-09 12:56:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:28,919][86122] Updated weights for policy 1, policy_version 18180 (0.0010) +[2023-10-09 12:56:29,287][86122] Updated weights for policy 1, policy_version 18190 (0.0011) +[2023-10-09 12:56:29,646][86122] Updated weights for policy 1, policy_version 18200 (0.0010) +[2023-10-09 12:56:31,822][86121] Updated weights for policy 0, policy_version 18120 (0.0007) +[2023-10-09 12:56:32,188][86121] Updated weights for policy 0, policy_version 18130 (0.0008) +[2023-10-09 12:56:32,557][86121] Updated weights for policy 0, policy_version 18140 (0.0007) +[2023-10-09 12:56:33,385][86122] Updated weights for policy 1, policy_version 18210 (0.0008) +[2023-10-09 12:56:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 37224448. Throughput: 0: 1811.2, 1: 1823.6. Samples: 9313044. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 12:56:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:33,757][86122] Updated weights for policy 1, policy_version 18220 (0.0009) +[2023-10-09 12:56:34,122][86122] Updated weights for policy 1, policy_version 18230 (0.0008) +[2023-10-09 12:56:34,486][86122] Updated weights for policy 1, policy_version 18240 (0.0008) +[2023-10-09 12:56:36,231][86121] Updated weights for policy 0, policy_version 18150 (0.0008) +[2023-10-09 12:56:36,593][86121] Updated weights for policy 0, policy_version 18160 (0.0007) +[2023-10-09 12:56:36,960][86121] Updated weights for policy 0, policy_version 18170 (0.0008) +[2023-10-09 12:56:38,283][86122] Updated weights for policy 1, policy_version 18250 (0.0010) +[2023-10-09 12:56:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 37289984. Throughput: 0: 1818.6, 1: 1819.7. Samples: 9335092. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 12:56:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:38,652][86122] Updated weights for policy 1, policy_version 18260 (0.0009) +[2023-10-09 12:56:39,012][86122] Updated weights for policy 1, policy_version 18270 (0.0008) +[2023-10-09 12:56:40,678][86121] Updated weights for policy 0, policy_version 18180 (0.0008) +[2023-10-09 12:56:41,042][86121] Updated weights for policy 0, policy_version 18190 (0.0010) +[2023-10-09 12:56:41,410][86121] Updated weights for policy 0, policy_version 18200 (0.0008) +[2023-10-09 12:56:42,848][86122] Updated weights for policy 1, policy_version 18280 (0.0008) +[2023-10-09 12:56:43,213][86122] Updated weights for policy 1, policy_version 18290 (0.0009) +[2023-10-09 12:56:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37355520. Throughput: 0: 1818.8, 1: 1824.4. Samples: 9346092. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 12:56:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:43,577][86122] Updated weights for policy 1, policy_version 18300 (0.0008) +[2023-10-09 12:56:45,142][86121] Updated weights for policy 0, policy_version 18210 (0.0008) +[2023-10-09 12:56:45,503][86121] Updated weights for policy 0, policy_version 18220 (0.0010) +[2023-10-09 12:56:45,871][86121] Updated weights for policy 0, policy_version 18230 (0.0010) +[2023-10-09 12:56:46,235][86121] Updated weights for policy 0, policy_version 18240 (0.0010) +[2023-10-09 12:56:47,185][86122] Updated weights for policy 1, policy_version 18310 (0.0008) +[2023-10-09 12:56:47,547][86122] Updated weights for policy 1, policy_version 18320 (0.0009) +[2023-10-09 12:56:47,914][86122] Updated weights for policy 1, policy_version 18330 (0.0009) +[2023-10-09 12:56:48,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37453824. Throughput: 0: 1821.3, 1: 1818.4. Samples: 9367678. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 12:56:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:49,948][86121] Updated weights for policy 0, policy_version 18250 (0.0009) +[2023-10-09 12:56:50,320][86121] Updated weights for policy 0, policy_version 18260 (0.0009) +[2023-10-09 12:56:50,690][86121] Updated weights for policy 0, policy_version 18270 (0.0007) +[2023-10-09 12:56:51,521][86122] Updated weights for policy 1, policy_version 18340 (0.0008) +[2023-10-09 12:56:51,887][86122] Updated weights for policy 1, policy_version 18350 (0.0008) +[2023-10-09 12:56:52,258][86122] Updated weights for policy 1, policy_version 18360 (0.0007) +[2023-10-09 12:56:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 37519360. Throughput: 0: 1819.9, 1: 1816.1. Samples: 9388942. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:56:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:54,408][86121] Updated weights for policy 0, policy_version 18280 (0.0008) +[2023-10-09 12:56:54,784][86121] Updated weights for policy 0, policy_version 18290 (0.0007) +[2023-10-09 12:56:55,156][86121] Updated weights for policy 0, policy_version 18300 (0.0009) +[2023-10-09 12:56:55,980][86122] Updated weights for policy 1, policy_version 18370 (0.0007) +[2023-10-09 12:56:56,376][86122] Updated weights for policy 1, policy_version 18380 (0.0007) +[2023-10-09 12:56:56,745][86122] Updated weights for policy 1, policy_version 18390 (0.0008) +[2023-10-09 12:56:57,109][86122] Updated weights for policy 1, policy_version 18400 (0.0009) +[2023-10-09 12:56:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37584896. Throughput: 0: 1825.0, 1: 1814.1. Samples: 9400336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:56:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:56:58,652][86121] Updated weights for policy 0, policy_version 18310 (0.0010) +[2023-10-09 12:56:59,020][86121] Updated weights for policy 0, policy_version 18320 (0.0008) +[2023-10-09 12:56:59,390][86121] Updated weights for policy 0, policy_version 18330 (0.0009) +[2023-10-09 12:57:00,755][86122] Updated weights for policy 1, policy_version 18410 (0.0010) +[2023-10-09 12:57:01,122][86122] Updated weights for policy 1, policy_version 18420 (0.0010) +[2023-10-09 12:57:01,483][86122] Updated weights for policy 1, policy_version 18430 (0.0007) +[2023-10-09 12:57:03,179][86121] Updated weights for policy 0, policy_version 18340 (0.0009) +[2023-10-09 12:57:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 37650432. Throughput: 0: 1821.4, 1: 1813.7. Samples: 9421638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:57:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:03,567][86121] Updated weights for policy 0, policy_version 18350 (0.0009) +[2023-10-09 12:57:03,923][86121] Updated weights for policy 0, policy_version 18360 (0.0009) +[2023-10-09 12:57:05,070][86122] Updated weights for policy 1, policy_version 18440 (0.0011) +[2023-10-09 12:57:05,437][86122] Updated weights for policy 1, policy_version 18450 (0.0010) +[2023-10-09 12:57:05,799][86122] Updated weights for policy 1, policy_version 18460 (0.0008) +[2023-10-09 12:57:07,656][86121] Updated weights for policy 0, policy_version 18370 (0.0009) +[2023-10-09 12:57:08,024][86121] Updated weights for policy 0, policy_version 18380 (0.0009) +[2023-10-09 12:57:08,391][86121] Updated weights for policy 0, policy_version 18390 (0.0009) +[2023-10-09 12:57:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 37715968. Throughput: 0: 1815.3, 1: 1818.1. Samples: 9444066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:57:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:08,765][86121] Updated weights for policy 0, policy_version 18400 (0.0010) +[2023-10-09 12:57:09,529][86122] Updated weights for policy 1, policy_version 18470 (0.0008) +[2023-10-09 12:57:09,899][86122] Updated weights for policy 1, policy_version 18480 (0.0008) +[2023-10-09 12:57:10,256][86122] Updated weights for policy 1, policy_version 18490 (0.0010) +[2023-10-09 12:57:12,468][86121] Updated weights for policy 0, policy_version 18410 (0.0009) +[2023-10-09 12:57:12,832][86121] Updated weights for policy 0, policy_version 18420 (0.0009) +[2023-10-09 12:57:13,199][86121] Updated weights for policy 0, policy_version 18430 (0.0009) +[2023-10-09 12:57:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37814272. Throughput: 0: 1808.4, 1: 1817.9. Samples: 9454254. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:57:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:14,050][86122] Updated weights for policy 1, policy_version 18500 (0.0009) +[2023-10-09 12:57:14,411][86122] Updated weights for policy 1, policy_version 18510 (0.0009) +[2023-10-09 12:57:14,775][86122] Updated weights for policy 1, policy_version 18520 (0.0009) +[2023-10-09 12:57:17,104][86121] Updated weights for policy 0, policy_version 18440 (0.0010) +[2023-10-09 12:57:17,466][86121] Updated weights for policy 0, policy_version 18450 (0.0009) +[2023-10-09 12:57:17,831][86121] Updated weights for policy 0, policy_version 18460 (0.0008) +[2023-10-09 12:57:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 37879808. Throughput: 0: 1818.6, 1: 1814.6. Samples: 9476538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:57:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:18,600][86122] Updated weights for policy 1, policy_version 18530 (0.0009) +[2023-10-09 12:57:18,976][86122] Updated weights for policy 1, policy_version 18540 (0.0009) +[2023-10-09 12:57:19,342][86122] Updated weights for policy 1, policy_version 18550 (0.0009) +[2023-10-09 12:57:19,708][86122] Updated weights for policy 1, policy_version 18560 (0.0009) +[2023-10-09 12:57:21,504][86121] Updated weights for policy 0, policy_version 18470 (0.0009) +[2023-10-09 12:57:21,870][86121] Updated weights for policy 0, policy_version 18480 (0.0010) +[2023-10-09 12:57:22,237][86121] Updated weights for policy 0, policy_version 18490 (0.0010) +[2023-10-09 12:57:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 37945344. Throughput: 0: 1799.7, 1: 1814.3. Samples: 9497720. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:57:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 12:57:23,412][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000018496_18939904.pth... +[2023-10-09 12:57:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000016800_17203200.pth +[2023-10-09 12:57:23,558][86122] Updated weights for policy 1, policy_version 18570 (0.0007) +[2023-10-09 12:57:23,919][86122] Updated weights for policy 1, policy_version 18580 (0.0008) +[2023-10-09 12:57:24,287][86122] Updated weights for policy 1, policy_version 18590 (0.0009) +[2023-10-09 12:57:24,354][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000018592_19038208.pth... +[2023-10-09 12:57:24,384][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000016864_17268736.pth +[2023-10-09 12:57:26,020][86121] Updated weights for policy 0, policy_version 18500 (0.0008) +[2023-10-09 12:57:26,391][86121] Updated weights for policy 0, policy_version 18510 (0.0008) +[2023-10-09 12:57:26,759][86121] Updated weights for policy 0, policy_version 18520 (0.0007) +[2023-10-09 12:57:27,875][86122] Updated weights for policy 1, policy_version 18600 (0.0010) +[2023-10-09 12:57:28,230][86122] Updated weights for policy 1, policy_version 18610 (0.0009) +[2023-10-09 12:57:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38010880. Throughput: 0: 1810.6, 1: 1808.5. Samples: 9508952. Policy #0 lag: (min: 24.0, avg: 45.7, max: 56.0) +[2023-10-09 12:57:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:28,598][86122] Updated weights for policy 1, policy_version 18620 (0.0008) +[2023-10-09 12:57:30,529][86121] Updated weights for policy 0, policy_version 18530 (0.0008) +[2023-10-09 12:57:30,898][86121] Updated weights for policy 0, policy_version 18540 (0.0009) +[2023-10-09 12:57:31,262][86121] Updated weights for policy 0, policy_version 18550 (0.0007) +[2023-10-09 12:57:31,631][86121] Updated weights for policy 0, policy_version 18560 (0.0009) +[2023-10-09 12:57:32,404][86122] Updated weights for policy 1, policy_version 18630 (0.0007) +[2023-10-09 12:57:32,761][86122] Updated weights for policy 1, policy_version 18640 (0.0009) +[2023-10-09 12:57:33,129][86122] Updated weights for policy 1, policy_version 18650 (0.0008) +[2023-10-09 12:57:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 38109184. Throughput: 0: 1799.2, 1: 1815.4. Samples: 9530338. Policy #0 lag: (min: 24.0, avg: 45.7, max: 56.0) +[2023-10-09 12:57:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 12:57:35,402][86121] Updated weights for policy 0, policy_version 18570 (0.0008) +[2023-10-09 12:57:35,762][86121] Updated weights for policy 0, policy_version 18580 (0.0007) +[2023-10-09 12:57:36,130][86121] Updated weights for policy 0, policy_version 18590 (0.0007) +[2023-10-09 12:57:36,704][86122] Updated weights for policy 1, policy_version 18660 (0.0009) +[2023-10-09 12:57:37,061][86122] Updated weights for policy 1, policy_version 18670 (0.0007) +[2023-10-09 12:57:37,422][86122] Updated weights for policy 1, policy_version 18680 (0.0008) +[2023-10-09 12:57:38,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38174720. Throughput: 0: 1802.5, 1: 1816.8. Samples: 9551812. Policy #0 lag: (min: 24.0, avg: 45.7, max: 56.0) +[2023-10-09 12:57:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:39,830][86121] Updated weights for policy 0, policy_version 18600 (0.0009) +[2023-10-09 12:57:40,197][86121] Updated weights for policy 0, policy_version 18610 (0.0009) +[2023-10-09 12:57:40,561][86121] Updated weights for policy 0, policy_version 18620 (0.0007) +[2023-10-09 12:57:41,268][86122] Updated weights for policy 1, policy_version 18690 (0.0009) +[2023-10-09 12:57:41,650][86122] Updated weights for policy 1, policy_version 18700 (0.0010) +[2023-10-09 12:57:42,015][86122] Updated weights for policy 1, policy_version 18710 (0.0008) +[2023-10-09 12:57:42,379][86122] Updated weights for policy 1, policy_version 18720 (0.0008) +[2023-10-09 12:57:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 38240256. Throughput: 0: 1798.5, 1: 1814.1. Samples: 9562906. Policy #0 lag: (min: 24.0, avg: 45.7, max: 56.0) +[2023-10-09 12:57:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:44,213][86121] Updated weights for policy 0, policy_version 18630 (0.0007) +[2023-10-09 12:57:44,589][86121] Updated weights for policy 0, policy_version 18640 (0.0007) +[2023-10-09 12:57:44,966][86121] Updated weights for policy 0, policy_version 18650 (0.0009) +[2023-10-09 12:57:45,865][86122] Updated weights for policy 1, policy_version 18730 (0.0008) +[2023-10-09 12:57:46,233][86122] Updated weights for policy 1, policy_version 18740 (0.0007) +[2023-10-09 12:57:46,607][86122] Updated weights for policy 1, policy_version 18750 (0.0008) +[2023-10-09 12:57:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 38305792. Throughput: 0: 1801.9, 1: 1820.6. Samples: 9584648. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) +[2023-10-09 12:57:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:48,769][86121] Updated weights for policy 0, policy_version 18660 (0.0008) +[2023-10-09 12:57:49,151][86121] Updated weights for policy 0, policy_version 18670 (0.0007) +[2023-10-09 12:57:49,525][86121] Updated weights for policy 0, policy_version 18680 (0.0007) +[2023-10-09 12:57:50,197][86122] Updated weights for policy 1, policy_version 18760 (0.0009) +[2023-10-09 12:57:50,556][86122] Updated weights for policy 1, policy_version 18770 (0.0010) +[2023-10-09 12:57:50,918][86122] Updated weights for policy 1, policy_version 18780 (0.0008) +[2023-10-09 12:57:53,081][86121] Updated weights for policy 0, policy_version 18690 (0.0007) +[2023-10-09 12:57:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38371328. Throughput: 0: 1814.0, 1: 1818.7. Samples: 9607536. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) +[2023-10-09 12:57:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:53,458][86121] Updated weights for policy 0, policy_version 18700 (0.0009) +[2023-10-09 12:57:53,823][86121] Updated weights for policy 0, policy_version 18710 (0.0008) +[2023-10-09 12:57:54,196][86121] Updated weights for policy 0, policy_version 18720 (0.0009) +[2023-10-09 12:57:54,595][86122] Updated weights for policy 1, policy_version 18790 (0.0008) +[2023-10-09 12:57:54,964][86122] Updated weights for policy 1, policy_version 18800 (0.0007) +[2023-10-09 12:57:55,326][86122] Updated weights for policy 1, policy_version 18810 (0.0008) +[2023-10-09 12:57:57,925][86121] Updated weights for policy 0, policy_version 18730 (0.0008) +[2023-10-09 12:57:58,297][86121] Updated weights for policy 0, policy_version 18740 (0.0008) +[2023-10-09 12:57:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 38436864. Throughput: 0: 1806.3, 1: 1821.2. Samples: 9617490. Policy #0 lag: (min: 31.0, avg: 44.8, max: 63.0) +[2023-10-09 12:57:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:57:58,670][86121] Updated weights for policy 0, policy_version 18750 (0.0009) +[2023-10-09 12:57:59,057][86122] Updated weights for policy 1, policy_version 18820 (0.0010) +[2023-10-09 12:57:59,420][86122] Updated weights for policy 1, policy_version 18830 (0.0008) +[2023-10-09 12:57:59,779][86122] Updated weights for policy 1, policy_version 18840 (0.0007) +[2023-10-09 12:58:02,372][86121] Updated weights for policy 0, policy_version 18760 (0.0008) +[2023-10-09 12:58:02,734][86121] Updated weights for policy 0, policy_version 18770 (0.0009) +[2023-10-09 12:58:03,111][86121] Updated weights for policy 0, policy_version 18780 (0.0010) +[2023-10-09 12:58:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 38535168. Throughput: 0: 1805.6, 1: 1824.8. Samples: 9639908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:03,439][86122] Updated weights for policy 1, policy_version 18850 (0.0009) +[2023-10-09 12:58:03,802][86122] Updated weights for policy 1, policy_version 18860 (0.0007) +[2023-10-09 12:58:04,170][86122] Updated weights for policy 1, policy_version 18870 (0.0007) +[2023-10-09 12:58:04,531][86122] Updated weights for policy 1, policy_version 18880 (0.0007) +[2023-10-09 12:58:06,866][86121] Updated weights for policy 0, policy_version 18790 (0.0008) +[2023-10-09 12:58:07,233][86121] Updated weights for policy 0, policy_version 18800 (0.0010) +[2023-10-09 12:58:07,599][86121] Updated weights for policy 0, policy_version 18810 (0.0010) +[2023-10-09 12:58:08,314][86122] Updated weights for policy 1, policy_version 18890 (0.0007) +[2023-10-09 12:58:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38600704. Throughput: 0: 1801.1, 1: 1829.3. Samples: 9661086. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:08,684][86122] Updated weights for policy 1, policy_version 18900 (0.0007) +[2023-10-09 12:58:09,055][86122] Updated weights for policy 1, policy_version 18910 (0.0009) +[2023-10-09 12:58:11,366][86121] Updated weights for policy 0, policy_version 18820 (0.0009) +[2023-10-09 12:58:11,727][86121] Updated weights for policy 0, policy_version 18830 (0.0008) +[2023-10-09 12:58:12,094][86121] Updated weights for policy 0, policy_version 18840 (0.0007) +[2023-10-09 12:58:12,717][86122] Updated weights for policy 1, policy_version 18920 (0.0009) +[2023-10-09 12:58:13,071][86122] Updated weights for policy 1, policy_version 18930 (0.0010) +[2023-10-09 12:58:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38666240. Throughput: 0: 1804.6, 1: 1829.6. Samples: 9672492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:13,441][86122] Updated weights for policy 1, policy_version 18940 (0.0011) +[2023-10-09 12:58:15,833][86121] Updated weights for policy 0, policy_version 18850 (0.0009) +[2023-10-09 12:58:16,209][86121] Updated weights for policy 0, policy_version 18860 (0.0009) +[2023-10-09 12:58:16,579][86121] Updated weights for policy 0, policy_version 18870 (0.0008) +[2023-10-09 12:58:16,950][86121] Updated weights for policy 0, policy_version 18880 (0.0008) +[2023-10-09 12:58:17,047][86122] Updated weights for policy 1, policy_version 18950 (0.0009) +[2023-10-09 12:58:17,415][86122] Updated weights for policy 1, policy_version 18960 (0.0008) +[2023-10-09 12:58:17,786][86122] Updated weights for policy 1, policy_version 18970 (0.0009) +[2023-10-09 12:58:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38764544. Throughput: 0: 1810.1, 1: 1827.6. Samples: 9694032. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:20,602][86121] Updated weights for policy 0, policy_version 18890 (0.0008) +[2023-10-09 12:58:20,963][86121] Updated weights for policy 0, policy_version 18900 (0.0013) +[2023-10-09 12:58:21,332][86121] Updated weights for policy 0, policy_version 18910 (0.0010) +[2023-10-09 12:58:21,597][86122] Updated weights for policy 1, policy_version 18980 (0.0008) +[2023-10-09 12:58:21,961][86122] Updated weights for policy 1, policy_version 18990 (0.0009) +[2023-10-09 12:58:22,325][86122] Updated weights for policy 1, policy_version 19000 (0.0009) +[2023-10-09 12:58:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38830080. Throughput: 0: 1804.8, 1: 1821.7. Samples: 9715008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:25,159][86121] Updated weights for policy 0, policy_version 18920 (0.0009) +[2023-10-09 12:58:25,523][86121] Updated weights for policy 0, policy_version 18930 (0.0011) +[2023-10-09 12:58:25,887][86121] Updated weights for policy 0, policy_version 18940 (0.0010) +[2023-10-09 12:58:26,265][86122] Updated weights for policy 1, policy_version 19010 (0.0007) +[2023-10-09 12:58:26,659][86122] Updated weights for policy 1, policy_version 19020 (0.0008) +[2023-10-09 12:58:27,026][86122] Updated weights for policy 1, policy_version 19030 (0.0010) +[2023-10-09 12:58:27,387][86122] Updated weights for policy 1, policy_version 19040 (0.0011) +[2023-10-09 12:58:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 38895616. Throughput: 0: 1809.1, 1: 1820.7. Samples: 9726248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:29,543][86121] Updated weights for policy 0, policy_version 18950 (0.0009) +[2023-10-09 12:58:29,915][86121] Updated weights for policy 0, policy_version 18960 (0.0008) +[2023-10-09 12:58:30,283][86121] Updated weights for policy 0, policy_version 18970 (0.0008) +[2023-10-09 12:58:31,135][86122] Updated weights for policy 1, policy_version 19050 (0.0009) +[2023-10-09 12:58:31,492][86122] Updated weights for policy 1, policy_version 19060 (0.0008) +[2023-10-09 12:58:31,848][86122] Updated weights for policy 1, policy_version 19070 (0.0010) +[2023-10-09 12:58:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 38961152. Throughput: 0: 1797.2, 1: 1816.1. Samples: 9747244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:33,977][86121] Updated weights for policy 0, policy_version 18980 (0.0007) +[2023-10-09 12:58:34,371][86121] Updated weights for policy 0, policy_version 18990 (0.0007) +[2023-10-09 12:58:34,742][86121] Updated weights for policy 0, policy_version 19000 (0.0008) +[2023-10-09 12:58:35,522][86122] Updated weights for policy 1, policy_version 19080 (0.0011) +[2023-10-09 12:58:35,879][86122] Updated weights for policy 1, policy_version 19090 (0.0010) +[2023-10-09 12:58:36,257][86122] Updated weights for policy 1, policy_version 19100 (0.0011) +[2023-10-09 12:58:38,233][86121] Updated weights for policy 0, policy_version 19010 (0.0010) +[2023-10-09 12:58:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39026688. Throughput: 0: 1805.3, 1: 1808.0. Samples: 9770136. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:38,603][86121] Updated weights for policy 0, policy_version 19020 (0.0008) +[2023-10-09 12:58:38,964][86121] Updated weights for policy 0, policy_version 19030 (0.0009) +[2023-10-09 12:58:39,332][86121] Updated weights for policy 0, policy_version 19040 (0.0008) +[2023-10-09 12:58:39,931][86122] Updated weights for policy 1, policy_version 19110 (0.0009) +[2023-10-09 12:58:40,297][86122] Updated weights for policy 1, policy_version 19120 (0.0009) +[2023-10-09 12:58:40,663][86122] Updated weights for policy 1, policy_version 19130 (0.0010) +[2023-10-09 12:58:42,962][86121] Updated weights for policy 0, policy_version 19050 (0.0008) +[2023-10-09 12:58:43,330][86121] Updated weights for policy 0, policy_version 19060 (0.0009) +[2023-10-09 12:58:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 39092224. Throughput: 0: 1809.3, 1: 1813.2. Samples: 9780502. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) +[2023-10-09 12:58:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:43,693][86121] Updated weights for policy 0, policy_version 19070 (0.0010) +[2023-10-09 12:58:44,277][86122] Updated weights for policy 1, policy_version 19140 (0.0012) +[2023-10-09 12:58:44,641][86122] Updated weights for policy 1, policy_version 19150 (0.0010) +[2023-10-09 12:58:45,008][86122] Updated weights for policy 1, policy_version 19160 (0.0009) +[2023-10-09 12:58:47,376][86121] Updated weights for policy 0, policy_version 19080 (0.0007) +[2023-10-09 12:58:47,745][86121] Updated weights for policy 0, policy_version 19090 (0.0009) +[2023-10-09 12:58:48,116][86121] Updated weights for policy 0, policy_version 19100 (0.0009) +[2023-10-09 12:58:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39190528. Throughput: 0: 1816.7, 1: 1814.7. Samples: 9803320. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) +[2023-10-09 12:58:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:48,760][86122] Updated weights for policy 1, policy_version 19170 (0.0010) +[2023-10-09 12:58:49,123][86122] Updated weights for policy 1, policy_version 19180 (0.0009) +[2023-10-09 12:58:49,483][86122] Updated weights for policy 1, policy_version 19190 (0.0007) +[2023-10-09 12:58:49,849][86122] Updated weights for policy 1, policy_version 19200 (0.0009) +[2023-10-09 12:58:51,848][86121] Updated weights for policy 0, policy_version 19110 (0.0008) +[2023-10-09 12:58:52,217][86121] Updated weights for policy 0, policy_version 19120 (0.0008) +[2023-10-09 12:58:52,576][86121] Updated weights for policy 0, policy_version 19130 (0.0008) +[2023-10-09 12:58:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39256064. Throughput: 0: 1821.0, 1: 1813.6. Samples: 9824644. Policy #0 lag: (min: 31.0, avg: 33.1, max: 61.0) +[2023-10-09 12:58:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:53,537][86122] Updated weights for policy 1, policy_version 19210 (0.0008) +[2023-10-09 12:58:53,896][86122] Updated weights for policy 1, policy_version 19220 (0.0008) +[2023-10-09 12:58:54,267][86122] Updated weights for policy 1, policy_version 19230 (0.0008) +[2023-10-09 12:58:56,365][86121] Updated weights for policy 0, policy_version 19140 (0.0008) +[2023-10-09 12:58:56,740][86121] Updated weights for policy 0, policy_version 19150 (0.0007) +[2023-10-09 12:58:57,108][86121] Updated weights for policy 0, policy_version 19160 (0.0007) +[2023-10-09 12:58:57,987][86122] Updated weights for policy 1, policy_version 19240 (0.0008) +[2023-10-09 12:58:58,354][86122] Updated weights for policy 1, policy_version 19250 (0.0008) +[2023-10-09 12:58:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39321600. Throughput: 0: 1817.3, 1: 1814.7. Samples: 9835930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:58:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:58:58,722][86122] Updated weights for policy 1, policy_version 19260 (0.0009) +[2023-10-09 12:59:00,974][86121] Updated weights for policy 0, policy_version 19170 (0.0008) +[2023-10-09 12:59:01,344][86121] Updated weights for policy 0, policy_version 19180 (0.0008) +[2023-10-09 12:59:01,718][86121] Updated weights for policy 0, policy_version 19190 (0.0008) +[2023-10-09 12:59:02,077][86121] Updated weights for policy 0, policy_version 19200 (0.0009) +[2023-10-09 12:59:02,357][86122] Updated weights for policy 1, policy_version 19270 (0.0007) +[2023-10-09 12:59:02,721][86122] Updated weights for policy 1, policy_version 19280 (0.0008) +[2023-10-09 12:59:03,100][86122] Updated weights for policy 1, policy_version 19290 (0.0008) +[2023-10-09 12:59:03,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39419904. Throughput: 0: 1812.2, 1: 1815.2. Samples: 9857268. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:59:05,760][86121] Updated weights for policy 0, policy_version 19210 (0.0010) +[2023-10-09 12:59:06,128][86121] Updated weights for policy 0, policy_version 19220 (0.0008) +[2023-10-09 12:59:06,492][86121] Updated weights for policy 0, policy_version 19230 (0.0007) +[2023-10-09 12:59:06,824][86122] Updated weights for policy 1, policy_version 19300 (0.0009) +[2023-10-09 12:59:07,199][86122] Updated weights for policy 1, policy_version 19310 (0.0009) +[2023-10-09 12:59:07,559][86122] Updated weights for policy 1, policy_version 19320 (0.0009) +[2023-10-09 12:59:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39485440. Throughput: 0: 1819.9, 1: 1817.2. Samples: 9878674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:59:09,923][86121] Updated weights for policy 0, policy_version 19240 (0.0009) +[2023-10-09 12:59:10,291][86121] Updated weights for policy 0, policy_version 19250 (0.0009) +[2023-10-09 12:59:10,658][86121] Updated weights for policy 0, policy_version 19260 (0.0010) +[2023-10-09 12:59:11,308][86122] Updated weights for policy 1, policy_version 19330 (0.0008) +[2023-10-09 12:59:11,682][86122] Updated weights for policy 1, policy_version 19340 (0.0009) +[2023-10-09 12:59:12,038][86122] Updated weights for policy 1, policy_version 19350 (0.0007) +[2023-10-09 12:59:12,400][86122] Updated weights for policy 1, policy_version 19360 (0.0008) +[2023-10-09 12:59:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39550976. Throughput: 0: 1819.4, 1: 1819.4. Samples: 9889992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:59:14,467][86121] Updated weights for policy 0, policy_version 19270 (0.0009) +[2023-10-09 12:59:14,833][86121] Updated weights for policy 0, policy_version 19280 (0.0007) +[2023-10-09 12:59:15,213][86121] Updated weights for policy 0, policy_version 19290 (0.0009) +[2023-10-09 12:59:15,942][86122] Updated weights for policy 1, policy_version 19370 (0.0009) +[2023-10-09 12:59:16,305][86122] Updated weights for policy 1, policy_version 19380 (0.0008) +[2023-10-09 12:59:16,674][86122] Updated weights for policy 1, policy_version 19390 (0.0007) +[2023-10-09 12:59:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39616512. Throughput: 0: 1824.6, 1: 1821.0. Samples: 9911296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:59:18,991][86121] Updated weights for policy 0, policy_version 19300 (0.0009) +[2023-10-09 12:59:19,369][86121] Updated weights for policy 0, policy_version 19310 (0.0008) +[2023-10-09 12:59:19,741][86121] Updated weights for policy 0, policy_version 19320 (0.0008) +[2023-10-09 12:59:20,327][86122] Updated weights for policy 1, policy_version 19400 (0.0009) +[2023-10-09 12:59:20,686][86122] Updated weights for policy 1, policy_version 19410 (0.0007) +[2023-10-09 12:59:21,054][86122] Updated weights for policy 1, policy_version 19420 (0.0010) +[2023-10-09 12:59:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 39682048. Throughput: 0: 1815.6, 1: 1827.0. Samples: 9934050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 12:59:23,406][86121] Updated weights for policy 0, policy_version 19330 (0.0010) +[2023-10-09 12:59:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000019424_19890176.pth... +[2023-10-09 12:59:23,448][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000017728_18153472.pth +[2023-10-09 12:59:23,818][86121] Updated weights for policy 0, policy_version 19340 (0.0010) +[2023-10-09 12:59:24,174][86121] Updated weights for policy 0, policy_version 19350 (0.0010) +[2023-10-09 12:59:24,540][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000019360_19824640.pth... +[2023-10-09 12:59:24,540][86121] Updated weights for policy 0, policy_version 19360 (0.0011) +[2023-10-09 12:59:24,573][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000017632_18055168.pth +[2023-10-09 12:59:24,681][86122] Updated weights for policy 1, policy_version 19430 (0.0008) +[2023-10-09 12:59:25,037][86122] Updated weights for policy 1, policy_version 19440 (0.0008) +[2023-10-09 12:59:25,395][86122] Updated weights for policy 1, policy_version 19450 (0.0007) +[2023-10-09 12:59:28,269][86121] Updated weights for policy 0, policy_version 19370 (0.0008) +[2023-10-09 12:59:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 39747584. Throughput: 0: 1805.7, 1: 1822.9. Samples: 9943790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 12:59:28,645][86121] Updated weights for policy 0, policy_version 19380 (0.0008) +[2023-10-09 12:59:29,017][86121] Updated weights for policy 0, policy_version 19390 (0.0008) +[2023-10-09 12:59:29,197][86122] Updated weights for policy 1, policy_version 19460 (0.0009) +[2023-10-09 12:59:29,563][86122] Updated weights for policy 1, policy_version 19470 (0.0008) +[2023-10-09 12:59:29,926][86122] Updated weights for policy 1, policy_version 19480 (0.0008) +[2023-10-09 12:59:32,782][86121] Updated weights for policy 0, policy_version 19400 (0.0009) +[2023-10-09 12:59:33,157][86121] Updated weights for policy 0, policy_version 19410 (0.0010) +[2023-10-09 12:59:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 39813120. Throughput: 0: 1811.3, 1: 1820.7. Samples: 9966762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 12:59:33,483][86122] Updated weights for policy 1, policy_version 19490 (0.0009) +[2023-10-09 12:59:33,532][86121] Updated weights for policy 0, policy_version 19420 (0.0009) +[2023-10-09 12:59:33,850][86122] Updated weights for policy 1, policy_version 19500 (0.0008) +[2023-10-09 12:59:34,203][86122] Updated weights for policy 1, policy_version 19510 (0.0009) +[2023-10-09 12:59:34,562][86122] Updated weights for policy 1, policy_version 19520 (0.0010) +[2023-10-09 12:59:37,263][86121] Updated weights for policy 0, policy_version 19430 (0.0009) +[2023-10-09 12:59:37,632][86121] Updated weights for policy 0, policy_version 19440 (0.0007) +[2023-10-09 12:59:38,000][86121] Updated weights for policy 0, policy_version 19450 (0.0008) +[2023-10-09 12:59:38,197][86122] Updated weights for policy 1, policy_version 19530 (0.0007) +[2023-10-09 12:59:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39911424. Throughput: 0: 1821.5, 1: 1826.5. Samples: 9988804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 12:59:38,559][86122] Updated weights for policy 1, policy_version 19540 (0.0008) +[2023-10-09 12:59:38,912][86122] Updated weights for policy 1, policy_version 19550 (0.0008) +[2023-10-09 12:59:41,786][86121] Updated weights for policy 0, policy_version 19460 (0.0010) +[2023-10-09 12:59:42,151][86121] Updated weights for policy 0, policy_version 19470 (0.0007) +[2023-10-09 12:59:42,524][86121] Updated weights for policy 0, policy_version 19480 (0.0010) +[2023-10-09 12:59:42,713][86122] Updated weights for policy 1, policy_version 19560 (0.0007) +[2023-10-09 12:59:43,072][86122] Updated weights for policy 1, policy_version 19570 (0.0009) +[2023-10-09 12:59:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 39976960. Throughput: 0: 1811.6, 1: 1825.9. Samples: 9999616. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 12:59:43,424][86122] Updated weights for policy 1, policy_version 19580 (0.0008) +[2023-10-09 12:59:46,234][86121] Updated weights for policy 0, policy_version 19490 (0.0010) +[2023-10-09 12:59:46,592][86121] Updated weights for policy 0, policy_version 19500 (0.0009) +[2023-10-09 12:59:46,961][86121] Updated weights for policy 0, policy_version 19510 (0.0007) +[2023-10-09 12:59:47,156][86122] Updated weights for policy 1, policy_version 19590 (0.0008) +[2023-10-09 12:59:47,331][86121] Updated weights for policy 0, policy_version 19520 (0.0007) +[2023-10-09 12:59:47,525][86122] Updated weights for policy 1, policy_version 19600 (0.0010) +[2023-10-09 12:59:47,879][86122] Updated weights for policy 1, policy_version 19610 (0.0011) +[2023-10-09 12:59:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40075264. Throughput: 0: 1824.7, 1: 1824.1. Samples: 10021464. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:48,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 12:59:51,019][86121] Updated weights for policy 0, policy_version 19530 (0.0007) +[2023-10-09 12:59:51,386][86121] Updated weights for policy 0, policy_version 19540 (0.0008) +[2023-10-09 12:59:51,688][86122] Updated weights for policy 1, policy_version 19620 (0.0010) +[2023-10-09 12:59:51,745][86121] Updated weights for policy 0, policy_version 19550 (0.0008) +[2023-10-09 12:59:52,042][86122] Updated weights for policy 1, policy_version 19630 (0.0009) +[2023-10-09 12:59:52,405][86122] Updated weights for policy 1, policy_version 19640 (0.0010) +[2023-10-09 12:59:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 40140800. Throughput: 0: 1812.5, 1: 1819.8. Samples: 10042128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 12:59:55,396][86121] Updated weights for policy 0, policy_version 19560 (0.0008) +[2023-10-09 12:59:55,758][86121] Updated weights for policy 0, policy_version 19570 (0.0009) +[2023-10-09 12:59:56,124][86121] Updated weights for policy 0, policy_version 19580 (0.0009) +[2023-10-09 12:59:56,329][86122] Updated weights for policy 1, policy_version 19650 (0.0010) +[2023-10-09 12:59:56,718][86122] Updated weights for policy 1, policy_version 19660 (0.0008) +[2023-10-09 12:59:57,094][86122] Updated weights for policy 1, policy_version 19670 (0.0011) +[2023-10-09 12:59:57,445][86122] Updated weights for policy 1, policy_version 19680 (0.0010) +[2023-10-09 12:59:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 40206336. Throughput: 0: 1818.9, 1: 1817.1. Samples: 10053614. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 12:59:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 12:59:59,837][86121] Updated weights for policy 0, policy_version 19590 (0.0009) +[2023-10-09 13:00:00,204][86121] Updated weights for policy 0, policy_version 19600 (0.0009) +[2023-10-09 13:00:00,567][86121] Updated weights for policy 0, policy_version 19610 (0.0010) +[2023-10-09 13:00:01,117][86122] Updated weights for policy 1, policy_version 19690 (0.0010) +[2023-10-09 13:00:01,485][86122] Updated weights for policy 1, policy_version 19700 (0.0007) +[2023-10-09 13:00:01,839][86122] Updated weights for policy 1, policy_version 19710 (0.0009) +[2023-10-09 13:00:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40271872. Throughput: 0: 1809.2, 1: 1822.7. Samples: 10074732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:00:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:04,265][86121] Updated weights for policy 0, policy_version 19620 (0.0008) +[2023-10-09 13:00:04,628][86121] Updated weights for policy 0, policy_version 19630 (0.0008) +[2023-10-09 13:00:05,003][86121] Updated weights for policy 0, policy_version 19640 (0.0010) +[2023-10-09 13:00:05,509][86122] Updated weights for policy 1, policy_version 19720 (0.0008) +[2023-10-09 13:00:05,875][86122] Updated weights for policy 1, policy_version 19730 (0.0008) +[2023-10-09 13:00:06,241][86122] Updated weights for policy 1, policy_version 19740 (0.0008) +[2023-10-09 13:00:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40337408. Throughput: 0: 1811.0, 1: 1821.4. Samples: 10097506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:00:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:08,796][86121] Updated weights for policy 0, policy_version 19650 (0.0008) +[2023-10-09 13:00:09,194][86121] Updated weights for policy 0, policy_version 19660 (0.0007) +[2023-10-09 13:00:09,557][86121] Updated weights for policy 0, policy_version 19670 (0.0007) +[2023-10-09 13:00:09,899][86122] Updated weights for policy 1, policy_version 19750 (0.0008) +[2023-10-09 13:00:09,929][86121] Updated weights for policy 0, policy_version 19680 (0.0008) +[2023-10-09 13:00:10,264][86122] Updated weights for policy 1, policy_version 19760 (0.0008) +[2023-10-09 13:00:10,629][86122] Updated weights for policy 1, policy_version 19770 (0.0008) +[2023-10-09 13:00:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40402944. Throughput: 0: 1813.3, 1: 1822.8. Samples: 10107412. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:00:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:13,419][86121] Updated weights for policy 0, policy_version 19690 (0.0008) +[2023-10-09 13:00:13,791][86121] Updated weights for policy 0, policy_version 19700 (0.0009) +[2023-10-09 13:00:14,146][86121] Updated weights for policy 0, policy_version 19710 (0.0009) +[2023-10-09 13:00:14,291][86122] Updated weights for policy 1, policy_version 19780 (0.0009) +[2023-10-09 13:00:14,657][86122] Updated weights for policy 1, policy_version 19790 (0.0009) +[2023-10-09 13:00:15,012][86122] Updated weights for policy 1, policy_version 19800 (0.0008) +[2023-10-09 13:00:17,931][86121] Updated weights for policy 0, policy_version 19720 (0.0009) +[2023-10-09 13:00:18,294][86121] Updated weights for policy 0, policy_version 19730 (0.0010) +[2023-10-09 13:00:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 40468480. Throughput: 0: 1810.8, 1: 1824.0. Samples: 10130330. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:00:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:18,529][86122] Updated weights for policy 1, policy_version 19810 (0.0008) +[2023-10-09 13:00:18,667][86121] Updated weights for policy 0, policy_version 19740 (0.0009) +[2023-10-09 13:00:18,890][86122] Updated weights for policy 1, policy_version 19820 (0.0008) +[2023-10-09 13:00:19,260][86122] Updated weights for policy 1, policy_version 19830 (0.0008) +[2023-10-09 13:00:19,625][86122] Updated weights for policy 1, policy_version 19840 (0.0009) +[2023-10-09 13:00:22,439][86121] Updated weights for policy 0, policy_version 19750 (0.0008) +[2023-10-09 13:00:22,821][86121] Updated weights for policy 0, policy_version 19760 (0.0009) +[2023-10-09 13:00:23,186][86121] Updated weights for policy 0, policy_version 19770 (0.0008) +[2023-10-09 13:00:23,221][86122] Updated weights for policy 1, policy_version 19850 (0.0007) +[2023-10-09 13:00:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 40534016. Throughput: 0: 1812.0, 1: 1823.4. Samples: 10152400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:00:23,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:23,582][86122] Updated weights for policy 1, policy_version 19860 (0.0008) +[2023-10-09 13:00:23,943][86122] Updated weights for policy 1, policy_version 19870 (0.0010) +[2023-10-09 13:00:27,010][86121] Updated weights for policy 0, policy_version 19780 (0.0008) +[2023-10-09 13:00:27,379][86121] Updated weights for policy 0, policy_version 19790 (0.0008) +[2023-10-09 13:00:27,484][86122] Updated weights for policy 1, policy_version 19880 (0.0007) +[2023-10-09 13:00:27,750][86121] Updated weights for policy 0, policy_version 19800 (0.0007) +[2023-10-09 13:00:27,848][86122] Updated weights for policy 1, policy_version 19890 (0.0009) +[2023-10-09 13:00:28,202][86122] Updated weights for policy 1, policy_version 19900 (0.0011) +[2023-10-09 13:00:28,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 40665088. Throughput: 0: 1805.7, 1: 1826.4. Samples: 10163058. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:00:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:31,404][86121] Updated weights for policy 0, policy_version 19810 (0.0008) +[2023-10-09 13:00:31,764][86121] Updated weights for policy 0, policy_version 19820 (0.0008) +[2023-10-09 13:00:31,987][86122] Updated weights for policy 1, policy_version 19910 (0.0009) +[2023-10-09 13:00:32,132][86121] Updated weights for policy 0, policy_version 19830 (0.0008) +[2023-10-09 13:00:32,346][86122] Updated weights for policy 1, policy_version 19920 (0.0008) +[2023-10-09 13:00:32,498][86121] Updated weights for policy 0, policy_version 19840 (0.0008) +[2023-10-09 13:00:32,717][86122] Updated weights for policy 1, policy_version 19930 (0.0008) +[2023-10-09 13:00:33,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 40730624. Throughput: 0: 1807.8, 1: 1822.5. Samples: 10184828. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:00:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:36,169][86121] Updated weights for policy 0, policy_version 19850 (0.0009) +[2023-10-09 13:00:36,363][86122] Updated weights for policy 1, policy_version 19940 (0.0008) +[2023-10-09 13:00:36,522][86121] Updated weights for policy 0, policy_version 19860 (0.0007) +[2023-10-09 13:00:36,733][86122] Updated weights for policy 1, policy_version 19950 (0.0008) +[2023-10-09 13:00:36,891][86121] Updated weights for policy 0, policy_version 19870 (0.0008) +[2023-10-09 13:00:37,103][86122] Updated weights for policy 1, policy_version 19960 (0.0009) +[2023-10-09 13:00:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 40796160. Throughput: 0: 1801.2, 1: 1832.4. Samples: 10205640. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:00:38,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:40,536][86121] Updated weights for policy 0, policy_version 19880 (0.0008) +[2023-10-09 13:00:40,908][86121] Updated weights for policy 0, policy_version 19890 (0.0009) +[2023-10-09 13:00:41,033][86122] Updated weights for policy 1, policy_version 19970 (0.0009) +[2023-10-09 13:00:41,267][86121] Updated weights for policy 0, policy_version 19900 (0.0010) +[2023-10-09 13:00:41,428][86122] Updated weights for policy 1, policy_version 19980 (0.0008) +[2023-10-09 13:00:41,795][86122] Updated weights for policy 1, policy_version 19990 (0.0009) +[2023-10-09 13:00:42,168][86122] Updated weights for policy 1, policy_version 20000 (0.0009) +[2023-10-09 13:00:43,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 40861696. Throughput: 0: 1808.4, 1: 1836.6. Samples: 10217638. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:00:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:45,129][86121] Updated weights for policy 0, policy_version 19910 (0.0007) +[2023-10-09 13:00:45,506][86121] Updated weights for policy 0, policy_version 19920 (0.0008) +[2023-10-09 13:00:45,669][86122] Updated weights for policy 1, policy_version 20010 (0.0009) +[2023-10-09 13:00:45,865][86121] Updated weights for policy 0, policy_version 19930 (0.0008) +[2023-10-09 13:00:46,029][86122] Updated weights for policy 1, policy_version 20020 (0.0008) +[2023-10-09 13:00:46,392][86122] Updated weights for policy 1, policy_version 20030 (0.0009) +[2023-10-09 13:00:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40927232. Throughput: 0: 1799.9, 1: 1830.8. Samples: 10238114. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:00:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:49,880][86121] Updated weights for policy 0, policy_version 19940 (0.0009) +[2023-10-09 13:00:50,185][86122] Updated weights for policy 1, policy_version 20040 (0.0007) +[2023-10-09 13:00:50,244][86121] Updated weights for policy 0, policy_version 19950 (0.0007) +[2023-10-09 13:00:50,555][86122] Updated weights for policy 1, policy_version 20050 (0.0009) +[2023-10-09 13:00:50,612][86121] Updated weights for policy 0, policy_version 19960 (0.0007) +[2023-10-09 13:00:50,921][86122] Updated weights for policy 1, policy_version 20060 (0.0009) +[2023-10-09 13:00:53,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 40992768. Throughput: 0: 1794.3, 1: 1829.4. Samples: 10260572. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:00:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:54,267][86121] Updated weights for policy 0, policy_version 19970 (0.0009) +[2023-10-09 13:00:54,531][86122] Updated weights for policy 1, policy_version 20070 (0.0008) +[2023-10-09 13:00:54,683][86121] Updated weights for policy 0, policy_version 19980 (0.0009) +[2023-10-09 13:00:54,897][86122] Updated weights for policy 1, policy_version 20080 (0.0007) +[2023-10-09 13:00:55,047][86121] Updated weights for policy 0, policy_version 19990 (0.0007) +[2023-10-09 13:00:55,264][86122] Updated weights for policy 1, policy_version 20090 (0.0008) +[2023-10-09 13:00:55,409][86121] Updated weights for policy 0, policy_version 20000 (0.0008) +[2023-10-09 13:00:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41058304. Throughput: 0: 1792.5, 1: 1826.8. Samples: 10270282. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:00:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:00:59,086][86121] Updated weights for policy 0, policy_version 20010 (0.0009) +[2023-10-09 13:00:59,111][86122] Updated weights for policy 1, policy_version 20100 (0.0008) +[2023-10-09 13:00:59,458][86121] Updated weights for policy 0, policy_version 20020 (0.0007) +[2023-10-09 13:00:59,480][86122] Updated weights for policy 1, policy_version 20110 (0.0009) +[2023-10-09 13:00:59,820][86121] Updated weights for policy 0, policy_version 20030 (0.0007) +[2023-10-09 13:00:59,834][86122] Updated weights for policy 1, policy_version 20120 (0.0007) +[2023-10-09 13:01:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41123840. Throughput: 0: 1789.8, 1: 1823.5. Samples: 10292932. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:01:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:01:03,519][86122] Updated weights for policy 1, policy_version 20130 (0.0009) +[2023-10-09 13:01:03,684][86121] Updated weights for policy 0, policy_version 20040 (0.0007) +[2023-10-09 13:01:03,884][86122] Updated weights for policy 1, policy_version 20140 (0.0008) +[2023-10-09 13:01:04,058][86121] Updated weights for policy 0, policy_version 20050 (0.0008) +[2023-10-09 13:01:04,244][86122] Updated weights for policy 1, policy_version 20150 (0.0008) +[2023-10-09 13:01:04,428][86121] Updated weights for policy 0, policy_version 20060 (0.0007) +[2023-10-09 13:01:04,608][86122] Updated weights for policy 1, policy_version 20160 (0.0007) +[2023-10-09 13:01:08,079][86121] Updated weights for policy 0, policy_version 20070 (0.0009) +[2023-10-09 13:01:08,259][86122] Updated weights for policy 1, policy_version 20170 (0.0007) +[2023-10-09 13:01:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41189376. Throughput: 0: 1812.1, 1: 1820.0. Samples: 10315844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:01:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:01:08,454][86121] Updated weights for policy 0, policy_version 20080 (0.0008) +[2023-10-09 13:01:08,627][86122] Updated weights for policy 1, policy_version 20180 (0.0008) +[2023-10-09 13:01:08,807][86121] Updated weights for policy 0, policy_version 20090 (0.0008) +[2023-10-09 13:01:08,980][86122] Updated weights for policy 1, policy_version 20190 (0.0010) +[2023-10-09 13:01:12,639][86121] Updated weights for policy 0, policy_version 20100 (0.0008) +[2023-10-09 13:01:12,769][86122] Updated weights for policy 1, policy_version 20200 (0.0009) +[2023-10-09 13:01:13,008][86121] Updated weights for policy 0, policy_version 20110 (0.0007) +[2023-10-09 13:01:13,122][86122] Updated weights for policy 1, policy_version 20210 (0.0007) +[2023-10-09 13:01:13,375][86121] Updated weights for policy 0, policy_version 20120 (0.0007) +[2023-10-09 13:01:13,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41254912. Throughput: 0: 1795.1, 1: 1816.5. Samples: 10325580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:01:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:01:13,495][86122] Updated weights for policy 1, policy_version 20220 (0.0007) +[2023-10-09 13:01:17,122][86121] Updated weights for policy 0, policy_version 20130 (0.0009) +[2023-10-09 13:01:17,244][86122] Updated weights for policy 1, policy_version 20230 (0.0008) +[2023-10-09 13:01:17,487][86121] Updated weights for policy 0, policy_version 20140 (0.0007) +[2023-10-09 13:01:17,612][86122] Updated weights for policy 1, policy_version 20240 (0.0009) +[2023-10-09 13:01:17,844][86121] Updated weights for policy 0, policy_version 20150 (0.0007) +[2023-10-09 13:01:17,966][86122] Updated weights for policy 1, policy_version 20250 (0.0008) +[2023-10-09 13:01:18,206][86121] Updated weights for policy 0, policy_version 20160 (0.0008) +[2023-10-09 13:01:18,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 41385984. Throughput: 0: 1814.3, 1: 1816.4. Samples: 10348208. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:01:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:01:21,739][86122] Updated weights for policy 1, policy_version 20260 (0.0010) +[2023-10-09 13:01:21,902][86121] Updated weights for policy 0, policy_version 20170 (0.0008) +[2023-10-09 13:01:22,109][86122] Updated weights for policy 1, policy_version 20270 (0.0009) +[2023-10-09 13:01:22,266][86121] Updated weights for policy 0, policy_version 20180 (0.0008) +[2023-10-09 13:01:22,466][86122] Updated weights for policy 1, policy_version 20280 (0.0008) +[2023-10-09 13:01:22,637][86121] Updated weights for policy 0, policy_version 20190 (0.0008) +[2023-10-09 13:01:23,397][85186] Fps is (10 sec: 19660.1, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 41451520. Throughput: 0: 1794.5, 1: 1810.4. Samples: 10367862. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 13:01:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:01:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000020288_20774912.pth... +[2023-10-09 13:01:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000020192_20676608.pth... +[2023-10-09 13:01:23,441][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000018496_18939904.pth +[2023-10-09 13:01:23,447][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000018592_19038208.pth +[2023-10-09 13:01:26,235][86122] Updated weights for policy 1, policy_version 20290 (0.0008) +[2023-10-09 13:01:26,465][86121] Updated weights for policy 0, policy_version 20200 (0.0007) +[2023-10-09 13:01:26,640][86122] Updated weights for policy 1, policy_version 20300 (0.0008) +[2023-10-09 13:01:26,824][86121] Updated weights for policy 0, policy_version 20210 (0.0007) +[2023-10-09 13:01:26,998][86122] Updated weights for policy 1, policy_version 20310 (0.0008) +[2023-10-09 13:01:27,198][86121] Updated weights for policy 0, policy_version 20220 (0.0007) +[2023-10-09 13:01:27,366][86122] Updated weights for policy 1, policy_version 20320 (0.0009) +[2023-10-09 13:01:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41517056. Throughput: 0: 1806.8, 1: 1811.3. Samples: 10380452. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 13:01:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:01:31,018][86121] Updated weights for policy 0, policy_version 20230 (0.0007) +[2023-10-09 13:01:31,181][86122] Updated weights for policy 1, policy_version 20330 (0.0007) +[2023-10-09 13:01:31,386][86121] Updated weights for policy 0, policy_version 20240 (0.0008) +[2023-10-09 13:01:31,544][86122] Updated weights for policy 1, policy_version 20340 (0.0007) +[2023-10-09 13:01:31,759][86121] Updated weights for policy 0, policy_version 20250 (0.0007) +[2023-10-09 13:01:31,897][86122] Updated weights for policy 1, policy_version 20350 (0.0007) +[2023-10-09 13:01:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41582592. Throughput: 0: 1794.5, 1: 1807.9. Samples: 10400224. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 13:01:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:01:35,539][86121] Updated weights for policy 0, policy_version 20260 (0.0008) +[2023-10-09 13:01:35,659][86122] Updated weights for policy 1, policy_version 20360 (0.0009) +[2023-10-09 13:01:35,912][86121] Updated weights for policy 0, policy_version 20270 (0.0008) +[2023-10-09 13:01:36,016][86122] Updated weights for policy 1, policy_version 20370 (0.0008) +[2023-10-09 13:01:36,276][86121] Updated weights for policy 0, policy_version 20280 (0.0007) +[2023-10-09 13:01:36,376][86122] Updated weights for policy 1, policy_version 20380 (0.0008) +[2023-10-09 13:01:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 41648128. Throughput: 0: 1789.6, 1: 1801.4. Samples: 10422166. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 13:01:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:01:39,998][86121] Updated weights for policy 0, policy_version 20290 (0.0007) +[2023-10-09 13:01:40,149][86122] Updated weights for policy 1, policy_version 20390 (0.0007) +[2023-10-09 13:01:40,412][86121] Updated weights for policy 0, policy_version 20300 (0.0010) +[2023-10-09 13:01:40,506][86122] Updated weights for policy 1, policy_version 20400 (0.0007) +[2023-10-09 13:01:40,780][86121] Updated weights for policy 0, policy_version 20310 (0.0010) +[2023-10-09 13:01:40,856][86122] Updated weights for policy 1, policy_version 20410 (0.0008) +[2023-10-09 13:01:41,144][86121] Updated weights for policy 0, policy_version 20320 (0.0009) +[2023-10-09 13:01:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 41713664. Throughput: 0: 1800.4, 1: 1806.0. Samples: 10432574. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) +[2023-10-09 13:01:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:01:44,572][86122] Updated weights for policy 1, policy_version 20420 (0.0008) +[2023-10-09 13:01:44,913][86121] Updated weights for policy 0, policy_version 20330 (0.0009) +[2023-10-09 13:01:44,944][86122] Updated weights for policy 1, policy_version 20430 (0.0008) +[2023-10-09 13:01:45,271][86121] Updated weights for policy 0, policy_version 20340 (0.0008) +[2023-10-09 13:01:45,308][86122] Updated weights for policy 1, policy_version 20440 (0.0008) +[2023-10-09 13:01:45,640][86121] Updated weights for policy 0, policy_version 20350 (0.0009) +[2023-10-09 13:01:48,398][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 41779200. Throughput: 0: 1785.5, 1: 1805.4. Samples: 10454524. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) +[2023-10-09 13:01:48,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:01:48,888][86122] Updated weights for policy 1, policy_version 20450 (0.0008) +[2023-10-09 13:01:49,252][86122] Updated weights for policy 1, policy_version 20460 (0.0009) +[2023-10-09 13:01:49,387][86121] Updated weights for policy 0, policy_version 20360 (0.0008) +[2023-10-09 13:01:49,613][86122] Updated weights for policy 1, policy_version 20470 (0.0009) +[2023-10-09 13:01:49,756][86121] Updated weights for policy 0, policy_version 20370 (0.0009) +[2023-10-09 13:01:49,973][86122] Updated weights for policy 1, policy_version 20480 (0.0010) +[2023-10-09 13:01:50,120][86121] Updated weights for policy 0, policy_version 20380 (0.0008) +[2023-10-09 13:01:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41844736. Throughput: 0: 1779.5, 1: 1800.4. Samples: 10476942. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) +[2023-10-09 13:01:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:01:53,911][86122] Updated weights for policy 1, policy_version 20490 (0.0009) +[2023-10-09 13:01:53,921][86121] Updated weights for policy 0, policy_version 20390 (0.0008) +[2023-10-09 13:01:54,272][86122] Updated weights for policy 1, policy_version 20500 (0.0008) +[2023-10-09 13:01:54,284][86121] Updated weights for policy 0, policy_version 20400 (0.0009) +[2023-10-09 13:01:54,635][86122] Updated weights for policy 1, policy_version 20510 (0.0009) +[2023-10-09 13:01:54,657][86121] Updated weights for policy 0, policy_version 20410 (0.0007) +[2023-10-09 13:01:58,293][86121] Updated weights for policy 0, policy_version 20420 (0.0009) +[2023-10-09 13:01:58,314][86122] Updated weights for policy 1, policy_version 20520 (0.0008) +[2023-10-09 13:01:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 41910272. Throughput: 0: 1779.9, 1: 1800.2. Samples: 10486684. Policy #0 lag: (min: 31.0, avg: 46.1, max: 63.0) +[2023-10-09 13:01:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:01:58,655][86121] Updated weights for policy 0, policy_version 20430 (0.0009) +[2023-10-09 13:01:58,680][86122] Updated weights for policy 1, policy_version 20530 (0.0009) +[2023-10-09 13:01:59,028][86121] Updated weights for policy 0, policy_version 20440 (0.0009) +[2023-10-09 13:01:59,032][86122] Updated weights for policy 1, policy_version 20540 (0.0007) +[2023-10-09 13:02:02,645][86122] Updated weights for policy 1, policy_version 20550 (0.0007) +[2023-10-09 13:02:02,767][86121] Updated weights for policy 0, policy_version 20450 (0.0008) +[2023-10-09 13:02:03,002][86122] Updated weights for policy 1, policy_version 20560 (0.0009) +[2023-10-09 13:02:03,134][86121] Updated weights for policy 0, policy_version 20460 (0.0008) +[2023-10-09 13:02:03,354][86122] Updated weights for policy 1, policy_version 20570 (0.0009) +[2023-10-09 13:02:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 41975808. Throughput: 0: 1777.5, 1: 1802.6. Samples: 10509312. Policy #0 lag: (min: 0.0, avg: 24.9, max: 32.0) +[2023-10-09 13:02:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:03,496][86121] Updated weights for policy 0, policy_version 20470 (0.0008) +[2023-10-09 13:02:03,865][86121] Updated weights for policy 0, policy_version 20480 (0.0010) +[2023-10-09 13:02:07,090][86122] Updated weights for policy 1, policy_version 20580 (0.0008) +[2023-10-09 13:02:07,458][86122] Updated weights for policy 1, policy_version 20590 (0.0007) +[2023-10-09 13:02:07,655][86121] Updated weights for policy 0, policy_version 20490 (0.0007) +[2023-10-09 13:02:07,814][86122] Updated weights for policy 1, policy_version 20600 (0.0007) +[2023-10-09 13:02:08,015][86121] Updated weights for policy 0, policy_version 20500 (0.0007) +[2023-10-09 13:02:08,379][86121] Updated weights for policy 0, policy_version 20510 (0.0007) +[2023-10-09 13:02:08,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 42074112. Throughput: 0: 1795.8, 1: 1814.5. Samples: 10530326. Policy #0 lag: (min: 0.0, avg: 24.9, max: 32.0) +[2023-10-09 13:02:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:11,618][86122] Updated weights for policy 1, policy_version 20610 (0.0009) +[2023-10-09 13:02:12,023][86122] Updated weights for policy 1, policy_version 20620 (0.0009) +[2023-10-09 13:02:12,089][86121] Updated weights for policy 0, policy_version 20520 (0.0008) +[2023-10-09 13:02:12,392][86122] Updated weights for policy 1, policy_version 20630 (0.0009) +[2023-10-09 13:02:12,458][86121] Updated weights for policy 0, policy_version 20530 (0.0008) +[2023-10-09 13:02:12,748][86122] Updated weights for policy 1, policy_version 20640 (0.0009) +[2023-10-09 13:02:12,829][86121] Updated weights for policy 0, policy_version 20540 (0.0008) +[2023-10-09 13:02:13,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 42172416. Throughput: 0: 1786.4, 1: 1800.4. Samples: 10541862. Policy #0 lag: (min: 0.0, avg: 24.9, max: 32.0) +[2023-10-09 13:02:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:16,436][86122] Updated weights for policy 1, policy_version 20650 (0.0007) +[2023-10-09 13:02:16,534][86121] Updated weights for policy 0, policy_version 20550 (0.0008) +[2023-10-09 13:02:16,805][86122] Updated weights for policy 1, policy_version 20660 (0.0008) +[2023-10-09 13:02:16,898][86121] Updated weights for policy 0, policy_version 20560 (0.0008) +[2023-10-09 13:02:17,172][86122] Updated weights for policy 1, policy_version 20670 (0.0009) +[2023-10-09 13:02:17,266][86121] Updated weights for policy 0, policy_version 20570 (0.0008) +[2023-10-09 13:02:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42237952. Throughput: 0: 1800.1, 1: 1814.2. Samples: 10562868. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:02:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:20,803][86122] Updated weights for policy 1, policy_version 20680 (0.0009) +[2023-10-09 13:02:20,989][86121] Updated weights for policy 0, policy_version 20580 (0.0010) +[2023-10-09 13:02:21,160][86122] Updated weights for policy 1, policy_version 20690 (0.0007) +[2023-10-09 13:02:21,357][86121] Updated weights for policy 0, policy_version 20590 (0.0008) +[2023-10-09 13:02:21,517][86122] Updated weights for policy 1, policy_version 20700 (0.0008) +[2023-10-09 13:02:21,713][86121] Updated weights for policy 0, policy_version 20600 (0.0009) +[2023-10-09 13:02:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 42303488. Throughput: 0: 1792.6, 1: 1813.9. Samples: 10584458. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:02:23,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:25,147][86122] Updated weights for policy 1, policy_version 20710 (0.0008) +[2023-10-09 13:02:25,474][86121] Updated weights for policy 0, policy_version 20610 (0.0008) +[2023-10-09 13:02:25,517][86122] Updated weights for policy 1, policy_version 20720 (0.0010) +[2023-10-09 13:02:25,873][86121] Updated weights for policy 0, policy_version 20620 (0.0008) +[2023-10-09 13:02:25,885][86122] Updated weights for policy 1, policy_version 20730 (0.0009) +[2023-10-09 13:02:26,235][86121] Updated weights for policy 0, policy_version 20630 (0.0009) +[2023-10-09 13:02:26,601][86121] Updated weights for policy 0, policy_version 20640 (0.0007) +[2023-10-09 13:02:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42369024. Throughput: 0: 1800.6, 1: 1817.8. Samples: 10595402. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:02:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:29,633][86122] Updated weights for policy 1, policy_version 20740 (0.0009) +[2023-10-09 13:02:30,009][86122] Updated weights for policy 1, policy_version 20750 (0.0008) +[2023-10-09 13:02:30,362][86122] Updated weights for policy 1, policy_version 20760 (0.0007) +[2023-10-09 13:02:30,391][86121] Updated weights for policy 0, policy_version 20650 (0.0008) +[2023-10-09 13:02:30,747][86121] Updated weights for policy 0, policy_version 20660 (0.0007) +[2023-10-09 13:02:31,119][86121] Updated weights for policy 0, policy_version 20670 (0.0008) +[2023-10-09 13:02:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42434560. Throughput: 0: 1791.7, 1: 1818.9. Samples: 10617002. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:02:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:33,927][86122] Updated weights for policy 1, policy_version 20770 (0.0008) +[2023-10-09 13:02:34,304][86122] Updated weights for policy 1, policy_version 20780 (0.0009) +[2023-10-09 13:02:34,663][86122] Updated weights for policy 1, policy_version 20790 (0.0007) +[2023-10-09 13:02:34,855][86121] Updated weights for policy 0, policy_version 20680 (0.0008) +[2023-10-09 13:02:35,030][86122] Updated weights for policy 1, policy_version 20800 (0.0008) +[2023-10-09 13:02:35,223][86121] Updated weights for policy 0, policy_version 20690 (0.0008) +[2023-10-09 13:02:35,594][86121] Updated weights for policy 0, policy_version 20700 (0.0008) +[2023-10-09 13:02:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42500096. Throughput: 0: 1798.1, 1: 1815.1. Samples: 10639538. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) +[2023-10-09 13:02:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:38,899][86122] Updated weights for policy 1, policy_version 20810 (0.0009) +[2023-10-09 13:02:39,265][86122] Updated weights for policy 1, policy_version 20820 (0.0008) +[2023-10-09 13:02:39,309][86121] Updated weights for policy 0, policy_version 20710 (0.0009) +[2023-10-09 13:02:39,631][86122] Updated weights for policy 1, policy_version 20830 (0.0007) +[2023-10-09 13:02:39,677][86121] Updated weights for policy 0, policy_version 20720 (0.0007) +[2023-10-09 13:02:40,057][86121] Updated weights for policy 0, policy_version 20730 (0.0010) +[2023-10-09 13:02:43,323][86122] Updated weights for policy 1, policy_version 20840 (0.0009) +[2023-10-09 13:02:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42565632. Throughput: 0: 1796.2, 1: 1817.2. Samples: 10649286. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) +[2023-10-09 13:02:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:43,691][86122] Updated weights for policy 1, policy_version 20850 (0.0009) +[2023-10-09 13:02:43,797][86121] Updated weights for policy 0, policy_version 20740 (0.0009) +[2023-10-09 13:02:44,062][86122] Updated weights for policy 1, policy_version 20860 (0.0009) +[2023-10-09 13:02:44,166][86121] Updated weights for policy 0, policy_version 20750 (0.0009) +[2023-10-09 13:02:44,527][86121] Updated weights for policy 0, policy_version 20760 (0.0007) +[2023-10-09 13:02:47,785][86122] Updated weights for policy 1, policy_version 20870 (0.0008) +[2023-10-09 13:02:48,148][86122] Updated weights for policy 1, policy_version 20880 (0.0008) +[2023-10-09 13:02:48,368][86121] Updated weights for policy 0, policy_version 20770 (0.0008) +[2023-10-09 13:02:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 42631168. Throughput: 0: 1799.6, 1: 1811.2. Samples: 10671800. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) +[2023-10-09 13:02:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:48,516][86122] Updated weights for policy 1, policy_version 20890 (0.0008) +[2023-10-09 13:02:48,736][86121] Updated weights for policy 0, policy_version 20780 (0.0008) +[2023-10-09 13:02:49,096][86121] Updated weights for policy 0, policy_version 20790 (0.0009) +[2023-10-09 13:02:49,458][86121] Updated weights for policy 0, policy_version 20800 (0.0009) +[2023-10-09 13:02:52,218][86122] Updated weights for policy 1, policy_version 20900 (0.0008) +[2023-10-09 13:02:52,579][86122] Updated weights for policy 1, policy_version 20910 (0.0008) +[2023-10-09 13:02:52,943][86122] Updated weights for policy 1, policy_version 20920 (0.0009) +[2023-10-09 13:02:53,191][86121] Updated weights for policy 0, policy_version 20810 (0.0009) +[2023-10-09 13:02:53,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42729472. Throughput: 0: 1810.9, 1: 1814.0. Samples: 10693448. Policy #0 lag: (min: 19.0, avg: 23.4, max: 51.0) +[2023-10-09 13:02:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:02:53,555][86121] Updated weights for policy 0, policy_version 20820 (0.0008) +[2023-10-09 13:02:53,926][86121] Updated weights for policy 0, policy_version 20830 (0.0008) +[2023-10-09 13:02:56,823][86122] Updated weights for policy 1, policy_version 20930 (0.0008) +[2023-10-09 13:02:57,230][86122] Updated weights for policy 1, policy_version 20940 (0.0008) +[2023-10-09 13:02:57,542][86121] Updated weights for policy 0, policy_version 20840 (0.0007) +[2023-10-09 13:02:57,589][86122] Updated weights for policy 1, policy_version 20950 (0.0007) +[2023-10-09 13:02:57,904][86121] Updated weights for policy 0, policy_version 20850 (0.0008) +[2023-10-09 13:02:57,953][86122] Updated weights for policy 1, policy_version 20960 (0.0007) +[2023-10-09 13:02:58,274][86121] Updated weights for policy 0, policy_version 20860 (0.0010) +[2023-10-09 13:02:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 42795008. Throughput: 0: 1797.1, 1: 1812.6. Samples: 10704298. Policy #0 lag: (min: 20.0, avg: 22.9, max: 52.0) +[2023-10-09 13:02:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:03:01,659][86122] Updated weights for policy 1, policy_version 20970 (0.0007) +[2023-10-09 13:03:02,037][86122] Updated weights for policy 1, policy_version 20980 (0.0007) +[2023-10-09 13:03:02,202][86121] Updated weights for policy 0, policy_version 20870 (0.0009) +[2023-10-09 13:03:02,396][86122] Updated weights for policy 1, policy_version 20990 (0.0008) +[2023-10-09 13:03:02,560][86121] Updated weights for policy 0, policy_version 20880 (0.0007) +[2023-10-09 13:03:02,927][86121] Updated weights for policy 0, policy_version 20890 (0.0008) +[2023-10-09 13:03:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 42893312. Throughput: 0: 1811.7, 1: 1814.5. Samples: 10726048. Policy #0 lag: (min: 20.0, avg: 22.9, max: 52.0) +[2023-10-09 13:03:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:03:06,139][86122] Updated weights for policy 1, policy_version 21000 (0.0010) +[2023-10-09 13:03:06,496][86122] Updated weights for policy 1, policy_version 21010 (0.0009) +[2023-10-09 13:03:06,542][86121] Updated weights for policy 0, policy_version 20900 (0.0007) +[2023-10-09 13:03:06,863][86122] Updated weights for policy 1, policy_version 21020 (0.0007) +[2023-10-09 13:03:06,909][86121] Updated weights for policy 0, policy_version 20910 (0.0008) +[2023-10-09 13:03:07,275][86121] Updated weights for policy 0, policy_version 20920 (0.0009) +[2023-10-09 13:03:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 42958848. Throughput: 0: 1797.2, 1: 1805.8. Samples: 10746592. Policy #0 lag: (min: 20.0, avg: 22.9, max: 52.0) +[2023-10-09 13:03:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:03:10,633][86122] Updated weights for policy 1, policy_version 21030 (0.0007) +[2023-10-09 13:03:10,872][86121] Updated weights for policy 0, policy_version 20930 (0.0007) +[2023-10-09 13:03:10,983][86122] Updated weights for policy 1, policy_version 21040 (0.0007) +[2023-10-09 13:03:11,290][86121] Updated weights for policy 0, policy_version 20940 (0.0009) +[2023-10-09 13:03:11,341][86122] Updated weights for policy 1, policy_version 21050 (0.0007) +[2023-10-09 13:03:11,661][86121] Updated weights for policy 0, policy_version 20950 (0.0009) +[2023-10-09 13:03:12,029][86121] Updated weights for policy 0, policy_version 20960 (0.0010) +[2023-10-09 13:03:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43024384. Throughput: 0: 1814.5, 1: 1816.0. Samples: 10758774. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:15,137][86122] Updated weights for policy 1, policy_version 21060 (0.0010) +[2023-10-09 13:03:15,504][86122] Updated weights for policy 1, policy_version 21070 (0.0010) +[2023-10-09 13:03:15,713][86121] Updated weights for policy 0, policy_version 20970 (0.0008) +[2023-10-09 13:03:15,859][86122] Updated weights for policy 1, policy_version 21080 (0.0007) +[2023-10-09 13:03:16,086][86121] Updated weights for policy 0, policy_version 20980 (0.0008) +[2023-10-09 13:03:16,446][86121] Updated weights for policy 0, policy_version 20990 (0.0008) +[2023-10-09 13:03:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 43089920. Throughput: 0: 1802.1, 1: 1795.7. Samples: 10778900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:03:19,423][86122] Updated weights for policy 1, policy_version 21090 (0.0008) +[2023-10-09 13:03:19,784][86122] Updated weights for policy 1, policy_version 21100 (0.0011) +[2023-10-09 13:03:20,148][86122] Updated weights for policy 1, policy_version 21110 (0.0008) +[2023-10-09 13:03:20,376][86121] Updated weights for policy 0, policy_version 21000 (0.0010) +[2023-10-09 13:03:20,509][86122] Updated weights for policy 1, policy_version 21120 (0.0011) +[2023-10-09 13:03:20,743][86121] Updated weights for policy 0, policy_version 21010 (0.0011) +[2023-10-09 13:03:21,113][86121] Updated weights for policy 0, policy_version 21020 (0.0007) +[2023-10-09 13:03:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43155456. Throughput: 0: 1794.8, 1: 1804.6. Samples: 10801512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:03:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000021024_21528576.pth... +[2023-10-09 13:03:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000021120_21626880.pth... +[2023-10-09 13:03:23,443][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000019424_19890176.pth +[2023-10-09 13:03:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000019360_19824640.pth +[2023-10-09 13:03:24,200][86122] Updated weights for policy 1, policy_version 21130 (0.0008) +[2023-10-09 13:03:24,561][86122] Updated weights for policy 1, policy_version 21140 (0.0010) +[2023-10-09 13:03:24,919][86122] Updated weights for policy 1, policy_version 21150 (0.0007) +[2023-10-09 13:03:24,950][86121] Updated weights for policy 0, policy_version 21030 (0.0008) +[2023-10-09 13:03:25,313][86121] Updated weights for policy 0, policy_version 21040 (0.0008) +[2023-10-09 13:03:25,681][86121] Updated weights for policy 0, policy_version 21050 (0.0010) +[2023-10-09 13:03:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43220992. Throughput: 0: 1797.3, 1: 1805.5. Samples: 10811410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:28,610][86122] Updated weights for policy 1, policy_version 21160 (0.0009) +[2023-10-09 13:03:28,974][86122] Updated weights for policy 1, policy_version 21170 (0.0008) +[2023-10-09 13:03:29,338][86122] Updated weights for policy 1, policy_version 21180 (0.0009) +[2023-10-09 13:03:29,545][86121] Updated weights for policy 0, policy_version 21060 (0.0009) +[2023-10-09 13:03:29,910][86121] Updated weights for policy 0, policy_version 21070 (0.0009) +[2023-10-09 13:03:30,283][86121] Updated weights for policy 0, policy_version 21080 (0.0011) +[2023-10-09 13:03:33,006][86122] Updated weights for policy 1, policy_version 21190 (0.0008) +[2023-10-09 13:03:33,378][86122] Updated weights for policy 1, policy_version 21200 (0.0008) +[2023-10-09 13:03:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43286528. Throughput: 0: 1786.5, 1: 1815.9. Samples: 10833908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:33,746][86122] Updated weights for policy 1, policy_version 21210 (0.0008) +[2023-10-09 13:03:33,961][86121] Updated weights for policy 0, policy_version 21090 (0.0007) +[2023-10-09 13:03:34,335][86121] Updated weights for policy 0, policy_version 21100 (0.0008) +[2023-10-09 13:03:34,700][86121] Updated weights for policy 0, policy_version 21110 (0.0008) +[2023-10-09 13:03:35,073][86121] Updated weights for policy 0, policy_version 21120 (0.0008) +[2023-10-09 13:03:37,554][86122] Updated weights for policy 1, policy_version 21220 (0.0009) +[2023-10-09 13:03:37,919][86122] Updated weights for policy 1, policy_version 21230 (0.0009) +[2023-10-09 13:03:38,279][86122] Updated weights for policy 1, policy_version 21240 (0.0007) +[2023-10-09 13:03:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 43352064. Throughput: 0: 1801.0, 1: 1817.1. Samples: 10856262. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:38,634][86121] Updated weights for policy 0, policy_version 21130 (0.0008) +[2023-10-09 13:03:39,007][86121] Updated weights for policy 0, policy_version 21140 (0.0009) +[2023-10-09 13:03:39,374][86121] Updated weights for policy 0, policy_version 21150 (0.0009) +[2023-10-09 13:03:42,022][86122] Updated weights for policy 1, policy_version 21250 (0.0009) +[2023-10-09 13:03:42,418][86122] Updated weights for policy 1, policy_version 21260 (0.0010) +[2023-10-09 13:03:42,780][86122] Updated weights for policy 1, policy_version 21270 (0.0010) +[2023-10-09 13:03:43,036][86121] Updated weights for policy 0, policy_version 21160 (0.0008) +[2023-10-09 13:03:43,149][86122] Updated weights for policy 1, policy_version 21280 (0.0007) +[2023-10-09 13:03:43,394][86121] Updated weights for policy 0, policy_version 21170 (0.0008) +[2023-10-09 13:03:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 43450368. Throughput: 0: 1796.7, 1: 1810.6. Samples: 10866628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:43,768][86121] Updated weights for policy 0, policy_version 21180 (0.0007) +[2023-10-09 13:03:46,870][86122] Updated weights for policy 1, policy_version 21290 (0.0007) +[2023-10-09 13:03:47,221][86122] Updated weights for policy 1, policy_version 21300 (0.0007) +[2023-10-09 13:03:47,503][86121] Updated weights for policy 0, policy_version 21190 (0.0008) +[2023-10-09 13:03:47,581][86122] Updated weights for policy 1, policy_version 21310 (0.0008) +[2023-10-09 13:03:47,867][86121] Updated weights for policy 0, policy_version 21200 (0.0008) +[2023-10-09 13:03:48,235][86121] Updated weights for policy 0, policy_version 21210 (0.0009) +[2023-10-09 13:03:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 43515904. Throughput: 0: 1798.0, 1: 1819.0. Samples: 10888812. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:03:48,399][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:51,109][86122] Updated weights for policy 1, policy_version 21320 (0.0009) +[2023-10-09 13:03:51,472][86122] Updated weights for policy 1, policy_version 21330 (0.0010) +[2023-10-09 13:03:51,815][86121] Updated weights for policy 0, policy_version 21220 (0.0009) +[2023-10-09 13:03:51,846][86122] Updated weights for policy 1, policy_version 21340 (0.0008) +[2023-10-09 13:03:52,177][86121] Updated weights for policy 0, policy_version 21230 (0.0008) +[2023-10-09 13:03:52,543][86121] Updated weights for policy 0, policy_version 21240 (0.0007) +[2023-10-09 13:03:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 43614208. Throughput: 0: 1798.7, 1: 1816.7. Samples: 10909286. Policy #0 lag: (min: 11.0, avg: 25.3, max: 43.0) +[2023-10-09 13:03:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:55,467][86122] Updated weights for policy 1, policy_version 21350 (0.0009) +[2023-10-09 13:03:55,827][86122] Updated weights for policy 1, policy_version 21360 (0.0008) +[2023-10-09 13:03:56,193][86122] Updated weights for policy 1, policy_version 21370 (0.0009) +[2023-10-09 13:03:56,201][86121] Updated weights for policy 0, policy_version 21250 (0.0007) +[2023-10-09 13:03:56,600][86121] Updated weights for policy 0, policy_version 21260 (0.0010) +[2023-10-09 13:03:56,966][86121] Updated weights for policy 0, policy_version 21270 (0.0008) +[2023-10-09 13:03:57,330][86121] Updated weights for policy 0, policy_version 21280 (0.0010) +[2023-10-09 13:03:58,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 43679744. Throughput: 0: 1803.3, 1: 1817.0. Samples: 10921686. Policy #0 lag: (min: 11.0, avg: 25.3, max: 43.0) +[2023-10-09 13:03:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:03:59,961][86122] Updated weights for policy 1, policy_version 21380 (0.0010) +[2023-10-09 13:04:00,317][86122] Updated weights for policy 1, policy_version 21390 (0.0010) +[2023-10-09 13:04:00,680][86122] Updated weights for policy 1, policy_version 21400 (0.0010) +[2023-10-09 13:04:01,190][86121] Updated weights for policy 0, policy_version 21290 (0.0009) +[2023-10-09 13:04:01,554][86121] Updated weights for policy 0, policy_version 21300 (0.0008) +[2023-10-09 13:04:01,918][86121] Updated weights for policy 0, policy_version 21310 (0.0009) +[2023-10-09 13:04:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 43745280. Throughput: 0: 1801.5, 1: 1827.3. Samples: 10942198. Policy #0 lag: (min: 11.0, avg: 25.3, max: 43.0) +[2023-10-09 13:04:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:04:04,149][86122] Updated weights for policy 1, policy_version 21410 (0.0008) +[2023-10-09 13:04:04,514][86122] Updated weights for policy 1, policy_version 21420 (0.0007) +[2023-10-09 13:04:04,870][86122] Updated weights for policy 1, policy_version 21430 (0.0007) +[2023-10-09 13:04:05,235][86122] Updated weights for policy 1, policy_version 21440 (0.0008) +[2023-10-09 13:04:05,640][86121] Updated weights for policy 0, policy_version 21320 (0.0011) +[2023-10-09 13:04:06,011][86121] Updated weights for policy 0, policy_version 21330 (0.0011) +[2023-10-09 13:04:06,383][86121] Updated weights for policy 0, policy_version 21340 (0.0007) +[2023-10-09 13:04:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43810816. Throughput: 0: 1803.6, 1: 1827.4. Samples: 10964904. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 13:04:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:04:08,943][86122] Updated weights for policy 1, policy_version 21450 (0.0010) +[2023-10-09 13:04:09,309][86122] Updated weights for policy 1, policy_version 21460 (0.0009) +[2023-10-09 13:04:09,680][86122] Updated weights for policy 1, policy_version 21470 (0.0009) +[2023-10-09 13:04:10,114][86121] Updated weights for policy 0, policy_version 21350 (0.0009) +[2023-10-09 13:04:10,466][86121] Updated weights for policy 0, policy_version 21360 (0.0011) +[2023-10-09 13:04:10,833][86121] Updated weights for policy 0, policy_version 21370 (0.0010) +[2023-10-09 13:04:13,290][86122] Updated weights for policy 1, policy_version 21480 (0.0007) +[2023-10-09 13:04:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43876352. Throughput: 0: 1815.3, 1: 1823.1. Samples: 10975136. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 13:04:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:04:13,654][86122] Updated weights for policy 1, policy_version 21490 (0.0008) +[2023-10-09 13:04:14,015][86122] Updated weights for policy 1, policy_version 21500 (0.0008) +[2023-10-09 13:04:14,450][86121] Updated weights for policy 0, policy_version 21380 (0.0007) +[2023-10-09 13:04:14,822][86121] Updated weights for policy 0, policy_version 21390 (0.0007) +[2023-10-09 13:04:15,182][86121] Updated weights for policy 0, policy_version 21400 (0.0007) +[2023-10-09 13:04:17,724][86122] Updated weights for policy 1, policy_version 21510 (0.0009) +[2023-10-09 13:04:18,087][86122] Updated weights for policy 1, policy_version 21520 (0.0011) +[2023-10-09 13:04:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 43941888. Throughput: 0: 1822.0, 1: 1823.0. Samples: 10997934. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 13:04:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:04:18,447][86122] Updated weights for policy 1, policy_version 21530 (0.0010) +[2023-10-09 13:04:18,815][86121] Updated weights for policy 0, policy_version 21410 (0.0009) +[2023-10-09 13:04:19,177][86121] Updated weights for policy 0, policy_version 21420 (0.0008) +[2023-10-09 13:04:19,543][86121] Updated weights for policy 0, policy_version 21430 (0.0007) +[2023-10-09 13:04:19,905][86121] Updated weights for policy 0, policy_version 21440 (0.0007) +[2023-10-09 13:04:22,274][86122] Updated weights for policy 1, policy_version 21540 (0.0010) +[2023-10-09 13:04:22,644][86122] Updated weights for policy 1, policy_version 21550 (0.0010) +[2023-10-09 13:04:23,009][86122] Updated weights for policy 1, policy_version 21560 (0.0010) +[2023-10-09 13:04:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 44040192. Throughput: 0: 1812.2, 1: 1820.8. Samples: 11019748. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 13:04:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:04:23,696][86121] Updated weights for policy 0, policy_version 21450 (0.0008) +[2023-10-09 13:04:24,066][86121] Updated weights for policy 0, policy_version 21460 (0.0008) +[2023-10-09 13:04:24,437][86121] Updated weights for policy 0, policy_version 21470 (0.0010) +[2023-10-09 13:04:26,887][86122] Updated weights for policy 1, policy_version 21570 (0.0009) +[2023-10-09 13:04:27,291][86122] Updated weights for policy 1, policy_version 21580 (0.0007) +[2023-10-09 13:04:27,654][86122] Updated weights for policy 1, policy_version 21590 (0.0008) +[2023-10-09 13:04:28,013][86122] Updated weights for policy 1, policy_version 21600 (0.0009) +[2023-10-09 13:04:28,088][86121] Updated weights for policy 0, policy_version 21480 (0.0008) +[2023-10-09 13:04:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44105728. Throughput: 0: 1816.4, 1: 1826.1. Samples: 11030538. Policy #0 lag: (min: 17.0, avg: 27.8, max: 49.0) +[2023-10-09 13:04:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:04:28,459][86121] Updated weights for policy 0, policy_version 21490 (0.0008) +[2023-10-09 13:04:28,826][86121] Updated weights for policy 0, policy_version 21500 (0.0007) +[2023-10-09 13:04:31,928][86122] Updated weights for policy 1, policy_version 21610 (0.0008) +[2023-10-09 13:04:32,298][86122] Updated weights for policy 1, policy_version 21620 (0.0007) +[2023-10-09 13:04:32,655][86121] Updated weights for policy 0, policy_version 21510 (0.0008) +[2023-10-09 13:04:32,658][86122] Updated weights for policy 1, policy_version 21630 (0.0009) +[2023-10-09 13:04:33,024][86121] Updated weights for policy 0, policy_version 21520 (0.0007) +[2023-10-09 13:04:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44171264. Throughput: 0: 1818.5, 1: 1824.2. Samples: 11052730. Policy #0 lag: (min: 17.0, avg: 27.8, max: 49.0) +[2023-10-09 13:04:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:04:33,401][86121] Updated weights for policy 0, policy_version 21530 (0.0007) +[2023-10-09 13:04:36,281][86122] Updated weights for policy 1, policy_version 21640 (0.0009) +[2023-10-09 13:04:36,659][86122] Updated weights for policy 1, policy_version 21650 (0.0010) +[2023-10-09 13:04:37,029][86122] Updated weights for policy 1, policy_version 21660 (0.0009) +[2023-10-09 13:04:37,175][86121] Updated weights for policy 0, policy_version 21540 (0.0010) +[2023-10-09 13:04:37,547][86121] Updated weights for policy 0, policy_version 21550 (0.0008) +[2023-10-09 13:04:37,911][86121] Updated weights for policy 0, policy_version 21560 (0.0008) +[2023-10-09 13:04:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 44269568. Throughput: 0: 1824.0, 1: 1821.9. Samples: 11073354. Policy #0 lag: (min: 17.0, avg: 27.8, max: 49.0) +[2023-10-09 13:04:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:04:40,615][86122] Updated weights for policy 1, policy_version 21670 (0.0009) +[2023-10-09 13:04:40,973][86122] Updated weights for policy 1, policy_version 21680 (0.0009) +[2023-10-09 13:04:41,335][86122] Updated weights for policy 1, policy_version 21690 (0.0010) +[2023-10-09 13:04:41,745][86121] Updated weights for policy 0, policy_version 21570 (0.0013) +[2023-10-09 13:04:42,111][86121] Updated weights for policy 0, policy_version 21580 (0.0009) +[2023-10-09 13:04:42,476][86121] Updated weights for policy 0, policy_version 21590 (0.0007) +[2023-10-09 13:04:42,838][86121] Updated weights for policy 0, policy_version 21600 (0.0007) +[2023-10-09 13:04:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44335104. Throughput: 0: 1811.0, 1: 1823.1. Samples: 11085220. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:04:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:04:45,123][86122] Updated weights for policy 1, policy_version 21700 (0.0011) +[2023-10-09 13:04:45,488][86122] Updated weights for policy 1, policy_version 21710 (0.0011) +[2023-10-09 13:04:45,865][86122] Updated weights for policy 1, policy_version 21720 (0.0011) +[2023-10-09 13:04:46,403][86121] Updated weights for policy 0, policy_version 21610 (0.0009) +[2023-10-09 13:04:46,769][86121] Updated weights for policy 0, policy_version 21620 (0.0009) +[2023-10-09 13:04:47,133][86121] Updated weights for policy 0, policy_version 21630 (0.0008) +[2023-10-09 13:04:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44400640. Throughput: 0: 1828.9, 1: 1812.4. Samples: 11106058. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:04:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:04:49,409][86122] Updated weights for policy 1, policy_version 21730 (0.0008) +[2023-10-09 13:04:49,775][86122] Updated weights for policy 1, policy_version 21740 (0.0009) +[2023-10-09 13:04:50,142][86122] Updated weights for policy 1, policy_version 21750 (0.0008) +[2023-10-09 13:04:50,499][86122] Updated weights for policy 1, policy_version 21760 (0.0008) +[2023-10-09 13:04:50,769][86121] Updated weights for policy 0, policy_version 21640 (0.0011) +[2023-10-09 13:04:51,140][86121] Updated weights for policy 0, policy_version 21650 (0.0011) +[2023-10-09 13:04:51,508][86121] Updated weights for policy 0, policy_version 21660 (0.0010) +[2023-10-09 13:04:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44466176. Throughput: 0: 1821.5, 1: 1809.4. Samples: 11128294. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:04:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:04:54,224][86122] Updated weights for policy 1, policy_version 21770 (0.0011) +[2023-10-09 13:04:54,573][86122] Updated weights for policy 1, policy_version 21780 (0.0010) +[2023-10-09 13:04:54,941][86122] Updated weights for policy 1, policy_version 21790 (0.0011) +[2023-10-09 13:04:55,274][86121] Updated weights for policy 0, policy_version 21670 (0.0008) +[2023-10-09 13:04:55,642][86121] Updated weights for policy 0, policy_version 21680 (0.0009) +[2023-10-09 13:04:56,022][86121] Updated weights for policy 0, policy_version 21690 (0.0009) +[2023-10-09 13:04:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44531712. Throughput: 0: 1825.3, 1: 1807.6. Samples: 11138616. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:04:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:04:58,729][86122] Updated weights for policy 1, policy_version 21800 (0.0009) +[2023-10-09 13:04:59,091][86122] Updated weights for policy 1, policy_version 21810 (0.0009) +[2023-10-09 13:04:59,454][86122] Updated weights for policy 1, policy_version 21820 (0.0010) +[2023-10-09 13:04:59,719][86121] Updated weights for policy 0, policy_version 21700 (0.0009) +[2023-10-09 13:05:00,086][86121] Updated weights for policy 0, policy_version 21710 (0.0007) +[2023-10-09 13:05:00,452][86121] Updated weights for policy 0, policy_version 21720 (0.0007) +[2023-10-09 13:05:03,080][86122] Updated weights for policy 1, policy_version 21830 (0.0009) +[2023-10-09 13:05:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 44597248. Throughput: 0: 1817.3, 1: 1806.3. Samples: 11160998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:05:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:05:03,439][86122] Updated weights for policy 1, policy_version 21840 (0.0007) +[2023-10-09 13:05:03,802][86122] Updated weights for policy 1, policy_version 21850 (0.0009) +[2023-10-09 13:05:04,064][86121] Updated weights for policy 0, policy_version 21730 (0.0008) +[2023-10-09 13:05:04,433][86121] Updated weights for policy 0, policy_version 21740 (0.0009) +[2023-10-09 13:05:04,794][86121] Updated weights for policy 0, policy_version 21750 (0.0008) +[2023-10-09 13:05:05,147][86121] Updated weights for policy 0, policy_version 21760 (0.0007) +[2023-10-09 13:05:07,615][86122] Updated weights for policy 1, policy_version 21860 (0.0008) +[2023-10-09 13:05:07,974][86122] Updated weights for policy 1, policy_version 21870 (0.0010) +[2023-10-09 13:05:08,330][86122] Updated weights for policy 1, policy_version 21880 (0.0008) +[2023-10-09 13:05:08,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 44662784. Throughput: 0: 1820.6, 1: 1814.1. Samples: 11183310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:05:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:05:08,839][86121] Updated weights for policy 0, policy_version 21770 (0.0009) +[2023-10-09 13:05:09,208][86121] Updated weights for policy 0, policy_version 21780 (0.0008) +[2023-10-09 13:05:09,577][86121] Updated weights for policy 0, policy_version 21790 (0.0007) +[2023-10-09 13:05:12,273][86122] Updated weights for policy 1, policy_version 21890 (0.0008) +[2023-10-09 13:05:12,666][86122] Updated weights for policy 1, policy_version 21900 (0.0008) +[2023-10-09 13:05:13,032][86122] Updated weights for policy 1, policy_version 21910 (0.0011) +[2023-10-09 13:05:13,264][86121] Updated weights for policy 0, policy_version 21800 (0.0008) +[2023-10-09 13:05:13,383][86122] Updated weights for policy 1, policy_version 21920 (0.0008) +[2023-10-09 13:05:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44761088. Throughput: 0: 1816.7, 1: 1804.9. Samples: 11193514. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:05:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:05:13,623][86121] Updated weights for policy 0, policy_version 21810 (0.0007) +[2023-10-09 13:05:13,985][86121] Updated weights for policy 0, policy_version 21820 (0.0008) +[2023-10-09 13:05:17,033][86122] Updated weights for policy 1, policy_version 21930 (0.0008) +[2023-10-09 13:05:17,393][86122] Updated weights for policy 1, policy_version 21940 (0.0008) +[2023-10-09 13:05:17,635][86121] Updated weights for policy 0, policy_version 21830 (0.0007) +[2023-10-09 13:05:17,743][86122] Updated weights for policy 1, policy_version 21950 (0.0008) +[2023-10-09 13:05:18,004][86121] Updated weights for policy 0, policy_version 21840 (0.0008) +[2023-10-09 13:05:18,379][86121] Updated weights for policy 0, policy_version 21850 (0.0007) +[2023-10-09 13:05:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 44826624. Throughput: 0: 1818.9, 1: 1807.6. Samples: 11215920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:05:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:05:21,506][86122] Updated weights for policy 1, policy_version 21960 (0.0010) +[2023-10-09 13:05:21,867][86122] Updated weights for policy 1, policy_version 21970 (0.0008) +[2023-10-09 13:05:22,225][86122] Updated weights for policy 1, policy_version 21980 (0.0010) +[2023-10-09 13:05:22,261][86121] Updated weights for policy 0, policy_version 21860 (0.0008) +[2023-10-09 13:05:22,620][86121] Updated weights for policy 0, policy_version 21870 (0.0008) +[2023-10-09 13:05:22,988][86121] Updated weights for policy 0, policy_version 21880 (0.0008) +[2023-10-09 13:05:23,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14440.1). Total num frames: 44924928. Throughput: 0: 1816.0, 1: 1797.5. Samples: 11235958. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 13:05:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000021984_22511616.pth... +[2023-10-09 13:05:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000021888_22413312.pth... +[2023-10-09 13:05:23,437][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000020288_20774912.pth +[2023-10-09 13:05:23,444][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000020192_20676608.pth +[2023-10-09 13:05:25,947][86122] Updated weights for policy 1, policy_version 21990 (0.0008) +[2023-10-09 13:05:26,307][86122] Updated weights for policy 1, policy_version 22000 (0.0008) +[2023-10-09 13:05:26,674][86122] Updated weights for policy 1, policy_version 22010 (0.0007) +[2023-10-09 13:05:26,709][86121] Updated weights for policy 0, policy_version 21890 (0.0008) +[2023-10-09 13:05:27,123][86121] Updated weights for policy 0, policy_version 21900 (0.0007) +[2023-10-09 13:05:27,484][86121] Updated weights for policy 0, policy_version 21910 (0.0009) +[2023-10-09 13:05:27,847][86121] Updated weights for policy 0, policy_version 21920 (0.0009) +[2023-10-09 13:05:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 44990464. Throughput: 0: 1816.2, 1: 1807.1. Samples: 11248266. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 13:05:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:30,342][86122] Updated weights for policy 1, policy_version 22020 (0.0008) +[2023-10-09 13:05:30,698][86122] Updated weights for policy 1, policy_version 22030 (0.0009) +[2023-10-09 13:05:31,061][86122] Updated weights for policy 1, policy_version 22040 (0.0010) +[2023-10-09 13:05:31,584][86121] Updated weights for policy 0, policy_version 21930 (0.0009) +[2023-10-09 13:05:31,941][86121] Updated weights for policy 0, policy_version 21940 (0.0010) +[2023-10-09 13:05:32,312][86121] Updated weights for policy 0, policy_version 21950 (0.0008) +[2023-10-09 13:05:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 45056000. Throughput: 0: 1813.8, 1: 1800.9. Samples: 11268720. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 13:05:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:34,805][86122] Updated weights for policy 1, policy_version 22050 (0.0008) +[2023-10-09 13:05:35,175][86122] Updated weights for policy 1, policy_version 22060 (0.0009) +[2023-10-09 13:05:35,541][86122] Updated weights for policy 1, policy_version 22070 (0.0009) +[2023-10-09 13:05:35,899][86122] Updated weights for policy 1, policy_version 22080 (0.0008) +[2023-10-09 13:05:35,979][86121] Updated weights for policy 0, policy_version 21960 (0.0008) +[2023-10-09 13:05:36,343][86121] Updated weights for policy 0, policy_version 21970 (0.0010) +[2023-10-09 13:05:36,716][86121] Updated weights for policy 0, policy_version 21980 (0.0009) +[2023-10-09 13:05:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45121536. Throughput: 0: 1813.0, 1: 1807.3. Samples: 11291208. Policy #0 lag: (min: 18.0, avg: 20.0, max: 48.0) +[2023-10-09 13:05:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:39,492][86122] Updated weights for policy 1, policy_version 22090 (0.0007) +[2023-10-09 13:05:39,843][86122] Updated weights for policy 1, policy_version 22100 (0.0007) +[2023-10-09 13:05:40,201][86122] Updated weights for policy 1, policy_version 22110 (0.0007) +[2023-10-09 13:05:40,307][86121] Updated weights for policy 0, policy_version 21990 (0.0009) +[2023-10-09 13:05:40,676][86121] Updated weights for policy 0, policy_version 22000 (0.0009) +[2023-10-09 13:05:41,037][86121] Updated weights for policy 0, policy_version 22010 (0.0009) +[2023-10-09 13:05:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45187072. Throughput: 0: 1812.5, 1: 1812.8. Samples: 11301756. Policy #0 lag: (min: 18.0, avg: 20.0, max: 48.0) +[2023-10-09 13:05:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:44,063][86122] Updated weights for policy 1, policy_version 22120 (0.0009) +[2023-10-09 13:05:44,430][86122] Updated weights for policy 1, policy_version 22130 (0.0010) +[2023-10-09 13:05:44,647][86121] Updated weights for policy 0, policy_version 22020 (0.0010) +[2023-10-09 13:05:44,787][86122] Updated weights for policy 1, policy_version 22140 (0.0008) +[2023-10-09 13:05:45,017][86121] Updated weights for policy 0, policy_version 22030 (0.0008) +[2023-10-09 13:05:45,384][86121] Updated weights for policy 0, policy_version 22040 (0.0008) +[2023-10-09 13:05:48,356][86122] Updated weights for policy 1, policy_version 22150 (0.0008) +[2023-10-09 13:05:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45252608. Throughput: 0: 1815.3, 1: 1805.3. Samples: 11323926. Policy #0 lag: (min: 18.0, avg: 20.0, max: 48.0) +[2023-10-09 13:05:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:48,718][86122] Updated weights for policy 1, policy_version 22160 (0.0007) +[2023-10-09 13:05:49,033][86121] Updated weights for policy 0, policy_version 22050 (0.0007) +[2023-10-09 13:05:49,075][86122] Updated weights for policy 1, policy_version 22170 (0.0008) +[2023-10-09 13:05:49,394][86121] Updated weights for policy 0, policy_version 22060 (0.0007) +[2023-10-09 13:05:49,771][86121] Updated weights for policy 0, policy_version 22070 (0.0008) +[2023-10-09 13:05:50,134][86121] Updated weights for policy 0, policy_version 22080 (0.0008) +[2023-10-09 13:05:52,823][86122] Updated weights for policy 1, policy_version 22180 (0.0011) +[2023-10-09 13:05:53,171][86122] Updated weights for policy 1, policy_version 22190 (0.0009) +[2023-10-09 13:05:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 45318144. Throughput: 0: 1810.9, 1: 1813.1. Samples: 11346392. Policy #0 lag: (min: 18.0, avg: 20.0, max: 48.0) +[2023-10-09 13:05:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:53,536][86122] Updated weights for policy 1, policy_version 22200 (0.0007) +[2023-10-09 13:05:53,903][86121] Updated weights for policy 0, policy_version 22090 (0.0008) +[2023-10-09 13:05:54,270][86121] Updated weights for policy 0, policy_version 22100 (0.0007) +[2023-10-09 13:05:54,632][86121] Updated weights for policy 0, policy_version 22110 (0.0008) +[2023-10-09 13:05:57,301][86122] Updated weights for policy 1, policy_version 22210 (0.0007) +[2023-10-09 13:05:57,695][86122] Updated weights for policy 1, policy_version 22220 (0.0008) +[2023-10-09 13:05:58,058][86122] Updated weights for policy 1, policy_version 22230 (0.0008) +[2023-10-09 13:05:58,236][86121] Updated weights for policy 0, policy_version 22120 (0.0007) +[2023-10-09 13:05:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45383680. Throughput: 0: 1813.1, 1: 1811.0. Samples: 11356598. Policy #0 lag: (min: 31.0, avg: 33.1, max: 62.0) +[2023-10-09 13:05:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:05:58,423][86122] Updated weights for policy 1, policy_version 22240 (0.0007) +[2023-10-09 13:05:58,597][86121] Updated weights for policy 0, policy_version 22130 (0.0007) +[2023-10-09 13:05:58,961][86121] Updated weights for policy 0, policy_version 22140 (0.0009) +[2023-10-09 13:06:02,181][86122] Updated weights for policy 1, policy_version 22250 (0.0008) +[2023-10-09 13:06:02,536][86122] Updated weights for policy 1, policy_version 22260 (0.0010) +[2023-10-09 13:06:02,775][86121] Updated weights for policy 0, policy_version 22150 (0.0007) +[2023-10-09 13:06:02,897][86122] Updated weights for policy 1, policy_version 22270 (0.0008) +[2023-10-09 13:06:03,131][86121] Updated weights for policy 0, policy_version 22160 (0.0008) +[2023-10-09 13:06:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 45481984. Throughput: 0: 1810.3, 1: 1816.0. Samples: 11379106. Policy #0 lag: (min: 31.0, avg: 33.1, max: 62.0) +[2023-10-09 13:06:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:03,506][86121] Updated weights for policy 0, policy_version 22170 (0.0008) +[2023-10-09 13:06:06,615][86122] Updated weights for policy 1, policy_version 22280 (0.0009) +[2023-10-09 13:06:06,982][86122] Updated weights for policy 1, policy_version 22290 (0.0009) +[2023-10-09 13:06:07,270][86121] Updated weights for policy 0, policy_version 22180 (0.0008) +[2023-10-09 13:06:07,349][86122] Updated weights for policy 1, policy_version 22300 (0.0008) +[2023-10-09 13:06:07,632][86121] Updated weights for policy 0, policy_version 22190 (0.0009) +[2023-10-09 13:06:08,014][86121] Updated weights for policy 0, policy_version 22200 (0.0010) +[2023-10-09 13:06:08,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 45580288. Throughput: 0: 1819.7, 1: 1817.4. Samples: 11399628. Policy #0 lag: (min: 31.0, avg: 33.1, max: 62.0) +[2023-10-09 13:06:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:11,140][86122] Updated weights for policy 1, policy_version 22310 (0.0008) +[2023-10-09 13:06:11,505][86122] Updated weights for policy 1, policy_version 22320 (0.0008) +[2023-10-09 13:06:11,657][86121] Updated weights for policy 0, policy_version 22210 (0.0009) +[2023-10-09 13:06:11,873][86122] Updated weights for policy 1, policy_version 22330 (0.0009) +[2023-10-09 13:06:12,028][86121] Updated weights for policy 0, policy_version 22220 (0.0008) +[2023-10-09 13:06:12,391][86121] Updated weights for policy 0, policy_version 22230 (0.0007) +[2023-10-09 13:06:12,750][86121] Updated weights for policy 0, policy_version 22240 (0.0007) +[2023-10-09 13:06:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 45645824. Throughput: 0: 1819.1, 1: 1820.5. Samples: 11412048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:06:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:15,438][86122] Updated weights for policy 1, policy_version 22340 (0.0009) +[2023-10-09 13:06:15,803][86122] Updated weights for policy 1, policy_version 22350 (0.0009) +[2023-10-09 13:06:16,167][86122] Updated weights for policy 1, policy_version 22360 (0.0009) +[2023-10-09 13:06:16,511][86121] Updated weights for policy 0, policy_version 22250 (0.0008) +[2023-10-09 13:06:16,881][86121] Updated weights for policy 0, policy_version 22260 (0.0009) +[2023-10-09 13:06:17,251][86121] Updated weights for policy 0, policy_version 22270 (0.0007) +[2023-10-09 13:06:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.2). Total num frames: 45711360. Throughput: 0: 1818.7, 1: 1820.0. Samples: 11432460. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:06:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:19,769][86122] Updated weights for policy 1, policy_version 22370 (0.0008) +[2023-10-09 13:06:20,137][86122] Updated weights for policy 1, policy_version 22380 (0.0007) +[2023-10-09 13:06:20,497][86122] Updated weights for policy 1, policy_version 22390 (0.0008) +[2023-10-09 13:06:20,861][86122] Updated weights for policy 1, policy_version 22400 (0.0008) +[2023-10-09 13:06:20,942][86121] Updated weights for policy 0, policy_version 22280 (0.0009) +[2023-10-09 13:06:21,308][86121] Updated weights for policy 0, policy_version 22290 (0.0010) +[2023-10-09 13:06:21,675][86121] Updated weights for policy 0, policy_version 22300 (0.0007) +[2023-10-09 13:06:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 45776896. Throughput: 0: 1822.3, 1: 1818.7. Samples: 11455050. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:06:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:24,378][86122] Updated weights for policy 1, policy_version 22410 (0.0008) +[2023-10-09 13:06:24,749][86122] Updated weights for policy 1, policy_version 22420 (0.0007) +[2023-10-09 13:06:25,118][86122] Updated weights for policy 1, policy_version 22430 (0.0007) +[2023-10-09 13:06:25,231][86121] Updated weights for policy 0, policy_version 22310 (0.0007) +[2023-10-09 13:06:25,595][86121] Updated weights for policy 0, policy_version 22320 (0.0010) +[2023-10-09 13:06:25,958][86121] Updated weights for policy 0, policy_version 22330 (0.0009) +[2023-10-09 13:06:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 45842432. Throughput: 0: 1819.7, 1: 1819.1. Samples: 11465502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:06:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:06:28,871][86122] Updated weights for policy 1, policy_version 22440 (0.0007) +[2023-10-09 13:06:29,231][86122] Updated weights for policy 1, policy_version 22450 (0.0007) +[2023-10-09 13:06:29,590][86122] Updated weights for policy 1, policy_version 22460 (0.0007) +[2023-10-09 13:06:29,704][86121] Updated weights for policy 0, policy_version 22340 (0.0008) +[2023-10-09 13:06:30,063][86121] Updated weights for policy 0, policy_version 22350 (0.0007) +[2023-10-09 13:06:30,437][86121] Updated weights for policy 0, policy_version 22360 (0.0008) +[2023-10-09 13:06:33,203][86122] Updated weights for policy 1, policy_version 22470 (0.0008) +[2023-10-09 13:06:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 45907968. Throughput: 0: 1816.0, 1: 1828.4. Samples: 11487926. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 13:06:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:06:33,568][86122] Updated weights for policy 1, policy_version 22480 (0.0011) +[2023-10-09 13:06:33,938][86122] Updated weights for policy 1, policy_version 22490 (0.0009) +[2023-10-09 13:06:34,089][86121] Updated weights for policy 0, policy_version 22370 (0.0010) +[2023-10-09 13:06:34,464][86121] Updated weights for policy 0, policy_version 22380 (0.0007) +[2023-10-09 13:06:34,823][86121] Updated weights for policy 0, policy_version 22390 (0.0008) +[2023-10-09 13:06:35,195][86121] Updated weights for policy 0, policy_version 22400 (0.0010) +[2023-10-09 13:06:37,625][86122] Updated weights for policy 1, policy_version 22500 (0.0008) +[2023-10-09 13:06:37,980][86122] Updated weights for policy 1, policy_version 22510 (0.0008) +[2023-10-09 13:06:38,347][86122] Updated weights for policy 1, policy_version 22520 (0.0008) +[2023-10-09 13:06:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 45973504. Throughput: 0: 1820.5, 1: 1826.3. Samples: 11510498. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 13:06:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:06:39,064][86121] Updated weights for policy 0, policy_version 22410 (0.0010) +[2023-10-09 13:06:39,435][86121] Updated weights for policy 0, policy_version 22420 (0.0011) +[2023-10-09 13:06:39,804][86121] Updated weights for policy 0, policy_version 22430 (0.0007) +[2023-10-09 13:06:42,138][86122] Updated weights for policy 1, policy_version 22530 (0.0008) +[2023-10-09 13:06:42,550][86122] Updated weights for policy 1, policy_version 22540 (0.0007) +[2023-10-09 13:06:42,913][86122] Updated weights for policy 1, policy_version 22550 (0.0011) +[2023-10-09 13:06:43,281][86122] Updated weights for policy 1, policy_version 22560 (0.0009) +[2023-10-09 13:06:43,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46071808. Throughput: 0: 1818.7, 1: 1833.4. Samples: 11520944. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 13:06:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:06:43,493][86121] Updated weights for policy 0, policy_version 22440 (0.0009) +[2023-10-09 13:06:43,863][86121] Updated weights for policy 0, policy_version 22450 (0.0008) +[2023-10-09 13:06:44,221][86121] Updated weights for policy 0, policy_version 22460 (0.0008) +[2023-10-09 13:06:46,809][86122] Updated weights for policy 1, policy_version 22570 (0.0007) +[2023-10-09 13:06:47,170][86122] Updated weights for policy 1, policy_version 22580 (0.0009) +[2023-10-09 13:06:47,534][86122] Updated weights for policy 1, policy_version 22590 (0.0008) +[2023-10-09 13:06:47,934][86121] Updated weights for policy 0, policy_version 22470 (0.0008) +[2023-10-09 13:06:48,304][86121] Updated weights for policy 0, policy_version 22480 (0.0008) +[2023-10-09 13:06:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46137344. Throughput: 0: 1823.4, 1: 1825.0. Samples: 11543284. Policy #0 lag: (min: 31.0, avg: 34.2, max: 63.0) +[2023-10-09 13:06:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:48,676][86121] Updated weights for policy 0, policy_version 22490 (0.0008) +[2023-10-09 13:06:51,254][86122] Updated weights for policy 1, policy_version 22600 (0.0010) +[2023-10-09 13:06:51,621][86122] Updated weights for policy 1, policy_version 22610 (0.0010) +[2023-10-09 13:06:51,984][86122] Updated weights for policy 1, policy_version 22620 (0.0008) +[2023-10-09 13:06:52,265][86121] Updated weights for policy 0, policy_version 22500 (0.0009) +[2023-10-09 13:06:52,626][86121] Updated weights for policy 0, policy_version 22510 (0.0009) +[2023-10-09 13:06:52,996][86121] Updated weights for policy 0, policy_version 22520 (0.0010) +[2023-10-09 13:06:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 46235648. Throughput: 0: 1818.7, 1: 1834.0. Samples: 11564000. Policy #0 lag: (min: 29.0, avg: 37.9, max: 61.0) +[2023-10-09 13:06:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:55,562][86122] Updated weights for policy 1, policy_version 22630 (0.0009) +[2023-10-09 13:06:55,932][86122] Updated weights for policy 1, policy_version 22640 (0.0009) +[2023-10-09 13:06:56,291][86122] Updated weights for policy 1, policy_version 22650 (0.0007) +[2023-10-09 13:06:56,810][86121] Updated weights for policy 0, policy_version 22530 (0.0010) +[2023-10-09 13:06:57,209][86121] Updated weights for policy 0, policy_version 22540 (0.0008) +[2023-10-09 13:06:57,569][86121] Updated weights for policy 0, policy_version 22550 (0.0008) +[2023-10-09 13:06:57,938][86121] Updated weights for policy 0, policy_version 22560 (0.0008) +[2023-10-09 13:06:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 46301184. Throughput: 0: 1814.5, 1: 1821.4. Samples: 11575664. Policy #0 lag: (min: 29.0, avg: 37.9, max: 61.0) +[2023-10-09 13:06:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:06:59,806][86122] Updated weights for policy 1, policy_version 22660 (0.0007) +[2023-10-09 13:07:00,176][86122] Updated weights for policy 1, policy_version 22670 (0.0008) +[2023-10-09 13:07:00,532][86122] Updated weights for policy 1, policy_version 22680 (0.0010) +[2023-10-09 13:07:01,652][86121] Updated weights for policy 0, policy_version 22570 (0.0009) +[2023-10-09 13:07:02,025][86121] Updated weights for policy 0, policy_version 22580 (0.0008) +[2023-10-09 13:07:02,393][86121] Updated weights for policy 0, policy_version 22590 (0.0008) +[2023-10-09 13:07:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46366720. Throughput: 0: 1815.8, 1: 1840.2. Samples: 11596982. Policy #0 lag: (min: 29.0, avg: 37.9, max: 61.0) +[2023-10-09 13:07:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:07:04,296][86122] Updated weights for policy 1, policy_version 22690 (0.0010) +[2023-10-09 13:07:04,659][86122] Updated weights for policy 1, policy_version 22700 (0.0007) +[2023-10-09 13:07:05,019][86122] Updated weights for policy 1, policy_version 22710 (0.0007) +[2023-10-09 13:07:05,383][86122] Updated weights for policy 1, policy_version 22720 (0.0008) +[2023-10-09 13:07:06,240][86121] Updated weights for policy 0, policy_version 22600 (0.0008) +[2023-10-09 13:07:06,602][86121] Updated weights for policy 0, policy_version 22610 (0.0009) +[2023-10-09 13:07:06,975][86121] Updated weights for policy 0, policy_version 22620 (0.0010) +[2023-10-09 13:07:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 46432256. Throughput: 0: 1802.3, 1: 1835.3. Samples: 11618744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:07:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:07:09,042][86122] Updated weights for policy 1, policy_version 22730 (0.0009) +[2023-10-09 13:07:09,414][86122] Updated weights for policy 1, policy_version 22740 (0.0009) +[2023-10-09 13:07:09,780][86122] Updated weights for policy 1, policy_version 22750 (0.0007) +[2023-10-09 13:07:10,683][86121] Updated weights for policy 0, policy_version 22630 (0.0008) +[2023-10-09 13:07:11,053][86121] Updated weights for policy 0, policy_version 22640 (0.0007) +[2023-10-09 13:07:11,417][86121] Updated weights for policy 0, policy_version 22650 (0.0009) +[2023-10-09 13:07:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46497792. Throughput: 0: 1810.8, 1: 1832.4. Samples: 11629444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:07:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:07:13,495][86122] Updated weights for policy 1, policy_version 22760 (0.0008) +[2023-10-09 13:07:13,847][86122] Updated weights for policy 1, policy_version 22770 (0.0009) +[2023-10-09 13:07:14,207][86122] Updated weights for policy 1, policy_version 22780 (0.0010) +[2023-10-09 13:07:15,000][86121] Updated weights for policy 0, policy_version 22660 (0.0009) +[2023-10-09 13:07:15,365][86121] Updated weights for policy 0, policy_version 22670 (0.0008) +[2023-10-09 13:07:15,730][86121] Updated weights for policy 0, policy_version 22680 (0.0007) +[2023-10-09 13:07:17,979][86122] Updated weights for policy 1, policy_version 22790 (0.0010) +[2023-10-09 13:07:18,339][86122] Updated weights for policy 1, policy_version 22800 (0.0007) +[2023-10-09 13:07:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 46563328. Throughput: 0: 1805.7, 1: 1829.6. Samples: 11651512. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:07:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:07:18,699][86122] Updated weights for policy 1, policy_version 22810 (0.0008) +[2023-10-09 13:07:19,469][86121] Updated weights for policy 0, policy_version 22690 (0.0008) +[2023-10-09 13:07:19,848][86121] Updated weights for policy 0, policy_version 22700 (0.0009) +[2023-10-09 13:07:20,207][86121] Updated weights for policy 0, policy_version 22710 (0.0008) +[2023-10-09 13:07:20,576][86121] Updated weights for policy 0, policy_version 22720 (0.0009) +[2023-10-09 13:07:22,391][86122] Updated weights for policy 1, policy_version 22820 (0.0008) +[2023-10-09 13:07:22,766][86122] Updated weights for policy 1, policy_version 22830 (0.0009) +[2023-10-09 13:07:23,130][86122] Updated weights for policy 1, policy_version 22840 (0.0008) +[2023-10-09 13:07:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 46628864. Throughput: 0: 1804.4, 1: 1824.1. Samples: 11673782. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:07:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:07:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000022720_23265280.pth... +[2023-10-09 13:07:23,418][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000022848_23396352.pth... +[2023-10-09 13:07:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000021024_21528576.pth +[2023-10-09 13:07:23,447][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000021120_21626880.pth +[2023-10-09 13:07:24,449][86121] Updated weights for policy 0, policy_version 22730 (0.0011) +[2023-10-09 13:07:24,814][86121] Updated weights for policy 0, policy_version 22740 (0.0008) +[2023-10-09 13:07:25,185][86121] Updated weights for policy 0, policy_version 22750 (0.0009) +[2023-10-09 13:07:26,843][86122] Updated weights for policy 1, policy_version 22850 (0.0007) +[2023-10-09 13:07:27,207][86122] Updated weights for policy 1, policy_version 22860 (0.0007) +[2023-10-09 13:07:27,571][86122] Updated weights for policy 1, policy_version 22870 (0.0008) +[2023-10-09 13:07:27,936][86122] Updated weights for policy 1, policy_version 22880 (0.0007) +[2023-10-09 13:07:28,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 46727168. Throughput: 0: 1800.7, 1: 1827.2. Samples: 11684198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:07:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:07:28,790][86121] Updated weights for policy 0, policy_version 22760 (0.0011) +[2023-10-09 13:07:29,155][86121] Updated weights for policy 0, policy_version 22770 (0.0009) +[2023-10-09 13:07:29,523][86121] Updated weights for policy 0, policy_version 22780 (0.0008) +[2023-10-09 13:07:31,870][86122] Updated weights for policy 1, policy_version 22890 (0.0009) +[2023-10-09 13:07:32,240][86122] Updated weights for policy 1, policy_version 22900 (0.0008) +[2023-10-09 13:07:32,595][86122] Updated weights for policy 1, policy_version 22910 (0.0008) +[2023-10-09 13:07:33,116][86121] Updated weights for policy 0, policy_version 22790 (0.0008) +[2023-10-09 13:07:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 46792704. Throughput: 0: 1802.0, 1: 1829.5. Samples: 11706704. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:07:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:07:33,485][86121] Updated weights for policy 0, policy_version 22800 (0.0008) +[2023-10-09 13:07:33,856][86121] Updated weights for policy 0, policy_version 22810 (0.0008) +[2023-10-09 13:07:36,139][86122] Updated weights for policy 1, policy_version 22920 (0.0007) +[2023-10-09 13:07:36,502][86122] Updated weights for policy 1, policy_version 22930 (0.0008) +[2023-10-09 13:07:36,859][86122] Updated weights for policy 1, policy_version 22940 (0.0010) +[2023-10-09 13:07:37,440][86121] Updated weights for policy 0, policy_version 22820 (0.0008) +[2023-10-09 13:07:37,813][86121] Updated weights for policy 0, policy_version 22830 (0.0008) +[2023-10-09 13:07:38,182][86121] Updated weights for policy 0, policy_version 22840 (0.0010) +[2023-10-09 13:07:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 46858240. Throughput: 0: 1817.7, 1: 1828.9. Samples: 11728098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:07:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:07:40,619][86122] Updated weights for policy 1, policy_version 22950 (0.0010) +[2023-10-09 13:07:40,982][86122] Updated weights for policy 1, policy_version 22960 (0.0011) +[2023-10-09 13:07:41,347][86122] Updated weights for policy 1, policy_version 22970 (0.0007) +[2023-10-09 13:07:41,892][86121] Updated weights for policy 0, policy_version 22850 (0.0008) +[2023-10-09 13:07:42,288][86121] Updated weights for policy 0, policy_version 22860 (0.0008) +[2023-10-09 13:07:42,659][86121] Updated weights for policy 0, policy_version 22870 (0.0007) +[2023-10-09 13:07:43,022][86121] Updated weights for policy 0, policy_version 22880 (0.0007) +[2023-10-09 13:07:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 46956544. Throughput: 0: 1817.9, 1: 1829.6. Samples: 11739802. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) +[2023-10-09 13:07:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:07:44,878][86122] Updated weights for policy 1, policy_version 22980 (0.0009) +[2023-10-09 13:07:45,243][86122] Updated weights for policy 1, policy_version 22990 (0.0008) +[2023-10-09 13:07:45,605][86122] Updated weights for policy 1, policy_version 23000 (0.0012) +[2023-10-09 13:07:46,733][86121] Updated weights for policy 0, policy_version 22890 (0.0007) +[2023-10-09 13:07:47,097][86121] Updated weights for policy 0, policy_version 22900 (0.0009) +[2023-10-09 13:07:47,467][86121] Updated weights for policy 0, policy_version 22910 (0.0007) +[2023-10-09 13:07:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47022080. Throughput: 0: 1816.9, 1: 1825.9. Samples: 11760908. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) +[2023-10-09 13:07:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:07:49,222][86122] Updated weights for policy 1, policy_version 23010 (0.0010) +[2023-10-09 13:07:49,584][86122] Updated weights for policy 1, policy_version 23020 (0.0008) +[2023-10-09 13:07:49,955][86122] Updated weights for policy 1, policy_version 23030 (0.0008) +[2023-10-09 13:07:50,319][86122] Updated weights for policy 1, policy_version 23040 (0.0009) +[2023-10-09 13:07:51,138][86121] Updated weights for policy 0, policy_version 22920 (0.0008) +[2023-10-09 13:07:51,499][86121] Updated weights for policy 0, policy_version 22930 (0.0007) +[2023-10-09 13:07:51,868][86121] Updated weights for policy 0, policy_version 22940 (0.0010) +[2023-10-09 13:07:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 47087616. Throughput: 0: 1828.9, 1: 1829.2. Samples: 11783356. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) +[2023-10-09 13:07:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:07:54,156][86122] Updated weights for policy 1, policy_version 23050 (0.0011) +[2023-10-09 13:07:54,514][86122] Updated weights for policy 1, policy_version 23060 (0.0008) +[2023-10-09 13:07:54,871][86122] Updated weights for policy 1, policy_version 23070 (0.0008) +[2023-10-09 13:07:55,465][86121] Updated weights for policy 0, policy_version 22950 (0.0008) +[2023-10-09 13:07:55,835][86121] Updated weights for policy 0, policy_version 22960 (0.0007) +[2023-10-09 13:07:56,193][86121] Updated weights for policy 0, policy_version 22970 (0.0007) +[2023-10-09 13:07:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 47153152. Throughput: 0: 1822.7, 1: 1829.5. Samples: 11793792. Policy #0 lag: (min: 16.0, avg: 42.1, max: 48.0) +[2023-10-09 13:07:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:07:58,539][86122] Updated weights for policy 1, policy_version 23080 (0.0008) +[2023-10-09 13:07:58,900][86122] Updated weights for policy 1, policy_version 23090 (0.0007) +[2023-10-09 13:07:59,264][86122] Updated weights for policy 1, policy_version 23100 (0.0007) +[2023-10-09 13:07:59,921][86121] Updated weights for policy 0, policy_version 22980 (0.0008) +[2023-10-09 13:08:00,288][86121] Updated weights for policy 0, policy_version 22990 (0.0010) +[2023-10-09 13:08:00,665][86121] Updated weights for policy 0, policy_version 23000 (0.0007) +[2023-10-09 13:08:02,962][86122] Updated weights for policy 1, policy_version 23110 (0.0007) +[2023-10-09 13:08:03,329][86122] Updated weights for policy 1, policy_version 23120 (0.0008) +[2023-10-09 13:08:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 47218688. Throughput: 0: 1821.2, 1: 1832.6. Samples: 11815934. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:08:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:03,695][86122] Updated weights for policy 1, policy_version 23130 (0.0008) +[2023-10-09 13:08:04,274][86121] Updated weights for policy 0, policy_version 23010 (0.0009) +[2023-10-09 13:08:04,646][86121] Updated weights for policy 0, policy_version 23020 (0.0008) +[2023-10-09 13:08:05,015][86121] Updated weights for policy 0, policy_version 23030 (0.0007) +[2023-10-09 13:08:05,374][86121] Updated weights for policy 0, policy_version 23040 (0.0009) +[2023-10-09 13:08:07,393][86122] Updated weights for policy 1, policy_version 23140 (0.0008) +[2023-10-09 13:08:07,748][86122] Updated weights for policy 1, policy_version 23150 (0.0007) +[2023-10-09 13:08:08,102][86122] Updated weights for policy 1, policy_version 23160 (0.0010) +[2023-10-09 13:08:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47316992. Throughput: 0: 1819.0, 1: 1830.3. Samples: 11838002. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:08:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:09,243][86121] Updated weights for policy 0, policy_version 23050 (0.0007) +[2023-10-09 13:08:09,603][86121] Updated weights for policy 0, policy_version 23060 (0.0008) +[2023-10-09 13:08:09,966][86121] Updated weights for policy 0, policy_version 23070 (0.0007) +[2023-10-09 13:08:11,711][86122] Updated weights for policy 1, policy_version 23170 (0.0011) +[2023-10-09 13:08:12,078][86122] Updated weights for policy 1, policy_version 23180 (0.0008) +[2023-10-09 13:08:12,436][86122] Updated weights for policy 1, policy_version 23190 (0.0007) +[2023-10-09 13:08:12,806][86122] Updated weights for policy 1, policy_version 23200 (0.0008) +[2023-10-09 13:08:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47382528. Throughput: 0: 1820.2, 1: 1833.8. Samples: 11848628. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:08:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:13,740][86121] Updated weights for policy 0, policy_version 23080 (0.0008) +[2023-10-09 13:08:14,104][86121] Updated weights for policy 0, policy_version 23090 (0.0009) +[2023-10-09 13:08:14,469][86121] Updated weights for policy 0, policy_version 23100 (0.0008) +[2023-10-09 13:08:16,460][86122] Updated weights for policy 1, policy_version 23210 (0.0007) +[2023-10-09 13:08:16,837][86122] Updated weights for policy 1, policy_version 23220 (0.0008) +[2023-10-09 13:08:17,192][86122] Updated weights for policy 1, policy_version 23230 (0.0008) +[2023-10-09 13:08:18,122][86121] Updated weights for policy 0, policy_version 23110 (0.0009) +[2023-10-09 13:08:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47448064. Throughput: 0: 1819.0, 1: 1823.1. Samples: 11870600. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:08:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:18,489][86121] Updated weights for policy 0, policy_version 23120 (0.0007) +[2023-10-09 13:08:18,849][86121] Updated weights for policy 0, policy_version 23130 (0.0012) +[2023-10-09 13:08:20,927][86122] Updated weights for policy 1, policy_version 23240 (0.0010) +[2023-10-09 13:08:21,278][86122] Updated weights for policy 1, policy_version 23250 (0.0010) +[2023-10-09 13:08:21,650][86122] Updated weights for policy 1, policy_version 23260 (0.0009) +[2023-10-09 13:08:22,536][86121] Updated weights for policy 0, policy_version 23140 (0.0009) +[2023-10-09 13:08:22,893][86121] Updated weights for policy 0, policy_version 23150 (0.0007) +[2023-10-09 13:08:23,263][86121] Updated weights for policy 0, policy_version 23160 (0.0010) +[2023-10-09 13:08:23,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47513600. Throughput: 0: 1815.2, 1: 1832.8. Samples: 11892256. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) +[2023-10-09 13:08:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:25,267][86122] Updated weights for policy 1, policy_version 23270 (0.0009) +[2023-10-09 13:08:25,626][86122] Updated weights for policy 1, policy_version 23280 (0.0010) +[2023-10-09 13:08:25,993][86122] Updated weights for policy 1, policy_version 23290 (0.0008) +[2023-10-09 13:08:26,952][86121] Updated weights for policy 0, policy_version 23170 (0.0009) +[2023-10-09 13:08:27,343][86121] Updated weights for policy 0, policy_version 23180 (0.0008) +[2023-10-09 13:08:27,708][86121] Updated weights for policy 0, policy_version 23190 (0.0008) +[2023-10-09 13:08:28,079][86121] Updated weights for policy 0, policy_version 23200 (0.0009) +[2023-10-09 13:08:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47611904. Throughput: 0: 1813.3, 1: 1824.2. Samples: 11903490. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) +[2023-10-09 13:08:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:29,704][86122] Updated weights for policy 1, policy_version 23300 (0.0008) +[2023-10-09 13:08:30,067][86122] Updated weights for policy 1, policy_version 23310 (0.0008) +[2023-10-09 13:08:30,429][86122] Updated weights for policy 1, policy_version 23320 (0.0010) +[2023-10-09 13:08:31,768][86121] Updated weights for policy 0, policy_version 23210 (0.0008) +[2023-10-09 13:08:32,135][86121] Updated weights for policy 0, policy_version 23220 (0.0008) +[2023-10-09 13:08:32,509][86121] Updated weights for policy 0, policy_version 23230 (0.0008) +[2023-10-09 13:08:33,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 47677440. Throughput: 0: 1822.0, 1: 1828.6. Samples: 11925186. Policy #0 lag: (min: 31.0, avg: 33.0, max: 62.0) +[2023-10-09 13:08:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:34,083][86122] Updated weights for policy 1, policy_version 23330 (0.0009) +[2023-10-09 13:08:34,445][86122] Updated weights for policy 1, policy_version 23340 (0.0009) +[2023-10-09 13:08:34,800][86122] Updated weights for policy 1, policy_version 23350 (0.0008) +[2023-10-09 13:08:35,163][86122] Updated weights for policy 1, policy_version 23360 (0.0011) +[2023-10-09 13:08:36,185][86121] Updated weights for policy 0, policy_version 23240 (0.0008) +[2023-10-09 13:08:36,547][86121] Updated weights for policy 0, policy_version 23250 (0.0008) +[2023-10-09 13:08:36,920][86121] Updated weights for policy 0, policy_version 23260 (0.0007) +[2023-10-09 13:08:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 47742976. Throughput: 0: 1814.1, 1: 1824.0. Samples: 11947070. Policy #0 lag: (min: 0.0, avg: 21.4, max: 32.0) +[2023-10-09 13:08:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:08:39,029][86122] Updated weights for policy 1, policy_version 23370 (0.0010) +[2023-10-09 13:08:39,403][86122] Updated weights for policy 1, policy_version 23380 (0.0009) +[2023-10-09 13:08:39,765][86122] Updated weights for policy 1, policy_version 23390 (0.0007) +[2023-10-09 13:08:40,603][86121] Updated weights for policy 0, policy_version 23270 (0.0008) +[2023-10-09 13:08:40,977][86121] Updated weights for policy 0, policy_version 23280 (0.0008) +[2023-10-09 13:08:41,345][86121] Updated weights for policy 0, policy_version 23290 (0.0008) +[2023-10-09 13:08:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 47808512. Throughput: 0: 1820.9, 1: 1821.3. Samples: 11957690. Policy #0 lag: (min: 0.0, avg: 21.4, max: 32.0) +[2023-10-09 13:08:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:43,515][86122] Updated weights for policy 1, policy_version 23400 (0.0008) +[2023-10-09 13:08:43,872][86122] Updated weights for policy 1, policy_version 23410 (0.0008) +[2023-10-09 13:08:44,246][86122] Updated weights for policy 1, policy_version 23420 (0.0009) +[2023-10-09 13:08:45,089][86121] Updated weights for policy 0, policy_version 23300 (0.0008) +[2023-10-09 13:08:45,448][86121] Updated weights for policy 0, policy_version 23310 (0.0008) +[2023-10-09 13:08:45,813][86121] Updated weights for policy 0, policy_version 23320 (0.0007) +[2023-10-09 13:08:47,922][86122] Updated weights for policy 1, policy_version 23430 (0.0008) +[2023-10-09 13:08:48,298][86122] Updated weights for policy 1, policy_version 23440 (0.0009) +[2023-10-09 13:08:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 47874048. Throughput: 0: 1819.6, 1: 1814.0. Samples: 11979444. Policy #0 lag: (min: 0.0, avg: 21.4, max: 32.0) +[2023-10-09 13:08:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:48,664][86122] Updated weights for policy 1, policy_version 23450 (0.0008) +[2023-10-09 13:08:49,673][86121] Updated weights for policy 0, policy_version 23330 (0.0008) +[2023-10-09 13:08:50,039][86121] Updated weights for policy 0, policy_version 23340 (0.0010) +[2023-10-09 13:08:50,413][86121] Updated weights for policy 0, policy_version 23350 (0.0009) +[2023-10-09 13:08:50,787][86121] Updated weights for policy 0, policy_version 23360 (0.0007) +[2023-10-09 13:08:52,515][86122] Updated weights for policy 1, policy_version 23460 (0.0010) +[2023-10-09 13:08:52,874][86122] Updated weights for policy 1, policy_version 23470 (0.0007) +[2023-10-09 13:08:53,234][86122] Updated weights for policy 1, policy_version 23480 (0.0007) +[2023-10-09 13:08:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 47939584. Throughput: 0: 1817.1, 1: 1813.3. Samples: 12001370. Policy #0 lag: (min: 0.0, avg: 21.4, max: 32.0) +[2023-10-09 13:08:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:54,512][86121] Updated weights for policy 0, policy_version 23370 (0.0010) +[2023-10-09 13:08:54,874][86121] Updated weights for policy 0, policy_version 23380 (0.0011) +[2023-10-09 13:08:55,242][86121] Updated weights for policy 0, policy_version 23390 (0.0009) +[2023-10-09 13:08:56,985][86122] Updated weights for policy 1, policy_version 23490 (0.0008) +[2023-10-09 13:08:57,349][86122] Updated weights for policy 1, policy_version 23500 (0.0007) +[2023-10-09 13:08:57,710][86122] Updated weights for policy 1, policy_version 23510 (0.0010) +[2023-10-09 13:08:58,068][86122] Updated weights for policy 1, policy_version 23520 (0.0007) +[2023-10-09 13:08:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48037888. Throughput: 0: 1815.4, 1: 1806.9. Samples: 12011634. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:08:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:08:58,907][86121] Updated weights for policy 0, policy_version 23400 (0.0008) +[2023-10-09 13:08:59,283][86121] Updated weights for policy 0, policy_version 23410 (0.0010) +[2023-10-09 13:08:59,644][86121] Updated weights for policy 0, policy_version 23420 (0.0010) +[2023-10-09 13:09:01,839][86122] Updated weights for policy 1, policy_version 23530 (0.0010) +[2023-10-09 13:09:02,194][86122] Updated weights for policy 1, policy_version 23540 (0.0011) +[2023-10-09 13:09:02,558][86122] Updated weights for policy 1, policy_version 23550 (0.0010) +[2023-10-09 13:09:03,352][86121] Updated weights for policy 0, policy_version 23430 (0.0009) +[2023-10-09 13:09:03,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48103424. Throughput: 0: 1816.6, 1: 1821.0. Samples: 12034294. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:09:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:09:03,723][86121] Updated weights for policy 0, policy_version 23440 (0.0008) +[2023-10-09 13:09:04,100][86121] Updated weights for policy 0, policy_version 23450 (0.0009) +[2023-10-09 13:09:06,241][86122] Updated weights for policy 1, policy_version 23560 (0.0007) +[2023-10-09 13:09:06,599][86122] Updated weights for policy 1, policy_version 23570 (0.0008) +[2023-10-09 13:09:06,971][86122] Updated weights for policy 1, policy_version 23580 (0.0008) +[2023-10-09 13:09:07,628][86121] Updated weights for policy 0, policy_version 23460 (0.0011) +[2023-10-09 13:09:07,994][86121] Updated weights for policy 0, policy_version 23470 (0.0011) +[2023-10-09 13:09:08,357][86121] Updated weights for policy 0, policy_version 23480 (0.0010) +[2023-10-09 13:09:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48168960. Throughput: 0: 1820.4, 1: 1808.0. Samples: 12055530. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:09:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:09:10,753][86122] Updated weights for policy 1, policy_version 23590 (0.0008) +[2023-10-09 13:09:11,107][86122] Updated weights for policy 1, policy_version 23600 (0.0008) +[2023-10-09 13:09:11,469][86122] Updated weights for policy 1, policy_version 23610 (0.0009) +[2023-10-09 13:09:11,962][86121] Updated weights for policy 0, policy_version 23490 (0.0010) +[2023-10-09 13:09:12,365][86121] Updated weights for policy 0, policy_version 23500 (0.0009) +[2023-10-09 13:09:12,731][86121] Updated weights for policy 0, policy_version 23510 (0.0009) +[2023-10-09 13:09:13,098][86121] Updated weights for policy 0, policy_version 23520 (0.0008) +[2023-10-09 13:09:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 48267264. Throughput: 0: 1816.8, 1: 1820.8. Samples: 12067180. Policy #0 lag: (min: 8.0, avg: 35.1, max: 40.0) +[2023-10-09 13:09:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:09:15,053][86122] Updated weights for policy 1, policy_version 23620 (0.0008) +[2023-10-09 13:09:15,416][86122] Updated weights for policy 1, policy_version 23630 (0.0009) +[2023-10-09 13:09:15,784][86122] Updated weights for policy 1, policy_version 23640 (0.0010) +[2023-10-09 13:09:16,793][86121] Updated weights for policy 0, policy_version 23530 (0.0008) +[2023-10-09 13:09:17,162][86121] Updated weights for policy 0, policy_version 23540 (0.0009) +[2023-10-09 13:09:17,525][86121] Updated weights for policy 0, policy_version 23550 (0.0008) +[2023-10-09 13:09:18,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48332800. Throughput: 0: 1813.6, 1: 1806.1. Samples: 12088074. Policy #0 lag: (min: 8.0, avg: 35.1, max: 40.0) +[2023-10-09 13:09:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:09:19,412][86122] Updated weights for policy 1, policy_version 23650 (0.0007) +[2023-10-09 13:09:19,781][86122] Updated weights for policy 1, policy_version 23660 (0.0008) +[2023-10-09 13:09:20,151][86122] Updated weights for policy 1, policy_version 23670 (0.0008) +[2023-10-09 13:09:20,505][86122] Updated weights for policy 1, policy_version 23680 (0.0007) +[2023-10-09 13:09:21,347][86121] Updated weights for policy 0, policy_version 23560 (0.0008) +[2023-10-09 13:09:21,721][86121] Updated weights for policy 0, policy_version 23570 (0.0008) +[2023-10-09 13:09:22,094][86121] Updated weights for policy 0, policy_version 23580 (0.0010) +[2023-10-09 13:09:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 48398336. Throughput: 0: 1811.7, 1: 1813.3. Samples: 12110194. Policy #0 lag: (min: 8.0, avg: 35.1, max: 40.0) +[2023-10-09 13:09:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:09:23,413][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000023680_24248320.pth... +[2023-10-09 13:09:23,413][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000023584_24150016.pth... +[2023-10-09 13:09:23,442][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000021984_22511616.pth +[2023-10-09 13:09:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000021888_22413312.pth +[2023-10-09 13:09:24,121][86122] Updated weights for policy 1, policy_version 23690 (0.0009) +[2023-10-09 13:09:24,487][86122] Updated weights for policy 1, policy_version 23700 (0.0008) +[2023-10-09 13:09:24,851][86122] Updated weights for policy 1, policy_version 23710 (0.0009) +[2023-10-09 13:09:25,896][86121] Updated weights for policy 0, policy_version 23590 (0.0008) +[2023-10-09 13:09:26,249][86121] Updated weights for policy 0, policy_version 23600 (0.0008) +[2023-10-09 13:09:26,620][86121] Updated weights for policy 0, policy_version 23610 (0.0007) +[2023-10-09 13:09:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 48463872. Throughput: 0: 1817.0, 1: 1815.2. Samples: 12121142. Policy #0 lag: (min: 8.0, avg: 35.1, max: 40.0) +[2023-10-09 13:09:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:09:28,647][86122] Updated weights for policy 1, policy_version 23720 (0.0008) +[2023-10-09 13:09:29,003][86122] Updated weights for policy 1, policy_version 23730 (0.0009) +[2023-10-09 13:09:29,377][86122] Updated weights for policy 1, policy_version 23740 (0.0009) +[2023-10-09 13:09:30,107][86121] Updated weights for policy 0, policy_version 23620 (0.0007) +[2023-10-09 13:09:30,469][86121] Updated weights for policy 0, policy_version 23630 (0.0007) +[2023-10-09 13:09:30,842][86121] Updated weights for policy 0, policy_version 23640 (0.0010) +[2023-10-09 13:09:33,071][86122] Updated weights for policy 1, policy_version 23750 (0.0007) +[2023-10-09 13:09:33,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48529408. Throughput: 0: 1811.9, 1: 1815.2. Samples: 12142664. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:09:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:09:33,433][86122] Updated weights for policy 1, policy_version 23760 (0.0008) +[2023-10-09 13:09:33,801][86122] Updated weights for policy 1, policy_version 23770 (0.0011) +[2023-10-09 13:09:34,728][86121] Updated weights for policy 0, policy_version 23650 (0.0009) +[2023-10-09 13:09:35,099][86121] Updated weights for policy 0, policy_version 23660 (0.0007) +[2023-10-09 13:09:35,461][86121] Updated weights for policy 0, policy_version 23670 (0.0008) +[2023-10-09 13:09:35,829][86121] Updated weights for policy 0, policy_version 23680 (0.0008) +[2023-10-09 13:09:37,618][86122] Updated weights for policy 1, policy_version 23780 (0.0010) +[2023-10-09 13:09:37,987][86122] Updated weights for policy 1, policy_version 23790 (0.0008) +[2023-10-09 13:09:38,347][86122] Updated weights for policy 1, policy_version 23800 (0.0010) +[2023-10-09 13:09:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 48594944. Throughput: 0: 1818.8, 1: 1822.7. Samples: 12165238. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:09:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:09:39,387][86121] Updated weights for policy 0, policy_version 23690 (0.0007) +[2023-10-09 13:09:39,752][86121] Updated weights for policy 0, policy_version 23700 (0.0009) +[2023-10-09 13:09:40,123][86121] Updated weights for policy 0, policy_version 23710 (0.0009) +[2023-10-09 13:09:42,003][86122] Updated weights for policy 1, policy_version 23810 (0.0010) +[2023-10-09 13:09:42,360][86122] Updated weights for policy 1, policy_version 23820 (0.0008) +[2023-10-09 13:09:42,728][86122] Updated weights for policy 1, policy_version 23830 (0.0009) +[2023-10-09 13:09:43,090][86122] Updated weights for policy 1, policy_version 23840 (0.0010) +[2023-10-09 13:09:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48693248. Throughput: 0: 1823.7, 1: 1821.1. Samples: 12175652. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:09:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:09:43,842][86121] Updated weights for policy 0, policy_version 23720 (0.0007) +[2023-10-09 13:09:44,208][86121] Updated weights for policy 0, policy_version 23730 (0.0008) +[2023-10-09 13:09:44,572][86121] Updated weights for policy 0, policy_version 23740 (0.0009) +[2023-10-09 13:09:46,774][86122] Updated weights for policy 1, policy_version 23850 (0.0009) +[2023-10-09 13:09:47,144][86122] Updated weights for policy 1, policy_version 23860 (0.0010) +[2023-10-09 13:09:47,506][86122] Updated weights for policy 1, policy_version 23870 (0.0009) +[2023-10-09 13:09:48,310][86121] Updated weights for policy 0, policy_version 23750 (0.0007) +[2023-10-09 13:09:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48758784. Throughput: 0: 1819.7, 1: 1816.6. Samples: 12197930. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:09:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:09:48,675][86121] Updated weights for policy 0, policy_version 23760 (0.0007) +[2023-10-09 13:09:49,044][86121] Updated weights for policy 0, policy_version 23770 (0.0008) +[2023-10-09 13:09:51,148][86122] Updated weights for policy 1, policy_version 23880 (0.0009) +[2023-10-09 13:09:51,512][86122] Updated weights for policy 1, policy_version 23890 (0.0010) +[2023-10-09 13:09:51,876][86122] Updated weights for policy 1, policy_version 23900 (0.0008) +[2023-10-09 13:09:52,784][86121] Updated weights for policy 0, policy_version 23780 (0.0009) +[2023-10-09 13:09:53,146][86121] Updated weights for policy 0, policy_version 23790 (0.0007) +[2023-10-09 13:09:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 48824320. Throughput: 0: 1821.7, 1: 1822.9. Samples: 12219538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:09:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:09:53,511][86121] Updated weights for policy 0, policy_version 23800 (0.0009) +[2023-10-09 13:09:55,649][86122] Updated weights for policy 1, policy_version 23910 (0.0008) +[2023-10-09 13:09:56,002][86122] Updated weights for policy 1, policy_version 23920 (0.0009) +[2023-10-09 13:09:56,365][86122] Updated weights for policy 1, policy_version 23930 (0.0009) +[2023-10-09 13:09:57,290][86121] Updated weights for policy 0, policy_version 23810 (0.0011) +[2023-10-09 13:09:57,695][86121] Updated weights for policy 0, policy_version 23820 (0.0010) +[2023-10-09 13:09:58,061][86121] Updated weights for policy 0, policy_version 23830 (0.0008) +[2023-10-09 13:09:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 48889856. Throughput: 0: 1816.6, 1: 1819.6. Samples: 12230808. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:09:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:09:58,432][86121] Updated weights for policy 0, policy_version 23840 (0.0007) +[2023-10-09 13:09:59,954][86122] Updated weights for policy 1, policy_version 23940 (0.0008) +[2023-10-09 13:10:00,315][86122] Updated weights for policy 1, policy_version 23950 (0.0008) +[2023-10-09 13:10:00,680][86122] Updated weights for policy 1, policy_version 23960 (0.0010) +[2023-10-09 13:10:02,197][86121] Updated weights for policy 0, policy_version 23850 (0.0011) +[2023-10-09 13:10:02,561][86121] Updated weights for policy 0, policy_version 23860 (0.0008) +[2023-10-09 13:10:02,935][86121] Updated weights for policy 0, policy_version 23870 (0.0008) +[2023-10-09 13:10:03,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 48988160. Throughput: 0: 1825.9, 1: 1821.1. Samples: 12252188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:10:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:10:04,339][86122] Updated weights for policy 1, policy_version 23970 (0.0008) +[2023-10-09 13:10:04,693][86122] Updated weights for policy 1, policy_version 23980 (0.0008) +[2023-10-09 13:10:05,063][86122] Updated weights for policy 1, policy_version 23990 (0.0008) +[2023-10-09 13:10:05,429][86122] Updated weights for policy 1, policy_version 24000 (0.0009) +[2023-10-09 13:10:06,571][86121] Updated weights for policy 0, policy_version 23880 (0.0007) +[2023-10-09 13:10:06,939][86121] Updated weights for policy 0, policy_version 23890 (0.0008) +[2023-10-09 13:10:07,305][86121] Updated weights for policy 0, policy_version 23900 (0.0009) +[2023-10-09 13:10:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49053696. Throughput: 0: 1816.4, 1: 1820.2. Samples: 12273838. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) +[2023-10-09 13:10:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:10:09,146][86122] Updated weights for policy 1, policy_version 24010 (0.0008) +[2023-10-09 13:10:09,507][86122] Updated weights for policy 1, policy_version 24020 (0.0008) +[2023-10-09 13:10:09,873][86122] Updated weights for policy 1, policy_version 24030 (0.0009) +[2023-10-09 13:10:10,989][86121] Updated weights for policy 0, policy_version 23910 (0.0009) +[2023-10-09 13:10:11,358][86121] Updated weights for policy 0, policy_version 23920 (0.0008) +[2023-10-09 13:10:11,723][86121] Updated weights for policy 0, policy_version 23930 (0.0008) +[2023-10-09 13:10:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49119232. Throughput: 0: 1819.5, 1: 1818.8. Samples: 12284864. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) +[2023-10-09 13:10:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:10:13,550][86122] Updated weights for policy 1, policy_version 24040 (0.0008) +[2023-10-09 13:10:13,915][86122] Updated weights for policy 1, policy_version 24050 (0.0007) +[2023-10-09 13:10:14,279][86122] Updated weights for policy 1, policy_version 24060 (0.0007) +[2023-10-09 13:10:15,340][86121] Updated weights for policy 0, policy_version 23940 (0.0008) +[2023-10-09 13:10:15,703][86121] Updated weights for policy 0, policy_version 23950 (0.0009) +[2023-10-09 13:10:16,065][86121] Updated weights for policy 0, policy_version 23960 (0.0007) +[2023-10-09 13:10:18,191][86122] Updated weights for policy 1, policy_version 24070 (0.0008) +[2023-10-09 13:10:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49184768. Throughput: 0: 1817.1, 1: 1824.3. Samples: 12306528. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) +[2023-10-09 13:10:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:10:18,548][86122] Updated weights for policy 1, policy_version 24080 (0.0009) +[2023-10-09 13:10:18,898][86122] Updated weights for policy 1, policy_version 24090 (0.0008) +[2023-10-09 13:10:19,794][86121] Updated weights for policy 0, policy_version 23970 (0.0008) +[2023-10-09 13:10:20,167][86121] Updated weights for policy 0, policy_version 23980 (0.0010) +[2023-10-09 13:10:20,526][86121] Updated weights for policy 0, policy_version 23990 (0.0009) +[2023-10-09 13:10:20,894][86121] Updated weights for policy 0, policy_version 24000 (0.0009) +[2023-10-09 13:10:22,448][86122] Updated weights for policy 1, policy_version 24100 (0.0008) +[2023-10-09 13:10:22,812][86122] Updated weights for policy 1, policy_version 24110 (0.0008) +[2023-10-09 13:10:23,182][86122] Updated weights for policy 1, policy_version 24120 (0.0007) +[2023-10-09 13:10:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49250304. Throughput: 0: 1812.9, 1: 1819.9. Samples: 12328714. Policy #0 lag: (min: 13.0, avg: 13.0, max: 16.0) +[2023-10-09 13:10:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:10:24,580][86121] Updated weights for policy 0, policy_version 24010 (0.0009) +[2023-10-09 13:10:24,945][86121] Updated weights for policy 0, policy_version 24020 (0.0009) +[2023-10-09 13:10:25,309][86121] Updated weights for policy 0, policy_version 24030 (0.0010) +[2023-10-09 13:10:26,871][86122] Updated weights for policy 1, policy_version 24130 (0.0009) +[2023-10-09 13:10:27,230][86122] Updated weights for policy 1, policy_version 24140 (0.0011) +[2023-10-09 13:10:27,595][86122] Updated weights for policy 1, policy_version 24150 (0.0010) +[2023-10-09 13:10:27,962][86122] Updated weights for policy 1, policy_version 24160 (0.0010) +[2023-10-09 13:10:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49348608. Throughput: 0: 1812.4, 1: 1824.3. Samples: 12339304. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) +[2023-10-09 13:10:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:10:28,966][86121] Updated weights for policy 0, policy_version 24040 (0.0010) +[2023-10-09 13:10:29,343][86121] Updated weights for policy 0, policy_version 24050 (0.0010) +[2023-10-09 13:10:29,711][86121] Updated weights for policy 0, policy_version 24060 (0.0008) +[2023-10-09 13:10:31,872][86122] Updated weights for policy 1, policy_version 24170 (0.0010) +[2023-10-09 13:10:32,238][86122] Updated weights for policy 1, policy_version 24180 (0.0007) +[2023-10-09 13:10:32,598][86122] Updated weights for policy 1, policy_version 24190 (0.0008) +[2023-10-09 13:10:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49414144. Throughput: 0: 1814.1, 1: 1819.4. Samples: 12361438. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) +[2023-10-09 13:10:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:10:33,414][86121] Updated weights for policy 0, policy_version 24070 (0.0009) +[2023-10-09 13:10:33,770][86121] Updated weights for policy 0, policy_version 24080 (0.0010) +[2023-10-09 13:10:34,134][86121] Updated weights for policy 0, policy_version 24090 (0.0010) +[2023-10-09 13:10:36,034][86122] Updated weights for policy 1, policy_version 24200 (0.0008) +[2023-10-09 13:10:36,400][86122] Updated weights for policy 1, policy_version 24210 (0.0007) +[2023-10-09 13:10:36,764][86122] Updated weights for policy 1, policy_version 24220 (0.0007) +[2023-10-09 13:10:37,796][86121] Updated weights for policy 0, policy_version 24100 (0.0011) +[2023-10-09 13:10:38,159][86121] Updated weights for policy 0, policy_version 24110 (0.0010) +[2023-10-09 13:10:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 49479680. Throughput: 0: 1812.3, 1: 1819.9. Samples: 12382988. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) +[2023-10-09 13:10:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:10:38,524][86121] Updated weights for policy 0, policy_version 24120 (0.0010) +[2023-10-09 13:10:40,562][86122] Updated weights for policy 1, policy_version 24230 (0.0007) +[2023-10-09 13:10:40,920][86122] Updated weights for policy 1, policy_version 24240 (0.0009) +[2023-10-09 13:10:41,290][86122] Updated weights for policy 1, policy_version 24250 (0.0007) +[2023-10-09 13:10:42,415][86121] Updated weights for policy 0, policy_version 24130 (0.0011) +[2023-10-09 13:10:42,800][86121] Updated weights for policy 0, policy_version 24140 (0.0008) +[2023-10-09 13:10:43,164][86121] Updated weights for policy 0, policy_version 24150 (0.0007) +[2023-10-09 13:10:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 49545216. Throughput: 0: 1806.8, 1: 1814.8. Samples: 12393780. Policy #0 lag: (min: 21.0, avg: 21.0, max: 21.0) +[2023-10-09 13:10:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:10:43,530][86121] Updated weights for policy 0, policy_version 24160 (0.0007) +[2023-10-09 13:10:45,154][86122] Updated weights for policy 1, policy_version 24260 (0.0008) +[2023-10-09 13:10:45,513][86122] Updated weights for policy 1, policy_version 24270 (0.0008) +[2023-10-09 13:10:45,885][86122] Updated weights for policy 1, policy_version 24280 (0.0010) +[2023-10-09 13:10:47,190][86121] Updated weights for policy 0, policy_version 24170 (0.0008) +[2023-10-09 13:10:47,556][86121] Updated weights for policy 0, policy_version 24180 (0.0007) +[2023-10-09 13:10:47,927][86121] Updated weights for policy 0, policy_version 24190 (0.0009) +[2023-10-09 13:10:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 49643520. Throughput: 0: 1809.3, 1: 1818.5. Samples: 12415442. Policy #0 lag: (min: 5.0, avg: 18.1, max: 37.0) +[2023-10-09 13:10:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:10:49,581][86122] Updated weights for policy 1, policy_version 24290 (0.0009) +[2023-10-09 13:10:49,946][86122] Updated weights for policy 1, policy_version 24300 (0.0009) +[2023-10-09 13:10:50,312][86122] Updated weights for policy 1, policy_version 24310 (0.0009) +[2023-10-09 13:10:50,672][86122] Updated weights for policy 1, policy_version 24320 (0.0009) +[2023-10-09 13:10:51,812][86121] Updated weights for policy 0, policy_version 24200 (0.0009) +[2023-10-09 13:10:52,177][86121] Updated weights for policy 0, policy_version 24210 (0.0007) +[2023-10-09 13:10:52,545][86121] Updated weights for policy 0, policy_version 24220 (0.0007) +[2023-10-09 13:10:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 49709056. Throughput: 0: 1805.6, 1: 1816.2. Samples: 12436820. Policy #0 lag: (min: 5.0, avg: 18.1, max: 37.0) +[2023-10-09 13:10:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:10:54,130][86122] Updated weights for policy 1, policy_version 24330 (0.0008) +[2023-10-09 13:10:54,498][86122] Updated weights for policy 1, policy_version 24340 (0.0010) +[2023-10-09 13:10:54,866][86122] Updated weights for policy 1, policy_version 24350 (0.0009) +[2023-10-09 13:10:56,289][86121] Updated weights for policy 0, policy_version 24230 (0.0010) +[2023-10-09 13:10:56,659][86121] Updated weights for policy 0, policy_version 24240 (0.0008) +[2023-10-09 13:10:57,025][86121] Updated weights for policy 0, policy_version 24250 (0.0008) +[2023-10-09 13:10:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49774592. Throughput: 0: 1810.9, 1: 1819.7. Samples: 12448242. Policy #0 lag: (min: 5.0, avg: 18.1, max: 37.0) +[2023-10-09 13:10:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:10:58,511][86122] Updated weights for policy 1, policy_version 24360 (0.0008) +[2023-10-09 13:10:58,879][86122] Updated weights for policy 1, policy_version 24370 (0.0008) +[2023-10-09 13:10:59,245][86122] Updated weights for policy 1, policy_version 24380 (0.0009) +[2023-10-09 13:11:00,740][86121] Updated weights for policy 0, policy_version 24260 (0.0009) +[2023-10-09 13:11:01,115][86121] Updated weights for policy 0, policy_version 24270 (0.0010) +[2023-10-09 13:11:01,474][86121] Updated weights for policy 0, policy_version 24280 (0.0011) +[2023-10-09 13:11:02,993][86122] Updated weights for policy 1, policy_version 24390 (0.0009) +[2023-10-09 13:11:03,363][86122] Updated weights for policy 1, policy_version 24400 (0.0011) +[2023-10-09 13:11:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 49840128. Throughput: 0: 1801.2, 1: 1818.7. Samples: 12469420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:11:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:11:03,727][86122] Updated weights for policy 1, policy_version 24410 (0.0009) +[2023-10-09 13:11:05,170][86121] Updated weights for policy 0, policy_version 24290 (0.0008) +[2023-10-09 13:11:05,539][86121] Updated weights for policy 0, policy_version 24300 (0.0010) +[2023-10-09 13:11:05,907][86121] Updated weights for policy 0, policy_version 24310 (0.0007) +[2023-10-09 13:11:06,271][86121] Updated weights for policy 0, policy_version 24320 (0.0008) +[2023-10-09 13:11:07,338][86122] Updated weights for policy 1, policy_version 24420 (0.0008) +[2023-10-09 13:11:07,695][86122] Updated weights for policy 1, policy_version 24430 (0.0007) +[2023-10-09 13:11:08,054][86122] Updated weights for policy 1, policy_version 24440 (0.0008) +[2023-10-09 13:11:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 49938432. Throughput: 0: 1806.0, 1: 1817.6. Samples: 12491778. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:11:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:11:09,866][86121] Updated weights for policy 0, policy_version 24330 (0.0009) +[2023-10-09 13:11:10,231][86121] Updated weights for policy 0, policy_version 24340 (0.0008) +[2023-10-09 13:11:10,602][86121] Updated weights for policy 0, policy_version 24350 (0.0010) +[2023-10-09 13:11:11,981][86122] Updated weights for policy 1, policy_version 24450 (0.0010) +[2023-10-09 13:11:12,348][86122] Updated weights for policy 1, policy_version 24460 (0.0007) +[2023-10-09 13:11:12,710][86122] Updated weights for policy 1, policy_version 24470 (0.0009) +[2023-10-09 13:11:13,082][86122] Updated weights for policy 1, policy_version 24480 (0.0008) +[2023-10-09 13:11:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50003968. Throughput: 0: 1806.3, 1: 1814.7. Samples: 12502248. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:11:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:11:14,314][86121] Updated weights for policy 0, policy_version 24360 (0.0008) +[2023-10-09 13:11:14,674][86121] Updated weights for policy 0, policy_version 24370 (0.0008) +[2023-10-09 13:11:15,045][86121] Updated weights for policy 0, policy_version 24380 (0.0009) +[2023-10-09 13:11:16,721][86122] Updated weights for policy 1, policy_version 24490 (0.0008) +[2023-10-09 13:11:17,083][86122] Updated weights for policy 1, policy_version 24500 (0.0008) +[2023-10-09 13:11:17,442][86122] Updated weights for policy 1, policy_version 24510 (0.0008) +[2023-10-09 13:11:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50069504. Throughput: 0: 1808.9, 1: 1815.6. Samples: 12524544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:11:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:11:18,638][86121] Updated weights for policy 0, policy_version 24390 (0.0008) +[2023-10-09 13:11:19,002][86121] Updated weights for policy 0, policy_version 24400 (0.0009) +[2023-10-09 13:11:19,370][86121] Updated weights for policy 0, policy_version 24410 (0.0009) +[2023-10-09 13:11:21,249][86122] Updated weights for policy 1, policy_version 24520 (0.0008) +[2023-10-09 13:11:21,612][86122] Updated weights for policy 1, policy_version 24530 (0.0010) +[2023-10-09 13:11:21,981][86122] Updated weights for policy 1, policy_version 24540 (0.0010) +[2023-10-09 13:11:23,136][86121] Updated weights for policy 0, policy_version 24420 (0.0008) +[2023-10-09 13:11:23,398][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 50135040. Throughput: 0: 1820.3, 1: 1812.7. Samples: 12546474. Policy #0 lag: (min: 18.0, avg: 44.7, max: 48.0) +[2023-10-09 13:11:23,399][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:11:23,411][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000024544_25133056.pth... +[2023-10-09 13:11:23,458][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000022848_23396352.pth +[2023-10-09 13:11:23,504][86121] Updated weights for policy 0, policy_version 24430 (0.0008) +[2023-10-09 13:11:23,867][86121] Updated weights for policy 0, policy_version 24440 (0.0008) +[2023-10-09 13:11:24,159][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000024448_25034752.pth... +[2023-10-09 13:11:24,198][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000022720_23265280.pth +[2023-10-09 13:11:25,565][86122] Updated weights for policy 1, policy_version 24550 (0.0010) +[2023-10-09 13:11:25,930][86122] Updated weights for policy 1, policy_version 24560 (0.0008) +[2023-10-09 13:11:26,293][86122] Updated weights for policy 1, policy_version 24570 (0.0007) +[2023-10-09 13:11:27,694][86121] Updated weights for policy 0, policy_version 24450 (0.0008) +[2023-10-09 13:11:28,084][86121] Updated weights for policy 0, policy_version 24460 (0.0010) +[2023-10-09 13:11:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 50200576. Throughput: 0: 1813.5, 1: 1817.2. Samples: 12557160. Policy #0 lag: (min: 18.0, avg: 44.7, max: 48.0) +[2023-10-09 13:11:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:11:28,455][86121] Updated weights for policy 0, policy_version 24470 (0.0010) +[2023-10-09 13:11:28,828][86121] Updated weights for policy 0, policy_version 24480 (0.0011) +[2023-10-09 13:11:30,082][86122] Updated weights for policy 1, policy_version 24580 (0.0009) +[2023-10-09 13:11:30,444][86122] Updated weights for policy 1, policy_version 24590 (0.0008) +[2023-10-09 13:11:30,811][86122] Updated weights for policy 1, policy_version 24600 (0.0009) +[2023-10-09 13:11:32,321][86121] Updated weights for policy 0, policy_version 24490 (0.0008) +[2023-10-09 13:11:32,702][86121] Updated weights for policy 0, policy_version 24500 (0.0007) +[2023-10-09 13:11:33,065][86121] Updated weights for policy 0, policy_version 24510 (0.0007) +[2023-10-09 13:11:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50298880. Throughput: 0: 1820.6, 1: 1816.2. Samples: 12579100. Policy #0 lag: (min: 18.0, avg: 44.7, max: 48.0) +[2023-10-09 13:11:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:11:34,558][86122] Updated weights for policy 1, policy_version 24610 (0.0009) +[2023-10-09 13:11:34,916][86122] Updated weights for policy 1, policy_version 24620 (0.0007) +[2023-10-09 13:11:35,275][86122] Updated weights for policy 1, policy_version 24630 (0.0007) +[2023-10-09 13:11:35,630][86122] Updated weights for policy 1, policy_version 24640 (0.0007) +[2023-10-09 13:11:36,735][86121] Updated weights for policy 0, policy_version 24520 (0.0008) +[2023-10-09 13:11:37,105][86121] Updated weights for policy 0, policy_version 24530 (0.0008) +[2023-10-09 13:11:37,467][86121] Updated weights for policy 0, policy_version 24540 (0.0007) +[2023-10-09 13:11:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50364416. Throughput: 0: 1822.7, 1: 1820.0. Samples: 12600742. Policy #0 lag: (min: 3.0, avg: 17.2, max: 35.0) +[2023-10-09 13:11:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:11:39,207][86122] Updated weights for policy 1, policy_version 24650 (0.0010) +[2023-10-09 13:11:39,572][86122] Updated weights for policy 1, policy_version 24660 (0.0010) +[2023-10-09 13:11:39,940][86122] Updated weights for policy 1, policy_version 24670 (0.0011) +[2023-10-09 13:11:41,184][86121] Updated weights for policy 0, policy_version 24550 (0.0009) +[2023-10-09 13:11:41,544][86121] Updated weights for policy 0, policy_version 24560 (0.0011) +[2023-10-09 13:11:41,905][86121] Updated weights for policy 0, policy_version 24570 (0.0010) +[2023-10-09 13:11:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50429952. Throughput: 0: 1823.1, 1: 1819.0. Samples: 12612134. Policy #0 lag: (min: 3.0, avg: 17.2, max: 35.0) +[2023-10-09 13:11:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:11:43,714][86122] Updated weights for policy 1, policy_version 24680 (0.0009) +[2023-10-09 13:11:44,083][86122] Updated weights for policy 1, policy_version 24690 (0.0010) +[2023-10-09 13:11:44,440][86122] Updated weights for policy 1, policy_version 24700 (0.0009) +[2023-10-09 13:11:45,684][86121] Updated weights for policy 0, policy_version 24580 (0.0010) +[2023-10-09 13:11:46,045][86121] Updated weights for policy 0, policy_version 24590 (0.0008) +[2023-10-09 13:11:46,417][86121] Updated weights for policy 0, policy_version 24600 (0.0007) +[2023-10-09 13:11:48,040][86122] Updated weights for policy 1, policy_version 24710 (0.0009) +[2023-10-09 13:11:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 50495488. Throughput: 0: 1822.9, 1: 1823.4. Samples: 12633506. Policy #0 lag: (min: 3.0, avg: 17.2, max: 35.0) +[2023-10-09 13:11:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:11:48,404][86122] Updated weights for policy 1, policy_version 24720 (0.0008) +[2023-10-09 13:11:48,769][86122] Updated weights for policy 1, policy_version 24730 (0.0008) +[2023-10-09 13:11:50,145][86121] Updated weights for policy 0, policy_version 24610 (0.0007) +[2023-10-09 13:11:50,513][86121] Updated weights for policy 0, policy_version 24620 (0.0007) +[2023-10-09 13:11:50,877][86121] Updated weights for policy 0, policy_version 24630 (0.0009) +[2023-10-09 13:11:51,239][86121] Updated weights for policy 0, policy_version 24640 (0.0010) +[2023-10-09 13:11:52,499][86122] Updated weights for policy 1, policy_version 24740 (0.0010) +[2023-10-09 13:11:52,862][86122] Updated weights for policy 1, policy_version 24750 (0.0009) +[2023-10-09 13:11:53,211][86122] Updated weights for policy 1, policy_version 24760 (0.0007) +[2023-10-09 13:11:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 50561024. Throughput: 0: 1821.4, 1: 1823.6. Samples: 12655800. Policy #0 lag: (min: 3.0, avg: 17.2, max: 35.0) +[2023-10-09 13:11:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:11:54,925][86121] Updated weights for policy 0, policy_version 24650 (0.0008) +[2023-10-09 13:11:55,286][86121] Updated weights for policy 0, policy_version 24660 (0.0007) +[2023-10-09 13:11:55,656][86121] Updated weights for policy 0, policy_version 24670 (0.0009) +[2023-10-09 13:11:56,952][86122] Updated weights for policy 1, policy_version 24770 (0.0008) +[2023-10-09 13:11:57,320][86122] Updated weights for policy 1, policy_version 24780 (0.0010) +[2023-10-09 13:11:57,683][86122] Updated weights for policy 1, policy_version 24790 (0.0007) +[2023-10-09 13:11:58,054][86122] Updated weights for policy 1, policy_version 24800 (0.0008) +[2023-10-09 13:11:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 50659328. Throughput: 0: 1816.0, 1: 1824.8. Samples: 12666084. Policy #0 lag: (min: 4.0, avg: 10.9, max: 36.0) +[2023-10-09 13:11:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:11:59,481][86121] Updated weights for policy 0, policy_version 24680 (0.0008) +[2023-10-09 13:11:59,850][86121] Updated weights for policy 0, policy_version 24690 (0.0008) +[2023-10-09 13:12:00,217][86121] Updated weights for policy 0, policy_version 24700 (0.0009) +[2023-10-09 13:12:01,881][86122] Updated weights for policy 1, policy_version 24810 (0.0009) +[2023-10-09 13:12:02,253][86122] Updated weights for policy 1, policy_version 24820 (0.0011) +[2023-10-09 13:12:02,612][86122] Updated weights for policy 1, policy_version 24830 (0.0010) +[2023-10-09 13:12:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 50724864. Throughput: 0: 1808.4, 1: 1832.6. Samples: 12688388. Policy #0 lag: (min: 4.0, avg: 10.9, max: 36.0) +[2023-10-09 13:12:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:12:03,867][86121] Updated weights for policy 0, policy_version 24710 (0.0008) +[2023-10-09 13:12:04,242][86121] Updated weights for policy 0, policy_version 24720 (0.0010) +[2023-10-09 13:12:04,608][86121] Updated weights for policy 0, policy_version 24730 (0.0009) +[2023-10-09 13:12:06,142][86122] Updated weights for policy 1, policy_version 24840 (0.0008) +[2023-10-09 13:12:06,508][86122] Updated weights for policy 1, policy_version 24850 (0.0008) +[2023-10-09 13:12:06,860][86122] Updated weights for policy 1, policy_version 24860 (0.0008) +[2023-10-09 13:12:08,242][86121] Updated weights for policy 0, policy_version 24740 (0.0008) +[2023-10-09 13:12:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50790400. Throughput: 0: 1809.9, 1: 1834.2. Samples: 12710460. Policy #0 lag: (min: 4.0, avg: 10.9, max: 36.0) +[2023-10-09 13:12:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:12:08,623][86121] Updated weights for policy 0, policy_version 24750 (0.0008) +[2023-10-09 13:12:08,986][86121] Updated weights for policy 0, policy_version 24760 (0.0009) +[2023-10-09 13:12:10,493][86122] Updated weights for policy 1, policy_version 24870 (0.0007) +[2023-10-09 13:12:10,857][86122] Updated weights for policy 1, policy_version 24880 (0.0009) +[2023-10-09 13:12:11,220][86122] Updated weights for policy 1, policy_version 24890 (0.0011) +[2023-10-09 13:12:12,682][86121] Updated weights for policy 0, policy_version 24770 (0.0010) +[2023-10-09 13:12:13,064][86121] Updated weights for policy 0, policy_version 24780 (0.0010) +[2023-10-09 13:12:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 50855936. Throughput: 0: 1812.7, 1: 1831.7. Samples: 12721156. Policy #0 lag: (min: 4.0, avg: 10.9, max: 36.0) +[2023-10-09 13:12:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:12:13,435][86121] Updated weights for policy 0, policy_version 24790 (0.0011) +[2023-10-09 13:12:13,801][86121] Updated weights for policy 0, policy_version 24800 (0.0011) +[2023-10-09 13:12:14,741][86122] Updated weights for policy 1, policy_version 24900 (0.0007) +[2023-10-09 13:12:15,104][86122] Updated weights for policy 1, policy_version 24910 (0.0007) +[2023-10-09 13:12:15,470][86122] Updated weights for policy 1, policy_version 24920 (0.0011) +[2023-10-09 13:12:17,461][86121] Updated weights for policy 0, policy_version 24810 (0.0008) +[2023-10-09 13:12:17,830][86121] Updated weights for policy 0, policy_version 24820 (0.0008) +[2023-10-09 13:12:18,203][86121] Updated weights for policy 0, policy_version 24830 (0.0008) +[2023-10-09 13:12:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 50954240. Throughput: 0: 1809.5, 1: 1838.8. Samples: 12743274. Policy #0 lag: (min: 7.0, avg: 7.0, max: 11.0) +[2023-10-09 13:12:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:12:19,083][86122] Updated weights for policy 1, policy_version 24930 (0.0007) +[2023-10-09 13:12:19,439][86122] Updated weights for policy 1, policy_version 24940 (0.0007) +[2023-10-09 13:12:19,804][86122] Updated weights for policy 1, policy_version 24950 (0.0008) +[2023-10-09 13:12:20,166][86122] Updated weights for policy 1, policy_version 24960 (0.0008) +[2023-10-09 13:12:21,927][86121] Updated weights for policy 0, policy_version 24840 (0.0007) +[2023-10-09 13:12:22,288][86121] Updated weights for policy 0, policy_version 24850 (0.0008) +[2023-10-09 13:12:22,654][86121] Updated weights for policy 0, policy_version 24860 (0.0007) +[2023-10-09 13:12:23,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 51019776. Throughput: 0: 1809.2, 1: 1836.3. Samples: 12764786. Policy #0 lag: (min: 7.0, avg: 7.0, max: 11.0) +[2023-10-09 13:12:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:12:23,847][86122] Updated weights for policy 1, policy_version 24970 (0.0007) +[2023-10-09 13:12:24,219][86122] Updated weights for policy 1, policy_version 24980 (0.0009) +[2023-10-09 13:12:24,572][86122] Updated weights for policy 1, policy_version 24990 (0.0010) +[2023-10-09 13:12:26,350][86121] Updated weights for policy 0, policy_version 24870 (0.0008) +[2023-10-09 13:12:26,721][86121] Updated weights for policy 0, policy_version 24880 (0.0009) +[2023-10-09 13:12:27,088][86121] Updated weights for policy 0, policy_version 24890 (0.0007) +[2023-10-09 13:12:28,354][86122] Updated weights for policy 1, policy_version 25000 (0.0011) +[2023-10-09 13:12:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51085312. Throughput: 0: 1807.2, 1: 1838.3. Samples: 12776186. Policy #0 lag: (min: 7.0, avg: 7.0, max: 11.0) +[2023-10-09 13:12:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:12:28,733][86122] Updated weights for policy 1, policy_version 25010 (0.0009) +[2023-10-09 13:12:29,085][86122] Updated weights for policy 1, policy_version 25020 (0.0011) +[2023-10-09 13:12:30,840][86121] Updated weights for policy 0, policy_version 24900 (0.0009) +[2023-10-09 13:12:31,207][86121] Updated weights for policy 0, policy_version 24910 (0.0008) +[2023-10-09 13:12:31,580][86121] Updated weights for policy 0, policy_version 24920 (0.0008) +[2023-10-09 13:12:32,715][86122] Updated weights for policy 1, policy_version 25030 (0.0009) +[2023-10-09 13:12:33,075][86122] Updated weights for policy 1, policy_version 25040 (0.0010) +[2023-10-09 13:12:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51150848. Throughput: 0: 1808.3, 1: 1831.7. Samples: 12797304. Policy #0 lag: (min: 29.0, avg: 41.9, max: 61.0) +[2023-10-09 13:12:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:12:33,439][86122] Updated weights for policy 1, policy_version 25050 (0.0008) +[2023-10-09 13:12:35,314][86121] Updated weights for policy 0, policy_version 24930 (0.0008) +[2023-10-09 13:12:35,682][86121] Updated weights for policy 0, policy_version 24940 (0.0008) +[2023-10-09 13:12:36,052][86121] Updated weights for policy 0, policy_version 24950 (0.0007) +[2023-10-09 13:12:36,422][86121] Updated weights for policy 0, policy_version 24960 (0.0007) +[2023-10-09 13:12:37,185][86122] Updated weights for policy 1, policy_version 25060 (0.0008) +[2023-10-09 13:12:37,552][86122] Updated weights for policy 1, policy_version 25070 (0.0007) +[2023-10-09 13:12:37,918][86122] Updated weights for policy 1, policy_version 25080 (0.0007) +[2023-10-09 13:12:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51249152. Throughput: 0: 1808.9, 1: 1824.1. Samples: 12819288. Policy #0 lag: (min: 29.0, avg: 41.9, max: 61.0) +[2023-10-09 13:12:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:12:40,093][86121] Updated weights for policy 0, policy_version 24970 (0.0007) +[2023-10-09 13:12:40,465][86121] Updated weights for policy 0, policy_version 24980 (0.0008) +[2023-10-09 13:12:40,827][86121] Updated weights for policy 0, policy_version 24990 (0.0007) +[2023-10-09 13:12:41,668][86122] Updated weights for policy 1, policy_version 25090 (0.0010) +[2023-10-09 13:12:42,040][86122] Updated weights for policy 1, policy_version 25100 (0.0009) +[2023-10-09 13:12:42,397][86122] Updated weights for policy 1, policy_version 25110 (0.0011) +[2023-10-09 13:12:42,757][86122] Updated weights for policy 1, policy_version 25120 (0.0011) +[2023-10-09 13:12:43,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51314688. Throughput: 0: 1815.4, 1: 1830.9. Samples: 12830166. Policy #0 lag: (min: 29.0, avg: 41.9, max: 61.0) +[2023-10-09 13:12:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:12:44,637][86121] Updated weights for policy 0, policy_version 25000 (0.0012) +[2023-10-09 13:12:44,994][86121] Updated weights for policy 0, policy_version 25010 (0.0010) +[2023-10-09 13:12:45,365][86121] Updated weights for policy 0, policy_version 25020 (0.0008) +[2023-10-09 13:12:46,350][86122] Updated weights for policy 1, policy_version 25130 (0.0007) +[2023-10-09 13:12:46,728][86122] Updated weights for policy 1, policy_version 25140 (0.0010) +[2023-10-09 13:12:47,083][86122] Updated weights for policy 1, policy_version 25150 (0.0009) +[2023-10-09 13:12:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 51380224. Throughput: 0: 1818.0, 1: 1817.1. Samples: 12851964. Policy #0 lag: (min: 29.0, avg: 41.9, max: 61.0) +[2023-10-09 13:12:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:12:49,034][86121] Updated weights for policy 0, policy_version 25030 (0.0008) +[2023-10-09 13:12:49,398][86121] Updated weights for policy 0, policy_version 25040 (0.0008) +[2023-10-09 13:12:49,766][86121] Updated weights for policy 0, policy_version 25050 (0.0008) +[2023-10-09 13:12:50,840][86122] Updated weights for policy 1, policy_version 25160 (0.0009) +[2023-10-09 13:12:51,200][86122] Updated weights for policy 1, policy_version 25170 (0.0007) +[2023-10-09 13:12:51,560][86122] Updated weights for policy 1, policy_version 25180 (0.0007) +[2023-10-09 13:12:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51445760. Throughput: 0: 1808.8, 1: 1824.3. Samples: 12873952. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) +[2023-10-09 13:12:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:12:53,479][86121] Updated weights for policy 0, policy_version 25060 (0.0007) +[2023-10-09 13:12:53,857][86121] Updated weights for policy 0, policy_version 25070 (0.0008) +[2023-10-09 13:12:54,226][86121] Updated weights for policy 0, policy_version 25080 (0.0007) +[2023-10-09 13:12:55,277][86122] Updated weights for policy 1, policy_version 25190 (0.0010) +[2023-10-09 13:12:55,635][86122] Updated weights for policy 1, policy_version 25200 (0.0010) +[2023-10-09 13:12:55,996][86122] Updated weights for policy 1, policy_version 25210 (0.0010) +[2023-10-09 13:12:57,888][86121] Updated weights for policy 0, policy_version 25090 (0.0007) +[2023-10-09 13:12:58,276][86121] Updated weights for policy 0, policy_version 25100 (0.0008) +[2023-10-09 13:12:58,398][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 51511296. Throughput: 0: 1810.2, 1: 1818.2. Samples: 12884436. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) +[2023-10-09 13:12:58,399][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:12:58,650][86121] Updated weights for policy 0, policy_version 25110 (0.0007) +[2023-10-09 13:12:59,010][86121] Updated weights for policy 0, policy_version 25120 (0.0008) +[2023-10-09 13:12:59,621][86122] Updated weights for policy 1, policy_version 25220 (0.0007) +[2023-10-09 13:12:59,974][86122] Updated weights for policy 1, policy_version 25230 (0.0010) +[2023-10-09 13:13:00,332][86122] Updated weights for policy 1, policy_version 25240 (0.0009) +[2023-10-09 13:13:02,659][86121] Updated weights for policy 0, policy_version 25130 (0.0009) +[2023-10-09 13:13:03,024][86121] Updated weights for policy 0, policy_version 25140 (0.0009) +[2023-10-09 13:13:03,394][86121] Updated weights for policy 0, policy_version 25150 (0.0007) +[2023-10-09 13:13:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 51576832. Throughput: 0: 1816.1, 1: 1820.7. Samples: 12906926. Policy #0 lag: (min: 1.0, avg: 8.7, max: 33.0) +[2023-10-09 13:13:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:13:04,149][86122] Updated weights for policy 1, policy_version 25250 (0.0009) +[2023-10-09 13:13:04,512][86122] Updated weights for policy 1, policy_version 25260 (0.0009) +[2023-10-09 13:13:04,870][86122] Updated weights for policy 1, policy_version 25270 (0.0009) +[2023-10-09 13:13:05,241][86122] Updated weights for policy 1, policy_version 25280 (0.0008) +[2023-10-09 13:13:07,123][86121] Updated weights for policy 0, policy_version 25160 (0.0008) +[2023-10-09 13:13:07,487][86121] Updated weights for policy 0, policy_version 25170 (0.0008) +[2023-10-09 13:13:07,861][86121] Updated weights for policy 0, policy_version 25180 (0.0007) +[2023-10-09 13:13:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51675136. Throughput: 0: 1820.9, 1: 1813.3. Samples: 12928324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:13:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:13:08,986][86122] Updated weights for policy 1, policy_version 25290 (0.0010) +[2023-10-09 13:13:09,351][86122] Updated weights for policy 1, policy_version 25300 (0.0009) +[2023-10-09 13:13:09,712][86122] Updated weights for policy 1, policy_version 25310 (0.0012) +[2023-10-09 13:13:11,447][86121] Updated weights for policy 0, policy_version 25190 (0.0010) +[2023-10-09 13:13:11,814][86121] Updated weights for policy 0, policy_version 25200 (0.0010) +[2023-10-09 13:13:12,180][86121] Updated weights for policy 0, policy_version 25210 (0.0009) +[2023-10-09 13:13:13,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51740672. Throughput: 0: 1816.1, 1: 1810.0. Samples: 12939362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:13:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:13:13,545][86122] Updated weights for policy 1, policy_version 25320 (0.0010) +[2023-10-09 13:13:13,915][86122] Updated weights for policy 1, policy_version 25330 (0.0009) +[2023-10-09 13:13:14,282][86122] Updated weights for policy 1, policy_version 25340 (0.0009) +[2023-10-09 13:13:15,932][86121] Updated weights for policy 0, policy_version 25220 (0.0010) +[2023-10-09 13:13:16,297][86121] Updated weights for policy 0, policy_version 25230 (0.0009) +[2023-10-09 13:13:16,656][86121] Updated weights for policy 0, policy_version 25240 (0.0008) +[2023-10-09 13:13:17,941][86122] Updated weights for policy 1, policy_version 25350 (0.0008) +[2023-10-09 13:13:18,296][86122] Updated weights for policy 1, policy_version 25360 (0.0008) +[2023-10-09 13:13:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 51806208. Throughput: 0: 1818.0, 1: 1811.9. Samples: 12960648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:13:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:13:18,670][86122] Updated weights for policy 1, policy_version 25370 (0.0008) +[2023-10-09 13:13:20,498][86121] Updated weights for policy 0, policy_version 25250 (0.0007) +[2023-10-09 13:13:20,857][86121] Updated weights for policy 0, policy_version 25260 (0.0008) +[2023-10-09 13:13:21,228][86121] Updated weights for policy 0, policy_version 25270 (0.0010) +[2023-10-09 13:13:21,598][86121] Updated weights for policy 0, policy_version 25280 (0.0008) +[2023-10-09 13:13:22,429][86122] Updated weights for policy 1, policy_version 25380 (0.0008) +[2023-10-09 13:13:22,795][86122] Updated weights for policy 1, policy_version 25390 (0.0009) +[2023-10-09 13:13:23,160][86122] Updated weights for policy 1, policy_version 25400 (0.0009) +[2023-10-09 13:13:23,398][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 51871744. Throughput: 0: 1808.6, 1: 1819.8. Samples: 12982566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:13:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000025280_25886720.pth... +[2023-10-09 13:13:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000023584_24150016.pth +[2023-10-09 13:13:23,447][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000025408_26017792.pth... +[2023-10-09 13:13:23,451][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000025280_25886720.pth +[2023-10-09 13:13:23,485][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000023680_24248320.pth +[2023-10-09 13:13:23,491][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000025408_26017792.pth +[2023-10-09 13:13:25,212][86121] Updated weights for policy 0, policy_version 25290 (0.0009) +[2023-10-09 13:13:25,573][86121] Updated weights for policy 0, policy_version 25300 (0.0009) +[2023-10-09 13:13:25,935][86121] Updated weights for policy 0, policy_version 25310 (0.0010) +[2023-10-09 13:13:26,978][86122] Updated weights for policy 1, policy_version 25410 (0.0009) +[2023-10-09 13:13:27,351][86122] Updated weights for policy 1, policy_version 25420 (0.0010) +[2023-10-09 13:13:27,717][86122] Updated weights for policy 1, policy_version 25430 (0.0011) +[2023-10-09 13:13:28,078][86122] Updated weights for policy 1, policy_version 25440 (0.0010) +[2023-10-09 13:13:28,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 51970048. Throughput: 0: 1810.7, 1: 1811.9. Samples: 12993180. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:13:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:29,670][86121] Updated weights for policy 0, policy_version 25320 (0.0008) +[2023-10-09 13:13:30,043][86121] Updated weights for policy 0, policy_version 25330 (0.0008) +[2023-10-09 13:13:30,399][86121] Updated weights for policy 0, policy_version 25340 (0.0008) +[2023-10-09 13:13:32,017][86122] Updated weights for policy 1, policy_version 25450 (0.0007) +[2023-10-09 13:13:32,377][86122] Updated weights for policy 1, policy_version 25460 (0.0009) +[2023-10-09 13:13:32,748][86122] Updated weights for policy 1, policy_version 25470 (0.0008) +[2023-10-09 13:13:33,397][85186] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52035584. Throughput: 0: 1807.4, 1: 1820.8. Samples: 13015234. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:13:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:34,087][86121] Updated weights for policy 0, policy_version 25350 (0.0009) +[2023-10-09 13:13:34,460][86121] Updated weights for policy 0, policy_version 25360 (0.0010) +[2023-10-09 13:13:34,819][86121] Updated weights for policy 0, policy_version 25370 (0.0011) +[2023-10-09 13:13:36,290][86122] Updated weights for policy 1, policy_version 25480 (0.0008) +[2023-10-09 13:13:36,652][86122] Updated weights for policy 1, policy_version 25490 (0.0010) +[2023-10-09 13:13:37,007][86122] Updated weights for policy 1, policy_version 25500 (0.0011) +[2023-10-09 13:13:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52101120. Throughput: 0: 1814.2, 1: 1807.2. Samples: 13036916. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:13:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:38,450][86121] Updated weights for policy 0, policy_version 25380 (0.0008) +[2023-10-09 13:13:38,820][86121] Updated weights for policy 0, policy_version 25390 (0.0009) +[2023-10-09 13:13:39,191][86121] Updated weights for policy 0, policy_version 25400 (0.0009) +[2023-10-09 13:13:40,565][86122] Updated weights for policy 1, policy_version 25510 (0.0011) +[2023-10-09 13:13:40,929][86122] Updated weights for policy 1, policy_version 25520 (0.0008) +[2023-10-09 13:13:41,296][86122] Updated weights for policy 1, policy_version 25530 (0.0009) +[2023-10-09 13:13:43,022][86121] Updated weights for policy 0, policy_version 25410 (0.0009) +[2023-10-09 13:13:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52166656. Throughput: 0: 1816.3, 1: 1819.7. Samples: 13048054. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:13:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:43,434][86121] Updated weights for policy 0, policy_version 25420 (0.0008) +[2023-10-09 13:13:43,797][86121] Updated weights for policy 0, policy_version 25430 (0.0008) +[2023-10-09 13:13:44,165][86121] Updated weights for policy 0, policy_version 25440 (0.0010) +[2023-10-09 13:13:45,105][86122] Updated weights for policy 1, policy_version 25540 (0.0007) +[2023-10-09 13:13:45,465][86122] Updated weights for policy 1, policy_version 25550 (0.0007) +[2023-10-09 13:13:45,827][86122] Updated weights for policy 1, policy_version 25560 (0.0008) +[2023-10-09 13:13:47,896][86121] Updated weights for policy 0, policy_version 25450 (0.0009) +[2023-10-09 13:13:48,265][86121] Updated weights for policy 0, policy_version 25460 (0.0007) +[2023-10-09 13:13:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52232192. Throughput: 0: 1807.1, 1: 1806.4. Samples: 13069534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:13:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:48,625][86121] Updated weights for policy 0, policy_version 25470 (0.0007) +[2023-10-09 13:13:49,601][86122] Updated weights for policy 1, policy_version 25570 (0.0009) +[2023-10-09 13:13:49,959][86122] Updated weights for policy 1, policy_version 25580 (0.0008) +[2023-10-09 13:13:50,332][86122] Updated weights for policy 1, policy_version 25590 (0.0008) +[2023-10-09 13:13:50,688][86122] Updated weights for policy 1, policy_version 25600 (0.0008) +[2023-10-09 13:13:52,431][86121] Updated weights for policy 0, policy_version 25480 (0.0008) +[2023-10-09 13:13:52,805][86121] Updated weights for policy 0, policy_version 25490 (0.0010) +[2023-10-09 13:13:53,175][86121] Updated weights for policy 0, policy_version 25500 (0.0010) +[2023-10-09 13:13:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52330496. Throughput: 0: 1815.2, 1: 1809.8. Samples: 13091452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:13:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:54,309][86122] Updated weights for policy 1, policy_version 25610 (0.0007) +[2023-10-09 13:13:54,673][86122] Updated weights for policy 1, policy_version 25620 (0.0008) +[2023-10-09 13:13:55,034][86122] Updated weights for policy 1, policy_version 25630 (0.0008) +[2023-10-09 13:13:56,959][86121] Updated weights for policy 0, policy_version 25510 (0.0008) +[2023-10-09 13:13:57,332][86121] Updated weights for policy 0, policy_version 25520 (0.0009) +[2023-10-09 13:13:57,702][86121] Updated weights for policy 0, policy_version 25530 (0.0008) +[2023-10-09 13:13:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 52396032. Throughput: 0: 1806.4, 1: 1816.3. Samples: 13102384. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:13:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:13:58,601][86122] Updated weights for policy 1, policy_version 25640 (0.0009) +[2023-10-09 13:13:58,959][86122] Updated weights for policy 1, policy_version 25650 (0.0009) +[2023-10-09 13:13:59,316][86122] Updated weights for policy 1, policy_version 25660 (0.0010) +[2023-10-09 13:14:01,554][86121] Updated weights for policy 0, policy_version 25540 (0.0007) +[2023-10-09 13:14:01,910][86121] Updated weights for policy 0, policy_version 25550 (0.0010) +[2023-10-09 13:14:02,281][86121] Updated weights for policy 0, policy_version 25560 (0.0008) +[2023-10-09 13:14:02,924][86122] Updated weights for policy 1, policy_version 25670 (0.0009) +[2023-10-09 13:14:03,284][86122] Updated weights for policy 1, policy_version 25680 (0.0010) +[2023-10-09 13:14:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52461568. Throughput: 0: 1821.7, 1: 1825.0. Samples: 13124750. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) +[2023-10-09 13:14:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:14:03,644][86122] Updated weights for policy 1, policy_version 25690 (0.0010) +[2023-10-09 13:14:05,718][86121] Updated weights for policy 0, policy_version 25570 (0.0007) +[2023-10-09 13:14:06,088][86121] Updated weights for policy 0, policy_version 25580 (0.0007) +[2023-10-09 13:14:06,442][86121] Updated weights for policy 0, policy_version 25590 (0.0008) +[2023-10-09 13:14:06,808][86121] Updated weights for policy 0, policy_version 25600 (0.0007) +[2023-10-09 13:14:07,353][86122] Updated weights for policy 1, policy_version 25700 (0.0009) +[2023-10-09 13:14:07,716][86122] Updated weights for policy 1, policy_version 25710 (0.0009) +[2023-10-09 13:14:08,079][86122] Updated weights for policy 1, policy_version 25720 (0.0008) +[2023-10-09 13:14:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52559872. Throughput: 0: 1814.6, 1: 1822.4. Samples: 13146230. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) +[2023-10-09 13:14:08,399][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:14:10,624][86121] Updated weights for policy 0, policy_version 25610 (0.0008) +[2023-10-09 13:14:10,992][86121] Updated weights for policy 0, policy_version 25620 (0.0008) +[2023-10-09 13:14:11,352][86121] Updated weights for policy 0, policy_version 25630 (0.0008) +[2023-10-09 13:14:11,650][86122] Updated weights for policy 1, policy_version 25730 (0.0009) +[2023-10-09 13:14:12,017][86122] Updated weights for policy 1, policy_version 25740 (0.0009) +[2023-10-09 13:14:12,381][86122] Updated weights for policy 1, policy_version 25750 (0.0010) +[2023-10-09 13:14:12,745][86122] Updated weights for policy 1, policy_version 25760 (0.0011) +[2023-10-09 13:14:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52625408. Throughput: 0: 1823.1, 1: 1827.2. Samples: 13157442. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) +[2023-10-09 13:14:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:14:14,847][86121] Updated weights for policy 0, policy_version 25640 (0.0010) +[2023-10-09 13:14:15,221][86121] Updated weights for policy 0, policy_version 25650 (0.0009) +[2023-10-09 13:14:15,591][86121] Updated weights for policy 0, policy_version 25660 (0.0009) +[2023-10-09 13:14:16,552][86122] Updated weights for policy 1, policy_version 25770 (0.0007) +[2023-10-09 13:14:16,921][86122] Updated weights for policy 1, policy_version 25780 (0.0007) +[2023-10-09 13:14:17,282][86122] Updated weights for policy 1, policy_version 25790 (0.0007) +[2023-10-09 13:14:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52690944. Throughput: 0: 1812.9, 1: 1820.7. Samples: 13178744. Policy #0 lag: (min: 15.0, avg: 22.0, max: 47.0) +[2023-10-09 13:14:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:14:19,193][86121] Updated weights for policy 0, policy_version 25670 (0.0012) +[2023-10-09 13:14:19,555][86121] Updated weights for policy 0, policy_version 25680 (0.0007) +[2023-10-09 13:14:19,922][86121] Updated weights for policy 0, policy_version 25690 (0.0011) +[2023-10-09 13:14:20,946][86122] Updated weights for policy 1, policy_version 25800 (0.0009) +[2023-10-09 13:14:21,311][86122] Updated weights for policy 1, policy_version 25810 (0.0010) +[2023-10-09 13:14:21,677][86122] Updated weights for policy 1, policy_version 25820 (0.0009) +[2023-10-09 13:14:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52756480. Throughput: 0: 1816.2, 1: 1827.5. Samples: 13200882. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) +[2023-10-09 13:14:23,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:14:23,640][86121] Updated weights for policy 0, policy_version 25700 (0.0008) +[2023-10-09 13:14:24,006][86121] Updated weights for policy 0, policy_version 25710 (0.0008) +[2023-10-09 13:14:24,373][86121] Updated weights for policy 0, policy_version 25720 (0.0007) +[2023-10-09 13:14:25,389][86122] Updated weights for policy 1, policy_version 25830 (0.0009) +[2023-10-09 13:14:25,757][86122] Updated weights for policy 1, policy_version 25840 (0.0007) +[2023-10-09 13:14:26,123][86122] Updated weights for policy 1, policy_version 25850 (0.0009) +[2023-10-09 13:14:28,215][86121] Updated weights for policy 0, policy_version 25730 (0.0008) +[2023-10-09 13:14:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 52822016. Throughput: 0: 1814.2, 1: 1819.6. Samples: 13211576. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) +[2023-10-09 13:14:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:14:28,602][86121] Updated weights for policy 0, policy_version 25740 (0.0008) +[2023-10-09 13:14:28,967][86121] Updated weights for policy 0, policy_version 25750 (0.0008) +[2023-10-09 13:14:29,328][86121] Updated weights for policy 0, policy_version 25760 (0.0007) +[2023-10-09 13:14:29,853][86122] Updated weights for policy 1, policy_version 25860 (0.0007) +[2023-10-09 13:14:30,217][86122] Updated weights for policy 1, policy_version 25870 (0.0007) +[2023-10-09 13:14:30,588][86122] Updated weights for policy 1, policy_version 25880 (0.0007) +[2023-10-09 13:14:32,895][86121] Updated weights for policy 0, policy_version 25770 (0.0008) +[2023-10-09 13:14:33,259][86121] Updated weights for policy 0, policy_version 25780 (0.0009) +[2023-10-09 13:14:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 52887552. Throughput: 0: 1821.9, 1: 1828.6. Samples: 13233806. Policy #0 lag: (min: 31.0, avg: 37.7, max: 63.0) +[2023-10-09 13:14:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:14:33,624][86121] Updated weights for policy 0, policy_version 25790 (0.0010) +[2023-10-09 13:14:34,356][86122] Updated weights for policy 1, policy_version 25890 (0.0009) +[2023-10-09 13:14:34,722][86122] Updated weights for policy 1, policy_version 25900 (0.0008) +[2023-10-09 13:14:35,095][86122] Updated weights for policy 1, policy_version 25910 (0.0009) +[2023-10-09 13:14:35,451][86122] Updated weights for policy 1, policy_version 25920 (0.0008) +[2023-10-09 13:14:37,440][86121] Updated weights for policy 0, policy_version 25800 (0.0008) +[2023-10-09 13:14:37,809][86121] Updated weights for policy 0, policy_version 25810 (0.0008) +[2023-10-09 13:14:38,179][86121] Updated weights for policy 0, policy_version 25820 (0.0010) +[2023-10-09 13:14:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 52985856. Throughput: 0: 1818.3, 1: 1828.8. Samples: 13255568. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:14:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:14:39,133][86122] Updated weights for policy 1, policy_version 25930 (0.0008) +[2023-10-09 13:14:39,480][86122] Updated weights for policy 1, policy_version 25940 (0.0007) +[2023-10-09 13:14:39,838][86122] Updated weights for policy 1, policy_version 25950 (0.0008) +[2023-10-09 13:14:41,783][86121] Updated weights for policy 0, policy_version 25830 (0.0009) +[2023-10-09 13:14:42,165][86121] Updated weights for policy 0, policy_version 25840 (0.0007) +[2023-10-09 13:14:42,529][86121] Updated weights for policy 0, policy_version 25850 (0.0007) +[2023-10-09 13:14:43,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53051392. Throughput: 0: 1818.7, 1: 1825.9. Samples: 13266390. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:14:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:14:43,590][86122] Updated weights for policy 1, policy_version 25960 (0.0008) +[2023-10-09 13:14:43,939][86122] Updated weights for policy 1, policy_version 25970 (0.0009) +[2023-10-09 13:14:44,297][86122] Updated weights for policy 1, policy_version 25980 (0.0010) +[2023-10-09 13:14:46,256][86121] Updated weights for policy 0, policy_version 25860 (0.0008) +[2023-10-09 13:14:46,615][86121] Updated weights for policy 0, policy_version 25870 (0.0008) +[2023-10-09 13:14:46,981][86121] Updated weights for policy 0, policy_version 25880 (0.0008) +[2023-10-09 13:14:47,929][86122] Updated weights for policy 1, policy_version 25990 (0.0008) +[2023-10-09 13:14:48,280][86122] Updated weights for policy 1, policy_version 26000 (0.0008) +[2023-10-09 13:14:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53116928. Throughput: 0: 1812.3, 1: 1814.1. Samples: 13287938. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:14:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:14:48,643][86122] Updated weights for policy 1, policy_version 26010 (0.0008) +[2023-10-09 13:14:50,786][86121] Updated weights for policy 0, policy_version 25890 (0.0009) +[2023-10-09 13:14:51,155][86121] Updated weights for policy 0, policy_version 25900 (0.0011) +[2023-10-09 13:14:51,514][86121] Updated weights for policy 0, policy_version 25910 (0.0008) +[2023-10-09 13:14:51,881][86121] Updated weights for policy 0, policy_version 25920 (0.0009) +[2023-10-09 13:14:52,214][86122] Updated weights for policy 1, policy_version 26020 (0.0008) +[2023-10-09 13:14:52,574][86122] Updated weights for policy 1, policy_version 26030 (0.0008) +[2023-10-09 13:14:52,937][86122] Updated weights for policy 1, policy_version 26040 (0.0010) +[2023-10-09 13:14:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 53215232. Throughput: 0: 1812.1, 1: 1817.3. Samples: 13309556. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:14:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:14:55,566][86121] Updated weights for policy 0, policy_version 25930 (0.0008) +[2023-10-09 13:14:55,933][86121] Updated weights for policy 0, policy_version 25940 (0.0007) +[2023-10-09 13:14:56,301][86121] Updated weights for policy 0, policy_version 25950 (0.0007) +[2023-10-09 13:14:56,726][86122] Updated weights for policy 1, policy_version 26050 (0.0009) +[2023-10-09 13:14:57,090][86122] Updated weights for policy 1, policy_version 26060 (0.0007) +[2023-10-09 13:14:57,454][86122] Updated weights for policy 1, policy_version 26070 (0.0008) +[2023-10-09 13:14:57,817][86122] Updated weights for policy 1, policy_version 26080 (0.0007) +[2023-10-09 13:14:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53280768. Throughput: 0: 1813.5, 1: 1817.0. Samples: 13320816. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:14:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:14:59,931][86121] Updated weights for policy 0, policy_version 25960 (0.0008) +[2023-10-09 13:15:00,296][86121] Updated weights for policy 0, policy_version 25970 (0.0007) +[2023-10-09 13:15:00,668][86121] Updated weights for policy 0, policy_version 25980 (0.0009) +[2023-10-09 13:15:01,444][86122] Updated weights for policy 1, policy_version 26090 (0.0008) +[2023-10-09 13:15:01,809][86122] Updated weights for policy 1, policy_version 26100 (0.0010) +[2023-10-09 13:15:02,170][86122] Updated weights for policy 1, policy_version 26110 (0.0007) +[2023-10-09 13:15:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53346304. Throughput: 0: 1815.8, 1: 1818.6. Samples: 13342294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:15:04,391][86121] Updated weights for policy 0, policy_version 25990 (0.0008) +[2023-10-09 13:15:04,752][86121] Updated weights for policy 0, policy_version 26000 (0.0007) +[2023-10-09 13:15:05,126][86121] Updated weights for policy 0, policy_version 26010 (0.0008) +[2023-10-09 13:15:05,793][86122] Updated weights for policy 1, policy_version 26120 (0.0008) +[2023-10-09 13:15:06,160][86122] Updated weights for policy 1, policy_version 26130 (0.0007) +[2023-10-09 13:15:06,530][86122] Updated weights for policy 1, policy_version 26140 (0.0008) +[2023-10-09 13:15:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53411840. Throughput: 0: 1805.7, 1: 1830.2. Samples: 13364498. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:15:08,923][86121] Updated weights for policy 0, policy_version 26020 (0.0008) +[2023-10-09 13:15:09,290][86121] Updated weights for policy 0, policy_version 26030 (0.0011) +[2023-10-09 13:15:09,660][86121] Updated weights for policy 0, policy_version 26040 (0.0009) +[2023-10-09 13:15:10,119][86122] Updated weights for policy 1, policy_version 26150 (0.0007) +[2023-10-09 13:15:10,489][86122] Updated weights for policy 1, policy_version 26160 (0.0011) +[2023-10-09 13:15:10,848][86122] Updated weights for policy 1, policy_version 26170 (0.0010) +[2023-10-09 13:15:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53477376. Throughput: 0: 1802.3, 1: 1822.4. Samples: 13374690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:15:13,402][86121] Updated weights for policy 0, policy_version 26050 (0.0010) +[2023-10-09 13:15:13,812][86121] Updated weights for policy 0, policy_version 26060 (0.0010) +[2023-10-09 13:15:14,181][86121] Updated weights for policy 0, policy_version 26070 (0.0009) +[2023-10-09 13:15:14,549][86121] Updated weights for policy 0, policy_version 26080 (0.0007) +[2023-10-09 13:15:14,693][86122] Updated weights for policy 1, policy_version 26180 (0.0009) +[2023-10-09 13:15:15,051][86122] Updated weights for policy 1, policy_version 26190 (0.0009) +[2023-10-09 13:15:15,414][86122] Updated weights for policy 1, policy_version 26200 (0.0009) +[2023-10-09 13:15:18,270][86121] Updated weights for policy 0, policy_version 26090 (0.0007) +[2023-10-09 13:15:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 53542912. Throughput: 0: 1798.8, 1: 1830.5. Samples: 13397124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:15:18,631][86121] Updated weights for policy 0, policy_version 26100 (0.0007) +[2023-10-09 13:15:18,990][86122] Updated weights for policy 1, policy_version 26210 (0.0008) +[2023-10-09 13:15:18,993][86121] Updated weights for policy 0, policy_version 26110 (0.0008) +[2023-10-09 13:15:19,357][86122] Updated weights for policy 1, policy_version 26220 (0.0008) +[2023-10-09 13:15:19,723][86122] Updated weights for policy 1, policy_version 26230 (0.0009) +[2023-10-09 13:15:20,091][86122] Updated weights for policy 1, policy_version 26240 (0.0008) +[2023-10-09 13:15:22,766][86121] Updated weights for policy 0, policy_version 26120 (0.0009) +[2023-10-09 13:15:23,123][86121] Updated weights for policy 0, policy_version 26130 (0.0010) +[2023-10-09 13:15:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 53608448. Throughput: 0: 1809.7, 1: 1836.3. Samples: 13419638. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:15:23,489][86121] Updated weights for policy 0, policy_version 26140 (0.0009) +[2023-10-09 13:15:23,628][86122] Updated weights for policy 1, policy_version 26250 (0.0007) +[2023-10-09 13:15:23,638][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000026144_26771456.pth... +[2023-10-09 13:15:23,666][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000024448_25034752.pth +[2023-10-09 13:15:23,990][86122] Updated weights for policy 1, policy_version 26260 (0.0009) +[2023-10-09 13:15:24,352][86122] Updated weights for policy 1, policy_version 26270 (0.0007) +[2023-10-09 13:15:24,422][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000026272_26902528.pth... +[2023-10-09 13:15:24,462][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000024544_25133056.pth +[2023-10-09 13:15:27,218][86121] Updated weights for policy 0, policy_version 26150 (0.0009) +[2023-10-09 13:15:27,595][86121] Updated weights for policy 0, policy_version 26160 (0.0009) +[2023-10-09 13:15:27,961][86121] Updated weights for policy 0, policy_version 26170 (0.0009) +[2023-10-09 13:15:28,171][86122] Updated weights for policy 1, policy_version 26280 (0.0008) +[2023-10-09 13:15:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53706752. Throughput: 0: 1802.9, 1: 1835.8. Samples: 13430130. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) +[2023-10-09 13:15:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:15:28,537][86122] Updated weights for policy 1, policy_version 26290 (0.0008) +[2023-10-09 13:15:28,903][86122] Updated weights for policy 1, policy_version 26300 (0.0008) +[2023-10-09 13:15:31,819][86121] Updated weights for policy 0, policy_version 26180 (0.0007) +[2023-10-09 13:15:32,187][86121] Updated weights for policy 0, policy_version 26190 (0.0008) +[2023-10-09 13:15:32,554][86121] Updated weights for policy 0, policy_version 26200 (0.0010) +[2023-10-09 13:15:32,693][86122] Updated weights for policy 1, policy_version 26310 (0.0008) +[2023-10-09 13:15:33,049][86122] Updated weights for policy 1, policy_version 26320 (0.0009) +[2023-10-09 13:15:33,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53772288. Throughput: 0: 1814.8, 1: 1834.5. Samples: 13452158. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) +[2023-10-09 13:15:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:15:33,412][86122] Updated weights for policy 1, policy_version 26330 (0.0008) +[2023-10-09 13:15:36,153][86121] Updated weights for policy 0, policy_version 26210 (0.0009) +[2023-10-09 13:15:36,510][86121] Updated weights for policy 0, policy_version 26220 (0.0007) +[2023-10-09 13:15:36,878][86121] Updated weights for policy 0, policy_version 26230 (0.0008) +[2023-10-09 13:15:37,200][86122] Updated weights for policy 1, policy_version 26340 (0.0009) +[2023-10-09 13:15:37,246][86121] Updated weights for policy 0, policy_version 26240 (0.0008) +[2023-10-09 13:15:37,563][86122] Updated weights for policy 1, policy_version 26350 (0.0009) +[2023-10-09 13:15:37,925][86122] Updated weights for policy 1, policy_version 26360 (0.0011) +[2023-10-09 13:15:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 53870592. Throughput: 0: 1804.8, 1: 1824.9. Samples: 13472894. Policy #0 lag: (min: 28.0, avg: 28.0, max: 28.0) +[2023-10-09 13:15:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:15:41,129][86121] Updated weights for policy 0, policy_version 26250 (0.0008) +[2023-10-09 13:15:41,490][86121] Updated weights for policy 0, policy_version 26260 (0.0009) +[2023-10-09 13:15:41,524][86122] Updated weights for policy 1, policy_version 26370 (0.0011) +[2023-10-09 13:15:41,858][86121] Updated weights for policy 0, policy_version 26270 (0.0009) +[2023-10-09 13:15:41,895][86122] Updated weights for policy 1, policy_version 26380 (0.0009) +[2023-10-09 13:15:42,262][86122] Updated weights for policy 1, policy_version 26390 (0.0010) +[2023-10-09 13:15:42,620][86122] Updated weights for policy 1, policy_version 26400 (0.0008) +[2023-10-09 13:15:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 53936128. Throughput: 0: 1819.0, 1: 1828.5. Samples: 13484952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:15:45,683][86121] Updated weights for policy 0, policy_version 26280 (0.0009) +[2023-10-09 13:15:46,045][86121] Updated weights for policy 0, policy_version 26290 (0.0007) +[2023-10-09 13:15:46,408][86121] Updated weights for policy 0, policy_version 26300 (0.0007) +[2023-10-09 13:15:46,450][86122] Updated weights for policy 1, policy_version 26410 (0.0010) +[2023-10-09 13:15:46,822][86122] Updated weights for policy 1, policy_version 26420 (0.0008) +[2023-10-09 13:15:47,191][86122] Updated weights for policy 1, policy_version 26430 (0.0007) +[2023-10-09 13:15:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54001664. Throughput: 0: 1799.0, 1: 1819.6. Samples: 13505130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:15:49,952][86121] Updated weights for policy 0, policy_version 26310 (0.0007) +[2023-10-09 13:15:50,319][86121] Updated weights for policy 0, policy_version 26320 (0.0009) +[2023-10-09 13:15:50,680][86121] Updated weights for policy 0, policy_version 26330 (0.0008) +[2023-10-09 13:15:50,909][86122] Updated weights for policy 1, policy_version 26440 (0.0009) +[2023-10-09 13:15:51,267][86122] Updated weights for policy 1, policy_version 26450 (0.0009) +[2023-10-09 13:15:51,622][86122] Updated weights for policy 1, policy_version 26460 (0.0007) +[2023-10-09 13:15:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54067200. Throughput: 0: 1805.9, 1: 1810.2. Samples: 13527222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 13:15:54,435][86121] Updated weights for policy 0, policy_version 26340 (0.0009) +[2023-10-09 13:15:54,803][86121] Updated weights for policy 0, policy_version 26350 (0.0009) +[2023-10-09 13:15:55,176][86121] Updated weights for policy 0, policy_version 26360 (0.0009) +[2023-10-09 13:15:55,336][86122] Updated weights for policy 1, policy_version 26470 (0.0009) +[2023-10-09 13:15:55,698][86122] Updated weights for policy 1, policy_version 26480 (0.0009) +[2023-10-09 13:15:56,066][86122] Updated weights for policy 1, policy_version 26490 (0.0009) +[2023-10-09 13:15:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54132736. Throughput: 0: 1810.7, 1: 1814.8. Samples: 13537836. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:15:58,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 13:15:58,991][86121] Updated weights for policy 0, policy_version 26370 (0.0009) +[2023-10-09 13:15:59,362][86121] Updated weights for policy 0, policy_version 26380 (0.0010) +[2023-10-09 13:15:59,727][86121] Updated weights for policy 0, policy_version 26390 (0.0010) +[2023-10-09 13:15:59,935][86122] Updated weights for policy 1, policy_version 26500 (0.0007) +[2023-10-09 13:16:00,096][86121] Updated weights for policy 0, policy_version 26400 (0.0007) +[2023-10-09 13:16:00,295][86122] Updated weights for policy 1, policy_version 26510 (0.0009) +[2023-10-09 13:16:00,653][86122] Updated weights for policy 1, policy_version 26520 (0.0008) +[2023-10-09 13:16:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54198272. Throughput: 0: 1812.8, 1: 1809.2. Samples: 13560118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:16:03,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 13:16:03,771][86121] Updated weights for policy 0, policy_version 26410 (0.0010) +[2023-10-09 13:16:04,152][86121] Updated weights for policy 0, policy_version 26420 (0.0008) +[2023-10-09 13:16:04,505][86121] Updated weights for policy 0, policy_version 26430 (0.0007) +[2023-10-09 13:16:04,558][86122] Updated weights for policy 1, policy_version 26530 (0.0007) +[2023-10-09 13:16:04,921][86122] Updated weights for policy 1, policy_version 26540 (0.0009) +[2023-10-09 13:16:05,287][86122] Updated weights for policy 1, policy_version 26550 (0.0008) +[2023-10-09 13:16:05,654][86122] Updated weights for policy 1, policy_version 26560 (0.0008) +[2023-10-09 13:16:08,113][86121] Updated weights for policy 0, policy_version 26440 (0.0011) +[2023-10-09 13:16:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54263808. Throughput: 0: 1818.6, 1: 1805.4. Samples: 13582718. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) +[2023-10-09 13:16:08,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 13:16:08,473][86121] Updated weights for policy 0, policy_version 26450 (0.0010) +[2023-10-09 13:16:08,846][86121] Updated weights for policy 0, policy_version 26460 (0.0008) +[2023-10-09 13:16:09,189][86122] Updated weights for policy 1, policy_version 26570 (0.0009) +[2023-10-09 13:16:09,552][86122] Updated weights for policy 1, policy_version 26580 (0.0011) +[2023-10-09 13:16:09,916][86122] Updated weights for policy 1, policy_version 26590 (0.0009) +[2023-10-09 13:16:12,439][86121] Updated weights for policy 0, policy_version 26470 (0.0009) +[2023-10-09 13:16:12,800][86121] Updated weights for policy 0, policy_version 26480 (0.0010) +[2023-10-09 13:16:13,164][86121] Updated weights for policy 0, policy_version 26490 (0.0011) +[2023-10-09 13:16:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54362112. Throughput: 0: 1805.7, 1: 1805.5. Samples: 13592632. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) +[2023-10-09 13:16:13,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:16:13,522][86122] Updated weights for policy 1, policy_version 26600 (0.0007) +[2023-10-09 13:16:13,893][86122] Updated weights for policy 1, policy_version 26610 (0.0009) +[2023-10-09 13:16:14,251][86122] Updated weights for policy 1, policy_version 26620 (0.0009) +[2023-10-09 13:16:16,967][86121] Updated weights for policy 0, policy_version 26500 (0.0010) +[2023-10-09 13:16:17,333][86121] Updated weights for policy 0, policy_version 26510 (0.0007) +[2023-10-09 13:16:17,704][86121] Updated weights for policy 0, policy_version 26520 (0.0007) +[2023-10-09 13:16:17,928][86122] Updated weights for policy 1, policy_version 26630 (0.0008) +[2023-10-09 13:16:18,286][86122] Updated weights for policy 1, policy_version 26640 (0.0008) +[2023-10-09 13:16:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54427648. Throughput: 0: 1814.7, 1: 1809.0. Samples: 13615224. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) +[2023-10-09 13:16:18,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:16:18,644][86122] Updated weights for policy 1, policy_version 26650 (0.0009) +[2023-10-09 13:16:21,384][86121] Updated weights for policy 0, policy_version 26530 (0.0010) +[2023-10-09 13:16:21,752][86121] Updated weights for policy 0, policy_version 26540 (0.0010) +[2023-10-09 13:16:22,113][86121] Updated weights for policy 0, policy_version 26550 (0.0008) +[2023-10-09 13:16:22,377][86122] Updated weights for policy 1, policy_version 26660 (0.0008) +[2023-10-09 13:16:22,480][86121] Updated weights for policy 0, policy_version 26560 (0.0007) +[2023-10-09 13:16:22,742][86122] Updated weights for policy 1, policy_version 26670 (0.0007) +[2023-10-09 13:16:23,103][86122] Updated weights for policy 1, policy_version 26680 (0.0007) +[2023-10-09 13:16:23,397][85186] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 54525952. Throughput: 0: 1809.8, 1: 1811.9. Samples: 13635868. Policy #0 lag: (min: 31.0, avg: 39.6, max: 63.0) +[2023-10-09 13:16:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:16:26,294][86121] Updated weights for policy 0, policy_version 26570 (0.0007) +[2023-10-09 13:16:26,619][86122] Updated weights for policy 1, policy_version 26690 (0.0009) +[2023-10-09 13:16:26,661][86121] Updated weights for policy 0, policy_version 26580 (0.0009) +[2023-10-09 13:16:26,987][86122] Updated weights for policy 1, policy_version 26700 (0.0008) +[2023-10-09 13:16:27,027][86121] Updated weights for policy 0, policy_version 26590 (0.0008) +[2023-10-09 13:16:27,346][86122] Updated weights for policy 1, policy_version 26710 (0.0008) +[2023-10-09 13:16:27,706][86122] Updated weights for policy 1, policy_version 26720 (0.0007) +[2023-10-09 13:16:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54591488. Throughput: 0: 1813.3, 1: 1810.1. Samples: 13648008. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 13:16:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:16:30,621][86121] Updated weights for policy 0, policy_version 26600 (0.0007) +[2023-10-09 13:16:30,994][86121] Updated weights for policy 0, policy_version 26610 (0.0008) +[2023-10-09 13:16:31,359][86121] Updated weights for policy 0, policy_version 26620 (0.0009) +[2023-10-09 13:16:31,599][86122] Updated weights for policy 1, policy_version 26730 (0.0008) +[2023-10-09 13:16:31,961][86122] Updated weights for policy 1, policy_version 26740 (0.0008) +[2023-10-09 13:16:32,328][86122] Updated weights for policy 1, policy_version 26750 (0.0009) +[2023-10-09 13:16:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 54657024. Throughput: 0: 1811.1, 1: 1821.6. Samples: 13668602. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 13:16:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:16:35,053][86121] Updated weights for policy 0, policy_version 26630 (0.0007) +[2023-10-09 13:16:35,422][86121] Updated weights for policy 0, policy_version 26640 (0.0009) +[2023-10-09 13:16:35,788][86121] Updated weights for policy 0, policy_version 26650 (0.0010) +[2023-10-09 13:16:36,016][86122] Updated weights for policy 1, policy_version 26760 (0.0008) +[2023-10-09 13:16:36,383][86122] Updated weights for policy 1, policy_version 26770 (0.0008) +[2023-10-09 13:16:36,745][86122] Updated weights for policy 1, policy_version 26780 (0.0007) +[2023-10-09 13:16:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54722560. Throughput: 0: 1811.6, 1: 1823.9. Samples: 13690818. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 13:16:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:16:39,534][86121] Updated weights for policy 0, policy_version 26660 (0.0009) +[2023-10-09 13:16:39,903][86121] Updated weights for policy 0, policy_version 26670 (0.0009) +[2023-10-09 13:16:40,266][86121] Updated weights for policy 0, policy_version 26680 (0.0008) +[2023-10-09 13:16:40,357][86122] Updated weights for policy 1, policy_version 26790 (0.0008) +[2023-10-09 13:16:40,713][86122] Updated weights for policy 1, policy_version 26800 (0.0009) +[2023-10-09 13:16:41,075][86122] Updated weights for policy 1, policy_version 26810 (0.0011) +[2023-10-09 13:16:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 54788096. Throughput: 0: 1807.9, 1: 1825.0. Samples: 13701316. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 13:16:43,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:16:44,104][86121] Updated weights for policy 0, policy_version 26690 (0.0009) +[2023-10-09 13:16:44,467][86121] Updated weights for policy 0, policy_version 26700 (0.0009) +[2023-10-09 13:16:44,767][86122] Updated weights for policy 1, policy_version 26820 (0.0009) +[2023-10-09 13:16:44,833][86121] Updated weights for policy 0, policy_version 26710 (0.0008) +[2023-10-09 13:16:45,127][86122] Updated weights for policy 1, policy_version 26830 (0.0008) +[2023-10-09 13:16:45,200][86121] Updated weights for policy 0, policy_version 26720 (0.0008) +[2023-10-09 13:16:45,489][86122] Updated weights for policy 1, policy_version 26840 (0.0008) +[2023-10-09 13:16:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 54853632. Throughput: 0: 1801.4, 1: 1825.8. Samples: 13723344. Policy #0 lag: (min: 24.0, avg: 49.3, max: 56.0) +[2023-10-09 13:16:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:16:49,072][86121] Updated weights for policy 0, policy_version 26730 (0.0009) +[2023-10-09 13:16:49,090][86122] Updated weights for policy 1, policy_version 26850 (0.0010) +[2023-10-09 13:16:49,436][86121] Updated weights for policy 0, policy_version 26740 (0.0007) +[2023-10-09 13:16:49,461][86122] Updated weights for policy 1, policy_version 26860 (0.0008) +[2023-10-09 13:16:49,804][86121] Updated weights for policy 0, policy_version 26750 (0.0008) +[2023-10-09 13:16:49,831][86122] Updated weights for policy 1, policy_version 26870 (0.0007) +[2023-10-09 13:16:50,193][86122] Updated weights for policy 1, policy_version 26880 (0.0009) +[2023-10-09 13:16:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 54919168. Throughput: 0: 1803.5, 1: 1825.9. Samples: 13746044. Policy #0 lag: (min: 24.0, avg: 49.3, max: 56.0) +[2023-10-09 13:16:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:16:53,411][86121] Updated weights for policy 0, policy_version 26760 (0.0008) +[2023-10-09 13:16:53,778][86121] Updated weights for policy 0, policy_version 26770 (0.0009) +[2023-10-09 13:16:54,002][86122] Updated weights for policy 1, policy_version 26890 (0.0008) +[2023-10-09 13:16:54,143][86121] Updated weights for policy 0, policy_version 26780 (0.0008) +[2023-10-09 13:16:54,363][86122] Updated weights for policy 1, policy_version 26900 (0.0008) +[2023-10-09 13:16:54,736][86122] Updated weights for policy 1, policy_version 26910 (0.0007) +[2023-10-09 13:16:57,856][86121] Updated weights for policy 0, policy_version 26790 (0.0008) +[2023-10-09 13:16:58,226][86121] Updated weights for policy 0, policy_version 26800 (0.0009) +[2023-10-09 13:16:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 54984704. Throughput: 0: 1802.8, 1: 1825.3. Samples: 13755894. Policy #0 lag: (min: 24.0, avg: 49.3, max: 56.0) +[2023-10-09 13:16:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:16:58,511][86122] Updated weights for policy 1, policy_version 26920 (0.0008) +[2023-10-09 13:16:58,586][86121] Updated weights for policy 0, policy_version 26810 (0.0008) +[2023-10-09 13:16:58,870][86122] Updated weights for policy 1, policy_version 26930 (0.0007) +[2023-10-09 13:16:59,235][86122] Updated weights for policy 1, policy_version 26940 (0.0007) +[2023-10-09 13:17:02,380][86121] Updated weights for policy 0, policy_version 26820 (0.0008) +[2023-10-09 13:17:02,757][86121] Updated weights for policy 0, policy_version 26830 (0.0008) +[2023-10-09 13:17:02,982][86122] Updated weights for policy 1, policy_version 26950 (0.0007) +[2023-10-09 13:17:03,116][86121] Updated weights for policy 0, policy_version 26840 (0.0008) +[2023-10-09 13:17:03,351][86122] Updated weights for policy 1, policy_version 26960 (0.0008) +[2023-10-09 13:17:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55050240. Throughput: 0: 1805.2, 1: 1822.2. Samples: 13778458. Policy #0 lag: (min: 24.0, avg: 49.3, max: 56.0) +[2023-10-09 13:17:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:17:03,721][86122] Updated weights for policy 1, policy_version 26970 (0.0009) +[2023-10-09 13:17:06,925][86121] Updated weights for policy 0, policy_version 26850 (0.0008) +[2023-10-09 13:17:07,294][86121] Updated weights for policy 0, policy_version 26860 (0.0007) +[2023-10-09 13:17:07,430][86122] Updated weights for policy 1, policy_version 26980 (0.0009) +[2023-10-09 13:17:07,653][86121] Updated weights for policy 0, policy_version 26870 (0.0008) +[2023-10-09 13:17:07,793][86122] Updated weights for policy 1, policy_version 26990 (0.0008) +[2023-10-09 13:17:08,023][86121] Updated weights for policy 0, policy_version 26880 (0.0008) +[2023-10-09 13:17:08,158][86122] Updated weights for policy 1, policy_version 27000 (0.0007) +[2023-10-09 13:17:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 55148544. Throughput: 0: 1798.4, 1: 1827.3. Samples: 13799024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:17:11,878][86122] Updated weights for policy 1, policy_version 27010 (0.0008) +[2023-10-09 13:17:11,901][86121] Updated weights for policy 0, policy_version 26890 (0.0008) +[2023-10-09 13:17:12,243][86122] Updated weights for policy 1, policy_version 27020 (0.0008) +[2023-10-09 13:17:12,268][86121] Updated weights for policy 0, policy_version 26900 (0.0007) +[2023-10-09 13:17:12,611][86122] Updated weights for policy 1, policy_version 27030 (0.0008) +[2023-10-09 13:17:12,641][86121] Updated weights for policy 0, policy_version 26910 (0.0008) +[2023-10-09 13:17:12,969][86122] Updated weights for policy 1, policy_version 27040 (0.0008) +[2023-10-09 13:17:13,397][85186] Fps is (10 sec: 19660.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 55246848. Throughput: 0: 1791.7, 1: 1822.4. Samples: 13810644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:17:16,456][86121] Updated weights for policy 0, policy_version 26920 (0.0009) +[2023-10-09 13:17:16,759][86122] Updated weights for policy 1, policy_version 27050 (0.0010) +[2023-10-09 13:17:16,836][86121] Updated weights for policy 0, policy_version 26930 (0.0008) +[2023-10-09 13:17:17,126][86122] Updated weights for policy 1, policy_version 27060 (0.0010) +[2023-10-09 13:17:17,197][86121] Updated weights for policy 0, policy_version 26940 (0.0008) +[2023-10-09 13:17:17,479][86122] Updated weights for policy 1, policy_version 27070 (0.0007) +[2023-10-09 13:17:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 55312384. Throughput: 0: 1800.9, 1: 1823.1. Samples: 13831684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:17:20,922][86121] Updated weights for policy 0, policy_version 26950 (0.0009) +[2023-10-09 13:17:21,098][86122] Updated weights for policy 1, policy_version 27080 (0.0008) +[2023-10-09 13:17:21,278][86121] Updated weights for policy 0, policy_version 26960 (0.0008) +[2023-10-09 13:17:21,461][86122] Updated weights for policy 1, policy_version 27090 (0.0007) +[2023-10-09 13:17:21,647][86121] Updated weights for policy 0, policy_version 26970 (0.0007) +[2023-10-09 13:17:21,831][86122] Updated weights for policy 1, policy_version 27100 (0.0009) +[2023-10-09 13:17:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 55377920. Throughput: 0: 1783.2, 1: 1820.0. Samples: 13852958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:17:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000026976_27623424.pth... +[2023-10-09 13:17:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000027104_27754496.pth... +[2023-10-09 13:17:23,443][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000025408_26017792.pth +[2023-10-09 13:17:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000025280_25886720.pth +[2023-10-09 13:17:25,338][86121] Updated weights for policy 0, policy_version 26980 (0.0009) +[2023-10-09 13:17:25,484][86122] Updated weights for policy 1, policy_version 27110 (0.0011) +[2023-10-09 13:17:25,709][86121] Updated weights for policy 0, policy_version 26990 (0.0009) +[2023-10-09 13:17:25,848][86122] Updated weights for policy 1, policy_version 27120 (0.0007) +[2023-10-09 13:17:26,066][86121] Updated weights for policy 0, policy_version 27000 (0.0009) +[2023-10-09 13:17:26,213][86122] Updated weights for policy 1, policy_version 27130 (0.0008) +[2023-10-09 13:17:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 55443456. Throughput: 0: 1802.4, 1: 1824.3. Samples: 13864518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:17:29,700][86121] Updated weights for policy 0, policy_version 27010 (0.0008) +[2023-10-09 13:17:29,939][86122] Updated weights for policy 1, policy_version 27140 (0.0007) +[2023-10-09 13:17:30,064][86121] Updated weights for policy 0, policy_version 27020 (0.0009) +[2023-10-09 13:17:30,296][86122] Updated weights for policy 1, policy_version 27150 (0.0008) +[2023-10-09 13:17:30,436][86121] Updated weights for policy 0, policy_version 27030 (0.0009) +[2023-10-09 13:17:30,663][86122] Updated weights for policy 1, policy_version 27160 (0.0007) +[2023-10-09 13:17:30,794][86121] Updated weights for policy 0, policy_version 27040 (0.0007) +[2023-10-09 13:17:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 55508992. Throughput: 0: 1795.1, 1: 1816.1. Samples: 13885850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:17:34,246][86122] Updated weights for policy 1, policy_version 27170 (0.0008) +[2023-10-09 13:17:34,607][86122] Updated weights for policy 1, policy_version 27180 (0.0008) +[2023-10-09 13:17:34,727][86121] Updated weights for policy 0, policy_version 27050 (0.0009) +[2023-10-09 13:17:34,966][86122] Updated weights for policy 1, policy_version 27190 (0.0008) +[2023-10-09 13:17:35,107][86121] Updated weights for policy 0, policy_version 27060 (0.0009) +[2023-10-09 13:17:35,339][86122] Updated weights for policy 1, policy_version 27200 (0.0009) +[2023-10-09 13:17:35,471][86121] Updated weights for policy 0, policy_version 27070 (0.0009) +[2023-10-09 13:17:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 55574528. Throughput: 0: 1794.3, 1: 1816.4. Samples: 13908524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:38,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:17:39,046][86122] Updated weights for policy 1, policy_version 27210 (0.0007) +[2023-10-09 13:17:39,093][86121] Updated weights for policy 0, policy_version 27080 (0.0008) +[2023-10-09 13:17:39,407][86122] Updated weights for policy 1, policy_version 27220 (0.0007) +[2023-10-09 13:17:39,454][86121] Updated weights for policy 0, policy_version 27090 (0.0009) +[2023-10-09 13:17:39,763][86122] Updated weights for policy 1, policy_version 27230 (0.0007) +[2023-10-09 13:17:39,821][86121] Updated weights for policy 0, policy_version 27100 (0.0007) +[2023-10-09 13:17:43,346][86122] Updated weights for policy 1, policy_version 27240 (0.0007) +[2023-10-09 13:17:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55640064. Throughput: 0: 1794.3, 1: 1818.3. Samples: 13918462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:17:43,544][86121] Updated weights for policy 0, policy_version 27110 (0.0008) +[2023-10-09 13:17:43,712][86122] Updated weights for policy 1, policy_version 27250 (0.0008) +[2023-10-09 13:17:43,908][86121] Updated weights for policy 0, policy_version 27120 (0.0008) +[2023-10-09 13:17:44,075][86122] Updated weights for policy 1, policy_version 27260 (0.0008) +[2023-10-09 13:17:44,266][86121] Updated weights for policy 0, policy_version 27130 (0.0008) +[2023-10-09 13:17:47,950][86122] Updated weights for policy 1, policy_version 27270 (0.0010) +[2023-10-09 13:17:48,242][86121] Updated weights for policy 0, policy_version 27140 (0.0008) +[2023-10-09 13:17:48,310][86122] Updated weights for policy 1, policy_version 27280 (0.0010) +[2023-10-09 13:17:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 55705600. Throughput: 0: 1792.9, 1: 1823.2. Samples: 13941184. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 13:17:48,606][86121] Updated weights for policy 0, policy_version 27150 (0.0008) +[2023-10-09 13:17:48,671][86122] Updated weights for policy 1, policy_version 27290 (0.0008) +[2023-10-09 13:17:48,967][86121] Updated weights for policy 0, policy_version 27160 (0.0008) +[2023-10-09 13:17:52,450][86122] Updated weights for policy 1, policy_version 27300 (0.0008) +[2023-10-09 13:17:52,687][86121] Updated weights for policy 0, policy_version 27170 (0.0010) +[2023-10-09 13:17:52,806][86122] Updated weights for policy 1, policy_version 27310 (0.0008) +[2023-10-09 13:17:53,047][86121] Updated weights for policy 0, policy_version 27180 (0.0007) +[2023-10-09 13:17:53,178][86122] Updated weights for policy 1, policy_version 27320 (0.0007) +[2023-10-09 13:17:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 55771136. Throughput: 0: 1814.9, 1: 1819.5. Samples: 13962568. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:17:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:17:53,409][86121] Updated weights for policy 0, policy_version 27190 (0.0009) +[2023-10-09 13:17:53,783][86121] Updated weights for policy 0, policy_version 27200 (0.0008) +[2023-10-09 13:17:56,852][86122] Updated weights for policy 1, policy_version 27330 (0.0009) +[2023-10-09 13:17:57,217][86122] Updated weights for policy 1, policy_version 27340 (0.0009) +[2023-10-09 13:17:57,509][86121] Updated weights for policy 0, policy_version 27210 (0.0007) +[2023-10-09 13:17:57,576][86122] Updated weights for policy 1, policy_version 27350 (0.0008) +[2023-10-09 13:17:57,869][86121] Updated weights for policy 0, policy_version 27220 (0.0007) +[2023-10-09 13:17:57,935][86122] Updated weights for policy 1, policy_version 27360 (0.0007) +[2023-10-09 13:17:58,226][86121] Updated weights for policy 0, policy_version 27230 (0.0008) +[2023-10-09 13:17:58,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 55902208. Throughput: 0: 1799.5, 1: 1819.8. Samples: 13973514. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) +[2023-10-09 13:17:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 13:18:01,690][86121] Updated weights for policy 0, policy_version 27240 (0.0009) +[2023-10-09 13:18:01,694][86122] Updated weights for policy 1, policy_version 27370 (0.0009) +[2023-10-09 13:18:02,053][86121] Updated weights for policy 0, policy_version 27250 (0.0007) +[2023-10-09 13:18:02,062][86122] Updated weights for policy 1, policy_version 27380 (0.0009) +[2023-10-09 13:18:02,426][86121] Updated weights for policy 0, policy_version 27260 (0.0007) +[2023-10-09 13:18:02,430][86122] Updated weights for policy 1, policy_version 27390 (0.0009) +[2023-10-09 13:18:03,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 55967744. Throughput: 0: 1817.4, 1: 1820.3. Samples: 13995380. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) +[2023-10-09 13:18:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:18:06,078][86122] Updated weights for policy 1, policy_version 27400 (0.0008) +[2023-10-09 13:18:06,330][86121] Updated weights for policy 0, policy_version 27270 (0.0008) +[2023-10-09 13:18:06,440][86122] Updated weights for policy 1, policy_version 27410 (0.0008) +[2023-10-09 13:18:06,696][86121] Updated weights for policy 0, policy_version 27280 (0.0008) +[2023-10-09 13:18:06,802][86122] Updated weights for policy 1, policy_version 27420 (0.0008) +[2023-10-09 13:18:07,056][86121] Updated weights for policy 0, policy_version 27290 (0.0008) +[2023-10-09 13:18:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56033280. Throughput: 0: 1807.5, 1: 1821.4. Samples: 14016258. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) +[2023-10-09 13:18:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:18:10,390][86122] Updated weights for policy 1, policy_version 27430 (0.0007) +[2023-10-09 13:18:10,640][86121] Updated weights for policy 0, policy_version 27300 (0.0008) +[2023-10-09 13:18:10,749][86122] Updated weights for policy 1, policy_version 27440 (0.0008) +[2023-10-09 13:18:10,999][86121] Updated weights for policy 0, policy_version 27310 (0.0009) +[2023-10-09 13:18:11,114][86122] Updated weights for policy 1, policy_version 27450 (0.0007) +[2023-10-09 13:18:11,361][86121] Updated weights for policy 0, policy_version 27320 (0.0008) +[2023-10-09 13:18:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 56098816. Throughput: 0: 1813.6, 1: 1816.4. Samples: 14027870. Policy #0 lag: (min: 31.0, avg: 42.5, max: 63.0) +[2023-10-09 13:18:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:18:14,862][86122] Updated weights for policy 1, policy_version 27460 (0.0009) +[2023-10-09 13:18:15,134][86121] Updated weights for policy 0, policy_version 27330 (0.0008) +[2023-10-09 13:18:15,227][86122] Updated weights for policy 1, policy_version 27470 (0.0009) +[2023-10-09 13:18:15,493][86121] Updated weights for policy 0, policy_version 27340 (0.0008) +[2023-10-09 13:18:15,583][86122] Updated weights for policy 1, policy_version 27480 (0.0008) +[2023-10-09 13:18:15,866][86121] Updated weights for policy 0, policy_version 27350 (0.0008) +[2023-10-09 13:18:16,228][86121] Updated weights for policy 0, policy_version 27360 (0.0008) +[2023-10-09 13:18:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 56164352. Throughput: 0: 1805.9, 1: 1826.1. Samples: 14049288. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) +[2023-10-09 13:18:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:18:19,266][86122] Updated weights for policy 1, policy_version 27490 (0.0009) +[2023-10-09 13:18:19,637][86122] Updated weights for policy 1, policy_version 27500 (0.0007) +[2023-10-09 13:18:19,958][86121] Updated weights for policy 0, policy_version 27370 (0.0007) +[2023-10-09 13:18:19,988][86122] Updated weights for policy 1, policy_version 27510 (0.0008) +[2023-10-09 13:18:20,320][86121] Updated weights for policy 0, policy_version 27380 (0.0007) +[2023-10-09 13:18:20,349][86122] Updated weights for policy 1, policy_version 27520 (0.0009) +[2023-10-09 13:18:20,691][86121] Updated weights for policy 0, policy_version 27390 (0.0008) +[2023-10-09 13:18:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56229888. Throughput: 0: 1809.0, 1: 1826.4. Samples: 14072116. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) +[2023-10-09 13:18:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:18:24,059][86122] Updated weights for policy 1, policy_version 27530 (0.0009) +[2023-10-09 13:18:24,268][86121] Updated weights for policy 0, policy_version 27400 (0.0008) +[2023-10-09 13:18:24,414][86122] Updated weights for policy 1, policy_version 27540 (0.0009) +[2023-10-09 13:18:24,627][86121] Updated weights for policy 0, policy_version 27410 (0.0007) +[2023-10-09 13:18:24,778][86122] Updated weights for policy 1, policy_version 27550 (0.0007) +[2023-10-09 13:18:24,996][86121] Updated weights for policy 0, policy_version 27420 (0.0008) +[2023-10-09 13:18:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56295424. Throughput: 0: 1808.4, 1: 1823.4. Samples: 14081892. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) +[2023-10-09 13:18:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:18:28,400][86122] Updated weights for policy 1, policy_version 27560 (0.0008) +[2023-10-09 13:18:28,767][86122] Updated weights for policy 1, policy_version 27570 (0.0009) +[2023-10-09 13:18:28,791][86121] Updated weights for policy 0, policy_version 27430 (0.0010) +[2023-10-09 13:18:29,128][86122] Updated weights for policy 1, policy_version 27580 (0.0008) +[2023-10-09 13:18:29,155][86121] Updated weights for policy 0, policy_version 27440 (0.0007) +[2023-10-09 13:18:29,530][86121] Updated weights for policy 0, policy_version 27450 (0.0008) +[2023-10-09 13:18:32,922][86122] Updated weights for policy 1, policy_version 27590 (0.0009) +[2023-10-09 13:18:33,286][86122] Updated weights for policy 1, policy_version 27600 (0.0009) +[2023-10-09 13:18:33,331][86121] Updated weights for policy 0, policy_version 27460 (0.0011) +[2023-10-09 13:18:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56360960. Throughput: 0: 1807.2, 1: 1823.3. Samples: 14104554. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) +[2023-10-09 13:18:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:18:33,648][86122] Updated weights for policy 1, policy_version 27610 (0.0008) +[2023-10-09 13:18:33,693][86121] Updated weights for policy 0, policy_version 27470 (0.0007) +[2023-10-09 13:18:34,065][86121] Updated weights for policy 0, policy_version 27480 (0.0010) +[2023-10-09 13:18:37,357][86122] Updated weights for policy 1, policy_version 27620 (0.0009) +[2023-10-09 13:18:37,686][86121] Updated weights for policy 0, policy_version 27490 (0.0009) +[2023-10-09 13:18:37,719][86122] Updated weights for policy 1, policy_version 27630 (0.0008) +[2023-10-09 13:18:38,046][86121] Updated weights for policy 0, policy_version 27500 (0.0009) +[2023-10-09 13:18:38,084][86122] Updated weights for policy 1, policy_version 27640 (0.0007) +[2023-10-09 13:18:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 56459264. Throughput: 0: 1811.8, 1: 1822.9. Samples: 14126132. Policy #0 lag: (min: 31.0, avg: 39.8, max: 63.0) +[2023-10-09 13:18:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:18:38,408][86121] Updated weights for policy 0, policy_version 27510 (0.0008) +[2023-10-09 13:18:38,778][86121] Updated weights for policy 0, policy_version 27520 (0.0007) +[2023-10-09 13:18:41,740][86122] Updated weights for policy 1, policy_version 27650 (0.0009) +[2023-10-09 13:18:42,107][86122] Updated weights for policy 1, policy_version 27660 (0.0008) +[2023-10-09 13:18:42,415][86121] Updated weights for policy 0, policy_version 27530 (0.0010) +[2023-10-09 13:18:42,477][86122] Updated weights for policy 1, policy_version 27670 (0.0008) +[2023-10-09 13:18:42,783][86121] Updated weights for policy 0, policy_version 27540 (0.0009) +[2023-10-09 13:18:42,837][86122] Updated weights for policy 1, policy_version 27680 (0.0007) +[2023-10-09 13:18:43,145][86121] Updated weights for policy 0, policy_version 27550 (0.0009) +[2023-10-09 13:18:43,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 56557568. Throughput: 0: 1811.6, 1: 1826.8. Samples: 14137238. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) +[2023-10-09 13:18:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:18:46,576][86122] Updated weights for policy 1, policy_version 27690 (0.0010) +[2023-10-09 13:18:46,853][86121] Updated weights for policy 0, policy_version 27560 (0.0008) +[2023-10-09 13:18:46,935][86122] Updated weights for policy 1, policy_version 27700 (0.0008) +[2023-10-09 13:18:47,223][86121] Updated weights for policy 0, policy_version 27570 (0.0009) +[2023-10-09 13:18:47,298][86122] Updated weights for policy 1, policy_version 27710 (0.0007) +[2023-10-09 13:18:47,585][86121] Updated weights for policy 0, policy_version 27580 (0.0009) +[2023-10-09 13:18:48,397][85186] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 56623104. Throughput: 0: 1808.8, 1: 1822.5. Samples: 14158788. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) +[2023-10-09 13:18:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:18:51,144][86122] Updated weights for policy 1, policy_version 27720 (0.0009) +[2023-10-09 13:18:51,241][86121] Updated weights for policy 0, policy_version 27590 (0.0008) +[2023-10-09 13:18:51,509][86122] Updated weights for policy 1, policy_version 27730 (0.0009) +[2023-10-09 13:18:51,599][86121] Updated weights for policy 0, policy_version 27600 (0.0007) +[2023-10-09 13:18:51,866][86122] Updated weights for policy 1, policy_version 27740 (0.0009) +[2023-10-09 13:18:51,965][86121] Updated weights for policy 0, policy_version 27610 (0.0009) +[2023-10-09 13:18:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 56688640. Throughput: 0: 1810.7, 1: 1816.3. Samples: 14179474. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) +[2023-10-09 13:18:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:18:55,589][86122] Updated weights for policy 1, policy_version 27750 (0.0008) +[2023-10-09 13:18:55,713][86121] Updated weights for policy 0, policy_version 27620 (0.0008) +[2023-10-09 13:18:55,946][86122] Updated weights for policy 1, policy_version 27760 (0.0008) +[2023-10-09 13:18:56,080][86121] Updated weights for policy 0, policy_version 27630 (0.0008) +[2023-10-09 13:18:56,313][86122] Updated weights for policy 1, policy_version 27770 (0.0008) +[2023-10-09 13:18:56,443][86121] Updated weights for policy 0, policy_version 27640 (0.0009) +[2023-10-09 13:18:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 56754176. Throughput: 0: 1813.6, 1: 1821.8. Samples: 14191464. Policy #0 lag: (min: 31.0, avg: 36.2, max: 63.0) +[2023-10-09 13:18:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:19:00,081][86122] Updated weights for policy 1, policy_version 27780 (0.0008) +[2023-10-09 13:19:00,123][86121] Updated weights for policy 0, policy_version 27650 (0.0007) +[2023-10-09 13:19:00,431][86122] Updated weights for policy 1, policy_version 27790 (0.0007) +[2023-10-09 13:19:00,490][86121] Updated weights for policy 0, policy_version 27660 (0.0007) +[2023-10-09 13:19:00,796][86122] Updated weights for policy 1, policy_version 27800 (0.0008) +[2023-10-09 13:19:00,849][86121] Updated weights for policy 0, policy_version 27670 (0.0009) +[2023-10-09 13:19:01,209][86121] Updated weights for policy 0, policy_version 27680 (0.0007) +[2023-10-09 13:19:03,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 56819712. Throughput: 0: 1809.1, 1: 1809.7. Samples: 14212134. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:19:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:19:04,364][86122] Updated weights for policy 1, policy_version 27810 (0.0008) +[2023-10-09 13:19:04,717][86122] Updated weights for policy 1, policy_version 27820 (0.0009) +[2023-10-09 13:19:05,078][86122] Updated weights for policy 1, policy_version 27830 (0.0009) +[2023-10-09 13:19:05,151][86121] Updated weights for policy 0, policy_version 27690 (0.0008) +[2023-10-09 13:19:05,443][86122] Updated weights for policy 1, policy_version 27840 (0.0009) +[2023-10-09 13:19:05,524][86121] Updated weights for policy 0, policy_version 27700 (0.0008) +[2023-10-09 13:19:05,886][86121] Updated weights for policy 0, policy_version 27710 (0.0010) +[2023-10-09 13:19:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56885248. Throughput: 0: 1799.7, 1: 1809.0. Samples: 14234510. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:19:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:19:09,207][86122] Updated weights for policy 1, policy_version 27850 (0.0011) +[2023-10-09 13:19:09,569][86122] Updated weights for policy 1, policy_version 27860 (0.0009) +[2023-10-09 13:19:09,618][86121] Updated weights for policy 0, policy_version 27720 (0.0008) +[2023-10-09 13:19:09,930][86122] Updated weights for policy 1, policy_version 27870 (0.0007) +[2023-10-09 13:19:09,981][86121] Updated weights for policy 0, policy_version 27730 (0.0009) +[2023-10-09 13:19:10,348][86121] Updated weights for policy 0, policy_version 27740 (0.0008) +[2023-10-09 13:19:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 56950784. Throughput: 0: 1797.3, 1: 1810.5. Samples: 14244244. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:19:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:13,590][86122] Updated weights for policy 1, policy_version 27880 (0.0008) +[2023-10-09 13:19:13,958][86122] Updated weights for policy 1, policy_version 27890 (0.0009) +[2023-10-09 13:19:14,023][86121] Updated weights for policy 0, policy_version 27750 (0.0008) +[2023-10-09 13:19:14,313][86122] Updated weights for policy 1, policy_version 27900 (0.0009) +[2023-10-09 13:19:14,380][86121] Updated weights for policy 0, policy_version 27760 (0.0008) +[2023-10-09 13:19:14,746][86121] Updated weights for policy 0, policy_version 27770 (0.0009) +[2023-10-09 13:19:18,072][86122] Updated weights for policy 1, policy_version 27910 (0.0009) +[2023-10-09 13:19:18,332][86121] Updated weights for policy 0, policy_version 27780 (0.0008) +[2023-10-09 13:19:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57016320. Throughput: 0: 1806.9, 1: 1809.2. Samples: 14267280. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:19:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:18,439][86122] Updated weights for policy 1, policy_version 27920 (0.0007) +[2023-10-09 13:19:18,702][86121] Updated weights for policy 0, policy_version 27790 (0.0008) +[2023-10-09 13:19:18,795][86122] Updated weights for policy 1, policy_version 27930 (0.0007) +[2023-10-09 13:19:19,061][86121] Updated weights for policy 0, policy_version 27800 (0.0008) +[2023-10-09 13:19:22,570][86122] Updated weights for policy 1, policy_version 27940 (0.0008) +[2023-10-09 13:19:22,820][86121] Updated weights for policy 0, policy_version 27810 (0.0008) +[2023-10-09 13:19:22,939][86122] Updated weights for policy 1, policy_version 27950 (0.0008) +[2023-10-09 13:19:23,181][86121] Updated weights for policy 0, policy_version 27820 (0.0007) +[2023-10-09 13:19:23,293][86122] Updated weights for policy 1, policy_version 27960 (0.0007) +[2023-10-09 13:19:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 57081856. Throughput: 0: 1811.2, 1: 1818.1. Samples: 14289452. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:19:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:23,553][86121] Updated weights for policy 0, policy_version 27830 (0.0007) +[2023-10-09 13:19:23,582][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000027968_28639232.pth... +[2023-10-09 13:19:23,611][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000026272_26902528.pth +[2023-10-09 13:19:23,911][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000027840_28508160.pth... +[2023-10-09 13:19:23,915][86121] Updated weights for policy 0, policy_version 27840 (0.0008) +[2023-10-09 13:19:23,940][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000026144_26771456.pth +[2023-10-09 13:19:26,907][86122] Updated weights for policy 1, policy_version 27970 (0.0008) +[2023-10-09 13:19:27,264][86122] Updated weights for policy 1, policy_version 27980 (0.0008) +[2023-10-09 13:19:27,629][86122] Updated weights for policy 1, policy_version 27990 (0.0007) +[2023-10-09 13:19:27,810][86121] Updated weights for policy 0, policy_version 27850 (0.0007) +[2023-10-09 13:19:27,985][86122] Updated weights for policy 1, policy_version 28000 (0.0009) +[2023-10-09 13:19:28,176][86121] Updated weights for policy 0, policy_version 27860 (0.0009) +[2023-10-09 13:19:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57180160. Throughput: 0: 1807.4, 1: 1813.2. Samples: 14300164. Policy #0 lag: (min: 4.0, avg: 6.4, max: 36.0) +[2023-10-09 13:19:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:28,546][86121] Updated weights for policy 0, policy_version 27870 (0.0010) +[2023-10-09 13:19:31,738][86122] Updated weights for policy 1, policy_version 28010 (0.0008) +[2023-10-09 13:19:32,097][86122] Updated weights for policy 1, policy_version 28020 (0.0009) +[2023-10-09 13:19:32,465][86121] Updated weights for policy 0, policy_version 27880 (0.0007) +[2023-10-09 13:19:32,468][86122] Updated weights for policy 1, policy_version 28030 (0.0009) +[2023-10-09 13:19:32,831][86121] Updated weights for policy 0, policy_version 27890 (0.0007) +[2023-10-09 13:19:33,198][86121] Updated weights for policy 0, policy_version 27900 (0.0007) +[2023-10-09 13:19:33,397][85186] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 57278464. Throughput: 0: 1812.7, 1: 1819.5. Samples: 14322234. Policy #0 lag: (min: 4.0, avg: 6.4, max: 36.0) +[2023-10-09 13:19:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:36,099][86122] Updated weights for policy 1, policy_version 28040 (0.0009) +[2023-10-09 13:19:36,455][86122] Updated weights for policy 1, policy_version 28050 (0.0010) +[2023-10-09 13:19:36,820][86122] Updated weights for policy 1, policy_version 28060 (0.0009) +[2023-10-09 13:19:37,036][86121] Updated weights for policy 0, policy_version 27910 (0.0007) +[2023-10-09 13:19:37,410][86121] Updated weights for policy 0, policy_version 27920 (0.0008) +[2023-10-09 13:19:37,777][86121] Updated weights for policy 0, policy_version 27930 (0.0009) +[2023-10-09 13:19:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57344000. Throughput: 0: 1804.7, 1: 1825.5. Samples: 14342834. Policy #0 lag: (min: 4.0, avg: 6.4, max: 36.0) +[2023-10-09 13:19:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:40,614][86122] Updated weights for policy 1, policy_version 28070 (0.0007) +[2023-10-09 13:19:40,983][86122] Updated weights for policy 1, policy_version 28080 (0.0007) +[2023-10-09 13:19:41,348][86122] Updated weights for policy 1, policy_version 28090 (0.0008) +[2023-10-09 13:19:41,395][86121] Updated weights for policy 0, policy_version 27940 (0.0008) +[2023-10-09 13:19:41,755][86121] Updated weights for policy 0, policy_version 27950 (0.0008) +[2023-10-09 13:19:42,121][86121] Updated weights for policy 0, policy_version 27960 (0.0008) +[2023-10-09 13:19:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 57409536. Throughput: 0: 1810.9, 1: 1821.6. Samples: 14354930. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) +[2023-10-09 13:19:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:45,080][86122] Updated weights for policy 1, policy_version 28100 (0.0007) +[2023-10-09 13:19:45,435][86122] Updated weights for policy 1, policy_version 28110 (0.0008) +[2023-10-09 13:19:45,777][86121] Updated weights for policy 0, policy_version 27970 (0.0008) +[2023-10-09 13:19:45,795][86122] Updated weights for policy 1, policy_version 28120 (0.0008) +[2023-10-09 13:19:46,148][86121] Updated weights for policy 0, policy_version 27980 (0.0009) +[2023-10-09 13:19:46,514][86121] Updated weights for policy 0, policy_version 27990 (0.0009) +[2023-10-09 13:19:46,884][86121] Updated weights for policy 0, policy_version 28000 (0.0008) +[2023-10-09 13:19:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57475072. Throughput: 0: 1805.2, 1: 1828.1. Samples: 14375634. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) +[2023-10-09 13:19:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:49,381][86122] Updated weights for policy 1, policy_version 28130 (0.0007) +[2023-10-09 13:19:49,746][86122] Updated weights for policy 1, policy_version 28140 (0.0009) +[2023-10-09 13:19:50,114][86122] Updated weights for policy 1, policy_version 28150 (0.0008) +[2023-10-09 13:19:50,471][86122] Updated weights for policy 1, policy_version 28160 (0.0008) +[2023-10-09 13:19:50,548][86121] Updated weights for policy 0, policy_version 28010 (0.0010) +[2023-10-09 13:19:50,911][86121] Updated weights for policy 0, policy_version 28020 (0.0011) +[2023-10-09 13:19:51,283][86121] Updated weights for policy 0, policy_version 28030 (0.0009) +[2023-10-09 13:19:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 57540608. Throughput: 0: 1810.6, 1: 1827.9. Samples: 14398246. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) +[2023-10-09 13:19:53,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:19:54,151][86122] Updated weights for policy 1, policy_version 28170 (0.0008) +[2023-10-09 13:19:54,503][86122] Updated weights for policy 1, policy_version 28180 (0.0007) +[2023-10-09 13:19:54,868][86122] Updated weights for policy 1, policy_version 28190 (0.0007) +[2023-10-09 13:19:55,124][86121] Updated weights for policy 0, policy_version 28040 (0.0010) +[2023-10-09 13:19:55,489][86121] Updated weights for policy 0, policy_version 28050 (0.0009) +[2023-10-09 13:19:55,852][86121] Updated weights for policy 0, policy_version 28060 (0.0012) +[2023-10-09 13:19:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 57606144. Throughput: 0: 1815.8, 1: 1827.5. Samples: 14408194. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) +[2023-10-09 13:19:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:19:58,480][86122] Updated weights for policy 1, policy_version 28200 (0.0008) +[2023-10-09 13:19:58,849][86122] Updated weights for policy 1, policy_version 28210 (0.0007) +[2023-10-09 13:19:59,201][86122] Updated weights for policy 1, policy_version 28220 (0.0007) +[2023-10-09 13:19:59,698][86121] Updated weights for policy 0, policy_version 28070 (0.0009) +[2023-10-09 13:20:00,058][86121] Updated weights for policy 0, policy_version 28080 (0.0007) +[2023-10-09 13:20:00,423][86121] Updated weights for policy 0, policy_version 28090 (0.0010) +[2023-10-09 13:20:02,824][86122] Updated weights for policy 1, policy_version 28230 (0.0007) +[2023-10-09 13:20:03,189][86122] Updated weights for policy 1, policy_version 28240 (0.0008) +[2023-10-09 13:20:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 57671680. Throughput: 0: 1801.4, 1: 1827.4. Samples: 14430576. Policy #0 lag: (min: 31.0, avg: 33.6, max: 63.0) +[2023-10-09 13:20:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:20:03,562][86122] Updated weights for policy 1, policy_version 28250 (0.0008) +[2023-10-09 13:20:03,984][86121] Updated weights for policy 0, policy_version 28100 (0.0009) +[2023-10-09 13:20:04,344][86121] Updated weights for policy 0, policy_version 28110 (0.0007) +[2023-10-09 13:20:04,716][86121] Updated weights for policy 0, policy_version 28120 (0.0007) +[2023-10-09 13:20:07,215][86122] Updated weights for policy 1, policy_version 28260 (0.0008) +[2023-10-09 13:20:07,581][86122] Updated weights for policy 1, policy_version 28270 (0.0008) +[2023-10-09 13:20:07,943][86122] Updated weights for policy 1, policy_version 28280 (0.0008) +[2023-10-09 13:20:08,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57769984. Throughput: 0: 1806.2, 1: 1819.6. Samples: 14452612. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) +[2023-10-09 13:20:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:20:08,519][86121] Updated weights for policy 0, policy_version 28130 (0.0009) +[2023-10-09 13:20:08,892][86121] Updated weights for policy 0, policy_version 28140 (0.0010) +[2023-10-09 13:20:09,260][86121] Updated weights for policy 0, policy_version 28150 (0.0007) +[2023-10-09 13:20:09,627][86121] Updated weights for policy 0, policy_version 28160 (0.0007) +[2023-10-09 13:20:11,653][86122] Updated weights for policy 1, policy_version 28290 (0.0009) +[2023-10-09 13:20:12,008][86122] Updated weights for policy 1, policy_version 28300 (0.0011) +[2023-10-09 13:20:12,370][86122] Updated weights for policy 1, policy_version 28310 (0.0010) +[2023-10-09 13:20:12,734][86122] Updated weights for policy 1, policy_version 28320 (0.0010) +[2023-10-09 13:20:13,185][86121] Updated weights for policy 0, policy_version 28170 (0.0007) +[2023-10-09 13:20:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 57835520. Throughput: 0: 1797.4, 1: 1829.1. Samples: 14463356. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) +[2023-10-09 13:20:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:20:13,552][86121] Updated weights for policy 0, policy_version 28180 (0.0009) +[2023-10-09 13:20:13,920][86121] Updated weights for policy 0, policy_version 28190 (0.0008) +[2023-10-09 13:20:16,547][86122] Updated weights for policy 1, policy_version 28330 (0.0010) +[2023-10-09 13:20:16,910][86122] Updated weights for policy 1, policy_version 28340 (0.0008) +[2023-10-09 13:20:17,267][86122] Updated weights for policy 1, policy_version 28350 (0.0008) +[2023-10-09 13:20:17,485][86121] Updated weights for policy 0, policy_version 28200 (0.0008) +[2023-10-09 13:20:17,846][86121] Updated weights for policy 0, policy_version 28210 (0.0009) +[2023-10-09 13:20:18,215][86121] Updated weights for policy 0, policy_version 28220 (0.0008) +[2023-10-09 13:20:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 57933824. Throughput: 0: 1807.8, 1: 1821.4. Samples: 14485548. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) +[2023-10-09 13:20:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:20:20,940][86122] Updated weights for policy 1, policy_version 28360 (0.0009) +[2023-10-09 13:20:21,305][86122] Updated weights for policy 1, policy_version 28370 (0.0007) +[2023-10-09 13:20:21,671][86122] Updated weights for policy 1, policy_version 28380 (0.0008) +[2023-10-09 13:20:21,896][86121] Updated weights for policy 0, policy_version 28230 (0.0009) +[2023-10-09 13:20:22,262][86121] Updated weights for policy 0, policy_version 28240 (0.0009) +[2023-10-09 13:20:22,630][86121] Updated weights for policy 0, policy_version 28250 (0.0010) +[2023-10-09 13:20:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 57999360. Throughput: 0: 1805.6, 1: 1821.4. Samples: 14506050. Policy #0 lag: (min: 31.0, avg: 33.0, max: 61.0) +[2023-10-09 13:20:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:20:25,314][86122] Updated weights for policy 1, policy_version 28390 (0.0008) +[2023-10-09 13:20:25,676][86122] Updated weights for policy 1, policy_version 28400 (0.0010) +[2023-10-09 13:20:26,032][86122] Updated weights for policy 1, policy_version 28410 (0.0007) +[2023-10-09 13:20:26,317][86121] Updated weights for policy 0, policy_version 28260 (0.0007) +[2023-10-09 13:20:26,672][86121] Updated weights for policy 0, policy_version 28270 (0.0008) +[2023-10-09 13:20:27,035][86121] Updated weights for policy 0, policy_version 28280 (0.0009) +[2023-10-09 13:20:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58064896. Throughput: 0: 1808.6, 1: 1817.2. Samples: 14518094. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 13:20:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:20:29,785][86122] Updated weights for policy 1, policy_version 28420 (0.0009) +[2023-10-09 13:20:30,155][86122] Updated weights for policy 1, policy_version 28430 (0.0008) +[2023-10-09 13:20:30,511][86122] Updated weights for policy 1, policy_version 28440 (0.0009) +[2023-10-09 13:20:30,890][86121] Updated weights for policy 0, policy_version 28290 (0.0008) +[2023-10-09 13:20:31,254][86121] Updated weights for policy 0, policy_version 28300 (0.0011) +[2023-10-09 13:20:31,630][86121] Updated weights for policy 0, policy_version 28310 (0.0010) +[2023-10-09 13:20:31,992][86121] Updated weights for policy 0, policy_version 28320 (0.0007) +[2023-10-09 13:20:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 58130432. Throughput: 0: 1808.7, 1: 1823.7. Samples: 14539096. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 13:20:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:20:34,260][86122] Updated weights for policy 1, policy_version 28450 (0.0007) +[2023-10-09 13:20:34,633][86122] Updated weights for policy 1, policy_version 28460 (0.0008) +[2023-10-09 13:20:34,989][86122] Updated weights for policy 1, policy_version 28470 (0.0008) +[2023-10-09 13:20:35,353][86122] Updated weights for policy 1, policy_version 28480 (0.0010) +[2023-10-09 13:20:35,701][86121] Updated weights for policy 0, policy_version 28330 (0.0010) +[2023-10-09 13:20:36,061][86121] Updated weights for policy 0, policy_version 28340 (0.0008) +[2023-10-09 13:20:36,432][86121] Updated weights for policy 0, policy_version 28350 (0.0009) +[2023-10-09 13:20:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58195968. Throughput: 0: 1807.0, 1: 1823.7. Samples: 14561630. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 13:20:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:20:38,873][86122] Updated weights for policy 1, policy_version 28490 (0.0009) +[2023-10-09 13:20:39,238][86122] Updated weights for policy 1, policy_version 28500 (0.0010) +[2023-10-09 13:20:39,610][86122] Updated weights for policy 1, policy_version 28510 (0.0009) +[2023-10-09 13:20:40,036][86121] Updated weights for policy 0, policy_version 28360 (0.0008) +[2023-10-09 13:20:40,400][86121] Updated weights for policy 0, policy_version 28370 (0.0008) +[2023-10-09 13:20:40,775][86121] Updated weights for policy 0, policy_version 28380 (0.0008) +[2023-10-09 13:20:43,313][86122] Updated weights for policy 1, policy_version 28520 (0.0007) +[2023-10-09 13:20:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58261504. Throughput: 0: 1808.4, 1: 1823.6. Samples: 14571634. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 13:20:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:20:43,680][86122] Updated weights for policy 1, policy_version 28530 (0.0009) +[2023-10-09 13:20:44,043][86122] Updated weights for policy 1, policy_version 28540 (0.0009) +[2023-10-09 13:20:44,403][86121] Updated weights for policy 0, policy_version 28390 (0.0008) +[2023-10-09 13:20:44,771][86121] Updated weights for policy 0, policy_version 28400 (0.0008) +[2023-10-09 13:20:45,147][86121] Updated weights for policy 0, policy_version 28410 (0.0009) +[2023-10-09 13:20:47,799][86122] Updated weights for policy 1, policy_version 28550 (0.0010) +[2023-10-09 13:20:48,165][86122] Updated weights for policy 1, policy_version 28560 (0.0008) +[2023-10-09 13:20:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58327040. Throughput: 0: 1814.9, 1: 1823.4. Samples: 14594300. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 13:20:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:20:48,523][86122] Updated weights for policy 1, policy_version 28570 (0.0007) +[2023-10-09 13:20:48,790][86121] Updated weights for policy 0, policy_version 28420 (0.0008) +[2023-10-09 13:20:49,162][86121] Updated weights for policy 0, policy_version 28430 (0.0010) +[2023-10-09 13:20:49,527][86121] Updated weights for policy 0, policy_version 28440 (0.0011) +[2023-10-09 13:20:52,339][86122] Updated weights for policy 1, policy_version 28580 (0.0007) +[2023-10-09 13:20:52,700][86122] Updated weights for policy 1, policy_version 28590 (0.0008) +[2023-10-09 13:20:53,059][86122] Updated weights for policy 1, policy_version 28600 (0.0010) +[2023-10-09 13:20:53,279][86121] Updated weights for policy 0, policy_version 28450 (0.0009) +[2023-10-09 13:20:53,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58425344. Throughput: 0: 1819.5, 1: 1820.3. Samples: 14616406. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) +[2023-10-09 13:20:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:20:53,646][86121] Updated weights for policy 0, policy_version 28460 (0.0009) +[2023-10-09 13:20:54,014][86121] Updated weights for policy 0, policy_version 28470 (0.0009) +[2023-10-09 13:20:54,378][86121] Updated weights for policy 0, policy_version 28480 (0.0007) +[2023-10-09 13:20:56,701][86122] Updated weights for policy 1, policy_version 28610 (0.0009) +[2023-10-09 13:20:57,064][86122] Updated weights for policy 1, policy_version 28620 (0.0010) +[2023-10-09 13:20:57,427][86122] Updated weights for policy 1, policy_version 28630 (0.0010) +[2023-10-09 13:20:57,786][86122] Updated weights for policy 1, policy_version 28640 (0.0009) +[2023-10-09 13:20:58,085][86121] Updated weights for policy 0, policy_version 28490 (0.0008) +[2023-10-09 13:20:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58490880. Throughput: 0: 1821.8, 1: 1814.6. Samples: 14626994. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) +[2023-10-09 13:20:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:20:58,454][86121] Updated weights for policy 0, policy_version 28500 (0.0008) +[2023-10-09 13:20:58,821][86121] Updated weights for policy 0, policy_version 28510 (0.0009) +[2023-10-09 13:21:01,773][86122] Updated weights for policy 1, policy_version 28650 (0.0009) +[2023-10-09 13:21:02,134][86122] Updated weights for policy 1, policy_version 28660 (0.0010) +[2023-10-09 13:21:02,494][86122] Updated weights for policy 1, policy_version 28670 (0.0007) +[2023-10-09 13:21:02,509][86121] Updated weights for policy 0, policy_version 28520 (0.0009) +[2023-10-09 13:21:02,879][86121] Updated weights for policy 0, policy_version 28530 (0.0011) +[2023-10-09 13:21:03,245][86121] Updated weights for policy 0, policy_version 28540 (0.0011) +[2023-10-09 13:21:03,397][85186] Fps is (10 sec: 16384.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 58589184. Throughput: 0: 1818.5, 1: 1813.3. Samples: 14648980. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) +[2023-10-09 13:21:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:06,081][86122] Updated weights for policy 1, policy_version 28680 (0.0008) +[2023-10-09 13:21:06,451][86122] Updated weights for policy 1, policy_version 28690 (0.0009) +[2023-10-09 13:21:06,805][86122] Updated weights for policy 1, policy_version 28700 (0.0007) +[2023-10-09 13:21:07,014][86121] Updated weights for policy 0, policy_version 28550 (0.0008) +[2023-10-09 13:21:07,371][86121] Updated weights for policy 0, policy_version 28560 (0.0007) +[2023-10-09 13:21:07,736][86121] Updated weights for policy 0, policy_version 28570 (0.0009) +[2023-10-09 13:21:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 58654720. Throughput: 0: 1817.2, 1: 1811.3. Samples: 14669334. Policy #0 lag: (min: 31.0, avg: 37.2, max: 63.0) +[2023-10-09 13:21:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:10,412][86122] Updated weights for policy 1, policy_version 28710 (0.0009) +[2023-10-09 13:21:10,776][86122] Updated weights for policy 1, policy_version 28720 (0.0008) +[2023-10-09 13:21:11,139][86122] Updated weights for policy 1, policy_version 28730 (0.0008) +[2023-10-09 13:21:11,572][86121] Updated weights for policy 0, policy_version 28580 (0.0010) +[2023-10-09 13:21:11,934][86121] Updated weights for policy 0, policy_version 28590 (0.0007) +[2023-10-09 13:21:12,302][86121] Updated weights for policy 0, policy_version 28600 (0.0007) +[2023-10-09 13:21:13,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 58720256. Throughput: 0: 1810.9, 1: 1818.0. Samples: 14681396. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) +[2023-10-09 13:21:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:14,985][86122] Updated weights for policy 1, policy_version 28740 (0.0009) +[2023-10-09 13:21:15,349][86122] Updated weights for policy 1, policy_version 28750 (0.0011) +[2023-10-09 13:21:15,714][86122] Updated weights for policy 1, policy_version 28760 (0.0011) +[2023-10-09 13:21:15,981][86121] Updated weights for policy 0, policy_version 28610 (0.0007) +[2023-10-09 13:21:16,350][86121] Updated weights for policy 0, policy_version 28620 (0.0007) +[2023-10-09 13:21:16,714][86121] Updated weights for policy 0, policy_version 28630 (0.0007) +[2023-10-09 13:21:17,080][86121] Updated weights for policy 0, policy_version 28640 (0.0008) +[2023-10-09 13:21:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58785792. Throughput: 0: 1813.5, 1: 1811.1. Samples: 14702204. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) +[2023-10-09 13:21:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:19,387][86122] Updated weights for policy 1, policy_version 28770 (0.0009) +[2023-10-09 13:21:19,749][86122] Updated weights for policy 1, policy_version 28780 (0.0008) +[2023-10-09 13:21:20,118][86122] Updated weights for policy 1, policy_version 28790 (0.0007) +[2023-10-09 13:21:20,476][86122] Updated weights for policy 1, policy_version 28800 (0.0008) +[2023-10-09 13:21:21,050][86121] Updated weights for policy 0, policy_version 28650 (0.0012) +[2023-10-09 13:21:21,424][86121] Updated weights for policy 0, policy_version 28660 (0.0010) +[2023-10-09 13:21:21,800][86121] Updated weights for policy 0, policy_version 28670 (0.0009) +[2023-10-09 13:21:23,398][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 58851328. Throughput: 0: 1807.5, 1: 1815.5. Samples: 14724668. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) +[2023-10-09 13:21:23,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000028672_29360128.pth... +[2023-10-09 13:21:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000028800_29491200.pth... +[2023-10-09 13:21:23,450][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000027104_27754496.pth +[2023-10-09 13:21:23,451][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000026976_27623424.pth +[2023-10-09 13:21:24,028][86122] Updated weights for policy 1, policy_version 28810 (0.0007) +[2023-10-09 13:21:24,393][86122] Updated weights for policy 1, policy_version 28820 (0.0008) +[2023-10-09 13:21:24,753][86122] Updated weights for policy 1, policy_version 28830 (0.0009) +[2023-10-09 13:21:25,454][86121] Updated weights for policy 0, policy_version 28680 (0.0008) +[2023-10-09 13:21:25,810][86121] Updated weights for policy 0, policy_version 28690 (0.0008) +[2023-10-09 13:21:26,176][86121] Updated weights for policy 0, policy_version 28700 (0.0008) +[2023-10-09 13:21:28,207][86122] Updated weights for policy 1, policy_version 28840 (0.0007) +[2023-10-09 13:21:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 58916864. Throughput: 0: 1817.0, 1: 1815.7. Samples: 14735106. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) +[2023-10-09 13:21:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:28,572][86122] Updated weights for policy 1, policy_version 28850 (0.0008) +[2023-10-09 13:21:28,938][86122] Updated weights for policy 1, policy_version 28860 (0.0009) +[2023-10-09 13:21:29,950][86121] Updated weights for policy 0, policy_version 28710 (0.0007) +[2023-10-09 13:21:30,323][86121] Updated weights for policy 0, policy_version 28720 (0.0009) +[2023-10-09 13:21:30,689][86121] Updated weights for policy 0, policy_version 28730 (0.0008) +[2023-10-09 13:21:32,841][86122] Updated weights for policy 1, policy_version 28870 (0.0010) +[2023-10-09 13:21:33,210][86122] Updated weights for policy 1, policy_version 28880 (0.0009) +[2023-10-09 13:21:33,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 58982400. Throughput: 0: 1805.1, 1: 1824.2. Samples: 14757618. Policy #0 lag: (min: 17.0, avg: 25.0, max: 49.0) +[2023-10-09 13:21:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:33,567][86122] Updated weights for policy 1, policy_version 28890 (0.0009) +[2023-10-09 13:21:34,499][86121] Updated weights for policy 0, policy_version 28740 (0.0008) +[2023-10-09 13:21:34,864][86121] Updated weights for policy 0, policy_version 28750 (0.0008) +[2023-10-09 13:21:35,223][86121] Updated weights for policy 0, policy_version 28760 (0.0010) +[2023-10-09 13:21:37,214][86122] Updated weights for policy 1, policy_version 28900 (0.0008) +[2023-10-09 13:21:37,574][86122] Updated weights for policy 1, policy_version 28910 (0.0008) +[2023-10-09 13:21:37,932][86122] Updated weights for policy 1, policy_version 28920 (0.0009) +[2023-10-09 13:21:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59080704. Throughput: 0: 1799.7, 1: 1822.6. Samples: 14779410. Policy #0 lag: (min: 3.0, avg: 6.1, max: 35.0) +[2023-10-09 13:21:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:38,901][86121] Updated weights for policy 0, policy_version 28770 (0.0010) +[2023-10-09 13:21:39,269][86121] Updated weights for policy 0, policy_version 28780 (0.0009) +[2023-10-09 13:21:39,629][86121] Updated weights for policy 0, policy_version 28790 (0.0009) +[2023-10-09 13:21:39,994][86121] Updated weights for policy 0, policy_version 28800 (0.0007) +[2023-10-09 13:21:41,724][86122] Updated weights for policy 1, policy_version 28930 (0.0011) +[2023-10-09 13:21:42,089][86122] Updated weights for policy 1, policy_version 28940 (0.0008) +[2023-10-09 13:21:42,443][86122] Updated weights for policy 1, policy_version 28950 (0.0007) +[2023-10-09 13:21:42,803][86122] Updated weights for policy 1, policy_version 28960 (0.0008) +[2023-10-09 13:21:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59146240. Throughput: 0: 1799.3, 1: 1825.3. Samples: 14790102. Policy #0 lag: (min: 3.0, avg: 6.1, max: 35.0) +[2023-10-09 13:21:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:21:43,650][86121] Updated weights for policy 0, policy_version 28810 (0.0010) +[2023-10-09 13:21:44,025][86121] Updated weights for policy 0, policy_version 28820 (0.0010) +[2023-10-09 13:21:44,393][86121] Updated weights for policy 0, policy_version 28830 (0.0011) +[2023-10-09 13:21:46,491][86122] Updated weights for policy 1, policy_version 28970 (0.0008) +[2023-10-09 13:21:46,850][86122] Updated weights for policy 1, policy_version 28980 (0.0007) +[2023-10-09 13:21:47,224][86122] Updated weights for policy 1, policy_version 28990 (0.0007) +[2023-10-09 13:21:47,968][86121] Updated weights for policy 0, policy_version 28840 (0.0010) +[2023-10-09 13:21:48,338][86121] Updated weights for policy 0, policy_version 28850 (0.0009) +[2023-10-09 13:21:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59211776. Throughput: 0: 1799.8, 1: 1833.6. Samples: 14812484. Policy #0 lag: (min: 3.0, avg: 6.1, max: 35.0) +[2023-10-09 13:21:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:21:48,702][86121] Updated weights for policy 0, policy_version 28860 (0.0010) +[2023-10-09 13:21:50,687][86122] Updated weights for policy 1, policy_version 29000 (0.0008) +[2023-10-09 13:21:51,048][86122] Updated weights for policy 1, policy_version 29010 (0.0009) +[2023-10-09 13:21:51,416][86122] Updated weights for policy 1, policy_version 29020 (0.0008) +[2023-10-09 13:21:52,420][86121] Updated weights for policy 0, policy_version 28870 (0.0009) +[2023-10-09 13:21:52,793][86121] Updated weights for policy 0, policy_version 28880 (0.0007) +[2023-10-09 13:21:53,166][86121] Updated weights for policy 0, policy_version 28890 (0.0007) +[2023-10-09 13:21:53,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59310080. Throughput: 0: 1815.3, 1: 1843.9. Samples: 14833998. Policy #0 lag: (min: 3.0, avg: 6.1, max: 35.0) +[2023-10-09 13:21:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:21:55,132][86122] Updated weights for policy 1, policy_version 29030 (0.0007) +[2023-10-09 13:21:55,498][86122] Updated weights for policy 1, policy_version 29040 (0.0008) +[2023-10-09 13:21:55,863][86122] Updated weights for policy 1, policy_version 29050 (0.0009) +[2023-10-09 13:21:56,825][86121] Updated weights for policy 0, policy_version 28900 (0.0008) +[2023-10-09 13:21:57,189][86121] Updated weights for policy 0, policy_version 28910 (0.0007) +[2023-10-09 13:21:57,556][86121] Updated weights for policy 0, policy_version 28920 (0.0009) +[2023-10-09 13:21:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59375616. Throughput: 0: 1806.2, 1: 1832.3. Samples: 14845126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:21:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 13:21:59,457][86122] Updated weights for policy 1, policy_version 29060 (0.0010) +[2023-10-09 13:21:59,814][86122] Updated weights for policy 1, policy_version 29070 (0.0008) +[2023-10-09 13:22:00,182][86122] Updated weights for policy 1, policy_version 29080 (0.0008) +[2023-10-09 13:22:01,252][86121] Updated weights for policy 0, policy_version 28930 (0.0008) +[2023-10-09 13:22:01,611][86121] Updated weights for policy 0, policy_version 28940 (0.0010) +[2023-10-09 13:22:01,971][86121] Updated weights for policy 0, policy_version 28950 (0.0010) +[2023-10-09 13:22:02,344][86121] Updated weights for policy 0, policy_version 28960 (0.0009) +[2023-10-09 13:22:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 59441152. Throughput: 0: 1811.4, 1: 1830.4. Samples: 14866084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 13:22:03,972][86122] Updated weights for policy 1, policy_version 29090 (0.0009) +[2023-10-09 13:22:04,336][86122] Updated weights for policy 1, policy_version 29100 (0.0008) +[2023-10-09 13:22:04,708][86122] Updated weights for policy 1, policy_version 29110 (0.0007) +[2023-10-09 13:22:05,068][86122] Updated weights for policy 1, policy_version 29120 (0.0011) +[2023-10-09 13:22:06,204][86121] Updated weights for policy 0, policy_version 28970 (0.0008) +[2023-10-09 13:22:06,562][86121] Updated weights for policy 0, policy_version 28980 (0.0007) +[2023-10-09 13:22:06,928][86121] Updated weights for policy 0, policy_version 28990 (0.0008) +[2023-10-09 13:22:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59506688. Throughput: 0: 1810.0, 1: 1829.9. Samples: 14888462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 13:22:08,858][86122] Updated weights for policy 1, policy_version 29130 (0.0007) +[2023-10-09 13:22:09,235][86122] Updated weights for policy 1, policy_version 29140 (0.0008) +[2023-10-09 13:22:09,595][86122] Updated weights for policy 1, policy_version 29150 (0.0007) +[2023-10-09 13:22:10,505][86121] Updated weights for policy 0, policy_version 29000 (0.0009) +[2023-10-09 13:22:10,875][86121] Updated weights for policy 0, policy_version 29010 (0.0009) +[2023-10-09 13:22:11,236][86121] Updated weights for policy 0, policy_version 29020 (0.0009) +[2023-10-09 13:22:13,302][86122] Updated weights for policy 1, policy_version 29160 (0.0011) +[2023-10-09 13:22:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59572224. Throughput: 0: 1814.6, 1: 1830.7. Samples: 14899142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.940')] +[2023-10-09 13:22:13,662][86122] Updated weights for policy 1, policy_version 29170 (0.0010) +[2023-10-09 13:22:14,033][86122] Updated weights for policy 1, policy_version 29180 (0.0009) +[2023-10-09 13:22:14,957][86121] Updated weights for policy 0, policy_version 29030 (0.0010) +[2023-10-09 13:22:15,314][86121] Updated weights for policy 0, policy_version 29040 (0.0009) +[2023-10-09 13:22:15,680][86121] Updated weights for policy 0, policy_version 29050 (0.0010) +[2023-10-09 13:22:17,761][86122] Updated weights for policy 1, policy_version 29190 (0.0008) +[2023-10-09 13:22:18,124][86122] Updated weights for policy 1, policy_version 29200 (0.0008) +[2023-10-09 13:22:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 59637760. Throughput: 0: 1817.9, 1: 1820.0. Samples: 14921320. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 13:22:18,484][86122] Updated weights for policy 1, policy_version 29210 (0.0007) +[2023-10-09 13:22:19,383][86121] Updated weights for policy 0, policy_version 29060 (0.0007) +[2023-10-09 13:22:19,745][86121] Updated weights for policy 0, policy_version 29070 (0.0008) +[2023-10-09 13:22:20,115][86121] Updated weights for policy 0, policy_version 29080 (0.0010) +[2023-10-09 13:22:22,108][86122] Updated weights for policy 1, policy_version 29220 (0.0009) +[2023-10-09 13:22:22,467][86122] Updated weights for policy 1, policy_version 29230 (0.0008) +[2023-10-09 13:22:22,842][86122] Updated weights for policy 1, policy_version 29240 (0.0010) +[2023-10-09 13:22:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 59736064. Throughput: 0: 1826.6, 1: 1820.8. Samples: 14943544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 13:22:23,640][86121] Updated weights for policy 0, policy_version 29090 (0.0007) +[2023-10-09 13:22:24,013][86121] Updated weights for policy 0, policy_version 29100 (0.0008) +[2023-10-09 13:22:24,372][86121] Updated weights for policy 0, policy_version 29110 (0.0010) +[2023-10-09 13:22:24,735][86121] Updated weights for policy 0, policy_version 29120 (0.0010) +[2023-10-09 13:22:26,623][86122] Updated weights for policy 1, policy_version 29250 (0.0009) +[2023-10-09 13:22:26,996][86122] Updated weights for policy 1, policy_version 29260 (0.0011) +[2023-10-09 13:22:27,351][86122] Updated weights for policy 1, policy_version 29270 (0.0010) +[2023-10-09 13:22:27,713][86122] Updated weights for policy 1, policy_version 29280 (0.0010) +[2023-10-09 13:22:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59801600. Throughput: 0: 1823.7, 1: 1825.0. Samples: 14954296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 13:22:28,421][86121] Updated weights for policy 0, policy_version 29130 (0.0008) +[2023-10-09 13:22:28,783][86121] Updated weights for policy 0, policy_version 29140 (0.0011) +[2023-10-09 13:22:29,155][86121] Updated weights for policy 0, policy_version 29150 (0.0010) +[2023-10-09 13:22:31,447][86122] Updated weights for policy 1, policy_version 29290 (0.0009) +[2023-10-09 13:22:31,817][86122] Updated weights for policy 1, policy_version 29300 (0.0009) +[2023-10-09 13:22:32,187][86122] Updated weights for policy 1, policy_version 29310 (0.0011) +[2023-10-09 13:22:33,020][86121] Updated weights for policy 0, policy_version 29160 (0.0009) +[2023-10-09 13:22:33,388][86121] Updated weights for policy 0, policy_version 29170 (0.0008) +[2023-10-09 13:22:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 59867136. Throughput: 0: 1815.5, 1: 1816.1. Samples: 14975908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 13:22:33,754][86121] Updated weights for policy 0, policy_version 29180 (0.0008) +[2023-10-09 13:22:35,842][86122] Updated weights for policy 1, policy_version 29320 (0.0010) +[2023-10-09 13:22:36,202][86122] Updated weights for policy 1, policy_version 29330 (0.0010) +[2023-10-09 13:22:36,572][86122] Updated weights for policy 1, policy_version 29340 (0.0009) +[2023-10-09 13:22:37,400][86121] Updated weights for policy 0, policy_version 29190 (0.0007) +[2023-10-09 13:22:37,765][86121] Updated weights for policy 0, policy_version 29200 (0.0008) +[2023-10-09 13:22:38,134][86121] Updated weights for policy 0, policy_version 29210 (0.0008) +[2023-10-09 13:22:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 59965440. Throughput: 0: 1819.6, 1: 1814.4. Samples: 14997524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:22:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 13:22:40,401][86122] Updated weights for policy 1, policy_version 29350 (0.0010) +[2023-10-09 13:22:40,762][86122] Updated weights for policy 1, policy_version 29360 (0.0007) +[2023-10-09 13:22:41,130][86122] Updated weights for policy 1, policy_version 29370 (0.0009) +[2023-10-09 13:22:41,703][86121] Updated weights for policy 0, policy_version 29220 (0.0008) +[2023-10-09 13:22:42,069][86121] Updated weights for policy 0, policy_version 29230 (0.0007) +[2023-10-09 13:22:42,449][86121] Updated weights for policy 0, policy_version 29240 (0.0009) +[2023-10-09 13:22:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60030976. Throughput: 0: 1821.7, 1: 1822.7. Samples: 15009126. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) +[2023-10-09 13:22:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.930')] +[2023-10-09 13:22:44,663][86122] Updated weights for policy 1, policy_version 29380 (0.0009) +[2023-10-09 13:22:45,029][86122] Updated weights for policy 1, policy_version 29390 (0.0008) +[2023-10-09 13:22:45,388][86122] Updated weights for policy 1, policy_version 29400 (0.0007) +[2023-10-09 13:22:46,229][86121] Updated weights for policy 0, policy_version 29250 (0.0010) +[2023-10-09 13:22:46,598][86121] Updated weights for policy 0, policy_version 29260 (0.0008) +[2023-10-09 13:22:46,958][86121] Updated weights for policy 0, policy_version 29270 (0.0008) +[2023-10-09 13:22:47,317][86121] Updated weights for policy 0, policy_version 29280 (0.0010) +[2023-10-09 13:22:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 60096512. Throughput: 0: 1822.3, 1: 1831.2. Samples: 15030490. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) +[2023-10-09 13:22:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.930')] +[2023-10-09 13:22:49,017][86122] Updated weights for policy 1, policy_version 29410 (0.0009) +[2023-10-09 13:22:49,386][86122] Updated weights for policy 1, policy_version 29420 (0.0010) +[2023-10-09 13:22:49,737][86122] Updated weights for policy 1, policy_version 29430 (0.0009) +[2023-10-09 13:22:50,100][86122] Updated weights for policy 1, policy_version 29440 (0.0008) +[2023-10-09 13:22:51,161][86121] Updated weights for policy 0, policy_version 29290 (0.0009) +[2023-10-09 13:22:51,534][86121] Updated weights for policy 0, policy_version 29300 (0.0009) +[2023-10-09 13:22:51,895][86121] Updated weights for policy 0, policy_version 29310 (0.0010) +[2023-10-09 13:22:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60162048. Throughput: 0: 1822.8, 1: 1833.2. Samples: 15052980. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) +[2023-10-09 13:22:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 13:22:53,697][86122] Updated weights for policy 1, policy_version 29450 (0.0007) +[2023-10-09 13:22:54,068][86122] Updated weights for policy 1, policy_version 29460 (0.0007) +[2023-10-09 13:22:54,422][86122] Updated weights for policy 1, policy_version 29470 (0.0007) +[2023-10-09 13:22:55,486][86121] Updated weights for policy 0, policy_version 29320 (0.0008) +[2023-10-09 13:22:55,859][86121] Updated weights for policy 0, policy_version 29330 (0.0008) +[2023-10-09 13:22:56,230][86121] Updated weights for policy 0, policy_version 29340 (0.0008) +[2023-10-09 13:22:58,028][86122] Updated weights for policy 1, policy_version 29480 (0.0008) +[2023-10-09 13:22:58,389][86122] Updated weights for policy 1, policy_version 29490 (0.0007) +[2023-10-09 13:22:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60227584. Throughput: 0: 1824.4, 1: 1830.9. Samples: 15063632. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) +[2023-10-09 13:22:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.920')] +[2023-10-09 13:22:58,751][86122] Updated weights for policy 1, policy_version 29500 (0.0008) +[2023-10-09 13:22:59,994][86121] Updated weights for policy 0, policy_version 29350 (0.0007) +[2023-10-09 13:23:00,369][86121] Updated weights for policy 0, policy_version 29360 (0.0008) +[2023-10-09 13:23:00,736][86121] Updated weights for policy 0, policy_version 29370 (0.0009) +[2023-10-09 13:23:02,504][86122] Updated weights for policy 1, policy_version 29510 (0.0010) +[2023-10-09 13:23:02,863][86122] Updated weights for policy 1, policy_version 29520 (0.0008) +[2023-10-09 13:23:03,235][86122] Updated weights for policy 1, policy_version 29530 (0.0009) +[2023-10-09 13:23:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 60293120. Throughput: 0: 1823.1, 1: 1830.8. Samples: 15085750. Policy #0 lag: (min: 31.0, avg: 32.0, max: 51.0) +[2023-10-09 13:23:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.910')] +[2023-10-09 13:23:04,356][86121] Updated weights for policy 0, policy_version 29380 (0.0008) +[2023-10-09 13:23:04,731][86121] Updated weights for policy 0, policy_version 29390 (0.0008) +[2023-10-09 13:23:05,085][86121] Updated weights for policy 0, policy_version 29400 (0.0010) +[2023-10-09 13:23:06,828][86122] Updated weights for policy 1, policy_version 29540 (0.0009) +[2023-10-09 13:23:07,195][86122] Updated weights for policy 1, policy_version 29550 (0.0007) +[2023-10-09 13:23:07,558][86122] Updated weights for policy 1, policy_version 29560 (0.0008) +[2023-10-09 13:23:08,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60391424. Throughput: 0: 1813.7, 1: 1826.9. Samples: 15107372. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.910')] +[2023-10-09 13:23:08,902][86121] Updated weights for policy 0, policy_version 29410 (0.0008) +[2023-10-09 13:23:09,262][86121] Updated weights for policy 0, policy_version 29420 (0.0010) +[2023-10-09 13:23:09,622][86121] Updated weights for policy 0, policy_version 29430 (0.0008) +[2023-10-09 13:23:09,996][86121] Updated weights for policy 0, policy_version 29440 (0.0009) +[2023-10-09 13:23:11,105][86122] Updated weights for policy 1, policy_version 29570 (0.0010) +[2023-10-09 13:23:11,474][86122] Updated weights for policy 1, policy_version 29580 (0.0009) +[2023-10-09 13:23:11,825][86122] Updated weights for policy 1, policy_version 29590 (0.0010) +[2023-10-09 13:23:12,183][86122] Updated weights for policy 1, policy_version 29600 (0.0008) +[2023-10-09 13:23:13,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60456960. Throughput: 0: 1814.9, 1: 1842.4. Samples: 15118878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.910')] +[2023-10-09 13:23:13,731][86121] Updated weights for policy 0, policy_version 29450 (0.0008) +[2023-10-09 13:23:14,098][86121] Updated weights for policy 0, policy_version 29460 (0.0009) +[2023-10-09 13:23:14,475][86121] Updated weights for policy 0, policy_version 29470 (0.0008) +[2023-10-09 13:23:15,916][86122] Updated weights for policy 1, policy_version 29610 (0.0010) +[2023-10-09 13:23:16,288][86122] Updated weights for policy 1, policy_version 29620 (0.0008) +[2023-10-09 13:23:16,648][86122] Updated weights for policy 1, policy_version 29630 (0.0007) +[2023-10-09 13:23:18,148][86121] Updated weights for policy 0, policy_version 29480 (0.0009) +[2023-10-09 13:23:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60522496. Throughput: 0: 1821.6, 1: 1830.5. Samples: 15140256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.910')] +[2023-10-09 13:23:18,509][86121] Updated weights for policy 0, policy_version 29490 (0.0007) +[2023-10-09 13:23:18,873][86121] Updated weights for policy 0, policy_version 29500 (0.0007) +[2023-10-09 13:23:20,513][86122] Updated weights for policy 1, policy_version 29640 (0.0008) +[2023-10-09 13:23:20,874][86122] Updated weights for policy 1, policy_version 29650 (0.0009) +[2023-10-09 13:23:21,244][86122] Updated weights for policy 1, policy_version 29660 (0.0009) +[2023-10-09 13:23:22,616][86121] Updated weights for policy 0, policy_version 29510 (0.0008) +[2023-10-09 13:23:22,989][86121] Updated weights for policy 0, policy_version 29520 (0.0008) +[2023-10-09 13:23:23,350][86121] Updated weights for policy 0, policy_version 29530 (0.0009) +[2023-10-09 13:23:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 60588032. Throughput: 0: 1826.6, 1: 1836.0. Samples: 15162342. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.910')] +[2023-10-09 13:23:23,405][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000029664_30375936.pth... +[2023-10-09 13:23:23,446][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000027968_28639232.pth +[2023-10-09 13:23:23,568][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000029536_30244864.pth... +[2023-10-09 13:23:23,598][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000027840_28508160.pth +[2023-10-09 13:23:24,945][86122] Updated weights for policy 1, policy_version 29670 (0.0008) +[2023-10-09 13:23:25,311][86122] Updated weights for policy 1, policy_version 29680 (0.0008) +[2023-10-09 13:23:25,671][86122] Updated weights for policy 1, policy_version 29690 (0.0011) +[2023-10-09 13:23:27,143][86121] Updated weights for policy 0, policy_version 29540 (0.0008) +[2023-10-09 13:23:27,513][86121] Updated weights for policy 0, policy_version 29550 (0.0007) +[2023-10-09 13:23:27,874][86121] Updated weights for policy 0, policy_version 29560 (0.0007) +[2023-10-09 13:23:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 60686336. Throughput: 0: 1816.4, 1: 1822.8. Samples: 15172886. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.900')] +[2023-10-09 13:23:29,446][86122] Updated weights for policy 1, policy_version 29700 (0.0008) +[2023-10-09 13:23:29,799][86122] Updated weights for policy 1, policy_version 29710 (0.0007) +[2023-10-09 13:23:30,171][86122] Updated weights for policy 1, policy_version 29720 (0.0009) +[2023-10-09 13:23:31,757][86121] Updated weights for policy 0, policy_version 29570 (0.0008) +[2023-10-09 13:23:32,127][86121] Updated weights for policy 0, policy_version 29580 (0.0012) +[2023-10-09 13:23:32,496][86121] Updated weights for policy 0, policy_version 29590 (0.0010) +[2023-10-09 13:23:32,862][86121] Updated weights for policy 0, policy_version 29600 (0.0008) +[2023-10-09 13:23:33,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 60751872. Throughput: 0: 1822.0, 1: 1831.7. Samples: 15194906. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.900')] +[2023-10-09 13:23:33,890][86122] Updated weights for policy 1, policy_version 29730 (0.0009) +[2023-10-09 13:23:34,257][86122] Updated weights for policy 1, policy_version 29740 (0.0009) +[2023-10-09 13:23:34,615][86122] Updated weights for policy 1, policy_version 29750 (0.0009) +[2023-10-09 13:23:34,985][86122] Updated weights for policy 1, policy_version 29760 (0.0010) +[2023-10-09 13:23:36,613][86121] Updated weights for policy 0, policy_version 29610 (0.0008) +[2023-10-09 13:23:36,979][86121] Updated weights for policy 0, policy_version 29620 (0.0008) +[2023-10-09 13:23:37,352][86121] Updated weights for policy 0, policy_version 29630 (0.0007) +[2023-10-09 13:23:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 60817408. Throughput: 0: 1807.1, 1: 1824.9. Samples: 15216420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.900')] +[2023-10-09 13:23:38,683][86122] Updated weights for policy 1, policy_version 29770 (0.0010) +[2023-10-09 13:23:39,047][86122] Updated weights for policy 1, policy_version 29780 (0.0008) +[2023-10-09 13:23:39,417][86122] Updated weights for policy 1, policy_version 29790 (0.0007) +[2023-10-09 13:23:41,143][86121] Updated weights for policy 0, policy_version 29640 (0.0007) +[2023-10-09 13:23:41,520][86121] Updated weights for policy 0, policy_version 29650 (0.0011) +[2023-10-09 13:23:41,884][86121] Updated weights for policy 0, policy_version 29660 (0.0010) +[2023-10-09 13:23:43,101][86122] Updated weights for policy 1, policy_version 29800 (0.0008) +[2023-10-09 13:23:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60882944. Throughput: 0: 1814.0, 1: 1820.3. Samples: 15227172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.910')] +[2023-10-09 13:23:43,479][86122] Updated weights for policy 1, policy_version 29810 (0.0008) +[2023-10-09 13:23:43,844][86122] Updated weights for policy 1, policy_version 29820 (0.0009) +[2023-10-09 13:23:45,571][86121] Updated weights for policy 0, policy_version 29670 (0.0011) +[2023-10-09 13:23:45,947][86121] Updated weights for policy 0, policy_version 29680 (0.0012) +[2023-10-09 13:23:46,302][86121] Updated weights for policy 0, policy_version 29690 (0.0009) +[2023-10-09 13:23:47,634][86122] Updated weights for policy 1, policy_version 29830 (0.0009) +[2023-10-09 13:23:47,991][86122] Updated weights for policy 1, policy_version 29840 (0.0007) +[2023-10-09 13:23:48,351][86122] Updated weights for policy 1, policy_version 29850 (0.0007) +[2023-10-09 13:23:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 60948480. Throughput: 0: 1796.9, 1: 1821.5. Samples: 15248576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.900')] +[2023-10-09 13:23:49,742][86121] Updated weights for policy 0, policy_version 29700 (0.0008) +[2023-10-09 13:23:50,105][86121] Updated weights for policy 0, policy_version 29710 (0.0007) +[2023-10-09 13:23:50,470][86121] Updated weights for policy 0, policy_version 29720 (0.0008) +[2023-10-09 13:23:52,039][86122] Updated weights for policy 1, policy_version 29860 (0.0008) +[2023-10-09 13:23:52,397][86122] Updated weights for policy 1, policy_version 29870 (0.0008) +[2023-10-09 13:23:52,769][86122] Updated weights for policy 1, policy_version 29880 (0.0007) +[2023-10-09 13:23:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61046784. Throughput: 0: 1807.2, 1: 1820.7. Samples: 15270630. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.910')] +[2023-10-09 13:23:54,121][86121] Updated weights for policy 0, policy_version 29730 (0.0008) +[2023-10-09 13:23:54,491][86121] Updated weights for policy 0, policy_version 29740 (0.0007) +[2023-10-09 13:23:54,861][86121] Updated weights for policy 0, policy_version 29750 (0.0008) +[2023-10-09 13:23:55,229][86121] Updated weights for policy 0, policy_version 29760 (0.0009) +[2023-10-09 13:23:56,475][86122] Updated weights for policy 1, policy_version 29890 (0.0007) +[2023-10-09 13:23:56,839][86122] Updated weights for policy 1, policy_version 29900 (0.0007) +[2023-10-09 13:23:57,194][86122] Updated weights for policy 1, policy_version 29910 (0.0008) +[2023-10-09 13:23:57,553][86122] Updated weights for policy 1, policy_version 29920 (0.0009) +[2023-10-09 13:23:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 61112320. Throughput: 0: 1808.7, 1: 1803.1. Samples: 15281414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:23:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.910')] +[2023-10-09 13:23:59,039][86121] Updated weights for policy 0, policy_version 29770 (0.0009) +[2023-10-09 13:23:59,408][86121] Updated weights for policy 0, policy_version 29780 (0.0009) +[2023-10-09 13:23:59,777][86121] Updated weights for policy 0, policy_version 29790 (0.0007) +[2023-10-09 13:24:01,323][86122] Updated weights for policy 1, policy_version 29930 (0.0008) +[2023-10-09 13:24:01,672][86122] Updated weights for policy 1, policy_version 29940 (0.0010) +[2023-10-09 13:24:02,040][86122] Updated weights for policy 1, policy_version 29950 (0.0011) +[2023-10-09 13:24:03,240][86121] Updated weights for policy 0, policy_version 29800 (0.0007) +[2023-10-09 13:24:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 61177856. Throughput: 0: 1798.8, 1: 1815.6. Samples: 15302904. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:24:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.910')] +[2023-10-09 13:24:03,611][86121] Updated weights for policy 0, policy_version 29810 (0.0008) +[2023-10-09 13:24:03,982][86121] Updated weights for policy 0, policy_version 29820 (0.0007) +[2023-10-09 13:24:05,701][86122] Updated weights for policy 1, policy_version 29960 (0.0009) +[2023-10-09 13:24:06,069][86122] Updated weights for policy 1, policy_version 29970 (0.0007) +[2023-10-09 13:24:06,435][86122] Updated weights for policy 1, policy_version 29980 (0.0007) +[2023-10-09 13:24:07,909][86121] Updated weights for policy 0, policy_version 29830 (0.0008) +[2023-10-09 13:24:08,287][86121] Updated weights for policy 0, policy_version 29840 (0.0010) +[2023-10-09 13:24:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 61243392. Throughput: 0: 1804.6, 1: 1812.8. Samples: 15325126. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:24:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.920')] +[2023-10-09 13:24:08,646][86121] Updated weights for policy 0, policy_version 29850 (0.0008) +[2023-10-09 13:24:09,989][86122] Updated weights for policy 1, policy_version 29990 (0.0007) +[2023-10-09 13:24:10,352][86122] Updated weights for policy 1, policy_version 30000 (0.0007) +[2023-10-09 13:24:10,711][86122] Updated weights for policy 1, policy_version 30010 (0.0008) +[2023-10-09 13:24:12,399][86121] Updated weights for policy 0, policy_version 29860 (0.0008) +[2023-10-09 13:24:12,763][86121] Updated weights for policy 0, policy_version 29870 (0.0007) +[2023-10-09 13:24:13,139][86121] Updated weights for policy 0, policy_version 29880 (0.0008) +[2023-10-09 13:24:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 61308928. Throughput: 0: 1797.2, 1: 1817.3. Samples: 15335538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:24:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 13:24:14,433][86122] Updated weights for policy 1, policy_version 30020 (0.0010) +[2023-10-09 13:24:14,797][86122] Updated weights for policy 1, policy_version 30030 (0.0007) +[2023-10-09 13:24:15,169][86122] Updated weights for policy 1, policy_version 30040 (0.0009) +[2023-10-09 13:24:16,763][86121] Updated weights for policy 0, policy_version 29890 (0.0008) +[2023-10-09 13:24:17,138][86121] Updated weights for policy 0, policy_version 29900 (0.0007) +[2023-10-09 13:24:17,499][86121] Updated weights for policy 0, policy_version 29910 (0.0007) +[2023-10-09 13:24:17,869][86121] Updated weights for policy 0, policy_version 29920 (0.0007) +[2023-10-09 13:24:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61407232. Throughput: 0: 1811.1, 1: 1811.8. Samples: 15357934. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) +[2023-10-09 13:24:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:24:18,816][86122] Updated weights for policy 1, policy_version 30050 (0.0009) +[2023-10-09 13:24:19,179][86122] Updated weights for policy 1, policy_version 30060 (0.0009) +[2023-10-09 13:24:19,543][86122] Updated weights for policy 1, policy_version 30070 (0.0007) +[2023-10-09 13:24:19,909][86122] Updated weights for policy 1, policy_version 30080 (0.0007) +[2023-10-09 13:24:21,619][86121] Updated weights for policy 0, policy_version 29930 (0.0007) +[2023-10-09 13:24:21,997][86121] Updated weights for policy 0, policy_version 29940 (0.0008) +[2023-10-09 13:24:22,362][86121] Updated weights for policy 0, policy_version 29950 (0.0008) +[2023-10-09 13:24:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 61472768. Throughput: 0: 1812.7, 1: 1816.5. Samples: 15379736. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) +[2023-10-09 13:24:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:24:23,548][86122] Updated weights for policy 1, policy_version 30090 (0.0008) +[2023-10-09 13:24:23,909][86122] Updated weights for policy 1, policy_version 30100 (0.0010) +[2023-10-09 13:24:24,276][86122] Updated weights for policy 1, policy_version 30110 (0.0009) +[2023-10-09 13:24:26,023][86121] Updated weights for policy 0, policy_version 29960 (0.0008) +[2023-10-09 13:24:26,392][86121] Updated weights for policy 0, policy_version 29970 (0.0007) +[2023-10-09 13:24:26,751][86121] Updated weights for policy 0, policy_version 29980 (0.0010) +[2023-10-09 13:24:28,169][86122] Updated weights for policy 1, policy_version 30120 (0.0010) +[2023-10-09 13:24:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 61538304. Throughput: 0: 1819.2, 1: 1820.2. Samples: 15390948. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) +[2023-10-09 13:24:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:24:28,533][86122] Updated weights for policy 1, policy_version 30130 (0.0012) +[2023-10-09 13:24:28,899][86122] Updated weights for policy 1, policy_version 30140 (0.0008) +[2023-10-09 13:24:30,422][86121] Updated weights for policy 0, policy_version 29990 (0.0009) +[2023-10-09 13:24:30,787][86121] Updated weights for policy 0, policy_version 30000 (0.0008) +[2023-10-09 13:24:31,156][86121] Updated weights for policy 0, policy_version 30010 (0.0010) +[2023-10-09 13:24:32,529][86122] Updated weights for policy 1, policy_version 30150 (0.0008) +[2023-10-09 13:24:32,898][86122] Updated weights for policy 1, policy_version 30160 (0.0007) +[2023-10-09 13:24:33,260][86122] Updated weights for policy 1, policy_version 30170 (0.0008) +[2023-10-09 13:24:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 61603840. Throughput: 0: 1820.5, 1: 1817.6. Samples: 15412290. Policy #0 lag: (min: 31.0, avg: 36.7, max: 63.0) +[2023-10-09 13:24:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:24:34,860][86121] Updated weights for policy 0, policy_version 30020 (0.0009) +[2023-10-09 13:24:35,224][86121] Updated weights for policy 0, policy_version 30030 (0.0010) +[2023-10-09 13:24:35,597][86121] Updated weights for policy 0, policy_version 30040 (0.0009) +[2023-10-09 13:24:36,757][86122] Updated weights for policy 1, policy_version 30180 (0.0009) +[2023-10-09 13:24:37,113][86122] Updated weights for policy 1, policy_version 30190 (0.0007) +[2023-10-09 13:24:37,477][86122] Updated weights for policy 1, policy_version 30200 (0.0009) +[2023-10-09 13:24:38,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61702144. Throughput: 0: 1817.7, 1: 1818.6. Samples: 15434266. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:24:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:24:38,887][86121] Updated weights for policy 0, policy_version 30050 (0.0008) +[2023-10-09 13:24:39,264][86121] Updated weights for policy 0, policy_version 30060 (0.0009) +[2023-10-09 13:24:39,626][86121] Updated weights for policy 0, policy_version 30070 (0.0009) +[2023-10-09 13:24:40,005][86121] Updated weights for policy 0, policy_version 30080 (0.0010) +[2023-10-09 13:24:41,192][86122] Updated weights for policy 1, policy_version 30210 (0.0007) +[2023-10-09 13:24:41,561][86122] Updated weights for policy 1, policy_version 30220 (0.0009) +[2023-10-09 13:24:41,925][86122] Updated weights for policy 1, policy_version 30230 (0.0010) +[2023-10-09 13:24:42,289][86122] Updated weights for policy 1, policy_version 30240 (0.0011) +[2023-10-09 13:24:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61767680. Throughput: 0: 1820.8, 1: 1828.1. Samples: 15445610. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:24:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:24:43,783][86121] Updated weights for policy 0, policy_version 30090 (0.0007) +[2023-10-09 13:24:44,145][86121] Updated weights for policy 0, policy_version 30100 (0.0010) +[2023-10-09 13:24:44,513][86121] Updated weights for policy 0, policy_version 30110 (0.0009) +[2023-10-09 13:24:46,093][86122] Updated weights for policy 1, policy_version 30250 (0.0007) +[2023-10-09 13:24:46,448][86122] Updated weights for policy 1, policy_version 30260 (0.0010) +[2023-10-09 13:24:46,809][86122] Updated weights for policy 1, policy_version 30270 (0.0007) +[2023-10-09 13:24:48,203][86121] Updated weights for policy 0, policy_version 30120 (0.0009) +[2023-10-09 13:24:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 61833216. Throughput: 0: 1836.3, 1: 1820.5. Samples: 15467458. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:24:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:24:48,570][86121] Updated weights for policy 0, policy_version 30130 (0.0011) +[2023-10-09 13:24:48,942][86121] Updated weights for policy 0, policy_version 30140 (0.0010) +[2023-10-09 13:24:50,363][86122] Updated weights for policy 1, policy_version 30280 (0.0007) +[2023-10-09 13:24:50,728][86122] Updated weights for policy 1, policy_version 30290 (0.0009) +[2023-10-09 13:24:51,082][86122] Updated weights for policy 1, policy_version 30300 (0.0010) +[2023-10-09 13:24:52,716][86121] Updated weights for policy 0, policy_version 30150 (0.0009) +[2023-10-09 13:24:53,072][86121] Updated weights for policy 0, policy_version 30160 (0.0008) +[2023-10-09 13:24:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 61898752. Throughput: 0: 1828.3, 1: 1826.0. Samples: 15489572. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:24:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:24:53,433][86121] Updated weights for policy 0, policy_version 30170 (0.0008) +[2023-10-09 13:24:54,769][86122] Updated weights for policy 1, policy_version 30310 (0.0009) +[2023-10-09 13:24:55,123][86122] Updated weights for policy 1, policy_version 30320 (0.0009) +[2023-10-09 13:24:55,483][86122] Updated weights for policy 1, policy_version 30330 (0.0009) +[2023-10-09 13:24:57,154][86121] Updated weights for policy 0, policy_version 30180 (0.0011) +[2023-10-09 13:24:57,519][86121] Updated weights for policy 0, policy_version 30190 (0.0008) +[2023-10-09 13:24:57,888][86121] Updated weights for policy 0, policy_version 30200 (0.0009) +[2023-10-09 13:24:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 61997056. Throughput: 0: 1835.9, 1: 1816.9. Samples: 15499912. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) +[2023-10-09 13:24:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:24:59,337][86122] Updated weights for policy 1, policy_version 30340 (0.0008) +[2023-10-09 13:24:59,696][86122] Updated weights for policy 1, policy_version 30350 (0.0007) +[2023-10-09 13:25:00,056][86122] Updated weights for policy 1, policy_version 30360 (0.0008) +[2023-10-09 13:25:01,638][86121] Updated weights for policy 0, policy_version 30210 (0.0008) +[2023-10-09 13:25:02,005][86121] Updated weights for policy 0, policy_version 30220 (0.0008) +[2023-10-09 13:25:02,379][86121] Updated weights for policy 0, policy_version 30230 (0.0008) +[2023-10-09 13:25:02,750][86121] Updated weights for policy 0, policy_version 30240 (0.0008) +[2023-10-09 13:25:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62062592. Throughput: 0: 1826.6, 1: 1822.5. Samples: 15522144. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) +[2023-10-09 13:25:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:25:03,777][86122] Updated weights for policy 1, policy_version 30370 (0.0010) +[2023-10-09 13:25:04,145][86122] Updated weights for policy 1, policy_version 30380 (0.0010) +[2023-10-09 13:25:04,507][86122] Updated weights for policy 1, policy_version 30390 (0.0008) +[2023-10-09 13:25:04,871][86122] Updated weights for policy 1, policy_version 30400 (0.0008) +[2023-10-09 13:25:06,519][86121] Updated weights for policy 0, policy_version 30250 (0.0010) +[2023-10-09 13:25:06,877][86121] Updated weights for policy 0, policy_version 30260 (0.0011) +[2023-10-09 13:25:07,244][86121] Updated weights for policy 0, policy_version 30270 (0.0008) +[2023-10-09 13:25:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62128128. Throughput: 0: 1829.7, 1: 1824.6. Samples: 15544176. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) +[2023-10-09 13:25:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:25:08,574][86122] Updated weights for policy 1, policy_version 30410 (0.0009) +[2023-10-09 13:25:08,936][86122] Updated weights for policy 1, policy_version 30420 (0.0008) +[2023-10-09 13:25:09,299][86122] Updated weights for policy 1, policy_version 30430 (0.0010) +[2023-10-09 13:25:10,909][86121] Updated weights for policy 0, policy_version 30280 (0.0009) +[2023-10-09 13:25:11,270][86121] Updated weights for policy 0, policy_version 30290 (0.0009) +[2023-10-09 13:25:11,637][86121] Updated weights for policy 0, policy_version 30300 (0.0009) +[2023-10-09 13:25:13,057][86122] Updated weights for policy 1, policy_version 30440 (0.0009) +[2023-10-09 13:25:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 62193664. Throughput: 0: 1826.5, 1: 1822.9. Samples: 15555172. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) +[2023-10-09 13:25:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:25:13,429][86122] Updated weights for policy 1, policy_version 30450 (0.0008) +[2023-10-09 13:25:13,791][86122] Updated weights for policy 1, policy_version 30460 (0.0007) +[2023-10-09 13:25:15,330][86121] Updated weights for policy 0, policy_version 30310 (0.0009) +[2023-10-09 13:25:15,696][86121] Updated weights for policy 0, policy_version 30320 (0.0011) +[2023-10-09 13:25:16,071][86121] Updated weights for policy 0, policy_version 30330 (0.0008) +[2023-10-09 13:25:17,502][86122] Updated weights for policy 1, policy_version 30470 (0.0008) +[2023-10-09 13:25:17,868][86122] Updated weights for policy 1, policy_version 30480 (0.0008) +[2023-10-09 13:25:18,230][86122] Updated weights for policy 1, policy_version 30490 (0.0008) +[2023-10-09 13:25:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62259200. Throughput: 0: 1829.9, 1: 1827.7. Samples: 15576884. Policy #0 lag: (min: 26.0, avg: 37.0, max: 58.0) +[2023-10-09 13:25:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:25:19,674][86121] Updated weights for policy 0, policy_version 30340 (0.0009) +[2023-10-09 13:25:20,038][86121] Updated weights for policy 0, policy_version 30350 (0.0008) +[2023-10-09 13:25:20,402][86121] Updated weights for policy 0, policy_version 30360 (0.0009) +[2023-10-09 13:25:21,791][86122] Updated weights for policy 1, policy_version 30500 (0.0009) +[2023-10-09 13:25:22,158][86122] Updated weights for policy 1, policy_version 30510 (0.0009) +[2023-10-09 13:25:22,526][86122] Updated weights for policy 1, policy_version 30520 (0.0008) +[2023-10-09 13:25:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62357504. Throughput: 0: 1825.7, 1: 1821.7. Samples: 15598400. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:25:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:25:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000030368_31096832.pth... +[2023-10-09 13:25:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000030528_31260672.pth... +[2023-10-09 13:25:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000028800_29491200.pth +[2023-10-09 13:25:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000028672_29360128.pth +[2023-10-09 13:25:24,069][86121] Updated weights for policy 0, policy_version 30370 (0.0009) +[2023-10-09 13:25:24,443][86121] Updated weights for policy 0, policy_version 30380 (0.0008) +[2023-10-09 13:25:24,808][86121] Updated weights for policy 0, policy_version 30390 (0.0008) +[2023-10-09 13:25:25,178][86121] Updated weights for policy 0, policy_version 30400 (0.0011) +[2023-10-09 13:25:26,289][86122] Updated weights for policy 1, policy_version 30530 (0.0008) +[2023-10-09 13:25:26,653][86122] Updated weights for policy 1, policy_version 30540 (0.0008) +[2023-10-09 13:25:27,010][86122] Updated weights for policy 1, policy_version 30550 (0.0010) +[2023-10-09 13:25:27,368][86122] Updated weights for policy 1, policy_version 30560 (0.0009) +[2023-10-09 13:25:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 62423040. Throughput: 0: 1825.2, 1: 1823.2. Samples: 15609788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:25:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:25:28,800][86121] Updated weights for policy 0, policy_version 30410 (0.0008) +[2023-10-09 13:25:29,169][86121] Updated weights for policy 0, policy_version 30420 (0.0008) +[2023-10-09 13:25:29,537][86121] Updated weights for policy 0, policy_version 30430 (0.0010) +[2023-10-09 13:25:31,187][86122] Updated weights for policy 1, policy_version 30570 (0.0008) +[2023-10-09 13:25:31,558][86122] Updated weights for policy 1, policy_version 30580 (0.0007) +[2023-10-09 13:25:31,912][86122] Updated weights for policy 1, policy_version 30590 (0.0009) +[2023-10-09 13:25:33,228][86121] Updated weights for policy 0, policy_version 30440 (0.0010) +[2023-10-09 13:25:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62488576. Throughput: 0: 1819.6, 1: 1819.8. Samples: 15631230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:25:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:25:33,594][86121] Updated weights for policy 0, policy_version 30450 (0.0007) +[2023-10-09 13:25:33,960][86121] Updated weights for policy 0, policy_version 30460 (0.0011) +[2023-10-09 13:25:35,471][86122] Updated weights for policy 1, policy_version 30600 (0.0011) +[2023-10-09 13:25:35,845][86122] Updated weights for policy 1, policy_version 30610 (0.0009) +[2023-10-09 13:25:36,213][86122] Updated weights for policy 1, policy_version 30620 (0.0008) +[2023-10-09 13:25:37,762][86121] Updated weights for policy 0, policy_version 30470 (0.0009) +[2023-10-09 13:25:38,132][86121] Updated weights for policy 0, policy_version 30480 (0.0009) +[2023-10-09 13:25:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 62554112. Throughput: 0: 1822.6, 1: 1819.2. Samples: 15653456. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:25:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:25:38,499][86121] Updated weights for policy 0, policy_version 30490 (0.0009) +[2023-10-09 13:25:39,782][86122] Updated weights for policy 1, policy_version 30630 (0.0011) +[2023-10-09 13:25:40,144][86122] Updated weights for policy 1, policy_version 30640 (0.0009) +[2023-10-09 13:25:40,514][86122] Updated weights for policy 1, policy_version 30650 (0.0010) +[2023-10-09 13:25:42,059][86121] Updated weights for policy 0, policy_version 30500 (0.0008) +[2023-10-09 13:25:42,424][86121] Updated weights for policy 0, policy_version 30510 (0.0007) +[2023-10-09 13:25:42,795][86121] Updated weights for policy 0, policy_version 30520 (0.0010) +[2023-10-09 13:25:43,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 62652416. Throughput: 0: 1821.6, 1: 1825.5. Samples: 15664030. Policy #0 lag: (min: 16.0, avg: 44.0, max: 48.0) +[2023-10-09 13:25:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:25:44,228][86122] Updated weights for policy 1, policy_version 30660 (0.0009) +[2023-10-09 13:25:44,593][86122] Updated weights for policy 1, policy_version 30670 (0.0010) +[2023-10-09 13:25:44,956][86122] Updated weights for policy 1, policy_version 30680 (0.0011) +[2023-10-09 13:25:46,537][86121] Updated weights for policy 0, policy_version 30530 (0.0010) +[2023-10-09 13:25:46,909][86121] Updated weights for policy 0, policy_version 30540 (0.0011) +[2023-10-09 13:25:47,278][86121] Updated weights for policy 0, policy_version 30550 (0.0011) +[2023-10-09 13:25:47,645][86121] Updated weights for policy 0, policy_version 30560 (0.0009) +[2023-10-09 13:25:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 62717952. Throughput: 0: 1818.4, 1: 1821.9. Samples: 15685960. Policy #0 lag: (min: 16.0, avg: 44.0, max: 48.0) +[2023-10-09 13:25:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 13:25:48,667][86122] Updated weights for policy 1, policy_version 30690 (0.0010) +[2023-10-09 13:25:49,037][86122] Updated weights for policy 1, policy_version 30700 (0.0009) +[2023-10-09 13:25:49,391][86122] Updated weights for policy 1, policy_version 30710 (0.0007) +[2023-10-09 13:25:49,760][86122] Updated weights for policy 1, policy_version 30720 (0.0007) +[2023-10-09 13:25:51,467][86121] Updated weights for policy 0, policy_version 30570 (0.0007) +[2023-10-09 13:25:51,826][86121] Updated weights for policy 0, policy_version 30580 (0.0008) +[2023-10-09 13:25:52,199][86121] Updated weights for policy 0, policy_version 30590 (0.0009) +[2023-10-09 13:25:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 62783488. Throughput: 0: 1814.4, 1: 1814.7. Samples: 15707484. Policy #0 lag: (min: 16.0, avg: 44.0, max: 48.0) +[2023-10-09 13:25:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:25:53,517][86122] Updated weights for policy 1, policy_version 30730 (0.0011) +[2023-10-09 13:25:53,880][86122] Updated weights for policy 1, policy_version 30740 (0.0010) +[2023-10-09 13:25:54,243][86122] Updated weights for policy 1, policy_version 30750 (0.0008) +[2023-10-09 13:25:55,881][86121] Updated weights for policy 0, policy_version 30600 (0.0008) +[2023-10-09 13:25:56,253][86121] Updated weights for policy 0, policy_version 30610 (0.0008) +[2023-10-09 13:25:56,617][86121] Updated weights for policy 0, policy_version 30620 (0.0007) +[2023-10-09 13:25:58,211][86122] Updated weights for policy 1, policy_version 30760 (0.0011) +[2023-10-09 13:25:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62849024. Throughput: 0: 1811.6, 1: 1817.7. Samples: 15718492. Policy #0 lag: (min: 16.0, avg: 44.0, max: 48.0) +[2023-10-09 13:25:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:25:58,580][86122] Updated weights for policy 1, policy_version 30770 (0.0008) +[2023-10-09 13:25:58,945][86122] Updated weights for policy 1, policy_version 30780 (0.0012) +[2023-10-09 13:26:00,321][86121] Updated weights for policy 0, policy_version 30630 (0.0009) +[2023-10-09 13:26:00,687][86121] Updated weights for policy 0, policy_version 30640 (0.0011) +[2023-10-09 13:26:01,056][86121] Updated weights for policy 0, policy_version 30650 (0.0010) +[2023-10-09 13:26:02,529][86122] Updated weights for policy 1, policy_version 30790 (0.0009) +[2023-10-09 13:26:02,894][86122] Updated weights for policy 1, policy_version 30800 (0.0008) +[2023-10-09 13:26:03,263][86122] Updated weights for policy 1, policy_version 30810 (0.0008) +[2023-10-09 13:26:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 62914560. Throughput: 0: 1813.1, 1: 1818.5. Samples: 15740308. Policy #0 lag: (min: 16.0, avg: 44.0, max: 48.0) +[2023-10-09 13:26:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:26:04,923][86121] Updated weights for policy 0, policy_version 30660 (0.0010) +[2023-10-09 13:26:05,287][86121] Updated weights for policy 0, policy_version 30670 (0.0007) +[2023-10-09 13:26:05,656][86121] Updated weights for policy 0, policy_version 30680 (0.0008) +[2023-10-09 13:26:06,850][86122] Updated weights for policy 1, policy_version 30820 (0.0007) +[2023-10-09 13:26:07,208][86122] Updated weights for policy 1, policy_version 30830 (0.0009) +[2023-10-09 13:26:07,572][86122] Updated weights for policy 1, policy_version 30840 (0.0008) +[2023-10-09 13:26:08,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 63012864. Throughput: 0: 1809.7, 1: 1822.9. Samples: 15761868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:26:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:26:09,427][86121] Updated weights for policy 0, policy_version 30690 (0.0010) +[2023-10-09 13:26:09,791][86121] Updated weights for policy 0, policy_version 30700 (0.0007) +[2023-10-09 13:26:10,160][86121] Updated weights for policy 0, policy_version 30710 (0.0009) +[2023-10-09 13:26:10,526][86121] Updated weights for policy 0, policy_version 30720 (0.0009) +[2023-10-09 13:26:11,268][86122] Updated weights for policy 1, policy_version 30850 (0.0007) +[2023-10-09 13:26:11,632][86122] Updated weights for policy 1, policy_version 30860 (0.0009) +[2023-10-09 13:26:11,987][86122] Updated weights for policy 1, policy_version 30870 (0.0007) +[2023-10-09 13:26:12,351][86122] Updated weights for policy 1, policy_version 30880 (0.0007) +[2023-10-09 13:26:13,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63078400. Throughput: 0: 1808.0, 1: 1823.8. Samples: 15773220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:26:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:26:14,220][86121] Updated weights for policy 0, policy_version 30730 (0.0010) +[2023-10-09 13:26:14,586][86121] Updated weights for policy 0, policy_version 30740 (0.0009) +[2023-10-09 13:26:14,962][86121] Updated weights for policy 0, policy_version 30750 (0.0007) +[2023-10-09 13:26:15,949][86122] Updated weights for policy 1, policy_version 30890 (0.0007) +[2023-10-09 13:26:16,309][86122] Updated weights for policy 1, policy_version 30900 (0.0008) +[2023-10-09 13:26:16,682][86122] Updated weights for policy 1, policy_version 30910 (0.0008) +[2023-10-09 13:26:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 63143936. Throughput: 0: 1805.9, 1: 1825.6. Samples: 15794650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:26:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 13:26:18,592][86121] Updated weights for policy 0, policy_version 30760 (0.0009) +[2023-10-09 13:26:18,952][86121] Updated weights for policy 0, policy_version 30770 (0.0011) +[2023-10-09 13:26:19,323][86121] Updated weights for policy 0, policy_version 30780 (0.0010) +[2023-10-09 13:26:20,429][86122] Updated weights for policy 1, policy_version 30920 (0.0008) +[2023-10-09 13:26:20,800][86122] Updated weights for policy 1, policy_version 30930 (0.0010) +[2023-10-09 13:26:21,162][86122] Updated weights for policy 1, policy_version 30940 (0.0010) +[2023-10-09 13:26:22,970][86121] Updated weights for policy 0, policy_version 30790 (0.0009) +[2023-10-09 13:26:23,322][86121] Updated weights for policy 0, policy_version 30800 (0.0008) +[2023-10-09 13:26:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63209472. Throughput: 0: 1813.6, 1: 1825.6. Samples: 15817220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:26:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 13:26:23,682][86121] Updated weights for policy 0, policy_version 30810 (0.0007) +[2023-10-09 13:26:24,625][86122] Updated weights for policy 1, policy_version 30950 (0.0008) +[2023-10-09 13:26:24,985][86122] Updated weights for policy 1, policy_version 30960 (0.0008) +[2023-10-09 13:26:25,346][86122] Updated weights for policy 1, policy_version 30970 (0.0008) +[2023-10-09 13:26:27,338][86121] Updated weights for policy 0, policy_version 30820 (0.0009) +[2023-10-09 13:26:27,711][86121] Updated weights for policy 0, policy_version 30830 (0.0007) +[2023-10-09 13:26:28,079][86121] Updated weights for policy 0, policy_version 30840 (0.0008) +[2023-10-09 13:26:28,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63307776. Throughput: 0: 1809.7, 1: 1825.0. Samples: 15827592. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 13:26:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 13:26:29,203][86122] Updated weights for policy 1, policy_version 30980 (0.0008) +[2023-10-09 13:26:29,566][86122] Updated weights for policy 1, policy_version 30990 (0.0008) +[2023-10-09 13:26:29,927][86122] Updated weights for policy 1, policy_version 31000 (0.0008) +[2023-10-09 13:26:31,701][86121] Updated weights for policy 0, policy_version 30850 (0.0009) +[2023-10-09 13:26:32,059][86121] Updated weights for policy 0, policy_version 30860 (0.0007) +[2023-10-09 13:26:32,436][86121] Updated weights for policy 0, policy_version 30870 (0.0008) +[2023-10-09 13:26:32,804][86121] Updated weights for policy 0, policy_version 30880 (0.0007) +[2023-10-09 13:26:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63373312. Throughput: 0: 1815.6, 1: 1828.3. Samples: 15849934. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 13:26:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 13:26:33,548][86122] Updated weights for policy 1, policy_version 31010 (0.0007) +[2023-10-09 13:26:33,913][86122] Updated weights for policy 1, policy_version 31020 (0.0008) +[2023-10-09 13:26:34,280][86122] Updated weights for policy 1, policy_version 31030 (0.0008) +[2023-10-09 13:26:34,637][86122] Updated weights for policy 1, policy_version 31040 (0.0010) +[2023-10-09 13:26:36,564][86121] Updated weights for policy 0, policy_version 30890 (0.0007) +[2023-10-09 13:26:36,932][86121] Updated weights for policy 0, policy_version 30900 (0.0008) +[2023-10-09 13:26:37,297][86121] Updated weights for policy 0, policy_version 30910 (0.0007) +[2023-10-09 13:26:38,358][86122] Updated weights for policy 1, policy_version 31050 (0.0009) +[2023-10-09 13:26:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63438848. Throughput: 0: 1819.6, 1: 1830.8. Samples: 15871750. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 13:26:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 13:26:38,715][86122] Updated weights for policy 1, policy_version 31060 (0.0009) +[2023-10-09 13:26:39,070][86122] Updated weights for policy 1, policy_version 31070 (0.0008) +[2023-10-09 13:26:40,794][86121] Updated weights for policy 0, policy_version 30920 (0.0009) +[2023-10-09 13:26:41,161][86121] Updated weights for policy 0, policy_version 30930 (0.0009) +[2023-10-09 13:26:41,529][86121] Updated weights for policy 0, policy_version 30940 (0.0010) +[2023-10-09 13:26:42,686][86122] Updated weights for policy 1, policy_version 31080 (0.0008) +[2023-10-09 13:26:43,051][86122] Updated weights for policy 1, policy_version 31090 (0.0008) +[2023-10-09 13:26:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 63504384. Throughput: 0: 1822.6, 1: 1834.1. Samples: 15883044. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 13:26:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 13:26:43,427][86122] Updated weights for policy 1, policy_version 31100 (0.0011) +[2023-10-09 13:26:45,193][86121] Updated weights for policy 0, policy_version 30950 (0.0010) +[2023-10-09 13:26:45,565][86121] Updated weights for policy 0, policy_version 30960 (0.0008) +[2023-10-09 13:26:45,943][86121] Updated weights for policy 0, policy_version 30970 (0.0008) +[2023-10-09 13:26:47,270][86122] Updated weights for policy 1, policy_version 31110 (0.0011) +[2023-10-09 13:26:47,624][86122] Updated weights for policy 1, policy_version 31120 (0.0012) +[2023-10-09 13:26:47,985][86122] Updated weights for policy 1, policy_version 31130 (0.0011) +[2023-10-09 13:26:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63602688. Throughput: 0: 1829.2, 1: 1835.2. Samples: 15905208. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 13:26:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 13:26:49,515][86121] Updated weights for policy 0, policy_version 30980 (0.0008) +[2023-10-09 13:26:49,876][86121] Updated weights for policy 0, policy_version 30990 (0.0008) +[2023-10-09 13:26:50,253][86121] Updated weights for policy 0, policy_version 31000 (0.0008) +[2023-10-09 13:26:51,787][86122] Updated weights for policy 1, policy_version 31140 (0.0010) +[2023-10-09 13:26:52,152][86122] Updated weights for policy 1, policy_version 31150 (0.0008) +[2023-10-09 13:26:52,527][86122] Updated weights for policy 1, policy_version 31160 (0.0007) +[2023-10-09 13:26:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63668224. Throughput: 0: 1839.2, 1: 1827.0. Samples: 15926846. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 13:26:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 13:26:53,814][86121] Updated weights for policy 0, policy_version 31010 (0.0008) +[2023-10-09 13:26:54,182][86121] Updated weights for policy 0, policy_version 31020 (0.0011) +[2023-10-09 13:26:54,551][86121] Updated weights for policy 0, policy_version 31030 (0.0010) +[2023-10-09 13:26:54,926][86121] Updated weights for policy 0, policy_version 31040 (0.0011) +[2023-10-09 13:26:56,262][86122] Updated weights for policy 1, policy_version 31170 (0.0008) +[2023-10-09 13:26:56,628][86122] Updated weights for policy 1, policy_version 31180 (0.0010) +[2023-10-09 13:26:57,002][86122] Updated weights for policy 1, policy_version 31190 (0.0011) +[2023-10-09 13:26:57,366][86122] Updated weights for policy 1, policy_version 31200 (0.0008) +[2023-10-09 13:26:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63733760. Throughput: 0: 1841.3, 1: 1822.3. Samples: 15938082. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 13:26:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 13:26:58,630][86121] Updated weights for policy 0, policy_version 31050 (0.0010) +[2023-10-09 13:26:58,999][86121] Updated weights for policy 0, policy_version 31060 (0.0008) +[2023-10-09 13:26:59,359][86121] Updated weights for policy 0, policy_version 31070 (0.0007) +[2023-10-09 13:27:01,080][86122] Updated weights for policy 1, policy_version 31210 (0.0008) +[2023-10-09 13:27:01,441][86122] Updated weights for policy 1, policy_version 31220 (0.0008) +[2023-10-09 13:27:01,795][86122] Updated weights for policy 1, policy_version 31230 (0.0009) +[2023-10-09 13:27:03,037][86121] Updated weights for policy 0, policy_version 31080 (0.0008) +[2023-10-09 13:27:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 63799296. Throughput: 0: 1844.3, 1: 1824.9. Samples: 15959764. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 13:27:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 13:27:03,400][86121] Updated weights for policy 0, policy_version 31090 (0.0008) +[2023-10-09 13:27:03,765][86121] Updated weights for policy 0, policy_version 31100 (0.0007) +[2023-10-09 13:27:05,515][86122] Updated weights for policy 1, policy_version 31240 (0.0009) +[2023-10-09 13:27:05,885][86122] Updated weights for policy 1, policy_version 31250 (0.0007) +[2023-10-09 13:27:06,257][86122] Updated weights for policy 1, policy_version 31260 (0.0008) +[2023-10-09 13:27:07,341][86121] Updated weights for policy 0, policy_version 31110 (0.0008) +[2023-10-09 13:27:07,708][86121] Updated weights for policy 0, policy_version 31120 (0.0009) +[2023-10-09 13:27:08,082][86121] Updated weights for policy 0, policy_version 31130 (0.0008) +[2023-10-09 13:27:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63897600. Throughput: 0: 1826.2, 1: 1820.2. Samples: 15981306. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 13:27:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 13:27:09,932][86122] Updated weights for policy 1, policy_version 31270 (0.0008) +[2023-10-09 13:27:10,297][86122] Updated weights for policy 1, policy_version 31280 (0.0009) +[2023-10-09 13:27:10,655][86122] Updated weights for policy 1, policy_version 31290 (0.0009) +[2023-10-09 13:27:11,763][86121] Updated weights for policy 0, policy_version 31140 (0.0011) +[2023-10-09 13:27:12,139][86121] Updated weights for policy 0, policy_version 31150 (0.0011) +[2023-10-09 13:27:12,504][86121] Updated weights for policy 0, policy_version 31160 (0.0010) +[2023-10-09 13:27:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 63963136. Throughput: 0: 1844.4, 1: 1822.9. Samples: 15992624. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) +[2023-10-09 13:27:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.930')] +[2023-10-09 13:27:14,041][86122] Updated weights for policy 1, policy_version 31300 (0.0007) +[2023-10-09 13:27:14,411][86122] Updated weights for policy 1, policy_version 31310 (0.0007) +[2023-10-09 13:27:14,774][86122] Updated weights for policy 1, policy_version 31320 (0.0008) +[2023-10-09 13:27:16,224][86121] Updated weights for policy 0, policy_version 31170 (0.0010) +[2023-10-09 13:27:16,597][86121] Updated weights for policy 0, policy_version 31180 (0.0008) +[2023-10-09 13:27:16,961][86121] Updated weights for policy 0, policy_version 31190 (0.0009) +[2023-10-09 13:27:17,325][86121] Updated weights for policy 0, policy_version 31200 (0.0009) +[2023-10-09 13:27:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64028672. Throughput: 0: 1833.8, 1: 1828.6. Samples: 16014744. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) +[2023-10-09 13:27:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.940')] +[2023-10-09 13:27:18,471][86122] Updated weights for policy 1, policy_version 31330 (0.0007) +[2023-10-09 13:27:18,840][86122] Updated weights for policy 1, policy_version 31340 (0.0008) +[2023-10-09 13:27:19,202][86122] Updated weights for policy 1, policy_version 31350 (0.0008) +[2023-10-09 13:27:19,561][86122] Updated weights for policy 1, policy_version 31360 (0.0008) +[2023-10-09 13:27:21,093][86121] Updated weights for policy 0, policy_version 31210 (0.0010) +[2023-10-09 13:27:21,469][86121] Updated weights for policy 0, policy_version 31220 (0.0009) +[2023-10-09 13:27:21,835][86121] Updated weights for policy 0, policy_version 31230 (0.0008) +[2023-10-09 13:27:23,360][86122] Updated weights for policy 1, policy_version 31370 (0.0008) +[2023-10-09 13:27:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 64094208. Throughput: 0: 1843.8, 1: 1827.5. Samples: 16036956. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) +[2023-10-09 13:27:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:27:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000031232_31981568.pth... +[2023-10-09 13:27:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000029536_30244864.pth +[2023-10-09 13:27:23,727][86122] Updated weights for policy 1, policy_version 31380 (0.0009) +[2023-10-09 13:27:24,093][86122] Updated weights for policy 1, policy_version 31390 (0.0010) +[2023-10-09 13:27:24,166][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000031392_32145408.pth... +[2023-10-09 13:27:24,205][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000029664_30375936.pth +[2023-10-09 13:27:25,681][86121] Updated weights for policy 0, policy_version 31240 (0.0009) +[2023-10-09 13:27:26,048][86121] Updated weights for policy 0, policy_version 31250 (0.0010) +[2023-10-09 13:27:26,414][86121] Updated weights for policy 0, policy_version 31260 (0.0007) +[2023-10-09 13:27:27,739][86122] Updated weights for policy 1, policy_version 31400 (0.0009) +[2023-10-09 13:27:28,100][86122] Updated weights for policy 1, policy_version 31410 (0.0008) +[2023-10-09 13:27:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 64159744. Throughput: 0: 1833.4, 1: 1823.9. Samples: 16047622. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) +[2023-10-09 13:27:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:27:28,475][86122] Updated weights for policy 1, policy_version 31420 (0.0010) +[2023-10-09 13:27:29,959][86121] Updated weights for policy 0, policy_version 31270 (0.0008) +[2023-10-09 13:27:30,321][86121] Updated weights for policy 0, policy_version 31280 (0.0008) +[2023-10-09 13:27:30,689][86121] Updated weights for policy 0, policy_version 31290 (0.0009) +[2023-10-09 13:27:32,210][86122] Updated weights for policy 1, policy_version 31430 (0.0008) +[2023-10-09 13:27:32,574][86122] Updated weights for policy 1, policy_version 31440 (0.0010) +[2023-10-09 13:27:32,934][86122] Updated weights for policy 1, policy_version 31450 (0.0010) +[2023-10-09 13:27:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 64258048. Throughput: 0: 1834.9, 1: 1818.0. Samples: 16069586. Policy #0 lag: (min: 31.0, avg: 31.6, max: 48.0) +[2023-10-09 13:27:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:27:34,281][86121] Updated weights for policy 0, policy_version 31300 (0.0008) +[2023-10-09 13:27:34,636][86121] Updated weights for policy 0, policy_version 31310 (0.0008) +[2023-10-09 13:27:35,004][86121] Updated weights for policy 0, policy_version 31320 (0.0009) +[2023-10-09 13:27:36,745][86122] Updated weights for policy 1, policy_version 31460 (0.0009) +[2023-10-09 13:27:37,109][86122] Updated weights for policy 1, policy_version 31470 (0.0010) +[2023-10-09 13:27:37,470][86122] Updated weights for policy 1, policy_version 31480 (0.0009) +[2023-10-09 13:27:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 64323584. Throughput: 0: 1824.2, 1: 1816.5. Samples: 16090680. Policy #0 lag: (min: 3.0, avg: 29.7, max: 32.0) +[2023-10-09 13:27:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:27:38,763][86121] Updated weights for policy 0, policy_version 31330 (0.0009) +[2023-10-09 13:27:39,120][86121] Updated weights for policy 0, policy_version 31340 (0.0009) +[2023-10-09 13:27:39,489][86121] Updated weights for policy 0, policy_version 31350 (0.0009) +[2023-10-09 13:27:39,859][86121] Updated weights for policy 0, policy_version 31360 (0.0010) +[2023-10-09 13:27:41,273][86122] Updated weights for policy 1, policy_version 31490 (0.0008) +[2023-10-09 13:27:41,627][86122] Updated weights for policy 1, policy_version 31500 (0.0010) +[2023-10-09 13:27:41,989][86122] Updated weights for policy 1, policy_version 31510 (0.0010) +[2023-10-09 13:27:42,358][86122] Updated weights for policy 1, policy_version 31520 (0.0010) +[2023-10-09 13:27:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64389120. Throughput: 0: 1818.7, 1: 1823.1. Samples: 16101962. Policy #0 lag: (min: 3.0, avg: 29.7, max: 32.0) +[2023-10-09 13:27:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:27:43,716][86121] Updated weights for policy 0, policy_version 31370 (0.0008) +[2023-10-09 13:27:44,084][86121] Updated weights for policy 0, policy_version 31380 (0.0011) +[2023-10-09 13:27:44,452][86121] Updated weights for policy 0, policy_version 31390 (0.0008) +[2023-10-09 13:27:45,952][86122] Updated weights for policy 1, policy_version 31530 (0.0009) +[2023-10-09 13:27:46,315][86122] Updated weights for policy 1, policy_version 31540 (0.0007) +[2023-10-09 13:27:46,684][86122] Updated weights for policy 1, policy_version 31550 (0.0008) +[2023-10-09 13:27:48,187][86121] Updated weights for policy 0, policy_version 31400 (0.0008) +[2023-10-09 13:27:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64454656. Throughput: 0: 1820.6, 1: 1818.3. Samples: 16123514. Policy #0 lag: (min: 3.0, avg: 29.7, max: 32.0) +[2023-10-09 13:27:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:27:48,552][86121] Updated weights for policy 0, policy_version 31410 (0.0009) +[2023-10-09 13:27:48,927][86121] Updated weights for policy 0, policy_version 31420 (0.0008) +[2023-10-09 13:27:50,471][86122] Updated weights for policy 1, policy_version 31560 (0.0008) +[2023-10-09 13:27:50,840][86122] Updated weights for policy 1, policy_version 31570 (0.0010) +[2023-10-09 13:27:51,210][86122] Updated weights for policy 1, policy_version 31580 (0.0010) +[2023-10-09 13:27:52,494][86121] Updated weights for policy 0, policy_version 31430 (0.0009) +[2023-10-09 13:27:52,869][86121] Updated weights for policy 0, policy_version 31440 (0.0008) +[2023-10-09 13:27:53,232][86121] Updated weights for policy 0, policy_version 31450 (0.0007) +[2023-10-09 13:27:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 64520192. Throughput: 0: 1823.3, 1: 1821.1. Samples: 16145304. Policy #0 lag: (min: 3.0, avg: 29.7, max: 32.0) +[2023-10-09 13:27:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 13:27:54,788][86122] Updated weights for policy 1, policy_version 31590 (0.0008) +[2023-10-09 13:27:55,151][86122] Updated weights for policy 1, policy_version 31600 (0.0008) +[2023-10-09 13:27:55,508][86122] Updated weights for policy 1, policy_version 31610 (0.0009) +[2023-10-09 13:27:56,999][86121] Updated weights for policy 0, policy_version 31460 (0.0007) +[2023-10-09 13:27:57,363][86121] Updated weights for policy 0, policy_version 31470 (0.0009) +[2023-10-09 13:27:57,730][86121] Updated weights for policy 0, policy_version 31480 (0.0007) +[2023-10-09 13:27:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64618496. Throughput: 0: 1813.8, 1: 1816.5. Samples: 16155986. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 13:27:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:27:59,338][86122] Updated weights for policy 1, policy_version 31620 (0.0009) +[2023-10-09 13:27:59,698][86122] Updated weights for policy 1, policy_version 31630 (0.0011) +[2023-10-09 13:28:00,061][86122] Updated weights for policy 1, policy_version 31640 (0.0010) +[2023-10-09 13:28:01,343][86121] Updated weights for policy 0, policy_version 31490 (0.0009) +[2023-10-09 13:28:01,719][86121] Updated weights for policy 0, policy_version 31500 (0.0009) +[2023-10-09 13:28:02,078][86121] Updated weights for policy 0, policy_version 31510 (0.0008) +[2023-10-09 13:28:02,442][86121] Updated weights for policy 0, policy_version 31520 (0.0009) +[2023-10-09 13:28:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 64684032. Throughput: 0: 1818.0, 1: 1810.9. Samples: 16178044. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 13:28:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:28:03,647][86122] Updated weights for policy 1, policy_version 31650 (0.0009) +[2023-10-09 13:28:04,008][86122] Updated weights for policy 1, policy_version 31660 (0.0012) +[2023-10-09 13:28:04,376][86122] Updated weights for policy 1, policy_version 31670 (0.0008) +[2023-10-09 13:28:04,745][86122] Updated weights for policy 1, policy_version 31680 (0.0009) +[2023-10-09 13:28:06,140][86121] Updated weights for policy 0, policy_version 31530 (0.0008) +[2023-10-09 13:28:06,498][86121] Updated weights for policy 0, policy_version 31540 (0.0007) +[2023-10-09 13:28:06,867][86121] Updated weights for policy 0, policy_version 31550 (0.0008) +[2023-10-09 13:28:08,334][86122] Updated weights for policy 1, policy_version 31690 (0.0008) +[2023-10-09 13:28:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 64749568. Throughput: 0: 1815.4, 1: 1809.9. Samples: 16200096. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 13:28:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:28:08,710][86122] Updated weights for policy 1, policy_version 31700 (0.0007) +[2023-10-09 13:28:09,064][86122] Updated weights for policy 1, policy_version 31710 (0.0009) +[2023-10-09 13:28:10,514][86121] Updated weights for policy 0, policy_version 31560 (0.0007) +[2023-10-09 13:28:10,891][86121] Updated weights for policy 0, policy_version 31570 (0.0008) +[2023-10-09 13:28:11,255][86121] Updated weights for policy 0, policy_version 31580 (0.0008) +[2023-10-09 13:28:12,709][86122] Updated weights for policy 1, policy_version 31720 (0.0009) +[2023-10-09 13:28:13,075][86122] Updated weights for policy 1, policy_version 31730 (0.0009) +[2023-10-09 13:28:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 64815104. Throughput: 0: 1808.7, 1: 1810.5. Samples: 16210486. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 13:28:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:28:13,436][86122] Updated weights for policy 1, policy_version 31740 (0.0008) +[2023-10-09 13:28:15,043][86121] Updated weights for policy 0, policy_version 31590 (0.0007) +[2023-10-09 13:28:15,400][86121] Updated weights for policy 0, policy_version 31600 (0.0008) +[2023-10-09 13:28:15,771][86121] Updated weights for policy 0, policy_version 31610 (0.0008) +[2023-10-09 13:28:17,059][86122] Updated weights for policy 1, policy_version 31750 (0.0007) +[2023-10-09 13:28:17,420][86122] Updated weights for policy 1, policy_version 31760 (0.0007) +[2023-10-09 13:28:17,780][86122] Updated weights for policy 1, policy_version 31770 (0.0007) +[2023-10-09 13:28:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 64913408. Throughput: 0: 1806.8, 1: 1821.4. Samples: 16232854. Policy #0 lag: (min: 31.0, avg: 38.4, max: 63.0) +[2023-10-09 13:28:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 13:28:19,499][86121] Updated weights for policy 0, policy_version 31620 (0.0009) +[2023-10-09 13:28:19,871][86121] Updated weights for policy 0, policy_version 31630 (0.0009) +[2023-10-09 13:28:20,240][86121] Updated weights for policy 0, policy_version 31640 (0.0009) +[2023-10-09 13:28:21,488][86122] Updated weights for policy 1, policy_version 31780 (0.0008) +[2023-10-09 13:28:21,838][86122] Updated weights for policy 1, policy_version 31790 (0.0010) +[2023-10-09 13:28:22,199][86122] Updated weights for policy 1, policy_version 31800 (0.0007) +[2023-10-09 13:28:23,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 64978944. Throughput: 0: 1810.9, 1: 1824.5. Samples: 16254272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 13:28:24,118][86121] Updated weights for policy 0, policy_version 31650 (0.0010) +[2023-10-09 13:28:24,480][86121] Updated weights for policy 0, policy_version 31660 (0.0011) +[2023-10-09 13:28:24,854][86121] Updated weights for policy 0, policy_version 31670 (0.0008) +[2023-10-09 13:28:25,225][86121] Updated weights for policy 0, policy_version 31680 (0.0011) +[2023-10-09 13:28:25,929][86122] Updated weights for policy 1, policy_version 31810 (0.0007) +[2023-10-09 13:28:26,289][86122] Updated weights for policy 1, policy_version 31820 (0.0007) +[2023-10-09 13:28:26,648][86122] Updated weights for policy 1, policy_version 31830 (0.0008) +[2023-10-09 13:28:27,004][86122] Updated weights for policy 1, policy_version 31840 (0.0008) +[2023-10-09 13:28:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65044480. Throughput: 0: 1812.6, 1: 1826.7. Samples: 16265728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 13:28:28,828][86121] Updated weights for policy 0, policy_version 31690 (0.0008) +[2023-10-09 13:28:29,201][86121] Updated weights for policy 0, policy_version 31700 (0.0010) +[2023-10-09 13:28:29,553][86121] Updated weights for policy 0, policy_version 31710 (0.0008) +[2023-10-09 13:28:30,641][86122] Updated weights for policy 1, policy_version 31850 (0.0007) +[2023-10-09 13:28:31,019][86122] Updated weights for policy 1, policy_version 31860 (0.0008) +[2023-10-09 13:28:31,389][86122] Updated weights for policy 1, policy_version 31870 (0.0007) +[2023-10-09 13:28:33,176][86121] Updated weights for policy 0, policy_version 31720 (0.0011) +[2023-10-09 13:28:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65110016. Throughput: 0: 1809.4, 1: 1829.7. Samples: 16287272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 13:28:33,534][86121] Updated weights for policy 0, policy_version 31730 (0.0010) +[2023-10-09 13:28:33,898][86121] Updated weights for policy 0, policy_version 31740 (0.0009) +[2023-10-09 13:28:35,246][86122] Updated weights for policy 1, policy_version 31880 (0.0009) +[2023-10-09 13:28:35,616][86122] Updated weights for policy 1, policy_version 31890 (0.0010) +[2023-10-09 13:28:35,993][86122] Updated weights for policy 1, policy_version 31900 (0.0009) +[2023-10-09 13:28:37,689][86121] Updated weights for policy 0, policy_version 31750 (0.0007) +[2023-10-09 13:28:38,051][86121] Updated weights for policy 0, policy_version 31760 (0.0008) +[2023-10-09 13:28:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65175552. Throughput: 0: 1815.5, 1: 1829.5. Samples: 16309326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 13:28:38,423][86121] Updated weights for policy 0, policy_version 31770 (0.0009) +[2023-10-09 13:28:39,639][86122] Updated weights for policy 1, policy_version 31910 (0.0008) +[2023-10-09 13:28:40,009][86122] Updated weights for policy 1, policy_version 31920 (0.0007) +[2023-10-09 13:28:40,366][86122] Updated weights for policy 1, policy_version 31930 (0.0007) +[2023-10-09 13:28:42,000][86121] Updated weights for policy 0, policy_version 31780 (0.0009) +[2023-10-09 13:28:42,367][86121] Updated weights for policy 0, policy_version 31790 (0.0008) +[2023-10-09 13:28:42,736][86121] Updated weights for policy 0, policy_version 31800 (0.0008) +[2023-10-09 13:28:43,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65273856. Throughput: 0: 1815.0, 1: 1831.3. Samples: 16320072. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 13:28:44,009][86122] Updated weights for policy 1, policy_version 31940 (0.0008) +[2023-10-09 13:28:44,375][86122] Updated weights for policy 1, policy_version 31950 (0.0008) +[2023-10-09 13:28:44,729][86122] Updated weights for policy 1, policy_version 31960 (0.0008) +[2023-10-09 13:28:46,260][86121] Updated weights for policy 0, policy_version 31810 (0.0009) +[2023-10-09 13:28:46,628][86121] Updated weights for policy 0, policy_version 31820 (0.0007) +[2023-10-09 13:28:46,997][86121] Updated weights for policy 0, policy_version 31830 (0.0007) +[2023-10-09 13:28:47,365][86121] Updated weights for policy 0, policy_version 31840 (0.0008) +[2023-10-09 13:28:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65339392. Throughput: 0: 1812.3, 1: 1834.0. Samples: 16342128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 13:28:48,503][86122] Updated weights for policy 1, policy_version 31970 (0.0007) +[2023-10-09 13:28:48,866][86122] Updated weights for policy 1, policy_version 31980 (0.0009) +[2023-10-09 13:28:49,236][86122] Updated weights for policy 1, policy_version 31990 (0.0008) +[2023-10-09 13:28:49,603][86122] Updated weights for policy 1, policy_version 32000 (0.0010) +[2023-10-09 13:28:51,103][86121] Updated weights for policy 0, policy_version 31850 (0.0009) +[2023-10-09 13:28:51,478][86121] Updated weights for policy 0, policy_version 31860 (0.0011) +[2023-10-09 13:28:51,843][86121] Updated weights for policy 0, policy_version 31870 (0.0009) +[2023-10-09 13:28:53,181][86122] Updated weights for policy 1, policy_version 32010 (0.0009) +[2023-10-09 13:28:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65404928. Throughput: 0: 1818.1, 1: 1835.2. Samples: 16364494. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 13:28:53,547][86122] Updated weights for policy 1, policy_version 32020 (0.0007) +[2023-10-09 13:28:53,915][86122] Updated weights for policy 1, policy_version 32030 (0.0010) +[2023-10-09 13:28:55,637][86121] Updated weights for policy 0, policy_version 31880 (0.0009) +[2023-10-09 13:28:56,006][86121] Updated weights for policy 0, policy_version 31890 (0.0007) +[2023-10-09 13:28:56,379][86121] Updated weights for policy 0, policy_version 31900 (0.0008) +[2023-10-09 13:28:57,569][86122] Updated weights for policy 1, policy_version 32040 (0.0010) +[2023-10-09 13:28:57,937][86122] Updated weights for policy 1, policy_version 32050 (0.0009) +[2023-10-09 13:28:58,305][86122] Updated weights for policy 1, policy_version 32060 (0.0008) +[2023-10-09 13:28:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65470464. Throughput: 0: 1821.9, 1: 1834.8. Samples: 16375036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:28:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 13:28:59,814][86121] Updated weights for policy 0, policy_version 31910 (0.0008) +[2023-10-09 13:29:00,181][86121] Updated weights for policy 0, policy_version 31920 (0.0008) +[2023-10-09 13:29:00,544][86121] Updated weights for policy 0, policy_version 31930 (0.0009) +[2023-10-09 13:29:01,988][86122] Updated weights for policy 1, policy_version 32070 (0.0010) +[2023-10-09 13:29:02,346][86122] Updated weights for policy 1, policy_version 32080 (0.0010) +[2023-10-09 13:29:02,707][86122] Updated weights for policy 1, policy_version 32090 (0.0010) +[2023-10-09 13:29:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65568768. Throughput: 0: 1827.6, 1: 1825.9. Samples: 16397260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:29:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 13:29:04,308][86121] Updated weights for policy 0, policy_version 31940 (0.0008) +[2023-10-09 13:29:04,667][86121] Updated weights for policy 0, policy_version 31950 (0.0007) +[2023-10-09 13:29:05,036][86121] Updated weights for policy 0, policy_version 31960 (0.0008) +[2023-10-09 13:29:06,539][86122] Updated weights for policy 1, policy_version 32100 (0.0010) +[2023-10-09 13:29:06,906][86122] Updated weights for policy 1, policy_version 32110 (0.0010) +[2023-10-09 13:29:07,267][86122] Updated weights for policy 1, policy_version 32120 (0.0010) +[2023-10-09 13:29:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65634304. Throughput: 0: 1828.7, 1: 1822.6. Samples: 16418578. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:29:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 13:29:08,812][86121] Updated weights for policy 0, policy_version 31970 (0.0010) +[2023-10-09 13:29:09,183][86121] Updated weights for policy 0, policy_version 31980 (0.0008) +[2023-10-09 13:29:09,544][86121] Updated weights for policy 0, policy_version 31990 (0.0008) +[2023-10-09 13:29:09,914][86121] Updated weights for policy 0, policy_version 32000 (0.0009) +[2023-10-09 13:29:10,833][86122] Updated weights for policy 1, policy_version 32130 (0.0009) +[2023-10-09 13:29:11,183][86122] Updated weights for policy 1, policy_version 32140 (0.0010) +[2023-10-09 13:29:11,547][86122] Updated weights for policy 1, policy_version 32150 (0.0009) +[2023-10-09 13:29:11,902][86122] Updated weights for policy 1, policy_version 32160 (0.0010) +[2023-10-09 13:29:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 65699840. Throughput: 0: 1829.0, 1: 1820.3. Samples: 16429944. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:29:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 13:29:13,730][86121] Updated weights for policy 0, policy_version 32010 (0.0009) +[2023-10-09 13:29:14,092][86121] Updated weights for policy 0, policy_version 32020 (0.0010) +[2023-10-09 13:29:14,458][86121] Updated weights for policy 0, policy_version 32030 (0.0008) +[2023-10-09 13:29:15,707][86122] Updated weights for policy 1, policy_version 32170 (0.0010) +[2023-10-09 13:29:16,073][86122] Updated weights for policy 1, policy_version 32180 (0.0007) +[2023-10-09 13:29:16,432][86122] Updated weights for policy 1, policy_version 32190 (0.0007) +[2023-10-09 13:29:17,939][86121] Updated weights for policy 0, policy_version 32040 (0.0009) +[2023-10-09 13:29:18,315][86121] Updated weights for policy 0, policy_version 32050 (0.0009) +[2023-10-09 13:29:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65765376. Throughput: 0: 1832.9, 1: 1815.5. Samples: 16451452. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:29:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 13:29:18,683][86121] Updated weights for policy 0, policy_version 32060 (0.0009) +[2023-10-09 13:29:20,129][86122] Updated weights for policy 1, policy_version 32200 (0.0008) +[2023-10-09 13:29:20,496][86122] Updated weights for policy 1, policy_version 32210 (0.0010) +[2023-10-09 13:29:20,858][86122] Updated weights for policy 1, policy_version 32220 (0.0011) +[2023-10-09 13:29:22,543][86121] Updated weights for policy 0, policy_version 32070 (0.0007) +[2023-10-09 13:29:22,909][86121] Updated weights for policy 0, policy_version 32080 (0.0008) +[2023-10-09 13:29:23,274][86121] Updated weights for policy 0, policy_version 32090 (0.0010) +[2023-10-09 13:29:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 65830912. Throughput: 0: 1824.8, 1: 1820.9. Samples: 16473384. Policy #0 lag: (min: 6.0, avg: 6.0, max: 6.0) +[2023-10-09 13:29:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 13:29:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000032224_32997376.pth... +[2023-10-09 13:29:23,442][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000030528_31260672.pth +[2023-10-09 13:29:23,492][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000032096_32866304.pth... +[2023-10-09 13:29:23,527][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000030368_31096832.pth +[2023-10-09 13:29:24,502][86122] Updated weights for policy 1, policy_version 32230 (0.0009) +[2023-10-09 13:29:24,863][86122] Updated weights for policy 1, policy_version 32240 (0.0007) +[2023-10-09 13:29:25,218][86122] Updated weights for policy 1, policy_version 32250 (0.0008) +[2023-10-09 13:29:26,946][86121] Updated weights for policy 0, policy_version 32100 (0.0010) +[2023-10-09 13:29:27,319][86121] Updated weights for policy 0, policy_version 32110 (0.0009) +[2023-10-09 13:29:27,686][86121] Updated weights for policy 0, policy_version 32120 (0.0007) +[2023-10-09 13:29:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 65929216. Throughput: 0: 1827.8, 1: 1817.4. Samples: 16484108. Policy #0 lag: (min: 14.0, avg: 23.8, max: 46.0) +[2023-10-09 13:29:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 13:29:29,061][86122] Updated weights for policy 1, policy_version 32260 (0.0009) +[2023-10-09 13:29:29,432][86122] Updated weights for policy 1, policy_version 32270 (0.0008) +[2023-10-09 13:29:29,790][86122] Updated weights for policy 1, policy_version 32280 (0.0008) +[2023-10-09 13:29:31,470][86121] Updated weights for policy 0, policy_version 32130 (0.0009) +[2023-10-09 13:29:31,834][86121] Updated weights for policy 0, policy_version 32140 (0.0007) +[2023-10-09 13:29:32,199][86121] Updated weights for policy 0, policy_version 32150 (0.0008) +[2023-10-09 13:29:32,565][86121] Updated weights for policy 0, policy_version 32160 (0.0009) +[2023-10-09 13:29:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 65994752. Throughput: 0: 1829.3, 1: 1814.6. Samples: 16506104. Policy #0 lag: (min: 14.0, avg: 23.8, max: 46.0) +[2023-10-09 13:29:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 13:29:33,451][86122] Updated weights for policy 1, policy_version 32290 (0.0007) +[2023-10-09 13:29:33,805][86122] Updated weights for policy 1, policy_version 32300 (0.0009) +[2023-10-09 13:29:34,175][86122] Updated weights for policy 1, policy_version 32310 (0.0009) +[2023-10-09 13:29:34,539][86122] Updated weights for policy 1, policy_version 32320 (0.0009) +[2023-10-09 13:29:36,265][86121] Updated weights for policy 0, policy_version 32170 (0.0010) +[2023-10-09 13:29:36,623][86121] Updated weights for policy 0, policy_version 32180 (0.0009) +[2023-10-09 13:29:36,987][86121] Updated weights for policy 0, policy_version 32190 (0.0009) +[2023-10-09 13:29:38,154][86122] Updated weights for policy 1, policy_version 32330 (0.0008) +[2023-10-09 13:29:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66060288. Throughput: 0: 1822.2, 1: 1815.7. Samples: 16528198. Policy #0 lag: (min: 14.0, avg: 23.8, max: 46.0) +[2023-10-09 13:29:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 13:29:38,524][86122] Updated weights for policy 1, policy_version 32340 (0.0008) +[2023-10-09 13:29:38,893][86122] Updated weights for policy 1, policy_version 32350 (0.0009) +[2023-10-09 13:29:40,779][86121] Updated weights for policy 0, policy_version 32200 (0.0008) +[2023-10-09 13:29:41,150][86121] Updated weights for policy 0, policy_version 32210 (0.0010) +[2023-10-09 13:29:41,526][86121] Updated weights for policy 0, policy_version 32220 (0.0008) +[2023-10-09 13:29:42,552][86122] Updated weights for policy 1, policy_version 32360 (0.0010) +[2023-10-09 13:29:42,912][86122] Updated weights for policy 1, policy_version 32370 (0.0012) +[2023-10-09 13:29:43,275][86122] Updated weights for policy 1, policy_version 32380 (0.0010) +[2023-10-09 13:29:43,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 66125824. Throughput: 0: 1822.7, 1: 1819.7. Samples: 16538942. Policy #0 lag: (min: 14.0, avg: 23.8, max: 46.0) +[2023-10-09 13:29:43,399][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:29:45,064][86121] Updated weights for policy 0, policy_version 32230 (0.0008) +[2023-10-09 13:29:45,427][86121] Updated weights for policy 0, policy_version 32240 (0.0009) +[2023-10-09 13:29:45,792][86121] Updated weights for policy 0, policy_version 32250 (0.0011) +[2023-10-09 13:29:47,099][86122] Updated weights for policy 1, policy_version 32390 (0.0011) +[2023-10-09 13:29:47,455][86122] Updated weights for policy 1, policy_version 32400 (0.0008) +[2023-10-09 13:29:47,812][86122] Updated weights for policy 1, policy_version 32410 (0.0008) +[2023-10-09 13:29:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66224128. Throughput: 0: 1817.7, 1: 1816.4. Samples: 16560790. Policy #0 lag: (min: 14.0, avg: 23.8, max: 46.0) +[2023-10-09 13:29:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:29:49,440][86121] Updated weights for policy 0, policy_version 32260 (0.0010) +[2023-10-09 13:29:49,806][86121] Updated weights for policy 0, policy_version 32270 (0.0007) +[2023-10-09 13:29:50,180][86121] Updated weights for policy 0, policy_version 32280 (0.0007) +[2023-10-09 13:29:51,514][86122] Updated weights for policy 1, policy_version 32420 (0.0008) +[2023-10-09 13:29:51,878][86122] Updated weights for policy 1, policy_version 32430 (0.0008) +[2023-10-09 13:29:52,236][86122] Updated weights for policy 1, policy_version 32440 (0.0008) +[2023-10-09 13:29:53,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66289664. Throughput: 0: 1818.3, 1: 1822.8. Samples: 16582430. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:29:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:29:53,869][86121] Updated weights for policy 0, policy_version 32290 (0.0008) +[2023-10-09 13:29:54,230][86121] Updated weights for policy 0, policy_version 32300 (0.0008) +[2023-10-09 13:29:54,600][86121] Updated weights for policy 0, policy_version 32310 (0.0007) +[2023-10-09 13:29:54,968][86121] Updated weights for policy 0, policy_version 32320 (0.0007) +[2023-10-09 13:29:55,995][86122] Updated weights for policy 1, policy_version 32450 (0.0008) +[2023-10-09 13:29:56,352][86122] Updated weights for policy 1, policy_version 32460 (0.0010) +[2023-10-09 13:29:56,723][86122] Updated weights for policy 1, policy_version 32470 (0.0008) +[2023-10-09 13:29:57,082][86122] Updated weights for policy 1, policy_version 32480 (0.0007) +[2023-10-09 13:29:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66355200. Throughput: 0: 1822.4, 1: 1821.2. Samples: 16593904. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:29:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 13:29:58,707][86121] Updated weights for policy 0, policy_version 32330 (0.0008) +[2023-10-09 13:29:59,076][86121] Updated weights for policy 0, policy_version 32340 (0.0007) +[2023-10-09 13:29:59,443][86121] Updated weights for policy 0, policy_version 32350 (0.0010) +[2023-10-09 13:30:00,844][86122] Updated weights for policy 1, policy_version 32490 (0.0009) +[2023-10-09 13:30:01,203][86122] Updated weights for policy 1, policy_version 32500 (0.0007) +[2023-10-09 13:30:01,567][86122] Updated weights for policy 1, policy_version 32510 (0.0007) +[2023-10-09 13:30:03,153][86121] Updated weights for policy 0, policy_version 32360 (0.0009) +[2023-10-09 13:30:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66420736. Throughput: 0: 1815.5, 1: 1820.5. Samples: 16615074. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:30:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 13:30:03,524][86121] Updated weights for policy 0, policy_version 32370 (0.0011) +[2023-10-09 13:30:03,896][86121] Updated weights for policy 0, policy_version 32380 (0.0009) +[2023-10-09 13:30:05,319][86122] Updated weights for policy 1, policy_version 32520 (0.0008) +[2023-10-09 13:30:05,681][86122] Updated weights for policy 1, policy_version 32530 (0.0009) +[2023-10-09 13:30:06,045][86122] Updated weights for policy 1, policy_version 32540 (0.0010) +[2023-10-09 13:30:07,627][86121] Updated weights for policy 0, policy_version 32390 (0.0009) +[2023-10-09 13:30:07,992][86121] Updated weights for policy 0, policy_version 32400 (0.0009) +[2023-10-09 13:30:08,370][86121] Updated weights for policy 0, policy_version 32410 (0.0008) +[2023-10-09 13:30:08,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 66486272. Throughput: 0: 1819.8, 1: 1819.0. Samples: 16637130. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 13:30:08,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 13:30:09,712][86122] Updated weights for policy 1, policy_version 32550 (0.0009) +[2023-10-09 13:30:10,080][86122] Updated weights for policy 1, policy_version 32560 (0.0011) +[2023-10-09 13:30:10,446][86122] Updated weights for policy 1, policy_version 32570 (0.0009) +[2023-10-09 13:30:12,069][86121] Updated weights for policy 0, policy_version 32420 (0.0009) +[2023-10-09 13:30:12,434][86121] Updated weights for policy 0, policy_version 32430 (0.0009) +[2023-10-09 13:30:12,809][86121] Updated weights for policy 0, policy_version 32440 (0.0008) +[2023-10-09 13:30:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 66584576. Throughput: 0: 1815.2, 1: 1818.1. Samples: 16647610. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:30:13,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 13:30:14,099][86122] Updated weights for policy 1, policy_version 32580 (0.0009) +[2023-10-09 13:30:14,472][86122] Updated weights for policy 1, policy_version 32590 (0.0007) +[2023-10-09 13:30:14,835][86122] Updated weights for policy 1, policy_version 32600 (0.0009) +[2023-10-09 13:30:16,514][86121] Updated weights for policy 0, policy_version 32450 (0.0008) +[2023-10-09 13:30:16,875][86121] Updated weights for policy 0, policy_version 32460 (0.0008) +[2023-10-09 13:30:17,243][86121] Updated weights for policy 0, policy_version 32470 (0.0008) +[2023-10-09 13:30:17,601][86121] Updated weights for policy 0, policy_version 32480 (0.0009) +[2023-10-09 13:30:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66650112. Throughput: 0: 1816.4, 1: 1825.6. Samples: 16669992. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:30:18,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 13:30:18,520][86122] Updated weights for policy 1, policy_version 32610 (0.0007) +[2023-10-09 13:30:18,879][86122] Updated weights for policy 1, policy_version 32620 (0.0008) +[2023-10-09 13:30:19,234][86122] Updated weights for policy 1, policy_version 32630 (0.0008) +[2023-10-09 13:30:19,591][86122] Updated weights for policy 1, policy_version 32640 (0.0009) +[2023-10-09 13:30:21,156][86121] Updated weights for policy 0, policy_version 32490 (0.0008) +[2023-10-09 13:30:21,519][86121] Updated weights for policy 0, policy_version 32500 (0.0009) +[2023-10-09 13:30:21,878][86121] Updated weights for policy 0, policy_version 32510 (0.0008) +[2023-10-09 13:30:23,336][86122] Updated weights for policy 1, policy_version 32650 (0.0008) +[2023-10-09 13:30:23,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66715648. Throughput: 0: 1819.2, 1: 1828.0. Samples: 16692324. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:30:23,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 13:30:23,694][86122] Updated weights for policy 1, policy_version 32660 (0.0007) +[2023-10-09 13:30:24,072][86122] Updated weights for policy 1, policy_version 32670 (0.0011) +[2023-10-09 13:30:25,720][86121] Updated weights for policy 0, policy_version 32520 (0.0011) +[2023-10-09 13:30:26,095][86121] Updated weights for policy 0, policy_version 32530 (0.0010) +[2023-10-09 13:30:26,469][86121] Updated weights for policy 0, policy_version 32540 (0.0009) +[2023-10-09 13:30:27,565][86122] Updated weights for policy 1, policy_version 32680 (0.0009) +[2023-10-09 13:30:27,930][86122] Updated weights for policy 1, policy_version 32690 (0.0008) +[2023-10-09 13:30:28,284][86122] Updated weights for policy 1, policy_version 32700 (0.0009) +[2023-10-09 13:30:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 66781184. Throughput: 0: 1824.4, 1: 1825.3. Samples: 16703178. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:30:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.970')] +[2023-10-09 13:30:30,162][86121] Updated weights for policy 0, policy_version 32550 (0.0007) +[2023-10-09 13:30:30,531][86121] Updated weights for policy 0, policy_version 32560 (0.0007) +[2023-10-09 13:30:30,895][86121] Updated weights for policy 0, policy_version 32570 (0.0009) +[2023-10-09 13:30:32,187][86122] Updated weights for policy 1, policy_version 32710 (0.0008) +[2023-10-09 13:30:32,543][86122] Updated weights for policy 1, policy_version 32720 (0.0009) +[2023-10-09 13:30:32,904][86122] Updated weights for policy 1, policy_version 32730 (0.0011) +[2023-10-09 13:30:33,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 66879488. Throughput: 0: 1819.6, 1: 1825.9. Samples: 16724836. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:30:33,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.970')] +[2023-10-09 13:30:34,638][86121] Updated weights for policy 0, policy_version 32580 (0.0007) +[2023-10-09 13:30:35,000][86121] Updated weights for policy 0, policy_version 32590 (0.0007) +[2023-10-09 13:30:35,361][86121] Updated weights for policy 0, policy_version 32600 (0.0008) +[2023-10-09 13:30:36,475][86122] Updated weights for policy 1, policy_version 32740 (0.0008) +[2023-10-09 13:30:36,849][86122] Updated weights for policy 1, policy_version 32750 (0.0009) +[2023-10-09 13:30:37,214][86122] Updated weights for policy 1, policy_version 32760 (0.0008) +[2023-10-09 13:30:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 66945024. Throughput: 0: 1815.1, 1: 1827.4. Samples: 16746344. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:30:38,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.960')] +[2023-10-09 13:30:38,971][86121] Updated weights for policy 0, policy_version 32610 (0.0008) +[2023-10-09 13:30:39,338][86121] Updated weights for policy 0, policy_version 32620 (0.0010) +[2023-10-09 13:30:39,690][86121] Updated weights for policy 0, policy_version 32630 (0.0007) +[2023-10-09 13:30:40,054][86121] Updated weights for policy 0, policy_version 32640 (0.0007) +[2023-10-09 13:30:40,842][86122] Updated weights for policy 1, policy_version 32770 (0.0010) +[2023-10-09 13:30:41,207][86122] Updated weights for policy 1, policy_version 32780 (0.0009) +[2023-10-09 13:30:41,571][86122] Updated weights for policy 1, policy_version 32790 (0.0011) +[2023-10-09 13:30:41,932][86122] Updated weights for policy 1, policy_version 32800 (0.0009) +[2023-10-09 13:30:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67010560. Throughput: 0: 1815.0, 1: 1826.3. Samples: 16757764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:30:43,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.960')] +[2023-10-09 13:30:43,676][86121] Updated weights for policy 0, policy_version 32650 (0.0009) +[2023-10-09 13:30:44,045][86121] Updated weights for policy 0, policy_version 32660 (0.0010) +[2023-10-09 13:30:44,414][86121] Updated weights for policy 0, policy_version 32670 (0.0009) +[2023-10-09 13:30:45,442][86122] Updated weights for policy 1, policy_version 32810 (0.0010) +[2023-10-09 13:30:45,796][86122] Updated weights for policy 1, policy_version 32820 (0.0008) +[2023-10-09 13:30:46,155][86122] Updated weights for policy 1, policy_version 32830 (0.0008) +[2023-10-09 13:30:48,120][86121] Updated weights for policy 0, policy_version 32680 (0.0009) +[2023-10-09 13:30:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67076096. Throughput: 0: 1815.7, 1: 1833.3. Samples: 16779278. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:30:48,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.960')] +[2023-10-09 13:30:48,495][86121] Updated weights for policy 0, policy_version 32690 (0.0007) +[2023-10-09 13:30:48,856][86121] Updated weights for policy 0, policy_version 32700 (0.0010) +[2023-10-09 13:30:49,762][86122] Updated weights for policy 1, policy_version 32840 (0.0008) +[2023-10-09 13:30:50,121][86122] Updated weights for policy 1, policy_version 32850 (0.0009) +[2023-10-09 13:30:50,483][86122] Updated weights for policy 1, policy_version 32860 (0.0010) +[2023-10-09 13:30:52,565][86121] Updated weights for policy 0, policy_version 32710 (0.0010) +[2023-10-09 13:30:52,936][86121] Updated weights for policy 0, policy_version 32720 (0.0007) +[2023-10-09 13:30:53,303][86121] Updated weights for policy 0, policy_version 32730 (0.0007) +[2023-10-09 13:30:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 67141632. Throughput: 0: 1818.9, 1: 1836.2. Samples: 16801610. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:30:53,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 13:30:54,305][86122] Updated weights for policy 1, policy_version 32870 (0.0008) +[2023-10-09 13:30:54,679][86122] Updated weights for policy 1, policy_version 32880 (0.0008) +[2023-10-09 13:30:55,033][86122] Updated weights for policy 1, policy_version 32890 (0.0008) +[2023-10-09 13:30:56,943][86121] Updated weights for policy 0, policy_version 32740 (0.0007) +[2023-10-09 13:30:57,304][86121] Updated weights for policy 0, policy_version 32750 (0.0008) +[2023-10-09 13:30:57,679][86121] Updated weights for policy 0, policy_version 32760 (0.0008) +[2023-10-09 13:30:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67239936. Throughput: 0: 1823.6, 1: 1833.9. Samples: 16812198. Policy #0 lag: (min: 12.0, avg: 13.7, max: 41.0) +[2023-10-09 13:30:58,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 13:30:58,868][86122] Updated weights for policy 1, policy_version 32900 (0.0008) +[2023-10-09 13:30:59,223][86122] Updated weights for policy 1, policy_version 32910 (0.0011) +[2023-10-09 13:30:59,595][86122] Updated weights for policy 1, policy_version 32920 (0.0008) +[2023-10-09 13:31:01,374][86121] Updated weights for policy 0, policy_version 32770 (0.0010) +[2023-10-09 13:31:01,740][86121] Updated weights for policy 0, policy_version 32780 (0.0008) +[2023-10-09 13:31:02,101][86121] Updated weights for policy 0, policy_version 32790 (0.0009) +[2023-10-09 13:31:02,467][86121] Updated weights for policy 0, policy_version 32800 (0.0011) +[2023-10-09 13:31:03,155][86122] Updated weights for policy 1, policy_version 32930 (0.0008) +[2023-10-09 13:31:03,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67305472. Throughput: 0: 1821.2, 1: 1831.8. Samples: 16834376. Policy #0 lag: (min: 12.0, avg: 13.7, max: 41.0) +[2023-10-09 13:31:03,398][85186] Avg episode reward: [(0, '9.850'), (1, '9.960')] +[2023-10-09 13:31:03,514][86122] Updated weights for policy 1, policy_version 32940 (0.0007) +[2023-10-09 13:31:03,884][86122] Updated weights for policy 1, policy_version 32950 (0.0008) +[2023-10-09 13:31:04,243][86122] Updated weights for policy 1, policy_version 32960 (0.0008) +[2023-10-09 13:31:06,176][86121] Updated weights for policy 0, policy_version 32810 (0.0008) +[2023-10-09 13:31:06,545][86121] Updated weights for policy 0, policy_version 32820 (0.0008) +[2023-10-09 13:31:06,905][86121] Updated weights for policy 0, policy_version 32830 (0.0008) +[2023-10-09 13:31:07,715][86122] Updated weights for policy 1, policy_version 32970 (0.0008) +[2023-10-09 13:31:08,071][86122] Updated weights for policy 1, policy_version 32980 (0.0009) +[2023-10-09 13:31:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67371008. Throughput: 0: 1820.3, 1: 1822.5. Samples: 16856254. Policy #0 lag: (min: 12.0, avg: 13.7, max: 41.0) +[2023-10-09 13:31:08,399][85186] Avg episode reward: [(0, '9.860'), (1, '9.960')] +[2023-10-09 13:31:08,437][86122] Updated weights for policy 1, policy_version 32990 (0.0008) +[2023-10-09 13:31:10,692][86121] Updated weights for policy 0, policy_version 32840 (0.0009) +[2023-10-09 13:31:11,068][86121] Updated weights for policy 0, policy_version 32850 (0.0009) +[2023-10-09 13:31:11,433][86121] Updated weights for policy 0, policy_version 32860 (0.0009) +[2023-10-09 13:31:12,162][86122] Updated weights for policy 1, policy_version 33000 (0.0009) +[2023-10-09 13:31:12,529][86122] Updated weights for policy 1, policy_version 33010 (0.0009) +[2023-10-09 13:31:12,908][86122] Updated weights for policy 1, policy_version 33020 (0.0010) +[2023-10-09 13:31:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67469312. Throughput: 0: 1814.4, 1: 1833.6. Samples: 16867340. Policy #0 lag: (min: 12.0, avg: 13.7, max: 41.0) +[2023-10-09 13:31:13,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.950')] +[2023-10-09 13:31:15,066][86121] Updated weights for policy 0, policy_version 32870 (0.0007) +[2023-10-09 13:31:15,433][86121] Updated weights for policy 0, policy_version 32880 (0.0009) +[2023-10-09 13:31:15,804][86121] Updated weights for policy 0, policy_version 32890 (0.0010) +[2023-10-09 13:31:16,792][86122] Updated weights for policy 1, policy_version 33030 (0.0011) +[2023-10-09 13:31:17,151][86122] Updated weights for policy 1, policy_version 33040 (0.0009) +[2023-10-09 13:31:17,502][86122] Updated weights for policy 1, policy_version 33050 (0.0009) +[2023-10-09 13:31:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 67534848. Throughput: 0: 1814.4, 1: 1830.3. Samples: 16888846. Policy #0 lag: (min: 12.0, avg: 13.7, max: 41.0) +[2023-10-09 13:31:18,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.950')] +[2023-10-09 13:31:19,557][86121] Updated weights for policy 0, policy_version 32900 (0.0009) +[2023-10-09 13:31:19,924][86121] Updated weights for policy 0, policy_version 32910 (0.0009) +[2023-10-09 13:31:20,291][86121] Updated weights for policy 0, policy_version 32920 (0.0009) +[2023-10-09 13:31:21,143][86122] Updated weights for policy 1, policy_version 33060 (0.0007) +[2023-10-09 13:31:21,508][86122] Updated weights for policy 1, policy_version 33070 (0.0007) +[2023-10-09 13:31:21,862][86122] Updated weights for policy 1, policy_version 33080 (0.0008) +[2023-10-09 13:31:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67600384. Throughput: 0: 1812.0, 1: 1830.9. Samples: 16910276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:31:23,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.950')] +[2023-10-09 13:31:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000032928_33718272.pth... +[2023-10-09 13:31:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000033088_33882112.pth... +[2023-10-09 13:31:23,448][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000031392_32145408.pth +[2023-10-09 13:31:23,450][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000031232_31981568.pth +[2023-10-09 13:31:24,012][86121] Updated weights for policy 0, policy_version 32930 (0.0009) +[2023-10-09 13:31:24,388][86121] Updated weights for policy 0, policy_version 32940 (0.0008) +[2023-10-09 13:31:24,752][86121] Updated weights for policy 0, policy_version 32950 (0.0009) +[2023-10-09 13:31:25,122][86121] Updated weights for policy 0, policy_version 32960 (0.0010) +[2023-10-09 13:31:25,348][86122] Updated weights for policy 1, policy_version 33090 (0.0008) +[2023-10-09 13:31:25,705][86122] Updated weights for policy 1, policy_version 33100 (0.0007) +[2023-10-09 13:31:26,066][86122] Updated weights for policy 1, policy_version 33110 (0.0007) +[2023-10-09 13:31:26,429][86122] Updated weights for policy 1, policy_version 33120 (0.0008) +[2023-10-09 13:31:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 67665920. Throughput: 0: 1808.3, 1: 1821.3. Samples: 16921100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:31:28,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.950')] +[2023-10-09 13:31:28,882][86121] Updated weights for policy 0, policy_version 32970 (0.0008) +[2023-10-09 13:31:29,244][86121] Updated weights for policy 0, policy_version 32980 (0.0008) +[2023-10-09 13:31:29,620][86121] Updated weights for policy 0, policy_version 32990 (0.0009) +[2023-10-09 13:31:30,152][86122] Updated weights for policy 1, policy_version 33130 (0.0010) +[2023-10-09 13:31:30,526][86122] Updated weights for policy 1, policy_version 33140 (0.0009) +[2023-10-09 13:31:30,886][86122] Updated weights for policy 1, policy_version 33150 (0.0010) +[2023-10-09 13:31:33,380][86121] Updated weights for policy 0, policy_version 33000 (0.0007) +[2023-10-09 13:31:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67731456. Throughput: 0: 1805.5, 1: 1834.5. Samples: 16943080. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:31:33,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.950')] +[2023-10-09 13:31:33,747][86121] Updated weights for policy 0, policy_version 33010 (0.0009) +[2023-10-09 13:31:34,105][86121] Updated weights for policy 0, policy_version 33020 (0.0008) +[2023-10-09 13:31:34,608][86122] Updated weights for policy 1, policy_version 33160 (0.0011) +[2023-10-09 13:31:34,979][86122] Updated weights for policy 1, policy_version 33170 (0.0009) +[2023-10-09 13:31:35,344][86122] Updated weights for policy 1, policy_version 33180 (0.0011) +[2023-10-09 13:31:37,954][86121] Updated weights for policy 0, policy_version 33030 (0.0008) +[2023-10-09 13:31:38,319][86121] Updated weights for policy 0, policy_version 33040 (0.0009) +[2023-10-09 13:31:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 67796992. Throughput: 0: 1809.4, 1: 1834.9. Samples: 16965604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:31:38,398][85186] Avg episode reward: [(0, '9.850'), (1, '9.950')] +[2023-10-09 13:31:38,689][86121] Updated weights for policy 0, policy_version 33050 (0.0008) +[2023-10-09 13:31:39,082][86122] Updated weights for policy 1, policy_version 33190 (0.0009) +[2023-10-09 13:31:39,446][86122] Updated weights for policy 1, policy_version 33200 (0.0008) +[2023-10-09 13:31:39,812][86122] Updated weights for policy 1, policy_version 33210 (0.0007) +[2023-10-09 13:31:42,618][86121] Updated weights for policy 0, policy_version 33060 (0.0008) +[2023-10-09 13:31:42,996][86121] Updated weights for policy 0, policy_version 33070 (0.0008) +[2023-10-09 13:31:43,367][86121] Updated weights for policy 0, policy_version 33080 (0.0007) +[2023-10-09 13:31:43,398][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 67862528. Throughput: 0: 1797.9, 1: 1838.7. Samples: 16975842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:31:43,399][85186] Avg episode reward: [(0, '9.850'), (1, '9.950')] +[2023-10-09 13:31:43,605][86122] Updated weights for policy 1, policy_version 33220 (0.0007) +[2023-10-09 13:31:43,989][86122] Updated weights for policy 1, policy_version 33230 (0.0008) +[2023-10-09 13:31:44,345][86122] Updated weights for policy 1, policy_version 33240 (0.0007) +[2023-10-09 13:31:47,180][86121] Updated weights for policy 0, policy_version 33090 (0.0007) +[2023-10-09 13:31:47,543][86121] Updated weights for policy 0, policy_version 33100 (0.0008) +[2023-10-09 13:31:47,914][86121] Updated weights for policy 0, policy_version 33110 (0.0008) +[2023-10-09 13:31:48,030][86122] Updated weights for policy 1, policy_version 33250 (0.0008) +[2023-10-09 13:31:48,275][86121] Updated weights for policy 0, policy_version 33120 (0.0009) +[2023-10-09 13:31:48,395][86122] Updated weights for policy 1, policy_version 33260 (0.0008) +[2023-10-09 13:31:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 67960832. Throughput: 0: 1811.1, 1: 1834.2. Samples: 16998414. Policy #0 lag: (min: 16.0, avg: 41.1, max: 48.0) +[2023-10-09 13:31:48,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.950')] +[2023-10-09 13:31:48,757][86122] Updated weights for policy 1, policy_version 33270 (0.0008) +[2023-10-09 13:31:49,117][86122] Updated weights for policy 1, policy_version 33280 (0.0008) +[2023-10-09 13:31:51,890][86121] Updated weights for policy 0, policy_version 33130 (0.0008) +[2023-10-09 13:31:52,262][86121] Updated weights for policy 0, policy_version 33140 (0.0008) +[2023-10-09 13:31:52,634][86121] Updated weights for policy 0, policy_version 33150 (0.0010) +[2023-10-09 13:31:52,795][86122] Updated weights for policy 1, policy_version 33290 (0.0008) +[2023-10-09 13:31:53,164][86122] Updated weights for policy 1, policy_version 33300 (0.0008) +[2023-10-09 13:31:53,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 68026368. Throughput: 0: 1798.1, 1: 1828.1. Samples: 17019432. Policy #0 lag: (min: 16.0, avg: 41.1, max: 48.0) +[2023-10-09 13:31:53,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.960')] +[2023-10-09 13:31:53,527][86122] Updated weights for policy 1, policy_version 33310 (0.0009) +[2023-10-09 13:31:56,334][86121] Updated weights for policy 0, policy_version 33160 (0.0007) +[2023-10-09 13:31:56,691][86121] Updated weights for policy 0, policy_version 33170 (0.0008) +[2023-10-09 13:31:57,064][86121] Updated weights for policy 0, policy_version 33180 (0.0009) +[2023-10-09 13:31:57,250][86122] Updated weights for policy 1, policy_version 33320 (0.0008) +[2023-10-09 13:31:57,607][86122] Updated weights for policy 1, policy_version 33330 (0.0007) +[2023-10-09 13:31:57,964][86122] Updated weights for policy 1, policy_version 33340 (0.0008) +[2023-10-09 13:31:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68124672. Throughput: 0: 1817.7, 1: 1826.2. Samples: 17031318. Policy #0 lag: (min: 16.0, avg: 41.1, max: 48.0) +[2023-10-09 13:31:58,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.960')] +[2023-10-09 13:32:00,650][86121] Updated weights for policy 0, policy_version 33190 (0.0008) +[2023-10-09 13:32:01,021][86121] Updated weights for policy 0, policy_version 33200 (0.0008) +[2023-10-09 13:32:01,379][86121] Updated weights for policy 0, policy_version 33210 (0.0007) +[2023-10-09 13:32:01,603][86122] Updated weights for policy 1, policy_version 33350 (0.0008) +[2023-10-09 13:32:01,957][86122] Updated weights for policy 1, policy_version 33360 (0.0008) +[2023-10-09 13:32:02,315][86122] Updated weights for policy 1, policy_version 33370 (0.0009) +[2023-10-09 13:32:03,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 68190208. Throughput: 0: 1804.4, 1: 1823.9. Samples: 17052120. Policy #0 lag: (min: 16.0, avg: 41.1, max: 48.0) +[2023-10-09 13:32:03,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.960')] +[2023-10-09 13:32:05,073][86121] Updated weights for policy 0, policy_version 33220 (0.0008) +[2023-10-09 13:32:05,447][86121] Updated weights for policy 0, policy_version 33230 (0.0009) +[2023-10-09 13:32:05,813][86121] Updated weights for policy 0, policy_version 33240 (0.0011) +[2023-10-09 13:32:06,091][86122] Updated weights for policy 1, policy_version 33380 (0.0008) +[2023-10-09 13:32:06,459][86122] Updated weights for policy 1, policy_version 33390 (0.0010) +[2023-10-09 13:32:06,823][86122] Updated weights for policy 1, policy_version 33400 (0.0008) +[2023-10-09 13:32:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68255744. Throughput: 0: 1812.3, 1: 1836.6. Samples: 17074476. Policy #0 lag: (min: 20.0, avg: 22.5, max: 52.0) +[2023-10-09 13:32:08,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.970')] +[2023-10-09 13:32:09,376][86121] Updated weights for policy 0, policy_version 33250 (0.0007) +[2023-10-09 13:32:09,743][86121] Updated weights for policy 0, policy_version 33260 (0.0008) +[2023-10-09 13:32:10,111][86121] Updated weights for policy 0, policy_version 33270 (0.0008) +[2023-10-09 13:32:10,378][86122] Updated weights for policy 1, policy_version 33410 (0.0008) +[2023-10-09 13:32:10,475][86121] Updated weights for policy 0, policy_version 33280 (0.0008) +[2023-10-09 13:32:10,729][86122] Updated weights for policy 1, policy_version 33420 (0.0009) +[2023-10-09 13:32:11,092][86122] Updated weights for policy 1, policy_version 33430 (0.0008) +[2023-10-09 13:32:11,451][86122] Updated weights for policy 1, policy_version 33440 (0.0010) +[2023-10-09 13:32:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68321280. Throughput: 0: 1813.2, 1: 1832.8. Samples: 17085170. Policy #0 lag: (min: 20.0, avg: 22.5, max: 52.0) +[2023-10-09 13:32:13,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.970')] +[2023-10-09 13:32:14,075][86121] Updated weights for policy 0, policy_version 33290 (0.0010) +[2023-10-09 13:32:14,443][86121] Updated weights for policy 0, policy_version 33300 (0.0008) +[2023-10-09 13:32:14,806][86121] Updated weights for policy 0, policy_version 33310 (0.0007) +[2023-10-09 13:32:15,226][86122] Updated weights for policy 1, policy_version 33450 (0.0007) +[2023-10-09 13:32:15,585][86122] Updated weights for policy 1, policy_version 33460 (0.0009) +[2023-10-09 13:32:15,948][86122] Updated weights for policy 1, policy_version 33470 (0.0008) +[2023-10-09 13:32:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68386816. Throughput: 0: 1818.0, 1: 1820.3. Samples: 17106804. Policy #0 lag: (min: 20.0, avg: 22.5, max: 52.0) +[2023-10-09 13:32:18,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.980')] +[2023-10-09 13:32:18,711][86121] Updated weights for policy 0, policy_version 33320 (0.0008) +[2023-10-09 13:32:19,082][86121] Updated weights for policy 0, policy_version 33330 (0.0009) +[2023-10-09 13:32:19,461][86121] Updated weights for policy 0, policy_version 33340 (0.0009) +[2023-10-09 13:32:19,574][86122] Updated weights for policy 1, policy_version 33480 (0.0008) +[2023-10-09 13:32:19,945][86122] Updated weights for policy 1, policy_version 33490 (0.0008) +[2023-10-09 13:32:20,300][86122] Updated weights for policy 1, policy_version 33500 (0.0008) +[2023-10-09 13:32:23,268][86121] Updated weights for policy 0, policy_version 33350 (0.0007) +[2023-10-09 13:32:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 68452352. Throughput: 0: 1825.7, 1: 1825.5. Samples: 17129908. Policy #0 lag: (min: 20.0, avg: 22.5, max: 52.0) +[2023-10-09 13:32:23,399][85186] Avg episode reward: [(0, '9.820'), (1, '9.980')] +[2023-10-09 13:32:23,632][86121] Updated weights for policy 0, policy_version 33360 (0.0007) +[2023-10-09 13:32:23,948][86122] Updated weights for policy 1, policy_version 33510 (0.0007) +[2023-10-09 13:32:23,993][86121] Updated weights for policy 0, policy_version 33370 (0.0010) +[2023-10-09 13:32:24,306][86122] Updated weights for policy 1, policy_version 33520 (0.0009) +[2023-10-09 13:32:24,671][86122] Updated weights for policy 1, policy_version 33530 (0.0008) +[2023-10-09 13:32:27,426][86121] Updated weights for policy 0, policy_version 33380 (0.0010) +[2023-10-09 13:32:27,790][86121] Updated weights for policy 0, policy_version 33390 (0.0007) +[2023-10-09 13:32:28,154][86121] Updated weights for policy 0, policy_version 33400 (0.0008) +[2023-10-09 13:32:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 68517888. Throughput: 0: 1822.7, 1: 1824.6. Samples: 17139972. Policy #0 lag: (min: 20.0, avg: 22.5, max: 52.0) +[2023-10-09 13:32:28,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.970')] +[2023-10-09 13:32:28,447][86122] Updated weights for policy 1, policy_version 33540 (0.0007) +[2023-10-09 13:32:28,827][86122] Updated weights for policy 1, policy_version 33550 (0.0007) +[2023-10-09 13:32:29,184][86122] Updated weights for policy 1, policy_version 33560 (0.0009) +[2023-10-09 13:32:31,787][86121] Updated weights for policy 0, policy_version 33410 (0.0008) +[2023-10-09 13:32:32,144][86121] Updated weights for policy 0, policy_version 33420 (0.0008) +[2023-10-09 13:32:32,520][86121] Updated weights for policy 0, policy_version 33430 (0.0007) +[2023-10-09 13:32:32,785][86122] Updated weights for policy 1, policy_version 33570 (0.0010) +[2023-10-09 13:32:32,878][86121] Updated weights for policy 0, policy_version 33440 (0.0007) +[2023-10-09 13:32:33,142][86122] Updated weights for policy 1, policy_version 33580 (0.0008) +[2023-10-09 13:32:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 68616192. Throughput: 0: 1828.3, 1: 1826.4. Samples: 17162874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:32:33,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.970')] +[2023-10-09 13:32:33,496][86122] Updated weights for policy 1, policy_version 33590 (0.0008) +[2023-10-09 13:32:33,863][86122] Updated weights for policy 1, policy_version 33600 (0.0008) +[2023-10-09 13:32:36,522][86121] Updated weights for policy 0, policy_version 33450 (0.0009) +[2023-10-09 13:32:36,885][86121] Updated weights for policy 0, policy_version 33460 (0.0011) +[2023-10-09 13:32:37,245][86121] Updated weights for policy 0, policy_version 33470 (0.0008) +[2023-10-09 13:32:37,692][86122] Updated weights for policy 1, policy_version 33610 (0.0007) +[2023-10-09 13:32:38,055][86122] Updated weights for policy 1, policy_version 33620 (0.0008) +[2023-10-09 13:32:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68681728. Throughput: 0: 1830.4, 1: 1822.8. Samples: 17183826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:32:38,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.970')] +[2023-10-09 13:32:38,423][86122] Updated weights for policy 1, policy_version 33630 (0.0009) +[2023-10-09 13:32:40,791][86121] Updated weights for policy 0, policy_version 33480 (0.0010) +[2023-10-09 13:32:41,155][86121] Updated weights for policy 0, policy_version 33490 (0.0009) +[2023-10-09 13:32:41,522][86121] Updated weights for policy 0, policy_version 33500 (0.0007) +[2023-10-09 13:32:42,136][86122] Updated weights for policy 1, policy_version 33640 (0.0009) +[2023-10-09 13:32:42,489][86122] Updated weights for policy 1, policy_version 33650 (0.0011) +[2023-10-09 13:32:42,850][86122] Updated weights for policy 1, policy_version 33660 (0.0011) +[2023-10-09 13:32:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 68780032. Throughput: 0: 1819.4, 1: 1827.4. Samples: 17195426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:32:43,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.960')] +[2023-10-09 13:32:45,281][86121] Updated weights for policy 0, policy_version 33510 (0.0010) +[2023-10-09 13:32:45,659][86121] Updated weights for policy 0, policy_version 33520 (0.0011) +[2023-10-09 13:32:46,019][86121] Updated weights for policy 0, policy_version 33530 (0.0009) +[2023-10-09 13:32:46,520][86122] Updated weights for policy 1, policy_version 33670 (0.0010) +[2023-10-09 13:32:46,873][86122] Updated weights for policy 1, policy_version 33680 (0.0009) +[2023-10-09 13:32:47,239][86122] Updated weights for policy 1, policy_version 33690 (0.0007) +[2023-10-09 13:32:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 68845568. Throughput: 0: 1834.3, 1: 1821.0. Samples: 17216608. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:32:48,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.960')] +[2023-10-09 13:32:49,670][86121] Updated weights for policy 0, policy_version 33540 (0.0009) +[2023-10-09 13:32:50,036][86121] Updated weights for policy 0, policy_version 33550 (0.0008) +[2023-10-09 13:32:50,401][86121] Updated weights for policy 0, policy_version 33560 (0.0008) +[2023-10-09 13:32:50,846][86122] Updated weights for policy 1, policy_version 33700 (0.0010) +[2023-10-09 13:32:51,211][86122] Updated weights for policy 1, policy_version 33710 (0.0010) +[2023-10-09 13:32:51,569][86122] Updated weights for policy 1, policy_version 33720 (0.0009) +[2023-10-09 13:32:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 68911104. Throughput: 0: 1829.4, 1: 1821.3. Samples: 17238754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:32:53,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.960')] +[2023-10-09 13:32:53,951][86121] Updated weights for policy 0, policy_version 33570 (0.0009) +[2023-10-09 13:32:54,320][86121] Updated weights for policy 0, policy_version 33580 (0.0009) +[2023-10-09 13:32:54,685][86121] Updated weights for policy 0, policy_version 33590 (0.0008) +[2023-10-09 13:32:55,050][86121] Updated weights for policy 0, policy_version 33600 (0.0008) +[2023-10-09 13:32:55,117][86122] Updated weights for policy 1, policy_version 33730 (0.0010) +[2023-10-09 13:32:55,469][86122] Updated weights for policy 1, policy_version 33740 (0.0010) +[2023-10-09 13:32:55,831][86122] Updated weights for policy 1, policy_version 33750 (0.0007) +[2023-10-09 13:32:56,203][86122] Updated weights for policy 1, policy_version 33760 (0.0008) +[2023-10-09 13:32:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 68976640. Throughput: 0: 1832.0, 1: 1818.9. Samples: 17249460. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:32:58,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.960')] +[2023-10-09 13:32:58,800][86121] Updated weights for policy 0, policy_version 33610 (0.0008) +[2023-10-09 13:32:59,169][86121] Updated weights for policy 0, policy_version 33620 (0.0009) +[2023-10-09 13:32:59,538][86121] Updated weights for policy 0, policy_version 33630 (0.0009) +[2023-10-09 13:32:59,873][86122] Updated weights for policy 1, policy_version 33770 (0.0010) +[2023-10-09 13:33:00,230][86122] Updated weights for policy 1, policy_version 33780 (0.0011) +[2023-10-09 13:33:00,592][86122] Updated weights for policy 1, policy_version 33790 (0.0011) +[2023-10-09 13:33:03,211][86121] Updated weights for policy 0, policy_version 33640 (0.0010) +[2023-10-09 13:33:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69042176. Throughput: 0: 1833.7, 1: 1834.1. Samples: 17271856. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:33:03,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.960')] +[2023-10-09 13:33:03,578][86121] Updated weights for policy 0, policy_version 33650 (0.0010) +[2023-10-09 13:33:03,943][86121] Updated weights for policy 0, policy_version 33660 (0.0009) +[2023-10-09 13:33:04,128][86122] Updated weights for policy 1, policy_version 33800 (0.0010) +[2023-10-09 13:33:04,495][86122] Updated weights for policy 1, policy_version 33810 (0.0008) +[2023-10-09 13:33:04,862][86122] Updated weights for policy 1, policy_version 33820 (0.0007) +[2023-10-09 13:33:07,611][86121] Updated weights for policy 0, policy_version 33670 (0.0008) +[2023-10-09 13:33:07,983][86121] Updated weights for policy 0, policy_version 33680 (0.0008) +[2023-10-09 13:33:08,341][86121] Updated weights for policy 0, policy_version 33690 (0.0008) +[2023-10-09 13:33:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69107712. Throughput: 0: 1820.6, 1: 1834.9. Samples: 17294404. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:33:08,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.960')] +[2023-10-09 13:33:08,536][86122] Updated weights for policy 1, policy_version 33830 (0.0007) +[2023-10-09 13:33:08,902][86122] Updated weights for policy 1, policy_version 33840 (0.0008) +[2023-10-09 13:33:09,258][86122] Updated weights for policy 1, policy_version 33850 (0.0007) +[2023-10-09 13:33:11,951][86121] Updated weights for policy 0, policy_version 33700 (0.0008) +[2023-10-09 13:33:12,314][86121] Updated weights for policy 0, policy_version 33710 (0.0008) +[2023-10-09 13:33:12,679][86121] Updated weights for policy 0, policy_version 33720 (0.0008) +[2023-10-09 13:33:12,902][86122] Updated weights for policy 1, policy_version 33860 (0.0007) +[2023-10-09 13:33:13,266][86122] Updated weights for policy 1, policy_version 33870 (0.0007) +[2023-10-09 13:33:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69206016. Throughput: 0: 1835.9, 1: 1837.2. Samples: 17305260. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:33:13,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.970')] +[2023-10-09 13:33:13,623][86122] Updated weights for policy 1, policy_version 33880 (0.0007) +[2023-10-09 13:33:16,416][86121] Updated weights for policy 0, policy_version 33730 (0.0008) +[2023-10-09 13:33:16,778][86121] Updated weights for policy 0, policy_version 33740 (0.0007) +[2023-10-09 13:33:17,143][86121] Updated weights for policy 0, policy_version 33750 (0.0010) +[2023-10-09 13:33:17,315][86122] Updated weights for policy 1, policy_version 33890 (0.0009) +[2023-10-09 13:33:17,513][86121] Updated weights for policy 0, policy_version 33760 (0.0008) +[2023-10-09 13:33:17,703][86122] Updated weights for policy 1, policy_version 33900 (0.0007) +[2023-10-09 13:33:18,063][86122] Updated weights for policy 1, policy_version 33910 (0.0009) +[2023-10-09 13:33:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69271552. Throughput: 0: 1819.5, 1: 1840.9. Samples: 17327592. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:33:18,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.970')] +[2023-10-09 13:33:18,425][86122] Updated weights for policy 1, policy_version 33920 (0.0008) +[2023-10-09 13:33:21,285][86121] Updated weights for policy 0, policy_version 33770 (0.0007) +[2023-10-09 13:33:21,652][86121] Updated weights for policy 0, policy_version 33780 (0.0008) +[2023-10-09 13:33:22,027][86121] Updated weights for policy 0, policy_version 33790 (0.0008) +[2023-10-09 13:33:22,095][86122] Updated weights for policy 1, policy_version 33930 (0.0007) +[2023-10-09 13:33:22,465][86122] Updated weights for policy 1, policy_version 33940 (0.0007) +[2023-10-09 13:33:22,822][86122] Updated weights for policy 1, policy_version 33950 (0.0008) +[2023-10-09 13:33:23,398][85186] Fps is (10 sec: 16383.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 69369856. Throughput: 0: 1830.8, 1: 1823.3. Samples: 17348264. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 13:33:23,399][85186] Avg episode reward: [(0, '9.810'), (1, '9.970')] +[2023-10-09 13:33:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000033952_34766848.pth... +[2023-10-09 13:33:23,411][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000033792_34603008.pth... +[2023-10-09 13:33:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000032096_32866304.pth +[2023-10-09 13:33:23,447][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000032224_32997376.pth +[2023-10-09 13:33:23,451][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000033792_34603008.pth +[2023-10-09 13:33:23,451][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000033952_34766848.pth +[2023-10-09 13:33:25,618][86121] Updated weights for policy 0, policy_version 33800 (0.0009) +[2023-10-09 13:33:25,987][86121] Updated weights for policy 0, policy_version 33810 (0.0008) +[2023-10-09 13:33:26,357][86121] Updated weights for policy 0, policy_version 33820 (0.0007) +[2023-10-09 13:33:26,583][86122] Updated weights for policy 1, policy_version 33960 (0.0007) +[2023-10-09 13:33:26,955][86122] Updated weights for policy 1, policy_version 33970 (0.0009) +[2023-10-09 13:33:27,324][86122] Updated weights for policy 1, policy_version 33980 (0.0007) +[2023-10-09 13:33:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 69435392. Throughput: 0: 1825.1, 1: 1838.9. Samples: 17360302. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 13:33:28,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.970')] +[2023-10-09 13:33:30,053][86121] Updated weights for policy 0, policy_version 33830 (0.0009) +[2023-10-09 13:33:30,423][86121] Updated weights for policy 0, policy_version 33840 (0.0011) +[2023-10-09 13:33:30,785][86121] Updated weights for policy 0, policy_version 33850 (0.0009) +[2023-10-09 13:33:30,994][86122] Updated weights for policy 1, policy_version 33990 (0.0008) +[2023-10-09 13:33:31,350][86122] Updated weights for policy 1, policy_version 34000 (0.0008) +[2023-10-09 13:33:31,698][86122] Updated weights for policy 1, policy_version 34010 (0.0010) +[2023-10-09 13:33:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 69500928. Throughput: 0: 1826.4, 1: 1825.9. Samples: 17380962. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 13:33:33,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.970')] +[2023-10-09 13:33:34,725][86121] Updated weights for policy 0, policy_version 33860 (0.0008) +[2023-10-09 13:33:35,119][86121] Updated weights for policy 0, policy_version 33870 (0.0009) +[2023-10-09 13:33:35,354][86122] Updated weights for policy 1, policy_version 34020 (0.0008) +[2023-10-09 13:33:35,479][86121] Updated weights for policy 0, policy_version 33880 (0.0007) +[2023-10-09 13:33:35,709][86122] Updated weights for policy 1, policy_version 34030 (0.0008) +[2023-10-09 13:33:36,065][86122] Updated weights for policy 1, policy_version 34040 (0.0009) +[2023-10-09 13:33:38,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 69566464. Throughput: 0: 1822.7, 1: 1835.8. Samples: 17403390. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 13:33:38,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.970')] +[2023-10-09 13:33:39,171][86121] Updated weights for policy 0, policy_version 33890 (0.0009) +[2023-10-09 13:33:39,531][86121] Updated weights for policy 0, policy_version 33900 (0.0007) +[2023-10-09 13:33:39,822][86122] Updated weights for policy 1, policy_version 34050 (0.0008) +[2023-10-09 13:33:39,904][86121] Updated weights for policy 0, policy_version 33910 (0.0007) +[2023-10-09 13:33:40,180][86122] Updated weights for policy 1, policy_version 34060 (0.0007) +[2023-10-09 13:33:40,259][86121] Updated weights for policy 0, policy_version 33920 (0.0008) +[2023-10-09 13:33:40,541][86122] Updated weights for policy 1, policy_version 34070 (0.0010) +[2023-10-09 13:33:40,900][86122] Updated weights for policy 1, policy_version 34080 (0.0007) +[2023-10-09 13:33:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69632000. Throughput: 0: 1820.0, 1: 1824.5. Samples: 17413466. Policy #0 lag: (min: 21.0, avg: 28.8, max: 53.0) +[2023-10-09 13:33:43,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.970')] +[2023-10-09 13:33:44,108][86121] Updated weights for policy 0, policy_version 33930 (0.0007) +[2023-10-09 13:33:44,478][86121] Updated weights for policy 0, policy_version 33940 (0.0007) +[2023-10-09 13:33:44,612][86122] Updated weights for policy 1, policy_version 34090 (0.0007) +[2023-10-09 13:33:44,835][86121] Updated weights for policy 0, policy_version 33950 (0.0007) +[2023-10-09 13:33:44,970][86122] Updated weights for policy 1, policy_version 34100 (0.0007) +[2023-10-09 13:33:45,330][86122] Updated weights for policy 1, policy_version 34110 (0.0008) +[2023-10-09 13:33:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 69697536. Throughput: 0: 1811.0, 1: 1837.1. Samples: 17436018. Policy #0 lag: (min: 3.0, avg: 27.6, max: 32.0) +[2023-10-09 13:33:48,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.970')] +[2023-10-09 13:33:48,581][86121] Updated weights for policy 0, policy_version 33960 (0.0008) +[2023-10-09 13:33:48,948][86121] Updated weights for policy 0, policy_version 33970 (0.0008) +[2023-10-09 13:33:49,085][86122] Updated weights for policy 1, policy_version 34120 (0.0009) +[2023-10-09 13:33:49,319][86121] Updated weights for policy 0, policy_version 33980 (0.0008) +[2023-10-09 13:33:49,454][86122] Updated weights for policy 1, policy_version 34130 (0.0007) +[2023-10-09 13:33:49,823][86122] Updated weights for policy 1, policy_version 34140 (0.0009) +[2023-10-09 13:33:53,026][86121] Updated weights for policy 0, policy_version 33990 (0.0009) +[2023-10-09 13:33:53,390][86121] Updated weights for policy 0, policy_version 34000 (0.0009) +[2023-10-09 13:33:53,398][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 69763072. Throughput: 0: 1820.6, 1: 1828.7. Samples: 17458622. Policy #0 lag: (min: 3.0, avg: 27.6, max: 32.0) +[2023-10-09 13:33:53,399][85186] Avg episode reward: [(0, '9.820'), (1, '9.970')] +[2023-10-09 13:33:53,422][86122] Updated weights for policy 1, policy_version 34150 (0.0008) +[2023-10-09 13:33:53,751][86121] Updated weights for policy 0, policy_version 34010 (0.0007) +[2023-10-09 13:33:53,779][86122] Updated weights for policy 1, policy_version 34160 (0.0008) +[2023-10-09 13:33:54,145][86122] Updated weights for policy 1, policy_version 34170 (0.0007) +[2023-10-09 13:33:57,298][86121] Updated weights for policy 0, policy_version 34020 (0.0008) +[2023-10-09 13:33:57,672][86121] Updated weights for policy 0, policy_version 34030 (0.0008) +[2023-10-09 13:33:57,740][86122] Updated weights for policy 1, policy_version 34180 (0.0008) +[2023-10-09 13:33:58,037][86121] Updated weights for policy 0, policy_version 34040 (0.0010) +[2023-10-09 13:33:58,095][86122] Updated weights for policy 1, policy_version 34190 (0.0009) +[2023-10-09 13:33:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69861376. Throughput: 0: 1807.5, 1: 1824.8. Samples: 17468714. Policy #0 lag: (min: 3.0, avg: 27.6, max: 32.0) +[2023-10-09 13:33:58,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.970')] +[2023-10-09 13:33:58,461][86122] Updated weights for policy 1, policy_version 34200 (0.0007) +[2023-10-09 13:34:01,729][86121] Updated weights for policy 0, policy_version 34050 (0.0009) +[2023-10-09 13:34:02,098][86121] Updated weights for policy 0, policy_version 34060 (0.0007) +[2023-10-09 13:34:02,319][86122] Updated weights for policy 1, policy_version 34210 (0.0008) +[2023-10-09 13:34:02,456][86121] Updated weights for policy 0, policy_version 34070 (0.0007) +[2023-10-09 13:34:02,722][86122] Updated weights for policy 1, policy_version 34220 (0.0008) +[2023-10-09 13:34:02,809][86121] Updated weights for policy 0, policy_version 34080 (0.0007) +[2023-10-09 13:34:03,073][86122] Updated weights for policy 1, policy_version 34230 (0.0010) +[2023-10-09 13:34:03,397][85186] Fps is (10 sec: 16384.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 69926912. Throughput: 0: 1814.7, 1: 1820.3. Samples: 17491164. Policy #0 lag: (min: 3.0, avg: 27.6, max: 32.0) +[2023-10-09 13:34:03,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.970')] +[2023-10-09 13:34:03,438][86122] Updated weights for policy 1, policy_version 34240 (0.0009) +[2023-10-09 13:34:06,647][86121] Updated weights for policy 0, policy_version 34090 (0.0008) +[2023-10-09 13:34:07,007][86121] Updated weights for policy 0, policy_version 34100 (0.0007) +[2023-10-09 13:34:07,089][86122] Updated weights for policy 1, policy_version 34250 (0.0007) +[2023-10-09 13:34:07,365][86121] Updated weights for policy 0, policy_version 34110 (0.0009) +[2023-10-09 13:34:07,457][86122] Updated weights for policy 1, policy_version 34260 (0.0007) +[2023-10-09 13:34:07,811][86122] Updated weights for policy 1, policy_version 34270 (0.0007) +[2023-10-09 13:34:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 70025216. Throughput: 0: 1807.3, 1: 1818.9. Samples: 17511444. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) +[2023-10-09 13:34:08,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.970')] +[2023-10-09 13:34:10,967][86121] Updated weights for policy 0, policy_version 34120 (0.0007) +[2023-10-09 13:34:11,333][86121] Updated weights for policy 0, policy_version 34130 (0.0008) +[2023-10-09 13:34:11,462][86122] Updated weights for policy 1, policy_version 34280 (0.0008) +[2023-10-09 13:34:11,705][86121] Updated weights for policy 0, policy_version 34140 (0.0007) +[2023-10-09 13:34:11,823][86122] Updated weights for policy 1, policy_version 34290 (0.0008) +[2023-10-09 13:34:12,200][86122] Updated weights for policy 1, policy_version 34300 (0.0009) +[2023-10-09 13:34:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70090752. Throughput: 0: 1813.7, 1: 1820.9. Samples: 17523860. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) +[2023-10-09 13:34:13,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.970')] +[2023-10-09 13:34:15,532][86121] Updated weights for policy 0, policy_version 34150 (0.0008) +[2023-10-09 13:34:15,895][86121] Updated weights for policy 0, policy_version 34160 (0.0008) +[2023-10-09 13:34:16,030][86122] Updated weights for policy 1, policy_version 34310 (0.0009) +[2023-10-09 13:34:16,258][86121] Updated weights for policy 0, policy_version 34170 (0.0009) +[2023-10-09 13:34:16,390][86122] Updated weights for policy 1, policy_version 34320 (0.0007) +[2023-10-09 13:34:16,756][86122] Updated weights for policy 1, policy_version 34330 (0.0010) +[2023-10-09 13:34:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70156288. Throughput: 0: 1806.6, 1: 1819.3. Samples: 17544128. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) +[2023-10-09 13:34:18,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.970')] +[2023-10-09 13:34:19,933][86121] Updated weights for policy 0, policy_version 34180 (0.0007) +[2023-10-09 13:34:20,325][86121] Updated weights for policy 0, policy_version 34190 (0.0007) +[2023-10-09 13:34:20,400][86122] Updated weights for policy 1, policy_version 34340 (0.0008) +[2023-10-09 13:34:20,688][86121] Updated weights for policy 0, policy_version 34200 (0.0007) +[2023-10-09 13:34:20,767][86122] Updated weights for policy 1, policy_version 34350 (0.0009) +[2023-10-09 13:34:21,123][86122] Updated weights for policy 1, policy_version 34360 (0.0008) +[2023-10-09 13:34:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70221824. Throughput: 0: 1807.4, 1: 1819.5. Samples: 17566598. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) +[2023-10-09 13:34:23,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.990')] +[2023-10-09 13:34:24,311][86121] Updated weights for policy 0, policy_version 34210 (0.0008) +[2023-10-09 13:34:24,673][86121] Updated weights for policy 0, policy_version 34220 (0.0010) +[2023-10-09 13:34:24,741][86122] Updated weights for policy 1, policy_version 34370 (0.0009) +[2023-10-09 13:34:25,039][86121] Updated weights for policy 0, policy_version 34230 (0.0008) +[2023-10-09 13:34:25,102][86122] Updated weights for policy 1, policy_version 34380 (0.0009) +[2023-10-09 13:34:25,403][86121] Updated weights for policy 0, policy_version 34240 (0.0007) +[2023-10-09 13:34:25,471][86122] Updated weights for policy 1, policy_version 34390 (0.0008) +[2023-10-09 13:34:25,832][86122] Updated weights for policy 1, policy_version 34400 (0.0009) +[2023-10-09 13:34:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 70287360. Throughput: 0: 1807.1, 1: 1818.1. Samples: 17576600. Policy #0 lag: (min: 10.0, avg: 13.0, max: 42.0) +[2023-10-09 13:34:28,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.990')] +[2023-10-09 13:34:29,111][86121] Updated weights for policy 0, policy_version 34250 (0.0007) +[2023-10-09 13:34:29,472][86121] Updated weights for policy 0, policy_version 34260 (0.0008) +[2023-10-09 13:34:29,623][86122] Updated weights for policy 1, policy_version 34410 (0.0008) +[2023-10-09 13:34:29,838][86121] Updated weights for policy 0, policy_version 34270 (0.0008) +[2023-10-09 13:34:29,992][86122] Updated weights for policy 1, policy_version 34420 (0.0010) +[2023-10-09 13:34:30,352][86122] Updated weights for policy 1, policy_version 34430 (0.0009) +[2023-10-09 13:34:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70352896. Throughput: 0: 1813.6, 1: 1813.6. Samples: 17599240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:34:33,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.990')] +[2023-10-09 13:34:33,566][86121] Updated weights for policy 0, policy_version 34280 (0.0009) +[2023-10-09 13:34:33,932][86121] Updated weights for policy 0, policy_version 34290 (0.0008) +[2023-10-09 13:34:34,101][86122] Updated weights for policy 1, policy_version 34440 (0.0009) +[2023-10-09 13:34:34,293][86121] Updated weights for policy 0, policy_version 34300 (0.0009) +[2023-10-09 13:34:34,452][86122] Updated weights for policy 1, policy_version 34450 (0.0007) +[2023-10-09 13:34:34,815][86122] Updated weights for policy 1, policy_version 34460 (0.0008) +[2023-10-09 13:34:37,913][86121] Updated weights for policy 0, policy_version 34310 (0.0011) +[2023-10-09 13:34:38,283][86121] Updated weights for policy 0, policy_version 34320 (0.0009) +[2023-10-09 13:34:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70418432. Throughput: 0: 1814.7, 1: 1814.1. Samples: 17621916. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:34:38,398][85186] Avg episode reward: [(0, '9.850'), (1, '10.000')] +[2023-10-09 13:34:38,643][86121] Updated weights for policy 0, policy_version 34330 (0.0007) +[2023-10-09 13:34:38,681][86122] Updated weights for policy 1, policy_version 34470 (0.0007) +[2023-10-09 13:34:39,044][86122] Updated weights for policy 1, policy_version 34480 (0.0007) +[2023-10-09 13:34:39,412][86122] Updated weights for policy 1, policy_version 34490 (0.0007) +[2023-10-09 13:34:42,429][86121] Updated weights for policy 0, policy_version 34340 (0.0008) +[2023-10-09 13:34:42,801][86121] Updated weights for policy 0, policy_version 34350 (0.0008) +[2023-10-09 13:34:43,158][86122] Updated weights for policy 1, policy_version 34500 (0.0009) +[2023-10-09 13:34:43,169][86121] Updated weights for policy 0, policy_version 34360 (0.0008) +[2023-10-09 13:34:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 70483968. Throughput: 0: 1811.9, 1: 1815.9. Samples: 17631964. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:34:43,398][85186] Avg episode reward: [(0, '9.860'), (1, '10.000')] +[2023-10-09 13:34:43,516][86122] Updated weights for policy 1, policy_version 34510 (0.0009) +[2023-10-09 13:34:43,874][86122] Updated weights for policy 1, policy_version 34520 (0.0009) +[2023-10-09 13:34:46,941][86121] Updated weights for policy 0, policy_version 34370 (0.0009) +[2023-10-09 13:34:47,312][86121] Updated weights for policy 0, policy_version 34380 (0.0009) +[2023-10-09 13:34:47,467][86122] Updated weights for policy 1, policy_version 34530 (0.0009) +[2023-10-09 13:34:47,678][86121] Updated weights for policy 0, policy_version 34390 (0.0008) +[2023-10-09 13:34:47,879][86122] Updated weights for policy 1, policy_version 34540 (0.0009) +[2023-10-09 13:34:48,042][86121] Updated weights for policy 0, policy_version 34400 (0.0009) +[2023-10-09 13:34:48,252][86122] Updated weights for policy 1, policy_version 34550 (0.0010) +[2023-10-09 13:34:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 70582272. Throughput: 0: 1817.2, 1: 1820.7. Samples: 17654874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:34:48,398][85186] Avg episode reward: [(0, '9.870'), (1, '10.000')] +[2023-10-09 13:34:48,612][86122] Updated weights for policy 1, policy_version 34560 (0.0011) +[2023-10-09 13:34:51,589][86121] Updated weights for policy 0, policy_version 34410 (0.0009) +[2023-10-09 13:34:51,961][86121] Updated weights for policy 0, policy_version 34420 (0.0008) +[2023-10-09 13:34:52,285][86122] Updated weights for policy 1, policy_version 34570 (0.0008) +[2023-10-09 13:34:52,328][86121] Updated weights for policy 0, policy_version 34430 (0.0007) +[2023-10-09 13:34:52,641][86122] Updated weights for policy 1, policy_version 34580 (0.0008) +[2023-10-09 13:34:53,003][86122] Updated weights for policy 1, policy_version 34590 (0.0008) +[2023-10-09 13:34:53,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 70680576. Throughput: 0: 1815.6, 1: 1823.4. Samples: 17675200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:34:53,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 13:34:56,181][86121] Updated weights for policy 0, policy_version 34440 (0.0008) +[2023-10-09 13:34:56,544][86121] Updated weights for policy 0, policy_version 34450 (0.0009) +[2023-10-09 13:34:56,763][86122] Updated weights for policy 1, policy_version 34600 (0.0007) +[2023-10-09 13:34:56,913][86121] Updated weights for policy 0, policy_version 34460 (0.0007) +[2023-10-09 13:34:57,122][86122] Updated weights for policy 1, policy_version 34610 (0.0007) +[2023-10-09 13:34:57,492][86122] Updated weights for policy 1, policy_version 34620 (0.0009) +[2023-10-09 13:34:58,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 70746112. Throughput: 0: 1820.0, 1: 1811.6. Samples: 17687282. Policy #0 lag: (min: 20.0, avg: 21.0, max: 40.0) +[2023-10-09 13:34:58,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 13:35:00,632][86121] Updated weights for policy 0, policy_version 34470 (0.0009) +[2023-10-09 13:35:00,996][86121] Updated weights for policy 0, policy_version 34480 (0.0009) +[2023-10-09 13:35:01,232][86122] Updated weights for policy 1, policy_version 34630 (0.0008) +[2023-10-09 13:35:01,356][86121] Updated weights for policy 0, policy_version 34490 (0.0008) +[2023-10-09 13:35:01,589][86122] Updated weights for policy 1, policy_version 34640 (0.0010) +[2023-10-09 13:35:01,948][86122] Updated weights for policy 1, policy_version 34650 (0.0010) +[2023-10-09 13:35:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 70811648. Throughput: 0: 1815.5, 1: 1821.8. Samples: 17707804. Policy #0 lag: (min: 20.0, avg: 21.0, max: 40.0) +[2023-10-09 13:35:03,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 13:35:05,004][86121] Updated weights for policy 0, policy_version 34500 (0.0008) +[2023-10-09 13:35:05,372][86121] Updated weights for policy 0, policy_version 34510 (0.0008) +[2023-10-09 13:35:05,661][86122] Updated weights for policy 1, policy_version 34660 (0.0009) +[2023-10-09 13:35:05,738][86121] Updated weights for policy 0, policy_version 34520 (0.0008) +[2023-10-09 13:35:06,014][86122] Updated weights for policy 1, policy_version 34670 (0.0008) +[2023-10-09 13:35:06,377][86122] Updated weights for policy 1, policy_version 34680 (0.0011) +[2023-10-09 13:35:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70877184. Throughput: 0: 1821.8, 1: 1818.0. Samples: 17730390. Policy #0 lag: (min: 20.0, avg: 21.0, max: 40.0) +[2023-10-09 13:35:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 13:35:09,424][86121] Updated weights for policy 0, policy_version 34530 (0.0008) +[2023-10-09 13:35:09,787][86121] Updated weights for policy 0, policy_version 34540 (0.0009) +[2023-10-09 13:35:10,024][86122] Updated weights for policy 1, policy_version 34690 (0.0009) +[2023-10-09 13:35:10,150][86121] Updated weights for policy 0, policy_version 34550 (0.0009) +[2023-10-09 13:35:10,385][86122] Updated weights for policy 1, policy_version 34700 (0.0007) +[2023-10-09 13:35:10,506][86121] Updated weights for policy 0, policy_version 34560 (0.0009) +[2023-10-09 13:35:10,750][86122] Updated weights for policy 1, policy_version 34710 (0.0008) +[2023-10-09 13:35:11,108][86122] Updated weights for policy 1, policy_version 34720 (0.0010) +[2023-10-09 13:35:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 70942720. Throughput: 0: 1821.1, 1: 1822.9. Samples: 17740580. Policy #0 lag: (min: 20.0, avg: 21.0, max: 40.0) +[2023-10-09 13:35:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 13:35:14,209][86121] Updated weights for policy 0, policy_version 34570 (0.0010) +[2023-10-09 13:35:14,569][86121] Updated weights for policy 0, policy_version 34580 (0.0009) +[2023-10-09 13:35:14,935][86121] Updated weights for policy 0, policy_version 34590 (0.0007) +[2023-10-09 13:35:15,015][86122] Updated weights for policy 1, policy_version 34730 (0.0007) +[2023-10-09 13:35:15,386][86122] Updated weights for policy 1, policy_version 34740 (0.0008) +[2023-10-09 13:35:15,745][86122] Updated weights for policy 1, policy_version 34750 (0.0009) +[2023-10-09 13:35:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71008256. Throughput: 0: 1817.5, 1: 1817.2. Samples: 17762804. Policy #0 lag: (min: 20.0, avg: 21.0, max: 40.0) +[2023-10-09 13:35:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 13:35:18,757][86121] Updated weights for policy 0, policy_version 34600 (0.0010) +[2023-10-09 13:35:19,131][86121] Updated weights for policy 0, policy_version 34610 (0.0009) +[2023-10-09 13:35:19,430][86122] Updated weights for policy 1, policy_version 34760 (0.0007) +[2023-10-09 13:35:19,492][86121] Updated weights for policy 0, policy_version 34620 (0.0007) +[2023-10-09 13:35:19,801][86122] Updated weights for policy 1, policy_version 34770 (0.0008) +[2023-10-09 13:35:20,157][86122] Updated weights for policy 1, policy_version 34780 (0.0007) +[2023-10-09 13:35:23,197][86121] Updated weights for policy 0, policy_version 34630 (0.0008) +[2023-10-09 13:35:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 71073792. Throughput: 0: 1817.9, 1: 1822.1. Samples: 17785718. Policy #0 lag: (min: 20.0, avg: 21.0, max: 40.0) +[2023-10-09 13:35:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 13:35:23,567][86121] Updated weights for policy 0, policy_version 34640 (0.0008) +[2023-10-09 13:35:23,705][86122] Updated weights for policy 1, policy_version 34790 (0.0007) +[2023-10-09 13:35:23,934][86121] Updated weights for policy 0, policy_version 34650 (0.0008) +[2023-10-09 13:35:24,060][86122] Updated weights for policy 1, policy_version 34800 (0.0007) +[2023-10-09 13:35:24,149][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000034656_35487744.pth... +[2023-10-09 13:35:24,188][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000032928_33718272.pth +[2023-10-09 13:35:24,426][86122] Updated weights for policy 1, policy_version 34810 (0.0007) +[2023-10-09 13:35:24,637][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000034816_35651584.pth... +[2023-10-09 13:35:24,666][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000033088_33882112.pth +[2023-10-09 13:35:27,658][86121] Updated weights for policy 0, policy_version 34660 (0.0008) +[2023-10-09 13:35:28,013][86122] Updated weights for policy 1, policy_version 34820 (0.0008) +[2023-10-09 13:35:28,030][86121] Updated weights for policy 0, policy_version 34670 (0.0010) +[2023-10-09 13:35:28,380][86122] Updated weights for policy 1, policy_version 34830 (0.0008) +[2023-10-09 13:35:28,389][86121] Updated weights for policy 0, policy_version 34680 (0.0008) +[2023-10-09 13:35:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71139328. Throughput: 0: 1818.0, 1: 1822.8. Samples: 17795802. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:35:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 13:35:28,730][86122] Updated weights for policy 1, policy_version 34840 (0.0009) +[2023-10-09 13:35:31,977][86121] Updated weights for policy 0, policy_version 34690 (0.0008) +[2023-10-09 13:35:32,336][86121] Updated weights for policy 0, policy_version 34700 (0.0009) +[2023-10-09 13:35:32,541][86122] Updated weights for policy 1, policy_version 34850 (0.0008) +[2023-10-09 13:35:32,713][86121] Updated weights for policy 0, policy_version 34710 (0.0009) +[2023-10-09 13:35:32,953][86122] Updated weights for policy 1, policy_version 34860 (0.0009) +[2023-10-09 13:35:33,076][86121] Updated weights for policy 0, policy_version 34720 (0.0009) +[2023-10-09 13:35:33,317][86122] Updated weights for policy 1, policy_version 34870 (0.0008) +[2023-10-09 13:35:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 71237632. Throughput: 0: 1821.1, 1: 1815.9. Samples: 17818536. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:35:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:35:33,681][86122] Updated weights for policy 1, policy_version 34880 (0.0007) +[2023-10-09 13:35:36,719][86121] Updated weights for policy 0, policy_version 34730 (0.0007) +[2023-10-09 13:35:37,083][86121] Updated weights for policy 0, policy_version 34740 (0.0008) +[2023-10-09 13:35:37,415][86122] Updated weights for policy 1, policy_version 34890 (0.0008) +[2023-10-09 13:35:37,455][86121] Updated weights for policy 0, policy_version 34750 (0.0008) +[2023-10-09 13:35:37,775][86122] Updated weights for policy 1, policy_version 34900 (0.0008) +[2023-10-09 13:35:38,144][86122] Updated weights for policy 1, policy_version 34910 (0.0008) +[2023-10-09 13:35:38,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 71335936. Throughput: 0: 1814.9, 1: 1821.2. Samples: 17838826. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:35:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:35:41,128][86121] Updated weights for policy 0, policy_version 34760 (0.0010) +[2023-10-09 13:35:41,496][86121] Updated weights for policy 0, policy_version 34770 (0.0008) +[2023-10-09 13:35:41,710][86122] Updated weights for policy 1, policy_version 34920 (0.0009) +[2023-10-09 13:35:41,859][86121] Updated weights for policy 0, policy_version 34780 (0.0009) +[2023-10-09 13:35:42,069][86122] Updated weights for policy 1, policy_version 34930 (0.0009) +[2023-10-09 13:35:42,426][86122] Updated weights for policy 1, policy_version 34940 (0.0010) +[2023-10-09 13:35:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 71401472. Throughput: 0: 1812.7, 1: 1825.9. Samples: 17851018. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:35:43,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 13:35:45,591][86121] Updated weights for policy 0, policy_version 34790 (0.0009) +[2023-10-09 13:35:45,946][86121] Updated weights for policy 0, policy_version 34800 (0.0008) +[2023-10-09 13:35:46,084][86122] Updated weights for policy 1, policy_version 34950 (0.0009) +[2023-10-09 13:35:46,320][86121] Updated weights for policy 0, policy_version 34810 (0.0008) +[2023-10-09 13:35:46,441][86122] Updated weights for policy 1, policy_version 34960 (0.0008) +[2023-10-09 13:35:46,797][86122] Updated weights for policy 1, policy_version 34970 (0.0008) +[2023-10-09 13:35:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 71467008. Throughput: 0: 1810.7, 1: 1822.3. Samples: 17871288. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:35:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:35:50,181][86121] Updated weights for policy 0, policy_version 34820 (0.0009) +[2023-10-09 13:35:50,555][86122] Updated weights for policy 1, policy_version 34980 (0.0010) +[2023-10-09 13:35:50,563][86121] Updated weights for policy 0, policy_version 34830 (0.0007) +[2023-10-09 13:35:50,910][86122] Updated weights for policy 1, policy_version 34990 (0.0008) +[2023-10-09 13:35:50,931][86121] Updated weights for policy 0, policy_version 34840 (0.0008) +[2023-10-09 13:35:51,277][86122] Updated weights for policy 1, policy_version 35000 (0.0009) +[2023-10-09 13:35:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71532544. Throughput: 0: 1802.8, 1: 1823.1. Samples: 17893554. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 13:35:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:35:54,801][86121] Updated weights for policy 0, policy_version 34850 (0.0008) +[2023-10-09 13:35:54,989][86122] Updated weights for policy 1, policy_version 35010 (0.0011) +[2023-10-09 13:35:55,174][86121] Updated weights for policy 0, policy_version 34860 (0.0007) +[2023-10-09 13:35:55,347][86122] Updated weights for policy 1, policy_version 35020 (0.0008) +[2023-10-09 13:35:55,541][86121] Updated weights for policy 0, policy_version 34870 (0.0007) +[2023-10-09 13:35:55,710][86122] Updated weights for policy 1, policy_version 35030 (0.0009) +[2023-10-09 13:35:55,903][86121] Updated weights for policy 0, policy_version 34880 (0.0009) +[2023-10-09 13:35:56,072][86122] Updated weights for policy 1, policy_version 35040 (0.0008) +[2023-10-09 13:35:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71598080. Throughput: 0: 1805.7, 1: 1821.1. Samples: 17903786. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 13:35:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 13:35:59,556][86121] Updated weights for policy 0, policy_version 34890 (0.0009) +[2023-10-09 13:35:59,776][86122] Updated weights for policy 1, policy_version 35050 (0.0008) +[2023-10-09 13:35:59,914][86121] Updated weights for policy 0, policy_version 34900 (0.0009) +[2023-10-09 13:36:00,137][86122] Updated weights for policy 1, policy_version 35060 (0.0007) +[2023-10-09 13:36:00,278][86121] Updated weights for policy 0, policy_version 34910 (0.0009) +[2023-10-09 13:36:00,500][86122] Updated weights for policy 1, policy_version 35070 (0.0009) +[2023-10-09 13:36:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 71663616. Throughput: 0: 1806.3, 1: 1822.5. Samples: 17926100. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 13:36:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:36:04,028][86121] Updated weights for policy 0, policy_version 34920 (0.0010) +[2023-10-09 13:36:04,301][86122] Updated weights for policy 1, policy_version 35080 (0.0009) +[2023-10-09 13:36:04,400][86121] Updated weights for policy 0, policy_version 34930 (0.0008) +[2023-10-09 13:36:04,652][86122] Updated weights for policy 1, policy_version 35090 (0.0009) +[2023-10-09 13:36:04,774][86121] Updated weights for policy 0, policy_version 34940 (0.0010) +[2023-10-09 13:36:05,011][86122] Updated weights for policy 1, policy_version 35100 (0.0009) +[2023-10-09 13:36:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71729152. Throughput: 0: 1808.3, 1: 1813.7. Samples: 17948708. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 13:36:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 13:36:08,466][86121] Updated weights for policy 0, policy_version 34950 (0.0008) +[2023-10-09 13:36:08,719][86122] Updated weights for policy 1, policy_version 35110 (0.0009) +[2023-10-09 13:36:08,833][86121] Updated weights for policy 0, policy_version 34960 (0.0007) +[2023-10-09 13:36:09,076][86122] Updated weights for policy 1, policy_version 35120 (0.0009) +[2023-10-09 13:36:09,192][86121] Updated weights for policy 0, policy_version 34970 (0.0007) +[2023-10-09 13:36:09,436][86122] Updated weights for policy 1, policy_version 35130 (0.0007) +[2023-10-09 13:36:13,012][86121] Updated weights for policy 0, policy_version 34980 (0.0009) +[2023-10-09 13:36:13,125][86122] Updated weights for policy 1, policy_version 35140 (0.0009) +[2023-10-09 13:36:13,386][86121] Updated weights for policy 0, policy_version 34990 (0.0007) +[2023-10-09 13:36:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71794688. Throughput: 0: 1805.3, 1: 1812.1. Samples: 17958584. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 13:36:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 13:36:13,484][86122] Updated weights for policy 1, policy_version 35150 (0.0007) +[2023-10-09 13:36:13,752][86121] Updated weights for policy 0, policy_version 35000 (0.0007) +[2023-10-09 13:36:13,849][86122] Updated weights for policy 1, policy_version 35160 (0.0008) +[2023-10-09 13:36:17,404][86121] Updated weights for policy 0, policy_version 35010 (0.0007) +[2023-10-09 13:36:17,488][86122] Updated weights for policy 1, policy_version 35170 (0.0009) +[2023-10-09 13:36:17,763][86121] Updated weights for policy 0, policy_version 35020 (0.0008) +[2023-10-09 13:36:17,872][86122] Updated weights for policy 1, policy_version 35180 (0.0008) +[2023-10-09 13:36:18,140][86121] Updated weights for policy 0, policy_version 35030 (0.0009) +[2023-10-09 13:36:18,239][86122] Updated weights for policy 1, policy_version 35190 (0.0008) +[2023-10-09 13:36:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 71860224. Throughput: 0: 1798.6, 1: 1816.9. Samples: 17981234. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 13:36:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 13:36:18,501][86121] Updated weights for policy 0, policy_version 35040 (0.0009) +[2023-10-09 13:36:18,596][86122] Updated weights for policy 1, policy_version 35200 (0.0008) +[2023-10-09 13:36:22,134][86121] Updated weights for policy 0, policy_version 35050 (0.0008) +[2023-10-09 13:36:22,316][86122] Updated weights for policy 1, policy_version 35210 (0.0008) +[2023-10-09 13:36:22,494][86121] Updated weights for policy 0, policy_version 35060 (0.0008) +[2023-10-09 13:36:22,671][86122] Updated weights for policy 1, policy_version 35220 (0.0008) +[2023-10-09 13:36:22,852][86121] Updated weights for policy 0, policy_version 35070 (0.0010) +[2023-10-09 13:36:23,037][86122] Updated weights for policy 1, policy_version 35230 (0.0008) +[2023-10-09 13:36:23,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 71991296. Throughput: 0: 1801.2, 1: 1820.6. Samples: 18001808. Policy #0 lag: (min: 12.0, avg: 12.8, max: 30.0) +[2023-10-09 13:36:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 13:36:26,520][86121] Updated weights for policy 0, policy_version 35080 (0.0008) +[2023-10-09 13:36:26,651][86122] Updated weights for policy 1, policy_version 35240 (0.0008) +[2023-10-09 13:36:26,889][86121] Updated weights for policy 0, policy_version 35090 (0.0008) +[2023-10-09 13:36:27,010][86122] Updated weights for policy 1, policy_version 35250 (0.0008) +[2023-10-09 13:36:27,248][86121] Updated weights for policy 0, policy_version 35100 (0.0008) +[2023-10-09 13:36:27,371][86122] Updated weights for policy 1, policy_version 35260 (0.0008) +[2023-10-09 13:36:28,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72056832. Throughput: 0: 1808.3, 1: 1818.3. Samples: 18014212. Policy #0 lag: (min: 12.0, avg: 12.8, max: 30.0) +[2023-10-09 13:36:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 13:36:30,977][86122] Updated weights for policy 1, policy_version 35270 (0.0008) +[2023-10-09 13:36:31,118][86121] Updated weights for policy 0, policy_version 35110 (0.0008) +[2023-10-09 13:36:31,330][86122] Updated weights for policy 1, policy_version 35280 (0.0009) +[2023-10-09 13:36:31,479][86121] Updated weights for policy 0, policy_version 35120 (0.0007) +[2023-10-09 13:36:31,695][86122] Updated weights for policy 1, policy_version 35290 (0.0009) +[2023-10-09 13:36:31,840][86121] Updated weights for policy 0, policy_version 35130 (0.0007) +[2023-10-09 13:36:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 72122368. Throughput: 0: 1810.7, 1: 1814.3. Samples: 18034412. Policy #0 lag: (min: 12.0, avg: 12.8, max: 30.0) +[2023-10-09 13:36:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 13:36:35,310][86122] Updated weights for policy 1, policy_version 35300 (0.0011) +[2023-10-09 13:36:35,578][86121] Updated weights for policy 0, policy_version 35140 (0.0008) +[2023-10-09 13:36:35,673][86122] Updated weights for policy 1, policy_version 35310 (0.0010) +[2023-10-09 13:36:35,969][86121] Updated weights for policy 0, policy_version 35150 (0.0007) +[2023-10-09 13:36:36,030][86122] Updated weights for policy 1, policy_version 35320 (0.0009) +[2023-10-09 13:36:36,323][86121] Updated weights for policy 0, policy_version 35160 (0.0009) +[2023-10-09 13:36:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 72187904. Throughput: 0: 1804.8, 1: 1819.0. Samples: 18056626. Policy #0 lag: (min: 12.0, avg: 12.8, max: 30.0) +[2023-10-09 13:36:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 13:36:39,722][86122] Updated weights for policy 1, policy_version 35330 (0.0007) +[2023-10-09 13:36:40,035][86121] Updated weights for policy 0, policy_version 35170 (0.0010) +[2023-10-09 13:36:40,091][86122] Updated weights for policy 1, policy_version 35340 (0.0008) +[2023-10-09 13:36:40,401][86121] Updated weights for policy 0, policy_version 35180 (0.0008) +[2023-10-09 13:36:40,462][86122] Updated weights for policy 1, policy_version 35350 (0.0007) +[2023-10-09 13:36:40,755][86121] Updated weights for policy 0, policy_version 35190 (0.0008) +[2023-10-09 13:36:40,821][86122] Updated weights for policy 1, policy_version 35360 (0.0010) +[2023-10-09 13:36:41,118][86121] Updated weights for policy 0, policy_version 35200 (0.0007) +[2023-10-09 13:36:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 72253440. Throughput: 0: 1812.4, 1: 1815.7. Samples: 18067050. Policy #0 lag: (min: 12.0, avg: 12.8, max: 30.0) +[2023-10-09 13:36:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 13:36:44,577][86122] Updated weights for policy 1, policy_version 35370 (0.0008) +[2023-10-09 13:36:44,783][86121] Updated weights for policy 0, policy_version 35210 (0.0008) +[2023-10-09 13:36:44,936][86122] Updated weights for policy 1, policy_version 35380 (0.0010) +[2023-10-09 13:36:45,147][86121] Updated weights for policy 0, policy_version 35220 (0.0007) +[2023-10-09 13:36:45,303][86122] Updated weights for policy 1, policy_version 35390 (0.0008) +[2023-10-09 13:36:45,503][86121] Updated weights for policy 0, policy_version 35230 (0.0011) +[2023-10-09 13:36:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 72318976. Throughput: 0: 1807.2, 1: 1823.2. Samples: 18089464. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:36:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 13:36:48,978][86122] Updated weights for policy 1, policy_version 35400 (0.0009) +[2023-10-09 13:36:49,338][86121] Updated weights for policy 0, policy_version 35240 (0.0008) +[2023-10-09 13:36:49,341][86122] Updated weights for policy 1, policy_version 35410 (0.0008) +[2023-10-09 13:36:49,698][86122] Updated weights for policy 1, policy_version 35420 (0.0007) +[2023-10-09 13:36:49,701][86121] Updated weights for policy 0, policy_version 35250 (0.0007) +[2023-10-09 13:36:50,070][86121] Updated weights for policy 0, policy_version 35260 (0.0009) +[2023-10-09 13:36:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 72384512. Throughput: 0: 1805.2, 1: 1826.4. Samples: 18112132. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:36:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.960')] +[2023-10-09 13:36:53,469][86122] Updated weights for policy 1, policy_version 35430 (0.0009) +[2023-10-09 13:36:53,831][86122] Updated weights for policy 1, policy_version 35440 (0.0009) +[2023-10-09 13:36:53,866][86121] Updated weights for policy 0, policy_version 35270 (0.0009) +[2023-10-09 13:36:54,190][86122] Updated weights for policy 1, policy_version 35450 (0.0007) +[2023-10-09 13:36:54,237][86121] Updated weights for policy 0, policy_version 35280 (0.0008) +[2023-10-09 13:36:54,601][86121] Updated weights for policy 0, policy_version 35290 (0.0008) +[2023-10-09 13:36:57,941][86122] Updated weights for policy 1, policy_version 35460 (0.0008) +[2023-10-09 13:36:58,205][86121] Updated weights for policy 0, policy_version 35300 (0.0008) +[2023-10-09 13:36:58,294][86122] Updated weights for policy 1, policy_version 35470 (0.0008) +[2023-10-09 13:36:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 72450048. Throughput: 0: 1804.9, 1: 1826.0. Samples: 18121978. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:36:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 13:36:58,568][86121] Updated weights for policy 0, policy_version 35310 (0.0007) +[2023-10-09 13:36:58,653][86122] Updated weights for policy 1, policy_version 35480 (0.0009) +[2023-10-09 13:36:58,934][86121] Updated weights for policy 0, policy_version 35320 (0.0008) +[2023-10-09 13:37:02,381][86122] Updated weights for policy 1, policy_version 35490 (0.0008) +[2023-10-09 13:37:02,703][86121] Updated weights for policy 0, policy_version 35330 (0.0010) +[2023-10-09 13:37:02,764][86122] Updated weights for policy 1, policy_version 35500 (0.0008) +[2023-10-09 13:37:03,061][86121] Updated weights for policy 0, policy_version 35340 (0.0007) +[2023-10-09 13:37:03,128][86122] Updated weights for policy 1, policy_version 35510 (0.0008) +[2023-10-09 13:37:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 72515584. Throughput: 0: 1808.2, 1: 1829.6. Samples: 18144932. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 13:37:03,436][86121] Updated weights for policy 0, policy_version 35350 (0.0009) +[2023-10-09 13:37:03,497][86122] Updated weights for policy 1, policy_version 35520 (0.0008) +[2023-10-09 13:37:03,793][86121] Updated weights for policy 0, policy_version 35360 (0.0008) +[2023-10-09 13:37:07,283][86122] Updated weights for policy 1, policy_version 35530 (0.0009) +[2023-10-09 13:37:07,644][86122] Updated weights for policy 1, policy_version 35540 (0.0008) +[2023-10-09 13:37:07,677][86121] Updated weights for policy 0, policy_version 35370 (0.0007) +[2023-10-09 13:37:08,008][86122] Updated weights for policy 1, policy_version 35550 (0.0008) +[2023-10-09 13:37:08,042][86121] Updated weights for policy 0, policy_version 35380 (0.0009) +[2023-10-09 13:37:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 72613888. Throughput: 0: 1815.6, 1: 1818.5. Samples: 18165344. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 13:37:08,405][86121] Updated weights for policy 0, policy_version 35390 (0.0007) +[2023-10-09 13:37:11,497][86122] Updated weights for policy 1, policy_version 35560 (0.0007) +[2023-10-09 13:37:11,863][86122] Updated weights for policy 1, policy_version 35570 (0.0007) +[2023-10-09 13:37:12,069][86121] Updated weights for policy 0, policy_version 35400 (0.0008) +[2023-10-09 13:37:12,216][86122] Updated weights for policy 1, policy_version 35580 (0.0007) +[2023-10-09 13:37:12,429][86121] Updated weights for policy 0, policy_version 35410 (0.0009) +[2023-10-09 13:37:12,807][86121] Updated weights for policy 0, policy_version 35420 (0.0011) +[2023-10-09 13:37:13,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72712192. Throughput: 0: 1798.7, 1: 1827.5. Samples: 18177390. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 13:37:15,795][86122] Updated weights for policy 1, policy_version 35590 (0.0008) +[2023-10-09 13:37:16,157][86122] Updated weights for policy 1, policy_version 35600 (0.0010) +[2023-10-09 13:37:16,416][86121] Updated weights for policy 0, policy_version 35430 (0.0010) +[2023-10-09 13:37:16,508][86122] Updated weights for policy 1, policy_version 35610 (0.0008) +[2023-10-09 13:37:16,781][86121] Updated weights for policy 0, policy_version 35440 (0.0007) +[2023-10-09 13:37:17,142][86121] Updated weights for policy 0, policy_version 35450 (0.0009) +[2023-10-09 13:37:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 72777728. Throughput: 0: 1810.6, 1: 1824.7. Samples: 18197998. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.930')] +[2023-10-09 13:37:20,301][86122] Updated weights for policy 1, policy_version 35620 (0.0009) +[2023-10-09 13:37:20,662][86122] Updated weights for policy 1, policy_version 35630 (0.0010) +[2023-10-09 13:37:21,008][86121] Updated weights for policy 0, policy_version 35460 (0.0011) +[2023-10-09 13:37:21,026][86122] Updated weights for policy 1, policy_version 35640 (0.0009) +[2023-10-09 13:37:21,403][86121] Updated weights for policy 0, policy_version 35470 (0.0009) +[2023-10-09 13:37:21,772][86121] Updated weights for policy 0, policy_version 35480 (0.0009) +[2023-10-09 13:37:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 72843264. Throughput: 0: 1801.8, 1: 1826.4. Samples: 18219898. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.930')] +[2023-10-09 13:37:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000035648_36503552.pth... +[2023-10-09 13:37:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000035488_36339712.pth... +[2023-10-09 13:37:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000033792_34603008.pth +[2023-10-09 13:37:23,447][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000033952_34766848.pth +[2023-10-09 13:37:24,687][86122] Updated weights for policy 1, policy_version 35650 (0.0008) +[2023-10-09 13:37:25,046][86122] Updated weights for policy 1, policy_version 35660 (0.0009) +[2023-10-09 13:37:25,366][86121] Updated weights for policy 0, policy_version 35490 (0.0008) +[2023-10-09 13:37:25,420][86122] Updated weights for policy 1, policy_version 35670 (0.0008) +[2023-10-09 13:37:25,734][86121] Updated weights for policy 0, policy_version 35500 (0.0009) +[2023-10-09 13:37:25,771][86122] Updated weights for policy 1, policy_version 35680 (0.0008) +[2023-10-09 13:37:26,092][86121] Updated weights for policy 0, policy_version 35510 (0.0010) +[2023-10-09 13:37:26,454][86121] Updated weights for policy 0, policy_version 35520 (0.0010) +[2023-10-09 13:37:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 72908800. Throughput: 0: 1813.8, 1: 1826.1. Samples: 18230848. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.930')] +[2023-10-09 13:37:29,353][86122] Updated weights for policy 1, policy_version 35690 (0.0007) +[2023-10-09 13:37:29,709][86122] Updated weights for policy 1, policy_version 35700 (0.0012) +[2023-10-09 13:37:30,075][86122] Updated weights for policy 1, policy_version 35710 (0.0009) +[2023-10-09 13:37:30,166][86121] Updated weights for policy 0, policy_version 35530 (0.0009) +[2023-10-09 13:37:30,533][86121] Updated weights for policy 0, policy_version 35540 (0.0008) +[2023-10-09 13:37:30,894][86121] Updated weights for policy 0, policy_version 35550 (0.0009) +[2023-10-09 13:37:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 72974336. Throughput: 0: 1801.1, 1: 1830.2. Samples: 18252872. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.930')] +[2023-10-09 13:37:33,812][86122] Updated weights for policy 1, policy_version 35720 (0.0010) +[2023-10-09 13:37:34,178][86122] Updated weights for policy 1, policy_version 35730 (0.0008) +[2023-10-09 13:37:34,536][86122] Updated weights for policy 1, policy_version 35740 (0.0007) +[2023-10-09 13:37:34,590][86121] Updated weights for policy 0, policy_version 35560 (0.0007) +[2023-10-09 13:37:34,959][86121] Updated weights for policy 0, policy_version 35570 (0.0007) +[2023-10-09 13:37:35,320][86121] Updated weights for policy 0, policy_version 35580 (0.0008) +[2023-10-09 13:37:38,188][86122] Updated weights for policy 1, policy_version 35750 (0.0009) +[2023-10-09 13:37:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73039872. Throughput: 0: 1802.3, 1: 1831.7. Samples: 18275666. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:37:38,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.920')] +[2023-10-09 13:37:38,550][86122] Updated weights for policy 1, policy_version 35760 (0.0009) +[2023-10-09 13:37:38,910][86122] Updated weights for policy 1, policy_version 35770 (0.0007) +[2023-10-09 13:37:39,086][86121] Updated weights for policy 0, policy_version 35590 (0.0007) +[2023-10-09 13:37:39,448][86121] Updated weights for policy 0, policy_version 35600 (0.0007) +[2023-10-09 13:37:39,808][86121] Updated weights for policy 0, policy_version 35610 (0.0007) +[2023-10-09 13:37:42,781][86122] Updated weights for policy 1, policy_version 35780 (0.0009) +[2023-10-09 13:37:43,152][86122] Updated weights for policy 1, policy_version 35790 (0.0011) +[2023-10-09 13:37:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73105408. Throughput: 0: 1802.0, 1: 1833.9. Samples: 18285592. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:37:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.920')] +[2023-10-09 13:37:43,515][86122] Updated weights for policy 1, policy_version 35800 (0.0008) +[2023-10-09 13:37:43,554][86121] Updated weights for policy 0, policy_version 35620 (0.0009) +[2023-10-09 13:37:43,929][86121] Updated weights for policy 0, policy_version 35630 (0.0008) +[2023-10-09 13:37:44,297][86121] Updated weights for policy 0, policy_version 35640 (0.0007) +[2023-10-09 13:37:47,358][86122] Updated weights for policy 1, policy_version 35810 (0.0008) +[2023-10-09 13:37:47,768][86122] Updated weights for policy 1, policy_version 35820 (0.0009) +[2023-10-09 13:37:48,122][86122] Updated weights for policy 1, policy_version 35830 (0.0007) +[2023-10-09 13:37:48,140][86121] Updated weights for policy 0, policy_version 35650 (0.0009) +[2023-10-09 13:37:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73170944. Throughput: 0: 1800.6, 1: 1824.6. Samples: 18308066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:37:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.900')] +[2023-10-09 13:37:48,479][86122] Updated weights for policy 1, policy_version 35840 (0.0009) +[2023-10-09 13:37:48,508][86121] Updated weights for policy 0, policy_version 35660 (0.0009) +[2023-10-09 13:37:48,868][86121] Updated weights for policy 0, policy_version 35670 (0.0007) +[2023-10-09 13:37:49,230][86121] Updated weights for policy 0, policy_version 35680 (0.0010) +[2023-10-09 13:37:52,098][86122] Updated weights for policy 1, policy_version 35850 (0.0009) +[2023-10-09 13:37:52,468][86122] Updated weights for policy 1, policy_version 35860 (0.0007) +[2023-10-09 13:37:52,820][86122] Updated weights for policy 1, policy_version 35870 (0.0009) +[2023-10-09 13:37:52,988][86121] Updated weights for policy 0, policy_version 35690 (0.0008) +[2023-10-09 13:37:53,350][86121] Updated weights for policy 0, policy_version 35700 (0.0009) +[2023-10-09 13:37:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73269248. Throughput: 0: 1813.6, 1: 1824.4. Samples: 18329054. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:37:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.880')] +[2023-10-09 13:37:53,711][86121] Updated weights for policy 0, policy_version 35710 (0.0011) +[2023-10-09 13:37:56,407][86122] Updated weights for policy 1, policy_version 35880 (0.0009) +[2023-10-09 13:37:56,781][86122] Updated weights for policy 1, policy_version 35890 (0.0008) +[2023-10-09 13:37:57,139][86122] Updated weights for policy 1, policy_version 35900 (0.0009) +[2023-10-09 13:37:57,405][86121] Updated weights for policy 0, policy_version 35720 (0.0009) +[2023-10-09 13:37:57,773][86121] Updated weights for policy 0, policy_version 35730 (0.0011) +[2023-10-09 13:37:58,148][86121] Updated weights for policy 0, policy_version 35740 (0.0009) +[2023-10-09 13:37:58,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 73367552. Throughput: 0: 1804.3, 1: 1823.8. Samples: 18340654. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:37:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.880')] +[2023-10-09 13:38:00,872][86122] Updated weights for policy 1, policy_version 35910 (0.0008) +[2023-10-09 13:38:01,236][86122] Updated weights for policy 1, policy_version 35920 (0.0010) +[2023-10-09 13:38:01,596][86122] Updated weights for policy 1, policy_version 35930 (0.0009) +[2023-10-09 13:38:01,778][86121] Updated weights for policy 0, policy_version 35750 (0.0009) +[2023-10-09 13:38:02,147][86121] Updated weights for policy 0, policy_version 35760 (0.0008) +[2023-10-09 13:38:02,507][86121] Updated weights for policy 0, policy_version 35770 (0.0007) +[2023-10-09 13:38:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 73433088. Throughput: 0: 1813.7, 1: 1820.7. Samples: 18361548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:38:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.860')] +[2023-10-09 13:38:05,158][86122] Updated weights for policy 1, policy_version 35940 (0.0008) +[2023-10-09 13:38:05,515][86122] Updated weights for policy 1, policy_version 35950 (0.0008) +[2023-10-09 13:38:05,883][86122] Updated weights for policy 1, policy_version 35960 (0.0010) +[2023-10-09 13:38:06,216][86121] Updated weights for policy 0, policy_version 35780 (0.0007) +[2023-10-09 13:38:06,615][86121] Updated weights for policy 0, policy_version 35790 (0.0008) +[2023-10-09 13:38:06,978][86121] Updated weights for policy 0, policy_version 35800 (0.0008) +[2023-10-09 13:38:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73498624. Throughput: 0: 1804.9, 1: 1831.0. Samples: 18383514. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:38:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.860')] +[2023-10-09 13:38:09,499][86122] Updated weights for policy 1, policy_version 35970 (0.0007) +[2023-10-09 13:38:09,870][86122] Updated weights for policy 1, policy_version 35980 (0.0008) +[2023-10-09 13:38:10,221][86122] Updated weights for policy 1, policy_version 35990 (0.0008) +[2023-10-09 13:38:10,582][86122] Updated weights for policy 1, policy_version 36000 (0.0009) +[2023-10-09 13:38:10,657][86121] Updated weights for policy 0, policy_version 35810 (0.0009) +[2023-10-09 13:38:11,023][86121] Updated weights for policy 0, policy_version 35820 (0.0008) +[2023-10-09 13:38:11,385][86121] Updated weights for policy 0, policy_version 35830 (0.0007) +[2023-10-09 13:38:11,751][86121] Updated weights for policy 0, policy_version 35840 (0.0008) +[2023-10-09 13:38:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 73564160. Throughput: 0: 1812.9, 1: 1825.3. Samples: 18394562. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:38:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.860')] +[2023-10-09 13:38:14,275][86122] Updated weights for policy 1, policy_version 36010 (0.0009) +[2023-10-09 13:38:14,635][86122] Updated weights for policy 1, policy_version 36020 (0.0007) +[2023-10-09 13:38:15,002][86122] Updated weights for policy 1, policy_version 36030 (0.0007) +[2023-10-09 13:38:15,374][86121] Updated weights for policy 0, policy_version 35850 (0.0007) +[2023-10-09 13:38:15,738][86121] Updated weights for policy 0, policy_version 35860 (0.0008) +[2023-10-09 13:38:16,105][86121] Updated weights for policy 0, policy_version 35870 (0.0010) +[2023-10-09 13:38:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 73629696. Throughput: 0: 1812.2, 1: 1826.9. Samples: 18416634. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:38:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.860')] +[2023-10-09 13:38:18,826][86122] Updated weights for policy 1, policy_version 36040 (0.0008) +[2023-10-09 13:38:19,190][86122] Updated weights for policy 1, policy_version 36050 (0.0010) +[2023-10-09 13:38:19,548][86122] Updated weights for policy 1, policy_version 36060 (0.0010) +[2023-10-09 13:38:19,807][86121] Updated weights for policy 0, policy_version 35880 (0.0008) +[2023-10-09 13:38:20,166][86121] Updated weights for policy 0, policy_version 35890 (0.0008) +[2023-10-09 13:38:20,537][86121] Updated weights for policy 0, policy_version 35900 (0.0007) +[2023-10-09 13:38:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73695232. Throughput: 0: 1812.0, 1: 1820.4. Samples: 18439124. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:38:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.850')] +[2023-10-09 13:38:23,409][86122] Updated weights for policy 1, policy_version 36070 (0.0007) +[2023-10-09 13:38:23,775][86122] Updated weights for policy 1, policy_version 36080 (0.0008) +[2023-10-09 13:38:24,144][86122] Updated weights for policy 1, policy_version 36090 (0.0009) +[2023-10-09 13:38:24,163][86121] Updated weights for policy 0, policy_version 35910 (0.0008) +[2023-10-09 13:38:24,535][86121] Updated weights for policy 0, policy_version 35920 (0.0008) +[2023-10-09 13:38:24,905][86121] Updated weights for policy 0, policy_version 35930 (0.0008) +[2023-10-09 13:38:27,739][86122] Updated weights for policy 1, policy_version 36100 (0.0007) +[2023-10-09 13:38:28,106][86122] Updated weights for policy 1, policy_version 36110 (0.0009) +[2023-10-09 13:38:28,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73760768. Throughput: 0: 1808.4, 1: 1817.5. Samples: 18448756. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:38:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.850')] +[2023-10-09 13:38:28,465][86122] Updated weights for policy 1, policy_version 36120 (0.0008) +[2023-10-09 13:38:28,811][86121] Updated weights for policy 0, policy_version 35940 (0.0009) +[2023-10-09 13:38:29,179][86121] Updated weights for policy 0, policy_version 35950 (0.0011) +[2023-10-09 13:38:29,534][86121] Updated weights for policy 0, policy_version 35960 (0.0008) +[2023-10-09 13:38:32,366][86122] Updated weights for policy 1, policy_version 36130 (0.0009) +[2023-10-09 13:38:32,733][86122] Updated weights for policy 1, policy_version 36140 (0.0008) +[2023-10-09 13:38:33,099][86122] Updated weights for policy 1, policy_version 36150 (0.0009) +[2023-10-09 13:38:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 73826304. Throughput: 0: 1805.8, 1: 1815.2. Samples: 18471014. Policy #0 lag: (min: 31.0, avg: 31.1, max: 38.0) +[2023-10-09 13:38:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.840')] +[2023-10-09 13:38:33,420][86121] Updated weights for policy 0, policy_version 35970 (0.0008) +[2023-10-09 13:38:33,459][86122] Updated weights for policy 1, policy_version 36160 (0.0008) +[2023-10-09 13:38:33,779][86121] Updated weights for policy 0, policy_version 35980 (0.0010) +[2023-10-09 13:38:34,143][86121] Updated weights for policy 0, policy_version 35990 (0.0010) +[2023-10-09 13:38:34,506][86121] Updated weights for policy 0, policy_version 36000 (0.0010) +[2023-10-09 13:38:37,074][86122] Updated weights for policy 1, policy_version 36170 (0.0008) +[2023-10-09 13:38:37,435][86122] Updated weights for policy 1, policy_version 36180 (0.0007) +[2023-10-09 13:38:37,793][86122] Updated weights for policy 1, policy_version 36190 (0.0008) +[2023-10-09 13:38:38,271][86121] Updated weights for policy 0, policy_version 36010 (0.0008) +[2023-10-09 13:38:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73924608. Throughput: 0: 1816.8, 1: 1817.1. Samples: 18492584. Policy #0 lag: (min: 7.0, avg: 29.0, max: 32.0) +[2023-10-09 13:38:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.840')] +[2023-10-09 13:38:38,639][86121] Updated weights for policy 0, policy_version 36020 (0.0010) +[2023-10-09 13:38:39,006][86121] Updated weights for policy 0, policy_version 36030 (0.0009) +[2023-10-09 13:38:41,497][86122] Updated weights for policy 1, policy_version 36200 (0.0008) +[2023-10-09 13:38:41,858][86122] Updated weights for policy 1, policy_version 36210 (0.0007) +[2023-10-09 13:38:42,216][86122] Updated weights for policy 1, policy_version 36220 (0.0008) +[2023-10-09 13:38:42,755][86121] Updated weights for policy 0, policy_version 36040 (0.0008) +[2023-10-09 13:38:43,120][86121] Updated weights for policy 0, policy_version 36050 (0.0007) +[2023-10-09 13:38:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 73990144. Throughput: 0: 1809.7, 1: 1821.6. Samples: 18504062. Policy #0 lag: (min: 7.0, avg: 29.0, max: 32.0) +[2023-10-09 13:38:43,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.840')] +[2023-10-09 13:38:43,474][86121] Updated weights for policy 0, policy_version 36060 (0.0007) +[2023-10-09 13:38:45,952][86122] Updated weights for policy 1, policy_version 36230 (0.0009) +[2023-10-09 13:38:46,321][86122] Updated weights for policy 1, policy_version 36240 (0.0007) +[2023-10-09 13:38:46,679][86122] Updated weights for policy 1, policy_version 36250 (0.0008) +[2023-10-09 13:38:47,154][86121] Updated weights for policy 0, policy_version 36070 (0.0008) +[2023-10-09 13:38:47,517][86121] Updated weights for policy 0, policy_version 36080 (0.0009) +[2023-10-09 13:38:47,880][86121] Updated weights for policy 0, policy_version 36090 (0.0008) +[2023-10-09 13:38:48,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 74088448. Throughput: 0: 1818.0, 1: 1823.4. Samples: 18525412. Policy #0 lag: (min: 7.0, avg: 29.0, max: 32.0) +[2023-10-09 13:38:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.860')] +[2023-10-09 13:38:50,392][86122] Updated weights for policy 1, policy_version 36260 (0.0008) +[2023-10-09 13:38:50,757][86122] Updated weights for policy 1, policy_version 36270 (0.0008) +[2023-10-09 13:38:51,121][86122] Updated weights for policy 1, policy_version 36280 (0.0007) +[2023-10-09 13:38:51,630][86121] Updated weights for policy 0, policy_version 36100 (0.0008) +[2023-10-09 13:38:51,998][86121] Updated weights for policy 0, policy_version 36110 (0.0008) +[2023-10-09 13:38:52,370][86121] Updated weights for policy 0, policy_version 36120 (0.0008) +[2023-10-09 13:38:53,398][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 74153984. Throughput: 0: 1809.8, 1: 1813.4. Samples: 18546556. Policy #0 lag: (min: 7.0, avg: 29.0, max: 32.0) +[2023-10-09 13:38:53,399][85186] Avg episode reward: [(0, '9.940'), (1, '9.860')] +[2023-10-09 13:38:54,648][86122] Updated weights for policy 1, policy_version 36290 (0.0007) +[2023-10-09 13:38:55,014][86122] Updated weights for policy 1, policy_version 36300 (0.0007) +[2023-10-09 13:38:55,376][86122] Updated weights for policy 1, policy_version 36310 (0.0008) +[2023-10-09 13:38:55,727][86122] Updated weights for policy 1, policy_version 36320 (0.0008) +[2023-10-09 13:38:56,136][86121] Updated weights for policy 0, policy_version 36130 (0.0007) +[2023-10-09 13:38:56,495][86121] Updated weights for policy 0, policy_version 36140 (0.0008) +[2023-10-09 13:38:56,860][86121] Updated weights for policy 0, policy_version 36150 (0.0007) +[2023-10-09 13:38:57,230][86121] Updated weights for policy 0, policy_version 36160 (0.0010) +[2023-10-09 13:38:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 74219520. Throughput: 0: 1815.3, 1: 1819.8. Samples: 18558142. Policy #0 lag: (min: 7.0, avg: 29.0, max: 32.0) +[2023-10-09 13:38:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.860')] +[2023-10-09 13:38:59,487][86122] Updated weights for policy 1, policy_version 36330 (0.0008) +[2023-10-09 13:38:59,847][86122] Updated weights for policy 1, policy_version 36340 (0.0007) +[2023-10-09 13:39:00,214][86122] Updated weights for policy 1, policy_version 36350 (0.0008) +[2023-10-09 13:39:00,904][86121] Updated weights for policy 0, policy_version 36170 (0.0009) +[2023-10-09 13:39:01,268][86121] Updated weights for policy 0, policy_version 36180 (0.0007) +[2023-10-09 13:39:01,636][86121] Updated weights for policy 0, policy_version 36190 (0.0007) +[2023-10-09 13:39:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74285056. Throughput: 0: 1798.4, 1: 1820.7. Samples: 18579494. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:39:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.860')] +[2023-10-09 13:39:03,807][86122] Updated weights for policy 1, policy_version 36360 (0.0009) +[2023-10-09 13:39:04,168][86122] Updated weights for policy 1, policy_version 36370 (0.0009) +[2023-10-09 13:39:04,520][86122] Updated weights for policy 1, policy_version 36380 (0.0008) +[2023-10-09 13:39:05,413][86121] Updated weights for policy 0, policy_version 36200 (0.0008) +[2023-10-09 13:39:05,781][86121] Updated weights for policy 0, policy_version 36210 (0.0008) +[2023-10-09 13:39:06,153][86121] Updated weights for policy 0, policy_version 36220 (0.0009) +[2023-10-09 13:39:08,199][86122] Updated weights for policy 1, policy_version 36390 (0.0008) +[2023-10-09 13:39:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74350592. Throughput: 0: 1792.8, 1: 1828.6. Samples: 18602088. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:39:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.870')] +[2023-10-09 13:39:08,562][86122] Updated weights for policy 1, policy_version 36400 (0.0008) +[2023-10-09 13:39:08,930][86122] Updated weights for policy 1, policy_version 36410 (0.0009) +[2023-10-09 13:39:09,894][86121] Updated weights for policy 0, policy_version 36230 (0.0007) +[2023-10-09 13:39:10,266][86121] Updated weights for policy 0, policy_version 36240 (0.0009) +[2023-10-09 13:39:10,623][86121] Updated weights for policy 0, policy_version 36250 (0.0007) +[2023-10-09 13:39:12,578][86122] Updated weights for policy 1, policy_version 36420 (0.0007) +[2023-10-09 13:39:12,942][86122] Updated weights for policy 1, policy_version 36430 (0.0007) +[2023-10-09 13:39:13,306][86122] Updated weights for policy 1, policy_version 36440 (0.0009) +[2023-10-09 13:39:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74416128. Throughput: 0: 1799.0, 1: 1833.3. Samples: 18612212. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:39:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.870')] +[2023-10-09 13:39:14,226][86121] Updated weights for policy 0, policy_version 36260 (0.0008) +[2023-10-09 13:39:14,602][86121] Updated weights for policy 0, policy_version 36270 (0.0009) +[2023-10-09 13:39:14,965][86121] Updated weights for policy 0, policy_version 36280 (0.0007) +[2023-10-09 13:39:17,047][86122] Updated weights for policy 1, policy_version 36450 (0.0008) +[2023-10-09 13:39:17,465][86122] Updated weights for policy 1, policy_version 36460 (0.0007) +[2023-10-09 13:39:17,822][86122] Updated weights for policy 1, policy_version 36470 (0.0007) +[2023-10-09 13:39:18,183][86122] Updated weights for policy 1, policy_version 36480 (0.0007) +[2023-10-09 13:39:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74514432. Throughput: 0: 1810.4, 1: 1838.2. Samples: 18635202. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:39:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.870')] +[2023-10-09 13:39:18,607][86121] Updated weights for policy 0, policy_version 36290 (0.0007) +[2023-10-09 13:39:18,964][86121] Updated weights for policy 0, policy_version 36300 (0.0007) +[2023-10-09 13:39:19,329][86121] Updated weights for policy 0, policy_version 36310 (0.0007) +[2023-10-09 13:39:19,702][86121] Updated weights for policy 0, policy_version 36320 (0.0009) +[2023-10-09 13:39:21,656][86122] Updated weights for policy 1, policy_version 36490 (0.0008) +[2023-10-09 13:39:22,015][86122] Updated weights for policy 1, policy_version 36500 (0.0008) +[2023-10-09 13:39:22,380][86122] Updated weights for policy 1, policy_version 36510 (0.0009) +[2023-10-09 13:39:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74579968. Throughput: 0: 1809.3, 1: 1835.6. Samples: 18656608. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:39:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.870')] +[2023-10-09 13:39:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000036512_37388288.pth... +[2023-10-09 13:39:23,449][86121] Updated weights for policy 0, policy_version 36330 (0.0007) +[2023-10-09 13:39:23,449][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000034816_35651584.pth +[2023-10-09 13:39:23,809][86121] Updated weights for policy 0, policy_version 36340 (0.0008) +[2023-10-09 13:39:24,185][86121] Updated weights for policy 0, policy_version 36350 (0.0007) +[2023-10-09 13:39:24,250][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000036352_37224448.pth... +[2023-10-09 13:39:24,290][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000034656_35487744.pth +[2023-10-09 13:39:26,172][86122] Updated weights for policy 1, policy_version 36520 (0.0009) +[2023-10-09 13:39:26,536][86122] Updated weights for policy 1, policy_version 36530 (0.0007) +[2023-10-09 13:39:26,901][86122] Updated weights for policy 1, policy_version 36540 (0.0009) +[2023-10-09 13:39:27,633][86121] Updated weights for policy 0, policy_version 36360 (0.0008) +[2023-10-09 13:39:28,002][86121] Updated weights for policy 0, policy_version 36370 (0.0007) +[2023-10-09 13:39:28,364][86121] Updated weights for policy 0, policy_version 36380 (0.0008) +[2023-10-09 13:39:28,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74645504. Throughput: 0: 1810.5, 1: 1834.2. Samples: 18668072. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 13:39:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.870')] +[2023-10-09 13:39:30,662][86122] Updated weights for policy 1, policy_version 36550 (0.0009) +[2023-10-09 13:39:31,015][86122] Updated weights for policy 1, policy_version 36560 (0.0010) +[2023-10-09 13:39:31,381][86122] Updated weights for policy 1, policy_version 36570 (0.0012) +[2023-10-09 13:39:32,234][86121] Updated weights for policy 0, policy_version 36390 (0.0009) +[2023-10-09 13:39:32,605][86121] Updated weights for policy 0, policy_version 36400 (0.0010) +[2023-10-09 13:39:32,965][86121] Updated weights for policy 0, policy_version 36410 (0.0008) +[2023-10-09 13:39:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 74743808. Throughput: 0: 1812.0, 1: 1828.8. Samples: 18689250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:39:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.880')] +[2023-10-09 13:39:35,096][86122] Updated weights for policy 1, policy_version 36580 (0.0010) +[2023-10-09 13:39:35,464][86122] Updated weights for policy 1, policy_version 36590 (0.0009) +[2023-10-09 13:39:35,830][86122] Updated weights for policy 1, policy_version 36600 (0.0008) +[2023-10-09 13:39:36,663][86121] Updated weights for policy 0, policy_version 36420 (0.0007) +[2023-10-09 13:39:37,052][86121] Updated weights for policy 0, policy_version 36430 (0.0008) +[2023-10-09 13:39:37,413][86121] Updated weights for policy 0, policy_version 36440 (0.0008) +[2023-10-09 13:39:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 74809344. Throughput: 0: 1814.1, 1: 1834.5. Samples: 18710740. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:39:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.880')] +[2023-10-09 13:39:39,397][86122] Updated weights for policy 1, policy_version 36610 (0.0008) +[2023-10-09 13:39:39,759][86122] Updated weights for policy 1, policy_version 36620 (0.0008) +[2023-10-09 13:39:40,120][86122] Updated weights for policy 1, policy_version 36630 (0.0008) +[2023-10-09 13:39:40,476][86122] Updated weights for policy 1, policy_version 36640 (0.0008) +[2023-10-09 13:39:41,042][86121] Updated weights for policy 0, policy_version 36450 (0.0009) +[2023-10-09 13:39:41,401][86121] Updated weights for policy 0, policy_version 36460 (0.0008) +[2023-10-09 13:39:41,769][86121] Updated weights for policy 0, policy_version 36470 (0.0007) +[2023-10-09 13:39:42,130][86121] Updated weights for policy 0, policy_version 36480 (0.0010) +[2023-10-09 13:39:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 74874880. Throughput: 0: 1814.2, 1: 1830.0. Samples: 18722130. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:39:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.900')] +[2023-10-09 13:39:44,139][86122] Updated weights for policy 1, policy_version 36650 (0.0008) +[2023-10-09 13:39:44,495][86122] Updated weights for policy 1, policy_version 36660 (0.0009) +[2023-10-09 13:39:44,860][86122] Updated weights for policy 1, policy_version 36670 (0.0011) +[2023-10-09 13:39:45,779][86121] Updated weights for policy 0, policy_version 36490 (0.0008) +[2023-10-09 13:39:46,144][86121] Updated weights for policy 0, policy_version 36500 (0.0007) +[2023-10-09 13:39:46,516][86121] Updated weights for policy 0, policy_version 36510 (0.0007) +[2023-10-09 13:39:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 74940416. Throughput: 0: 1823.8, 1: 1830.2. Samples: 18743926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:39:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.910')] +[2023-10-09 13:39:48,553][86122] Updated weights for policy 1, policy_version 36680 (0.0008) +[2023-10-09 13:39:48,921][86122] Updated weights for policy 1, policy_version 36690 (0.0008) +[2023-10-09 13:39:49,284][86122] Updated weights for policy 1, policy_version 36700 (0.0007) +[2023-10-09 13:39:50,017][86121] Updated weights for policy 0, policy_version 36520 (0.0007) +[2023-10-09 13:39:50,376][86121] Updated weights for policy 0, policy_version 36530 (0.0010) +[2023-10-09 13:39:50,744][86121] Updated weights for policy 0, policy_version 36540 (0.0008) +[2023-10-09 13:39:52,769][86122] Updated weights for policy 1, policy_version 36710 (0.0008) +[2023-10-09 13:39:53,135][86122] Updated weights for policy 1, policy_version 36720 (0.0009) +[2023-10-09 13:39:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75005952. Throughput: 0: 1831.2, 1: 1827.8. Samples: 18766742. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:39:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.890')] +[2023-10-09 13:39:53,494][86122] Updated weights for policy 1, policy_version 36730 (0.0008) +[2023-10-09 13:39:54,633][86121] Updated weights for policy 0, policy_version 36550 (0.0008) +[2023-10-09 13:39:54,999][86121] Updated weights for policy 0, policy_version 36560 (0.0007) +[2023-10-09 13:39:55,374][86121] Updated weights for policy 0, policy_version 36570 (0.0008) +[2023-10-09 13:39:57,024][86122] Updated weights for policy 1, policy_version 36740 (0.0009) +[2023-10-09 13:39:57,391][86122] Updated weights for policy 1, policy_version 36750 (0.0010) +[2023-10-09 13:39:57,757][86122] Updated weights for policy 1, policy_version 36760 (0.0010) +[2023-10-09 13:39:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75104256. Throughput: 0: 1828.7, 1: 1837.7. Samples: 18777200. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) +[2023-10-09 13:39:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.910')] +[2023-10-09 13:39:59,153][86121] Updated weights for policy 0, policy_version 36580 (0.0009) +[2023-10-09 13:39:59,530][86121] Updated weights for policy 0, policy_version 36590 (0.0010) +[2023-10-09 13:39:59,904][86121] Updated weights for policy 0, policy_version 36600 (0.0007) +[2023-10-09 13:40:01,316][86122] Updated weights for policy 1, policy_version 36770 (0.0008) +[2023-10-09 13:40:01,681][86122] Updated weights for policy 1, policy_version 36780 (0.0011) +[2023-10-09 13:40:02,044][86122] Updated weights for policy 1, policy_version 36790 (0.0011) +[2023-10-09 13:40:02,399][86122] Updated weights for policy 1, policy_version 36800 (0.0009) +[2023-10-09 13:40:03,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75169792. Throughput: 0: 1819.4, 1: 1830.1. Samples: 18799430. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) +[2023-10-09 13:40:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.920')] +[2023-10-09 13:40:03,595][86121] Updated weights for policy 0, policy_version 36610 (0.0007) +[2023-10-09 13:40:03,968][86121] Updated weights for policy 0, policy_version 36620 (0.0009) +[2023-10-09 13:40:04,324][86121] Updated weights for policy 0, policy_version 36630 (0.0007) +[2023-10-09 13:40:04,688][86121] Updated weights for policy 0, policy_version 36640 (0.0009) +[2023-10-09 13:40:06,153][86122] Updated weights for policy 1, policy_version 36810 (0.0010) +[2023-10-09 13:40:06,518][86122] Updated weights for policy 1, policy_version 36820 (0.0009) +[2023-10-09 13:40:06,879][86122] Updated weights for policy 1, policy_version 36830 (0.0009) +[2023-10-09 13:40:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75235328. Throughput: 0: 1815.4, 1: 1846.4. Samples: 18821390. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) +[2023-10-09 13:40:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.920')] +[2023-10-09 13:40:08,474][86121] Updated weights for policy 0, policy_version 36650 (0.0011) +[2023-10-09 13:40:08,837][86121] Updated weights for policy 0, policy_version 36660 (0.0008) +[2023-10-09 13:40:09,198][86121] Updated weights for policy 0, policy_version 36670 (0.0010) +[2023-10-09 13:40:10,424][86122] Updated weights for policy 1, policy_version 36840 (0.0009) +[2023-10-09 13:40:10,791][86122] Updated weights for policy 1, policy_version 36850 (0.0008) +[2023-10-09 13:40:11,155][86122] Updated weights for policy 1, policy_version 36860 (0.0010) +[2023-10-09 13:40:12,966][86121] Updated weights for policy 0, policy_version 36680 (0.0009) +[2023-10-09 13:40:13,328][86121] Updated weights for policy 0, policy_version 36690 (0.0008) +[2023-10-09 13:40:13,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75300864. Throughput: 0: 1814.1, 1: 1830.9. Samples: 18832098. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) +[2023-10-09 13:40:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.940')] +[2023-10-09 13:40:13,696][86121] Updated weights for policy 0, policy_version 36700 (0.0008) +[2023-10-09 13:40:14,863][86122] Updated weights for policy 1, policy_version 36870 (0.0009) +[2023-10-09 13:40:15,215][86122] Updated weights for policy 1, policy_version 36880 (0.0008) +[2023-10-09 13:40:15,582][86122] Updated weights for policy 1, policy_version 36890 (0.0009) +[2023-10-09 13:40:17,545][86121] Updated weights for policy 0, policy_version 36710 (0.0010) +[2023-10-09 13:40:17,912][86121] Updated weights for policy 0, policy_version 36720 (0.0010) +[2023-10-09 13:40:18,285][86121] Updated weights for policy 0, policy_version 36730 (0.0010) +[2023-10-09 13:40:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 75366400. Throughput: 0: 1815.2, 1: 1853.3. Samples: 18854336. Policy #0 lag: (min: 31.0, avg: 32.7, max: 59.0) +[2023-10-09 13:40:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 13:40:19,141][86122] Updated weights for policy 1, policy_version 36900 (0.0010) +[2023-10-09 13:40:19,499][86122] Updated weights for policy 1, policy_version 36910 (0.0007) +[2023-10-09 13:40:19,863][86122] Updated weights for policy 1, policy_version 36920 (0.0008) +[2023-10-09 13:40:21,845][86121] Updated weights for policy 0, policy_version 36740 (0.0010) +[2023-10-09 13:40:22,224][86121] Updated weights for policy 0, policy_version 36750 (0.0009) +[2023-10-09 13:40:22,590][86121] Updated weights for policy 0, policy_version 36760 (0.0009) +[2023-10-09 13:40:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 75464704. Throughput: 0: 1814.8, 1: 1854.4. Samples: 18875852. Policy #0 lag: (min: 6.0, avg: 14.1, max: 38.0) +[2023-10-09 13:40:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 13:40:23,645][86122] Updated weights for policy 1, policy_version 36930 (0.0009) +[2023-10-09 13:40:24,015][86122] Updated weights for policy 1, policy_version 36940 (0.0009) +[2023-10-09 13:40:24,370][86122] Updated weights for policy 1, policy_version 36950 (0.0009) +[2023-10-09 13:40:24,736][86122] Updated weights for policy 1, policy_version 36960 (0.0008) +[2023-10-09 13:40:26,206][86121] Updated weights for policy 0, policy_version 36770 (0.0010) +[2023-10-09 13:40:26,573][86121] Updated weights for policy 0, policy_version 36780 (0.0007) +[2023-10-09 13:40:26,936][86121] Updated weights for policy 0, policy_version 36790 (0.0010) +[2023-10-09 13:40:27,303][86121] Updated weights for policy 0, policy_version 36800 (0.0010) +[2023-10-09 13:40:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75530240. Throughput: 0: 1814.5, 1: 1852.3. Samples: 18887134. Policy #0 lag: (min: 6.0, avg: 14.1, max: 38.0) +[2023-10-09 13:40:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 13:40:28,436][86122] Updated weights for policy 1, policy_version 36970 (0.0007) +[2023-10-09 13:40:28,793][86122] Updated weights for policy 1, policy_version 36980 (0.0008) +[2023-10-09 13:40:29,157][86122] Updated weights for policy 1, policy_version 36990 (0.0007) +[2023-10-09 13:40:31,090][86121] Updated weights for policy 0, policy_version 36810 (0.0008) +[2023-10-09 13:40:31,447][86121] Updated weights for policy 0, policy_version 36820 (0.0007) +[2023-10-09 13:40:31,815][86121] Updated weights for policy 0, policy_version 36830 (0.0010) +[2023-10-09 13:40:32,868][86122] Updated weights for policy 1, policy_version 37000 (0.0007) +[2023-10-09 13:40:33,229][86122] Updated weights for policy 1, policy_version 37010 (0.0008) +[2023-10-09 13:40:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 75595776. Throughput: 0: 1809.4, 1: 1848.7. Samples: 18908540. Policy #0 lag: (min: 6.0, avg: 14.1, max: 38.0) +[2023-10-09 13:40:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 13:40:33,586][86122] Updated weights for policy 1, policy_version 37020 (0.0008) +[2023-10-09 13:40:35,450][86121] Updated weights for policy 0, policy_version 36840 (0.0008) +[2023-10-09 13:40:35,827][86121] Updated weights for policy 0, policy_version 36850 (0.0008) +[2023-10-09 13:40:36,195][86121] Updated weights for policy 0, policy_version 36860 (0.0007) +[2023-10-09 13:40:37,385][86122] Updated weights for policy 1, policy_version 37030 (0.0009) +[2023-10-09 13:40:37,750][86122] Updated weights for policy 1, policy_version 37040 (0.0011) +[2023-10-09 13:40:38,118][86122] Updated weights for policy 1, policy_version 37050 (0.0009) +[2023-10-09 13:40:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75694080. Throughput: 0: 1810.5, 1: 1827.2. Samples: 18930440. Policy #0 lag: (min: 6.0, avg: 14.1, max: 38.0) +[2023-10-09 13:40:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 13:40:39,887][86121] Updated weights for policy 0, policy_version 36870 (0.0009) +[2023-10-09 13:40:40,263][86121] Updated weights for policy 0, policy_version 36880 (0.0009) +[2023-10-09 13:40:40,618][86121] Updated weights for policy 0, policy_version 36890 (0.0010) +[2023-10-09 13:40:41,774][86122] Updated weights for policy 1, policy_version 37060 (0.0010) +[2023-10-09 13:40:42,145][86122] Updated weights for policy 1, policy_version 37070 (0.0010) +[2023-10-09 13:40:42,505][86122] Updated weights for policy 1, policy_version 37080 (0.0008) +[2023-10-09 13:40:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75759616. Throughput: 0: 1807.1, 1: 1832.9. Samples: 18941002. Policy #0 lag: (min: 6.0, avg: 14.1, max: 38.0) +[2023-10-09 13:40:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 13:40:44,380][86121] Updated weights for policy 0, policy_version 36900 (0.0009) +[2023-10-09 13:40:44,745][86121] Updated weights for policy 0, policy_version 36910 (0.0010) +[2023-10-09 13:40:45,105][86121] Updated weights for policy 0, policy_version 36920 (0.0011) +[2023-10-09 13:40:46,172][86122] Updated weights for policy 1, policy_version 37090 (0.0008) +[2023-10-09 13:40:46,536][86122] Updated weights for policy 1, policy_version 37100 (0.0008) +[2023-10-09 13:40:46,891][86122] Updated weights for policy 1, policy_version 37110 (0.0007) +[2023-10-09 13:40:47,257][86122] Updated weights for policy 1, policy_version 37120 (0.0010) +[2023-10-09 13:40:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75825152. Throughput: 0: 1812.9, 1: 1821.2. Samples: 18962964. Policy #0 lag: (min: 6.0, avg: 14.1, max: 38.0) +[2023-10-09 13:40:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 13:40:48,672][86121] Updated weights for policy 0, policy_version 36930 (0.0009) +[2023-10-09 13:40:49,039][86121] Updated weights for policy 0, policy_version 36940 (0.0008) +[2023-10-09 13:40:49,405][86121] Updated weights for policy 0, policy_version 36950 (0.0009) +[2023-10-09 13:40:49,779][86121] Updated weights for policy 0, policy_version 36960 (0.0009) +[2023-10-09 13:40:50,989][86122] Updated weights for policy 1, policy_version 37130 (0.0009) +[2023-10-09 13:40:51,355][86122] Updated weights for policy 1, policy_version 37140 (0.0007) +[2023-10-09 13:40:51,715][86122] Updated weights for policy 1, policy_version 37150 (0.0007) +[2023-10-09 13:40:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 75890688. Throughput: 0: 1816.8, 1: 1821.2. Samples: 18985100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:40:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 13:40:53,509][86121] Updated weights for policy 0, policy_version 36970 (0.0010) +[2023-10-09 13:40:53,875][86121] Updated weights for policy 0, policy_version 36980 (0.0010) +[2023-10-09 13:40:54,244][86121] Updated weights for policy 0, policy_version 36990 (0.0009) +[2023-10-09 13:40:55,528][86122] Updated weights for policy 1, policy_version 37160 (0.0008) +[2023-10-09 13:40:55,888][86122] Updated weights for policy 1, policy_version 37170 (0.0009) +[2023-10-09 13:40:56,256][86122] Updated weights for policy 1, policy_version 37180 (0.0008) +[2023-10-09 13:40:57,854][86121] Updated weights for policy 0, policy_version 37000 (0.0009) +[2023-10-09 13:40:58,229][86121] Updated weights for policy 0, policy_version 37010 (0.0008) +[2023-10-09 13:40:58,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 75956224. Throughput: 0: 1816.3, 1: 1819.9. Samples: 18995726. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:40:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 13:40:58,586][86121] Updated weights for policy 0, policy_version 37020 (0.0007) +[2023-10-09 13:40:59,914][86122] Updated weights for policy 1, policy_version 37190 (0.0007) +[2023-10-09 13:41:00,268][86122] Updated weights for policy 1, policy_version 37200 (0.0008) +[2023-10-09 13:41:00,630][86122] Updated weights for policy 1, policy_version 37210 (0.0009) +[2023-10-09 13:41:02,370][86121] Updated weights for policy 0, policy_version 37030 (0.0009) +[2023-10-09 13:41:02,744][86121] Updated weights for policy 0, policy_version 37040 (0.0008) +[2023-10-09 13:41:03,117][86121] Updated weights for policy 0, policy_version 37050 (0.0010) +[2023-10-09 13:41:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 76054528. Throughput: 0: 1814.6, 1: 1816.8. Samples: 19017748. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.950')] +[2023-10-09 13:41:04,382][86122] Updated weights for policy 1, policy_version 37220 (0.0008) +[2023-10-09 13:41:04,745][86122] Updated weights for policy 1, policy_version 37230 (0.0011) +[2023-10-09 13:41:05,106][86122] Updated weights for policy 1, policy_version 37240 (0.0011) +[2023-10-09 13:41:06,760][86121] Updated weights for policy 0, policy_version 37060 (0.0008) +[2023-10-09 13:41:07,140][86121] Updated weights for policy 0, policy_version 37070 (0.0007) +[2023-10-09 13:41:07,507][86121] Updated weights for policy 0, policy_version 37080 (0.0007) +[2023-10-09 13:41:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76120064. Throughput: 0: 1814.6, 1: 1815.8. Samples: 19039220. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.950')] +[2023-10-09 13:41:08,737][86122] Updated weights for policy 1, policy_version 37250 (0.0009) +[2023-10-09 13:41:09,098][86122] Updated weights for policy 1, policy_version 37260 (0.0008) +[2023-10-09 13:41:09,471][86122] Updated weights for policy 1, policy_version 37270 (0.0007) +[2023-10-09 13:41:09,832][86122] Updated weights for policy 1, policy_version 37280 (0.0008) +[2023-10-09 13:41:11,130][86121] Updated weights for policy 0, policy_version 37090 (0.0007) +[2023-10-09 13:41:11,492][86121] Updated weights for policy 0, policy_version 37100 (0.0010) +[2023-10-09 13:41:11,855][86121] Updated weights for policy 0, policy_version 37110 (0.0009) +[2023-10-09 13:41:12,218][86121] Updated weights for policy 0, policy_version 37120 (0.0007) +[2023-10-09 13:41:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76185600. Throughput: 0: 1813.4, 1: 1817.4. Samples: 19050520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.940')] +[2023-10-09 13:41:13,746][86122] Updated weights for policy 1, policy_version 37290 (0.0008) +[2023-10-09 13:41:14,111][86122] Updated weights for policy 1, policy_version 37300 (0.0008) +[2023-10-09 13:41:14,472][86122] Updated weights for policy 1, policy_version 37310 (0.0008) +[2023-10-09 13:41:16,060][86121] Updated weights for policy 0, policy_version 37130 (0.0010) +[2023-10-09 13:41:16,430][86121] Updated weights for policy 0, policy_version 37140 (0.0008) +[2023-10-09 13:41:16,783][86121] Updated weights for policy 0, policy_version 37150 (0.0008) +[2023-10-09 13:41:18,078][86122] Updated weights for policy 1, policy_version 37320 (0.0008) +[2023-10-09 13:41:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 76251136. Throughput: 0: 1808.0, 1: 1817.4. Samples: 19071684. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.930')] +[2023-10-09 13:41:18,440][86122] Updated weights for policy 1, policy_version 37330 (0.0009) +[2023-10-09 13:41:18,806][86122] Updated weights for policy 1, policy_version 37340 (0.0009) +[2023-10-09 13:41:20,479][86121] Updated weights for policy 0, policy_version 37160 (0.0009) +[2023-10-09 13:41:20,841][86121] Updated weights for policy 0, policy_version 37170 (0.0008) +[2023-10-09 13:41:21,217][86121] Updated weights for policy 0, policy_version 37180 (0.0007) +[2023-10-09 13:41:22,393][86122] Updated weights for policy 1, policy_version 37350 (0.0008) +[2023-10-09 13:41:22,758][86122] Updated weights for policy 1, policy_version 37360 (0.0008) +[2023-10-09 13:41:23,122][86122] Updated weights for policy 1, policy_version 37370 (0.0008) +[2023-10-09 13:41:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76349440. Throughput: 0: 1816.7, 1: 1825.9. Samples: 19094354. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.930')] +[2023-10-09 13:41:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000037184_38076416.pth... +[2023-10-09 13:41:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000037376_38273024.pth... +[2023-10-09 13:41:23,436][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000035488_36339712.pth +[2023-10-09 13:41:23,442][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000035648_36503552.pth +[2023-10-09 13:41:24,729][86121] Updated weights for policy 0, policy_version 37190 (0.0008) +[2023-10-09 13:41:25,091][86121] Updated weights for policy 0, policy_version 37200 (0.0008) +[2023-10-09 13:41:25,454][86121] Updated weights for policy 0, policy_version 37210 (0.0009) +[2023-10-09 13:41:26,724][86122] Updated weights for policy 1, policy_version 37380 (0.0008) +[2023-10-09 13:41:27,096][86122] Updated weights for policy 1, policy_version 37390 (0.0008) +[2023-10-09 13:41:27,465][86122] Updated weights for policy 1, policy_version 37400 (0.0007) +[2023-10-09 13:41:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 76414976. Throughput: 0: 1822.2, 1: 1825.0. Samples: 19105128. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.930')] +[2023-10-09 13:41:29,215][86121] Updated weights for policy 0, policy_version 37220 (0.0008) +[2023-10-09 13:41:29,588][86121] Updated weights for policy 0, policy_version 37230 (0.0008) +[2023-10-09 13:41:29,945][86121] Updated weights for policy 0, policy_version 37240 (0.0008) +[2023-10-09 13:41:31,114][86122] Updated weights for policy 1, policy_version 37410 (0.0008) +[2023-10-09 13:41:31,473][86122] Updated weights for policy 1, policy_version 37420 (0.0007) +[2023-10-09 13:41:31,839][86122] Updated weights for policy 1, policy_version 37430 (0.0007) +[2023-10-09 13:41:32,201][86122] Updated weights for policy 1, policy_version 37440 (0.0008) +[2023-10-09 13:41:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 76480512. Throughput: 0: 1821.2, 1: 1828.5. Samples: 19127204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.920')] +[2023-10-09 13:41:33,621][86121] Updated weights for policy 0, policy_version 37250 (0.0008) +[2023-10-09 13:41:33,987][86121] Updated weights for policy 0, policy_version 37260 (0.0008) +[2023-10-09 13:41:34,339][86121] Updated weights for policy 0, policy_version 37270 (0.0009) +[2023-10-09 13:41:34,700][86121] Updated weights for policy 0, policy_version 37280 (0.0010) +[2023-10-09 13:41:35,775][86122] Updated weights for policy 1, policy_version 37450 (0.0009) +[2023-10-09 13:41:36,144][86122] Updated weights for policy 1, policy_version 37460 (0.0007) +[2023-10-09 13:41:36,502][86122] Updated weights for policy 1, policy_version 37470 (0.0010) +[2023-10-09 13:41:38,387][86121] Updated weights for policy 0, policy_version 37290 (0.0007) +[2023-10-09 13:41:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 76546048. Throughput: 0: 1820.5, 1: 1836.1. Samples: 19149648. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.920')] +[2023-10-09 13:41:38,759][86121] Updated weights for policy 0, policy_version 37300 (0.0007) +[2023-10-09 13:41:39,129][86121] Updated weights for policy 0, policy_version 37310 (0.0007) +[2023-10-09 13:41:40,220][86122] Updated weights for policy 1, policy_version 37480 (0.0010) +[2023-10-09 13:41:40,587][86122] Updated weights for policy 1, policy_version 37490 (0.0011) +[2023-10-09 13:41:40,957][86122] Updated weights for policy 1, policy_version 37500 (0.0008) +[2023-10-09 13:41:42,912][86121] Updated weights for policy 0, policy_version 37320 (0.0007) +[2023-10-09 13:41:43,286][86121] Updated weights for policy 0, policy_version 37330 (0.0007) +[2023-10-09 13:41:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 76611584. Throughput: 0: 1819.7, 1: 1828.6. Samples: 19159900. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:41:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.920')] +[2023-10-09 13:41:43,646][86121] Updated weights for policy 0, policy_version 37340 (0.0008) +[2023-10-09 13:41:44,578][86122] Updated weights for policy 1, policy_version 37510 (0.0009) +[2023-10-09 13:41:44,938][86122] Updated weights for policy 1, policy_version 37520 (0.0008) +[2023-10-09 13:41:45,299][86122] Updated weights for policy 1, policy_version 37530 (0.0010) +[2023-10-09 13:41:47,573][86121] Updated weights for policy 0, policy_version 37350 (0.0010) +[2023-10-09 13:41:47,933][86121] Updated weights for policy 0, policy_version 37360 (0.0011) +[2023-10-09 13:41:48,301][86121] Updated weights for policy 0, policy_version 37370 (0.0009) +[2023-10-09 13:41:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 76677120. Throughput: 0: 1814.1, 1: 1839.5. Samples: 19182160. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:41:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.920')] +[2023-10-09 13:41:48,941][86122] Updated weights for policy 1, policy_version 37540 (0.0009) +[2023-10-09 13:41:49,301][86122] Updated weights for policy 1, policy_version 37550 (0.0007) +[2023-10-09 13:41:49,667][86122] Updated weights for policy 1, policy_version 37560 (0.0007) +[2023-10-09 13:41:52,191][86121] Updated weights for policy 0, policy_version 37380 (0.0010) +[2023-10-09 13:41:52,574][86121] Updated weights for policy 0, policy_version 37390 (0.0007) +[2023-10-09 13:41:52,940][86121] Updated weights for policy 0, policy_version 37400 (0.0007) +[2023-10-09 13:41:53,332][86122] Updated weights for policy 1, policy_version 37570 (0.0007) +[2023-10-09 13:41:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76775424. Throughput: 0: 1817.5, 1: 1843.3. Samples: 19203956. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:41:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.920')] +[2023-10-09 13:41:53,699][86122] Updated weights for policy 1, policy_version 37580 (0.0007) +[2023-10-09 13:41:54,056][86122] Updated weights for policy 1, policy_version 37590 (0.0008) +[2023-10-09 13:41:54,412][86122] Updated weights for policy 1, policy_version 37600 (0.0010) +[2023-10-09 13:41:56,563][86121] Updated weights for policy 0, policy_version 37410 (0.0008) +[2023-10-09 13:41:56,922][86121] Updated weights for policy 0, policy_version 37420 (0.0008) +[2023-10-09 13:41:57,290][86121] Updated weights for policy 0, policy_version 37430 (0.0011) +[2023-10-09 13:41:57,650][86121] Updated weights for policy 0, policy_version 37440 (0.0007) +[2023-10-09 13:41:58,036][86122] Updated weights for policy 1, policy_version 37610 (0.0008) +[2023-10-09 13:41:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76840960. Throughput: 0: 1805.6, 1: 1844.4. Samples: 19214770. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:41:58,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.920')] +[2023-10-09 13:41:58,400][86122] Updated weights for policy 1, policy_version 37620 (0.0007) +[2023-10-09 13:41:58,767][86122] Updated weights for policy 1, policy_version 37630 (0.0007) +[2023-10-09 13:42:01,409][86121] Updated weights for policy 0, policy_version 37450 (0.0010) +[2023-10-09 13:42:01,777][86121] Updated weights for policy 0, policy_version 37460 (0.0009) +[2023-10-09 13:42:02,143][86121] Updated weights for policy 0, policy_version 37470 (0.0008) +[2023-10-09 13:42:02,466][86122] Updated weights for policy 1, policy_version 37640 (0.0009) +[2023-10-09 13:42:02,824][86122] Updated weights for policy 1, policy_version 37650 (0.0007) +[2023-10-09 13:42:03,193][86122] Updated weights for policy 1, policy_version 37660 (0.0010) +[2023-10-09 13:42:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 76939264. Throughput: 0: 1819.9, 1: 1844.0. Samples: 19236558. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:42:03,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.920')] +[2023-10-09 13:42:05,811][86121] Updated weights for policy 0, policy_version 37480 (0.0009) +[2023-10-09 13:42:06,184][86121] Updated weights for policy 0, policy_version 37490 (0.0010) +[2023-10-09 13:42:06,553][86121] Updated weights for policy 0, policy_version 37500 (0.0010) +[2023-10-09 13:42:06,970][86122] Updated weights for policy 1, policy_version 37670 (0.0009) +[2023-10-09 13:42:07,320][86122] Updated weights for policy 1, policy_version 37680 (0.0007) +[2023-10-09 13:42:07,688][86122] Updated weights for policy 1, policy_version 37690 (0.0007) +[2023-10-09 13:42:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77004800. Throughput: 0: 1798.2, 1: 1827.4. Samples: 19257506. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:42:08,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.920')] +[2023-10-09 13:42:10,313][86121] Updated weights for policy 0, policy_version 37510 (0.0010) +[2023-10-09 13:42:10,684][86121] Updated weights for policy 0, policy_version 37520 (0.0009) +[2023-10-09 13:42:11,051][86121] Updated weights for policy 0, policy_version 37530 (0.0010) +[2023-10-09 13:42:11,437][86122] Updated weights for policy 1, policy_version 37700 (0.0009) +[2023-10-09 13:42:11,803][86122] Updated weights for policy 1, policy_version 37710 (0.0008) +[2023-10-09 13:42:12,169][86122] Updated weights for policy 1, policy_version 37720 (0.0008) +[2023-10-09 13:42:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77070336. Throughput: 0: 1807.7, 1: 1837.0. Samples: 19269140. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 13:42:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.910')] +[2023-10-09 13:42:14,724][86121] Updated weights for policy 0, policy_version 37540 (0.0008) +[2023-10-09 13:42:15,091][86121] Updated weights for policy 0, policy_version 37550 (0.0008) +[2023-10-09 13:42:15,453][86121] Updated weights for policy 0, policy_version 37560 (0.0008) +[2023-10-09 13:42:15,798][86122] Updated weights for policy 1, policy_version 37730 (0.0008) +[2023-10-09 13:42:16,167][86122] Updated weights for policy 1, policy_version 37740 (0.0011) +[2023-10-09 13:42:16,531][86122] Updated weights for policy 1, policy_version 37750 (0.0010) +[2023-10-09 13:42:16,896][86122] Updated weights for policy 1, policy_version 37760 (0.0010) +[2023-10-09 13:42:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77135872. Throughput: 0: 1799.8, 1: 1823.3. Samples: 19290242. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 13:42:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.910')] +[2023-10-09 13:42:19,116][86121] Updated weights for policy 0, policy_version 37570 (0.0008) +[2023-10-09 13:42:19,490][86121] Updated weights for policy 0, policy_version 37580 (0.0010) +[2023-10-09 13:42:19,850][86121] Updated weights for policy 0, policy_version 37590 (0.0008) +[2023-10-09 13:42:20,224][86121] Updated weights for policy 0, policy_version 37600 (0.0009) +[2023-10-09 13:42:20,807][86122] Updated weights for policy 1, policy_version 37770 (0.0007) +[2023-10-09 13:42:21,167][86122] Updated weights for policy 1, policy_version 37780 (0.0007) +[2023-10-09 13:42:21,525][86122] Updated weights for policy 1, policy_version 37790 (0.0008) +[2023-10-09 13:42:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77201408. Throughput: 0: 1799.3, 1: 1820.2. Samples: 19312526. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 13:42:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.910')] +[2023-10-09 13:42:23,955][86121] Updated weights for policy 0, policy_version 37610 (0.0008) +[2023-10-09 13:42:24,337][86121] Updated weights for policy 0, policy_version 37620 (0.0008) +[2023-10-09 13:42:24,700][86121] Updated weights for policy 0, policy_version 37630 (0.0009) +[2023-10-09 13:42:25,311][86122] Updated weights for policy 1, policy_version 37800 (0.0009) +[2023-10-09 13:42:25,668][86122] Updated weights for policy 1, policy_version 37810 (0.0008) +[2023-10-09 13:42:26,025][86122] Updated weights for policy 1, policy_version 37820 (0.0007) +[2023-10-09 13:42:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77266944. Throughput: 0: 1799.6, 1: 1820.7. Samples: 19322816. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 13:42:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.900')] +[2023-10-09 13:42:28,432][86121] Updated weights for policy 0, policy_version 37640 (0.0009) +[2023-10-09 13:42:28,790][86121] Updated weights for policy 0, policy_version 37650 (0.0009) +[2023-10-09 13:42:29,154][86121] Updated weights for policy 0, policy_version 37660 (0.0007) +[2023-10-09 13:42:29,730][86122] Updated weights for policy 1, policy_version 37830 (0.0007) +[2023-10-09 13:42:30,101][86122] Updated weights for policy 1, policy_version 37840 (0.0009) +[2023-10-09 13:42:30,468][86122] Updated weights for policy 1, policy_version 37850 (0.0009) +[2023-10-09 13:42:32,805][86121] Updated weights for policy 0, policy_version 37670 (0.0010) +[2023-10-09 13:42:33,165][86121] Updated weights for policy 0, policy_version 37680 (0.0009) +[2023-10-09 13:42:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77332480. Throughput: 0: 1808.6, 1: 1815.3. Samples: 19345236. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 13:42:33,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.900')] +[2023-10-09 13:42:33,533][86121] Updated weights for policy 0, policy_version 37690 (0.0007) +[2023-10-09 13:42:34,182][86122] Updated weights for policy 1, policy_version 37860 (0.0008) +[2023-10-09 13:42:34,537][86122] Updated weights for policy 1, policy_version 37870 (0.0008) +[2023-10-09 13:42:34,899][86122] Updated weights for policy 1, policy_version 37880 (0.0010) +[2023-10-09 13:42:37,300][86121] Updated weights for policy 0, policy_version 37700 (0.0009) +[2023-10-09 13:42:37,690][86121] Updated weights for policy 0, policy_version 37710 (0.0010) +[2023-10-09 13:42:38,051][86121] Updated weights for policy 0, policy_version 37720 (0.0007) +[2023-10-09 13:42:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77430784. Throughput: 0: 1815.7, 1: 1807.8. Samples: 19367012. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) +[2023-10-09 13:42:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.900')] +[2023-10-09 13:42:38,537][86122] Updated weights for policy 1, policy_version 37890 (0.0007) +[2023-10-09 13:42:38,904][86122] Updated weights for policy 1, policy_version 37900 (0.0008) +[2023-10-09 13:42:39,265][86122] Updated weights for policy 1, policy_version 37910 (0.0010) +[2023-10-09 13:42:39,622][86122] Updated weights for policy 1, policy_version 37920 (0.0009) +[2023-10-09 13:42:41,629][86121] Updated weights for policy 0, policy_version 37730 (0.0008) +[2023-10-09 13:42:41,993][86121] Updated weights for policy 0, policy_version 37740 (0.0007) +[2023-10-09 13:42:42,355][86121] Updated weights for policy 0, policy_version 37750 (0.0007) +[2023-10-09 13:42:42,727][86121] Updated weights for policy 0, policy_version 37760 (0.0007) +[2023-10-09 13:42:43,372][86122] Updated weights for policy 1, policy_version 37930 (0.0007) +[2023-10-09 13:42:43,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 77496320. Throughput: 0: 1818.3, 1: 1804.1. Samples: 19377776. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) +[2023-10-09 13:42:43,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.890')] +[2023-10-09 13:42:43,741][86122] Updated weights for policy 1, policy_version 37940 (0.0007) +[2023-10-09 13:42:44,111][86122] Updated weights for policy 1, policy_version 37950 (0.0010) +[2023-10-09 13:42:46,482][86121] Updated weights for policy 0, policy_version 37770 (0.0007) +[2023-10-09 13:42:46,843][86121] Updated weights for policy 0, policy_version 37780 (0.0007) +[2023-10-09 13:42:47,208][86121] Updated weights for policy 0, policy_version 37790 (0.0011) +[2023-10-09 13:42:47,832][86122] Updated weights for policy 1, policy_version 37960 (0.0009) +[2023-10-09 13:42:48,190][86122] Updated weights for policy 1, policy_version 37970 (0.0009) +[2023-10-09 13:42:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77561856. Throughput: 0: 1820.0, 1: 1809.0. Samples: 19399866. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) +[2023-10-09 13:42:48,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.900')] +[2023-10-09 13:42:48,562][86122] Updated weights for policy 1, policy_version 37980 (0.0008) +[2023-10-09 13:42:50,898][86121] Updated weights for policy 0, policy_version 37800 (0.0011) +[2023-10-09 13:42:51,264][86121] Updated weights for policy 0, policy_version 37810 (0.0010) +[2023-10-09 13:42:51,635][86121] Updated weights for policy 0, policy_version 37820 (0.0010) +[2023-10-09 13:42:52,249][86122] Updated weights for policy 1, policy_version 37990 (0.0007) +[2023-10-09 13:42:52,616][86122] Updated weights for policy 1, policy_version 38000 (0.0007) +[2023-10-09 13:42:52,974][86122] Updated weights for policy 1, policy_version 38010 (0.0007) +[2023-10-09 13:42:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 77660160. Throughput: 0: 1824.9, 1: 1816.3. Samples: 19421360. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) +[2023-10-09 13:42:53,399][85186] Avg episode reward: [(0, '9.900'), (1, '9.900')] +[2023-10-09 13:42:55,210][86121] Updated weights for policy 0, policy_version 37830 (0.0009) +[2023-10-09 13:42:55,581][86121] Updated weights for policy 0, policy_version 37840 (0.0009) +[2023-10-09 13:42:55,948][86121] Updated weights for policy 0, policy_version 37850 (0.0007) +[2023-10-09 13:42:56,619][86122] Updated weights for policy 1, policy_version 38020 (0.0007) +[2023-10-09 13:42:56,995][86122] Updated weights for policy 1, policy_version 38030 (0.0009) +[2023-10-09 13:42:57,363][86122] Updated weights for policy 1, policy_version 38040 (0.0008) +[2023-10-09 13:42:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 77725696. Throughput: 0: 1826.4, 1: 1811.2. Samples: 19432836. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) +[2023-10-09 13:42:58,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.900')] +[2023-10-09 13:42:59,610][86121] Updated weights for policy 0, policy_version 37860 (0.0007) +[2023-10-09 13:42:59,981][86121] Updated weights for policy 0, policy_version 37870 (0.0008) +[2023-10-09 13:43:00,349][86121] Updated weights for policy 0, policy_version 37880 (0.0007) +[2023-10-09 13:43:01,117][86122] Updated weights for policy 1, policy_version 38050 (0.0009) +[2023-10-09 13:43:01,487][86122] Updated weights for policy 1, policy_version 38060 (0.0007) +[2023-10-09 13:43:01,838][86122] Updated weights for policy 1, policy_version 38070 (0.0008) +[2023-10-09 13:43:02,201][86122] Updated weights for policy 1, policy_version 38080 (0.0009) +[2023-10-09 13:43:03,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77791232. Throughput: 0: 1829.1, 1: 1815.1. Samples: 19454228. Policy #0 lag: (min: 7.0, avg: 7.0, max: 9.0) +[2023-10-09 13:43:03,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.900')] +[2023-10-09 13:43:03,935][86121] Updated weights for policy 0, policy_version 37890 (0.0009) +[2023-10-09 13:43:04,309][86121] Updated weights for policy 0, policy_version 37900 (0.0008) +[2023-10-09 13:43:04,670][86121] Updated weights for policy 0, policy_version 37910 (0.0010) +[2023-10-09 13:43:05,045][86121] Updated weights for policy 0, policy_version 37920 (0.0010) +[2023-10-09 13:43:05,936][86122] Updated weights for policy 1, policy_version 38090 (0.0008) +[2023-10-09 13:43:06,300][86122] Updated weights for policy 1, policy_version 38100 (0.0009) +[2023-10-09 13:43:06,657][86122] Updated weights for policy 1, policy_version 38110 (0.0007) +[2023-10-09 13:43:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 77856768. Throughput: 0: 1835.7, 1: 1818.5. Samples: 19476968. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) +[2023-10-09 13:43:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.910')] +[2023-10-09 13:43:08,749][86121] Updated weights for policy 0, policy_version 37930 (0.0008) +[2023-10-09 13:43:09,124][86121] Updated weights for policy 0, policy_version 37940 (0.0009) +[2023-10-09 13:43:09,489][86121] Updated weights for policy 0, policy_version 37950 (0.0007) +[2023-10-09 13:43:10,181][86122] Updated weights for policy 1, policy_version 38120 (0.0007) +[2023-10-09 13:43:10,554][86122] Updated weights for policy 1, policy_version 38130 (0.0009) +[2023-10-09 13:43:10,917][86122] Updated weights for policy 1, policy_version 38140 (0.0008) +[2023-10-09 13:43:13,063][86121] Updated weights for policy 0, policy_version 37960 (0.0008) +[2023-10-09 13:43:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 77922304. Throughput: 0: 1838.1, 1: 1821.1. Samples: 19487480. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) +[2023-10-09 13:43:13,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.910')] +[2023-10-09 13:43:13,430][86121] Updated weights for policy 0, policy_version 37970 (0.0007) +[2023-10-09 13:43:13,805][86121] Updated weights for policy 0, policy_version 37980 (0.0007) +[2023-10-09 13:43:14,658][86122] Updated weights for policy 1, policy_version 38150 (0.0010) +[2023-10-09 13:43:15,014][86122] Updated weights for policy 1, policy_version 38160 (0.0010) +[2023-10-09 13:43:15,383][86122] Updated weights for policy 1, policy_version 38170 (0.0011) +[2023-10-09 13:43:17,266][86121] Updated weights for policy 0, policy_version 37990 (0.0008) +[2023-10-09 13:43:17,628][86121] Updated weights for policy 0, policy_version 38000 (0.0007) +[2023-10-09 13:43:17,995][86121] Updated weights for policy 0, policy_version 38010 (0.0007) +[2023-10-09 13:43:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78020608. Throughput: 0: 1843.2, 1: 1820.8. Samples: 19510116. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) +[2023-10-09 13:43:18,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.920')] +[2023-10-09 13:43:19,169][86122] Updated weights for policy 1, policy_version 38180 (0.0009) +[2023-10-09 13:43:19,537][86122] Updated weights for policy 1, policy_version 38190 (0.0011) +[2023-10-09 13:43:19,901][86122] Updated weights for policy 1, policy_version 38200 (0.0011) +[2023-10-09 13:43:21,718][86121] Updated weights for policy 0, policy_version 38020 (0.0008) +[2023-10-09 13:43:22,082][86121] Updated weights for policy 0, policy_version 38030 (0.0010) +[2023-10-09 13:43:22,443][86121] Updated weights for policy 0, policy_version 38040 (0.0007) +[2023-10-09 13:43:23,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78086144. Throughput: 0: 1832.0, 1: 1824.8. Samples: 19531570. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) +[2023-10-09 13:43:23,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.920')] +[2023-10-09 13:43:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000038048_38961152.pth... +[2023-10-09 13:43:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000036352_37224448.pth +[2023-10-09 13:43:23,528][86122] Updated weights for policy 1, policy_version 38210 (0.0009) +[2023-10-09 13:43:23,897][86122] Updated weights for policy 1, policy_version 38220 (0.0008) +[2023-10-09 13:43:24,258][86122] Updated weights for policy 1, policy_version 38230 (0.0009) +[2023-10-09 13:43:24,621][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000038240_39157760.pth... +[2023-10-09 13:43:24,623][86122] Updated weights for policy 1, policy_version 38240 (0.0009) +[2023-10-09 13:43:24,661][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000036512_37388288.pth +[2023-10-09 13:43:26,180][86121] Updated weights for policy 0, policy_version 38050 (0.0008) +[2023-10-09 13:43:26,564][86121] Updated weights for policy 0, policy_version 38060 (0.0009) +[2023-10-09 13:43:26,938][86121] Updated weights for policy 0, policy_version 38070 (0.0011) +[2023-10-09 13:43:27,306][86121] Updated weights for policy 0, policy_version 38080 (0.0007) +[2023-10-09 13:43:28,236][86122] Updated weights for policy 1, policy_version 38250 (0.0007) +[2023-10-09 13:43:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78151680. Throughput: 0: 1843.0, 1: 1827.3. Samples: 19542938. Policy #0 lag: (min: 31.0, avg: 32.9, max: 60.0) +[2023-10-09 13:43:28,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.930')] +[2023-10-09 13:43:28,598][86122] Updated weights for policy 1, policy_version 38260 (0.0009) +[2023-10-09 13:43:28,960][86122] Updated weights for policy 1, policy_version 38270 (0.0007) +[2023-10-09 13:43:31,065][86121] Updated weights for policy 0, policy_version 38090 (0.0008) +[2023-10-09 13:43:31,434][86121] Updated weights for policy 0, policy_version 38100 (0.0008) +[2023-10-09 13:43:31,793][86121] Updated weights for policy 0, policy_version 38110 (0.0009) +[2023-10-09 13:43:32,654][86122] Updated weights for policy 1, policy_version 38280 (0.0007) +[2023-10-09 13:43:33,014][86122] Updated weights for policy 1, policy_version 38290 (0.0010) +[2023-10-09 13:43:33,376][86122] Updated weights for policy 1, policy_version 38300 (0.0008) +[2023-10-09 13:43:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78217216. Throughput: 0: 1831.6, 1: 1826.6. Samples: 19564482. Policy #0 lag: (min: 2.0, avg: 7.4, max: 34.0) +[2023-10-09 13:43:33,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.940')] +[2023-10-09 13:43:35,564][86121] Updated weights for policy 0, policy_version 38120 (0.0011) +[2023-10-09 13:43:35,933][86121] Updated weights for policy 0, policy_version 38130 (0.0010) +[2023-10-09 13:43:36,299][86121] Updated weights for policy 0, policy_version 38140 (0.0010) +[2023-10-09 13:43:37,115][86122] Updated weights for policy 1, policy_version 38310 (0.0007) +[2023-10-09 13:43:37,473][86122] Updated weights for policy 1, policy_version 38320 (0.0009) +[2023-10-09 13:43:37,828][86122] Updated weights for policy 1, policy_version 38330 (0.0010) +[2023-10-09 13:43:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 78315520. Throughput: 0: 1836.6, 1: 1818.9. Samples: 19585858. Policy #0 lag: (min: 2.0, avg: 7.4, max: 34.0) +[2023-10-09 13:43:38,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.960')] +[2023-10-09 13:43:39,978][86121] Updated weights for policy 0, policy_version 38150 (0.0008) +[2023-10-09 13:43:40,343][86121] Updated weights for policy 0, policy_version 38160 (0.0007) +[2023-10-09 13:43:40,706][86121] Updated weights for policy 0, policy_version 38170 (0.0007) +[2023-10-09 13:43:41,585][86122] Updated weights for policy 1, policy_version 38340 (0.0008) +[2023-10-09 13:43:41,942][86122] Updated weights for policy 1, policy_version 38350 (0.0010) +[2023-10-09 13:43:42,306][86122] Updated weights for policy 1, policy_version 38360 (0.0009) +[2023-10-09 13:43:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78381056. Throughput: 0: 1827.1, 1: 1820.2. Samples: 19596964. Policy #0 lag: (min: 2.0, avg: 7.4, max: 34.0) +[2023-10-09 13:43:43,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.960')] +[2023-10-09 13:43:44,341][86121] Updated weights for policy 0, policy_version 38180 (0.0008) +[2023-10-09 13:43:44,708][86121] Updated weights for policy 0, policy_version 38190 (0.0008) +[2023-10-09 13:43:45,070][86121] Updated weights for policy 0, policy_version 38200 (0.0008) +[2023-10-09 13:43:46,003][86122] Updated weights for policy 1, policy_version 38370 (0.0008) +[2023-10-09 13:43:46,365][86122] Updated weights for policy 1, policy_version 38380 (0.0008) +[2023-10-09 13:43:46,724][86122] Updated weights for policy 1, policy_version 38390 (0.0008) +[2023-10-09 13:43:47,087][86122] Updated weights for policy 1, policy_version 38400 (0.0007) +[2023-10-09 13:43:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78446592. Throughput: 0: 1835.9, 1: 1823.7. Samples: 19618912. Policy #0 lag: (min: 2.0, avg: 7.4, max: 34.0) +[2023-10-09 13:43:48,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.960')] +[2023-10-09 13:43:48,723][86121] Updated weights for policy 0, policy_version 38210 (0.0009) +[2023-10-09 13:43:49,093][86121] Updated weights for policy 0, policy_version 38220 (0.0009) +[2023-10-09 13:43:49,455][86121] Updated weights for policy 0, policy_version 38230 (0.0008) +[2023-10-09 13:43:49,823][86121] Updated weights for policy 0, policy_version 38240 (0.0008) +[2023-10-09 13:43:50,832][86122] Updated weights for policy 1, policy_version 38410 (0.0010) +[2023-10-09 13:43:51,198][86122] Updated weights for policy 1, policy_version 38420 (0.0011) +[2023-10-09 13:43:51,565][86122] Updated weights for policy 1, policy_version 38430 (0.0011) +[2023-10-09 13:43:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78512128. Throughput: 0: 1829.0, 1: 1825.5. Samples: 19641420. Policy #0 lag: (min: 2.0, avg: 7.4, max: 34.0) +[2023-10-09 13:43:53,399][85186] Avg episode reward: [(0, '9.910'), (1, '9.960')] +[2023-10-09 13:43:53,689][86121] Updated weights for policy 0, policy_version 38250 (0.0009) +[2023-10-09 13:43:54,057][86121] Updated weights for policy 0, policy_version 38260 (0.0009) +[2023-10-09 13:43:54,417][86121] Updated weights for policy 0, policy_version 38270 (0.0011) +[2023-10-09 13:43:55,190][86122] Updated weights for policy 1, policy_version 38440 (0.0008) +[2023-10-09 13:43:55,548][86122] Updated weights for policy 1, policy_version 38450 (0.0009) +[2023-10-09 13:43:55,917][86122] Updated weights for policy 1, policy_version 38460 (0.0008) +[2023-10-09 13:43:58,136][86121] Updated weights for policy 0, policy_version 38280 (0.0010) +[2023-10-09 13:43:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78577664. Throughput: 0: 1822.7, 1: 1823.6. Samples: 19651562. Policy #0 lag: (min: 2.0, avg: 7.4, max: 34.0) +[2023-10-09 13:43:58,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.950')] +[2023-10-09 13:43:58,503][86121] Updated weights for policy 0, policy_version 38290 (0.0008) +[2023-10-09 13:43:58,876][86121] Updated weights for policy 0, policy_version 38300 (0.0007) +[2023-10-09 13:43:59,684][86122] Updated weights for policy 1, policy_version 38470 (0.0007) +[2023-10-09 13:44:00,037][86122] Updated weights for policy 1, policy_version 38480 (0.0007) +[2023-10-09 13:44:00,406][86122] Updated weights for policy 1, policy_version 38490 (0.0009) +[2023-10-09 13:44:02,499][86121] Updated weights for policy 0, policy_version 38310 (0.0007) +[2023-10-09 13:44:02,866][86121] Updated weights for policy 0, policy_version 38320 (0.0009) +[2023-10-09 13:44:03,241][86121] Updated weights for policy 0, policy_version 38330 (0.0008) +[2023-10-09 13:44:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 78643200. Throughput: 0: 1815.8, 1: 1823.7. Samples: 19673892. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) +[2023-10-09 13:44:03,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.950')] +[2023-10-09 13:44:04,060][86122] Updated weights for policy 1, policy_version 38500 (0.0009) +[2023-10-09 13:44:04,416][86122] Updated weights for policy 1, policy_version 38510 (0.0007) +[2023-10-09 13:44:04,774][86122] Updated weights for policy 1, policy_version 38520 (0.0010) +[2023-10-09 13:44:06,995][86121] Updated weights for policy 0, policy_version 38340 (0.0010) +[2023-10-09 13:44:07,374][86121] Updated weights for policy 0, policy_version 38350 (0.0008) +[2023-10-09 13:44:07,739][86121] Updated weights for policy 0, policy_version 38360 (0.0010) +[2023-10-09 13:44:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 78741504. Throughput: 0: 1819.8, 1: 1823.6. Samples: 19695520. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) +[2023-10-09 13:44:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.950')] +[2023-10-09 13:44:08,516][86122] Updated weights for policy 1, policy_version 38530 (0.0009) +[2023-10-09 13:44:08,881][86122] Updated weights for policy 1, policy_version 38540 (0.0008) +[2023-10-09 13:44:09,248][86122] Updated weights for policy 1, policy_version 38550 (0.0009) +[2023-10-09 13:44:09,612][86122] Updated weights for policy 1, policy_version 38560 (0.0007) +[2023-10-09 13:44:11,532][86121] Updated weights for policy 0, policy_version 38370 (0.0010) +[2023-10-09 13:44:11,920][86121] Updated weights for policy 0, policy_version 38380 (0.0008) +[2023-10-09 13:44:12,289][86121] Updated weights for policy 0, policy_version 38390 (0.0007) +[2023-10-09 13:44:12,649][86121] Updated weights for policy 0, policy_version 38400 (0.0007) +[2023-10-09 13:44:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 78807040. Throughput: 0: 1815.3, 1: 1820.6. Samples: 19706556. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) +[2023-10-09 13:44:13,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.950')] +[2023-10-09 13:44:13,439][86122] Updated weights for policy 1, policy_version 38570 (0.0007) +[2023-10-09 13:44:13,799][86122] Updated weights for policy 1, policy_version 38580 (0.0008) +[2023-10-09 13:44:14,166][86122] Updated weights for policy 1, policy_version 38590 (0.0008) +[2023-10-09 13:44:16,366][86121] Updated weights for policy 0, policy_version 38410 (0.0008) +[2023-10-09 13:44:16,732][86121] Updated weights for policy 0, policy_version 38420 (0.0010) +[2023-10-09 13:44:17,100][86121] Updated weights for policy 0, policy_version 38430 (0.0009) +[2023-10-09 13:44:17,773][86122] Updated weights for policy 1, policy_version 38600 (0.0008) +[2023-10-09 13:44:18,135][86122] Updated weights for policy 1, policy_version 38610 (0.0007) +[2023-10-09 13:44:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 78872576. Throughput: 0: 1823.8, 1: 1816.1. Samples: 19728276. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) +[2023-10-09 13:44:18,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.950')] +[2023-10-09 13:44:18,503][86122] Updated weights for policy 1, policy_version 38620 (0.0010) +[2023-10-09 13:44:20,578][86121] Updated weights for policy 0, policy_version 38440 (0.0008) +[2023-10-09 13:44:20,948][86121] Updated weights for policy 0, policy_version 38450 (0.0007) +[2023-10-09 13:44:21,313][86121] Updated weights for policy 0, policy_version 38460 (0.0007) +[2023-10-09 13:44:22,122][86122] Updated weights for policy 1, policy_version 38630 (0.0009) +[2023-10-09 13:44:22,495][86122] Updated weights for policy 1, policy_version 38640 (0.0007) +[2023-10-09 13:44:22,869][86122] Updated weights for policy 1, policy_version 38650 (0.0011) +[2023-10-09 13:44:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 78970880. Throughput: 0: 1817.7, 1: 1827.7. Samples: 19749896. Policy #0 lag: (min: 31.0, avg: 43.3, max: 63.0) +[2023-10-09 13:44:23,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.960')] +[2023-10-09 13:44:24,881][86121] Updated weights for policy 0, policy_version 38470 (0.0007) +[2023-10-09 13:44:25,247][86121] Updated weights for policy 0, policy_version 38480 (0.0008) +[2023-10-09 13:44:25,614][86121] Updated weights for policy 0, policy_version 38490 (0.0008) +[2023-10-09 13:44:26,460][86122] Updated weights for policy 1, policy_version 38660 (0.0009) +[2023-10-09 13:44:26,817][86122] Updated weights for policy 1, policy_version 38670 (0.0007) +[2023-10-09 13:44:27,186][86122] Updated weights for policy 1, policy_version 38680 (0.0008) +[2023-10-09 13:44:28,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79036416. Throughput: 0: 1817.0, 1: 1831.3. Samples: 19761136. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 13:44:28,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.970')] +[2023-10-09 13:44:29,274][86121] Updated weights for policy 0, policy_version 38500 (0.0008) +[2023-10-09 13:44:29,632][86121] Updated weights for policy 0, policy_version 38510 (0.0009) +[2023-10-09 13:44:29,998][86121] Updated weights for policy 0, policy_version 38520 (0.0007) +[2023-10-09 13:44:30,894][86122] Updated weights for policy 1, policy_version 38690 (0.0008) +[2023-10-09 13:44:31,257][86122] Updated weights for policy 1, policy_version 38700 (0.0008) +[2023-10-09 13:44:31,614][86122] Updated weights for policy 1, policy_version 38710 (0.0009) +[2023-10-09 13:44:31,981][86122] Updated weights for policy 1, policy_version 38720 (0.0009) +[2023-10-09 13:44:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79101952. Throughput: 0: 1820.6, 1: 1826.5. Samples: 19783030. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 13:44:33,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.970')] +[2023-10-09 13:44:33,733][86121] Updated weights for policy 0, policy_version 38530 (0.0008) +[2023-10-09 13:44:34,101][86121] Updated weights for policy 0, policy_version 38540 (0.0009) +[2023-10-09 13:44:34,477][86121] Updated weights for policy 0, policy_version 38550 (0.0007) +[2023-10-09 13:44:34,848][86121] Updated weights for policy 0, policy_version 38560 (0.0008) +[2023-10-09 13:44:35,896][86122] Updated weights for policy 1, policy_version 38730 (0.0009) +[2023-10-09 13:44:36,254][86122] Updated weights for policy 1, policy_version 38740 (0.0008) +[2023-10-09 13:44:36,620][86122] Updated weights for policy 1, policy_version 38750 (0.0008) +[2023-10-09 13:44:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79167488. Throughput: 0: 1819.2, 1: 1822.8. Samples: 19805310. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 13:44:38,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.970')] +[2023-10-09 13:44:38,585][86121] Updated weights for policy 0, policy_version 38570 (0.0009) +[2023-10-09 13:44:38,949][86121] Updated weights for policy 0, policy_version 38580 (0.0008) +[2023-10-09 13:44:39,317][86121] Updated weights for policy 0, policy_version 38590 (0.0009) +[2023-10-09 13:44:40,110][86122] Updated weights for policy 1, policy_version 38760 (0.0007) +[2023-10-09 13:44:40,463][86122] Updated weights for policy 1, policy_version 38770 (0.0010) +[2023-10-09 13:44:40,831][86122] Updated weights for policy 1, policy_version 38780 (0.0009) +[2023-10-09 13:44:42,842][86121] Updated weights for policy 0, policy_version 38600 (0.0008) +[2023-10-09 13:44:43,199][86121] Updated weights for policy 0, policy_version 38610 (0.0008) +[2023-10-09 13:44:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79233024. Throughput: 0: 1826.0, 1: 1822.6. Samples: 19815752. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 13:44:43,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 13:44:43,566][86121] Updated weights for policy 0, policy_version 38620 (0.0008) +[2023-10-09 13:44:44,445][86122] Updated weights for policy 1, policy_version 38790 (0.0009) +[2023-10-09 13:44:44,803][86122] Updated weights for policy 1, policy_version 38800 (0.0007) +[2023-10-09 13:44:45,167][86122] Updated weights for policy 1, policy_version 38810 (0.0008) +[2023-10-09 13:44:47,332][86121] Updated weights for policy 0, policy_version 38630 (0.0008) +[2023-10-09 13:44:47,705][86121] Updated weights for policy 0, policy_version 38640 (0.0009) +[2023-10-09 13:44:48,065][86121] Updated weights for policy 0, policy_version 38650 (0.0009) +[2023-10-09 13:44:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79331328. Throughput: 0: 1827.2, 1: 1830.0. Samples: 19838466. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 13:44:48,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 13:44:48,955][86122] Updated weights for policy 1, policy_version 38820 (0.0009) +[2023-10-09 13:44:49,310][86122] Updated weights for policy 1, policy_version 38830 (0.0009) +[2023-10-09 13:44:49,672][86122] Updated weights for policy 1, policy_version 38840 (0.0010) +[2023-10-09 13:44:51,737][86121] Updated weights for policy 0, policy_version 38660 (0.0008) +[2023-10-09 13:44:52,094][86121] Updated weights for policy 0, policy_version 38670 (0.0007) +[2023-10-09 13:44:52,457][86121] Updated weights for policy 0, policy_version 38680 (0.0007) +[2023-10-09 13:44:53,339][86122] Updated weights for policy 1, policy_version 38850 (0.0010) +[2023-10-09 13:44:53,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 79396864. Throughput: 0: 1826.2, 1: 1825.7. Samples: 19859856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:44:53,398][85186] Avg episode reward: [(0, '9.850'), (1, '9.960')] +[2023-10-09 13:44:53,698][86122] Updated weights for policy 1, policy_version 38860 (0.0011) +[2023-10-09 13:44:54,072][86122] Updated weights for policy 1, policy_version 38870 (0.0009) +[2023-10-09 13:44:54,432][86122] Updated weights for policy 1, policy_version 38880 (0.0009) +[2023-10-09 13:44:56,292][86121] Updated weights for policy 0, policy_version 38690 (0.0008) +[2023-10-09 13:44:56,662][86121] Updated weights for policy 0, policy_version 38700 (0.0009) +[2023-10-09 13:44:57,027][86121] Updated weights for policy 0, policy_version 38710 (0.0007) +[2023-10-09 13:44:57,393][86121] Updated weights for policy 0, policy_version 38720 (0.0008) +[2023-10-09 13:44:58,201][86122] Updated weights for policy 1, policy_version 38890 (0.0010) +[2023-10-09 13:44:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79462400. Throughput: 0: 1832.6, 1: 1824.3. Samples: 19871118. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:44:58,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.960')] +[2023-10-09 13:44:58,572][86122] Updated weights for policy 1, policy_version 38900 (0.0010) +[2023-10-09 13:44:58,934][86122] Updated weights for policy 1, policy_version 38910 (0.0012) +[2023-10-09 13:45:01,166][86121] Updated weights for policy 0, policy_version 38730 (0.0008) +[2023-10-09 13:45:01,541][86121] Updated weights for policy 0, policy_version 38740 (0.0009) +[2023-10-09 13:45:01,902][86121] Updated weights for policy 0, policy_version 38750 (0.0008) +[2023-10-09 13:45:02,728][86122] Updated weights for policy 1, policy_version 38920 (0.0010) +[2023-10-09 13:45:03,093][86122] Updated weights for policy 1, policy_version 38930 (0.0008) +[2023-10-09 13:45:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79527936. Throughput: 0: 1825.3, 1: 1822.2. Samples: 19892414. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:45:03,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.960')] +[2023-10-09 13:45:03,457][86122] Updated weights for policy 1, policy_version 38940 (0.0008) +[2023-10-09 13:45:05,583][86121] Updated weights for policy 0, policy_version 38760 (0.0010) +[2023-10-09 13:45:05,949][86121] Updated weights for policy 0, policy_version 38770 (0.0010) +[2023-10-09 13:45:06,314][86121] Updated weights for policy 0, policy_version 38780 (0.0011) +[2023-10-09 13:45:07,213][86122] Updated weights for policy 1, policy_version 38950 (0.0008) +[2023-10-09 13:45:07,578][86122] Updated weights for policy 1, policy_version 38960 (0.0009) +[2023-10-09 13:45:07,941][86122] Updated weights for policy 1, policy_version 38970 (0.0009) +[2023-10-09 13:45:08,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 79626240. Throughput: 0: 1829.5, 1: 1815.8. Samples: 19913934. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:45:08,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.950')] +[2023-10-09 13:45:09,913][86121] Updated weights for policy 0, policy_version 38790 (0.0009) +[2023-10-09 13:45:10,274][86121] Updated weights for policy 0, policy_version 38800 (0.0010) +[2023-10-09 13:45:10,639][86121] Updated weights for policy 0, policy_version 38810 (0.0010) +[2023-10-09 13:45:11,617][86122] Updated weights for policy 1, policy_version 38980 (0.0008) +[2023-10-09 13:45:11,986][86122] Updated weights for policy 1, policy_version 38990 (0.0007) +[2023-10-09 13:45:12,350][86122] Updated weights for policy 1, policy_version 39000 (0.0009) +[2023-10-09 13:45:13,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 79691776. Throughput: 0: 1826.1, 1: 1811.2. Samples: 19924814. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:45:13,399][85186] Avg episode reward: [(0, '9.860'), (1, '9.950')] +[2023-10-09 13:45:14,359][86121] Updated weights for policy 0, policy_version 38820 (0.0011) +[2023-10-09 13:45:14,736][86121] Updated weights for policy 0, policy_version 38830 (0.0008) +[2023-10-09 13:45:15,096][86121] Updated weights for policy 0, policy_version 38840 (0.0007) +[2023-10-09 13:45:16,006][86122] Updated weights for policy 1, policy_version 39010 (0.0008) +[2023-10-09 13:45:16,377][86122] Updated weights for policy 1, policy_version 39020 (0.0007) +[2023-10-09 13:45:16,734][86122] Updated weights for policy 1, policy_version 39030 (0.0009) +[2023-10-09 13:45:17,097][86122] Updated weights for policy 1, policy_version 39040 (0.0009) +[2023-10-09 13:45:18,398][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 79757312. Throughput: 0: 1822.6, 1: 1814.8. Samples: 19946714. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:45:18,399][85186] Avg episode reward: [(0, '9.870'), (1, '9.950')] +[2023-10-09 13:45:18,785][86121] Updated weights for policy 0, policy_version 38850 (0.0008) +[2023-10-09 13:45:19,157][86121] Updated weights for policy 0, policy_version 38860 (0.0008) +[2023-10-09 13:45:19,518][86121] Updated weights for policy 0, policy_version 38870 (0.0008) +[2023-10-09 13:45:19,887][86121] Updated weights for policy 0, policy_version 38880 (0.0009) +[2023-10-09 13:45:20,904][86122] Updated weights for policy 1, policy_version 39050 (0.0008) +[2023-10-09 13:45:21,268][86122] Updated weights for policy 1, policy_version 39060 (0.0009) +[2023-10-09 13:45:21,628][86122] Updated weights for policy 1, policy_version 39070 (0.0011) +[2023-10-09 13:45:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 79822848. Throughput: 0: 1817.1, 1: 1813.6. Samples: 19968690. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:45:23,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.950')] +[2023-10-09 13:45:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000039072_40009728.pth... +[2023-10-09 13:45:23,440][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000037376_38273024.pth +[2023-10-09 13:45:23,658][86121] Updated weights for policy 0, policy_version 38890 (0.0010) +[2023-10-09 13:45:24,029][86121] Updated weights for policy 0, policy_version 38900 (0.0010) +[2023-10-09 13:45:24,393][86121] Updated weights for policy 0, policy_version 38910 (0.0011) +[2023-10-09 13:45:24,465][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000038912_39845888.pth... +[2023-10-09 13:45:24,503][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000037184_38076416.pth +[2023-10-09 13:45:25,295][86122] Updated weights for policy 1, policy_version 39080 (0.0010) +[2023-10-09 13:45:25,660][86122] Updated weights for policy 1, policy_version 39090 (0.0008) +[2023-10-09 13:45:26,019][86122] Updated weights for policy 1, policy_version 39100 (0.0009) +[2023-10-09 13:45:27,976][86121] Updated weights for policy 0, policy_version 38920 (0.0008) +[2023-10-09 13:45:28,341][86121] Updated weights for policy 0, policy_version 38930 (0.0007) +[2023-10-09 13:45:28,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 79888384. Throughput: 0: 1816.4, 1: 1816.1. Samples: 19979214. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:45:28,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.950')] +[2023-10-09 13:45:28,708][86121] Updated weights for policy 0, policy_version 38940 (0.0008) +[2023-10-09 13:45:29,683][86122] Updated weights for policy 1, policy_version 39110 (0.0008) +[2023-10-09 13:45:30,049][86122] Updated weights for policy 1, policy_version 39120 (0.0008) +[2023-10-09 13:45:30,426][86122] Updated weights for policy 1, policy_version 39130 (0.0007) +[2023-10-09 13:45:32,425][86121] Updated weights for policy 0, policy_version 38950 (0.0009) +[2023-10-09 13:45:32,790][86121] Updated weights for policy 0, policy_version 38960 (0.0008) +[2023-10-09 13:45:33,163][86121] Updated weights for policy 0, policy_version 38970 (0.0007) +[2023-10-09 13:45:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 79986688. Throughput: 0: 1811.2, 1: 1810.2. Samples: 20001430. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:45:33,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.950')] +[2023-10-09 13:45:34,131][86122] Updated weights for policy 1, policy_version 39140 (0.0007) +[2023-10-09 13:45:34,495][86122] Updated weights for policy 1, policy_version 39150 (0.0008) +[2023-10-09 13:45:34,861][86122] Updated weights for policy 1, policy_version 39160 (0.0007) +[2023-10-09 13:45:36,911][86121] Updated weights for policy 0, policy_version 38980 (0.0008) +[2023-10-09 13:45:37,278][86121] Updated weights for policy 0, policy_version 38990 (0.0009) +[2023-10-09 13:45:37,648][86121] Updated weights for policy 0, policy_version 39000 (0.0011) +[2023-10-09 13:45:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80052224. Throughput: 0: 1810.0, 1: 1815.0. Samples: 20022980. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:45:38,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.940')] +[2023-10-09 13:45:38,585][86122] Updated weights for policy 1, policy_version 39170 (0.0010) +[2023-10-09 13:45:38,939][86122] Updated weights for policy 1, policy_version 39180 (0.0007) +[2023-10-09 13:45:39,303][86122] Updated weights for policy 1, policy_version 39190 (0.0008) +[2023-10-09 13:45:39,660][86122] Updated weights for policy 1, policy_version 39200 (0.0012) +[2023-10-09 13:45:41,502][86121] Updated weights for policy 0, policy_version 39010 (0.0010) +[2023-10-09 13:45:41,907][86121] Updated weights for policy 0, policy_version 39020 (0.0009) +[2023-10-09 13:45:42,271][86121] Updated weights for policy 0, policy_version 39030 (0.0008) +[2023-10-09 13:45:42,631][86121] Updated weights for policy 0, policy_version 39040 (0.0008) +[2023-10-09 13:45:43,306][86122] Updated weights for policy 1, policy_version 39210 (0.0010) +[2023-10-09 13:45:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80117760. Throughput: 0: 1803.3, 1: 1818.8. Samples: 20034112. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 13:45:43,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.930')] +[2023-10-09 13:45:43,669][86122] Updated weights for policy 1, policy_version 39220 (0.0011) +[2023-10-09 13:45:44,040][86122] Updated weights for policy 1, policy_version 39230 (0.0008) +[2023-10-09 13:45:46,116][86121] Updated weights for policy 0, policy_version 39050 (0.0011) +[2023-10-09 13:45:46,482][86121] Updated weights for policy 0, policy_version 39060 (0.0010) +[2023-10-09 13:45:46,849][86121] Updated weights for policy 0, policy_version 39070 (0.0011) +[2023-10-09 13:45:47,636][86122] Updated weights for policy 1, policy_version 39240 (0.0007) +[2023-10-09 13:45:47,995][86122] Updated weights for policy 1, policy_version 39250 (0.0008) +[2023-10-09 13:45:48,350][86122] Updated weights for policy 1, policy_version 39260 (0.0009) +[2023-10-09 13:45:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80183296. Throughput: 0: 1807.6, 1: 1821.8. Samples: 20055740. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:45:48,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.920')] +[2023-10-09 13:45:50,553][86121] Updated weights for policy 0, policy_version 39080 (0.0008) +[2023-10-09 13:45:50,923][86121] Updated weights for policy 0, policy_version 39090 (0.0009) +[2023-10-09 13:45:51,280][86121] Updated weights for policy 0, policy_version 39100 (0.0007) +[2023-10-09 13:45:52,017][86122] Updated weights for policy 1, policy_version 39270 (0.0009) +[2023-10-09 13:45:52,383][86122] Updated weights for policy 1, policy_version 39280 (0.0008) +[2023-10-09 13:45:52,749][86122] Updated weights for policy 1, policy_version 39290 (0.0007) +[2023-10-09 13:45:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80281600. Throughput: 0: 1811.3, 1: 1823.2. Samples: 20077488. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:45:53,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.920')] +[2023-10-09 13:45:54,929][86121] Updated weights for policy 0, policy_version 39110 (0.0009) +[2023-10-09 13:45:55,292][86121] Updated weights for policy 0, policy_version 39120 (0.0007) +[2023-10-09 13:45:55,654][86121] Updated weights for policy 0, policy_version 39130 (0.0009) +[2023-10-09 13:45:56,454][86122] Updated weights for policy 1, policy_version 39300 (0.0008) +[2023-10-09 13:45:56,823][86122] Updated weights for policy 1, policy_version 39310 (0.0008) +[2023-10-09 13:45:57,178][86122] Updated weights for policy 1, policy_version 39320 (0.0007) +[2023-10-09 13:45:58,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80347136. Throughput: 0: 1816.0, 1: 1830.8. Samples: 20088920. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:45:58,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.920')] +[2023-10-09 13:45:59,444][86121] Updated weights for policy 0, policy_version 39140 (0.0009) +[2023-10-09 13:45:59,811][86121] Updated weights for policy 0, policy_version 39150 (0.0011) +[2023-10-09 13:46:00,177][86121] Updated weights for policy 0, policy_version 39160 (0.0010) +[2023-10-09 13:46:00,837][86122] Updated weights for policy 1, policy_version 39330 (0.0008) +[2023-10-09 13:46:01,198][86122] Updated weights for policy 1, policy_version 39340 (0.0009) +[2023-10-09 13:46:01,569][86122] Updated weights for policy 1, policy_version 39350 (0.0008) +[2023-10-09 13:46:01,922][86122] Updated weights for policy 1, policy_version 39360 (0.0010) +[2023-10-09 13:46:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80412672. Throughput: 0: 1809.1, 1: 1822.9. Samples: 20110154. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:46:03,398][85186] Avg episode reward: [(0, '9.790'), (1, '9.930')] +[2023-10-09 13:46:03,888][86121] Updated weights for policy 0, policy_version 39170 (0.0010) +[2023-10-09 13:46:04,256][86121] Updated weights for policy 0, policy_version 39180 (0.0008) +[2023-10-09 13:46:04,626][86121] Updated weights for policy 0, policy_version 39190 (0.0008) +[2023-10-09 13:46:04,989][86121] Updated weights for policy 0, policy_version 39200 (0.0007) +[2023-10-09 13:46:05,475][86122] Updated weights for policy 1, policy_version 39370 (0.0010) +[2023-10-09 13:46:05,838][86122] Updated weights for policy 1, policy_version 39380 (0.0009) +[2023-10-09 13:46:06,202][86122] Updated weights for policy 1, policy_version 39390 (0.0011) +[2023-10-09 13:46:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80478208. Throughput: 0: 1811.7, 1: 1832.9. Samples: 20132694. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:46:08,398][85186] Avg episode reward: [(0, '9.780'), (1, '9.910')] +[2023-10-09 13:46:08,749][86121] Updated weights for policy 0, policy_version 39210 (0.0008) +[2023-10-09 13:46:09,119][86121] Updated weights for policy 0, policy_version 39220 (0.0007) +[2023-10-09 13:46:09,475][86121] Updated weights for policy 0, policy_version 39230 (0.0009) +[2023-10-09 13:46:09,809][86122] Updated weights for policy 1, policy_version 39400 (0.0011) +[2023-10-09 13:46:10,176][86122] Updated weights for policy 1, policy_version 39410 (0.0009) +[2023-10-09 13:46:10,543][86122] Updated weights for policy 1, policy_version 39420 (0.0010) +[2023-10-09 13:46:13,087][86121] Updated weights for policy 0, policy_version 39240 (0.0009) +[2023-10-09 13:46:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 80543744. Throughput: 0: 1813.3, 1: 1824.5. Samples: 20142914. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 13:46:13,398][85186] Avg episode reward: [(0, '9.770'), (1, '9.910')] +[2023-10-09 13:46:13,454][86121] Updated weights for policy 0, policy_version 39250 (0.0010) +[2023-10-09 13:46:13,814][86121] Updated weights for policy 0, policy_version 39260 (0.0011) +[2023-10-09 13:46:14,070][86122] Updated weights for policy 1, policy_version 39430 (0.0008) +[2023-10-09 13:46:14,438][86122] Updated weights for policy 1, policy_version 39440 (0.0008) +[2023-10-09 13:46:14,797][86122] Updated weights for policy 1, policy_version 39450 (0.0008) +[2023-10-09 13:46:17,550][86121] Updated weights for policy 0, policy_version 39270 (0.0009) +[2023-10-09 13:46:17,918][86121] Updated weights for policy 0, policy_version 39280 (0.0007) +[2023-10-09 13:46:18,280][86121] Updated weights for policy 0, policy_version 39290 (0.0010) +[2023-10-09 13:46:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 80609280. Throughput: 0: 1810.1, 1: 1838.2. Samples: 20165604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:46:18,398][85186] Avg episode reward: [(0, '9.780'), (1, '9.910')] +[2023-10-09 13:46:18,544][86122] Updated weights for policy 1, policy_version 39460 (0.0008) +[2023-10-09 13:46:18,901][86122] Updated weights for policy 1, policy_version 39470 (0.0009) +[2023-10-09 13:46:19,264][86122] Updated weights for policy 1, policy_version 39480 (0.0009) +[2023-10-09 13:46:21,994][86121] Updated weights for policy 0, policy_version 39300 (0.0008) +[2023-10-09 13:46:22,359][86121] Updated weights for policy 0, policy_version 39310 (0.0009) +[2023-10-09 13:46:22,730][86121] Updated weights for policy 0, policy_version 39320 (0.0007) +[2023-10-09 13:46:22,963][86122] Updated weights for policy 1, policy_version 39490 (0.0010) +[2023-10-09 13:46:23,319][86122] Updated weights for policy 1, policy_version 39500 (0.0009) +[2023-10-09 13:46:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80707584. Throughput: 0: 1813.2, 1: 1828.9. Samples: 20186878. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:46:23,398][85186] Avg episode reward: [(0, '9.780'), (1, '9.900')] +[2023-10-09 13:46:23,687][86122] Updated weights for policy 1, policy_version 39510 (0.0008) +[2023-10-09 13:46:24,044][86122] Updated weights for policy 1, policy_version 39520 (0.0008) +[2023-10-09 13:46:26,578][86121] Updated weights for policy 0, policy_version 39330 (0.0007) +[2023-10-09 13:46:26,979][86121] Updated weights for policy 0, policy_version 39340 (0.0008) +[2023-10-09 13:46:27,335][86121] Updated weights for policy 0, policy_version 39350 (0.0010) +[2023-10-09 13:46:27,699][86121] Updated weights for policy 0, policy_version 39360 (0.0009) +[2023-10-09 13:46:27,795][86122] Updated weights for policy 1, policy_version 39530 (0.0008) +[2023-10-09 13:46:28,154][86122] Updated weights for policy 1, policy_version 39540 (0.0007) +[2023-10-09 13:46:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 80773120. Throughput: 0: 1814.4, 1: 1830.1. Samples: 20198112. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:46:28,398][85186] Avg episode reward: [(0, '9.770'), (1, '9.880')] +[2023-10-09 13:46:28,515][86122] Updated weights for policy 1, policy_version 39550 (0.0007) +[2023-10-09 13:46:31,337][86121] Updated weights for policy 0, policy_version 39370 (0.0010) +[2023-10-09 13:46:31,712][86121] Updated weights for policy 0, policy_version 39380 (0.0010) +[2023-10-09 13:46:32,077][86121] Updated weights for policy 0, policy_version 39390 (0.0007) +[2023-10-09 13:46:32,428][86122] Updated weights for policy 1, policy_version 39560 (0.0007) +[2023-10-09 13:46:32,796][86122] Updated weights for policy 1, policy_version 39570 (0.0007) +[2023-10-09 13:46:33,151][86122] Updated weights for policy 1, policy_version 39580 (0.0008) +[2023-10-09 13:46:33,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 80871424. Throughput: 0: 1809.2, 1: 1830.2. Samples: 20219516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:46:33,398][85186] Avg episode reward: [(0, '9.790'), (1, '9.890')] +[2023-10-09 13:46:35,716][86121] Updated weights for policy 0, policy_version 39400 (0.0007) +[2023-10-09 13:46:36,087][86121] Updated weights for policy 0, policy_version 39410 (0.0008) +[2023-10-09 13:46:36,451][86121] Updated weights for policy 0, policy_version 39420 (0.0008) +[2023-10-09 13:46:36,827][86122] Updated weights for policy 1, policy_version 39590 (0.0008) +[2023-10-09 13:46:37,187][86122] Updated weights for policy 1, policy_version 39600 (0.0008) +[2023-10-09 13:46:37,549][86122] Updated weights for policy 1, policy_version 39610 (0.0009) +[2023-10-09 13:46:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 80936960. Throughput: 0: 1803.3, 1: 1823.6. Samples: 20240698. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:46:38,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.900')] +[2023-10-09 13:46:40,278][86121] Updated weights for policy 0, policy_version 39430 (0.0007) +[2023-10-09 13:46:40,648][86121] Updated weights for policy 0, policy_version 39440 (0.0007) +[2023-10-09 13:46:41,014][86121] Updated weights for policy 0, policy_version 39450 (0.0009) +[2023-10-09 13:46:41,422][86122] Updated weights for policy 1, policy_version 39620 (0.0009) +[2023-10-09 13:46:41,790][86122] Updated weights for policy 1, policy_version 39630 (0.0009) +[2023-10-09 13:46:42,149][86122] Updated weights for policy 1, policy_version 39640 (0.0007) +[2023-10-09 13:46:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81002496. Throughput: 0: 1812.7, 1: 1821.0. Samples: 20252438. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:46:43,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.900')] +[2023-10-09 13:46:44,862][86121] Updated weights for policy 0, policy_version 39460 (0.0008) +[2023-10-09 13:46:45,221][86121] Updated weights for policy 0, policy_version 39470 (0.0008) +[2023-10-09 13:46:45,588][86121] Updated weights for policy 0, policy_version 39480 (0.0008) +[2023-10-09 13:46:46,039][86122] Updated weights for policy 1, policy_version 39650 (0.0008) +[2023-10-09 13:46:46,411][86122] Updated weights for policy 1, policy_version 39660 (0.0007) +[2023-10-09 13:46:46,769][86122] Updated weights for policy 1, policy_version 39670 (0.0009) +[2023-10-09 13:46:47,134][86122] Updated weights for policy 1, policy_version 39680 (0.0009) +[2023-10-09 13:46:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81068032. Throughput: 0: 1807.7, 1: 1824.1. Samples: 20273586. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:46:48,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.900')] +[2023-10-09 13:46:49,040][86121] Updated weights for policy 0, policy_version 39490 (0.0008) +[2023-10-09 13:46:49,407][86121] Updated weights for policy 0, policy_version 39500 (0.0007) +[2023-10-09 13:46:49,786][86121] Updated weights for policy 0, policy_version 39510 (0.0009) +[2023-10-09 13:46:50,142][86121] Updated weights for policy 0, policy_version 39520 (0.0007) +[2023-10-09 13:46:50,868][86122] Updated weights for policy 1, policy_version 39690 (0.0009) +[2023-10-09 13:46:51,228][86122] Updated weights for policy 1, policy_version 39700 (0.0008) +[2023-10-09 13:46:51,590][86122] Updated weights for policy 1, policy_version 39710 (0.0011) +[2023-10-09 13:46:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 81133568. Throughput: 0: 1816.9, 1: 1813.1. Samples: 20296048. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:46:53,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.900')] +[2023-10-09 13:46:53,867][86121] Updated weights for policy 0, policy_version 39530 (0.0010) +[2023-10-09 13:46:54,241][86121] Updated weights for policy 0, policy_version 39540 (0.0010) +[2023-10-09 13:46:54,605][86121] Updated weights for policy 0, policy_version 39550 (0.0009) +[2023-10-09 13:46:55,286][86122] Updated weights for policy 1, policy_version 39720 (0.0008) +[2023-10-09 13:46:55,653][86122] Updated weights for policy 1, policy_version 39730 (0.0009) +[2023-10-09 13:46:56,020][86122] Updated weights for policy 1, policy_version 39740 (0.0009) +[2023-10-09 13:46:58,314][86121] Updated weights for policy 0, policy_version 39560 (0.0009) +[2023-10-09 13:46:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81199104. Throughput: 0: 1815.8, 1: 1824.2. Samples: 20306714. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:46:58,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.900')] +[2023-10-09 13:46:58,674][86121] Updated weights for policy 0, policy_version 39570 (0.0010) +[2023-10-09 13:46:59,040][86121] Updated weights for policy 0, policy_version 39580 (0.0009) +[2023-10-09 13:46:59,566][86122] Updated weights for policy 1, policy_version 39750 (0.0012) +[2023-10-09 13:46:59,920][86122] Updated weights for policy 1, policy_version 39760 (0.0008) +[2023-10-09 13:47:00,293][86122] Updated weights for policy 1, policy_version 39770 (0.0009) +[2023-10-09 13:47:02,778][86121] Updated weights for policy 0, policy_version 39590 (0.0007) +[2023-10-09 13:47:03,139][86121] Updated weights for policy 0, policy_version 39600 (0.0007) +[2023-10-09 13:47:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 81264640. Throughput: 0: 1818.8, 1: 1816.0. Samples: 20329168. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 13:47:03,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.900')] +[2023-10-09 13:47:03,501][86121] Updated weights for policy 0, policy_version 39610 (0.0007) +[2023-10-09 13:47:03,856][86122] Updated weights for policy 1, policy_version 39780 (0.0009) +[2023-10-09 13:47:04,227][86122] Updated weights for policy 1, policy_version 39790 (0.0009) +[2023-10-09 13:47:04,589][86122] Updated weights for policy 1, policy_version 39800 (0.0009) +[2023-10-09 13:47:07,246][86121] Updated weights for policy 0, policy_version 39620 (0.0007) +[2023-10-09 13:47:07,613][86121] Updated weights for policy 0, policy_version 39630 (0.0009) +[2023-10-09 13:47:07,989][86121] Updated weights for policy 0, policy_version 39640 (0.0009) +[2023-10-09 13:47:08,308][86122] Updated weights for policy 1, policy_version 39810 (0.0009) +[2023-10-09 13:47:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81362944. Throughput: 0: 1825.3, 1: 1824.5. Samples: 20351120. Policy #0 lag: (min: 13.0, avg: 13.2, max: 22.0) +[2023-10-09 13:47:08,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.890')] +[2023-10-09 13:47:08,672][86122] Updated weights for policy 1, policy_version 39820 (0.0007) +[2023-10-09 13:47:09,024][86122] Updated weights for policy 1, policy_version 39830 (0.0010) +[2023-10-09 13:47:09,389][86122] Updated weights for policy 1, policy_version 39840 (0.0008) +[2023-10-09 13:47:11,640][86121] Updated weights for policy 0, policy_version 39650 (0.0008) +[2023-10-09 13:47:12,034][86121] Updated weights for policy 0, policy_version 39660 (0.0007) +[2023-10-09 13:47:12,404][86121] Updated weights for policy 0, policy_version 39670 (0.0007) +[2023-10-09 13:47:12,769][86121] Updated weights for policy 0, policy_version 39680 (0.0007) +[2023-10-09 13:47:13,124][86122] Updated weights for policy 1, policy_version 39850 (0.0011) +[2023-10-09 13:47:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81428480. Throughput: 0: 1816.4, 1: 1824.4. Samples: 20361950. Policy #0 lag: (min: 13.0, avg: 13.2, max: 22.0) +[2023-10-09 13:47:13,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.890')] +[2023-10-09 13:47:13,482][86122] Updated weights for policy 1, policy_version 39860 (0.0011) +[2023-10-09 13:47:13,844][86122] Updated weights for policy 1, policy_version 39870 (0.0012) +[2023-10-09 13:47:16,372][86121] Updated weights for policy 0, policy_version 39690 (0.0007) +[2023-10-09 13:47:16,739][86121] Updated weights for policy 0, policy_version 39700 (0.0010) +[2023-10-09 13:47:17,103][86121] Updated weights for policy 0, policy_version 39710 (0.0008) +[2023-10-09 13:47:17,566][86122] Updated weights for policy 1, policy_version 39880 (0.0008) +[2023-10-09 13:47:17,920][86122] Updated weights for policy 1, policy_version 39890 (0.0007) +[2023-10-09 13:47:18,277][86122] Updated weights for policy 1, policy_version 39900 (0.0009) +[2023-10-09 13:47:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 81494016. Throughput: 0: 1822.9, 1: 1822.2. Samples: 20383544. Policy #0 lag: (min: 13.0, avg: 13.2, max: 22.0) +[2023-10-09 13:47:18,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.890')] +[2023-10-09 13:47:20,980][86121] Updated weights for policy 0, policy_version 39720 (0.0007) +[2023-10-09 13:47:21,350][86121] Updated weights for policy 0, policy_version 39730 (0.0008) +[2023-10-09 13:47:21,710][86121] Updated weights for policy 0, policy_version 39740 (0.0008) +[2023-10-09 13:47:22,017][86122] Updated weights for policy 1, policy_version 39910 (0.0008) +[2023-10-09 13:47:22,374][86122] Updated weights for policy 1, policy_version 39920 (0.0008) +[2023-10-09 13:47:22,731][86122] Updated weights for policy 1, policy_version 39930 (0.0008) +[2023-10-09 13:47:23,398][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81592320. Throughput: 0: 1821.1, 1: 1825.4. Samples: 20404790. Policy #0 lag: (min: 13.0, avg: 13.2, max: 22.0) +[2023-10-09 13:47:23,399][85186] Avg episode reward: [(0, '9.820'), (1, '9.890')] +[2023-10-09 13:47:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000039936_40894464.pth... +[2023-10-09 13:47:23,411][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000039744_40697856.pth... +[2023-10-09 13:47:23,453][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000038240_39157760.pth +[2023-10-09 13:47:23,453][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000038048_38961152.pth +[2023-10-09 13:47:25,325][86121] Updated weights for policy 0, policy_version 39750 (0.0009) +[2023-10-09 13:47:25,696][86121] Updated weights for policy 0, policy_version 39760 (0.0008) +[2023-10-09 13:47:26,064][86121] Updated weights for policy 0, policy_version 39770 (0.0009) +[2023-10-09 13:47:26,332][86122] Updated weights for policy 1, policy_version 39940 (0.0007) +[2023-10-09 13:47:26,692][86122] Updated weights for policy 1, policy_version 39950 (0.0010) +[2023-10-09 13:47:27,051][86122] Updated weights for policy 1, policy_version 39960 (0.0009) +[2023-10-09 13:47:28,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 81657856. Throughput: 0: 1816.7, 1: 1827.5. Samples: 20416428. Policy #0 lag: (min: 13.0, avg: 13.2, max: 22.0) +[2023-10-09 13:47:28,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.890')] +[2023-10-09 13:47:29,655][86121] Updated weights for policy 0, policy_version 39780 (0.0007) +[2023-10-09 13:47:30,025][86121] Updated weights for policy 0, policy_version 39790 (0.0008) +[2023-10-09 13:47:30,389][86121] Updated weights for policy 0, policy_version 39800 (0.0011) +[2023-10-09 13:47:30,826][86122] Updated weights for policy 1, policy_version 39970 (0.0009) +[2023-10-09 13:47:31,202][86122] Updated weights for policy 1, policy_version 39980 (0.0010) +[2023-10-09 13:47:31,567][86122] Updated weights for policy 1, policy_version 39990 (0.0007) +[2023-10-09 13:47:31,935][86122] Updated weights for policy 1, policy_version 40000 (0.0008) +[2023-10-09 13:47:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 81723392. Throughput: 0: 1814.4, 1: 1825.5. Samples: 20437384. Policy #0 lag: (min: 13.0, avg: 13.2, max: 22.0) +[2023-10-09 13:47:33,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.900')] +[2023-10-09 13:47:34,104][86121] Updated weights for policy 0, policy_version 39810 (0.0010) +[2023-10-09 13:47:34,478][86121] Updated weights for policy 0, policy_version 39820 (0.0009) +[2023-10-09 13:47:34,843][86121] Updated weights for policy 0, policy_version 39830 (0.0009) +[2023-10-09 13:47:35,209][86121] Updated weights for policy 0, policy_version 39840 (0.0007) +[2023-10-09 13:47:35,675][86122] Updated weights for policy 1, policy_version 40010 (0.0009) +[2023-10-09 13:47:36,036][86122] Updated weights for policy 1, policy_version 40020 (0.0010) +[2023-10-09 13:47:36,409][86122] Updated weights for policy 1, policy_version 40030 (0.0009) +[2023-10-09 13:47:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 81788928. Throughput: 0: 1812.1, 1: 1833.9. Samples: 20460118. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:47:38,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.910')] +[2023-10-09 13:47:38,977][86121] Updated weights for policy 0, policy_version 39850 (0.0007) +[2023-10-09 13:47:39,341][86121] Updated weights for policy 0, policy_version 39860 (0.0008) +[2023-10-09 13:47:39,712][86121] Updated weights for policy 0, policy_version 39870 (0.0008) +[2023-10-09 13:47:39,946][86122] Updated weights for policy 1, policy_version 40040 (0.0007) +[2023-10-09 13:47:40,306][86122] Updated weights for policy 1, policy_version 40050 (0.0007) +[2023-10-09 13:47:40,663][86122] Updated weights for policy 1, policy_version 40060 (0.0008) +[2023-10-09 13:47:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 81854464. Throughput: 0: 1810.1, 1: 1825.3. Samples: 20470308. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:47:43,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.930')] +[2023-10-09 13:47:43,427][86121] Updated weights for policy 0, policy_version 39880 (0.0007) +[2023-10-09 13:47:43,800][86121] Updated weights for policy 0, policy_version 39890 (0.0008) +[2023-10-09 13:47:44,159][86121] Updated weights for policy 0, policy_version 39900 (0.0007) +[2023-10-09 13:47:44,318][86122] Updated weights for policy 1, policy_version 40070 (0.0008) +[2023-10-09 13:47:44,677][86122] Updated weights for policy 1, policy_version 40080 (0.0010) +[2023-10-09 13:47:45,039][86122] Updated weights for policy 1, policy_version 40090 (0.0010) +[2023-10-09 13:47:47,800][86121] Updated weights for policy 0, policy_version 39910 (0.0008) +[2023-10-09 13:47:48,161][86121] Updated weights for policy 0, policy_version 39920 (0.0008) +[2023-10-09 13:47:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 81920000. Throughput: 0: 1816.3, 1: 1832.6. Samples: 20493368. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:47:48,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.930')] +[2023-10-09 13:47:48,539][86121] Updated weights for policy 0, policy_version 39930 (0.0009) +[2023-10-09 13:47:48,649][86122] Updated weights for policy 1, policy_version 40100 (0.0008) +[2023-10-09 13:47:49,010][86122] Updated weights for policy 1, policy_version 40110 (0.0008) +[2023-10-09 13:47:49,381][86122] Updated weights for policy 1, policy_version 40120 (0.0008) +[2023-10-09 13:47:52,347][86121] Updated weights for policy 0, policy_version 39940 (0.0009) +[2023-10-09 13:47:52,710][86121] Updated weights for policy 0, policy_version 39950 (0.0008) +[2023-10-09 13:47:53,074][86121] Updated weights for policy 0, policy_version 39960 (0.0007) +[2023-10-09 13:47:53,163][86122] Updated weights for policy 1, policy_version 40130 (0.0007) +[2023-10-09 13:47:53,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 82018304. Throughput: 0: 1818.8, 1: 1827.8. Samples: 20515216. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:47:53,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.930')] +[2023-10-09 13:47:53,530][86122] Updated weights for policy 1, policy_version 40140 (0.0008) +[2023-10-09 13:47:53,879][86122] Updated weights for policy 1, policy_version 40150 (0.0009) +[2023-10-09 13:47:54,240][86122] Updated weights for policy 1, policy_version 40160 (0.0007) +[2023-10-09 13:47:56,845][86121] Updated weights for policy 0, policy_version 39970 (0.0008) +[2023-10-09 13:47:57,245][86121] Updated weights for policy 0, policy_version 39980 (0.0009) +[2023-10-09 13:47:57,611][86121] Updated weights for policy 0, policy_version 39990 (0.0008) +[2023-10-09 13:47:57,871][86122] Updated weights for policy 1, policy_version 40170 (0.0009) +[2023-10-09 13:47:57,979][86121] Updated weights for policy 0, policy_version 40000 (0.0008) +[2023-10-09 13:47:58,227][86122] Updated weights for policy 1, policy_version 40180 (0.0008) +[2023-10-09 13:47:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82083840. Throughput: 0: 1817.7, 1: 1827.6. Samples: 20525986. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 13:47:58,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.930')] +[2023-10-09 13:47:58,593][86122] Updated weights for policy 1, policy_version 40190 (0.0007) +[2023-10-09 13:48:01,639][86121] Updated weights for policy 0, policy_version 40010 (0.0009) +[2023-10-09 13:48:02,013][86121] Updated weights for policy 0, policy_version 40020 (0.0009) +[2023-10-09 13:48:02,272][86122] Updated weights for policy 1, policy_version 40200 (0.0009) +[2023-10-09 13:48:02,381][86121] Updated weights for policy 0, policy_version 40030 (0.0009) +[2023-10-09 13:48:02,639][86122] Updated weights for policy 1, policy_version 40210 (0.0009) +[2023-10-09 13:48:03,002][86122] Updated weights for policy 1, policy_version 40220 (0.0007) +[2023-10-09 13:48:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 82182144. Throughput: 0: 1825.2, 1: 1830.0. Samples: 20548024. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:48:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.940')] +[2023-10-09 13:48:06,100][86121] Updated weights for policy 0, policy_version 40040 (0.0007) +[2023-10-09 13:48:06,477][86121] Updated weights for policy 0, policy_version 40050 (0.0007) +[2023-10-09 13:48:06,799][86122] Updated weights for policy 1, policy_version 40230 (0.0007) +[2023-10-09 13:48:06,843][86121] Updated weights for policy 0, policy_version 40060 (0.0009) +[2023-10-09 13:48:07,149][86122] Updated weights for policy 1, policy_version 40240 (0.0008) +[2023-10-09 13:48:07,510][86122] Updated weights for policy 1, policy_version 40250 (0.0007) +[2023-10-09 13:48:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82247680. Throughput: 0: 1813.9, 1: 1825.3. Samples: 20568550. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:48:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 13:48:10,579][86121] Updated weights for policy 0, policy_version 40070 (0.0009) +[2023-10-09 13:48:10,953][86121] Updated weights for policy 0, policy_version 40080 (0.0007) +[2023-10-09 13:48:11,170][86122] Updated weights for policy 1, policy_version 40260 (0.0008) +[2023-10-09 13:48:11,309][86121] Updated weights for policy 0, policy_version 40090 (0.0007) +[2023-10-09 13:48:11,533][86122] Updated weights for policy 1, policy_version 40270 (0.0008) +[2023-10-09 13:48:11,899][86122] Updated weights for policy 1, policy_version 40280 (0.0007) +[2023-10-09 13:48:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 82313216. Throughput: 0: 1823.5, 1: 1831.5. Samples: 20580904. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:48:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.950')] +[2023-10-09 13:48:15,017][86121] Updated weights for policy 0, policy_version 40100 (0.0009) +[2023-10-09 13:48:15,385][86121] Updated weights for policy 0, policy_version 40110 (0.0008) +[2023-10-09 13:48:15,527][86122] Updated weights for policy 1, policy_version 40290 (0.0007) +[2023-10-09 13:48:15,750][86121] Updated weights for policy 0, policy_version 40120 (0.0007) +[2023-10-09 13:48:15,891][86122] Updated weights for policy 1, policy_version 40300 (0.0008) +[2023-10-09 13:48:16,253][86122] Updated weights for policy 1, policy_version 40310 (0.0010) +[2023-10-09 13:48:16,614][86122] Updated weights for policy 1, policy_version 40320 (0.0011) +[2023-10-09 13:48:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 82378752. Throughput: 0: 1810.3, 1: 1831.1. Samples: 20601252. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:48:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.950')] +[2023-10-09 13:48:19,632][86121] Updated weights for policy 0, policy_version 40130 (0.0007) +[2023-10-09 13:48:19,998][86121] Updated weights for policy 0, policy_version 40140 (0.0007) +[2023-10-09 13:48:20,349][86122] Updated weights for policy 1, policy_version 40330 (0.0009) +[2023-10-09 13:48:20,370][86121] Updated weights for policy 0, policy_version 40150 (0.0008) +[2023-10-09 13:48:20,708][86122] Updated weights for policy 1, policy_version 40340 (0.0008) +[2023-10-09 13:48:20,736][86121] Updated weights for policy 0, policy_version 40160 (0.0007) +[2023-10-09 13:48:21,076][86122] Updated weights for policy 1, policy_version 40350 (0.0007) +[2023-10-09 13:48:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 82444288. Throughput: 0: 1802.8, 1: 1832.9. Samples: 20623726. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:48:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 13:48:24,373][86121] Updated weights for policy 0, policy_version 40170 (0.0009) +[2023-10-09 13:48:24,577][86122] Updated weights for policy 1, policy_version 40360 (0.0007) +[2023-10-09 13:48:24,735][86121] Updated weights for policy 0, policy_version 40180 (0.0009) +[2023-10-09 13:48:24,935][86122] Updated weights for policy 1, policy_version 40370 (0.0008) +[2023-10-09 13:48:25,105][86121] Updated weights for policy 0, policy_version 40190 (0.0009) +[2023-10-09 13:48:25,291][86122] Updated weights for policy 1, policy_version 40380 (0.0009) +[2023-10-09 13:48:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 82509824. Throughput: 0: 1803.2, 1: 1826.4. Samples: 20633644. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 13:48:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 13:48:28,687][86121] Updated weights for policy 0, policy_version 40200 (0.0007) +[2023-10-09 13:48:29,058][86121] Updated weights for policy 0, policy_version 40210 (0.0007) +[2023-10-09 13:48:29,060][86122] Updated weights for policy 1, policy_version 40390 (0.0008) +[2023-10-09 13:48:29,418][86122] Updated weights for policy 1, policy_version 40400 (0.0007) +[2023-10-09 13:48:29,419][86121] Updated weights for policy 0, policy_version 40220 (0.0008) +[2023-10-09 13:48:29,780][86122] Updated weights for policy 1, policy_version 40410 (0.0007) +[2023-10-09 13:48:32,964][86121] Updated weights for policy 0, policy_version 40230 (0.0007) +[2023-10-09 13:48:33,331][86121] Updated weights for policy 0, policy_version 40240 (0.0007) +[2023-10-09 13:48:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 82575360. Throughput: 0: 1811.1, 1: 1822.8. Samples: 20656896. Policy #0 lag: (min: 9.0, avg: 21.6, max: 41.0) +[2023-10-09 13:48:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 13:48:33,560][86122] Updated weights for policy 1, policy_version 40420 (0.0008) +[2023-10-09 13:48:33,691][86121] Updated weights for policy 0, policy_version 40250 (0.0007) +[2023-10-09 13:48:33,920][86122] Updated weights for policy 1, policy_version 40430 (0.0009) +[2023-10-09 13:48:34,277][86122] Updated weights for policy 1, policy_version 40440 (0.0009) +[2023-10-09 13:48:37,442][86121] Updated weights for policy 0, policy_version 40260 (0.0009) +[2023-10-09 13:48:37,803][86121] Updated weights for policy 0, policy_version 40270 (0.0007) +[2023-10-09 13:48:37,825][86122] Updated weights for policy 1, policy_version 40450 (0.0008) +[2023-10-09 13:48:38,165][86121] Updated weights for policy 0, policy_version 40280 (0.0008) +[2023-10-09 13:48:38,186][86122] Updated weights for policy 1, policy_version 40460 (0.0007) +[2023-10-09 13:48:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 82640896. Throughput: 0: 1815.7, 1: 1826.3. Samples: 20679104. Policy #0 lag: (min: 9.0, avg: 21.6, max: 41.0) +[2023-10-09 13:48:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 13:48:38,543][86122] Updated weights for policy 1, policy_version 40470 (0.0008) +[2023-10-09 13:48:38,901][86122] Updated weights for policy 1, policy_version 40480 (0.0009) +[2023-10-09 13:48:41,907][86121] Updated weights for policy 0, policy_version 40290 (0.0008) +[2023-10-09 13:48:42,288][86121] Updated weights for policy 0, policy_version 40300 (0.0008) +[2023-10-09 13:48:42,543][86122] Updated weights for policy 1, policy_version 40490 (0.0009) +[2023-10-09 13:48:42,655][86121] Updated weights for policy 0, policy_version 40310 (0.0007) +[2023-10-09 13:48:42,906][86122] Updated weights for policy 1, policy_version 40500 (0.0007) +[2023-10-09 13:48:43,020][86121] Updated weights for policy 0, policy_version 40320 (0.0007) +[2023-10-09 13:48:43,271][86122] Updated weights for policy 1, policy_version 40510 (0.0007) +[2023-10-09 13:48:43,397][85186] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 82771968. Throughput: 0: 1812.6, 1: 1831.0. Samples: 20689950. Policy #0 lag: (min: 9.0, avg: 21.6, max: 41.0) +[2023-10-09 13:48:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 13:48:46,709][86121] Updated weights for policy 0, policy_version 40330 (0.0008) +[2023-10-09 13:48:46,872][86122] Updated weights for policy 1, policy_version 40520 (0.0008) +[2023-10-09 13:48:47,071][86121] Updated weights for policy 0, policy_version 40340 (0.0008) +[2023-10-09 13:48:47,234][86122] Updated weights for policy 1, policy_version 40530 (0.0007) +[2023-10-09 13:48:47,440][86121] Updated weights for policy 0, policy_version 40350 (0.0008) +[2023-10-09 13:48:47,583][86122] Updated weights for policy 1, policy_version 40540 (0.0008) +[2023-10-09 13:48:48,397][85186] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 82837504. Throughput: 0: 1810.9, 1: 1824.0. Samples: 20711594. Policy #0 lag: (min: 9.0, avg: 21.6, max: 41.0) +[2023-10-09 13:48:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 13:48:51,254][86121] Updated weights for policy 0, policy_version 40360 (0.0009) +[2023-10-09 13:48:51,393][86122] Updated weights for policy 1, policy_version 40550 (0.0010) +[2023-10-09 13:48:51,617][86121] Updated weights for policy 0, policy_version 40370 (0.0008) +[2023-10-09 13:48:51,750][86122] Updated weights for policy 1, policy_version 40560 (0.0008) +[2023-10-09 13:48:51,982][86121] Updated weights for policy 0, policy_version 40380 (0.0008) +[2023-10-09 13:48:52,114][86122] Updated weights for policy 1, policy_version 40570 (0.0010) +[2023-10-09 13:48:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 82903040. Throughput: 0: 1808.6, 1: 1828.1. Samples: 20732200. Policy #0 lag: (min: 9.0, avg: 21.6, max: 41.0) +[2023-10-09 13:48:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 13:48:55,625][86121] Updated weights for policy 0, policy_version 40390 (0.0008) +[2023-10-09 13:48:55,902][86122] Updated weights for policy 1, policy_version 40580 (0.0008) +[2023-10-09 13:48:55,991][86121] Updated weights for policy 0, policy_version 40400 (0.0007) +[2023-10-09 13:48:56,272][86122] Updated weights for policy 1, policy_version 40590 (0.0009) +[2023-10-09 13:48:56,352][86121] Updated weights for policy 0, policy_version 40410 (0.0008) +[2023-10-09 13:48:56,638][86122] Updated weights for policy 1, policy_version 40600 (0.0008) +[2023-10-09 13:48:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 82968576. Throughput: 0: 1812.9, 1: 1820.9. Samples: 20744428. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 13:48:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 13:49:00,131][86121] Updated weights for policy 0, policy_version 40420 (0.0007) +[2023-10-09 13:49:00,449][86122] Updated weights for policy 1, policy_version 40610 (0.0009) +[2023-10-09 13:49:00,490][86121] Updated weights for policy 0, policy_version 40430 (0.0010) +[2023-10-09 13:49:00,814][86122] Updated weights for policy 1, policy_version 40620 (0.0011) +[2023-10-09 13:49:00,855][86121] Updated weights for policy 0, policy_version 40440 (0.0010) +[2023-10-09 13:49:01,183][86122] Updated weights for policy 1, policy_version 40630 (0.0008) +[2023-10-09 13:49:01,538][86122] Updated weights for policy 1, policy_version 40640 (0.0008) +[2023-10-09 13:49:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83034112. Throughput: 0: 1815.7, 1: 1818.5. Samples: 20764790. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 13:49:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 13:49:04,648][86121] Updated weights for policy 0, policy_version 40450 (0.0008) +[2023-10-09 13:49:05,013][86121] Updated weights for policy 0, policy_version 40460 (0.0008) +[2023-10-09 13:49:05,375][86122] Updated weights for policy 1, policy_version 40650 (0.0008) +[2023-10-09 13:49:05,386][86121] Updated weights for policy 0, policy_version 40470 (0.0009) +[2023-10-09 13:49:05,745][86122] Updated weights for policy 1, policy_version 40660 (0.0009) +[2023-10-09 13:49:05,748][86121] Updated weights for policy 0, policy_version 40480 (0.0009) +[2023-10-09 13:49:06,101][86122] Updated weights for policy 1, policy_version 40670 (0.0010) +[2023-10-09 13:49:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83099648. Throughput: 0: 1821.0, 1: 1823.0. Samples: 20787708. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 13:49:08,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 13:49:09,387][86121] Updated weights for policy 0, policy_version 40490 (0.0008) +[2023-10-09 13:49:09,601][86122] Updated weights for policy 1, policy_version 40680 (0.0008) +[2023-10-09 13:49:09,761][86121] Updated weights for policy 0, policy_version 40500 (0.0009) +[2023-10-09 13:49:09,964][86122] Updated weights for policy 1, policy_version 40690 (0.0008) +[2023-10-09 13:49:10,129][86121] Updated weights for policy 0, policy_version 40510 (0.0007) +[2023-10-09 13:49:10,327][86122] Updated weights for policy 1, policy_version 40700 (0.0008) +[2023-10-09 13:49:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 83165184. Throughput: 0: 1825.1, 1: 1823.3. Samples: 20797820. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 13:49:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 13:49:13,732][86121] Updated weights for policy 0, policy_version 40520 (0.0009) +[2023-10-09 13:49:14,085][86121] Updated weights for policy 0, policy_version 40530 (0.0009) +[2023-10-09 13:49:14,141][86122] Updated weights for policy 1, policy_version 40710 (0.0007) +[2023-10-09 13:49:14,450][86121] Updated weights for policy 0, policy_version 40540 (0.0009) +[2023-10-09 13:49:14,499][86122] Updated weights for policy 1, policy_version 40720 (0.0009) +[2023-10-09 13:49:14,862][86122] Updated weights for policy 1, policy_version 40730 (0.0008) +[2023-10-09 13:49:18,074][86121] Updated weights for policy 0, policy_version 40550 (0.0010) +[2023-10-09 13:49:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83230720. Throughput: 0: 1818.5, 1: 1821.4. Samples: 20820690. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 13:49:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 13:49:18,444][86121] Updated weights for policy 0, policy_version 40560 (0.0009) +[2023-10-09 13:49:18,600][86122] Updated weights for policy 1, policy_version 40740 (0.0010) +[2023-10-09 13:49:18,802][86121] Updated weights for policy 0, policy_version 40570 (0.0008) +[2023-10-09 13:49:18,956][86122] Updated weights for policy 1, policy_version 40750 (0.0008) +[2023-10-09 13:49:19,316][86122] Updated weights for policy 1, policy_version 40760 (0.0010) +[2023-10-09 13:49:22,398][86121] Updated weights for policy 0, policy_version 40580 (0.0008) +[2023-10-09 13:49:22,761][86121] Updated weights for policy 0, policy_version 40590 (0.0007) +[2023-10-09 13:49:22,999][86122] Updated weights for policy 1, policy_version 40770 (0.0009) +[2023-10-09 13:49:23,136][86121] Updated weights for policy 0, policy_version 40600 (0.0008) +[2023-10-09 13:49:23,367][86122] Updated weights for policy 1, policy_version 40780 (0.0009) +[2023-10-09 13:49:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83296256. Throughput: 0: 1823.8, 1: 1816.8. Samples: 20842932. Policy #0 lag: (min: 31.0, avg: 35.1, max: 63.0) +[2023-10-09 13:49:23,399][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 13:49:23,422][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000040608_41582592.pth... +[2023-10-09 13:49:23,456][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000038912_39845888.pth +[2023-10-09 13:49:23,721][86122] Updated weights for policy 1, policy_version 40790 (0.0007) +[2023-10-09 13:49:24,082][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000040800_41779200.pth... +[2023-10-09 13:49:24,087][86122] Updated weights for policy 1, policy_version 40800 (0.0011) +[2023-10-09 13:49:24,122][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000039072_40009728.pth +[2023-10-09 13:49:26,847][86121] Updated weights for policy 0, policy_version 40610 (0.0009) +[2023-10-09 13:49:27,246][86121] Updated weights for policy 0, policy_version 40620 (0.0008) +[2023-10-09 13:49:27,617][86121] Updated weights for policy 0, policy_version 40630 (0.0007) +[2023-10-09 13:49:27,814][86122] Updated weights for policy 1, policy_version 40810 (0.0009) +[2023-10-09 13:49:27,977][86121] Updated weights for policy 0, policy_version 40640 (0.0007) +[2023-10-09 13:49:28,168][86122] Updated weights for policy 1, policy_version 40820 (0.0008) +[2023-10-09 13:49:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 83394560. Throughput: 0: 1826.5, 1: 1809.0. Samples: 20853548. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:49:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 13:49:28,531][86122] Updated weights for policy 1, policy_version 40830 (0.0008) +[2023-10-09 13:49:31,833][86121] Updated weights for policy 0, policy_version 40650 (0.0007) +[2023-10-09 13:49:32,200][86121] Updated weights for policy 0, policy_version 40660 (0.0008) +[2023-10-09 13:49:32,218][86122] Updated weights for policy 1, policy_version 40840 (0.0008) +[2023-10-09 13:49:32,563][86121] Updated weights for policy 0, policy_version 40670 (0.0007) +[2023-10-09 13:49:32,581][86122] Updated weights for policy 1, policy_version 40850 (0.0009) +[2023-10-09 13:49:32,943][86122] Updated weights for policy 1, policy_version 40860 (0.0010) +[2023-10-09 13:49:33,397][85186] Fps is (10 sec: 19661.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 83492864. Throughput: 0: 1829.2, 1: 1817.4. Samples: 20875690. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:49:33,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 13:49:36,176][86121] Updated weights for policy 0, policy_version 40680 (0.0009) +[2023-10-09 13:49:36,535][86122] Updated weights for policy 1, policy_version 40870 (0.0007) +[2023-10-09 13:49:36,535][86121] Updated weights for policy 0, policy_version 40690 (0.0008) +[2023-10-09 13:49:36,893][86122] Updated weights for policy 1, policy_version 40880 (0.0008) +[2023-10-09 13:49:36,902][86121] Updated weights for policy 0, policy_version 40700 (0.0008) +[2023-10-09 13:49:37,250][86122] Updated weights for policy 1, policy_version 40890 (0.0007) +[2023-10-09 13:49:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 83558400. Throughput: 0: 1829.7, 1: 1816.3. Samples: 20896266. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:49:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 13:49:40,527][86121] Updated weights for policy 0, policy_version 40710 (0.0008) +[2023-10-09 13:49:40,826][86122] Updated weights for policy 1, policy_version 40900 (0.0007) +[2023-10-09 13:49:40,892][86121] Updated weights for policy 0, policy_version 40720 (0.0009) +[2023-10-09 13:49:41,180][86122] Updated weights for policy 1, policy_version 40910 (0.0008) +[2023-10-09 13:49:41,266][86121] Updated weights for policy 0, policy_version 40730 (0.0007) +[2023-10-09 13:49:41,538][86122] Updated weights for policy 1, policy_version 40920 (0.0007) +[2023-10-09 13:49:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 83623936. Throughput: 0: 1825.1, 1: 1819.3. Samples: 20908426. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:49:43,399][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 13:49:44,943][86121] Updated weights for policy 0, policy_version 40740 (0.0008) +[2023-10-09 13:49:45,309][86121] Updated weights for policy 0, policy_version 40750 (0.0008) +[2023-10-09 13:49:45,384][86122] Updated weights for policy 1, policy_version 40930 (0.0011) +[2023-10-09 13:49:45,674][86121] Updated weights for policy 0, policy_version 40760 (0.0008) +[2023-10-09 13:49:45,740][86122] Updated weights for policy 1, policy_version 40940 (0.0010) +[2023-10-09 13:49:46,101][86122] Updated weights for policy 1, policy_version 40950 (0.0010) +[2023-10-09 13:49:46,471][86122] Updated weights for policy 1, policy_version 40960 (0.0010) +[2023-10-09 13:49:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 83689472. Throughput: 0: 1828.3, 1: 1822.3. Samples: 20929068. Policy #0 lag: (min: 14.0, avg: 19.4, max: 46.0) +[2023-10-09 13:49:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 13:49:49,344][86121] Updated weights for policy 0, policy_version 40770 (0.0008) +[2023-10-09 13:49:49,706][86121] Updated weights for policy 0, policy_version 40780 (0.0007) +[2023-10-09 13:49:50,070][86121] Updated weights for policy 0, policy_version 40790 (0.0009) +[2023-10-09 13:49:50,072][86122] Updated weights for policy 1, policy_version 40970 (0.0009) +[2023-10-09 13:49:50,432][86122] Updated weights for policy 1, policy_version 40980 (0.0008) +[2023-10-09 13:49:50,442][86121] Updated weights for policy 0, policy_version 40800 (0.0008) +[2023-10-09 13:49:50,802][86122] Updated weights for policy 1, policy_version 40990 (0.0008) +[2023-10-09 13:49:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83755008. Throughput: 0: 1833.8, 1: 1822.1. Samples: 20952226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:49:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 13:49:53,956][86121] Updated weights for policy 0, policy_version 40810 (0.0010) +[2023-10-09 13:49:54,327][86121] Updated weights for policy 0, policy_version 40820 (0.0009) +[2023-10-09 13:49:54,693][86121] Updated weights for policy 0, policy_version 40830 (0.0007) +[2023-10-09 13:49:54,747][86122] Updated weights for policy 1, policy_version 41000 (0.0007) +[2023-10-09 13:49:55,108][86122] Updated weights for policy 1, policy_version 41010 (0.0008) +[2023-10-09 13:49:55,470][86122] Updated weights for policy 1, policy_version 41020 (0.0008) +[2023-10-09 13:49:58,357][86121] Updated weights for policy 0, policy_version 40840 (0.0008) +[2023-10-09 13:49:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 83820544. Throughput: 0: 1833.2, 1: 1814.3. Samples: 20961954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:49:58,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 13:49:58,726][86121] Updated weights for policy 0, policy_version 40850 (0.0009) +[2023-10-09 13:49:59,091][86121] Updated weights for policy 0, policy_version 40860 (0.0008) +[2023-10-09 13:49:59,244][86122] Updated weights for policy 1, policy_version 41030 (0.0008) +[2023-10-09 13:49:59,617][86122] Updated weights for policy 1, policy_version 41040 (0.0009) +[2023-10-09 13:49:59,978][86122] Updated weights for policy 1, policy_version 41050 (0.0009) +[2023-10-09 13:50:02,874][86121] Updated weights for policy 0, policy_version 40870 (0.0009) +[2023-10-09 13:50:03,234][86121] Updated weights for policy 0, policy_version 40880 (0.0008) +[2023-10-09 13:50:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83886080. Throughput: 0: 1824.5, 1: 1821.3. Samples: 20984754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:50:03,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 13:50:03,493][86122] Updated weights for policy 1, policy_version 41060 (0.0008) +[2023-10-09 13:50:03,598][86121] Updated weights for policy 0, policy_version 40890 (0.0008) +[2023-10-09 13:50:03,858][86122] Updated weights for policy 1, policy_version 41070 (0.0010) +[2023-10-09 13:50:04,226][86122] Updated weights for policy 1, policy_version 41080 (0.0010) +[2023-10-09 13:50:07,479][86121] Updated weights for policy 0, policy_version 40900 (0.0009) +[2023-10-09 13:50:07,833][86122] Updated weights for policy 1, policy_version 41090 (0.0010) +[2023-10-09 13:50:07,847][86121] Updated weights for policy 0, policy_version 40910 (0.0008) +[2023-10-09 13:50:08,192][86122] Updated weights for policy 1, policy_version 41100 (0.0009) +[2023-10-09 13:50:08,210][86121] Updated weights for policy 0, policy_version 40920 (0.0008) +[2023-10-09 13:50:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 83951616. Throughput: 0: 1820.7, 1: 1824.6. Samples: 21006970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:50:08,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 13:50:08,562][86122] Updated weights for policy 1, policy_version 41110 (0.0008) +[2023-10-09 13:50:08,923][86122] Updated weights for policy 1, policy_version 41120 (0.0009) +[2023-10-09 13:50:11,923][86121] Updated weights for policy 0, policy_version 40930 (0.0009) +[2023-10-09 13:50:12,332][86121] Updated weights for policy 0, policy_version 40940 (0.0008) +[2023-10-09 13:50:12,697][86121] Updated weights for policy 0, policy_version 40950 (0.0008) +[2023-10-09 13:50:12,699][86122] Updated weights for policy 1, policy_version 41130 (0.0007) +[2023-10-09 13:50:13,054][86121] Updated weights for policy 0, policy_version 40960 (0.0008) +[2023-10-09 13:50:13,059][86122] Updated weights for policy 1, policy_version 41140 (0.0007) +[2023-10-09 13:50:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84049920. Throughput: 0: 1817.2, 1: 1826.6. Samples: 21017520. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:50:13,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 13:50:13,429][86122] Updated weights for policy 1, policy_version 41150 (0.0008) +[2023-10-09 13:50:16,774][86121] Updated weights for policy 0, policy_version 40970 (0.0011) +[2023-10-09 13:50:17,075][86122] Updated weights for policy 1, policy_version 41160 (0.0008) +[2023-10-09 13:50:17,145][86121] Updated weights for policy 0, policy_version 40980 (0.0007) +[2023-10-09 13:50:17,437][86122] Updated weights for policy 1, policy_version 41170 (0.0009) +[2023-10-09 13:50:17,517][86121] Updated weights for policy 0, policy_version 40990 (0.0009) +[2023-10-09 13:50:17,803][86122] Updated weights for policy 1, policy_version 41180 (0.0008) +[2023-10-09 13:50:18,397][85186] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 84148224. Throughput: 0: 1821.4, 1: 1825.3. Samples: 21039792. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) +[2023-10-09 13:50:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 13:50:21,068][86121] Updated weights for policy 0, policy_version 41000 (0.0008) +[2023-10-09 13:50:21,444][86121] Updated weights for policy 0, policy_version 41010 (0.0008) +[2023-10-09 13:50:21,607][86122] Updated weights for policy 1, policy_version 41190 (0.0009) +[2023-10-09 13:50:21,818][86121] Updated weights for policy 0, policy_version 41020 (0.0008) +[2023-10-09 13:50:21,965][86122] Updated weights for policy 1, policy_version 41200 (0.0007) +[2023-10-09 13:50:22,332][86122] Updated weights for policy 1, policy_version 41210 (0.0008) +[2023-10-09 13:50:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 84213760. Throughput: 0: 1824.1, 1: 1816.4. Samples: 21060088. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) +[2023-10-09 13:50:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 13:50:25,345][86121] Updated weights for policy 0, policy_version 41030 (0.0010) +[2023-10-09 13:50:25,706][86121] Updated weights for policy 0, policy_version 41040 (0.0009) +[2023-10-09 13:50:26,085][86121] Updated weights for policy 0, policy_version 41050 (0.0009) +[2023-10-09 13:50:26,142][86122] Updated weights for policy 1, policy_version 41220 (0.0007) +[2023-10-09 13:50:26,517][86122] Updated weights for policy 1, policy_version 41230 (0.0009) +[2023-10-09 13:50:26,870][86122] Updated weights for policy 1, policy_version 41240 (0.0008) +[2023-10-09 13:50:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84279296. Throughput: 0: 1817.7, 1: 1816.1. Samples: 21071944. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) +[2023-10-09 13:50:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 13:50:29,886][86121] Updated weights for policy 0, policy_version 41060 (0.0009) +[2023-10-09 13:50:30,250][86121] Updated weights for policy 0, policy_version 41070 (0.0011) +[2023-10-09 13:50:30,493][86122] Updated weights for policy 1, policy_version 41250 (0.0008) +[2023-10-09 13:50:30,616][86121] Updated weights for policy 0, policy_version 41080 (0.0009) +[2023-10-09 13:50:30,846][86122] Updated weights for policy 1, policy_version 41260 (0.0008) +[2023-10-09 13:50:31,216][86122] Updated weights for policy 1, policy_version 41270 (0.0007) +[2023-10-09 13:50:31,583][86122] Updated weights for policy 1, policy_version 41280 (0.0009) +[2023-10-09 13:50:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84344832. Throughput: 0: 1824.3, 1: 1813.3. Samples: 21092758. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) +[2023-10-09 13:50:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 13:50:34,297][86121] Updated weights for policy 0, policy_version 41090 (0.0009) +[2023-10-09 13:50:34,663][86121] Updated weights for policy 0, policy_version 41100 (0.0009) +[2023-10-09 13:50:35,031][86121] Updated weights for policy 0, policy_version 41110 (0.0008) +[2023-10-09 13:50:35,247][86122] Updated weights for policy 1, policy_version 41290 (0.0009) +[2023-10-09 13:50:35,393][86121] Updated weights for policy 0, policy_version 41120 (0.0007) +[2023-10-09 13:50:35,603][86122] Updated weights for policy 1, policy_version 41300 (0.0010) +[2023-10-09 13:50:35,972][86122] Updated weights for policy 1, policy_version 41310 (0.0011) +[2023-10-09 13:50:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84410368. Throughput: 0: 1817.6, 1: 1812.3. Samples: 21115574. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) +[2023-10-09 13:50:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 13:50:39,219][86121] Updated weights for policy 0, policy_version 41130 (0.0007) +[2023-10-09 13:50:39,585][86121] Updated weights for policy 0, policy_version 41140 (0.0007) +[2023-10-09 13:50:39,824][86122] Updated weights for policy 1, policy_version 41320 (0.0008) +[2023-10-09 13:50:39,947][86121] Updated weights for policy 0, policy_version 41150 (0.0008) +[2023-10-09 13:50:40,190][86122] Updated weights for policy 1, policy_version 41330 (0.0010) +[2023-10-09 13:50:40,559][86122] Updated weights for policy 1, policy_version 41340 (0.0009) +[2023-10-09 13:50:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 84475904. Throughput: 0: 1813.6, 1: 1815.1. Samples: 21125246. Policy #0 lag: (min: 3.0, avg: 3.0, max: 5.0) +[2023-10-09 13:50:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 13:50:43,760][86121] Updated weights for policy 0, policy_version 41160 (0.0008) +[2023-10-09 13:50:44,128][86121] Updated weights for policy 0, policy_version 41170 (0.0008) +[2023-10-09 13:50:44,210][86122] Updated weights for policy 1, policy_version 41350 (0.0009) +[2023-10-09 13:50:44,499][86121] Updated weights for policy 0, policy_version 41180 (0.0008) +[2023-10-09 13:50:44,567][86122] Updated weights for policy 1, policy_version 41360 (0.0007) +[2023-10-09 13:50:44,930][86122] Updated weights for policy 1, policy_version 41370 (0.0008) +[2023-10-09 13:50:48,274][86121] Updated weights for policy 0, policy_version 41190 (0.0008) +[2023-10-09 13:50:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84541440. Throughput: 0: 1819.0, 1: 1815.6. Samples: 21148310. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 13:50:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 13:50:48,591][86122] Updated weights for policy 1, policy_version 41380 (0.0008) +[2023-10-09 13:50:48,633][86121] Updated weights for policy 0, policy_version 41200 (0.0008) +[2023-10-09 13:50:48,957][86122] Updated weights for policy 1, policy_version 41390 (0.0008) +[2023-10-09 13:50:48,997][86121] Updated weights for policy 0, policy_version 41210 (0.0008) +[2023-10-09 13:50:49,310][86122] Updated weights for policy 1, policy_version 41400 (0.0009) +[2023-10-09 13:50:52,685][86121] Updated weights for policy 0, policy_version 41220 (0.0009) +[2023-10-09 13:50:52,909][86122] Updated weights for policy 1, policy_version 41410 (0.0007) +[2023-10-09 13:50:53,045][86121] Updated weights for policy 0, policy_version 41230 (0.0007) +[2023-10-09 13:50:53,275][86122] Updated weights for policy 1, policy_version 41420 (0.0009) +[2023-10-09 13:50:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 84606976. Throughput: 0: 1822.8, 1: 1815.6. Samples: 21170700. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 13:50:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 13:50:53,412][86121] Updated weights for policy 0, policy_version 41240 (0.0007) +[2023-10-09 13:50:53,637][86122] Updated weights for policy 1, policy_version 41430 (0.0007) +[2023-10-09 13:50:53,996][86122] Updated weights for policy 1, policy_version 41440 (0.0011) +[2023-10-09 13:50:57,151][86121] Updated weights for policy 0, policy_version 41250 (0.0007) +[2023-10-09 13:50:57,550][86121] Updated weights for policy 0, policy_version 41260 (0.0008) +[2023-10-09 13:50:57,736][86122] Updated weights for policy 1, policy_version 41450 (0.0007) +[2023-10-09 13:50:57,918][86121] Updated weights for policy 0, policy_version 41270 (0.0011) +[2023-10-09 13:50:58,105][86122] Updated weights for policy 1, policy_version 41460 (0.0008) +[2023-10-09 13:50:58,279][86121] Updated weights for policy 0, policy_version 41280 (0.0008) +[2023-10-09 13:50:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 84705280. Throughput: 0: 1818.8, 1: 1817.7. Samples: 21181166. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 13:50:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 13:50:58,476][86122] Updated weights for policy 1, policy_version 41470 (0.0008) +[2023-10-09 13:51:01,943][86121] Updated weights for policy 0, policy_version 41290 (0.0010) +[2023-10-09 13:51:02,312][86122] Updated weights for policy 1, policy_version 41480 (0.0008) +[2023-10-09 13:51:02,316][86121] Updated weights for policy 0, policy_version 41300 (0.0009) +[2023-10-09 13:51:02,674][86122] Updated weights for policy 1, policy_version 41490 (0.0009) +[2023-10-09 13:51:02,677][86121] Updated weights for policy 0, policy_version 41310 (0.0008) +[2023-10-09 13:51:03,036][86122] Updated weights for policy 1, policy_version 41500 (0.0007) +[2023-10-09 13:51:03,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 84803584. Throughput: 0: 1815.1, 1: 1813.1. Samples: 21203062. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 13:51:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 13:51:06,446][86121] Updated weights for policy 0, policy_version 41320 (0.0009) +[2023-10-09 13:51:06,819][86121] Updated weights for policy 0, policy_version 41330 (0.0009) +[2023-10-09 13:51:06,897][86122] Updated weights for policy 1, policy_version 41510 (0.0009) +[2023-10-09 13:51:07,178][86121] Updated weights for policy 0, policy_version 41340 (0.0007) +[2023-10-09 13:51:07,244][86122] Updated weights for policy 1, policy_version 41520 (0.0007) +[2023-10-09 13:51:07,614][86122] Updated weights for policy 1, policy_version 41530 (0.0008) +[2023-10-09 13:51:08,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 84869120. Throughput: 0: 1808.4, 1: 1820.8. Samples: 21223404. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 13:51:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 13:51:10,884][86121] Updated weights for policy 0, policy_version 41350 (0.0008) +[2023-10-09 13:51:11,260][86121] Updated weights for policy 0, policy_version 41360 (0.0007) +[2023-10-09 13:51:11,419][86122] Updated weights for policy 1, policy_version 41540 (0.0008) +[2023-10-09 13:51:11,626][86121] Updated weights for policy 0, policy_version 41370 (0.0007) +[2023-10-09 13:51:11,777][86122] Updated weights for policy 1, policy_version 41550 (0.0007) +[2023-10-09 13:51:12,135][86122] Updated weights for policy 1, policy_version 41560 (0.0007) +[2023-10-09 13:51:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 84934656. Throughput: 0: 1825.6, 1: 1816.4. Samples: 21235834. Policy #0 lag: (min: 6.0, avg: 6.9, max: 26.0) +[2023-10-09 13:51:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 13:51:15,146][86121] Updated weights for policy 0, policy_version 41380 (0.0007) +[2023-10-09 13:51:15,506][86121] Updated weights for policy 0, policy_version 41390 (0.0009) +[2023-10-09 13:51:15,763][86122] Updated weights for policy 1, policy_version 41570 (0.0008) +[2023-10-09 13:51:15,869][86121] Updated weights for policy 0, policy_version 41400 (0.0007) +[2023-10-09 13:51:16,123][86122] Updated weights for policy 1, policy_version 41580 (0.0008) +[2023-10-09 13:51:16,488][86122] Updated weights for policy 1, policy_version 41590 (0.0007) +[2023-10-09 13:51:16,854][86122] Updated weights for policy 1, policy_version 41600 (0.0007) +[2023-10-09 13:51:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85000192. Throughput: 0: 1816.7, 1: 1822.9. Samples: 21256542. Policy #0 lag: (min: 6.0, avg: 6.9, max: 26.0) +[2023-10-09 13:51:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:51:19,671][86121] Updated weights for policy 0, policy_version 41410 (0.0007) +[2023-10-09 13:51:20,033][86121] Updated weights for policy 0, policy_version 41420 (0.0008) +[2023-10-09 13:51:20,375][86122] Updated weights for policy 1, policy_version 41610 (0.0008) +[2023-10-09 13:51:20,403][86121] Updated weights for policy 0, policy_version 41430 (0.0009) +[2023-10-09 13:51:20,735][86122] Updated weights for policy 1, policy_version 41620 (0.0011) +[2023-10-09 13:51:20,761][86121] Updated weights for policy 0, policy_version 41440 (0.0008) +[2023-10-09 13:51:21,099][86122] Updated weights for policy 1, policy_version 41630 (0.0011) +[2023-10-09 13:51:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85065728. Throughput: 0: 1816.4, 1: 1819.5. Samples: 21279190. Policy #0 lag: (min: 6.0, avg: 6.9, max: 26.0) +[2023-10-09 13:51:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:51:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000041440_42434560.pth... +[2023-10-09 13:51:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000041632_42631168.pth... +[2023-10-09 13:51:23,440][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000039936_40894464.pth +[2023-10-09 13:51:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000039744_40697856.pth +[2023-10-09 13:51:24,341][86121] Updated weights for policy 0, policy_version 41450 (0.0007) +[2023-10-09 13:51:24,704][86121] Updated weights for policy 0, policy_version 41460 (0.0007) +[2023-10-09 13:51:24,888][86122] Updated weights for policy 1, policy_version 41640 (0.0008) +[2023-10-09 13:51:25,075][86121] Updated weights for policy 0, policy_version 41470 (0.0007) +[2023-10-09 13:51:25,258][86122] Updated weights for policy 1, policy_version 41650 (0.0009) +[2023-10-09 13:51:25,622][86122] Updated weights for policy 1, policy_version 41660 (0.0010) +[2023-10-09 13:51:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 85131264. Throughput: 0: 1816.6, 1: 1822.9. Samples: 21289024. Policy #0 lag: (min: 6.0, avg: 6.9, max: 26.0) +[2023-10-09 13:51:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 13:51:28,787][86121] Updated weights for policy 0, policy_version 41480 (0.0008) +[2023-10-09 13:51:29,157][86121] Updated weights for policy 0, policy_version 41490 (0.0009) +[2023-10-09 13:51:29,298][86122] Updated weights for policy 1, policy_version 41670 (0.0008) +[2023-10-09 13:51:29,524][86121] Updated weights for policy 0, policy_version 41500 (0.0009) +[2023-10-09 13:51:29,662][86122] Updated weights for policy 1, policy_version 41680 (0.0007) +[2023-10-09 13:51:30,026][86122] Updated weights for policy 1, policy_version 41690 (0.0009) +[2023-10-09 13:51:33,311][86121] Updated weights for policy 0, policy_version 41510 (0.0008) +[2023-10-09 13:51:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85196800. Throughput: 0: 1812.9, 1: 1819.4. Samples: 21311764. Policy #0 lag: (min: 6.0, avg: 6.9, max: 26.0) +[2023-10-09 13:51:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:51:33,621][86122] Updated weights for policy 1, policy_version 41700 (0.0008) +[2023-10-09 13:51:33,674][86121] Updated weights for policy 0, policy_version 41520 (0.0008) +[2023-10-09 13:51:33,984][86122] Updated weights for policy 1, policy_version 41710 (0.0009) +[2023-10-09 13:51:34,046][86121] Updated weights for policy 0, policy_version 41530 (0.0009) +[2023-10-09 13:51:34,356][86122] Updated weights for policy 1, policy_version 41720 (0.0008) +[2023-10-09 13:51:37,824][86121] Updated weights for policy 0, policy_version 41540 (0.0009) +[2023-10-09 13:51:38,030][86122] Updated weights for policy 1, policy_version 41730 (0.0008) +[2023-10-09 13:51:38,188][86121] Updated weights for policy 0, policy_version 41550 (0.0009) +[2023-10-09 13:51:38,395][86122] Updated weights for policy 1, policy_version 41740 (0.0007) +[2023-10-09 13:51:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 85262336. Throughput: 0: 1812.5, 1: 1822.9. Samples: 21334294. Policy #0 lag: (min: 6.0, avg: 6.9, max: 26.0) +[2023-10-09 13:51:38,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:51:38,556][86121] Updated weights for policy 0, policy_version 41560 (0.0008) +[2023-10-09 13:51:38,750][86122] Updated weights for policy 1, policy_version 41750 (0.0009) +[2023-10-09 13:51:39,111][86122] Updated weights for policy 1, policy_version 41760 (0.0008) +[2023-10-09 13:51:42,335][86121] Updated weights for policy 0, policy_version 41570 (0.0008) +[2023-10-09 13:51:42,734][86121] Updated weights for policy 0, policy_version 41580 (0.0007) +[2023-10-09 13:51:42,847][86122] Updated weights for policy 1, policy_version 41770 (0.0008) +[2023-10-09 13:51:43,106][86121] Updated weights for policy 0, policy_version 41590 (0.0007) +[2023-10-09 13:51:43,209][86122] Updated weights for policy 1, policy_version 41780 (0.0009) +[2023-10-09 13:51:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85327872. Throughput: 0: 1803.9, 1: 1824.3. Samples: 21344434. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) +[2023-10-09 13:51:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:51:43,467][86121] Updated weights for policy 0, policy_version 41600 (0.0007) +[2023-10-09 13:51:43,561][86122] Updated weights for policy 1, policy_version 41790 (0.0008) +[2023-10-09 13:51:47,239][86122] Updated weights for policy 1, policy_version 41800 (0.0008) +[2023-10-09 13:51:47,277][86121] Updated weights for policy 0, policy_version 41610 (0.0009) +[2023-10-09 13:51:47,603][86122] Updated weights for policy 1, policy_version 41810 (0.0009) +[2023-10-09 13:51:47,642][86121] Updated weights for policy 0, policy_version 41620 (0.0010) +[2023-10-09 13:51:47,963][86122] Updated weights for policy 1, policy_version 41820 (0.0007) +[2023-10-09 13:51:48,000][86121] Updated weights for policy 0, policy_version 41630 (0.0009) +[2023-10-09 13:51:48,397][85186] Fps is (10 sec: 19661.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 85458944. Throughput: 0: 1815.2, 1: 1823.2. Samples: 21366790. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) +[2023-10-09 13:51:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:51:51,743][86121] Updated weights for policy 0, policy_version 41640 (0.0008) +[2023-10-09 13:51:51,770][86122] Updated weights for policy 1, policy_version 41830 (0.0008) +[2023-10-09 13:51:52,105][86121] Updated weights for policy 0, policy_version 41650 (0.0008) +[2023-10-09 13:51:52,124][86122] Updated weights for policy 1, policy_version 41840 (0.0007) +[2023-10-09 13:51:52,473][86121] Updated weights for policy 0, policy_version 41660 (0.0007) +[2023-10-09 13:51:52,483][86122] Updated weights for policy 1, policy_version 41850 (0.0008) +[2023-10-09 13:51:53,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 85524480. Throughput: 0: 1805.2, 1: 1818.5. Samples: 21386472. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) +[2023-10-09 13:51:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 13:51:56,184][86122] Updated weights for policy 1, policy_version 41860 (0.0008) +[2023-10-09 13:51:56,228][86121] Updated weights for policy 0, policy_version 41670 (0.0008) +[2023-10-09 13:51:56,552][86122] Updated weights for policy 1, policy_version 41870 (0.0008) +[2023-10-09 13:51:56,595][86121] Updated weights for policy 0, policy_version 41680 (0.0008) +[2023-10-09 13:51:56,915][86122] Updated weights for policy 1, policy_version 41880 (0.0009) +[2023-10-09 13:51:56,959][86121] Updated weights for policy 0, policy_version 41690 (0.0010) +[2023-10-09 13:51:58,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 85590016. Throughput: 0: 1808.7, 1: 1824.5. Samples: 21399328. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) +[2023-10-09 13:51:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:52:00,714][86122] Updated weights for policy 1, policy_version 41890 (0.0009) +[2023-10-09 13:52:00,730][86121] Updated weights for policy 0, policy_version 41700 (0.0009) +[2023-10-09 13:52:01,065][86122] Updated weights for policy 1, policy_version 41900 (0.0007) +[2023-10-09 13:52:01,087][86121] Updated weights for policy 0, policy_version 41710 (0.0007) +[2023-10-09 13:52:01,433][86122] Updated weights for policy 1, policy_version 41910 (0.0007) +[2023-10-09 13:52:01,456][86121] Updated weights for policy 0, policy_version 41720 (0.0008) +[2023-10-09 13:52:01,790][86122] Updated weights for policy 1, policy_version 41920 (0.0009) +[2023-10-09 13:52:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 85655552. Throughput: 0: 1795.0, 1: 1813.9. Samples: 21418942. Policy #0 lag: (min: 8.0, avg: 35.3, max: 40.0) +[2023-10-09 13:52:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:52:05,206][86121] Updated weights for policy 0, policy_version 41730 (0.0008) +[2023-10-09 13:52:05,561][86121] Updated weights for policy 0, policy_version 41740 (0.0009) +[2023-10-09 13:52:05,565][86122] Updated weights for policy 1, policy_version 41930 (0.0008) +[2023-10-09 13:52:05,924][86122] Updated weights for policy 1, policy_version 41940 (0.0009) +[2023-10-09 13:52:05,928][86121] Updated weights for policy 0, policy_version 41750 (0.0009) +[2023-10-09 13:52:06,286][86122] Updated weights for policy 1, policy_version 41950 (0.0009) +[2023-10-09 13:52:06,294][86121] Updated weights for policy 0, policy_version 41760 (0.0008) +[2023-10-09 13:52:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 85721088. Throughput: 0: 1797.1, 1: 1811.5. Samples: 21441574. Policy #0 lag: (min: 25.0, avg: 41.1, max: 57.0) +[2023-10-09 13:52:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:52:09,745][86121] Updated weights for policy 0, policy_version 41770 (0.0008) +[2023-10-09 13:52:09,976][86122] Updated weights for policy 1, policy_version 41960 (0.0009) +[2023-10-09 13:52:10,114][86121] Updated weights for policy 0, policy_version 41780 (0.0008) +[2023-10-09 13:52:10,345][86122] Updated weights for policy 1, policy_version 41970 (0.0008) +[2023-10-09 13:52:10,477][86121] Updated weights for policy 0, policy_version 41790 (0.0007) +[2023-10-09 13:52:10,698][86122] Updated weights for policy 1, policy_version 41980 (0.0008) +[2023-10-09 13:52:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 85786624. Throughput: 0: 1796.2, 1: 1816.5. Samples: 21451596. Policy #0 lag: (min: 25.0, avg: 41.1, max: 57.0) +[2023-10-09 13:52:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:52:14,132][86121] Updated weights for policy 0, policy_version 41800 (0.0007) +[2023-10-09 13:52:14,444][86122] Updated weights for policy 1, policy_version 41990 (0.0008) +[2023-10-09 13:52:14,505][86121] Updated weights for policy 0, policy_version 41810 (0.0007) +[2023-10-09 13:52:14,800][86122] Updated weights for policy 1, policy_version 42000 (0.0008) +[2023-10-09 13:52:14,863][86121] Updated weights for policy 0, policy_version 41820 (0.0007) +[2023-10-09 13:52:15,159][86122] Updated weights for policy 1, policy_version 42010 (0.0010) +[2023-10-09 13:52:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85852160. Throughput: 0: 1803.7, 1: 1811.5. Samples: 21474452. Policy #0 lag: (min: 25.0, avg: 41.1, max: 57.0) +[2023-10-09 13:52:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:52:18,541][86121] Updated weights for policy 0, policy_version 41830 (0.0009) +[2023-10-09 13:52:18,894][86122] Updated weights for policy 1, policy_version 42020 (0.0008) +[2023-10-09 13:52:18,911][86121] Updated weights for policy 0, policy_version 41840 (0.0010) +[2023-10-09 13:52:19,251][86122] Updated weights for policy 1, policy_version 42030 (0.0008) +[2023-10-09 13:52:19,267][86121] Updated weights for policy 0, policy_version 41850 (0.0007) +[2023-10-09 13:52:19,607][86122] Updated weights for policy 1, policy_version 42040 (0.0009) +[2023-10-09 13:52:22,932][86121] Updated weights for policy 0, policy_version 41860 (0.0007) +[2023-10-09 13:52:23,295][86121] Updated weights for policy 0, policy_version 41870 (0.0007) +[2023-10-09 13:52:23,308][86122] Updated weights for policy 1, policy_version 42050 (0.0009) +[2023-10-09 13:52:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 85917696. Throughput: 0: 1808.9, 1: 1808.4. Samples: 21497068. Policy #0 lag: (min: 25.0, avg: 41.1, max: 57.0) +[2023-10-09 13:52:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:52:23,652][86121] Updated weights for policy 0, policy_version 41880 (0.0008) +[2023-10-09 13:52:23,672][86122] Updated weights for policy 1, policy_version 42060 (0.0008) +[2023-10-09 13:52:24,029][86122] Updated weights for policy 1, policy_version 42070 (0.0008) +[2023-10-09 13:52:24,396][86122] Updated weights for policy 1, policy_version 42080 (0.0008) +[2023-10-09 13:52:27,305][86121] Updated weights for policy 0, policy_version 41890 (0.0008) +[2023-10-09 13:52:27,716][86121] Updated weights for policy 0, policy_version 41900 (0.0008) +[2023-10-09 13:52:27,867][86122] Updated weights for policy 1, policy_version 42090 (0.0008) +[2023-10-09 13:52:28,081][86121] Updated weights for policy 0, policy_version 41910 (0.0009) +[2023-10-09 13:52:28,225][86122] Updated weights for policy 1, policy_version 42100 (0.0010) +[2023-10-09 13:52:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 85983232. Throughput: 0: 1808.5, 1: 1806.9. Samples: 21507128. Policy #0 lag: (min: 25.0, avg: 41.1, max: 57.0) +[2023-10-09 13:52:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:52:28,446][86121] Updated weights for policy 0, policy_version 41920 (0.0008) +[2023-10-09 13:52:28,589][86122] Updated weights for policy 1, policy_version 42110 (0.0008) +[2023-10-09 13:52:32,184][86121] Updated weights for policy 0, policy_version 41930 (0.0007) +[2023-10-09 13:52:32,384][86122] Updated weights for policy 1, policy_version 42120 (0.0010) +[2023-10-09 13:52:32,557][86121] Updated weights for policy 0, policy_version 41940 (0.0007) +[2023-10-09 13:52:32,744][86122] Updated weights for policy 1, policy_version 42130 (0.0008) +[2023-10-09 13:52:32,930][86121] Updated weights for policy 0, policy_version 41950 (0.0008) +[2023-10-09 13:52:33,100][86122] Updated weights for policy 1, policy_version 42140 (0.0008) +[2023-10-09 13:52:33,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 86114304. Throughput: 0: 1811.9, 1: 1810.8. Samples: 21529814. Policy #0 lag: (min: 25.0, avg: 41.1, max: 57.0) +[2023-10-09 13:52:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:52:36,595][86121] Updated weights for policy 0, policy_version 41960 (0.0007) +[2023-10-09 13:52:36,824][86122] Updated weights for policy 1, policy_version 42150 (0.0007) +[2023-10-09 13:52:36,960][86121] Updated weights for policy 0, policy_version 41970 (0.0008) +[2023-10-09 13:52:37,186][86122] Updated weights for policy 1, policy_version 42160 (0.0008) +[2023-10-09 13:52:37,333][86121] Updated weights for policy 0, policy_version 41980 (0.0008) +[2023-10-09 13:52:37,555][86122] Updated weights for policy 1, policy_version 42170 (0.0008) +[2023-10-09 13:52:38,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 86179840. Throughput: 0: 1814.8, 1: 1814.6. Samples: 21549794. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 13:52:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:52:41,155][86121] Updated weights for policy 0, policy_version 41990 (0.0009) +[2023-10-09 13:52:41,431][86122] Updated weights for policy 1, policy_version 42180 (0.0007) +[2023-10-09 13:52:41,520][86121] Updated weights for policy 0, policy_version 42000 (0.0008) +[2023-10-09 13:52:41,791][86122] Updated weights for policy 1, policy_version 42190 (0.0010) +[2023-10-09 13:52:41,889][86121] Updated weights for policy 0, policy_version 42010 (0.0009) +[2023-10-09 13:52:42,159][86122] Updated weights for policy 1, policy_version 42200 (0.0010) +[2023-10-09 13:52:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 86245376. Throughput: 0: 1810.9, 1: 1810.8. Samples: 21562304. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 13:52:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:52:45,602][86121] Updated weights for policy 0, policy_version 42020 (0.0008) +[2023-10-09 13:52:45,777][86122] Updated weights for policy 1, policy_version 42210 (0.0009) +[2023-10-09 13:52:45,970][86121] Updated weights for policy 0, policy_version 42030 (0.0008) +[2023-10-09 13:52:46,125][86122] Updated weights for policy 1, policy_version 42220 (0.0008) +[2023-10-09 13:52:46,340][86121] Updated weights for policy 0, policy_version 42040 (0.0008) +[2023-10-09 13:52:46,493][86122] Updated weights for policy 1, policy_version 42230 (0.0007) +[2023-10-09 13:52:46,853][86122] Updated weights for policy 1, policy_version 42240 (0.0008) +[2023-10-09 13:52:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 86310912. Throughput: 0: 1812.6, 1: 1822.6. Samples: 21582528. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 13:52:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:52:50,022][86121] Updated weights for policy 0, policy_version 42050 (0.0009) +[2023-10-09 13:52:50,380][86121] Updated weights for policy 0, policy_version 42060 (0.0008) +[2023-10-09 13:52:50,492][86122] Updated weights for policy 1, policy_version 42250 (0.0008) +[2023-10-09 13:52:50,744][86121] Updated weights for policy 0, policy_version 42070 (0.0008) +[2023-10-09 13:52:50,853][86122] Updated weights for policy 1, policy_version 42260 (0.0009) +[2023-10-09 13:52:51,106][86121] Updated weights for policy 0, policy_version 42080 (0.0007) +[2023-10-09 13:52:51,207][86122] Updated weights for policy 1, policy_version 42270 (0.0009) +[2023-10-09 13:52:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 86376448. Throughput: 0: 1807.5, 1: 1821.2. Samples: 21604864. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 13:52:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:52:54,883][86121] Updated weights for policy 0, policy_version 42090 (0.0010) +[2023-10-09 13:52:55,059][86122] Updated weights for policy 1, policy_version 42280 (0.0008) +[2023-10-09 13:52:55,252][86121] Updated weights for policy 0, policy_version 42100 (0.0007) +[2023-10-09 13:52:55,414][86122] Updated weights for policy 1, policy_version 42290 (0.0008) +[2023-10-09 13:52:55,608][86121] Updated weights for policy 0, policy_version 42110 (0.0008) +[2023-10-09 13:52:55,775][86122] Updated weights for policy 1, policy_version 42300 (0.0008) +[2023-10-09 13:52:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 86441984. Throughput: 0: 1810.0, 1: 1820.7. Samples: 21614976. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 13:52:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:52:59,330][86121] Updated weights for policy 0, policy_version 42120 (0.0008) +[2023-10-09 13:52:59,508][86122] Updated weights for policy 1, policy_version 42310 (0.0008) +[2023-10-09 13:52:59,695][86121] Updated weights for policy 0, policy_version 42130 (0.0008) +[2023-10-09 13:52:59,871][86122] Updated weights for policy 1, policy_version 42320 (0.0008) +[2023-10-09 13:53:00,059][86121] Updated weights for policy 0, policy_version 42140 (0.0007) +[2023-10-09 13:53:00,226][86122] Updated weights for policy 1, policy_version 42330 (0.0007) +[2023-10-09 13:53:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 86507520. Throughput: 0: 1804.1, 1: 1822.5. Samples: 21637648. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 13:53:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:53:03,796][86121] Updated weights for policy 0, policy_version 42150 (0.0008) +[2023-10-09 13:53:03,847][86122] Updated weights for policy 1, policy_version 42340 (0.0009) +[2023-10-09 13:53:04,158][86121] Updated weights for policy 0, policy_version 42160 (0.0008) +[2023-10-09 13:53:04,208][86122] Updated weights for policy 1, policy_version 42350 (0.0009) +[2023-10-09 13:53:04,532][86121] Updated weights for policy 0, policy_version 42170 (0.0008) +[2023-10-09 13:53:04,567][86122] Updated weights for policy 1, policy_version 42360 (0.0010) +[2023-10-09 13:53:08,277][86122] Updated weights for policy 1, policy_version 42370 (0.0010) +[2023-10-09 13:53:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 86573056. Throughput: 0: 1806.7, 1: 1822.3. Samples: 21660370. Policy #0 lag: (min: 23.0, avg: 23.0, max: 23.0) +[2023-10-09 13:53:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:53:08,412][86121] Updated weights for policy 0, policy_version 42180 (0.0008) +[2023-10-09 13:53:08,638][86122] Updated weights for policy 1, policy_version 42380 (0.0007) +[2023-10-09 13:53:08,777][86121] Updated weights for policy 0, policy_version 42190 (0.0009) +[2023-10-09 13:53:08,992][86122] Updated weights for policy 1, policy_version 42390 (0.0007) +[2023-10-09 13:53:09,143][86121] Updated weights for policy 0, policy_version 42200 (0.0010) +[2023-10-09 13:53:09,356][86122] Updated weights for policy 1, policy_version 42400 (0.0007) +[2023-10-09 13:53:12,977][86122] Updated weights for policy 1, policy_version 42410 (0.0007) +[2023-10-09 13:53:13,032][86121] Updated weights for policy 0, policy_version 42210 (0.0007) +[2023-10-09 13:53:13,334][86122] Updated weights for policy 1, policy_version 42420 (0.0007) +[2023-10-09 13:53:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 86638592. Throughput: 0: 1804.2, 1: 1822.4. Samples: 21670326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:53:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 13:53:13,433][86121] Updated weights for policy 0, policy_version 42220 (0.0008) +[2023-10-09 13:53:13,692][86122] Updated weights for policy 1, policy_version 42430 (0.0009) +[2023-10-09 13:53:13,802][86121] Updated weights for policy 0, policy_version 42230 (0.0008) +[2023-10-09 13:53:14,170][86121] Updated weights for policy 0, policy_version 42240 (0.0007) +[2023-10-09 13:53:17,428][86122] Updated weights for policy 1, policy_version 42440 (0.0008) +[2023-10-09 13:53:17,787][86122] Updated weights for policy 1, policy_version 42450 (0.0007) +[2023-10-09 13:53:17,800][86121] Updated weights for policy 0, policy_version 42250 (0.0007) +[2023-10-09 13:53:18,153][86122] Updated weights for policy 1, policy_version 42460 (0.0008) +[2023-10-09 13:53:18,160][86121] Updated weights for policy 0, policy_version 42260 (0.0007) +[2023-10-09 13:53:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 86736896. Throughput: 0: 1804.2, 1: 1819.6. Samples: 21692884. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:53:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:53:18,534][86121] Updated weights for policy 0, policy_version 42270 (0.0008) +[2023-10-09 13:53:21,852][86122] Updated weights for policy 1, policy_version 42470 (0.0009) +[2023-10-09 13:53:22,210][86122] Updated weights for policy 1, policy_version 42480 (0.0007) +[2023-10-09 13:53:22,290][86121] Updated weights for policy 0, policy_version 42280 (0.0007) +[2023-10-09 13:53:22,573][86122] Updated weights for policy 1, policy_version 42490 (0.0009) +[2023-10-09 13:53:22,663][86121] Updated weights for policy 0, policy_version 42290 (0.0009) +[2023-10-09 13:53:23,020][86121] Updated weights for policy 0, policy_version 42300 (0.0008) +[2023-10-09 13:53:23,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 86835200. Throughput: 0: 1809.6, 1: 1817.5. Samples: 21713016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:53:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:53:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000042496_43515904.pth... +[2023-10-09 13:53:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000042304_43319296.pth... +[2023-10-09 13:53:23,439][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000040608_41582592.pth +[2023-10-09 13:53:23,443][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000042304_43319296.pth +[2023-10-09 13:53:23,446][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000040800_41779200.pth +[2023-10-09 13:53:23,451][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000042496_43515904.pth +[2023-10-09 13:53:26,203][86122] Updated weights for policy 1, policy_version 42500 (0.0007) +[2023-10-09 13:53:26,570][86122] Updated weights for policy 1, policy_version 42510 (0.0008) +[2023-10-09 13:53:26,674][86121] Updated weights for policy 0, policy_version 42310 (0.0008) +[2023-10-09 13:53:26,928][86122] Updated weights for policy 1, policy_version 42520 (0.0008) +[2023-10-09 13:53:27,027][86121] Updated weights for policy 0, policy_version 42320 (0.0008) +[2023-10-09 13:53:27,391][86121] Updated weights for policy 0, policy_version 42330 (0.0007) +[2023-10-09 13:53:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 86900736. Throughput: 0: 1807.7, 1: 1819.3. Samples: 21725518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:53:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:53:30,660][86122] Updated weights for policy 1, policy_version 42530 (0.0009) +[2023-10-09 13:53:31,030][86122] Updated weights for policy 1, policy_version 42540 (0.0008) +[2023-10-09 13:53:31,138][86121] Updated weights for policy 0, policy_version 42340 (0.0008) +[2023-10-09 13:53:31,388][86122] Updated weights for policy 1, policy_version 42550 (0.0007) +[2023-10-09 13:53:31,501][86121] Updated weights for policy 0, policy_version 42350 (0.0007) +[2023-10-09 13:53:31,750][86122] Updated weights for policy 1, policy_version 42560 (0.0008) +[2023-10-09 13:53:31,863][86121] Updated weights for policy 0, policy_version 42360 (0.0009) +[2023-10-09 13:53:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 86966272. Throughput: 0: 1817.8, 1: 1810.3. Samples: 21745790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:53:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 13:53:35,390][86121] Updated weights for policy 0, policy_version 42370 (0.0010) +[2023-10-09 13:53:35,430][86122] Updated weights for policy 1, policy_version 42570 (0.0008) +[2023-10-09 13:53:35,744][86121] Updated weights for policy 0, policy_version 42380 (0.0007) +[2023-10-09 13:53:35,786][86122] Updated weights for policy 1, policy_version 42580 (0.0007) +[2023-10-09 13:53:36,114][86121] Updated weights for policy 0, policy_version 42390 (0.0007) +[2023-10-09 13:53:36,142][86122] Updated weights for policy 1, policy_version 42590 (0.0007) +[2023-10-09 13:53:36,474][86121] Updated weights for policy 0, policy_version 42400 (0.0011) +[2023-10-09 13:53:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 87031808. Throughput: 0: 1818.7, 1: 1818.2. Samples: 21768522. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:53:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 13:53:40,018][86122] Updated weights for policy 1, policy_version 42600 (0.0008) +[2023-10-09 13:53:40,231][86121] Updated weights for policy 0, policy_version 42410 (0.0008) +[2023-10-09 13:53:40,395][86122] Updated weights for policy 1, policy_version 42610 (0.0009) +[2023-10-09 13:53:40,602][86121] Updated weights for policy 0, policy_version 42420 (0.0008) +[2023-10-09 13:53:40,754][86122] Updated weights for policy 1, policy_version 42620 (0.0008) +[2023-10-09 13:53:40,974][86121] Updated weights for policy 0, policy_version 42430 (0.0007) +[2023-10-09 13:53:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87097344. Throughput: 0: 1823.5, 1: 1812.6. Samples: 21778598. Policy #0 lag: (min: 49.0, avg: 55.7, max: 56.0) +[2023-10-09 13:53:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:53:44,447][86122] Updated weights for policy 1, policy_version 42630 (0.0007) +[2023-10-09 13:53:44,614][86121] Updated weights for policy 0, policy_version 42440 (0.0007) +[2023-10-09 13:53:44,808][86122] Updated weights for policy 1, policy_version 42640 (0.0008) +[2023-10-09 13:53:44,977][86121] Updated weights for policy 0, policy_version 42450 (0.0008) +[2023-10-09 13:53:45,175][86122] Updated weights for policy 1, policy_version 42650 (0.0007) +[2023-10-09 13:53:45,343][86121] Updated weights for policy 0, policy_version 42460 (0.0007) +[2023-10-09 13:53:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87162880. Throughput: 0: 1812.9, 1: 1817.5. Samples: 21801014. Policy #0 lag: (min: 49.0, avg: 55.7, max: 56.0) +[2023-10-09 13:53:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:53:48,928][86122] Updated weights for policy 1, policy_version 42660 (0.0008) +[2023-10-09 13:53:49,053][86121] Updated weights for policy 0, policy_version 42470 (0.0007) +[2023-10-09 13:53:49,290][86122] Updated weights for policy 1, policy_version 42670 (0.0007) +[2023-10-09 13:53:49,419][86121] Updated weights for policy 0, policy_version 42480 (0.0009) +[2023-10-09 13:53:49,652][86122] Updated weights for policy 1, policy_version 42680 (0.0007) +[2023-10-09 13:53:49,796][86121] Updated weights for policy 0, policy_version 42490 (0.0007) +[2023-10-09 13:53:53,394][86122] Updated weights for policy 1, policy_version 42690 (0.0009) +[2023-10-09 13:53:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87228416. Throughput: 0: 1810.4, 1: 1812.1. Samples: 21823380. Policy #0 lag: (min: 49.0, avg: 55.7, max: 56.0) +[2023-10-09 13:53:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:53:53,481][86121] Updated weights for policy 0, policy_version 42500 (0.0007) +[2023-10-09 13:53:53,750][86122] Updated weights for policy 1, policy_version 42700 (0.0009) +[2023-10-09 13:53:53,848][86121] Updated weights for policy 0, policy_version 42510 (0.0008) +[2023-10-09 13:53:54,110][86122] Updated weights for policy 1, policy_version 42710 (0.0008) +[2023-10-09 13:53:54,214][86121] Updated weights for policy 0, policy_version 42520 (0.0008) +[2023-10-09 13:53:54,470][86122] Updated weights for policy 1, policy_version 42720 (0.0008) +[2023-10-09 13:53:57,947][86121] Updated weights for policy 0, policy_version 42530 (0.0008) +[2023-10-09 13:53:58,223][86122] Updated weights for policy 1, policy_version 42730 (0.0009) +[2023-10-09 13:53:58,333][86121] Updated weights for policy 0, policy_version 42540 (0.0007) +[2023-10-09 13:53:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87293952. Throughput: 0: 1810.2, 1: 1811.0. Samples: 21833280. Policy #0 lag: (min: 49.0, avg: 55.7, max: 56.0) +[2023-10-09 13:53:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:53:58,582][86122] Updated weights for policy 1, policy_version 42740 (0.0008) +[2023-10-09 13:53:58,688][86121] Updated weights for policy 0, policy_version 42550 (0.0007) +[2023-10-09 13:53:58,950][86122] Updated weights for policy 1, policy_version 42750 (0.0008) +[2023-10-09 13:53:59,048][86121] Updated weights for policy 0, policy_version 42560 (0.0008) +[2023-10-09 13:54:02,465][86122] Updated weights for policy 1, policy_version 42760 (0.0007) +[2023-10-09 13:54:02,729][86121] Updated weights for policy 0, policy_version 42570 (0.0008) +[2023-10-09 13:54:02,832][86122] Updated weights for policy 1, policy_version 42770 (0.0007) +[2023-10-09 13:54:03,097][86121] Updated weights for policy 0, policy_version 42580 (0.0007) +[2023-10-09 13:54:03,191][86122] Updated weights for policy 1, policy_version 42780 (0.0007) +[2023-10-09 13:54:03,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 87392256. Throughput: 0: 1808.8, 1: 1812.2. Samples: 21855830. Policy #0 lag: (min: 49.0, avg: 55.7, max: 56.0) +[2023-10-09 13:54:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:54:03,463][86121] Updated weights for policy 0, policy_version 42590 (0.0009) +[2023-10-09 13:54:06,991][86122] Updated weights for policy 1, policy_version 42790 (0.0008) +[2023-10-09 13:54:07,252][86121] Updated weights for policy 0, policy_version 42600 (0.0007) +[2023-10-09 13:54:07,351][86122] Updated weights for policy 1, policy_version 42800 (0.0007) +[2023-10-09 13:54:07,614][86121] Updated weights for policy 0, policy_version 42610 (0.0008) +[2023-10-09 13:54:07,714][86122] Updated weights for policy 1, policy_version 42810 (0.0008) +[2023-10-09 13:54:07,980][86121] Updated weights for policy 0, policy_version 42620 (0.0009) +[2023-10-09 13:54:08,397][85186] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 87490560. Throughput: 0: 1805.9, 1: 1815.8. Samples: 21875994. Policy #0 lag: (min: 49.0, avg: 55.7, max: 56.0) +[2023-10-09 13:54:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:54:11,388][86122] Updated weights for policy 1, policy_version 42820 (0.0009) +[2023-10-09 13:54:11,757][86122] Updated weights for policy 1, policy_version 42830 (0.0009) +[2023-10-09 13:54:11,790][86121] Updated weights for policy 0, policy_version 42630 (0.0009) +[2023-10-09 13:54:12,117][86122] Updated weights for policy 1, policy_version 42840 (0.0007) +[2023-10-09 13:54:12,152][86121] Updated weights for policy 0, policy_version 42640 (0.0008) +[2023-10-09 13:54:12,519][86121] Updated weights for policy 0, policy_version 42650 (0.0007) +[2023-10-09 13:54:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 87556096. Throughput: 0: 1803.4, 1: 1813.4. Samples: 21888272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:54:15,690][86122] Updated weights for policy 1, policy_version 42850 (0.0008) +[2023-10-09 13:54:16,044][86122] Updated weights for policy 1, policy_version 42860 (0.0011) +[2023-10-09 13:54:16,304][86121] Updated weights for policy 0, policy_version 42660 (0.0007) +[2023-10-09 13:54:16,405][86122] Updated weights for policy 1, policy_version 42870 (0.0008) +[2023-10-09 13:54:16,671][86121] Updated weights for policy 0, policy_version 42670 (0.0008) +[2023-10-09 13:54:16,763][86122] Updated weights for policy 1, policy_version 42880 (0.0008) +[2023-10-09 13:54:17,040][86121] Updated weights for policy 0, policy_version 42680 (0.0008) +[2023-10-09 13:54:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 87621632. Throughput: 0: 1802.5, 1: 1819.7. Samples: 21908790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:54:20,490][86122] Updated weights for policy 1, policy_version 42890 (0.0007) +[2023-10-09 13:54:20,851][86122] Updated weights for policy 1, policy_version 42900 (0.0008) +[2023-10-09 13:54:20,875][86121] Updated weights for policy 0, policy_version 42690 (0.0007) +[2023-10-09 13:54:21,217][86122] Updated weights for policy 1, policy_version 42910 (0.0009) +[2023-10-09 13:54:21,245][86121] Updated weights for policy 0, policy_version 42700 (0.0007) +[2023-10-09 13:54:21,612][86121] Updated weights for policy 0, policy_version 42710 (0.0008) +[2023-10-09 13:54:21,979][86121] Updated weights for policy 0, policy_version 42720 (0.0012) +[2023-10-09 13:54:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 87687168. Throughput: 0: 1791.9, 1: 1819.1. Samples: 21931016. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:54:24,905][86122] Updated weights for policy 1, policy_version 42920 (0.0010) +[2023-10-09 13:54:25,278][86122] Updated weights for policy 1, policy_version 42930 (0.0010) +[2023-10-09 13:54:25,556][86121] Updated weights for policy 0, policy_version 42730 (0.0007) +[2023-10-09 13:54:25,631][86122] Updated weights for policy 1, policy_version 42940 (0.0009) +[2023-10-09 13:54:25,920][86121] Updated weights for policy 0, policy_version 42740 (0.0008) +[2023-10-09 13:54:26,291][86121] Updated weights for policy 0, policy_version 42750 (0.0008) +[2023-10-09 13:54:28,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87752704. Throughput: 0: 1805.2, 1: 1822.6. Samples: 21941850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 13:54:29,391][86122] Updated weights for policy 1, policy_version 42950 (0.0009) +[2023-10-09 13:54:29,767][86122] Updated weights for policy 1, policy_version 42960 (0.0008) +[2023-10-09 13:54:29,958][86121] Updated weights for policy 0, policy_version 42760 (0.0008) +[2023-10-09 13:54:30,119][86122] Updated weights for policy 1, policy_version 42970 (0.0008) +[2023-10-09 13:54:30,327][86121] Updated weights for policy 0, policy_version 42770 (0.0007) +[2023-10-09 13:54:30,692][86121] Updated weights for policy 0, policy_version 42780 (0.0009) +[2023-10-09 13:54:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87818240. Throughput: 0: 1798.5, 1: 1816.6. Samples: 21963694. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:54:33,900][86122] Updated weights for policy 1, policy_version 42980 (0.0008) +[2023-10-09 13:54:34,266][86122] Updated weights for policy 1, policy_version 42990 (0.0007) +[2023-10-09 13:54:34,393][86121] Updated weights for policy 0, policy_version 42790 (0.0009) +[2023-10-09 13:54:34,637][86122] Updated weights for policy 1, policy_version 43000 (0.0007) +[2023-10-09 13:54:34,763][86121] Updated weights for policy 0, policy_version 42800 (0.0008) +[2023-10-09 13:54:35,127][86121] Updated weights for policy 0, policy_version 42810 (0.0007) +[2023-10-09 13:54:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87883776. Throughput: 0: 1808.5, 1: 1818.1. Samples: 21986578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:54:38,450][86122] Updated weights for policy 1, policy_version 43010 (0.0007) +[2023-10-09 13:54:38,818][86122] Updated weights for policy 1, policy_version 43020 (0.0008) +[2023-10-09 13:54:38,843][86121] Updated weights for policy 0, policy_version 42820 (0.0008) +[2023-10-09 13:54:39,182][86122] Updated weights for policy 1, policy_version 43030 (0.0009) +[2023-10-09 13:54:39,209][86121] Updated weights for policy 0, policy_version 42830 (0.0007) +[2023-10-09 13:54:39,544][86122] Updated weights for policy 1, policy_version 43040 (0.0008) +[2023-10-09 13:54:39,584][86121] Updated weights for policy 0, policy_version 42840 (0.0008) +[2023-10-09 13:54:43,272][86122] Updated weights for policy 1, policy_version 43050 (0.0007) +[2023-10-09 13:54:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 87949312. Throughput: 0: 1804.8, 1: 1818.4. Samples: 21996326. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:54:43,476][86121] Updated weights for policy 0, policy_version 42850 (0.0008) +[2023-10-09 13:54:43,625][86122] Updated weights for policy 1, policy_version 43060 (0.0007) +[2023-10-09 13:54:43,845][86121] Updated weights for policy 0, policy_version 42860 (0.0008) +[2023-10-09 13:54:43,995][86122] Updated weights for policy 1, policy_version 43070 (0.0008) +[2023-10-09 13:54:44,220][86121] Updated weights for policy 0, policy_version 42870 (0.0007) +[2023-10-09 13:54:44,597][86121] Updated weights for policy 0, policy_version 42880 (0.0007) +[2023-10-09 13:54:47,581][86122] Updated weights for policy 1, policy_version 43080 (0.0009) +[2023-10-09 13:54:47,946][86122] Updated weights for policy 1, policy_version 43090 (0.0009) +[2023-10-09 13:54:48,300][86121] Updated weights for policy 0, policy_version 42890 (0.0007) +[2023-10-09 13:54:48,302][86122] Updated weights for policy 1, policy_version 43100 (0.0008) +[2023-10-09 13:54:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88014848. Throughput: 0: 1806.1, 1: 1825.1. Samples: 22019232. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:54:48,665][86121] Updated weights for policy 0, policy_version 42900 (0.0008) +[2023-10-09 13:54:49,020][86121] Updated weights for policy 0, policy_version 42910 (0.0010) +[2023-10-09 13:54:52,116][86122] Updated weights for policy 1, policy_version 43110 (0.0008) +[2023-10-09 13:54:52,483][86122] Updated weights for policy 1, policy_version 43120 (0.0010) +[2023-10-09 13:54:52,823][86121] Updated weights for policy 0, policy_version 42920 (0.0008) +[2023-10-09 13:54:52,842][86122] Updated weights for policy 1, policy_version 43130 (0.0008) +[2023-10-09 13:54:53,187][86121] Updated weights for policy 0, policy_version 42930 (0.0007) +[2023-10-09 13:54:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88113152. Throughput: 0: 1823.1, 1: 1832.3. Samples: 22040488. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:54:53,559][86121] Updated weights for policy 0, policy_version 42940 (0.0007) +[2023-10-09 13:54:56,489][86122] Updated weights for policy 1, policy_version 43140 (0.0008) +[2023-10-09 13:54:56,843][86122] Updated weights for policy 1, policy_version 43150 (0.0009) +[2023-10-09 13:54:57,205][86122] Updated weights for policy 1, policy_version 43160 (0.0008) +[2023-10-09 13:54:57,236][86121] Updated weights for policy 0, policy_version 42950 (0.0007) +[2023-10-09 13:54:57,595][86121] Updated weights for policy 0, policy_version 42960 (0.0007) +[2023-10-09 13:54:57,965][86121] Updated weights for policy 0, policy_version 42970 (0.0007) +[2023-10-09 13:54:58,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 88211456. Throughput: 0: 1809.8, 1: 1829.4. Samples: 22052036. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:54:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:55:00,821][86122] Updated weights for policy 1, policy_version 43170 (0.0008) +[2023-10-09 13:55:01,179][86122] Updated weights for policy 1, policy_version 43180 (0.0007) +[2023-10-09 13:55:01,545][86122] Updated weights for policy 1, policy_version 43190 (0.0008) +[2023-10-09 13:55:01,784][86121] Updated weights for policy 0, policy_version 42980 (0.0007) +[2023-10-09 13:55:01,909][86122] Updated weights for policy 1, policy_version 43200 (0.0008) +[2023-10-09 13:55:02,153][86121] Updated weights for policy 0, policy_version 42990 (0.0007) +[2023-10-09 13:55:02,512][86121] Updated weights for policy 0, policy_version 43000 (0.0008) +[2023-10-09 13:55:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 88276992. Throughput: 0: 1822.8, 1: 1829.4. Samples: 22073138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:55:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:55:05,585][86122] Updated weights for policy 1, policy_version 43210 (0.0013) +[2023-10-09 13:55:05,949][86122] Updated weights for policy 1, policy_version 43220 (0.0010) +[2023-10-09 13:55:06,199][86121] Updated weights for policy 0, policy_version 43010 (0.0008) +[2023-10-09 13:55:06,310][86122] Updated weights for policy 1, policy_version 43230 (0.0009) +[2023-10-09 13:55:06,566][86121] Updated weights for policy 0, policy_version 43020 (0.0008) +[2023-10-09 13:55:06,932][86121] Updated weights for policy 0, policy_version 43030 (0.0007) +[2023-10-09 13:55:07,306][86121] Updated weights for policy 0, policy_version 43040 (0.0007) +[2023-10-09 13:55:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 88342528. Throughput: 0: 1810.6, 1: 1825.7. Samples: 22094650. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:55:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 13:55:10,013][86122] Updated weights for policy 1, policy_version 43240 (0.0010) +[2023-10-09 13:55:10,383][86122] Updated weights for policy 1, policy_version 43250 (0.0011) +[2023-10-09 13:55:10,751][86122] Updated weights for policy 1, policy_version 43260 (0.0009) +[2023-10-09 13:55:10,890][86121] Updated weights for policy 0, policy_version 43050 (0.0007) +[2023-10-09 13:55:11,259][86121] Updated weights for policy 0, policy_version 43060 (0.0008) +[2023-10-09 13:55:11,631][86121] Updated weights for policy 0, policy_version 43070 (0.0010) +[2023-10-09 13:55:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 88408064. Throughput: 0: 1810.5, 1: 1823.0. Samples: 22105358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:55:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 13:55:14,411][86122] Updated weights for policy 1, policy_version 43270 (0.0008) +[2023-10-09 13:55:14,775][86122] Updated weights for policy 1, policy_version 43280 (0.0007) +[2023-10-09 13:55:15,140][86122] Updated weights for policy 1, policy_version 43290 (0.0009) +[2023-10-09 13:55:15,232][86121] Updated weights for policy 0, policy_version 43080 (0.0007) +[2023-10-09 13:55:15,593][86121] Updated weights for policy 0, policy_version 43090 (0.0009) +[2023-10-09 13:55:15,957][86121] Updated weights for policy 0, policy_version 43100 (0.0010) +[2023-10-09 13:55:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 88473600. Throughput: 0: 1810.4, 1: 1822.5. Samples: 22127172. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) +[2023-10-09 13:55:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:55:18,763][86122] Updated weights for policy 1, policy_version 43300 (0.0007) +[2023-10-09 13:55:19,126][86122] Updated weights for policy 1, policy_version 43310 (0.0008) +[2023-10-09 13:55:19,495][86122] Updated weights for policy 1, policy_version 43320 (0.0010) +[2023-10-09 13:55:19,772][86121] Updated weights for policy 0, policy_version 43110 (0.0007) +[2023-10-09 13:55:20,135][86121] Updated weights for policy 0, policy_version 43120 (0.0007) +[2023-10-09 13:55:20,509][86121] Updated weights for policy 0, policy_version 43130 (0.0009) +[2023-10-09 13:55:23,209][86122] Updated weights for policy 1, policy_version 43330 (0.0009) +[2023-10-09 13:55:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 88539136. Throughput: 0: 1804.8, 1: 1825.7. Samples: 22149950. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) +[2023-10-09 13:55:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:55:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000043136_44171264.pth... +[2023-10-09 13:55:23,442][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000041440_42434560.pth +[2023-10-09 13:55:23,568][86122] Updated weights for policy 1, policy_version 43340 (0.0007) +[2023-10-09 13:55:23,934][86122] Updated weights for policy 1, policy_version 43350 (0.0010) +[2023-10-09 13:55:24,242][86121] Updated weights for policy 0, policy_version 43140 (0.0010) +[2023-10-09 13:55:24,296][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000043360_44400640.pth... +[2023-10-09 13:55:24,298][86122] Updated weights for policy 1, policy_version 43360 (0.0010) +[2023-10-09 13:55:24,329][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000041632_42631168.pth +[2023-10-09 13:55:24,608][86121] Updated weights for policy 0, policy_version 43150 (0.0008) +[2023-10-09 13:55:24,985][86121] Updated weights for policy 0, policy_version 43160 (0.0009) +[2023-10-09 13:55:28,060][86122] Updated weights for policy 1, policy_version 43370 (0.0008) +[2023-10-09 13:55:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88604672. Throughput: 0: 1805.7, 1: 1826.4. Samples: 22159774. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) +[2023-10-09 13:55:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:55:28,411][86122] Updated weights for policy 1, policy_version 43380 (0.0008) +[2023-10-09 13:55:28,772][86122] Updated weights for policy 1, policy_version 43390 (0.0008) +[2023-10-09 13:55:28,805][86121] Updated weights for policy 0, policy_version 43170 (0.0008) +[2023-10-09 13:55:29,177][86121] Updated weights for policy 0, policy_version 43180 (0.0009) +[2023-10-09 13:55:29,549][86121] Updated weights for policy 0, policy_version 43190 (0.0009) +[2023-10-09 13:55:29,914][86121] Updated weights for policy 0, policy_version 43200 (0.0008) +[2023-10-09 13:55:32,542][86122] Updated weights for policy 1, policy_version 43400 (0.0009) +[2023-10-09 13:55:32,912][86122] Updated weights for policy 1, policy_version 43410 (0.0009) +[2023-10-09 13:55:33,281][86122] Updated weights for policy 1, policy_version 43420 (0.0007) +[2023-10-09 13:55:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 88670208. Throughput: 0: 1800.7, 1: 1825.3. Samples: 22182402. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) +[2023-10-09 13:55:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:55:33,722][86121] Updated weights for policy 0, policy_version 43210 (0.0007) +[2023-10-09 13:55:34,088][86121] Updated weights for policy 0, policy_version 43220 (0.0009) +[2023-10-09 13:55:34,451][86121] Updated weights for policy 0, policy_version 43230 (0.0008) +[2023-10-09 13:55:36,942][86122] Updated weights for policy 1, policy_version 43430 (0.0008) +[2023-10-09 13:55:37,307][86122] Updated weights for policy 1, policy_version 43440 (0.0007) +[2023-10-09 13:55:37,667][86122] Updated weights for policy 1, policy_version 43450 (0.0008) +[2023-10-09 13:55:38,106][86121] Updated weights for policy 0, policy_version 43240 (0.0008) +[2023-10-09 13:55:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88768512. Throughput: 0: 1811.3, 1: 1819.1. Samples: 22203858. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) +[2023-10-09 13:55:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:55:38,481][86121] Updated weights for policy 0, policy_version 43250 (0.0009) +[2023-10-09 13:55:38,845][86121] Updated weights for policy 0, policy_version 43260 (0.0010) +[2023-10-09 13:55:41,465][86122] Updated weights for policy 1, policy_version 43460 (0.0007) +[2023-10-09 13:55:41,838][86122] Updated weights for policy 1, policy_version 43470 (0.0010) +[2023-10-09 13:55:42,203][86122] Updated weights for policy 1, policy_version 43480 (0.0008) +[2023-10-09 13:55:42,553][86121] Updated weights for policy 0, policy_version 43270 (0.0008) +[2023-10-09 13:55:42,920][86121] Updated weights for policy 0, policy_version 43280 (0.0008) +[2023-10-09 13:55:43,280][86121] Updated weights for policy 0, policy_version 43290 (0.0008) +[2023-10-09 13:55:43,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88834048. Throughput: 0: 1799.8, 1: 1820.1. Samples: 22214934. Policy #0 lag: (min: 31.0, avg: 34.6, max: 63.0) +[2023-10-09 13:55:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:55:45,703][86122] Updated weights for policy 1, policy_version 43490 (0.0010) +[2023-10-09 13:55:46,076][86122] Updated weights for policy 1, policy_version 43500 (0.0009) +[2023-10-09 13:55:46,433][86122] Updated weights for policy 1, policy_version 43510 (0.0008) +[2023-10-09 13:55:46,792][86122] Updated weights for policy 1, policy_version 43520 (0.0009) +[2023-10-09 13:55:46,948][86121] Updated weights for policy 0, policy_version 43300 (0.0007) +[2023-10-09 13:55:47,328][86121] Updated weights for policy 0, policy_version 43310 (0.0007) +[2023-10-09 13:55:47,704][86121] Updated weights for policy 0, policy_version 43320 (0.0010) +[2023-10-09 13:55:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 88932352. Throughput: 0: 1805.5, 1: 1819.9. Samples: 22236282. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) +[2023-10-09 13:55:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:55:50,427][86122] Updated weights for policy 1, policy_version 43530 (0.0009) +[2023-10-09 13:55:50,789][86122] Updated weights for policy 1, policy_version 43540 (0.0009) +[2023-10-09 13:55:51,140][86122] Updated weights for policy 1, policy_version 43550 (0.0011) +[2023-10-09 13:55:51,404][86121] Updated weights for policy 0, policy_version 43330 (0.0009) +[2023-10-09 13:55:51,764][86121] Updated weights for policy 0, policy_version 43340 (0.0008) +[2023-10-09 13:55:52,129][86121] Updated weights for policy 0, policy_version 43350 (0.0008) +[2023-10-09 13:55:52,500][86121] Updated weights for policy 0, policy_version 43360 (0.0008) +[2023-10-09 13:55:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 88997888. Throughput: 0: 1801.1, 1: 1828.5. Samples: 22257980. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) +[2023-10-09 13:55:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:55:54,808][86122] Updated weights for policy 1, policy_version 43560 (0.0008) +[2023-10-09 13:55:55,190][86122] Updated weights for policy 1, policy_version 43570 (0.0009) +[2023-10-09 13:55:55,544][86122] Updated weights for policy 1, policy_version 43580 (0.0010) +[2023-10-09 13:55:56,173][86121] Updated weights for policy 0, policy_version 43370 (0.0011) +[2023-10-09 13:55:56,541][86121] Updated weights for policy 0, policy_version 43380 (0.0007) +[2023-10-09 13:55:56,909][86121] Updated weights for policy 0, policy_version 43390 (0.0008) +[2023-10-09 13:55:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89063424. Throughput: 0: 1812.4, 1: 1826.6. Samples: 22269110. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) +[2023-10-09 13:55:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:55:59,202][86122] Updated weights for policy 1, policy_version 43590 (0.0008) +[2023-10-09 13:55:59,564][86122] Updated weights for policy 1, policy_version 43600 (0.0009) +[2023-10-09 13:55:59,926][86122] Updated weights for policy 1, policy_version 43610 (0.0007) +[2023-10-09 13:56:00,649][86121] Updated weights for policy 0, policy_version 43400 (0.0007) +[2023-10-09 13:56:01,003][86121] Updated weights for policy 0, policy_version 43410 (0.0008) +[2023-10-09 13:56:01,368][86121] Updated weights for policy 0, policy_version 43420 (0.0009) +[2023-10-09 13:56:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89128960. Throughput: 0: 1799.9, 1: 1835.2. Samples: 22290748. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) +[2023-10-09 13:56:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:56:03,688][86122] Updated weights for policy 1, policy_version 43620 (0.0008) +[2023-10-09 13:56:04,048][86122] Updated weights for policy 1, policy_version 43630 (0.0009) +[2023-10-09 13:56:04,414][86122] Updated weights for policy 1, policy_version 43640 (0.0008) +[2023-10-09 13:56:04,960][86121] Updated weights for policy 0, policy_version 43430 (0.0009) +[2023-10-09 13:56:05,325][86121] Updated weights for policy 0, policy_version 43440 (0.0008) +[2023-10-09 13:56:05,695][86121] Updated weights for policy 0, policy_version 43450 (0.0007) +[2023-10-09 13:56:08,147][86122] Updated weights for policy 1, policy_version 43650 (0.0010) +[2023-10-09 13:56:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89194496. Throughput: 0: 1801.2, 1: 1831.9. Samples: 22313436. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) +[2023-10-09 13:56:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:56:08,510][86122] Updated weights for policy 1, policy_version 43660 (0.0008) +[2023-10-09 13:56:08,874][86122] Updated weights for policy 1, policy_version 43670 (0.0008) +[2023-10-09 13:56:09,240][86122] Updated weights for policy 1, policy_version 43680 (0.0007) +[2023-10-09 13:56:09,361][86121] Updated weights for policy 0, policy_version 43460 (0.0009) +[2023-10-09 13:56:09,724][86121] Updated weights for policy 0, policy_version 43470 (0.0011) +[2023-10-09 13:56:10,083][86121] Updated weights for policy 0, policy_version 43480 (0.0008) +[2023-10-09 13:56:12,720][86122] Updated weights for policy 1, policy_version 43690 (0.0009) +[2023-10-09 13:56:13,083][86122] Updated weights for policy 1, policy_version 43700 (0.0008) +[2023-10-09 13:56:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 89260032. Throughput: 0: 1803.6, 1: 1834.1. Samples: 22323474. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) +[2023-10-09 13:56:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:56:13,449][86122] Updated weights for policy 1, policy_version 43710 (0.0007) +[2023-10-09 13:56:13,932][86121] Updated weights for policy 0, policy_version 43490 (0.0009) +[2023-10-09 13:56:14,325][86121] Updated weights for policy 0, policy_version 43500 (0.0009) +[2023-10-09 13:56:14,684][86121] Updated weights for policy 0, policy_version 43510 (0.0007) +[2023-10-09 13:56:15,049][86121] Updated weights for policy 0, policy_version 43520 (0.0007) +[2023-10-09 13:56:17,035][86122] Updated weights for policy 1, policy_version 43720 (0.0007) +[2023-10-09 13:56:17,398][86122] Updated weights for policy 1, policy_version 43730 (0.0009) +[2023-10-09 13:56:17,756][86122] Updated weights for policy 1, policy_version 43740 (0.0008) +[2023-10-09 13:56:18,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89358336. Throughput: 0: 1806.0, 1: 1832.3. Samples: 22346128. Policy #0 lag: (min: 31.0, avg: 31.2, max: 40.0) +[2023-10-09 13:56:18,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:56:18,808][86121] Updated weights for policy 0, policy_version 43530 (0.0010) +[2023-10-09 13:56:19,172][86121] Updated weights for policy 0, policy_version 43540 (0.0010) +[2023-10-09 13:56:19,542][86121] Updated weights for policy 0, policy_version 43550 (0.0007) +[2023-10-09 13:56:21,407][86122] Updated weights for policy 1, policy_version 43750 (0.0010) +[2023-10-09 13:56:21,772][86122] Updated weights for policy 1, policy_version 43760 (0.0011) +[2023-10-09 13:56:22,126][86122] Updated weights for policy 1, policy_version 43770 (0.0008) +[2023-10-09 13:56:23,301][86121] Updated weights for policy 0, policy_version 43560 (0.0010) +[2023-10-09 13:56:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89423872. Throughput: 0: 1808.1, 1: 1831.9. Samples: 22367656. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) +[2023-10-09 13:56:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:56:23,668][86121] Updated weights for policy 0, policy_version 43570 (0.0011) +[2023-10-09 13:56:24,037][86121] Updated weights for policy 0, policy_version 43580 (0.0010) +[2023-10-09 13:56:25,894][86122] Updated weights for policy 1, policy_version 43780 (0.0009) +[2023-10-09 13:56:26,253][86122] Updated weights for policy 1, policy_version 43790 (0.0008) +[2023-10-09 13:56:26,612][86122] Updated weights for policy 1, policy_version 43800 (0.0008) +[2023-10-09 13:56:27,660][86121] Updated weights for policy 0, policy_version 43590 (0.0010) +[2023-10-09 13:56:28,025][86121] Updated weights for policy 0, policy_version 43600 (0.0009) +[2023-10-09 13:56:28,395][86121] Updated weights for policy 0, policy_version 43610 (0.0007) +[2023-10-09 13:56:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 89489408. Throughput: 0: 1810.0, 1: 1835.5. Samples: 22378980. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) +[2023-10-09 13:56:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:56:30,261][86122] Updated weights for policy 1, policy_version 43810 (0.0008) +[2023-10-09 13:56:30,627][86122] Updated weights for policy 1, policy_version 43820 (0.0009) +[2023-10-09 13:56:30,985][86122] Updated weights for policy 1, policy_version 43830 (0.0009) +[2023-10-09 13:56:31,345][86122] Updated weights for policy 1, policy_version 43840 (0.0007) +[2023-10-09 13:56:32,089][86121] Updated weights for policy 0, policy_version 43620 (0.0008) +[2023-10-09 13:56:32,459][86121] Updated weights for policy 0, policy_version 43630 (0.0009) +[2023-10-09 13:56:32,833][86121] Updated weights for policy 0, policy_version 43640 (0.0007) +[2023-10-09 13:56:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 89587712. Throughput: 0: 1810.8, 1: 1833.4. Samples: 22400270. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) +[2023-10-09 13:56:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:56:35,049][86122] Updated weights for policy 1, policy_version 43850 (0.0009) +[2023-10-09 13:56:35,412][86122] Updated weights for policy 1, policy_version 43860 (0.0009) +[2023-10-09 13:56:35,777][86122] Updated weights for policy 1, policy_version 43870 (0.0010) +[2023-10-09 13:56:36,560][86121] Updated weights for policy 0, policy_version 43650 (0.0009) +[2023-10-09 13:56:36,927][86121] Updated weights for policy 0, policy_version 43660 (0.0007) +[2023-10-09 13:56:37,294][86121] Updated weights for policy 0, policy_version 43670 (0.0009) +[2023-10-09 13:56:37,657][86121] Updated weights for policy 0, policy_version 43680 (0.0010) +[2023-10-09 13:56:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 89653248. Throughput: 0: 1813.8, 1: 1829.9. Samples: 22421948. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) +[2023-10-09 13:56:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:56:39,461][86122] Updated weights for policy 1, policy_version 43880 (0.0009) +[2023-10-09 13:56:39,811][86122] Updated weights for policy 1, policy_version 43890 (0.0008) +[2023-10-09 13:56:40,174][86122] Updated weights for policy 1, policy_version 43900 (0.0007) +[2023-10-09 13:56:41,260][86121] Updated weights for policy 0, policy_version 43690 (0.0007) +[2023-10-09 13:56:41,627][86121] Updated weights for policy 0, policy_version 43700 (0.0008) +[2023-10-09 13:56:41,991][86121] Updated weights for policy 0, policy_version 43710 (0.0009) +[2023-10-09 13:56:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 89718784. Throughput: 0: 1815.6, 1: 1828.4. Samples: 22433088. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) +[2023-10-09 13:56:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 13:56:43,860][86122] Updated weights for policy 1, policy_version 43910 (0.0010) +[2023-10-09 13:56:44,211][86122] Updated weights for policy 1, policy_version 43920 (0.0008) +[2023-10-09 13:56:44,579][86122] Updated weights for policy 1, policy_version 43930 (0.0009) +[2023-10-09 13:56:45,830][86121] Updated weights for policy 0, policy_version 43720 (0.0009) +[2023-10-09 13:56:46,202][86121] Updated weights for policy 0, policy_version 43730 (0.0009) +[2023-10-09 13:56:46,573][86121] Updated weights for policy 0, policy_version 43740 (0.0007) +[2023-10-09 13:56:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89784320. Throughput: 0: 1809.4, 1: 1824.0. Samples: 22454248. Policy #0 lag: (min: 31.0, avg: 37.5, max: 63.0) +[2023-10-09 13:56:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:56:48,544][86122] Updated weights for policy 1, policy_version 43940 (0.0007) +[2023-10-09 13:56:48,913][86122] Updated weights for policy 1, policy_version 43950 (0.0008) +[2023-10-09 13:56:49,275][86122] Updated weights for policy 1, policy_version 43960 (0.0007) +[2023-10-09 13:56:50,285][86121] Updated weights for policy 0, policy_version 43750 (0.0010) +[2023-10-09 13:56:50,646][86121] Updated weights for policy 0, policy_version 43760 (0.0007) +[2023-10-09 13:56:51,008][86121] Updated weights for policy 0, policy_version 43770 (0.0007) +[2023-10-09 13:56:53,055][86122] Updated weights for policy 1, policy_version 43970 (0.0008) +[2023-10-09 13:56:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89849856. Throughput: 0: 1809.6, 1: 1823.6. Samples: 22476930. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 13:56:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:56:53,422][86122] Updated weights for policy 1, policy_version 43980 (0.0008) +[2023-10-09 13:56:53,789][86122] Updated weights for policy 1, policy_version 43990 (0.0008) +[2023-10-09 13:56:54,148][86122] Updated weights for policy 1, policy_version 44000 (0.0008) +[2023-10-09 13:56:54,730][86121] Updated weights for policy 0, policy_version 43780 (0.0008) +[2023-10-09 13:56:55,098][86121] Updated weights for policy 0, policy_version 43790 (0.0012) +[2023-10-09 13:56:55,457][86121] Updated weights for policy 0, policy_version 43800 (0.0009) +[2023-10-09 13:56:57,848][86122] Updated weights for policy 1, policy_version 44010 (0.0009) +[2023-10-09 13:56:58,216][86122] Updated weights for policy 1, policy_version 44020 (0.0008) +[2023-10-09 13:56:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 89915392. Throughput: 0: 1808.9, 1: 1823.6. Samples: 22486934. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 13:56:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:56:58,573][86122] Updated weights for policy 1, policy_version 44030 (0.0007) +[2023-10-09 13:56:59,199][86121] Updated weights for policy 0, policy_version 43810 (0.0010) +[2023-10-09 13:56:59,569][86121] Updated weights for policy 0, policy_version 43820 (0.0007) +[2023-10-09 13:56:59,933][86121] Updated weights for policy 0, policy_version 43830 (0.0010) +[2023-10-09 13:57:00,302][86121] Updated weights for policy 0, policy_version 43840 (0.0011) +[2023-10-09 13:57:02,208][86122] Updated weights for policy 1, policy_version 44040 (0.0009) +[2023-10-09 13:57:02,570][86122] Updated weights for policy 1, policy_version 44050 (0.0008) +[2023-10-09 13:57:02,938][86122] Updated weights for policy 1, policy_version 44060 (0.0010) +[2023-10-09 13:57:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90013696. Throughput: 0: 1809.3, 1: 1827.7. Samples: 22509790. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 13:57:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:04,110][86121] Updated weights for policy 0, policy_version 43850 (0.0010) +[2023-10-09 13:57:04,489][86121] Updated weights for policy 0, policy_version 43860 (0.0008) +[2023-10-09 13:57:04,850][86121] Updated weights for policy 0, policy_version 43870 (0.0008) +[2023-10-09 13:57:06,574][86122] Updated weights for policy 1, policy_version 44070 (0.0009) +[2023-10-09 13:57:06,937][86122] Updated weights for policy 1, policy_version 44080 (0.0008) +[2023-10-09 13:57:07,309][86122] Updated weights for policy 1, policy_version 44090 (0.0008) +[2023-10-09 13:57:08,376][86121] Updated weights for policy 0, policy_version 43880 (0.0007) +[2023-10-09 13:57:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90079232. Throughput: 0: 1818.4, 1: 1825.2. Samples: 22531620. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 13:57:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:08,738][86121] Updated weights for policy 0, policy_version 43890 (0.0007) +[2023-10-09 13:57:09,101][86121] Updated weights for policy 0, policy_version 43900 (0.0007) +[2023-10-09 13:57:10,899][86122] Updated weights for policy 1, policy_version 44100 (0.0007) +[2023-10-09 13:57:11,259][86122] Updated weights for policy 1, policy_version 44110 (0.0008) +[2023-10-09 13:57:11,622][86122] Updated weights for policy 1, policy_version 44120 (0.0008) +[2023-10-09 13:57:12,846][86121] Updated weights for policy 0, policy_version 43910 (0.0008) +[2023-10-09 13:57:13,208][86121] Updated weights for policy 0, policy_version 43920 (0.0008) +[2023-10-09 13:57:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90144768. Throughput: 0: 1817.1, 1: 1828.4. Samples: 22543026. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 13:57:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:13,565][86121] Updated weights for policy 0, policy_version 43930 (0.0007) +[2023-10-09 13:57:15,055][86122] Updated weights for policy 1, policy_version 44130 (0.0010) +[2023-10-09 13:57:15,413][86122] Updated weights for policy 1, policy_version 44140 (0.0010) +[2023-10-09 13:57:15,774][86122] Updated weights for policy 1, policy_version 44150 (0.0011) +[2023-10-09 13:57:16,129][86122] Updated weights for policy 1, policy_version 44160 (0.0010) +[2023-10-09 13:57:17,427][86121] Updated weights for policy 0, policy_version 43940 (0.0008) +[2023-10-09 13:57:17,795][86121] Updated weights for policy 0, policy_version 43950 (0.0008) +[2023-10-09 13:57:18,165][86121] Updated weights for policy 0, policy_version 43960 (0.0008) +[2023-10-09 13:57:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 90210304. Throughput: 0: 1821.2, 1: 1833.1. Samples: 22564712. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 13:57:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:19,905][86122] Updated weights for policy 1, policy_version 44170 (0.0009) +[2023-10-09 13:57:20,252][86122] Updated weights for policy 1, policy_version 44180 (0.0008) +[2023-10-09 13:57:20,616][86122] Updated weights for policy 1, policy_version 44190 (0.0008) +[2023-10-09 13:57:21,859][86121] Updated weights for policy 0, policy_version 43970 (0.0008) +[2023-10-09 13:57:22,237][86121] Updated weights for policy 0, policy_version 43980 (0.0008) +[2023-10-09 13:57:22,603][86121] Updated weights for policy 0, policy_version 43990 (0.0007) +[2023-10-09 13:57:22,972][86121] Updated weights for policy 0, policy_version 44000 (0.0008) +[2023-10-09 13:57:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90308608. Throughput: 0: 1819.3, 1: 1833.2. Samples: 22586310. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:57:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000044000_45056000.pth... +[2023-10-09 13:57:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000044192_45252608.pth... +[2023-10-09 13:57:23,438][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000042304_43319296.pth +[2023-10-09 13:57:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000042496_43515904.pth +[2023-10-09 13:57:24,410][86122] Updated weights for policy 1, policy_version 44200 (0.0009) +[2023-10-09 13:57:24,783][86122] Updated weights for policy 1, policy_version 44210 (0.0010) +[2023-10-09 13:57:25,154][86122] Updated weights for policy 1, policy_version 44220 (0.0008) +[2023-10-09 13:57:26,480][86121] Updated weights for policy 0, policy_version 44010 (0.0010) +[2023-10-09 13:57:26,838][86121] Updated weights for policy 0, policy_version 44020 (0.0010) +[2023-10-09 13:57:27,210][86121] Updated weights for policy 0, policy_version 44030 (0.0010) +[2023-10-09 13:57:28,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 90374144. Throughput: 0: 1823.9, 1: 1830.7. Samples: 22597550. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:57:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:28,854][86122] Updated weights for policy 1, policy_version 44230 (0.0009) +[2023-10-09 13:57:29,214][86122] Updated weights for policy 1, policy_version 44240 (0.0009) +[2023-10-09 13:57:29,564][86122] Updated weights for policy 1, policy_version 44250 (0.0008) +[2023-10-09 13:57:30,960][86121] Updated weights for policy 0, policy_version 44040 (0.0009) +[2023-10-09 13:57:31,326][86121] Updated weights for policy 0, policy_version 44050 (0.0009) +[2023-10-09 13:57:31,696][86121] Updated weights for policy 0, policy_version 44060 (0.0007) +[2023-10-09 13:57:33,274][86122] Updated weights for policy 1, policy_version 44260 (0.0009) +[2023-10-09 13:57:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90439680. Throughput: 0: 1828.0, 1: 1832.5. Samples: 22618972. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:57:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:33,643][86122] Updated weights for policy 1, policy_version 44270 (0.0007) +[2023-10-09 13:57:34,002][86122] Updated weights for policy 1, policy_version 44280 (0.0010) +[2023-10-09 13:57:35,096][86121] Updated weights for policy 0, policy_version 44070 (0.0008) +[2023-10-09 13:57:35,447][86121] Updated weights for policy 0, policy_version 44080 (0.0008) +[2023-10-09 13:57:35,810][86121] Updated weights for policy 0, policy_version 44090 (0.0008) +[2023-10-09 13:57:37,677][86122] Updated weights for policy 1, policy_version 44290 (0.0009) +[2023-10-09 13:57:38,037][86122] Updated weights for policy 1, policy_version 44300 (0.0010) +[2023-10-09 13:57:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 90505216. Throughput: 0: 1833.9, 1: 1833.0. Samples: 22641940. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:57:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:57:38,399][86122] Updated weights for policy 1, policy_version 44310 (0.0007) +[2023-10-09 13:57:38,757][86122] Updated weights for policy 1, policy_version 44320 (0.0007) +[2023-10-09 13:57:39,512][86121] Updated weights for policy 0, policy_version 44100 (0.0009) +[2023-10-09 13:57:39,870][86121] Updated weights for policy 0, policy_version 44110 (0.0010) +[2023-10-09 13:57:40,229][86121] Updated weights for policy 0, policy_version 44120 (0.0008) +[2023-10-09 13:57:42,479][86122] Updated weights for policy 1, policy_version 44330 (0.0008) +[2023-10-09 13:57:42,844][86122] Updated weights for policy 1, policy_version 44340 (0.0007) +[2023-10-09 13:57:43,212][86122] Updated weights for policy 1, policy_version 44350 (0.0010) +[2023-10-09 13:57:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 90603520. Throughput: 0: 1836.1, 1: 1838.9. Samples: 22652308. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:57:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:57:43,986][86121] Updated weights for policy 0, policy_version 44130 (0.0007) +[2023-10-09 13:57:44,359][86121] Updated weights for policy 0, policy_version 44140 (0.0009) +[2023-10-09 13:57:44,725][86121] Updated weights for policy 0, policy_version 44150 (0.0009) +[2023-10-09 13:57:45,079][86121] Updated weights for policy 0, policy_version 44160 (0.0007) +[2023-10-09 13:57:46,942][86122] Updated weights for policy 1, policy_version 44360 (0.0007) +[2023-10-09 13:57:47,309][86122] Updated weights for policy 1, policy_version 44370 (0.0007) +[2023-10-09 13:57:47,674][86122] Updated weights for policy 1, policy_version 44380 (0.0008) +[2023-10-09 13:57:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90669056. Throughput: 0: 1836.7, 1: 1825.6. Samples: 22674592. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:57:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:57:49,018][86121] Updated weights for policy 0, policy_version 44170 (0.0010) +[2023-10-09 13:57:49,376][86121] Updated weights for policy 0, policy_version 44180 (0.0008) +[2023-10-09 13:57:49,745][86121] Updated weights for policy 0, policy_version 44190 (0.0007) +[2023-10-09 13:57:51,189][86122] Updated weights for policy 1, policy_version 44390 (0.0009) +[2023-10-09 13:57:51,550][86122] Updated weights for policy 1, policy_version 44400 (0.0007) +[2023-10-09 13:57:51,915][86122] Updated weights for policy 1, policy_version 44410 (0.0008) +[2023-10-09 13:57:53,336][86121] Updated weights for policy 0, policy_version 44200 (0.0007) +[2023-10-09 13:57:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 90734592. Throughput: 0: 1827.7, 1: 1831.0. Samples: 22696264. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 13:57:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:57:53,697][86121] Updated weights for policy 0, policy_version 44210 (0.0008) +[2023-10-09 13:57:54,057][86121] Updated weights for policy 0, policy_version 44220 (0.0007) +[2023-10-09 13:57:55,517][86122] Updated weights for policy 1, policy_version 44420 (0.0009) +[2023-10-09 13:57:55,874][86122] Updated weights for policy 1, policy_version 44430 (0.0007) +[2023-10-09 13:57:56,247][86122] Updated weights for policy 1, policy_version 44440 (0.0007) +[2023-10-09 13:57:57,817][86121] Updated weights for policy 0, policy_version 44230 (0.0008) +[2023-10-09 13:57:58,178][86121] Updated weights for policy 0, policy_version 44240 (0.0007) +[2023-10-09 13:57:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 90800128. Throughput: 0: 1826.7, 1: 1822.0. Samples: 22707222. Policy #0 lag: (min: 1.0, avg: 17.4, max: 33.0) +[2023-10-09 13:57:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:57:58,548][86121] Updated weights for policy 0, policy_version 44250 (0.0007) +[2023-10-09 13:57:59,967][86122] Updated weights for policy 1, policy_version 44450 (0.0007) +[2023-10-09 13:58:00,325][86122] Updated weights for policy 1, policy_version 44460 (0.0009) +[2023-10-09 13:58:00,689][86122] Updated weights for policy 1, policy_version 44470 (0.0008) +[2023-10-09 13:58:01,048][86122] Updated weights for policy 1, policy_version 44480 (0.0010) +[2023-10-09 13:58:02,122][86121] Updated weights for policy 0, policy_version 44260 (0.0008) +[2023-10-09 13:58:02,472][86121] Updated weights for policy 0, policy_version 44270 (0.0009) +[2023-10-09 13:58:02,833][86121] Updated weights for policy 0, policy_version 44280 (0.0008) +[2023-10-09 13:58:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90898432. Throughput: 0: 1826.4, 1: 1831.5. Samples: 22729318. Policy #0 lag: (min: 1.0, avg: 17.4, max: 33.0) +[2023-10-09 13:58:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:04,643][86122] Updated weights for policy 1, policy_version 44490 (0.0009) +[2023-10-09 13:58:05,005][86122] Updated weights for policy 1, policy_version 44500 (0.0008) +[2023-10-09 13:58:05,365][86122] Updated weights for policy 1, policy_version 44510 (0.0009) +[2023-10-09 13:58:06,445][86121] Updated weights for policy 0, policy_version 44290 (0.0008) +[2023-10-09 13:58:06,816][86121] Updated weights for policy 0, policy_version 44300 (0.0007) +[2023-10-09 13:58:07,173][86121] Updated weights for policy 0, policy_version 44310 (0.0008) +[2023-10-09 13:58:07,543][86121] Updated weights for policy 0, policy_version 44320 (0.0011) +[2023-10-09 13:58:08,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 90963968. Throughput: 0: 1827.1, 1: 1832.5. Samples: 22750988. Policy #0 lag: (min: 1.0, avg: 17.4, max: 33.0) +[2023-10-09 13:58:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:08,971][86122] Updated weights for policy 1, policy_version 44520 (0.0008) +[2023-10-09 13:58:09,332][86122] Updated weights for policy 1, policy_version 44530 (0.0007) +[2023-10-09 13:58:09,693][86122] Updated weights for policy 1, policy_version 44540 (0.0009) +[2023-10-09 13:58:11,233][86121] Updated weights for policy 0, policy_version 44330 (0.0008) +[2023-10-09 13:58:11,609][86121] Updated weights for policy 0, policy_version 44340 (0.0010) +[2023-10-09 13:58:11,966][86121] Updated weights for policy 0, policy_version 44350 (0.0009) +[2023-10-09 13:58:13,387][86122] Updated weights for policy 1, policy_version 44550 (0.0009) +[2023-10-09 13:58:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91029504. Throughput: 0: 1821.1, 1: 1840.1. Samples: 22762302. Policy #0 lag: (min: 1.0, avg: 17.4, max: 33.0) +[2023-10-09 13:58:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:13,749][86122] Updated weights for policy 1, policy_version 44560 (0.0007) +[2023-10-09 13:58:14,111][86122] Updated weights for policy 1, policy_version 44570 (0.0010) +[2023-10-09 13:58:15,566][86121] Updated weights for policy 0, policy_version 44360 (0.0008) +[2023-10-09 13:58:15,935][86121] Updated weights for policy 0, policy_version 44370 (0.0008) +[2023-10-09 13:58:16,294][86121] Updated weights for policy 0, policy_version 44380 (0.0007) +[2023-10-09 13:58:17,861][86122] Updated weights for policy 1, policy_version 44580 (0.0008) +[2023-10-09 13:58:18,214][86122] Updated weights for policy 1, policy_version 44590 (0.0008) +[2023-10-09 13:58:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 91095040. Throughput: 0: 1828.6, 1: 1841.6. Samples: 22784132. Policy #0 lag: (min: 1.0, avg: 17.4, max: 33.0) +[2023-10-09 13:58:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:18,586][86122] Updated weights for policy 1, policy_version 44600 (0.0007) +[2023-10-09 13:58:19,859][86121] Updated weights for policy 0, policy_version 44390 (0.0011) +[2023-10-09 13:58:20,218][86121] Updated weights for policy 0, policy_version 44400 (0.0009) +[2023-10-09 13:58:20,583][86121] Updated weights for policy 0, policy_version 44410 (0.0010) +[2023-10-09 13:58:22,451][86122] Updated weights for policy 1, policy_version 44610 (0.0008) +[2023-10-09 13:58:22,818][86122] Updated weights for policy 1, policy_version 44620 (0.0009) +[2023-10-09 13:58:23,178][86122] Updated weights for policy 1, policy_version 44630 (0.0009) +[2023-10-09 13:58:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 91160576. Throughput: 0: 1830.2, 1: 1830.9. Samples: 22806692. Policy #0 lag: (min: 1.0, avg: 17.4, max: 33.0) +[2023-10-09 13:58:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:23,544][86122] Updated weights for policy 1, policy_version 44640 (0.0007) +[2023-10-09 13:58:24,320][86121] Updated weights for policy 0, policy_version 44420 (0.0009) +[2023-10-09 13:58:24,686][86121] Updated weights for policy 0, policy_version 44430 (0.0009) +[2023-10-09 13:58:25,058][86121] Updated weights for policy 0, policy_version 44440 (0.0008) +[2023-10-09 13:58:27,154][86122] Updated weights for policy 1, policy_version 44650 (0.0009) +[2023-10-09 13:58:27,510][86122] Updated weights for policy 1, policy_version 44660 (0.0010) +[2023-10-09 13:58:27,876][86122] Updated weights for policy 1, policy_version 44670 (0.0009) +[2023-10-09 13:58:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 91258880. Throughput: 0: 1826.5, 1: 1834.8. Samples: 22817064. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:58:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:28,899][86121] Updated weights for policy 0, policy_version 44450 (0.0010) +[2023-10-09 13:58:29,262][86121] Updated weights for policy 0, policy_version 44460 (0.0009) +[2023-10-09 13:58:29,632][86121] Updated weights for policy 0, policy_version 44470 (0.0009) +[2023-10-09 13:58:29,997][86121] Updated weights for policy 0, policy_version 44480 (0.0007) +[2023-10-09 13:58:31,549][86122] Updated weights for policy 1, policy_version 44680 (0.0009) +[2023-10-09 13:58:31,910][86122] Updated weights for policy 1, policy_version 44690 (0.0010) +[2023-10-09 13:58:32,264][86122] Updated weights for policy 1, policy_version 44700 (0.0010) +[2023-10-09 13:58:33,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91324416. Throughput: 0: 1831.9, 1: 1827.4. Samples: 22839258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:58:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:33,695][86121] Updated weights for policy 0, policy_version 44490 (0.0008) +[2023-10-09 13:58:34,062][86121] Updated weights for policy 0, policy_version 44500 (0.0007) +[2023-10-09 13:58:34,426][86121] Updated weights for policy 0, policy_version 44510 (0.0009) +[2023-10-09 13:58:35,797][86122] Updated weights for policy 1, policy_version 44710 (0.0010) +[2023-10-09 13:58:36,154][86122] Updated weights for policy 1, policy_version 44720 (0.0007) +[2023-10-09 13:58:36,513][86122] Updated weights for policy 1, policy_version 44730 (0.0009) +[2023-10-09 13:58:38,104][86121] Updated weights for policy 0, policy_version 44520 (0.0007) +[2023-10-09 13:58:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91389952. Throughput: 0: 1829.9, 1: 1842.6. Samples: 22861526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:58:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:38,478][86121] Updated weights for policy 0, policy_version 44530 (0.0008) +[2023-10-09 13:58:38,845][86121] Updated weights for policy 0, policy_version 44540 (0.0007) +[2023-10-09 13:58:40,135][86122] Updated weights for policy 1, policy_version 44740 (0.0008) +[2023-10-09 13:58:40,492][86122] Updated weights for policy 1, policy_version 44750 (0.0007) +[2023-10-09 13:58:40,853][86122] Updated weights for policy 1, policy_version 44760 (0.0008) +[2023-10-09 13:58:42,590][86121] Updated weights for policy 0, policy_version 44550 (0.0008) +[2023-10-09 13:58:42,953][86121] Updated weights for policy 0, policy_version 44560 (0.0007) +[2023-10-09 13:58:43,325][86121] Updated weights for policy 0, policy_version 44570 (0.0009) +[2023-10-09 13:58:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 91455488. Throughput: 0: 1830.3, 1: 1829.0. Samples: 22871890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:58:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:44,652][86122] Updated weights for policy 1, policy_version 44770 (0.0008) +[2023-10-09 13:58:45,021][86122] Updated weights for policy 1, policy_version 44780 (0.0008) +[2023-10-09 13:58:45,374][86122] Updated weights for policy 1, policy_version 44790 (0.0008) +[2023-10-09 13:58:45,738][86122] Updated weights for policy 1, policy_version 44800 (0.0007) +[2023-10-09 13:58:46,907][86121] Updated weights for policy 0, policy_version 44580 (0.0007) +[2023-10-09 13:58:47,276][86121] Updated weights for policy 0, policy_version 44590 (0.0009) +[2023-10-09 13:58:47,645][86121] Updated weights for policy 0, policy_version 44600 (0.0010) +[2023-10-09 13:58:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91553792. Throughput: 0: 1830.4, 1: 1839.7. Samples: 22894472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:58:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:49,358][86122] Updated weights for policy 1, policy_version 44810 (0.0009) +[2023-10-09 13:58:49,725][86122] Updated weights for policy 1, policy_version 44820 (0.0008) +[2023-10-09 13:58:50,089][86122] Updated weights for policy 1, policy_version 44830 (0.0008) +[2023-10-09 13:58:51,306][86121] Updated weights for policy 0, policy_version 44610 (0.0010) +[2023-10-09 13:58:51,672][86121] Updated weights for policy 0, policy_version 44620 (0.0010) +[2023-10-09 13:58:52,029][86121] Updated weights for policy 0, policy_version 44630 (0.0007) +[2023-10-09 13:58:52,392][86121] Updated weights for policy 0, policy_version 44640 (0.0007) +[2023-10-09 13:58:53,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 91619328. Throughput: 0: 1833.3, 1: 1837.1. Samples: 22916156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 13:58:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:53,795][86122] Updated weights for policy 1, policy_version 44840 (0.0008) +[2023-10-09 13:58:54,161][86122] Updated weights for policy 1, policy_version 44850 (0.0008) +[2023-10-09 13:58:54,521][86122] Updated weights for policy 1, policy_version 44860 (0.0008) +[2023-10-09 13:58:55,863][86121] Updated weights for policy 0, policy_version 44650 (0.0008) +[2023-10-09 13:58:56,232][86121] Updated weights for policy 0, policy_version 44660 (0.0007) +[2023-10-09 13:58:56,600][86121] Updated weights for policy 0, policy_version 44670 (0.0008) +[2023-10-09 13:58:58,367][86122] Updated weights for policy 1, policy_version 44870 (0.0007) +[2023-10-09 13:58:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91684864. Throughput: 0: 1826.2, 1: 1836.1. Samples: 22927108. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) +[2023-10-09 13:58:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:58:58,746][86122] Updated weights for policy 1, policy_version 44880 (0.0008) +[2023-10-09 13:58:59,108][86122] Updated weights for policy 1, policy_version 44890 (0.0008) +[2023-10-09 13:59:00,416][86121] Updated weights for policy 0, policy_version 44680 (0.0008) +[2023-10-09 13:59:00,789][86121] Updated weights for policy 0, policy_version 44690 (0.0008) +[2023-10-09 13:59:01,153][86121] Updated weights for policy 0, policy_version 44700 (0.0007) +[2023-10-09 13:59:02,670][86122] Updated weights for policy 1, policy_version 44900 (0.0009) +[2023-10-09 13:59:03,026][86122] Updated weights for policy 1, policy_version 44910 (0.0010) +[2023-10-09 13:59:03,396][86122] Updated weights for policy 1, policy_version 44920 (0.0011) +[2023-10-09 13:59:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 91750400. Throughput: 0: 1829.6, 1: 1823.8. Samples: 22948538. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) +[2023-10-09 13:59:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:04,848][86121] Updated weights for policy 0, policy_version 44710 (0.0007) +[2023-10-09 13:59:05,205][86121] Updated weights for policy 0, policy_version 44720 (0.0009) +[2023-10-09 13:59:05,571][86121] Updated weights for policy 0, policy_version 44730 (0.0008) +[2023-10-09 13:59:07,068][86122] Updated weights for policy 1, policy_version 44930 (0.0008) +[2023-10-09 13:59:07,433][86122] Updated weights for policy 1, policy_version 44940 (0.0007) +[2023-10-09 13:59:07,793][86122] Updated weights for policy 1, policy_version 44950 (0.0008) +[2023-10-09 13:59:08,143][86122] Updated weights for policy 1, policy_version 44960 (0.0008) +[2023-10-09 13:59:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 91848704. Throughput: 0: 1830.3, 1: 1819.4. Samples: 22970930. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) +[2023-10-09 13:59:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:09,251][86121] Updated weights for policy 0, policy_version 44740 (0.0008) +[2023-10-09 13:59:09,616][86121] Updated weights for policy 0, policy_version 44750 (0.0007) +[2023-10-09 13:59:09,982][86121] Updated weights for policy 0, policy_version 44760 (0.0007) +[2023-10-09 13:59:11,837][86122] Updated weights for policy 1, policy_version 44970 (0.0009) +[2023-10-09 13:59:12,194][86122] Updated weights for policy 1, policy_version 44980 (0.0009) +[2023-10-09 13:59:12,552][86122] Updated weights for policy 1, policy_version 44990 (0.0007) +[2023-10-09 13:59:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91914240. Throughput: 0: 1834.2, 1: 1828.8. Samples: 22981900. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) +[2023-10-09 13:59:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:13,573][86121] Updated weights for policy 0, policy_version 44770 (0.0010) +[2023-10-09 13:59:13,949][86121] Updated weights for policy 0, policy_version 44780 (0.0009) +[2023-10-09 13:59:14,312][86121] Updated weights for policy 0, policy_version 44790 (0.0008) +[2023-10-09 13:59:14,674][86121] Updated weights for policy 0, policy_version 44800 (0.0009) +[2023-10-09 13:59:16,111][86122] Updated weights for policy 1, policy_version 45000 (0.0008) +[2023-10-09 13:59:16,467][86122] Updated weights for policy 1, policy_version 45010 (0.0011) +[2023-10-09 13:59:16,823][86122] Updated weights for policy 1, policy_version 45020 (0.0010) +[2023-10-09 13:59:18,395][86121] Updated weights for policy 0, policy_version 44810 (0.0010) +[2023-10-09 13:59:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 91979776. Throughput: 0: 1837.4, 1: 1818.3. Samples: 23003764. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) +[2023-10-09 13:59:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:18,766][86121] Updated weights for policy 0, policy_version 44820 (0.0007) +[2023-10-09 13:59:19,133][86121] Updated weights for policy 0, policy_version 44830 (0.0009) +[2023-10-09 13:59:20,633][86122] Updated weights for policy 1, policy_version 45030 (0.0009) +[2023-10-09 13:59:20,986][86122] Updated weights for policy 1, policy_version 45040 (0.0010) +[2023-10-09 13:59:21,346][86122] Updated weights for policy 1, policy_version 45050 (0.0010) +[2023-10-09 13:59:22,856][86121] Updated weights for policy 0, policy_version 44840 (0.0010) +[2023-10-09 13:59:23,221][86121] Updated weights for policy 0, policy_version 44850 (0.0011) +[2023-10-09 13:59:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92045312. Throughput: 0: 1825.3, 1: 1821.0. Samples: 23025608. Policy #0 lag: (min: 31.0, avg: 31.6, max: 49.0) +[2023-10-09 13:59:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000045056_46137344.pth... +[2023-10-09 13:59:23,440][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000043360_44400640.pth +[2023-10-09 13:59:23,586][86121] Updated weights for policy 0, policy_version 44860 (0.0009) +[2023-10-09 13:59:23,731][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000044864_45940736.pth... +[2023-10-09 13:59:23,760][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000043136_44171264.pth +[2023-10-09 13:59:25,099][86122] Updated weights for policy 1, policy_version 45060 (0.0010) +[2023-10-09 13:59:25,462][86122] Updated weights for policy 1, policy_version 45070 (0.0008) +[2023-10-09 13:59:25,825][86122] Updated weights for policy 1, policy_version 45080 (0.0009) +[2023-10-09 13:59:27,045][86121] Updated weights for policy 0, policy_version 44870 (0.0008) +[2023-10-09 13:59:27,416][86121] Updated weights for policy 0, policy_version 44880 (0.0010) +[2023-10-09 13:59:27,782][86121] Updated weights for policy 0, policy_version 44890 (0.0008) +[2023-10-09 13:59:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92143616. Throughput: 0: 1840.7, 1: 1821.0. Samples: 23036666. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) +[2023-10-09 13:59:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:29,558][86122] Updated weights for policy 1, policy_version 45090 (0.0010) +[2023-10-09 13:59:29,918][86122] Updated weights for policy 1, policy_version 45100 (0.0008) +[2023-10-09 13:59:30,277][86122] Updated weights for policy 1, policy_version 45110 (0.0007) +[2023-10-09 13:59:30,642][86122] Updated weights for policy 1, policy_version 45120 (0.0007) +[2023-10-09 13:59:31,348][86121] Updated weights for policy 0, policy_version 44900 (0.0007) +[2023-10-09 13:59:31,699][86121] Updated weights for policy 0, policy_version 44910 (0.0007) +[2023-10-09 13:59:32,069][86121] Updated weights for policy 0, policy_version 44920 (0.0008) +[2023-10-09 13:59:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92209152. Throughput: 0: 1825.1, 1: 1815.2. Samples: 23058288. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) +[2023-10-09 13:59:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:34,368][86122] Updated weights for policy 1, policy_version 45130 (0.0009) +[2023-10-09 13:59:34,731][86122] Updated weights for policy 1, policy_version 45140 (0.0008) +[2023-10-09 13:59:35,096][86122] Updated weights for policy 1, policy_version 45150 (0.0008) +[2023-10-09 13:59:35,834][86121] Updated weights for policy 0, policy_version 44930 (0.0008) +[2023-10-09 13:59:36,198][86121] Updated weights for policy 0, policy_version 44940 (0.0009) +[2023-10-09 13:59:36,568][86121] Updated weights for policy 0, policy_version 44950 (0.0009) +[2023-10-09 13:59:36,927][86121] Updated weights for policy 0, policy_version 44960 (0.0009) +[2023-10-09 13:59:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92274688. Throughput: 0: 1839.4, 1: 1812.8. Samples: 23080504. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) +[2023-10-09 13:59:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 13:59:38,815][86122] Updated weights for policy 1, policy_version 45160 (0.0008) +[2023-10-09 13:59:39,174][86122] Updated weights for policy 1, policy_version 45170 (0.0009) +[2023-10-09 13:59:39,546][86122] Updated weights for policy 1, policy_version 45180 (0.0009) +[2023-10-09 13:59:40,454][86121] Updated weights for policy 0, policy_version 44970 (0.0010) +[2023-10-09 13:59:40,820][86121] Updated weights for policy 0, policy_version 44980 (0.0007) +[2023-10-09 13:59:41,176][86121] Updated weights for policy 0, policy_version 44990 (0.0007) +[2023-10-09 13:59:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 92340224. Throughput: 0: 1828.6, 1: 1812.5. Samples: 23090958. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) +[2023-10-09 13:59:43,399][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 13:59:43,485][86122] Updated weights for policy 1, policy_version 45190 (0.0010) +[2023-10-09 13:59:43,866][86122] Updated weights for policy 1, policy_version 45200 (0.0009) +[2023-10-09 13:59:44,229][86122] Updated weights for policy 1, policy_version 45210 (0.0008) +[2023-10-09 13:59:44,889][86121] Updated weights for policy 0, policy_version 45000 (0.0009) +[2023-10-09 13:59:45,254][86121] Updated weights for policy 0, policy_version 45010 (0.0009) +[2023-10-09 13:59:45,620][86121] Updated weights for policy 0, policy_version 45020 (0.0009) +[2023-10-09 13:59:47,823][86122] Updated weights for policy 1, policy_version 45220 (0.0009) +[2023-10-09 13:59:48,185][86122] Updated weights for policy 1, policy_version 45230 (0.0010) +[2023-10-09 13:59:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 92405760. Throughput: 0: 1837.0, 1: 1813.9. Samples: 23112830. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) +[2023-10-09 13:59:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 13:59:48,552][86122] Updated weights for policy 1, policy_version 45240 (0.0009) +[2023-10-09 13:59:49,335][86121] Updated weights for policy 0, policy_version 45030 (0.0008) +[2023-10-09 13:59:49,705][86121] Updated weights for policy 0, policy_version 45040 (0.0009) +[2023-10-09 13:59:50,075][86121] Updated weights for policy 0, policy_version 45050 (0.0007) +[2023-10-09 13:59:52,163][86122] Updated weights for policy 1, policy_version 45250 (0.0009) +[2023-10-09 13:59:52,521][86122] Updated weights for policy 1, policy_version 45260 (0.0009) +[2023-10-09 13:59:52,894][86122] Updated weights for policy 1, policy_version 45270 (0.0008) +[2023-10-09 13:59:53,258][86122] Updated weights for policy 1, policy_version 45280 (0.0008) +[2023-10-09 13:59:53,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 92504064. Throughput: 0: 1832.4, 1: 1815.0. Samples: 23135062. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) +[2023-10-09 13:59:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 13:59:53,750][86121] Updated weights for policy 0, policy_version 45060 (0.0010) +[2023-10-09 13:59:54,114][86121] Updated weights for policy 0, policy_version 45070 (0.0009) +[2023-10-09 13:59:54,484][86121] Updated weights for policy 0, policy_version 45080 (0.0009) +[2023-10-09 13:59:56,745][86122] Updated weights for policy 1, policy_version 45290 (0.0008) +[2023-10-09 13:59:57,106][86122] Updated weights for policy 1, policy_version 45300 (0.0007) +[2023-10-09 13:59:57,474][86122] Updated weights for policy 1, policy_version 45310 (0.0007) +[2023-10-09 13:59:58,236][86121] Updated weights for policy 0, policy_version 45090 (0.0009) +[2023-10-09 13:59:58,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92569600. Throughput: 0: 1829.1, 1: 1819.9. Samples: 23146102. Policy #0 lag: (min: 7.0, avg: 15.0, max: 39.0) +[2023-10-09 13:59:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 13:59:58,617][86121] Updated weights for policy 0, policy_version 45100 (0.0008) +[2023-10-09 13:59:58,982][86121] Updated weights for policy 0, policy_version 45110 (0.0008) +[2023-10-09 13:59:59,348][86121] Updated weights for policy 0, policy_version 45120 (0.0008) +[2023-10-09 14:00:01,192][86122] Updated weights for policy 1, policy_version 45320 (0.0009) +[2023-10-09 14:00:01,548][86122] Updated weights for policy 1, policy_version 45330 (0.0009) +[2023-10-09 14:00:01,915][86122] Updated weights for policy 1, policy_version 45340 (0.0008) +[2023-10-09 14:00:03,179][86121] Updated weights for policy 0, policy_version 45130 (0.0009) +[2023-10-09 14:00:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 92635136. Throughput: 0: 1824.9, 1: 1819.3. Samples: 23167754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:00:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:00:03,545][86121] Updated weights for policy 0, policy_version 45140 (0.0009) +[2023-10-09 14:00:03,921][86121] Updated weights for policy 0, policy_version 45150 (0.0009) +[2023-10-09 14:00:05,591][86122] Updated weights for policy 1, policy_version 45350 (0.0008) +[2023-10-09 14:00:05,944][86122] Updated weights for policy 1, policy_version 45360 (0.0010) +[2023-10-09 14:00:06,311][86122] Updated weights for policy 1, policy_version 45370 (0.0010) +[2023-10-09 14:00:07,373][86121] Updated weights for policy 0, policy_version 45160 (0.0008) +[2023-10-09 14:00:07,740][86121] Updated weights for policy 0, policy_version 45170 (0.0008) +[2023-10-09 14:00:08,101][86121] Updated weights for policy 0, policy_version 45180 (0.0009) +[2023-10-09 14:00:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92733440. Throughput: 0: 1820.4, 1: 1816.3. Samples: 23189258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:00:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:00:10,005][86122] Updated weights for policy 1, policy_version 45380 (0.0008) +[2023-10-09 14:00:10,364][86122] Updated weights for policy 1, policy_version 45390 (0.0008) +[2023-10-09 14:00:10,724][86122] Updated weights for policy 1, policy_version 45400 (0.0009) +[2023-10-09 14:00:11,847][86121] Updated weights for policy 0, policy_version 45190 (0.0009) +[2023-10-09 14:00:12,219][86121] Updated weights for policy 0, policy_version 45200 (0.0007) +[2023-10-09 14:00:12,594][86121] Updated weights for policy 0, policy_version 45210 (0.0008) +[2023-10-09 14:00:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92798976. Throughput: 0: 1827.0, 1: 1814.0. Samples: 23200510. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:00:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:00:14,471][86122] Updated weights for policy 1, policy_version 45410 (0.0010) +[2023-10-09 14:00:14,827][86122] Updated weights for policy 1, policy_version 45420 (0.0009) +[2023-10-09 14:00:15,190][86122] Updated weights for policy 1, policy_version 45430 (0.0009) +[2023-10-09 14:00:15,551][86122] Updated weights for policy 1, policy_version 45440 (0.0010) +[2023-10-09 14:00:16,337][86121] Updated weights for policy 0, policy_version 45220 (0.0008) +[2023-10-09 14:00:16,708][86121] Updated weights for policy 0, policy_version 45230 (0.0009) +[2023-10-09 14:00:17,071][86121] Updated weights for policy 0, policy_version 45240 (0.0007) +[2023-10-09 14:00:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92864512. Throughput: 0: 1823.2, 1: 1819.9. Samples: 23222224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:00:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:00:19,172][86122] Updated weights for policy 1, policy_version 45450 (0.0008) +[2023-10-09 14:00:19,543][86122] Updated weights for policy 1, policy_version 45460 (0.0009) +[2023-10-09 14:00:19,915][86122] Updated weights for policy 1, policy_version 45470 (0.0008) +[2023-10-09 14:00:20,707][86121] Updated weights for policy 0, policy_version 45250 (0.0007) +[2023-10-09 14:00:21,067][86121] Updated weights for policy 0, policy_version 45260 (0.0007) +[2023-10-09 14:00:21,433][86121] Updated weights for policy 0, policy_version 45270 (0.0007) +[2023-10-09 14:00:21,796][86121] Updated weights for policy 0, policy_version 45280 (0.0008) +[2023-10-09 14:00:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 92930048. Throughput: 0: 1825.4, 1: 1825.5. Samples: 23244792. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:00:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:00:23,726][86122] Updated weights for policy 1, policy_version 45480 (0.0008) +[2023-10-09 14:00:24,090][86122] Updated weights for policy 1, policy_version 45490 (0.0008) +[2023-10-09 14:00:24,449][86122] Updated weights for policy 1, policy_version 45500 (0.0008) +[2023-10-09 14:00:25,453][86121] Updated weights for policy 0, policy_version 45290 (0.0010) +[2023-10-09 14:00:25,825][86121] Updated weights for policy 0, policy_version 45300 (0.0008) +[2023-10-09 14:00:26,188][86121] Updated weights for policy 0, policy_version 45310 (0.0009) +[2023-10-09 14:00:28,091][86122] Updated weights for policy 1, policy_version 45510 (0.0009) +[2023-10-09 14:00:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 92995584. Throughput: 0: 1824.6, 1: 1829.0. Samples: 23255368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:00:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:00:28,447][86122] Updated weights for policy 1, policy_version 45520 (0.0010) +[2023-10-09 14:00:28,813][86122] Updated weights for policy 1, policy_version 45530 (0.0010) +[2023-10-09 14:00:29,838][86121] Updated weights for policy 0, policy_version 45320 (0.0007) +[2023-10-09 14:00:30,211][86121] Updated weights for policy 0, policy_version 45330 (0.0011) +[2023-10-09 14:00:30,569][86121] Updated weights for policy 0, policy_version 45340 (0.0010) +[2023-10-09 14:00:32,637][86122] Updated weights for policy 1, policy_version 45540 (0.0008) +[2023-10-09 14:00:33,029][86122] Updated weights for policy 1, policy_version 45550 (0.0009) +[2023-10-09 14:00:33,393][86122] Updated weights for policy 1, policy_version 45560 (0.0008) +[2023-10-09 14:00:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 93061120. Throughput: 0: 1828.6, 1: 1837.2. Samples: 23277790. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:00:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:00:34,172][86121] Updated weights for policy 0, policy_version 45350 (0.0007) +[2023-10-09 14:00:34,541][86121] Updated weights for policy 0, policy_version 45360 (0.0009) +[2023-10-09 14:00:34,910][86121] Updated weights for policy 0, policy_version 45370 (0.0007) +[2023-10-09 14:00:37,162][86122] Updated weights for policy 1, policy_version 45570 (0.0008) +[2023-10-09 14:00:37,522][86122] Updated weights for policy 1, policy_version 45580 (0.0008) +[2023-10-09 14:00:37,875][86122] Updated weights for policy 1, policy_version 45590 (0.0009) +[2023-10-09 14:00:38,236][86122] Updated weights for policy 1, policy_version 45600 (0.0008) +[2023-10-09 14:00:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 93159424. Throughput: 0: 1826.6, 1: 1821.2. Samples: 23299212. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:00:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:00:38,711][86121] Updated weights for policy 0, policy_version 45380 (0.0007) +[2023-10-09 14:00:39,087][86121] Updated weights for policy 0, policy_version 45390 (0.0007) +[2023-10-09 14:00:39,438][86121] Updated weights for policy 0, policy_version 45400 (0.0007) +[2023-10-09 14:00:41,969][86122] Updated weights for policy 1, policy_version 45610 (0.0010) +[2023-10-09 14:00:42,321][86122] Updated weights for policy 1, policy_version 45620 (0.0008) +[2023-10-09 14:00:42,689][86122] Updated weights for policy 1, policy_version 45630 (0.0007) +[2023-10-09 14:00:43,154][86121] Updated weights for policy 0, policy_version 45410 (0.0009) +[2023-10-09 14:00:43,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 93224960. Throughput: 0: 1827.9, 1: 1813.8. Samples: 23309978. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:00:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:00:43,527][86121] Updated weights for policy 0, policy_version 45420 (0.0010) +[2023-10-09 14:00:43,888][86121] Updated weights for policy 0, policy_version 45430 (0.0011) +[2023-10-09 14:00:44,252][86121] Updated weights for policy 0, policy_version 45440 (0.0010) +[2023-10-09 14:00:46,361][86122] Updated weights for policy 1, policy_version 45640 (0.0007) +[2023-10-09 14:00:46,726][86122] Updated weights for policy 1, policy_version 45650 (0.0008) +[2023-10-09 14:00:47,072][86122] Updated weights for policy 1, policy_version 45660 (0.0010) +[2023-10-09 14:00:47,813][86121] Updated weights for policy 0, policy_version 45450 (0.0009) +[2023-10-09 14:00:48,176][86121] Updated weights for policy 0, policy_version 45460 (0.0010) +[2023-10-09 14:00:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93290496. Throughput: 0: 1834.7, 1: 1822.0. Samples: 23332308. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:00:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:00:48,547][86121] Updated weights for policy 0, policy_version 45470 (0.0010) +[2023-10-09 14:00:50,734][86122] Updated weights for policy 1, policy_version 45670 (0.0009) +[2023-10-09 14:00:51,110][86122] Updated weights for policy 1, policy_version 45680 (0.0008) +[2023-10-09 14:00:51,481][86122] Updated weights for policy 1, policy_version 45690 (0.0009) +[2023-10-09 14:00:52,298][86121] Updated weights for policy 0, policy_version 45480 (0.0009) +[2023-10-09 14:00:52,661][86121] Updated weights for policy 0, policy_version 45490 (0.0009) +[2023-10-09 14:00:53,035][86121] Updated weights for policy 0, policy_version 45500 (0.0010) +[2023-10-09 14:00:53,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 93388800. Throughput: 0: 1825.2, 1: 1819.9. Samples: 23353284. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:00:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:00:55,245][86122] Updated weights for policy 1, policy_version 45700 (0.0009) +[2023-10-09 14:00:55,602][86122] Updated weights for policy 1, policy_version 45710 (0.0010) +[2023-10-09 14:00:55,966][86122] Updated weights for policy 1, policy_version 45720 (0.0009) +[2023-10-09 14:00:56,886][86121] Updated weights for policy 0, policy_version 45510 (0.0008) +[2023-10-09 14:00:57,262][86121] Updated weights for policy 0, policy_version 45520 (0.0008) +[2023-10-09 14:00:57,628][86121] Updated weights for policy 0, policy_version 45530 (0.0009) +[2023-10-09 14:00:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93454336. Throughput: 0: 1825.1, 1: 1824.7. Samples: 23364752. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:00:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:00:59,700][86122] Updated weights for policy 1, policy_version 45730 (0.0009) +[2023-10-09 14:01:00,064][86122] Updated weights for policy 1, policy_version 45740 (0.0008) +[2023-10-09 14:01:00,413][86122] Updated weights for policy 1, policy_version 45750 (0.0007) +[2023-10-09 14:01:00,776][86122] Updated weights for policy 1, policy_version 45760 (0.0010) +[2023-10-09 14:01:01,472][86121] Updated weights for policy 0, policy_version 45540 (0.0010) +[2023-10-09 14:01:01,828][86121] Updated weights for policy 0, policy_version 45550 (0.0009) +[2023-10-09 14:01:02,193][86121] Updated weights for policy 0, policy_version 45560 (0.0010) +[2023-10-09 14:01:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93519872. Throughput: 0: 1825.3, 1: 1816.5. Samples: 23386108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:01:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:04,391][86122] Updated weights for policy 1, policy_version 45770 (0.0009) +[2023-10-09 14:01:04,757][86122] Updated weights for policy 1, policy_version 45780 (0.0007) +[2023-10-09 14:01:05,115][86122] Updated weights for policy 1, policy_version 45790 (0.0009) +[2023-10-09 14:01:05,769][86121] Updated weights for policy 0, policy_version 45570 (0.0009) +[2023-10-09 14:01:06,140][86121] Updated weights for policy 0, policy_version 45580 (0.0007) +[2023-10-09 14:01:06,502][86121] Updated weights for policy 0, policy_version 45590 (0.0007) +[2023-10-09 14:01:06,872][86121] Updated weights for policy 0, policy_version 45600 (0.0008) +[2023-10-09 14:01:08,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 93585408. Throughput: 0: 1821.5, 1: 1820.0. Samples: 23408658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:01:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:08,792][86122] Updated weights for policy 1, policy_version 45800 (0.0009) +[2023-10-09 14:01:09,154][86122] Updated weights for policy 1, policy_version 45810 (0.0008) +[2023-10-09 14:01:09,514][86122] Updated weights for policy 1, policy_version 45820 (0.0008) +[2023-10-09 14:01:10,467][86121] Updated weights for policy 0, policy_version 45610 (0.0010) +[2023-10-09 14:01:10,839][86121] Updated weights for policy 0, policy_version 45620 (0.0008) +[2023-10-09 14:01:11,203][86121] Updated weights for policy 0, policy_version 45630 (0.0007) +[2023-10-09 14:01:13,259][86122] Updated weights for policy 1, policy_version 45830 (0.0007) +[2023-10-09 14:01:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 93650944. Throughput: 0: 1826.0, 1: 1816.0. Samples: 23419258. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:01:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:13,627][86122] Updated weights for policy 1, policy_version 45840 (0.0008) +[2023-10-09 14:01:13,986][86122] Updated weights for policy 1, policy_version 45850 (0.0009) +[2023-10-09 14:01:14,697][86121] Updated weights for policy 0, policy_version 45640 (0.0008) +[2023-10-09 14:01:15,060][86121] Updated weights for policy 0, policy_version 45650 (0.0007) +[2023-10-09 14:01:15,433][86121] Updated weights for policy 0, policy_version 45660 (0.0009) +[2023-10-09 14:01:17,631][86122] Updated weights for policy 1, policy_version 45860 (0.0008) +[2023-10-09 14:01:18,024][86122] Updated weights for policy 1, policy_version 45870 (0.0010) +[2023-10-09 14:01:18,387][86122] Updated weights for policy 1, policy_version 45880 (0.0007) +[2023-10-09 14:01:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 93716480. Throughput: 0: 1831.8, 1: 1822.5. Samples: 23442234. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:01:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:19,103][86121] Updated weights for policy 0, policy_version 45670 (0.0009) +[2023-10-09 14:01:19,471][86121] Updated weights for policy 0, policy_version 45680 (0.0007) +[2023-10-09 14:01:19,839][86121] Updated weights for policy 0, policy_version 45690 (0.0007) +[2023-10-09 14:01:21,915][86122] Updated weights for policy 1, policy_version 45890 (0.0011) +[2023-10-09 14:01:22,278][86122] Updated weights for policy 1, policy_version 45900 (0.0008) +[2023-10-09 14:01:22,637][86122] Updated weights for policy 1, policy_version 45910 (0.0007) +[2023-10-09 14:01:23,000][86122] Updated weights for policy 1, policy_version 45920 (0.0008) +[2023-10-09 14:01:23,306][86121] Updated weights for policy 0, policy_version 45700 (0.0008) +[2023-10-09 14:01:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 93814784. Throughput: 0: 1833.5, 1: 1831.1. Samples: 23464116. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:01:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:23,405][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000045920_47022080.pth... +[2023-10-09 14:01:23,442][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000044192_45252608.pth +[2023-10-09 14:01:23,666][86121] Updated weights for policy 0, policy_version 45710 (0.0010) +[2023-10-09 14:01:24,035][86121] Updated weights for policy 0, policy_version 45720 (0.0010) +[2023-10-09 14:01:24,327][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000045728_46825472.pth... +[2023-10-09 14:01:24,365][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000044000_45056000.pth +[2023-10-09 14:01:26,719][86122] Updated weights for policy 1, policy_version 45930 (0.0010) +[2023-10-09 14:01:27,089][86122] Updated weights for policy 1, policy_version 45940 (0.0008) +[2023-10-09 14:01:27,453][86122] Updated weights for policy 1, policy_version 45950 (0.0009) +[2023-10-09 14:01:27,743][86121] Updated weights for policy 0, policy_version 45730 (0.0010) +[2023-10-09 14:01:28,106][86121] Updated weights for policy 0, policy_version 45740 (0.0007) +[2023-10-09 14:01:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 93880320. Throughput: 0: 1834.8, 1: 1841.5. Samples: 23475410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:01:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:28,470][86121] Updated weights for policy 0, policy_version 45750 (0.0008) +[2023-10-09 14:01:28,828][86121] Updated weights for policy 0, policy_version 45760 (0.0009) +[2023-10-09 14:01:31,134][86122] Updated weights for policy 1, policy_version 45960 (0.0008) +[2023-10-09 14:01:31,508][86122] Updated weights for policy 1, policy_version 45970 (0.0009) +[2023-10-09 14:01:31,869][86122] Updated weights for policy 1, policy_version 45980 (0.0008) +[2023-10-09 14:01:32,621][86121] Updated weights for policy 0, policy_version 45770 (0.0008) +[2023-10-09 14:01:32,978][86121] Updated weights for policy 0, policy_version 45780 (0.0007) +[2023-10-09 14:01:33,342][86121] Updated weights for policy 0, policy_version 45790 (0.0007) +[2023-10-09 14:01:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 93945856. Throughput: 0: 1829.3, 1: 1831.4. Samples: 23497038. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:01:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:35,289][86122] Updated weights for policy 1, policy_version 45990 (0.0008) +[2023-10-09 14:01:35,651][86122] Updated weights for policy 1, policy_version 46000 (0.0010) +[2023-10-09 14:01:36,010][86122] Updated weights for policy 1, policy_version 46010 (0.0007) +[2023-10-09 14:01:37,060][86121] Updated weights for policy 0, policy_version 45800 (0.0007) +[2023-10-09 14:01:37,426][86121] Updated weights for policy 0, policy_version 45810 (0.0008) +[2023-10-09 14:01:37,798][86121] Updated weights for policy 0, policy_version 45820 (0.0008) +[2023-10-09 14:01:38,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94044160. Throughput: 0: 1825.4, 1: 1850.4. Samples: 23518692. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 14:01:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:01:39,549][86122] Updated weights for policy 1, policy_version 46020 (0.0007) +[2023-10-09 14:01:39,902][86122] Updated weights for policy 1, policy_version 46030 (0.0008) +[2023-10-09 14:01:40,257][86122] Updated weights for policy 1, policy_version 46040 (0.0008) +[2023-10-09 14:01:41,595][86121] Updated weights for policy 0, policy_version 45830 (0.0009) +[2023-10-09 14:01:41,957][86121] Updated weights for policy 0, policy_version 45840 (0.0008) +[2023-10-09 14:01:42,325][86121] Updated weights for policy 0, policy_version 45850 (0.0008) +[2023-10-09 14:01:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94109696. Throughput: 0: 1839.2, 1: 1839.9. Samples: 23530314. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 14:01:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:01:43,960][86122] Updated weights for policy 1, policy_version 46050 (0.0011) +[2023-10-09 14:01:44,309][86122] Updated weights for policy 1, policy_version 46060 (0.0010) +[2023-10-09 14:01:44,666][86122] Updated weights for policy 1, policy_version 46070 (0.0007) +[2023-10-09 14:01:45,026][86122] Updated weights for policy 1, policy_version 46080 (0.0007) +[2023-10-09 14:01:45,967][86121] Updated weights for policy 0, policy_version 45860 (0.0009) +[2023-10-09 14:01:46,331][86121] Updated weights for policy 0, policy_version 45870 (0.0008) +[2023-10-09 14:01:46,706][86121] Updated weights for policy 0, policy_version 45880 (0.0009) +[2023-10-09 14:01:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94175232. Throughput: 0: 1830.9, 1: 1858.3. Samples: 23552118. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 14:01:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:01:48,674][86122] Updated weights for policy 1, policy_version 46090 (0.0008) +[2023-10-09 14:01:49,040][86122] Updated weights for policy 1, policy_version 46100 (0.0008) +[2023-10-09 14:01:49,409][86122] Updated weights for policy 1, policy_version 46110 (0.0009) +[2023-10-09 14:01:50,333][86121] Updated weights for policy 0, policy_version 45890 (0.0009) +[2023-10-09 14:01:50,695][86121] Updated weights for policy 0, policy_version 45900 (0.0008) +[2023-10-09 14:01:51,062][86121] Updated weights for policy 0, policy_version 45910 (0.0007) +[2023-10-09 14:01:51,421][86121] Updated weights for policy 0, policy_version 45920 (0.0009) +[2023-10-09 14:01:53,052][86122] Updated weights for policy 1, policy_version 46120 (0.0008) +[2023-10-09 14:01:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 94240768. Throughput: 0: 1849.0, 1: 1851.3. Samples: 23575172. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 14:01:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:01:53,424][86122] Updated weights for policy 1, policy_version 46130 (0.0007) +[2023-10-09 14:01:53,789][86122] Updated weights for policy 1, policy_version 46140 (0.0008) +[2023-10-09 14:01:55,000][86121] Updated weights for policy 0, policy_version 45930 (0.0010) +[2023-10-09 14:01:55,360][86121] Updated weights for policy 0, policy_version 45940 (0.0008) +[2023-10-09 14:01:55,735][86121] Updated weights for policy 0, policy_version 45950 (0.0008) +[2023-10-09 14:01:57,435][86122] Updated weights for policy 1, policy_version 46150 (0.0009) +[2023-10-09 14:01:57,802][86122] Updated weights for policy 1, policy_version 46160 (0.0009) +[2023-10-09 14:01:58,156][86122] Updated weights for policy 1, policy_version 46170 (0.0009) +[2023-10-09 14:01:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94339072. Throughput: 0: 1831.6, 1: 1857.6. Samples: 23585270. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 14:01:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:01:59,479][86121] Updated weights for policy 0, policy_version 45960 (0.0008) +[2023-10-09 14:01:59,841][86121] Updated weights for policy 0, policy_version 45970 (0.0007) +[2023-10-09 14:02:00,203][86121] Updated weights for policy 0, policy_version 45980 (0.0007) +[2023-10-09 14:02:01,911][86122] Updated weights for policy 1, policy_version 46180 (0.0009) +[2023-10-09 14:02:02,277][86122] Updated weights for policy 1, policy_version 46190 (0.0008) +[2023-10-09 14:02:02,646][86122] Updated weights for policy 1, policy_version 46200 (0.0009) +[2023-10-09 14:02:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94404608. Throughput: 0: 1832.5, 1: 1848.8. Samples: 23607892. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 14:02:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:02:03,960][86121] Updated weights for policy 0, policy_version 45990 (0.0008) +[2023-10-09 14:02:04,317][86121] Updated weights for policy 0, policy_version 46000 (0.0008) +[2023-10-09 14:02:04,686][86121] Updated weights for policy 0, policy_version 46010 (0.0007) +[2023-10-09 14:02:06,436][86122] Updated weights for policy 1, policy_version 46210 (0.0008) +[2023-10-09 14:02:06,843][86122] Updated weights for policy 1, policy_version 46220 (0.0009) +[2023-10-09 14:02:07,215][86122] Updated weights for policy 1, policy_version 46230 (0.0007) +[2023-10-09 14:02:07,580][86122] Updated weights for policy 1, policy_version 46240 (0.0008) +[2023-10-09 14:02:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94470144. Throughput: 0: 1829.9, 1: 1838.6. Samples: 23629200. Policy #0 lag: (min: 4.0, avg: 7.5, max: 36.0) +[2023-10-09 14:02:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:02:08,502][86121] Updated weights for policy 0, policy_version 46020 (0.0009) +[2023-10-09 14:02:08,864][86121] Updated weights for policy 0, policy_version 46030 (0.0009) +[2023-10-09 14:02:09,221][86121] Updated weights for policy 0, policy_version 46040 (0.0009) +[2023-10-09 14:02:11,162][86122] Updated weights for policy 1, policy_version 46250 (0.0008) +[2023-10-09 14:02:11,526][86122] Updated weights for policy 1, policy_version 46260 (0.0007) +[2023-10-09 14:02:11,893][86122] Updated weights for policy 1, policy_version 46270 (0.0009) +[2023-10-09 14:02:12,863][86121] Updated weights for policy 0, policy_version 46050 (0.0007) +[2023-10-09 14:02:13,234][86121] Updated weights for policy 0, policy_version 46060 (0.0007) +[2023-10-09 14:02:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94535680. Throughput: 0: 1828.4, 1: 1842.0. Samples: 23640580. Policy #0 lag: (min: 4.0, avg: 7.5, max: 36.0) +[2023-10-09 14:02:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:02:13,597][86121] Updated weights for policy 0, policy_version 46070 (0.0008) +[2023-10-09 14:02:13,960][86121] Updated weights for policy 0, policy_version 46080 (0.0008) +[2023-10-09 14:02:15,583][86122] Updated weights for policy 1, policy_version 46280 (0.0009) +[2023-10-09 14:02:15,933][86122] Updated weights for policy 1, policy_version 46290 (0.0009) +[2023-10-09 14:02:16,294][86122] Updated weights for policy 1, policy_version 46300 (0.0010) +[2023-10-09 14:02:17,657][86121] Updated weights for policy 0, policy_version 46090 (0.0008) +[2023-10-09 14:02:18,024][86121] Updated weights for policy 0, policy_version 46100 (0.0008) +[2023-10-09 14:02:18,395][86121] Updated weights for policy 0, policy_version 46110 (0.0009) +[2023-10-09 14:02:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 94601216. Throughput: 0: 1828.0, 1: 1839.1. Samples: 23662056. Policy #0 lag: (min: 4.0, avg: 7.5, max: 36.0) +[2023-10-09 14:02:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:02:20,008][86122] Updated weights for policy 1, policy_version 46310 (0.0010) +[2023-10-09 14:02:20,373][86122] Updated weights for policy 1, policy_version 46320 (0.0009) +[2023-10-09 14:02:20,735][86122] Updated weights for policy 1, policy_version 46330 (0.0007) +[2023-10-09 14:02:22,003][86121] Updated weights for policy 0, policy_version 46120 (0.0010) +[2023-10-09 14:02:22,364][86121] Updated weights for policy 0, policy_version 46130 (0.0008) +[2023-10-09 14:02:22,741][86121] Updated weights for policy 0, policy_version 46140 (0.0008) +[2023-10-09 14:02:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94699520. Throughput: 0: 1832.4, 1: 1834.5. Samples: 23683704. Policy #0 lag: (min: 4.0, avg: 7.5, max: 36.0) +[2023-10-09 14:02:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:02:24,304][86122] Updated weights for policy 1, policy_version 46340 (0.0008) +[2023-10-09 14:02:24,665][86122] Updated weights for policy 1, policy_version 46350 (0.0007) +[2023-10-09 14:02:25,031][86122] Updated weights for policy 1, policy_version 46360 (0.0007) +[2023-10-09 14:02:26,560][86121] Updated weights for policy 0, policy_version 46150 (0.0009) +[2023-10-09 14:02:26,946][86121] Updated weights for policy 0, policy_version 46160 (0.0008) +[2023-10-09 14:02:27,321][86121] Updated weights for policy 0, policy_version 46170 (0.0007) +[2023-10-09 14:02:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94765056. Throughput: 0: 1828.7, 1: 1829.6. Samples: 23694940. Policy #0 lag: (min: 4.0, avg: 7.5, max: 36.0) +[2023-10-09 14:02:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:02:28,758][86122] Updated weights for policy 1, policy_version 46370 (0.0008) +[2023-10-09 14:02:29,118][86122] Updated weights for policy 1, policy_version 46380 (0.0008) +[2023-10-09 14:02:29,476][86122] Updated weights for policy 1, policy_version 46390 (0.0007) +[2023-10-09 14:02:29,834][86122] Updated weights for policy 1, policy_version 46400 (0.0009) +[2023-10-09 14:02:30,862][86121] Updated weights for policy 0, policy_version 46180 (0.0008) +[2023-10-09 14:02:31,231][86121] Updated weights for policy 0, policy_version 46190 (0.0008) +[2023-10-09 14:02:31,598][86121] Updated weights for policy 0, policy_version 46200 (0.0009) +[2023-10-09 14:02:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94830592. Throughput: 0: 1820.4, 1: 1829.4. Samples: 23716360. Policy #0 lag: (min: 4.0, avg: 7.5, max: 36.0) +[2023-10-09 14:02:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:02:33,628][86122] Updated weights for policy 1, policy_version 46410 (0.0008) +[2023-10-09 14:02:33,993][86122] Updated weights for policy 1, policy_version 46420 (0.0008) +[2023-10-09 14:02:34,347][86122] Updated weights for policy 1, policy_version 46430 (0.0008) +[2023-10-09 14:02:35,204][86121] Updated weights for policy 0, policy_version 46210 (0.0008) +[2023-10-09 14:02:35,562][86121] Updated weights for policy 0, policy_version 46220 (0.0008) +[2023-10-09 14:02:35,921][86121] Updated weights for policy 0, policy_version 46230 (0.0009) +[2023-10-09 14:02:36,297][86121] Updated weights for policy 0, policy_version 46240 (0.0008) +[2023-10-09 14:02:37,955][86122] Updated weights for policy 1, policy_version 46440 (0.0007) +[2023-10-09 14:02:38,311][86122] Updated weights for policy 1, policy_version 46450 (0.0009) +[2023-10-09 14:02:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 94896128. Throughput: 0: 1816.4, 1: 1824.5. Samples: 23739016. Policy #0 lag: (min: 23.0, avg: 32.6, max: 55.0) +[2023-10-09 14:02:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:02:38,670][86122] Updated weights for policy 1, policy_version 46460 (0.0009) +[2023-10-09 14:02:40,017][86121] Updated weights for policy 0, policy_version 46250 (0.0011) +[2023-10-09 14:02:40,375][86121] Updated weights for policy 0, policy_version 46260 (0.0009) +[2023-10-09 14:02:40,741][86121] Updated weights for policy 0, policy_version 46270 (0.0008) +[2023-10-09 14:02:42,212][86122] Updated weights for policy 1, policy_version 46470 (0.0008) +[2023-10-09 14:02:42,565][86122] Updated weights for policy 1, policy_version 46480 (0.0007) +[2023-10-09 14:02:42,933][86122] Updated weights for policy 1, policy_version 46490 (0.0007) +[2023-10-09 14:02:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 94994432. Throughput: 0: 1812.8, 1: 1831.1. Samples: 23749248. Policy #0 lag: (min: 23.0, avg: 32.6, max: 55.0) +[2023-10-09 14:02:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:02:44,432][86121] Updated weights for policy 0, policy_version 46280 (0.0010) +[2023-10-09 14:02:44,785][86121] Updated weights for policy 0, policy_version 46290 (0.0007) +[2023-10-09 14:02:45,159][86121] Updated weights for policy 0, policy_version 46300 (0.0008) +[2023-10-09 14:02:46,527][86122] Updated weights for policy 1, policy_version 46500 (0.0007) +[2023-10-09 14:02:46,881][86122] Updated weights for policy 1, policy_version 46510 (0.0008) +[2023-10-09 14:02:47,244][86122] Updated weights for policy 1, policy_version 46520 (0.0008) +[2023-10-09 14:02:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95059968. Throughput: 0: 1816.0, 1: 1820.3. Samples: 23771524. Policy #0 lag: (min: 23.0, avg: 32.6, max: 55.0) +[2023-10-09 14:02:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:02:49,082][86121] Updated weights for policy 0, policy_version 46310 (0.0007) +[2023-10-09 14:02:49,447][86121] Updated weights for policy 0, policy_version 46320 (0.0008) +[2023-10-09 14:02:49,813][86121] Updated weights for policy 0, policy_version 46330 (0.0010) +[2023-10-09 14:02:51,007][86122] Updated weights for policy 1, policy_version 46530 (0.0010) +[2023-10-09 14:02:51,372][86122] Updated weights for policy 1, policy_version 46540 (0.0008) +[2023-10-09 14:02:51,745][86122] Updated weights for policy 1, policy_version 46550 (0.0008) +[2023-10-09 14:02:52,111][86122] Updated weights for policy 1, policy_version 46560 (0.0008) +[2023-10-09 14:02:53,394][86121] Updated weights for policy 0, policy_version 46340 (0.0011) +[2023-10-09 14:02:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95125504. Throughput: 0: 1821.2, 1: 1826.3. Samples: 23793338. Policy #0 lag: (min: 23.0, avg: 32.6, max: 55.0) +[2023-10-09 14:02:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:02:53,760][86121] Updated weights for policy 0, policy_version 46350 (0.0007) +[2023-10-09 14:02:54,131][86121] Updated weights for policy 0, policy_version 46360 (0.0007) +[2023-10-09 14:02:55,857][86122] Updated weights for policy 1, policy_version 46570 (0.0008) +[2023-10-09 14:02:56,220][86122] Updated weights for policy 1, policy_version 46580 (0.0009) +[2023-10-09 14:02:56,577][86122] Updated weights for policy 1, policy_version 46590 (0.0012) +[2023-10-09 14:02:57,829][86121] Updated weights for policy 0, policy_version 46370 (0.0009) +[2023-10-09 14:02:58,190][86121] Updated weights for policy 0, policy_version 46380 (0.0011) +[2023-10-09 14:02:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95191040. Throughput: 0: 1821.6, 1: 1818.1. Samples: 23804364. Policy #0 lag: (min: 23.0, avg: 32.6, max: 55.0) +[2023-10-09 14:02:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:02:58,571][86121] Updated weights for policy 0, policy_version 46390 (0.0011) +[2023-10-09 14:02:58,937][86121] Updated weights for policy 0, policy_version 46400 (0.0009) +[2023-10-09 14:03:00,208][86122] Updated weights for policy 1, policy_version 46600 (0.0009) +[2023-10-09 14:03:00,570][86122] Updated weights for policy 1, policy_version 46610 (0.0007) +[2023-10-09 14:03:00,934][86122] Updated weights for policy 1, policy_version 46620 (0.0008) +[2023-10-09 14:03:02,752][86121] Updated weights for policy 0, policy_version 46410 (0.0008) +[2023-10-09 14:03:03,120][86121] Updated weights for policy 0, policy_version 46420 (0.0010) +[2023-10-09 14:03:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95256576. Throughput: 0: 1820.4, 1: 1828.1. Samples: 23826234. Policy #0 lag: (min: 23.0, avg: 32.6, max: 55.0) +[2023-10-09 14:03:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:03:03,497][86121] Updated weights for policy 0, policy_version 46430 (0.0008) +[2023-10-09 14:03:04,583][86122] Updated weights for policy 1, policy_version 46630 (0.0010) +[2023-10-09 14:03:04,944][86122] Updated weights for policy 1, policy_version 46640 (0.0010) +[2023-10-09 14:03:05,310][86122] Updated weights for policy 1, policy_version 46650 (0.0008) +[2023-10-09 14:03:07,046][86121] Updated weights for policy 0, policy_version 46440 (0.0007) +[2023-10-09 14:03:07,407][86121] Updated weights for policy 0, policy_version 46450 (0.0007) +[2023-10-09 14:03:07,771][86121] Updated weights for policy 0, policy_version 46460 (0.0007) +[2023-10-09 14:03:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95354880. Throughput: 0: 1814.9, 1: 1830.2. Samples: 23847732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:03:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:03:09,071][86122] Updated weights for policy 1, policy_version 46660 (0.0010) +[2023-10-09 14:03:09,429][86122] Updated weights for policy 1, policy_version 46670 (0.0007) +[2023-10-09 14:03:09,781][86122] Updated weights for policy 1, policy_version 46680 (0.0009) +[2023-10-09 14:03:11,497][86121] Updated weights for policy 0, policy_version 46470 (0.0008) +[2023-10-09 14:03:11,864][86121] Updated weights for policy 0, policy_version 46480 (0.0007) +[2023-10-09 14:03:12,231][86121] Updated weights for policy 0, policy_version 46490 (0.0009) +[2023-10-09 14:03:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95420416. Throughput: 0: 1815.4, 1: 1833.2. Samples: 23859124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:03:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:03:13,488][86122] Updated weights for policy 1, policy_version 46690 (0.0008) +[2023-10-09 14:03:13,848][86122] Updated weights for policy 1, policy_version 46700 (0.0007) +[2023-10-09 14:03:14,208][86122] Updated weights for policy 1, policy_version 46710 (0.0008) +[2023-10-09 14:03:14,565][86122] Updated weights for policy 1, policy_version 46720 (0.0008) +[2023-10-09 14:03:15,770][86121] Updated weights for policy 0, policy_version 46500 (0.0009) +[2023-10-09 14:03:16,146][86121] Updated weights for policy 0, policy_version 46510 (0.0010) +[2023-10-09 14:03:16,511][86121] Updated weights for policy 0, policy_version 46520 (0.0009) +[2023-10-09 14:03:18,297][86122] Updated weights for policy 1, policy_version 46730 (0.0007) +[2023-10-09 14:03:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95485952. Throughput: 0: 1816.7, 1: 1830.9. Samples: 23880504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:03:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:03:18,665][86122] Updated weights for policy 1, policy_version 46740 (0.0008) +[2023-10-09 14:03:19,021][86122] Updated weights for policy 1, policy_version 46750 (0.0009) +[2023-10-09 14:03:20,086][86121] Updated weights for policy 0, policy_version 46530 (0.0007) +[2023-10-09 14:03:20,445][86121] Updated weights for policy 0, policy_version 46540 (0.0009) +[2023-10-09 14:03:20,821][86121] Updated weights for policy 0, policy_version 46550 (0.0007) +[2023-10-09 14:03:21,181][86121] Updated weights for policy 0, policy_version 46560 (0.0007) +[2023-10-09 14:03:22,787][86122] Updated weights for policy 1, policy_version 46760 (0.0008) +[2023-10-09 14:03:23,157][86122] Updated weights for policy 1, policy_version 46770 (0.0007) +[2023-10-09 14:03:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95551488. Throughput: 0: 1823.6, 1: 1825.0. Samples: 23903204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:03:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:03:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000046560_47677440.pth... +[2023-10-09 14:03:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000044864_45940736.pth +[2023-10-09 14:03:23,521][86122] Updated weights for policy 1, policy_version 46780 (0.0008) +[2023-10-09 14:03:23,657][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000046784_47906816.pth... +[2023-10-09 14:03:23,696][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000045056_46137344.pth +[2023-10-09 14:03:24,879][86121] Updated weights for policy 0, policy_version 46570 (0.0007) +[2023-10-09 14:03:25,251][86121] Updated weights for policy 0, policy_version 46580 (0.0008) +[2023-10-09 14:03:25,605][86121] Updated weights for policy 0, policy_version 46590 (0.0008) +[2023-10-09 14:03:27,122][86122] Updated weights for policy 1, policy_version 46790 (0.0007) +[2023-10-09 14:03:27,477][86122] Updated weights for policy 1, policy_version 46800 (0.0009) +[2023-10-09 14:03:27,842][86122] Updated weights for policy 1, policy_version 46810 (0.0008) +[2023-10-09 14:03:28,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95649792. Throughput: 0: 1825.6, 1: 1824.3. Samples: 23913492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:03:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:03:29,325][86121] Updated weights for policy 0, policy_version 46600 (0.0008) +[2023-10-09 14:03:29,694][86121] Updated weights for policy 0, policy_version 46610 (0.0009) +[2023-10-09 14:03:30,064][86121] Updated weights for policy 0, policy_version 46620 (0.0008) +[2023-10-09 14:03:31,601][86122] Updated weights for policy 1, policy_version 46820 (0.0008) +[2023-10-09 14:03:31,959][86122] Updated weights for policy 1, policy_version 46830 (0.0009) +[2023-10-09 14:03:32,311][86122] Updated weights for policy 1, policy_version 46840 (0.0010) +[2023-10-09 14:03:33,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95715328. Throughput: 0: 1825.3, 1: 1822.0. Samples: 23935652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:03:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:03:33,752][86121] Updated weights for policy 0, policy_version 46630 (0.0008) +[2023-10-09 14:03:34,116][86121] Updated weights for policy 0, policy_version 46640 (0.0009) +[2023-10-09 14:03:34,487][86121] Updated weights for policy 0, policy_version 46650 (0.0007) +[2023-10-09 14:03:36,189][86122] Updated weights for policy 1, policy_version 46850 (0.0008) +[2023-10-09 14:03:36,548][86122] Updated weights for policy 1, policy_version 46860 (0.0010) +[2023-10-09 14:03:36,906][86122] Updated weights for policy 1, policy_version 46870 (0.0010) +[2023-10-09 14:03:37,265][86122] Updated weights for policy 1, policy_version 46880 (0.0010) +[2023-10-09 14:03:38,135][86121] Updated weights for policy 0, policy_version 46660 (0.0008) +[2023-10-09 14:03:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95780864. Throughput: 0: 1826.0, 1: 1817.3. Samples: 23957288. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:03:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:03:38,506][86121] Updated weights for policy 0, policy_version 46670 (0.0008) +[2023-10-09 14:03:38,869][86121] Updated weights for policy 0, policy_version 46680 (0.0008) +[2023-10-09 14:03:41,084][86122] Updated weights for policy 1, policy_version 46890 (0.0007) +[2023-10-09 14:03:41,451][86122] Updated weights for policy 1, policy_version 46900 (0.0007) +[2023-10-09 14:03:41,821][86122] Updated weights for policy 1, policy_version 46910 (0.0010) +[2023-10-09 14:03:42,534][86121] Updated weights for policy 0, policy_version 46690 (0.0010) +[2023-10-09 14:03:42,911][86121] Updated weights for policy 0, policy_version 46700 (0.0009) +[2023-10-09 14:03:43,280][86121] Updated weights for policy 0, policy_version 46710 (0.0007) +[2023-10-09 14:03:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 95846400. Throughput: 0: 1829.1, 1: 1819.9. Samples: 23968570. Policy #0 lag: (min: 18.0, avg: 31.3, max: 50.0) +[2023-10-09 14:03:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:03:43,658][86121] Updated weights for policy 0, policy_version 46720 (0.0007) +[2023-10-09 14:03:45,505][86122] Updated weights for policy 1, policy_version 46920 (0.0008) +[2023-10-09 14:03:45,870][86122] Updated weights for policy 1, policy_version 46930 (0.0007) +[2023-10-09 14:03:46,228][86122] Updated weights for policy 1, policy_version 46940 (0.0008) +[2023-10-09 14:03:47,326][86121] Updated weights for policy 0, policy_version 46730 (0.0007) +[2023-10-09 14:03:47,693][86121] Updated weights for policy 0, policy_version 46740 (0.0008) +[2023-10-09 14:03:48,056][86121] Updated weights for policy 0, policy_version 46750 (0.0009) +[2023-10-09 14:03:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 95944704. Throughput: 0: 1828.2, 1: 1812.2. Samples: 23990054. Policy #0 lag: (min: 18.0, avg: 31.3, max: 50.0) +[2023-10-09 14:03:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:03:50,011][86122] Updated weights for policy 1, policy_version 46950 (0.0008) +[2023-10-09 14:03:50,366][86122] Updated weights for policy 1, policy_version 46960 (0.0009) +[2023-10-09 14:03:50,725][86122] Updated weights for policy 1, policy_version 46970 (0.0010) +[2023-10-09 14:03:51,614][86121] Updated weights for policy 0, policy_version 46760 (0.0008) +[2023-10-09 14:03:51,977][86121] Updated weights for policy 0, policy_version 46770 (0.0007) +[2023-10-09 14:03:52,344][86121] Updated weights for policy 0, policy_version 46780 (0.0007) +[2023-10-09 14:03:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96010240. Throughput: 0: 1830.5, 1: 1809.8. Samples: 24011546. Policy #0 lag: (min: 18.0, avg: 31.3, max: 50.0) +[2023-10-09 14:03:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:03:54,502][86122] Updated weights for policy 1, policy_version 46980 (0.0010) +[2023-10-09 14:03:54,877][86122] Updated weights for policy 1, policy_version 46990 (0.0008) +[2023-10-09 14:03:55,231][86122] Updated weights for policy 1, policy_version 47000 (0.0007) +[2023-10-09 14:03:55,962][86121] Updated weights for policy 0, policy_version 46790 (0.0008) +[2023-10-09 14:03:56,332][86121] Updated weights for policy 0, policy_version 46800 (0.0009) +[2023-10-09 14:03:56,698][86121] Updated weights for policy 0, policy_version 46810 (0.0007) +[2023-10-09 14:03:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96075776. Throughput: 0: 1831.0, 1: 1805.1. Samples: 24022748. Policy #0 lag: (min: 18.0, avg: 31.3, max: 50.0) +[2023-10-09 14:03:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:03:58,938][86122] Updated weights for policy 1, policy_version 47010 (0.0010) +[2023-10-09 14:03:59,308][86122] Updated weights for policy 1, policy_version 47020 (0.0011) +[2023-10-09 14:03:59,663][86122] Updated weights for policy 1, policy_version 47030 (0.0010) +[2023-10-09 14:04:00,020][86122] Updated weights for policy 1, policy_version 47040 (0.0010) +[2023-10-09 14:04:00,410][86121] Updated weights for policy 0, policy_version 46820 (0.0008) +[2023-10-09 14:04:00,775][86121] Updated weights for policy 0, policy_version 46830 (0.0007) +[2023-10-09 14:04:01,147][86121] Updated weights for policy 0, policy_version 46840 (0.0008) +[2023-10-09 14:04:03,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 96141312. Throughput: 0: 1836.0, 1: 1805.1. Samples: 24044350. Policy #0 lag: (min: 18.0, avg: 31.3, max: 50.0) +[2023-10-09 14:04:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:04:03,788][86122] Updated weights for policy 1, policy_version 47050 (0.0009) +[2023-10-09 14:04:04,151][86122] Updated weights for policy 1, policy_version 47060 (0.0008) +[2023-10-09 14:04:04,512][86122] Updated weights for policy 1, policy_version 47070 (0.0007) +[2023-10-09 14:04:04,859][86121] Updated weights for policy 0, policy_version 46850 (0.0009) +[2023-10-09 14:04:05,226][86121] Updated weights for policy 0, policy_version 46860 (0.0009) +[2023-10-09 14:04:05,596][86121] Updated weights for policy 0, policy_version 46870 (0.0008) +[2023-10-09 14:04:05,969][86121] Updated weights for policy 0, policy_version 46880 (0.0010) +[2023-10-09 14:04:08,119][86122] Updated weights for policy 1, policy_version 47080 (0.0009) +[2023-10-09 14:04:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96206848. Throughput: 0: 1833.7, 1: 1813.0. Samples: 24067308. Policy #0 lag: (min: 18.0, avg: 31.3, max: 50.0) +[2023-10-09 14:04:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:04:08,485][86122] Updated weights for policy 1, policy_version 47090 (0.0007) +[2023-10-09 14:04:08,839][86122] Updated weights for policy 1, policy_version 47100 (0.0007) +[2023-10-09 14:04:09,751][86121] Updated weights for policy 0, policy_version 46890 (0.0011) +[2023-10-09 14:04:10,121][86121] Updated weights for policy 0, policy_version 46900 (0.0007) +[2023-10-09 14:04:10,488][86121] Updated weights for policy 0, policy_version 46910 (0.0010) +[2023-10-09 14:04:12,516][86122] Updated weights for policy 1, policy_version 47110 (0.0008) +[2023-10-09 14:04:12,885][86122] Updated weights for policy 1, policy_version 47120 (0.0008) +[2023-10-09 14:04:13,242][86122] Updated weights for policy 1, policy_version 47130 (0.0007) +[2023-10-09 14:04:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 96272384. Throughput: 0: 1830.7, 1: 1808.2. Samples: 24077242. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:04:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:04:14,043][86121] Updated weights for policy 0, policy_version 46920 (0.0007) +[2023-10-09 14:04:14,419][86121] Updated weights for policy 0, policy_version 46930 (0.0010) +[2023-10-09 14:04:14,787][86121] Updated weights for policy 0, policy_version 46940 (0.0008) +[2023-10-09 14:04:16,915][86122] Updated weights for policy 1, policy_version 47140 (0.0008) +[2023-10-09 14:04:17,276][86122] Updated weights for policy 1, policy_version 47150 (0.0008) +[2023-10-09 14:04:17,640][86122] Updated weights for policy 1, policy_version 47160 (0.0010) +[2023-10-09 14:04:18,307][86121] Updated weights for policy 0, policy_version 46950 (0.0009) +[2023-10-09 14:04:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 96370688. Throughput: 0: 1840.3, 1: 1823.7. Samples: 24100530. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:04:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:04:18,687][86121] Updated weights for policy 0, policy_version 46960 (0.0010) +[2023-10-09 14:04:19,045][86121] Updated weights for policy 0, policy_version 46970 (0.0007) +[2023-10-09 14:04:21,245][86122] Updated weights for policy 1, policy_version 47170 (0.0008) +[2023-10-09 14:04:21,601][86122] Updated weights for policy 1, policy_version 47180 (0.0010) +[2023-10-09 14:04:21,959][86122] Updated weights for policy 1, policy_version 47190 (0.0010) +[2023-10-09 14:04:22,318][86122] Updated weights for policy 1, policy_version 47200 (0.0011) +[2023-10-09 14:04:22,806][86121] Updated weights for policy 0, policy_version 46980 (0.0009) +[2023-10-09 14:04:23,170][86121] Updated weights for policy 0, policy_version 46990 (0.0008) +[2023-10-09 14:04:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 96436224. Throughput: 0: 1822.4, 1: 1823.7. Samples: 24121366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:04:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:04:23,537][86121] Updated weights for policy 0, policy_version 47000 (0.0008) +[2023-10-09 14:04:26,035][86122] Updated weights for policy 1, policy_version 47210 (0.0007) +[2023-10-09 14:04:26,393][86122] Updated weights for policy 1, policy_version 47220 (0.0008) +[2023-10-09 14:04:26,768][86122] Updated weights for policy 1, policy_version 47230 (0.0007) +[2023-10-09 14:04:27,390][86121] Updated weights for policy 0, policy_version 47010 (0.0007) +[2023-10-09 14:04:27,762][86121] Updated weights for policy 0, policy_version 47020 (0.0011) +[2023-10-09 14:04:28,122][86121] Updated weights for policy 0, policy_version 47030 (0.0010) +[2023-10-09 14:04:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96501760. Throughput: 0: 1823.7, 1: 1826.9. Samples: 24132848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:04:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:04:28,486][86121] Updated weights for policy 0, policy_version 47040 (0.0008) +[2023-10-09 14:04:30,417][86122] Updated weights for policy 1, policy_version 47240 (0.0008) +[2023-10-09 14:04:30,778][86122] Updated weights for policy 1, policy_version 47250 (0.0008) +[2023-10-09 14:04:31,135][86122] Updated weights for policy 1, policy_version 47260 (0.0009) +[2023-10-09 14:04:32,119][86121] Updated weights for policy 0, policy_version 47050 (0.0010) +[2023-10-09 14:04:32,488][86121] Updated weights for policy 0, policy_version 47060 (0.0010) +[2023-10-09 14:04:32,853][86121] Updated weights for policy 0, policy_version 47070 (0.0008) +[2023-10-09 14:04:33,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 96600064. Throughput: 0: 1822.4, 1: 1826.6. Samples: 24154256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:04:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:04:34,634][86122] Updated weights for policy 1, policy_version 47270 (0.0010) +[2023-10-09 14:04:34,995][86122] Updated weights for policy 1, policy_version 47280 (0.0011) +[2023-10-09 14:04:35,361][86122] Updated weights for policy 1, policy_version 47290 (0.0009) +[2023-10-09 14:04:36,557][86121] Updated weights for policy 0, policy_version 47080 (0.0009) +[2023-10-09 14:04:36,930][86121] Updated weights for policy 0, policy_version 47090 (0.0009) +[2023-10-09 14:04:37,285][86121] Updated weights for policy 0, policy_version 47100 (0.0007) +[2023-10-09 14:04:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96665600. Throughput: 0: 1825.2, 1: 1834.1. Samples: 24176214. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:04:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:04:38,967][86122] Updated weights for policy 1, policy_version 47300 (0.0007) +[2023-10-09 14:04:39,329][86122] Updated weights for policy 1, policy_version 47310 (0.0007) +[2023-10-09 14:04:39,694][86122] Updated weights for policy 1, policy_version 47320 (0.0007) +[2023-10-09 14:04:41,059][86121] Updated weights for policy 0, policy_version 47110 (0.0009) +[2023-10-09 14:04:41,435][86121] Updated weights for policy 0, policy_version 47120 (0.0008) +[2023-10-09 14:04:41,795][86121] Updated weights for policy 0, policy_version 47130 (0.0007) +[2023-10-09 14:04:43,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96731136. Throughput: 0: 1824.2, 1: 1834.4. Samples: 24187388. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) +[2023-10-09 14:04:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:04:43,427][86122] Updated weights for policy 1, policy_version 47330 (0.0007) +[2023-10-09 14:04:43,788][86122] Updated weights for policy 1, policy_version 47340 (0.0009) +[2023-10-09 14:04:44,144][86122] Updated weights for policy 1, policy_version 47350 (0.0011) +[2023-10-09 14:04:44,512][86122] Updated weights for policy 1, policy_version 47360 (0.0009) +[2023-10-09 14:04:45,476][86121] Updated weights for policy 0, policy_version 47140 (0.0009) +[2023-10-09 14:04:45,861][86121] Updated weights for policy 0, policy_version 47150 (0.0007) +[2023-10-09 14:04:46,224][86121] Updated weights for policy 0, policy_version 47160 (0.0008) +[2023-10-09 14:04:48,324][86122] Updated weights for policy 1, policy_version 47370 (0.0009) +[2023-10-09 14:04:48,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 96796672. Throughput: 0: 1824.4, 1: 1833.3. Samples: 24208948. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) +[2023-10-09 14:04:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:04:48,674][86122] Updated weights for policy 1, policy_version 47380 (0.0008) +[2023-10-09 14:04:49,037][86122] Updated weights for policy 1, policy_version 47390 (0.0010) +[2023-10-09 14:04:49,895][86121] Updated weights for policy 0, policy_version 47170 (0.0007) +[2023-10-09 14:04:50,265][86121] Updated weights for policy 0, policy_version 47180 (0.0007) +[2023-10-09 14:04:50,626][86121] Updated weights for policy 0, policy_version 47190 (0.0008) +[2023-10-09 14:04:50,986][86121] Updated weights for policy 0, policy_version 47200 (0.0010) +[2023-10-09 14:04:52,822][86122] Updated weights for policy 1, policy_version 47400 (0.0010) +[2023-10-09 14:04:53,184][86122] Updated weights for policy 1, policy_version 47410 (0.0010) +[2023-10-09 14:04:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 96862208. Throughput: 0: 1822.8, 1: 1826.9. Samples: 24231548. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) +[2023-10-09 14:04:53,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:04:53,554][86122] Updated weights for policy 1, policy_version 47420 (0.0010) +[2023-10-09 14:04:54,533][86121] Updated weights for policy 0, policy_version 47210 (0.0010) +[2023-10-09 14:04:54,896][86121] Updated weights for policy 0, policy_version 47220 (0.0010) +[2023-10-09 14:04:55,262][86121] Updated weights for policy 0, policy_version 47230 (0.0008) +[2023-10-09 14:04:57,206][86122] Updated weights for policy 1, policy_version 47430 (0.0010) +[2023-10-09 14:04:57,569][86122] Updated weights for policy 1, policy_version 47440 (0.0008) +[2023-10-09 14:04:57,929][86122] Updated weights for policy 1, policy_version 47450 (0.0008) +[2023-10-09 14:04:58,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 96960512. Throughput: 0: 1828.1, 1: 1831.3. Samples: 24241914. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) +[2023-10-09 14:04:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:04:59,004][86121] Updated weights for policy 0, policy_version 47240 (0.0010) +[2023-10-09 14:04:59,369][86121] Updated weights for policy 0, policy_version 47250 (0.0008) +[2023-10-09 14:04:59,727][86121] Updated weights for policy 0, policy_version 47260 (0.0009) +[2023-10-09 14:05:01,574][86122] Updated weights for policy 1, policy_version 47460 (0.0007) +[2023-10-09 14:05:01,938][86122] Updated weights for policy 1, policy_version 47470 (0.0007) +[2023-10-09 14:05:02,299][86122] Updated weights for policy 1, policy_version 47480 (0.0007) +[2023-10-09 14:05:03,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 97026048. Throughput: 0: 1811.5, 1: 1820.6. Samples: 24263974. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) +[2023-10-09 14:05:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:05:03,578][86121] Updated weights for policy 0, policy_version 47270 (0.0008) +[2023-10-09 14:05:03,945][86121] Updated weights for policy 0, policy_version 47280 (0.0009) +[2023-10-09 14:05:04,310][86121] Updated weights for policy 0, policy_version 47290 (0.0007) +[2023-10-09 14:05:06,099][86122] Updated weights for policy 1, policy_version 47490 (0.0007) +[2023-10-09 14:05:06,459][86122] Updated weights for policy 1, policy_version 47500 (0.0007) +[2023-10-09 14:05:06,820][86122] Updated weights for policy 1, policy_version 47510 (0.0007) +[2023-10-09 14:05:07,183][86122] Updated weights for policy 1, policy_version 47520 (0.0009) +[2023-10-09 14:05:08,129][86121] Updated weights for policy 0, policy_version 47300 (0.0009) +[2023-10-09 14:05:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 97091584. Throughput: 0: 1823.2, 1: 1828.9. Samples: 24285712. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) +[2023-10-09 14:05:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:05:08,497][86121] Updated weights for policy 0, policy_version 47310 (0.0008) +[2023-10-09 14:05:08,857][86121] Updated weights for policy 0, policy_version 47320 (0.0008) +[2023-10-09 14:05:10,988][86122] Updated weights for policy 1, policy_version 47530 (0.0009) +[2023-10-09 14:05:11,355][86122] Updated weights for policy 1, policy_version 47540 (0.0011) +[2023-10-09 14:05:11,716][86122] Updated weights for policy 1, policy_version 47550 (0.0009) +[2023-10-09 14:05:12,414][86121] Updated weights for policy 0, policy_version 47330 (0.0009) +[2023-10-09 14:05:12,780][86121] Updated weights for policy 0, policy_version 47340 (0.0008) +[2023-10-09 14:05:13,147][86121] Updated weights for policy 0, policy_version 47350 (0.0007) +[2023-10-09 14:05:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 97157120. Throughput: 0: 1819.4, 1: 1821.9. Samples: 24296708. Policy #0 lag: (min: 31.0, avg: 35.7, max: 63.0) +[2023-10-09 14:05:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:05:13,511][86121] Updated weights for policy 0, policy_version 47360 (0.0009) +[2023-10-09 14:05:15,214][86122] Updated weights for policy 1, policy_version 47560 (0.0008) +[2023-10-09 14:05:15,576][86122] Updated weights for policy 1, policy_version 47570 (0.0010) +[2023-10-09 14:05:15,936][86122] Updated weights for policy 1, policy_version 47580 (0.0008) +[2023-10-09 14:05:17,238][86121] Updated weights for policy 0, policy_version 47370 (0.0009) +[2023-10-09 14:05:17,596][86121] Updated weights for policy 0, policy_version 47380 (0.0009) +[2023-10-09 14:05:17,954][86121] Updated weights for policy 0, policy_version 47390 (0.0008) +[2023-10-09 14:05:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97255424. Throughput: 0: 1820.8, 1: 1827.5. Samples: 24318426. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:05:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:05:19,438][86122] Updated weights for policy 1, policy_version 47590 (0.0008) +[2023-10-09 14:05:19,795][86122] Updated weights for policy 1, policy_version 47600 (0.0007) +[2023-10-09 14:05:20,162][86122] Updated weights for policy 1, policy_version 47610 (0.0007) +[2023-10-09 14:05:21,800][86121] Updated weights for policy 0, policy_version 47400 (0.0008) +[2023-10-09 14:05:22,167][86121] Updated weights for policy 0, policy_version 47410 (0.0008) +[2023-10-09 14:05:22,540][86121] Updated weights for policy 0, policy_version 47420 (0.0011) +[2023-10-09 14:05:23,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97320960. Throughput: 0: 1812.1, 1: 1829.6. Samples: 24340092. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:05:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:05:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000047616_48758784.pth... +[2023-10-09 14:05:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000047424_48562176.pth... +[2023-10-09 14:05:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000045728_46825472.pth +[2023-10-09 14:05:23,452][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000045920_47022080.pth +[2023-10-09 14:05:23,726][86122] Updated weights for policy 1, policy_version 47620 (0.0008) +[2023-10-09 14:05:24,087][86122] Updated weights for policy 1, policy_version 47630 (0.0008) +[2023-10-09 14:05:24,449][86122] Updated weights for policy 1, policy_version 47640 (0.0011) +[2023-10-09 14:05:26,177][86121] Updated weights for policy 0, policy_version 47430 (0.0008) +[2023-10-09 14:05:26,541][86121] Updated weights for policy 0, policy_version 47440 (0.0009) +[2023-10-09 14:05:26,905][86121] Updated weights for policy 0, policy_version 47450 (0.0007) +[2023-10-09 14:05:28,126][86122] Updated weights for policy 1, policy_version 47650 (0.0009) +[2023-10-09 14:05:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97386496. Throughput: 0: 1815.0, 1: 1831.3. Samples: 24351474. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:05:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:05:28,485][86122] Updated weights for policy 1, policy_version 47660 (0.0008) +[2023-10-09 14:05:28,858][86122] Updated weights for policy 1, policy_version 47670 (0.0010) +[2023-10-09 14:05:29,210][86122] Updated weights for policy 1, policy_version 47680 (0.0010) +[2023-10-09 14:05:30,765][86121] Updated weights for policy 0, policy_version 47460 (0.0008) +[2023-10-09 14:05:31,127][86121] Updated weights for policy 0, policy_version 47470 (0.0010) +[2023-10-09 14:05:31,497][86121] Updated weights for policy 0, policy_version 47480 (0.0008) +[2023-10-09 14:05:32,941][86122] Updated weights for policy 1, policy_version 47690 (0.0007) +[2023-10-09 14:05:33,303][86122] Updated weights for policy 1, policy_version 47700 (0.0007) +[2023-10-09 14:05:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97452032. Throughput: 0: 1801.1, 1: 1837.2. Samples: 24372670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:05:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:05:33,663][86122] Updated weights for policy 1, policy_version 47710 (0.0009) +[2023-10-09 14:05:35,329][86121] Updated weights for policy 0, policy_version 47490 (0.0011) +[2023-10-09 14:05:35,727][86121] Updated weights for policy 0, policy_version 47500 (0.0008) +[2023-10-09 14:05:36,090][86121] Updated weights for policy 0, policy_version 47510 (0.0010) +[2023-10-09 14:05:36,454][86121] Updated weights for policy 0, policy_version 47520 (0.0008) +[2023-10-09 14:05:37,403][86122] Updated weights for policy 1, policy_version 47720 (0.0008) +[2023-10-09 14:05:37,768][86122] Updated weights for policy 1, policy_version 47730 (0.0010) +[2023-10-09 14:05:38,136][86122] Updated weights for policy 1, policy_version 47740 (0.0008) +[2023-10-09 14:05:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97550336. Throughput: 0: 1794.4, 1: 1827.7. Samples: 24394544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:05:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:05:40,102][86121] Updated weights for policy 0, policy_version 47530 (0.0008) +[2023-10-09 14:05:40,468][86121] Updated weights for policy 0, policy_version 47540 (0.0010) +[2023-10-09 14:05:40,831][86121] Updated weights for policy 0, policy_version 47550 (0.0007) +[2023-10-09 14:05:41,779][86122] Updated weights for policy 1, policy_version 47750 (0.0010) +[2023-10-09 14:05:42,147][86122] Updated weights for policy 1, policy_version 47760 (0.0008) +[2023-10-09 14:05:42,517][86122] Updated weights for policy 1, policy_version 47770 (0.0008) +[2023-10-09 14:05:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 97615872. Throughput: 0: 1795.9, 1: 1840.4. Samples: 24405552. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:05:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:05:44,589][86121] Updated weights for policy 0, policy_version 47560 (0.0009) +[2023-10-09 14:05:44,957][86121] Updated weights for policy 0, policy_version 47570 (0.0011) +[2023-10-09 14:05:45,329][86121] Updated weights for policy 0, policy_version 47580 (0.0009) +[2023-10-09 14:05:46,158][86122] Updated weights for policy 1, policy_version 47780 (0.0008) +[2023-10-09 14:05:46,522][86122] Updated weights for policy 1, policy_version 47790 (0.0007) +[2023-10-09 14:05:46,890][86122] Updated weights for policy 1, policy_version 47800 (0.0008) +[2023-10-09 14:05:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 97681408. Throughput: 0: 1799.7, 1: 1828.3. Samples: 24427234. Policy #0 lag: (min: 25.0, avg: 35.6, max: 57.0) +[2023-10-09 14:05:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:05:49,028][86121] Updated weights for policy 0, policy_version 47590 (0.0009) +[2023-10-09 14:05:49,390][86121] Updated weights for policy 0, policy_version 47600 (0.0007) +[2023-10-09 14:05:49,754][86121] Updated weights for policy 0, policy_version 47610 (0.0008) +[2023-10-09 14:05:50,540][86122] Updated weights for policy 1, policy_version 47810 (0.0008) +[2023-10-09 14:05:50,900][86122] Updated weights for policy 1, policy_version 47820 (0.0008) +[2023-10-09 14:05:51,267][86122] Updated weights for policy 1, policy_version 47830 (0.0010) +[2023-10-09 14:05:51,633][86122] Updated weights for policy 1, policy_version 47840 (0.0008) +[2023-10-09 14:05:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 97746944. Throughput: 0: 1802.8, 1: 1842.3. Samples: 24449744. Policy #0 lag: (min: 25.0, avg: 35.6, max: 57.0) +[2023-10-09 14:05:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:05:53,411][86121] Updated weights for policy 0, policy_version 47620 (0.0009) +[2023-10-09 14:05:53,782][86121] Updated weights for policy 0, policy_version 47630 (0.0009) +[2023-10-09 14:05:54,139][86121] Updated weights for policy 0, policy_version 47640 (0.0007) +[2023-10-09 14:05:55,348][86122] Updated weights for policy 1, policy_version 47850 (0.0010) +[2023-10-09 14:05:55,712][86122] Updated weights for policy 1, policy_version 47860 (0.0011) +[2023-10-09 14:05:56,068][86122] Updated weights for policy 1, policy_version 47870 (0.0010) +[2023-10-09 14:05:57,965][86121] Updated weights for policy 0, policy_version 47650 (0.0008) +[2023-10-09 14:05:58,333][86121] Updated weights for policy 0, policy_version 47660 (0.0009) +[2023-10-09 14:05:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97812480. Throughput: 0: 1800.7, 1: 1827.1. Samples: 24459958. Policy #0 lag: (min: 25.0, avg: 35.6, max: 57.0) +[2023-10-09 14:05:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:05:58,697][86121] Updated weights for policy 0, policy_version 47670 (0.0008) +[2023-10-09 14:05:59,064][86121] Updated weights for policy 0, policy_version 47680 (0.0008) +[2023-10-09 14:05:59,691][86122] Updated weights for policy 1, policy_version 47880 (0.0009) +[2023-10-09 14:06:00,051][86122] Updated weights for policy 1, policy_version 47890 (0.0009) +[2023-10-09 14:06:00,409][86122] Updated weights for policy 1, policy_version 47900 (0.0008) +[2023-10-09 14:06:02,740][86121] Updated weights for policy 0, policy_version 47690 (0.0009) +[2023-10-09 14:06:03,115][86121] Updated weights for policy 0, policy_version 47700 (0.0008) +[2023-10-09 14:06:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 97878016. Throughput: 0: 1802.0, 1: 1840.6. Samples: 24482340. Policy #0 lag: (min: 25.0, avg: 35.6, max: 57.0) +[2023-10-09 14:06:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:06:03,480][86121] Updated weights for policy 0, policy_version 47710 (0.0008) +[2023-10-09 14:06:04,157][86122] Updated weights for policy 1, policy_version 47910 (0.0008) +[2023-10-09 14:06:04,513][86122] Updated weights for policy 1, policy_version 47920 (0.0007) +[2023-10-09 14:06:04,883][86122] Updated weights for policy 1, policy_version 47930 (0.0008) +[2023-10-09 14:06:07,144][86121] Updated weights for policy 0, policy_version 47720 (0.0009) +[2023-10-09 14:06:07,503][86121] Updated weights for policy 0, policy_version 47730 (0.0008) +[2023-10-09 14:06:07,872][86121] Updated weights for policy 0, policy_version 47740 (0.0008) +[2023-10-09 14:06:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 97976320. Throughput: 0: 1813.3, 1: 1839.3. Samples: 24504460. Policy #0 lag: (min: 25.0, avg: 35.6, max: 57.0) +[2023-10-09 14:06:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:06:08,579][86122] Updated weights for policy 1, policy_version 47940 (0.0008) +[2023-10-09 14:06:08,945][86122] Updated weights for policy 1, policy_version 47950 (0.0008) +[2023-10-09 14:06:09,318][86122] Updated weights for policy 1, policy_version 47960 (0.0008) +[2023-10-09 14:06:11,642][86121] Updated weights for policy 0, policy_version 47750 (0.0010) +[2023-10-09 14:06:12,011][86121] Updated weights for policy 0, policy_version 47760 (0.0008) +[2023-10-09 14:06:12,383][86121] Updated weights for policy 0, policy_version 47770 (0.0009) +[2023-10-09 14:06:12,958][86122] Updated weights for policy 1, policy_version 47970 (0.0010) +[2023-10-09 14:06:13,329][86122] Updated weights for policy 1, policy_version 47980 (0.0010) +[2023-10-09 14:06:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98041856. Throughput: 0: 1811.2, 1: 1835.4. Samples: 24515572. Policy #0 lag: (min: 25.0, avg: 35.6, max: 57.0) +[2023-10-09 14:06:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:06:13,699][86122] Updated weights for policy 1, policy_version 47990 (0.0007) +[2023-10-09 14:06:14,057][86122] Updated weights for policy 1, policy_version 48000 (0.0007) +[2023-10-09 14:06:15,915][86121] Updated weights for policy 0, policy_version 47780 (0.0007) +[2023-10-09 14:06:16,277][86121] Updated weights for policy 0, policy_version 47790 (0.0009) +[2023-10-09 14:06:16,646][86121] Updated weights for policy 0, policy_version 47800 (0.0010) +[2023-10-09 14:06:17,711][86122] Updated weights for policy 1, policy_version 48010 (0.0009) +[2023-10-09 14:06:18,075][86122] Updated weights for policy 1, policy_version 48020 (0.0009) +[2023-10-09 14:06:18,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 98107392. Throughput: 0: 1824.6, 1: 1832.6. Samples: 24537242. Policy #0 lag: (min: 38.0, avg: 55.1, max: 56.0) +[2023-10-09 14:06:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:06:18,438][86122] Updated weights for policy 1, policy_version 48030 (0.0008) +[2023-10-09 14:06:20,189][86121] Updated weights for policy 0, policy_version 47810 (0.0008) +[2023-10-09 14:06:20,573][86121] Updated weights for policy 0, policy_version 47820 (0.0009) +[2023-10-09 14:06:20,936][86121] Updated weights for policy 0, policy_version 47830 (0.0009) +[2023-10-09 14:06:21,309][86121] Updated weights for policy 0, policy_version 47840 (0.0008) +[2023-10-09 14:06:22,055][86122] Updated weights for policy 1, policy_version 48040 (0.0008) +[2023-10-09 14:06:22,417][86122] Updated weights for policy 1, policy_version 48050 (0.0009) +[2023-10-09 14:06:22,777][86122] Updated weights for policy 1, policy_version 48060 (0.0008) +[2023-10-09 14:06:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98205696. Throughput: 0: 1827.9, 1: 1826.1. Samples: 24558978. Policy #0 lag: (min: 38.0, avg: 55.1, max: 56.0) +[2023-10-09 14:06:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:06:24,983][86121] Updated weights for policy 0, policy_version 47850 (0.0008) +[2023-10-09 14:06:25,341][86121] Updated weights for policy 0, policy_version 47860 (0.0008) +[2023-10-09 14:06:25,704][86121] Updated weights for policy 0, policy_version 47870 (0.0011) +[2023-10-09 14:06:26,400][86122] Updated weights for policy 1, policy_version 48070 (0.0009) +[2023-10-09 14:06:26,754][86122] Updated weights for policy 1, policy_version 48080 (0.0010) +[2023-10-09 14:06:27,118][86122] Updated weights for policy 1, policy_version 48090 (0.0010) +[2023-10-09 14:06:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98271232. Throughput: 0: 1825.6, 1: 1831.9. Samples: 24570140. Policy #0 lag: (min: 38.0, avg: 55.1, max: 56.0) +[2023-10-09 14:06:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:06:29,397][86121] Updated weights for policy 0, policy_version 47880 (0.0009) +[2023-10-09 14:06:29,769][86121] Updated weights for policy 0, policy_version 47890 (0.0007) +[2023-10-09 14:06:30,142][86121] Updated weights for policy 0, policy_version 47900 (0.0009) +[2023-10-09 14:06:30,901][86122] Updated weights for policy 1, policy_version 48100 (0.0008) +[2023-10-09 14:06:31,271][86122] Updated weights for policy 1, policy_version 48110 (0.0010) +[2023-10-09 14:06:31,626][86122] Updated weights for policy 1, policy_version 48120 (0.0010) +[2023-10-09 14:06:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98336768. Throughput: 0: 1831.2, 1: 1826.2. Samples: 24591818. Policy #0 lag: (min: 38.0, avg: 55.1, max: 56.0) +[2023-10-09 14:06:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:06:33,787][86121] Updated weights for policy 0, policy_version 47910 (0.0008) +[2023-10-09 14:06:34,149][86121] Updated weights for policy 0, policy_version 47920 (0.0007) +[2023-10-09 14:06:34,520][86121] Updated weights for policy 0, policy_version 47930 (0.0007) +[2023-10-09 14:06:35,118][86122] Updated weights for policy 1, policy_version 48130 (0.0007) +[2023-10-09 14:06:35,474][86122] Updated weights for policy 1, policy_version 48140 (0.0007) +[2023-10-09 14:06:35,846][86122] Updated weights for policy 1, policy_version 48150 (0.0009) +[2023-10-09 14:06:36,200][86122] Updated weights for policy 1, policy_version 48160 (0.0009) +[2023-10-09 14:06:38,117][86121] Updated weights for policy 0, policy_version 47940 (0.0007) +[2023-10-09 14:06:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 98402304. Throughput: 0: 1831.8, 1: 1832.5. Samples: 24614636. Policy #0 lag: (min: 38.0, avg: 55.1, max: 56.0) +[2023-10-09 14:06:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:06:38,483][86121] Updated weights for policy 0, policy_version 47950 (0.0010) +[2023-10-09 14:06:38,848][86121] Updated weights for policy 0, policy_version 47960 (0.0010) +[2023-10-09 14:06:39,915][86122] Updated weights for policy 1, policy_version 48170 (0.0008) +[2023-10-09 14:06:40,280][86122] Updated weights for policy 1, policy_version 48180 (0.0008) +[2023-10-09 14:06:40,645][86122] Updated weights for policy 1, policy_version 48190 (0.0011) +[2023-10-09 14:06:42,677][86121] Updated weights for policy 0, policy_version 47970 (0.0010) +[2023-10-09 14:06:43,046][86121] Updated weights for policy 0, policy_version 47980 (0.0007) +[2023-10-09 14:06:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 98467840. Throughput: 0: 1835.1, 1: 1825.5. Samples: 24624684. Policy #0 lag: (min: 38.0, avg: 55.1, max: 56.0) +[2023-10-09 14:06:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:06:43,410][86121] Updated weights for policy 0, policy_version 47990 (0.0007) +[2023-10-09 14:06:43,784][86121] Updated weights for policy 0, policy_version 48000 (0.0008) +[2023-10-09 14:06:44,509][86122] Updated weights for policy 1, policy_version 48200 (0.0007) +[2023-10-09 14:06:44,879][86122] Updated weights for policy 1, policy_version 48210 (0.0007) +[2023-10-09 14:06:45,240][86122] Updated weights for policy 1, policy_version 48220 (0.0007) +[2023-10-09 14:06:47,379][86121] Updated weights for policy 0, policy_version 48010 (0.0010) +[2023-10-09 14:06:47,743][86121] Updated weights for policy 0, policy_version 48020 (0.0007) +[2023-10-09 14:06:48,109][86121] Updated weights for policy 0, policy_version 48030 (0.0008) +[2023-10-09 14:06:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98566144. Throughput: 0: 1833.5, 1: 1832.2. Samples: 24647294. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:06:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:06:48,894][86122] Updated weights for policy 1, policy_version 48230 (0.0009) +[2023-10-09 14:06:49,264][86122] Updated weights for policy 1, policy_version 48240 (0.0007) +[2023-10-09 14:06:49,636][86122] Updated weights for policy 1, policy_version 48250 (0.0010) +[2023-10-09 14:06:51,861][86121] Updated weights for policy 0, policy_version 48040 (0.0007) +[2023-10-09 14:06:52,236][86121] Updated weights for policy 0, policy_version 48050 (0.0007) +[2023-10-09 14:06:52,604][86121] Updated weights for policy 0, policy_version 48060 (0.0009) +[2023-10-09 14:06:53,373][86122] Updated weights for policy 1, policy_version 48260 (0.0009) +[2023-10-09 14:06:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 98631680. Throughput: 0: 1826.8, 1: 1822.3. Samples: 24668668. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:06:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 14:06:53,746][86122] Updated weights for policy 1, policy_version 48270 (0.0008) +[2023-10-09 14:06:54,109][86122] Updated weights for policy 1, policy_version 48280 (0.0007) +[2023-10-09 14:06:56,330][86121] Updated weights for policy 0, policy_version 48070 (0.0008) +[2023-10-09 14:06:56,705][86121] Updated weights for policy 0, policy_version 48080 (0.0008) +[2023-10-09 14:06:57,065][86121] Updated weights for policy 0, policy_version 48090 (0.0011) +[2023-10-09 14:06:57,786][86122] Updated weights for policy 1, policy_version 48290 (0.0008) +[2023-10-09 14:06:58,141][86122] Updated weights for policy 1, policy_version 48300 (0.0009) +[2023-10-09 14:06:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98697216. Throughput: 0: 1824.5, 1: 1825.4. Samples: 24679818. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:06:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 14:06:58,492][86122] Updated weights for policy 1, policy_version 48310 (0.0008) +[2023-10-09 14:06:58,865][86122] Updated weights for policy 1, policy_version 48320 (0.0009) +[2023-10-09 14:07:00,789][86121] Updated weights for policy 0, policy_version 48100 (0.0010) +[2023-10-09 14:07:01,150][86121] Updated weights for policy 0, policy_version 48110 (0.0008) +[2023-10-09 14:07:01,520][86121] Updated weights for policy 0, policy_version 48120 (0.0009) +[2023-10-09 14:07:02,695][86122] Updated weights for policy 1, policy_version 48330 (0.0011) +[2023-10-09 14:07:03,059][86122] Updated weights for policy 1, policy_version 48340 (0.0010) +[2023-10-09 14:07:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98762752. Throughput: 0: 1822.1, 1: 1821.1. Samples: 24701184. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:07:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 14:07:03,419][86122] Updated weights for policy 1, policy_version 48350 (0.0010) +[2023-10-09 14:07:05,102][86121] Updated weights for policy 0, policy_version 48130 (0.0010) +[2023-10-09 14:07:05,511][86121] Updated weights for policy 0, policy_version 48140 (0.0009) +[2023-10-09 14:07:05,883][86121] Updated weights for policy 0, policy_version 48150 (0.0009) +[2023-10-09 14:07:06,242][86121] Updated weights for policy 0, policy_version 48160 (0.0009) +[2023-10-09 14:07:06,971][86122] Updated weights for policy 1, policy_version 48360 (0.0008) +[2023-10-09 14:07:07,344][86122] Updated weights for policy 1, policy_version 48370 (0.0008) +[2023-10-09 14:07:07,702][86122] Updated weights for policy 1, policy_version 48380 (0.0009) +[2023-10-09 14:07:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 98861056. Throughput: 0: 1825.6, 1: 1819.8. Samples: 24723020. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:07:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:07:09,939][86121] Updated weights for policy 0, policy_version 48170 (0.0010) +[2023-10-09 14:07:10,307][86121] Updated weights for policy 0, policy_version 48180 (0.0008) +[2023-10-09 14:07:10,675][86121] Updated weights for policy 0, policy_version 48190 (0.0007) +[2023-10-09 14:07:11,368][86122] Updated weights for policy 1, policy_version 48390 (0.0007) +[2023-10-09 14:07:11,728][86122] Updated weights for policy 1, policy_version 48400 (0.0009) +[2023-10-09 14:07:12,091][86122] Updated weights for policy 1, policy_version 48410 (0.0011) +[2023-10-09 14:07:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 98926592. Throughput: 0: 1821.8, 1: 1824.8. Samples: 24734236. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:07:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:07:14,245][86121] Updated weights for policy 0, policy_version 48200 (0.0009) +[2023-10-09 14:07:14,622][86121] Updated weights for policy 0, policy_version 48210 (0.0010) +[2023-10-09 14:07:14,977][86121] Updated weights for policy 0, policy_version 48220 (0.0008) +[2023-10-09 14:07:15,697][86122] Updated weights for policy 1, policy_version 48420 (0.0009) +[2023-10-09 14:07:16,062][86122] Updated weights for policy 1, policy_version 48430 (0.0008) +[2023-10-09 14:07:16,426][86122] Updated weights for policy 1, policy_version 48440 (0.0008) +[2023-10-09 14:07:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 98992128. Throughput: 0: 1829.7, 1: 1823.8. Samples: 24756228. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:07:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:07:18,589][86121] Updated weights for policy 0, policy_version 48230 (0.0009) +[2023-10-09 14:07:18,948][86121] Updated weights for policy 0, policy_version 48240 (0.0009) +[2023-10-09 14:07:19,319][86121] Updated weights for policy 0, policy_version 48250 (0.0007) +[2023-10-09 14:07:20,151][86122] Updated weights for policy 1, policy_version 48450 (0.0008) +[2023-10-09 14:07:20,513][86122] Updated weights for policy 1, policy_version 48460 (0.0008) +[2023-10-09 14:07:20,881][86122] Updated weights for policy 1, policy_version 48470 (0.0009) +[2023-10-09 14:07:21,245][86122] Updated weights for policy 1, policy_version 48480 (0.0011) +[2023-10-09 14:07:23,046][86121] Updated weights for policy 0, policy_version 48260 (0.0009) +[2023-10-09 14:07:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99057664. Throughput: 0: 1825.2, 1: 1833.5. Samples: 24779278. Policy #0 lag: (min: 18.0, avg: 19.3, max: 42.0) +[2023-10-09 14:07:23,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:07:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000048480_49643520.pth... +[2023-10-09 14:07:23,423][86121] Updated weights for policy 0, policy_version 48270 (0.0009) +[2023-10-09 14:07:23,449][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000046784_47906816.pth +[2023-10-09 14:07:23,794][86121] Updated weights for policy 0, policy_version 48280 (0.0007) +[2023-10-09 14:07:24,076][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000048288_49446912.pth... +[2023-10-09 14:07:24,115][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000046560_47677440.pth +[2023-10-09 14:07:24,903][86122] Updated weights for policy 1, policy_version 48490 (0.0009) +[2023-10-09 14:07:25,263][86122] Updated weights for policy 1, policy_version 48500 (0.0010) +[2023-10-09 14:07:25,623][86122] Updated weights for policy 1, policy_version 48510 (0.0011) +[2023-10-09 14:07:27,478][86121] Updated weights for policy 0, policy_version 48290 (0.0008) +[2023-10-09 14:07:27,848][86121] Updated weights for policy 0, policy_version 48300 (0.0009) +[2023-10-09 14:07:28,211][86121] Updated weights for policy 0, policy_version 48310 (0.0008) +[2023-10-09 14:07:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99123200. Throughput: 0: 1826.3, 1: 1833.2. Samples: 24789360. Policy #0 lag: (min: 18.0, avg: 19.3, max: 42.0) +[2023-10-09 14:07:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:07:28,579][86121] Updated weights for policy 0, policy_version 48320 (0.0009) +[2023-10-09 14:07:29,404][86122] Updated weights for policy 1, policy_version 48520 (0.0009) +[2023-10-09 14:07:29,764][86122] Updated weights for policy 1, policy_version 48530 (0.0007) +[2023-10-09 14:07:30,121][86122] Updated weights for policy 1, policy_version 48540 (0.0008) +[2023-10-09 14:07:32,259][86121] Updated weights for policy 0, policy_version 48330 (0.0010) +[2023-10-09 14:07:32,621][86121] Updated weights for policy 0, policy_version 48340 (0.0008) +[2023-10-09 14:07:32,983][86121] Updated weights for policy 0, policy_version 48350 (0.0010) +[2023-10-09 14:07:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99221504. Throughput: 0: 1825.7, 1: 1834.8. Samples: 24812014. Policy #0 lag: (min: 18.0, avg: 19.3, max: 42.0) +[2023-10-09 14:07:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:07:33,836][86122] Updated weights for policy 1, policy_version 48550 (0.0008) +[2023-10-09 14:07:34,208][86122] Updated weights for policy 1, policy_version 48560 (0.0007) +[2023-10-09 14:07:34,563][86122] Updated weights for policy 1, policy_version 48570 (0.0007) +[2023-10-09 14:07:36,629][86121] Updated weights for policy 0, policy_version 48360 (0.0008) +[2023-10-09 14:07:36,989][86121] Updated weights for policy 0, policy_version 48370 (0.0007) +[2023-10-09 14:07:37,357][86121] Updated weights for policy 0, policy_version 48380 (0.0007) +[2023-10-09 14:07:38,253][86122] Updated weights for policy 1, policy_version 48580 (0.0009) +[2023-10-09 14:07:38,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99287040. Throughput: 0: 1824.8, 1: 1835.5. Samples: 24833380. Policy #0 lag: (min: 18.0, avg: 19.3, max: 42.0) +[2023-10-09 14:07:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:07:38,613][86122] Updated weights for policy 1, policy_version 48590 (0.0010) +[2023-10-09 14:07:38,983][86122] Updated weights for policy 1, policy_version 48600 (0.0007) +[2023-10-09 14:07:41,006][86121] Updated weights for policy 0, policy_version 48390 (0.0008) +[2023-10-09 14:07:41,376][86121] Updated weights for policy 0, policy_version 48400 (0.0007) +[2023-10-09 14:07:41,744][86121] Updated weights for policy 0, policy_version 48410 (0.0007) +[2023-10-09 14:07:42,630][86122] Updated weights for policy 1, policy_version 48610 (0.0007) +[2023-10-09 14:07:42,995][86122] Updated weights for policy 1, policy_version 48620 (0.0008) +[2023-10-09 14:07:43,360][86122] Updated weights for policy 1, policy_version 48630 (0.0009) +[2023-10-09 14:07:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99352576. Throughput: 0: 1828.9, 1: 1837.8. Samples: 24844822. Policy #0 lag: (min: 18.0, avg: 19.3, max: 42.0) +[2023-10-09 14:07:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:07:43,723][86122] Updated weights for policy 1, policy_version 48640 (0.0008) +[2023-10-09 14:07:45,535][86121] Updated weights for policy 0, policy_version 48420 (0.0009) +[2023-10-09 14:07:45,902][86121] Updated weights for policy 0, policy_version 48430 (0.0008) +[2023-10-09 14:07:46,270][86121] Updated weights for policy 0, policy_version 48440 (0.0007) +[2023-10-09 14:07:47,478][86122] Updated weights for policy 1, policy_version 48650 (0.0008) +[2023-10-09 14:07:47,841][86122] Updated weights for policy 1, policy_version 48660 (0.0007) +[2023-10-09 14:07:48,205][86122] Updated weights for policy 1, policy_version 48670 (0.0010) +[2023-10-09 14:07:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99450880. Throughput: 0: 1831.3, 1: 1844.3. Samples: 24866588. Policy #0 lag: (min: 18.0, avg: 19.3, max: 42.0) +[2023-10-09 14:07:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:07:49,931][86121] Updated weights for policy 0, policy_version 48450 (0.0007) +[2023-10-09 14:07:50,288][86121] Updated weights for policy 0, policy_version 48460 (0.0007) +[2023-10-09 14:07:50,659][86121] Updated weights for policy 0, policy_version 48470 (0.0008) +[2023-10-09 14:07:51,019][86121] Updated weights for policy 0, policy_version 48480 (0.0008) +[2023-10-09 14:07:51,796][86122] Updated weights for policy 1, policy_version 48680 (0.0009) +[2023-10-09 14:07:52,158][86122] Updated weights for policy 1, policy_version 48690 (0.0007) +[2023-10-09 14:07:52,524][86122] Updated weights for policy 1, policy_version 48700 (0.0007) +[2023-10-09 14:07:53,398][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 99516416. Throughput: 0: 1834.2, 1: 1837.8. Samples: 24888262. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) +[2023-10-09 14:07:53,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:07:54,704][86121] Updated weights for policy 0, policy_version 48490 (0.0008) +[2023-10-09 14:07:55,074][86121] Updated weights for policy 0, policy_version 48500 (0.0007) +[2023-10-09 14:07:55,448][86121] Updated weights for policy 0, policy_version 48510 (0.0011) +[2023-10-09 14:07:56,288][86122] Updated weights for policy 1, policy_version 48710 (0.0009) +[2023-10-09 14:07:56,648][86122] Updated weights for policy 1, policy_version 48720 (0.0009) +[2023-10-09 14:07:57,016][86122] Updated weights for policy 1, policy_version 48730 (0.0008) +[2023-10-09 14:07:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 99581952. Throughput: 0: 1836.9, 1: 1838.2. Samples: 24899614. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) +[2023-10-09 14:07:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:07:59,016][86121] Updated weights for policy 0, policy_version 48520 (0.0009) +[2023-10-09 14:07:59,379][86121] Updated weights for policy 0, policy_version 48530 (0.0007) +[2023-10-09 14:07:59,750][86121] Updated weights for policy 0, policy_version 48540 (0.0007) +[2023-10-09 14:08:00,656][86122] Updated weights for policy 1, policy_version 48740 (0.0009) +[2023-10-09 14:08:01,014][86122] Updated weights for policy 1, policy_version 48750 (0.0007) +[2023-10-09 14:08:01,376][86122] Updated weights for policy 1, policy_version 48760 (0.0007) +[2023-10-09 14:08:03,351][86121] Updated weights for policy 0, policy_version 48550 (0.0007) +[2023-10-09 14:08:03,397][85186] Fps is (10 sec: 13107.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99647488. Throughput: 0: 1833.1, 1: 1834.5. Samples: 24921268. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) +[2023-10-09 14:08:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:08:03,726][86121] Updated weights for policy 0, policy_version 48560 (0.0007) +[2023-10-09 14:08:04,084][86121] Updated weights for policy 0, policy_version 48570 (0.0008) +[2023-10-09 14:08:04,959][86122] Updated weights for policy 1, policy_version 48770 (0.0008) +[2023-10-09 14:08:05,330][86122] Updated weights for policy 1, policy_version 48780 (0.0008) +[2023-10-09 14:08:05,693][86122] Updated weights for policy 1, policy_version 48790 (0.0010) +[2023-10-09 14:08:06,064][86122] Updated weights for policy 1, policy_version 48800 (0.0007) +[2023-10-09 14:08:07,753][86121] Updated weights for policy 0, policy_version 48580 (0.0009) +[2023-10-09 14:08:08,120][86121] Updated weights for policy 0, policy_version 48590 (0.0007) +[2023-10-09 14:08:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 99713024. Throughput: 0: 1828.3, 1: 1831.3. Samples: 24943960. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) +[2023-10-09 14:08:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:08:08,484][86121] Updated weights for policy 0, policy_version 48600 (0.0009) +[2023-10-09 14:08:09,735][86122] Updated weights for policy 1, policy_version 48810 (0.0008) +[2023-10-09 14:08:10,095][86122] Updated weights for policy 1, policy_version 48820 (0.0007) +[2023-10-09 14:08:10,460][86122] Updated weights for policy 1, policy_version 48830 (0.0009) +[2023-10-09 14:08:12,192][86121] Updated weights for policy 0, policy_version 48610 (0.0009) +[2023-10-09 14:08:12,568][86121] Updated weights for policy 0, policy_version 48620 (0.0008) +[2023-10-09 14:08:12,932][86121] Updated weights for policy 0, policy_version 48630 (0.0009) +[2023-10-09 14:08:13,299][86121] Updated weights for policy 0, policy_version 48640 (0.0011) +[2023-10-09 14:08:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99811328. Throughput: 0: 1836.9, 1: 1831.1. Samples: 24954424. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) +[2023-10-09 14:08:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:13,989][86122] Updated weights for policy 1, policy_version 48840 (0.0010) +[2023-10-09 14:08:14,345][86122] Updated weights for policy 1, policy_version 48850 (0.0007) +[2023-10-09 14:08:14,709][86122] Updated weights for policy 1, policy_version 48860 (0.0007) +[2023-10-09 14:08:16,915][86121] Updated weights for policy 0, policy_version 48650 (0.0008) +[2023-10-09 14:08:17,280][86121] Updated weights for policy 0, policy_version 48660 (0.0009) +[2023-10-09 14:08:17,646][86121] Updated weights for policy 0, policy_version 48670 (0.0007) +[2023-10-09 14:08:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 99876864. Throughput: 0: 1831.1, 1: 1835.6. Samples: 24977018. Policy #0 lag: (min: 25.0, avg: 35.5, max: 57.0) +[2023-10-09 14:08:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:18,613][86122] Updated weights for policy 1, policy_version 48870 (0.0007) +[2023-10-09 14:08:18,996][86122] Updated weights for policy 1, policy_version 48880 (0.0007) +[2023-10-09 14:08:19,353][86122] Updated weights for policy 1, policy_version 48890 (0.0010) +[2023-10-09 14:08:21,307][86121] Updated weights for policy 0, policy_version 48680 (0.0007) +[2023-10-09 14:08:21,678][86121] Updated weights for policy 0, policy_version 48690 (0.0010) +[2023-10-09 14:08:22,045][86121] Updated weights for policy 0, policy_version 48700 (0.0010) +[2023-10-09 14:08:23,030][86122] Updated weights for policy 1, policy_version 48900 (0.0009) +[2023-10-09 14:08:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 99942400. Throughput: 0: 1837.0, 1: 1832.9. Samples: 24998524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:08:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:23,409][86122] Updated weights for policy 1, policy_version 48910 (0.0008) +[2023-10-09 14:08:23,768][86122] Updated weights for policy 1, policy_version 48920 (0.0007) +[2023-10-09 14:08:25,753][86121] Updated weights for policy 0, policy_version 48710 (0.0010) +[2023-10-09 14:08:26,121][86121] Updated weights for policy 0, policy_version 48720 (0.0009) +[2023-10-09 14:08:26,487][86121] Updated weights for policy 0, policy_version 48730 (0.0008) +[2023-10-09 14:08:27,355][86122] Updated weights for policy 1, policy_version 48930 (0.0009) +[2023-10-09 14:08:27,717][86122] Updated weights for policy 1, policy_version 48940 (0.0007) +[2023-10-09 14:08:28,083][86122] Updated weights for policy 1, policy_version 48950 (0.0007) +[2023-10-09 14:08:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 100007936. Throughput: 0: 1824.0, 1: 1832.8. Samples: 25009380. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:08:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:28,439][86122] Updated weights for policy 1, policy_version 48960 (0.0009) +[2023-10-09 14:08:30,265][86121] Updated weights for policy 0, policy_version 48740 (0.0010) +[2023-10-09 14:08:30,628][86121] Updated weights for policy 0, policy_version 48750 (0.0010) +[2023-10-09 14:08:30,993][86121] Updated weights for policy 0, policy_version 48760 (0.0011) +[2023-10-09 14:08:32,154][86122] Updated weights for policy 1, policy_version 48970 (0.0008) +[2023-10-09 14:08:32,515][86122] Updated weights for policy 1, policy_version 48980 (0.0007) +[2023-10-09 14:08:32,882][86122] Updated weights for policy 1, policy_version 48990 (0.0007) +[2023-10-09 14:08:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100106240. Throughput: 0: 1829.2, 1: 1830.2. Samples: 25031260. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:08:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:34,697][86121] Updated weights for policy 0, policy_version 48770 (0.0009) +[2023-10-09 14:08:35,069][86121] Updated weights for policy 0, policy_version 48780 (0.0008) +[2023-10-09 14:08:35,437][86121] Updated weights for policy 0, policy_version 48790 (0.0009) +[2023-10-09 14:08:35,804][86121] Updated weights for policy 0, policy_version 48800 (0.0007) +[2023-10-09 14:08:36,494][86122] Updated weights for policy 1, policy_version 49000 (0.0008) +[2023-10-09 14:08:36,854][86122] Updated weights for policy 1, policy_version 49010 (0.0009) +[2023-10-09 14:08:37,231][86122] Updated weights for policy 1, policy_version 49020 (0.0008) +[2023-10-09 14:08:38,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100171776. Throughput: 0: 1823.5, 1: 1832.6. Samples: 25052784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:08:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:39,487][86121] Updated weights for policy 0, policy_version 48810 (0.0008) +[2023-10-09 14:08:39,855][86121] Updated weights for policy 0, policy_version 48820 (0.0009) +[2023-10-09 14:08:40,228][86121] Updated weights for policy 0, policy_version 48830 (0.0009) +[2023-10-09 14:08:40,856][86122] Updated weights for policy 1, policy_version 49030 (0.0009) +[2023-10-09 14:08:41,221][86122] Updated weights for policy 1, policy_version 49040 (0.0009) +[2023-10-09 14:08:41,582][86122] Updated weights for policy 1, policy_version 49050 (0.0008) +[2023-10-09 14:08:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 100237312. Throughput: 0: 1822.1, 1: 1828.1. Samples: 25063874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:08:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:44,003][86121] Updated weights for policy 0, policy_version 48840 (0.0009) +[2023-10-09 14:08:44,376][86121] Updated weights for policy 0, policy_version 48850 (0.0009) +[2023-10-09 14:08:44,755][86121] Updated weights for policy 0, policy_version 48860 (0.0009) +[2023-10-09 14:08:45,355][86122] Updated weights for policy 1, policy_version 49060 (0.0009) +[2023-10-09 14:08:45,726][86122] Updated weights for policy 1, policy_version 49070 (0.0008) +[2023-10-09 14:08:46,092][86122] Updated weights for policy 1, policy_version 49080 (0.0009) +[2023-10-09 14:08:48,353][86121] Updated weights for policy 0, policy_version 48870 (0.0009) +[2023-10-09 14:08:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100302848. Throughput: 0: 1818.9, 1: 1830.8. Samples: 25085506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:08:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:08:48,725][86121] Updated weights for policy 0, policy_version 48880 (0.0007) +[2023-10-09 14:08:49,088][86121] Updated weights for policy 0, policy_version 48890 (0.0008) +[2023-10-09 14:08:49,735][86122] Updated weights for policy 1, policy_version 49090 (0.0009) +[2023-10-09 14:08:50,099][86122] Updated weights for policy 1, policy_version 49100 (0.0007) +[2023-10-09 14:08:50,469][86122] Updated weights for policy 1, policy_version 49110 (0.0010) +[2023-10-09 14:08:50,824][86122] Updated weights for policy 1, policy_version 49120 (0.0012) +[2023-10-09 14:08:52,852][86121] Updated weights for policy 0, policy_version 48900 (0.0011) +[2023-10-09 14:08:53,217][86121] Updated weights for policy 0, policy_version 48910 (0.0007) +[2023-10-09 14:08:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.6, 300 sec: 14551.2). Total num frames: 100368384. Throughput: 0: 1815.7, 1: 1828.9. Samples: 25107970. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:08:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:08:53,586][86121] Updated weights for policy 0, policy_version 48920 (0.0009) +[2023-10-09 14:08:54,520][86122] Updated weights for policy 1, policy_version 49130 (0.0009) +[2023-10-09 14:08:54,877][86122] Updated weights for policy 1, policy_version 49140 (0.0007) +[2023-10-09 14:08:55,238][86122] Updated weights for policy 1, policy_version 49150 (0.0007) +[2023-10-09 14:08:57,184][86121] Updated weights for policy 0, policy_version 48930 (0.0007) +[2023-10-09 14:08:57,552][86121] Updated weights for policy 0, policy_version 48940 (0.0010) +[2023-10-09 14:08:57,916][86121] Updated weights for policy 0, policy_version 48950 (0.0009) +[2023-10-09 14:08:58,278][86121] Updated weights for policy 0, policy_version 48960 (0.0008) +[2023-10-09 14:08:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100466688. Throughput: 0: 1813.9, 1: 1826.1. Samples: 25118222. Policy #0 lag: (min: 14.0, avg: 14.0, max: 17.0) +[2023-10-09 14:08:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:08:58,881][86122] Updated weights for policy 1, policy_version 49160 (0.0011) +[2023-10-09 14:08:59,245][86122] Updated weights for policy 1, policy_version 49170 (0.0009) +[2023-10-09 14:08:59,605][86122] Updated weights for policy 1, policy_version 49180 (0.0009) +[2023-10-09 14:09:02,149][86121] Updated weights for policy 0, policy_version 48970 (0.0009) +[2023-10-09 14:09:02,511][86121] Updated weights for policy 0, policy_version 48980 (0.0007) +[2023-10-09 14:09:02,884][86121] Updated weights for policy 0, policy_version 48990 (0.0007) +[2023-10-09 14:09:03,299][86122] Updated weights for policy 1, policy_version 49190 (0.0008) +[2023-10-09 14:09:03,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100532224. Throughput: 0: 1813.8, 1: 1827.7. Samples: 25140886. Policy #0 lag: (min: 14.0, avg: 14.0, max: 17.0) +[2023-10-09 14:09:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:03,682][86122] Updated weights for policy 1, policy_version 49200 (0.0010) +[2023-10-09 14:09:04,033][86122] Updated weights for policy 1, policy_version 49210 (0.0008) +[2023-10-09 14:09:06,628][86121] Updated weights for policy 0, policy_version 49000 (0.0010) +[2023-10-09 14:09:06,997][86121] Updated weights for policy 0, policy_version 49010 (0.0010) +[2023-10-09 14:09:07,358][86121] Updated weights for policy 0, policy_version 49020 (0.0010) +[2023-10-09 14:09:07,563][86122] Updated weights for policy 1, policy_version 49220 (0.0010) +[2023-10-09 14:09:07,917][86122] Updated weights for policy 1, policy_version 49230 (0.0007) +[2023-10-09 14:09:08,276][86122] Updated weights for policy 1, policy_version 49240 (0.0008) +[2023-10-09 14:09:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100597760. Throughput: 0: 1808.6, 1: 1823.1. Samples: 25161950. Policy #0 lag: (min: 14.0, avg: 14.0, max: 17.0) +[2023-10-09 14:09:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:11,040][86121] Updated weights for policy 0, policy_version 49030 (0.0009) +[2023-10-09 14:09:11,411][86121] Updated weights for policy 0, policy_version 49040 (0.0009) +[2023-10-09 14:09:11,770][86121] Updated weights for policy 0, policy_version 49050 (0.0007) +[2023-10-09 14:09:11,888][86122] Updated weights for policy 1, policy_version 49250 (0.0008) +[2023-10-09 14:09:12,243][86122] Updated weights for policy 1, policy_version 49260 (0.0008) +[2023-10-09 14:09:12,612][86122] Updated weights for policy 1, policy_version 49270 (0.0008) +[2023-10-09 14:09:12,968][86122] Updated weights for policy 1, policy_version 49280 (0.0008) +[2023-10-09 14:09:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 100696064. Throughput: 0: 1815.8, 1: 1836.9. Samples: 25173752. Policy #0 lag: (min: 14.0, avg: 14.0, max: 17.0) +[2023-10-09 14:09:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:09:15,379][86121] Updated weights for policy 0, policy_version 49060 (0.0007) +[2023-10-09 14:09:15,740][86121] Updated weights for policy 0, policy_version 49070 (0.0008) +[2023-10-09 14:09:16,109][86121] Updated weights for policy 0, policy_version 49080 (0.0007) +[2023-10-09 14:09:16,706][86122] Updated weights for policy 1, policy_version 49290 (0.0008) +[2023-10-09 14:09:17,069][86122] Updated weights for policy 1, policy_version 49300 (0.0007) +[2023-10-09 14:09:17,434][86122] Updated weights for policy 1, policy_version 49310 (0.0007) +[2023-10-09 14:09:18,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100761600. Throughput: 0: 1816.1, 1: 1820.3. Samples: 25194896. Policy #0 lag: (min: 14.0, avg: 14.0, max: 17.0) +[2023-10-09 14:09:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:19,717][86121] Updated weights for policy 0, policy_version 49090 (0.0008) +[2023-10-09 14:09:20,088][86121] Updated weights for policy 0, policy_version 49100 (0.0007) +[2023-10-09 14:09:20,448][86121] Updated weights for policy 0, policy_version 49110 (0.0008) +[2023-10-09 14:09:20,815][86121] Updated weights for policy 0, policy_version 49120 (0.0008) +[2023-10-09 14:09:21,155][86122] Updated weights for policy 1, policy_version 49320 (0.0008) +[2023-10-09 14:09:21,517][86122] Updated weights for policy 1, policy_version 49330 (0.0008) +[2023-10-09 14:09:21,872][86122] Updated weights for policy 1, policy_version 49340 (0.0008) +[2023-10-09 14:09:23,397][85186] Fps is (10 sec: 13106.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 100827136. Throughput: 0: 1821.5, 1: 1829.6. Samples: 25217084. Policy #0 lag: (min: 14.0, avg: 14.0, max: 17.0) +[2023-10-09 14:09:23,399][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000049344_50528256.pth... +[2023-10-09 14:09:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000049120_50298880.pth... +[2023-10-09 14:09:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000047424_48562176.pth +[2023-10-09 14:09:23,448][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000047616_48758784.pth +[2023-10-09 14:09:24,508][86121] Updated weights for policy 0, policy_version 49130 (0.0010) +[2023-10-09 14:09:24,871][86121] Updated weights for policy 0, policy_version 49140 (0.0007) +[2023-10-09 14:09:25,232][86121] Updated weights for policy 0, policy_version 49150 (0.0010) +[2023-10-09 14:09:25,479][86122] Updated weights for policy 1, policy_version 49350 (0.0008) +[2023-10-09 14:09:25,845][86122] Updated weights for policy 1, policy_version 49360 (0.0009) +[2023-10-09 14:09:26,205][86122] Updated weights for policy 1, policy_version 49370 (0.0010) +[2023-10-09 14:09:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 100892672. Throughput: 0: 1821.3, 1: 1822.9. Samples: 25227864. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:09:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:29,060][86121] Updated weights for policy 0, policy_version 49160 (0.0009) +[2023-10-09 14:09:29,425][86121] Updated weights for policy 0, policy_version 49170 (0.0010) +[2023-10-09 14:09:29,790][86121] Updated weights for policy 0, policy_version 49180 (0.0008) +[2023-10-09 14:09:29,883][86122] Updated weights for policy 1, policy_version 49380 (0.0010) +[2023-10-09 14:09:30,253][86122] Updated weights for policy 1, policy_version 49390 (0.0010) +[2023-10-09 14:09:30,606][86122] Updated weights for policy 1, policy_version 49400 (0.0010) +[2023-10-09 14:09:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 100958208. Throughput: 0: 1816.0, 1: 1834.2. Samples: 25249764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:09:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:33,522][86121] Updated weights for policy 0, policy_version 49190 (0.0009) +[2023-10-09 14:09:33,899][86121] Updated weights for policy 0, policy_version 49200 (0.0008) +[2023-10-09 14:09:34,266][86121] Updated weights for policy 0, policy_version 49210 (0.0009) +[2023-10-09 14:09:34,343][86122] Updated weights for policy 1, policy_version 49410 (0.0010) +[2023-10-09 14:09:34,715][86122] Updated weights for policy 1, policy_version 49420 (0.0010) +[2023-10-09 14:09:35,074][86122] Updated weights for policy 1, policy_version 49430 (0.0009) +[2023-10-09 14:09:35,441][86122] Updated weights for policy 1, policy_version 49440 (0.0009) +[2023-10-09 14:09:37,983][86121] Updated weights for policy 0, policy_version 49220 (0.0009) +[2023-10-09 14:09:38,352][86121] Updated weights for policy 0, policy_version 49230 (0.0011) +[2023-10-09 14:09:38,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 101023744. Throughput: 0: 1817.7, 1: 1836.4. Samples: 25272408. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:09:38,399][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:38,714][86121] Updated weights for policy 0, policy_version 49240 (0.0007) +[2023-10-09 14:09:39,169][86122] Updated weights for policy 1, policy_version 49450 (0.0009) +[2023-10-09 14:09:39,533][86122] Updated weights for policy 1, policy_version 49460 (0.0008) +[2023-10-09 14:09:39,891][86122] Updated weights for policy 1, policy_version 49470 (0.0008) +[2023-10-09 14:09:42,497][86121] Updated weights for policy 0, policy_version 49250 (0.0007) +[2023-10-09 14:09:42,860][86121] Updated weights for policy 0, policy_version 49260 (0.0007) +[2023-10-09 14:09:43,227][86121] Updated weights for policy 0, policy_version 49270 (0.0008) +[2023-10-09 14:09:43,324][86122] Updated weights for policy 1, policy_version 49480 (0.0008) +[2023-10-09 14:09:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 101089280. Throughput: 0: 1813.8, 1: 1835.2. Samples: 25282430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:09:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:09:43,583][86121] Updated weights for policy 0, policy_version 49280 (0.0008) +[2023-10-09 14:09:43,693][86122] Updated weights for policy 1, policy_version 49490 (0.0008) +[2023-10-09 14:09:44,046][86122] Updated weights for policy 1, policy_version 49500 (0.0011) +[2023-10-09 14:09:47,266][86121] Updated weights for policy 0, policy_version 49290 (0.0007) +[2023-10-09 14:09:47,635][86121] Updated weights for policy 0, policy_version 49300 (0.0007) +[2023-10-09 14:09:47,751][86122] Updated weights for policy 1, policy_version 49510 (0.0009) +[2023-10-09 14:09:47,999][86121] Updated weights for policy 0, policy_version 49310 (0.0007) +[2023-10-09 14:09:48,105][86122] Updated weights for policy 1, policy_version 49520 (0.0008) +[2023-10-09 14:09:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101187584. Throughput: 0: 1822.4, 1: 1834.1. Samples: 25305430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:09:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:09:48,474][86122] Updated weights for policy 1, policy_version 49530 (0.0008) +[2023-10-09 14:09:51,634][86121] Updated weights for policy 0, policy_version 49320 (0.0011) +[2023-10-09 14:09:52,011][86121] Updated weights for policy 0, policy_version 49330 (0.0009) +[2023-10-09 14:09:52,268][86122] Updated weights for policy 1, policy_version 49540 (0.0009) +[2023-10-09 14:09:52,370][86121] Updated weights for policy 0, policy_version 49340 (0.0008) +[2023-10-09 14:09:52,661][86122] Updated weights for policy 1, policy_version 49550 (0.0008) +[2023-10-09 14:09:53,015][86122] Updated weights for policy 1, policy_version 49560 (0.0009) +[2023-10-09 14:09:53,397][85186] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 101285888. Throughput: 0: 1815.2, 1: 1818.7. Samples: 25325478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:09:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:09:56,248][86121] Updated weights for policy 0, policy_version 49350 (0.0010) +[2023-10-09 14:09:56,618][86121] Updated weights for policy 0, policy_version 49360 (0.0007) +[2023-10-09 14:09:56,743][86122] Updated weights for policy 1, policy_version 49570 (0.0009) +[2023-10-09 14:09:56,990][86121] Updated weights for policy 0, policy_version 49370 (0.0007) +[2023-10-09 14:09:57,101][86122] Updated weights for policy 1, policy_version 49580 (0.0008) +[2023-10-09 14:09:57,463][86122] Updated weights for policy 1, policy_version 49590 (0.0009) +[2023-10-09 14:09:57,825][86122] Updated weights for policy 1, policy_version 49600 (0.0009) +[2023-10-09 14:09:58,398][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 101351424. Throughput: 0: 1814.3, 1: 1817.8. Samples: 25337196. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:09:58,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:10:00,777][86121] Updated weights for policy 0, policy_version 49380 (0.0007) +[2023-10-09 14:10:01,146][86121] Updated weights for policy 0, policy_version 49390 (0.0009) +[2023-10-09 14:10:01,511][86121] Updated weights for policy 0, policy_version 49400 (0.0009) +[2023-10-09 14:10:01,639][86122] Updated weights for policy 1, policy_version 49610 (0.0008) +[2023-10-09 14:10:02,000][86122] Updated weights for policy 1, policy_version 49620 (0.0008) +[2023-10-09 14:10:02,363][86122] Updated weights for policy 1, policy_version 49630 (0.0011) +[2023-10-09 14:10:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101416960. Throughput: 0: 1797.3, 1: 1817.3. Samples: 25357554. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:10:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:10:05,135][86121] Updated weights for policy 0, policy_version 49410 (0.0009) +[2023-10-09 14:10:05,506][86121] Updated weights for policy 0, policy_version 49420 (0.0011) +[2023-10-09 14:10:05,862][86121] Updated weights for policy 0, policy_version 49430 (0.0008) +[2023-10-09 14:10:06,177][86122] Updated weights for policy 1, policy_version 49640 (0.0010) +[2023-10-09 14:10:06,231][86121] Updated weights for policy 0, policy_version 49440 (0.0008) +[2023-10-09 14:10:06,549][86122] Updated weights for policy 1, policy_version 49650 (0.0010) +[2023-10-09 14:10:06,907][86122] Updated weights for policy 1, policy_version 49660 (0.0010) +[2023-10-09 14:10:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 101482496. Throughput: 0: 1793.7, 1: 1821.9. Samples: 25379788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:10:08,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:09,964][86121] Updated weights for policy 0, policy_version 49450 (0.0007) +[2023-10-09 14:10:10,330][86121] Updated weights for policy 0, policy_version 49460 (0.0008) +[2023-10-09 14:10:10,617][86122] Updated weights for policy 1, policy_version 49670 (0.0009) +[2023-10-09 14:10:10,690][86121] Updated weights for policy 0, policy_version 49470 (0.0009) +[2023-10-09 14:10:10,984][86122] Updated weights for policy 1, policy_version 49680 (0.0007) +[2023-10-09 14:10:11,336][86122] Updated weights for policy 1, policy_version 49690 (0.0007) +[2023-10-09 14:10:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 101548032. Throughput: 0: 1794.7, 1: 1819.1. Samples: 25390486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:10:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:14,610][86121] Updated weights for policy 0, policy_version 49480 (0.0007) +[2023-10-09 14:10:14,900][86122] Updated weights for policy 1, policy_version 49700 (0.0008) +[2023-10-09 14:10:14,972][86121] Updated weights for policy 0, policy_version 49490 (0.0008) +[2023-10-09 14:10:15,272][86122] Updated weights for policy 1, policy_version 49710 (0.0008) +[2023-10-09 14:10:15,336][86121] Updated weights for policy 0, policy_version 49500 (0.0009) +[2023-10-09 14:10:15,633][86122] Updated weights for policy 1, policy_version 49720 (0.0008) +[2023-10-09 14:10:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 101613568. Throughput: 0: 1792.6, 1: 1823.3. Samples: 25412480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:10:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:19,016][86121] Updated weights for policy 0, policy_version 49510 (0.0008) +[2023-10-09 14:10:19,318][86122] Updated weights for policy 1, policy_version 49730 (0.0008) +[2023-10-09 14:10:19,391][86121] Updated weights for policy 0, policy_version 49520 (0.0007) +[2023-10-09 14:10:19,680][86122] Updated weights for policy 1, policy_version 49740 (0.0007) +[2023-10-09 14:10:19,749][86121] Updated weights for policy 0, policy_version 49530 (0.0008) +[2023-10-09 14:10:20,048][86122] Updated weights for policy 1, policy_version 49750 (0.0007) +[2023-10-09 14:10:20,406][86122] Updated weights for policy 1, policy_version 49760 (0.0009) +[2023-10-09 14:10:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 101679104. Throughput: 0: 1800.3, 1: 1821.9. Samples: 25435406. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:10:23,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:23,609][86121] Updated weights for policy 0, policy_version 49540 (0.0009) +[2023-10-09 14:10:23,968][86121] Updated weights for policy 0, policy_version 49550 (0.0008) +[2023-10-09 14:10:24,023][86122] Updated weights for policy 1, policy_version 49770 (0.0007) +[2023-10-09 14:10:24,320][86121] Updated weights for policy 0, policy_version 49560 (0.0007) +[2023-10-09 14:10:24,382][86122] Updated weights for policy 1, policy_version 49780 (0.0008) +[2023-10-09 14:10:24,750][86122] Updated weights for policy 1, policy_version 49790 (0.0008) +[2023-10-09 14:10:28,024][86121] Updated weights for policy 0, policy_version 49570 (0.0007) +[2023-10-09 14:10:28,381][86121] Updated weights for policy 0, policy_version 49580 (0.0009) +[2023-10-09 14:10:28,398][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 101744640. Throughput: 0: 1794.2, 1: 1822.7. Samples: 25445192. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:10:28,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:28,455][86122] Updated weights for policy 1, policy_version 49800 (0.0008) +[2023-10-09 14:10:28,744][86121] Updated weights for policy 0, policy_version 49590 (0.0009) +[2023-10-09 14:10:28,818][86122] Updated weights for policy 1, policy_version 49810 (0.0007) +[2023-10-09 14:10:29,112][86121] Updated weights for policy 0, policy_version 49600 (0.0008) +[2023-10-09 14:10:29,171][86122] Updated weights for policy 1, policy_version 49820 (0.0008) +[2023-10-09 14:10:32,834][86121] Updated weights for policy 0, policy_version 49610 (0.0008) +[2023-10-09 14:10:32,948][86122] Updated weights for policy 1, policy_version 49830 (0.0007) +[2023-10-09 14:10:33,211][86121] Updated weights for policy 0, policy_version 49620 (0.0008) +[2023-10-09 14:10:33,311][86122] Updated weights for policy 1, policy_version 49840 (0.0007) +[2023-10-09 14:10:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 101810176. Throughput: 0: 1795.1, 1: 1818.1. Samples: 25468022. Policy #0 lag: (min: 33.0, avg: 46.9, max: 48.0) +[2023-10-09 14:10:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:33,584][86121] Updated weights for policy 0, policy_version 49630 (0.0008) +[2023-10-09 14:10:33,668][86122] Updated weights for policy 1, policy_version 49850 (0.0008) +[2023-10-09 14:10:37,333][86121] Updated weights for policy 0, policy_version 49640 (0.0008) +[2023-10-09 14:10:37,380][86122] Updated weights for policy 1, policy_version 49860 (0.0010) +[2023-10-09 14:10:37,695][86121] Updated weights for policy 0, policy_version 49650 (0.0009) +[2023-10-09 14:10:37,771][86122] Updated weights for policy 1, policy_version 49870 (0.0009) +[2023-10-09 14:10:38,064][86121] Updated weights for policy 0, policy_version 49660 (0.0010) +[2023-10-09 14:10:38,136][86122] Updated weights for policy 1, policy_version 49880 (0.0007) +[2023-10-09 14:10:38,397][85186] Fps is (10 sec: 16384.6, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 101908480. Throughput: 0: 1807.3, 1: 1827.7. Samples: 25489052. Policy #0 lag: (min: 33.0, avg: 46.9, max: 48.0) +[2023-10-09 14:10:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:41,645][86121] Updated weights for policy 0, policy_version 49670 (0.0008) +[2023-10-09 14:10:41,756][86122] Updated weights for policy 1, policy_version 49890 (0.0008) +[2023-10-09 14:10:42,012][86121] Updated weights for policy 0, policy_version 49680 (0.0009) +[2023-10-09 14:10:42,117][86122] Updated weights for policy 1, policy_version 49900 (0.0008) +[2023-10-09 14:10:42,373][86121] Updated weights for policy 0, policy_version 49690 (0.0008) +[2023-10-09 14:10:42,479][86122] Updated weights for policy 1, policy_version 49910 (0.0008) +[2023-10-09 14:10:42,846][86122] Updated weights for policy 1, policy_version 49920 (0.0009) +[2023-10-09 14:10:43,397][85186] Fps is (10 sec: 19661.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 102006784. Throughput: 0: 1801.8, 1: 1830.0. Samples: 25500626. Policy #0 lag: (min: 33.0, avg: 46.9, max: 48.0) +[2023-10-09 14:10:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:46,056][86121] Updated weights for policy 0, policy_version 49700 (0.0009) +[2023-10-09 14:10:46,416][86121] Updated weights for policy 0, policy_version 49710 (0.0007) +[2023-10-09 14:10:46,638][86122] Updated weights for policy 1, policy_version 49930 (0.0008) +[2023-10-09 14:10:46,781][86121] Updated weights for policy 0, policy_version 49720 (0.0007) +[2023-10-09 14:10:46,996][86122] Updated weights for policy 1, policy_version 49940 (0.0007) +[2023-10-09 14:10:47,359][86122] Updated weights for policy 1, policy_version 49950 (0.0010) +[2023-10-09 14:10:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102072320. Throughput: 0: 1815.7, 1: 1829.4. Samples: 25521586. Policy #0 lag: (min: 33.0, avg: 46.9, max: 48.0) +[2023-10-09 14:10:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:50,568][86121] Updated weights for policy 0, policy_version 49730 (0.0008) +[2023-10-09 14:10:50,937][86121] Updated weights for policy 0, policy_version 49740 (0.0008) +[2023-10-09 14:10:51,054][86122] Updated weights for policy 1, policy_version 49960 (0.0010) +[2023-10-09 14:10:51,306][86121] Updated weights for policy 0, policy_version 49750 (0.0007) +[2023-10-09 14:10:51,412][86122] Updated weights for policy 1, policy_version 49970 (0.0008) +[2023-10-09 14:10:51,671][86121] Updated weights for policy 0, policy_version 49760 (0.0009) +[2023-10-09 14:10:51,776][86122] Updated weights for policy 1, policy_version 49980 (0.0007) +[2023-10-09 14:10:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 102137856. Throughput: 0: 1804.1, 1: 1828.2. Samples: 25543240. Policy #0 lag: (min: 33.0, avg: 46.9, max: 48.0) +[2023-10-09 14:10:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:10:55,245][86122] Updated weights for policy 1, policy_version 49990 (0.0008) +[2023-10-09 14:10:55,457][86121] Updated weights for policy 0, policy_version 49770 (0.0008) +[2023-10-09 14:10:55,601][86122] Updated weights for policy 1, policy_version 50000 (0.0008) +[2023-10-09 14:10:55,826][86121] Updated weights for policy 0, policy_version 49780 (0.0009) +[2023-10-09 14:10:55,972][86122] Updated weights for policy 1, policy_version 50010 (0.0007) +[2023-10-09 14:10:56,200][86121] Updated weights for policy 0, policy_version 49790 (0.0008) +[2023-10-09 14:10:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 102203392. Throughput: 0: 1815.1, 1: 1829.9. Samples: 25554512. Policy #0 lag: (min: 33.0, avg: 46.9, max: 48.0) +[2023-10-09 14:10:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:10:59,623][86121] Updated weights for policy 0, policy_version 49800 (0.0009) +[2023-10-09 14:10:59,792][86122] Updated weights for policy 1, policy_version 50020 (0.0009) +[2023-10-09 14:10:59,993][86121] Updated weights for policy 0, policy_version 49810 (0.0007) +[2023-10-09 14:11:00,155][86122] Updated weights for policy 1, policy_version 50030 (0.0009) +[2023-10-09 14:11:00,355][86121] Updated weights for policy 0, policy_version 49820 (0.0008) +[2023-10-09 14:11:00,510][86122] Updated weights for policy 1, policy_version 50040 (0.0009) +[2023-10-09 14:11:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102268928. Throughput: 0: 1812.8, 1: 1831.3. Samples: 25576464. Policy #0 lag: (min: 33.0, avg: 46.9, max: 48.0) +[2023-10-09 14:11:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:11:04,120][86121] Updated weights for policy 0, policy_version 49830 (0.0008) +[2023-10-09 14:11:04,228][86122] Updated weights for policy 1, policy_version 50050 (0.0008) +[2023-10-09 14:11:04,495][86121] Updated weights for policy 0, policy_version 49840 (0.0007) +[2023-10-09 14:11:04,593][86122] Updated weights for policy 1, policy_version 50060 (0.0007) +[2023-10-09 14:11:04,860][86121] Updated weights for policy 0, policy_version 49850 (0.0009) +[2023-10-09 14:11:04,955][86122] Updated weights for policy 1, policy_version 50070 (0.0007) +[2023-10-09 14:11:05,315][86122] Updated weights for policy 1, policy_version 50080 (0.0010) +[2023-10-09 14:11:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102334464. Throughput: 0: 1812.1, 1: 1830.3. Samples: 25599310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:11:08,569][86121] Updated weights for policy 0, policy_version 49860 (0.0009) +[2023-10-09 14:11:08,928][86122] Updated weights for policy 1, policy_version 50090 (0.0007) +[2023-10-09 14:11:08,936][86121] Updated weights for policy 0, policy_version 49870 (0.0007) +[2023-10-09 14:11:09,283][86122] Updated weights for policy 1, policy_version 50100 (0.0007) +[2023-10-09 14:11:09,303][86121] Updated weights for policy 0, policy_version 49880 (0.0008) +[2023-10-09 14:11:09,638][86122] Updated weights for policy 1, policy_version 50110 (0.0010) +[2023-10-09 14:11:12,922][86121] Updated weights for policy 0, policy_version 49890 (0.0008) +[2023-10-09 14:11:13,294][86121] Updated weights for policy 0, policy_version 49900 (0.0008) +[2023-10-09 14:11:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102400000. Throughput: 0: 1811.2, 1: 1832.5. Samples: 25609156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:11:13,398][86122] Updated weights for policy 1, policy_version 50120 (0.0007) +[2023-10-09 14:11:13,663][86121] Updated weights for policy 0, policy_version 49910 (0.0009) +[2023-10-09 14:11:13,757][86122] Updated weights for policy 1, policy_version 50130 (0.0008) +[2023-10-09 14:11:14,024][86121] Updated weights for policy 0, policy_version 49920 (0.0007) +[2023-10-09 14:11:14,124][86122] Updated weights for policy 1, policy_version 50140 (0.0008) +[2023-10-09 14:11:17,776][86121] Updated weights for policy 0, policy_version 49930 (0.0008) +[2023-10-09 14:11:17,928][86122] Updated weights for policy 1, policy_version 50150 (0.0008) +[2023-10-09 14:11:18,139][86121] Updated weights for policy 0, policy_version 49940 (0.0009) +[2023-10-09 14:11:18,286][86122] Updated weights for policy 1, policy_version 50160 (0.0008) +[2023-10-09 14:11:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 102465536. Throughput: 0: 1816.8, 1: 1827.6. Samples: 25632018. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:11:18,496][86121] Updated weights for policy 0, policy_version 49950 (0.0008) +[2023-10-09 14:11:18,657][86122] Updated weights for policy 1, policy_version 50170 (0.0009) +[2023-10-09 14:11:22,139][86121] Updated weights for policy 0, policy_version 49960 (0.0008) +[2023-10-09 14:11:22,456][86122] Updated weights for policy 1, policy_version 50180 (0.0008) +[2023-10-09 14:11:22,511][86121] Updated weights for policy 0, policy_version 49970 (0.0008) +[2023-10-09 14:11:22,834][86122] Updated weights for policy 1, policy_version 50190 (0.0007) +[2023-10-09 14:11:22,871][86121] Updated weights for policy 0, policy_version 49980 (0.0008) +[2023-10-09 14:11:23,194][86122] Updated weights for policy 1, policy_version 50200 (0.0009) +[2023-10-09 14:11:23,398][85186] Fps is (10 sec: 16383.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 102563840. Throughput: 0: 1820.4, 1: 1821.3. Samples: 25652930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:23,399][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:11:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000049984_51183616.pth... +[2023-10-09 14:11:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000048288_49446912.pth +[2023-10-09 14:11:23,484][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000050208_51412992.pth... +[2023-10-09 14:11:23,514][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000048480_49643520.pth +[2023-10-09 14:11:26,594][86121] Updated weights for policy 0, policy_version 49990 (0.0008) +[2023-10-09 14:11:26,891][86122] Updated weights for policy 1, policy_version 50210 (0.0008) +[2023-10-09 14:11:26,957][86121] Updated weights for policy 0, policy_version 50000 (0.0008) +[2023-10-09 14:11:27,258][86122] Updated weights for policy 1, policy_version 50220 (0.0008) +[2023-10-09 14:11:27,322][86121] Updated weights for policy 0, policy_version 50010 (0.0008) +[2023-10-09 14:11:27,628][86122] Updated weights for policy 1, policy_version 50230 (0.0007) +[2023-10-09 14:11:27,984][86122] Updated weights for policy 1, policy_version 50240 (0.0007) +[2023-10-09 14:11:28,397][85186] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 102662144. Throughput: 0: 1826.7, 1: 1816.0. Samples: 25664544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:11:31,074][86121] Updated weights for policy 0, policy_version 50020 (0.0008) +[2023-10-09 14:11:31,435][86121] Updated weights for policy 0, policy_version 50030 (0.0007) +[2023-10-09 14:11:31,596][86122] Updated weights for policy 1, policy_version 50250 (0.0007) +[2023-10-09 14:11:31,798][86121] Updated weights for policy 0, policy_version 50040 (0.0007) +[2023-10-09 14:11:31,953][86122] Updated weights for policy 1, policy_version 50260 (0.0008) +[2023-10-09 14:11:32,309][86122] Updated weights for policy 1, policy_version 50270 (0.0007) +[2023-10-09 14:11:33,397][85186] Fps is (10 sec: 16384.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 102727680. Throughput: 0: 1825.5, 1: 1821.3. Samples: 25685690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 14:11:35,493][86121] Updated weights for policy 0, policy_version 50050 (0.0008) +[2023-10-09 14:11:35,849][86121] Updated weights for policy 0, policy_version 50060 (0.0010) +[2023-10-09 14:11:36,112][86122] Updated weights for policy 1, policy_version 50280 (0.0007) +[2023-10-09 14:11:36,211][86121] Updated weights for policy 0, policy_version 50070 (0.0007) +[2023-10-09 14:11:36,475][86122] Updated weights for policy 1, policy_version 50290 (0.0008) +[2023-10-09 14:11:36,570][86121] Updated weights for policy 0, policy_version 50080 (0.0007) +[2023-10-09 14:11:36,843][86122] Updated weights for policy 1, policy_version 50300 (0.0009) +[2023-10-09 14:11:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 102793216. Throughput: 0: 1827.7, 1: 1814.3. Samples: 25707134. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 14:11:40,338][86121] Updated weights for policy 0, policy_version 50090 (0.0009) +[2023-10-09 14:11:40,556][86122] Updated weights for policy 1, policy_version 50310 (0.0009) +[2023-10-09 14:11:40,697][86121] Updated weights for policy 0, policy_version 50100 (0.0010) +[2023-10-09 14:11:40,919][86122] Updated weights for policy 1, policy_version 50320 (0.0009) +[2023-10-09 14:11:41,070][86121] Updated weights for policy 0, policy_version 50110 (0.0009) +[2023-10-09 14:11:41,272][86122] Updated weights for policy 1, policy_version 50330 (0.0009) +[2023-10-09 14:11:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102858752. Throughput: 0: 1823.2, 1: 1811.6. Samples: 25718078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:11:44,794][86121] Updated weights for policy 0, policy_version 50120 (0.0008) +[2023-10-09 14:11:44,804][86122] Updated weights for policy 1, policy_version 50340 (0.0008) +[2023-10-09 14:11:45,156][86121] Updated weights for policy 0, policy_version 50130 (0.0008) +[2023-10-09 14:11:45,167][86122] Updated weights for policy 1, policy_version 50350 (0.0009) +[2023-10-09 14:11:45,520][86121] Updated weights for policy 0, policy_version 50140 (0.0009) +[2023-10-09 14:11:45,526][86122] Updated weights for policy 1, policy_version 50360 (0.0011) +[2023-10-09 14:11:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 102924288. Throughput: 0: 1822.2, 1: 1810.4. Samples: 25739928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 14:11:49,278][86122] Updated weights for policy 1, policy_version 50370 (0.0008) +[2023-10-09 14:11:49,309][86121] Updated weights for policy 0, policy_version 50150 (0.0008) +[2023-10-09 14:11:49,636][86122] Updated weights for policy 1, policy_version 50380 (0.0008) +[2023-10-09 14:11:49,689][86121] Updated weights for policy 0, policy_version 50160 (0.0008) +[2023-10-09 14:11:50,003][86122] Updated weights for policy 1, policy_version 50390 (0.0009) +[2023-10-09 14:11:50,061][86121] Updated weights for policy 0, policy_version 50170 (0.0008) +[2023-10-09 14:11:50,358][86122] Updated weights for policy 1, policy_version 50400 (0.0009) +[2023-10-09 14:11:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 102989824. Throughput: 0: 1820.3, 1: 1809.8. Samples: 25762666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 14:11:53,552][86121] Updated weights for policy 0, policy_version 50180 (0.0007) +[2023-10-09 14:11:53,923][86121] Updated weights for policy 0, policy_version 50190 (0.0009) +[2023-10-09 14:11:54,165][86122] Updated weights for policy 1, policy_version 50410 (0.0007) +[2023-10-09 14:11:54,286][86121] Updated weights for policy 0, policy_version 50200 (0.0008) +[2023-10-09 14:11:54,536][86122] Updated weights for policy 1, policy_version 50420 (0.0008) +[2023-10-09 14:11:54,888][86122] Updated weights for policy 1, policy_version 50430 (0.0007) +[2023-10-09 14:11:57,960][86121] Updated weights for policy 0, policy_version 50210 (0.0008) +[2023-10-09 14:11:58,332][86121] Updated weights for policy 0, policy_version 50220 (0.0009) +[2023-10-09 14:11:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 103055360. Throughput: 0: 1825.4, 1: 1809.4. Samples: 25772722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:11:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 14:11:58,534][86122] Updated weights for policy 1, policy_version 50440 (0.0007) +[2023-10-09 14:11:58,703][86121] Updated weights for policy 0, policy_version 50230 (0.0009) +[2023-10-09 14:11:58,889][86122] Updated weights for policy 1, policy_version 50450 (0.0008) +[2023-10-09 14:11:59,067][86121] Updated weights for policy 0, policy_version 50240 (0.0008) +[2023-10-09 14:11:59,247][86122] Updated weights for policy 1, policy_version 50460 (0.0007) +[2023-10-09 14:12:02,826][86121] Updated weights for policy 0, policy_version 50250 (0.0007) +[2023-10-09 14:12:03,019][86122] Updated weights for policy 1, policy_version 50470 (0.0007) +[2023-10-09 14:12:03,201][86121] Updated weights for policy 0, policy_version 50260 (0.0010) +[2023-10-09 14:12:03,391][86122] Updated weights for policy 1, policy_version 50480 (0.0007) +[2023-10-09 14:12:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103120896. Throughput: 0: 1816.0, 1: 1820.6. Samples: 25795668. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:12:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 14:12:03,580][86121] Updated weights for policy 0, policy_version 50270 (0.0007) +[2023-10-09 14:12:03,750][86122] Updated weights for policy 1, policy_version 50490 (0.0008) +[2023-10-09 14:12:07,304][86121] Updated weights for policy 0, policy_version 50280 (0.0007) +[2023-10-09 14:12:07,565][86122] Updated weights for policy 1, policy_version 50500 (0.0007) +[2023-10-09 14:12:07,673][86121] Updated weights for policy 0, policy_version 50290 (0.0007) +[2023-10-09 14:12:07,966][86122] Updated weights for policy 1, policy_version 50510 (0.0008) +[2023-10-09 14:12:08,050][86121] Updated weights for policy 0, policy_version 50300 (0.0010) +[2023-10-09 14:12:08,331][86122] Updated weights for policy 1, policy_version 50520 (0.0007) +[2023-10-09 14:12:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 103219200. Throughput: 0: 1816.4, 1: 1824.6. Samples: 25816776. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:12:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 14:12:11,856][86121] Updated weights for policy 0, policy_version 50310 (0.0010) +[2023-10-09 14:12:12,035][86122] Updated weights for policy 1, policy_version 50530 (0.0008) +[2023-10-09 14:12:12,219][86121] Updated weights for policy 0, policy_version 50320 (0.0007) +[2023-10-09 14:12:12,392][86122] Updated weights for policy 1, policy_version 50540 (0.0008) +[2023-10-09 14:12:12,591][86121] Updated weights for policy 0, policy_version 50330 (0.0007) +[2023-10-09 14:12:12,750][86122] Updated weights for policy 1, policy_version 50550 (0.0009) +[2023-10-09 14:12:13,113][86122] Updated weights for policy 1, policy_version 50560 (0.0010) +[2023-10-09 14:12:13,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103317504. Throughput: 0: 1808.6, 1: 1822.3. Samples: 25827932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:12:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 14:12:16,248][86121] Updated weights for policy 0, policy_version 50340 (0.0009) +[2023-10-09 14:12:16,622][86121] Updated weights for policy 0, policy_version 50350 (0.0007) +[2023-10-09 14:12:16,691][86122] Updated weights for policy 1, policy_version 50570 (0.0008) +[2023-10-09 14:12:16,988][86121] Updated weights for policy 0, policy_version 50360 (0.0007) +[2023-10-09 14:12:17,051][86122] Updated weights for policy 1, policy_version 50580 (0.0008) +[2023-10-09 14:12:17,424][86122] Updated weights for policy 1, policy_version 50590 (0.0009) +[2023-10-09 14:12:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 103383040. Throughput: 0: 1813.6, 1: 1820.4. Samples: 25849216. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:12:20,864][86121] Updated weights for policy 0, policy_version 50370 (0.0010) +[2023-10-09 14:12:21,227][86122] Updated weights for policy 1, policy_version 50600 (0.0009) +[2023-10-09 14:12:21,231][86121] Updated weights for policy 0, policy_version 50380 (0.0008) +[2023-10-09 14:12:21,587][86122] Updated weights for policy 1, policy_version 50610 (0.0009) +[2023-10-09 14:12:21,597][86121] Updated weights for policy 0, policy_version 50390 (0.0008) +[2023-10-09 14:12:21,947][86122] Updated weights for policy 1, policy_version 50620 (0.0009) +[2023-10-09 14:12:21,958][86121] Updated weights for policy 0, policy_version 50400 (0.0008) +[2023-10-09 14:12:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 103448576. Throughput: 0: 1809.7, 1: 1824.1. Samples: 25870656. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:23,399][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:12:25,489][86121] Updated weights for policy 0, policy_version 50410 (0.0008) +[2023-10-09 14:12:25,670][86122] Updated weights for policy 1, policy_version 50630 (0.0007) +[2023-10-09 14:12:25,861][86121] Updated weights for policy 0, policy_version 50420 (0.0007) +[2023-10-09 14:12:26,023][86122] Updated weights for policy 1, policy_version 50640 (0.0007) +[2023-10-09 14:12:26,220][86121] Updated weights for policy 0, policy_version 50430 (0.0007) +[2023-10-09 14:12:26,385][86122] Updated weights for policy 1, policy_version 50650 (0.0008) +[2023-10-09 14:12:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 103514112. Throughput: 0: 1818.7, 1: 1829.5. Samples: 25882244. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:12:29,972][86121] Updated weights for policy 0, policy_version 50440 (0.0008) +[2023-10-09 14:12:30,029][86122] Updated weights for policy 1, policy_version 50660 (0.0010) +[2023-10-09 14:12:30,339][86121] Updated weights for policy 0, policy_version 50450 (0.0008) +[2023-10-09 14:12:30,391][86122] Updated weights for policy 1, policy_version 50670 (0.0007) +[2023-10-09 14:12:30,703][86121] Updated weights for policy 0, policy_version 50460 (0.0009) +[2023-10-09 14:12:30,755][86122] Updated weights for policy 1, policy_version 50680 (0.0008) +[2023-10-09 14:12:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 103579648. Throughput: 0: 1809.1, 1: 1824.6. Samples: 25903446. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:12:34,291][86122] Updated weights for policy 1, policy_version 50690 (0.0009) +[2023-10-09 14:12:34,505][86121] Updated weights for policy 0, policy_version 50470 (0.0008) +[2023-10-09 14:12:34,655][86122] Updated weights for policy 1, policy_version 50700 (0.0007) +[2023-10-09 14:12:34,876][86121] Updated weights for policy 0, policy_version 50480 (0.0007) +[2023-10-09 14:12:35,019][86122] Updated weights for policy 1, policy_version 50710 (0.0007) +[2023-10-09 14:12:35,235][86121] Updated weights for policy 0, policy_version 50490 (0.0011) +[2023-10-09 14:12:35,379][86122] Updated weights for policy 1, policy_version 50720 (0.0008) +[2023-10-09 14:12:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 103645184. Throughput: 0: 1807.3, 1: 1828.2. Samples: 25926264. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 14:12:38,828][86121] Updated weights for policy 0, policy_version 50500 (0.0008) +[2023-10-09 14:12:39,189][86122] Updated weights for policy 1, policy_version 50730 (0.0007) +[2023-10-09 14:12:39,198][86121] Updated weights for policy 0, policy_version 50510 (0.0008) +[2023-10-09 14:12:39,547][86122] Updated weights for policy 1, policy_version 50740 (0.0007) +[2023-10-09 14:12:39,561][86121] Updated weights for policy 0, policy_version 50520 (0.0008) +[2023-10-09 14:12:39,911][86122] Updated weights for policy 1, policy_version 50750 (0.0007) +[2023-10-09 14:12:43,389][86121] Updated weights for policy 0, policy_version 50530 (0.0007) +[2023-10-09 14:12:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 103710720. Throughput: 0: 1801.7, 1: 1826.1. Samples: 25935974. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 14:12:43,666][86122] Updated weights for policy 1, policy_version 50760 (0.0007) +[2023-10-09 14:12:43,750][86121] Updated weights for policy 0, policy_version 50540 (0.0008) +[2023-10-09 14:12:44,029][86122] Updated weights for policy 1, policy_version 50770 (0.0007) +[2023-10-09 14:12:44,124][86121] Updated weights for policy 0, policy_version 50550 (0.0008) +[2023-10-09 14:12:44,386][86122] Updated weights for policy 1, policy_version 50780 (0.0008) +[2023-10-09 14:12:44,491][86121] Updated weights for policy 0, policy_version 50560 (0.0007) +[2023-10-09 14:12:48,013][86122] Updated weights for policy 1, policy_version 50790 (0.0008) +[2023-10-09 14:12:48,268][86121] Updated weights for policy 0, policy_version 50570 (0.0009) +[2023-10-09 14:12:48,376][86122] Updated weights for policy 1, policy_version 50800 (0.0009) +[2023-10-09 14:12:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 103776256. Throughput: 0: 1806.7, 1: 1818.3. Samples: 25958792. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 14:12:48,640][86121] Updated weights for policy 0, policy_version 50580 (0.0008) +[2023-10-09 14:12:48,737][86122] Updated weights for policy 1, policy_version 50810 (0.0008) +[2023-10-09 14:12:48,998][86121] Updated weights for policy 0, policy_version 50590 (0.0010) +[2023-10-09 14:12:52,608][86122] Updated weights for policy 1, policy_version 50820 (0.0007) +[2023-10-09 14:12:52,618][86121] Updated weights for policy 0, policy_version 50600 (0.0008) +[2023-10-09 14:12:52,987][86121] Updated weights for policy 0, policy_version 50610 (0.0009) +[2023-10-09 14:12:52,991][86122] Updated weights for policy 1, policy_version 50830 (0.0007) +[2023-10-09 14:12:53,350][86121] Updated weights for policy 0, policy_version 50620 (0.0007) +[2023-10-09 14:12:53,356][86122] Updated weights for policy 1, policy_version 50840 (0.0010) +[2023-10-09 14:12:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 103841792. Throughput: 0: 1814.1, 1: 1819.9. Samples: 25980310. Policy #0 lag: (min: 14.0, avg: 14.0, max: 14.0) +[2023-10-09 14:12:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:12:56,880][86122] Updated weights for policy 1, policy_version 50850 (0.0008) +[2023-10-09 14:12:57,021][86121] Updated weights for policy 0, policy_version 50630 (0.0007) +[2023-10-09 14:12:57,246][86122] Updated weights for policy 1, policy_version 50860 (0.0007) +[2023-10-09 14:12:57,393][86121] Updated weights for policy 0, policy_version 50640 (0.0009) +[2023-10-09 14:12:57,605][86122] Updated weights for policy 1, policy_version 50870 (0.0008) +[2023-10-09 14:12:57,749][86121] Updated weights for policy 0, policy_version 50650 (0.0007) +[2023-10-09 14:12:57,964][86122] Updated weights for policy 1, policy_version 50880 (0.0007) +[2023-10-09 14:12:58,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 103972864. Throughput: 0: 1809.0, 1: 1823.5. Samples: 25991396. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) +[2023-10-09 14:12:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:13:01,456][86121] Updated weights for policy 0, policy_version 50660 (0.0007) +[2023-10-09 14:13:01,656][86122] Updated weights for policy 1, policy_version 50890 (0.0008) +[2023-10-09 14:13:01,828][86121] Updated weights for policy 0, policy_version 50670 (0.0007) +[2023-10-09 14:13:02,010][86122] Updated weights for policy 1, policy_version 50900 (0.0008) +[2023-10-09 14:13:02,198][86121] Updated weights for policy 0, policy_version 50680 (0.0008) +[2023-10-09 14:13:02,377][86122] Updated weights for policy 1, policy_version 50910 (0.0009) +[2023-10-09 14:13:03,397][85186] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 104038400. Throughput: 0: 1815.0, 1: 1817.7. Samples: 26012688. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) +[2023-10-09 14:13:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:13:05,884][86121] Updated weights for policy 0, policy_version 50690 (0.0008) +[2023-10-09 14:13:06,202][86122] Updated weights for policy 1, policy_version 50920 (0.0009) +[2023-10-09 14:13:06,240][86121] Updated weights for policy 0, policy_version 50700 (0.0010) +[2023-10-09 14:13:06,556][86122] Updated weights for policy 1, policy_version 50930 (0.0007) +[2023-10-09 14:13:06,599][86121] Updated weights for policy 0, policy_version 50710 (0.0009) +[2023-10-09 14:13:06,917][86122] Updated weights for policy 1, policy_version 50940 (0.0007) +[2023-10-09 14:13:06,968][86121] Updated weights for policy 0, policy_version 50720 (0.0010) +[2023-10-09 14:13:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 104103936. Throughput: 0: 1807.7, 1: 1817.9. Samples: 26033808. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) +[2023-10-09 14:13:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 14:13:10,569][86122] Updated weights for policy 1, policy_version 50950 (0.0009) +[2023-10-09 14:13:10,792][86121] Updated weights for policy 0, policy_version 50730 (0.0007) +[2023-10-09 14:13:10,927][86122] Updated weights for policy 1, policy_version 50960 (0.0007) +[2023-10-09 14:13:11,163][86121] Updated weights for policy 0, policy_version 50740 (0.0007) +[2023-10-09 14:13:11,288][86122] Updated weights for policy 1, policy_version 50970 (0.0009) +[2023-10-09 14:13:11,523][86121] Updated weights for policy 0, policy_version 50750 (0.0007) +[2023-10-09 14:13:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 104169472. Throughput: 0: 1811.6, 1: 1814.6. Samples: 26045420. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) +[2023-10-09 14:13:13,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 14:13:15,137][86122] Updated weights for policy 1, policy_version 50980 (0.0008) +[2023-10-09 14:13:15,193][86121] Updated weights for policy 0, policy_version 50760 (0.0009) +[2023-10-09 14:13:15,492][86122] Updated weights for policy 1, policy_version 50990 (0.0008) +[2023-10-09 14:13:15,557][86121] Updated weights for policy 0, policy_version 50770 (0.0008) +[2023-10-09 14:13:15,864][86122] Updated weights for policy 1, policy_version 51000 (0.0008) +[2023-10-09 14:13:15,921][86121] Updated weights for policy 0, policy_version 50780 (0.0009) +[2023-10-09 14:13:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 104235008. Throughput: 0: 1808.1, 1: 1808.1. Samples: 26066174. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) +[2023-10-09 14:13:18,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 14:13:19,663][86122] Updated weights for policy 1, policy_version 51010 (0.0011) +[2023-10-09 14:13:19,695][86121] Updated weights for policy 0, policy_version 50790 (0.0007) +[2023-10-09 14:13:20,022][86122] Updated weights for policy 1, policy_version 51020 (0.0009) +[2023-10-09 14:13:20,068][86121] Updated weights for policy 0, policy_version 50800 (0.0008) +[2023-10-09 14:13:20,385][86122] Updated weights for policy 1, policy_version 51030 (0.0009) +[2023-10-09 14:13:20,430][86121] Updated weights for policy 0, policy_version 50810 (0.0008) +[2023-10-09 14:13:20,749][86122] Updated weights for policy 1, policy_version 51040 (0.0010) +[2023-10-09 14:13:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 104300544. Throughput: 0: 1812.7, 1: 1798.4. Samples: 26088766. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) +[2023-10-09 14:13:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:13:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000051040_52264960.pth... +[2023-10-09 14:13:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000050816_52035584.pth... +[2023-10-09 14:13:23,437][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000049344_50528256.pth +[2023-10-09 14:13:23,441][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000051040_52264960.pth +[2023-10-09 14:13:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000049120_50298880.pth +[2023-10-09 14:13:23,451][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000050816_52035584.pth +[2023-10-09 14:13:24,066][86121] Updated weights for policy 0, policy_version 50820 (0.0007) +[2023-10-09 14:13:24,446][86121] Updated weights for policy 0, policy_version 50830 (0.0008) +[2023-10-09 14:13:24,456][86122] Updated weights for policy 1, policy_version 51050 (0.0007) +[2023-10-09 14:13:24,810][86121] Updated weights for policy 0, policy_version 50840 (0.0008) +[2023-10-09 14:13:24,818][86122] Updated weights for policy 1, policy_version 51060 (0.0007) +[2023-10-09 14:13:25,186][86122] Updated weights for policy 1, policy_version 51070 (0.0009) +[2023-10-09 14:13:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104366080. Throughput: 0: 1811.9, 1: 1798.8. Samples: 26098454. Policy #0 lag: (min: 31.0, avg: 32.4, max: 57.0) +[2023-10-09 14:13:28,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 14:13:28,503][86121] Updated weights for policy 0, policy_version 50850 (0.0010) +[2023-10-09 14:13:28,864][86121] Updated weights for policy 0, policy_version 50860 (0.0008) +[2023-10-09 14:13:29,026][86122] Updated weights for policy 1, policy_version 51080 (0.0008) +[2023-10-09 14:13:29,230][86121] Updated weights for policy 0, policy_version 50870 (0.0007) +[2023-10-09 14:13:29,387][86122] Updated weights for policy 1, policy_version 51090 (0.0007) +[2023-10-09 14:13:29,599][86121] Updated weights for policy 0, policy_version 50880 (0.0008) +[2023-10-09 14:13:29,748][86122] Updated weights for policy 1, policy_version 51100 (0.0009) +[2023-10-09 14:13:33,121][86121] Updated weights for policy 0, policy_version 50890 (0.0008) +[2023-10-09 14:13:33,325][86122] Updated weights for policy 1, policy_version 51110 (0.0008) +[2023-10-09 14:13:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104431616. Throughput: 0: 1817.7, 1: 1794.7. Samples: 26121348. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) +[2023-10-09 14:13:33,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:13:33,483][86121] Updated weights for policy 0, policy_version 50900 (0.0008) +[2023-10-09 14:13:33,688][86122] Updated weights for policy 1, policy_version 51120 (0.0008) +[2023-10-09 14:13:33,844][86121] Updated weights for policy 0, policy_version 50910 (0.0008) +[2023-10-09 14:13:34,048][86122] Updated weights for policy 1, policy_version 51130 (0.0008) +[2023-10-09 14:13:37,571][86121] Updated weights for policy 0, policy_version 50920 (0.0010) +[2023-10-09 14:13:37,739][86122] Updated weights for policy 1, policy_version 51140 (0.0009) +[2023-10-09 14:13:37,935][86121] Updated weights for policy 0, policy_version 50930 (0.0008) +[2023-10-09 14:13:38,111][86122] Updated weights for policy 1, policy_version 51150 (0.0008) +[2023-10-09 14:13:38,295][86121] Updated weights for policy 0, policy_version 50940 (0.0009) +[2023-10-09 14:13:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104497152. Throughput: 0: 1818.9, 1: 1803.7. Samples: 26143328. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) +[2023-10-09 14:13:38,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:13:38,470][86122] Updated weights for policy 1, policy_version 51160 (0.0007) +[2023-10-09 14:13:42,110][86121] Updated weights for policy 0, policy_version 50950 (0.0008) +[2023-10-09 14:13:42,153][86122] Updated weights for policy 1, policy_version 51170 (0.0009) +[2023-10-09 14:13:42,478][86121] Updated weights for policy 0, policy_version 50960 (0.0009) +[2023-10-09 14:13:42,513][86122] Updated weights for policy 1, policy_version 51180 (0.0007) +[2023-10-09 14:13:42,848][86121] Updated weights for policy 0, policy_version 50970 (0.0008) +[2023-10-09 14:13:42,877][86122] Updated weights for policy 1, policy_version 51190 (0.0008) +[2023-10-09 14:13:43,235][86122] Updated weights for policy 1, policy_version 51200 (0.0007) +[2023-10-09 14:13:43,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 104628224. Throughput: 0: 1821.8, 1: 1797.3. Samples: 26154258. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) +[2023-10-09 14:13:43,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:13:46,572][86121] Updated weights for policy 0, policy_version 50980 (0.0008) +[2023-10-09 14:13:46,935][86121] Updated weights for policy 0, policy_version 50990 (0.0008) +[2023-10-09 14:13:47,073][86122] Updated weights for policy 1, policy_version 51210 (0.0009) +[2023-10-09 14:13:47,305][86121] Updated weights for policy 0, policy_version 51000 (0.0009) +[2023-10-09 14:13:47,432][86122] Updated weights for policy 1, policy_version 51220 (0.0008) +[2023-10-09 14:13:47,801][86122] Updated weights for policy 1, policy_version 51230 (0.0008) +[2023-10-09 14:13:48,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 104693760. Throughput: 0: 1825.3, 1: 1807.3. Samples: 26176154. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) +[2023-10-09 14:13:48,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:13:50,986][86121] Updated weights for policy 0, policy_version 51010 (0.0010) +[2023-10-09 14:13:51,357][86121] Updated weights for policy 0, policy_version 51020 (0.0008) +[2023-10-09 14:13:51,469][86122] Updated weights for policy 1, policy_version 51240 (0.0008) +[2023-10-09 14:13:51,725][86121] Updated weights for policy 0, policy_version 51030 (0.0009) +[2023-10-09 14:13:51,838][86122] Updated weights for policy 1, policy_version 51250 (0.0008) +[2023-10-09 14:13:52,091][86121] Updated weights for policy 0, policy_version 51040 (0.0008) +[2023-10-09 14:13:52,201][86122] Updated weights for policy 1, policy_version 51260 (0.0007) +[2023-10-09 14:13:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 15291.8, 300 sec: 14551.2). Total num frames: 104759296. Throughput: 0: 1824.3, 1: 1796.4. Samples: 26196740. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) +[2023-10-09 14:13:53,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:13:55,808][86121] Updated weights for policy 0, policy_version 51050 (0.0008) +[2023-10-09 14:13:55,888][86122] Updated weights for policy 1, policy_version 51270 (0.0008) +[2023-10-09 14:13:56,162][86121] Updated weights for policy 0, policy_version 51060 (0.0008) +[2023-10-09 14:13:56,245][86122] Updated weights for policy 1, policy_version 51280 (0.0008) +[2023-10-09 14:13:56,523][86121] Updated weights for policy 0, policy_version 51070 (0.0008) +[2023-10-09 14:13:56,601][86122] Updated weights for policy 1, policy_version 51290 (0.0008) +[2023-10-09 14:13:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 104824832. Throughput: 0: 1826.4, 1: 1806.7. Samples: 26208912. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) +[2023-10-09 14:13:58,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:14:00,137][86121] Updated weights for policy 0, policy_version 51080 (0.0008) +[2023-10-09 14:14:00,295][86122] Updated weights for policy 1, policy_version 51300 (0.0008) +[2023-10-09 14:14:00,512][86121] Updated weights for policy 0, policy_version 51090 (0.0008) +[2023-10-09 14:14:00,657][86122] Updated weights for policy 1, policy_version 51310 (0.0008) +[2023-10-09 14:14:00,880][86121] Updated weights for policy 0, policy_version 51100 (0.0009) +[2023-10-09 14:14:01,013][86122] Updated weights for policy 1, policy_version 51320 (0.0009) +[2023-10-09 14:14:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 104890368. Throughput: 0: 1829.8, 1: 1803.3. Samples: 26229664. Policy #0 lag: (min: 28.0, avg: 35.8, max: 60.0) +[2023-10-09 14:14:03,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:04,521][86121] Updated weights for policy 0, policy_version 51110 (0.0008) +[2023-10-09 14:14:04,611][86122] Updated weights for policy 1, policy_version 51330 (0.0010) +[2023-10-09 14:14:04,896][86121] Updated weights for policy 0, policy_version 51120 (0.0008) +[2023-10-09 14:14:04,983][86122] Updated weights for policy 1, policy_version 51340 (0.0007) +[2023-10-09 14:14:05,263][86121] Updated weights for policy 0, policy_version 51130 (0.0008) +[2023-10-09 14:14:05,340][86122] Updated weights for policy 1, policy_version 51350 (0.0009) +[2023-10-09 14:14:05,695][86122] Updated weights for policy 1, policy_version 51360 (0.0008) +[2023-10-09 14:14:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 104955904. Throughput: 0: 1827.9, 1: 1813.4. Samples: 26252624. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) +[2023-10-09 14:14:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:09,044][86121] Updated weights for policy 0, policy_version 51140 (0.0008) +[2023-10-09 14:14:09,295][86122] Updated weights for policy 1, policy_version 51370 (0.0007) +[2023-10-09 14:14:09,410][86121] Updated weights for policy 0, policy_version 51150 (0.0008) +[2023-10-09 14:14:09,658][86122] Updated weights for policy 1, policy_version 51380 (0.0008) +[2023-10-09 14:14:09,776][86121] Updated weights for policy 0, policy_version 51160 (0.0008) +[2023-10-09 14:14:10,015][86122] Updated weights for policy 1, policy_version 51390 (0.0007) +[2023-10-09 14:14:13,362][86121] Updated weights for policy 0, policy_version 51170 (0.0007) +[2023-10-09 14:14:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105021440. Throughput: 0: 1830.2, 1: 1816.2. Samples: 26262542. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) +[2023-10-09 14:14:13,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:13,730][86121] Updated weights for policy 0, policy_version 51180 (0.0008) +[2023-10-09 14:14:13,925][86122] Updated weights for policy 1, policy_version 51400 (0.0008) +[2023-10-09 14:14:14,094][86121] Updated weights for policy 0, policy_version 51190 (0.0008) +[2023-10-09 14:14:14,288][86122] Updated weights for policy 1, policy_version 51410 (0.0007) +[2023-10-09 14:14:14,453][86121] Updated weights for policy 0, policy_version 51200 (0.0008) +[2023-10-09 14:14:14,647][86122] Updated weights for policy 1, policy_version 51420 (0.0011) +[2023-10-09 14:14:18,098][86121] Updated weights for policy 0, policy_version 51210 (0.0012) +[2023-10-09 14:14:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 105086976. Throughput: 0: 1824.9, 1: 1816.3. Samples: 26285202. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) +[2023-10-09 14:14:18,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:14:18,465][86121] Updated weights for policy 0, policy_version 51220 (0.0008) +[2023-10-09 14:14:18,468][86122] Updated weights for policy 1, policy_version 51430 (0.0009) +[2023-10-09 14:14:18,825][86121] Updated weights for policy 0, policy_version 51230 (0.0007) +[2023-10-09 14:14:18,829][86122] Updated weights for policy 1, policy_version 51440 (0.0009) +[2023-10-09 14:14:19,180][86122] Updated weights for policy 1, policy_version 51450 (0.0009) +[2023-10-09 14:14:22,537][86121] Updated weights for policy 0, policy_version 51240 (0.0008) +[2023-10-09 14:14:22,894][86122] Updated weights for policy 1, policy_version 51460 (0.0007) +[2023-10-09 14:14:22,905][86121] Updated weights for policy 0, policy_version 51250 (0.0008) +[2023-10-09 14:14:23,272][86121] Updated weights for policy 0, policy_version 51260 (0.0008) +[2023-10-09 14:14:23,277][86122] Updated weights for policy 1, policy_version 51470 (0.0007) +[2023-10-09 14:14:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105152512. Throughput: 0: 1817.5, 1: 1819.9. Samples: 26307014. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) +[2023-10-09 14:14:23,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:23,641][86122] Updated weights for policy 1, policy_version 51480 (0.0007) +[2023-10-09 14:14:27,058][86121] Updated weights for policy 0, policy_version 51270 (0.0009) +[2023-10-09 14:14:27,323][86122] Updated weights for policy 1, policy_version 51490 (0.0009) +[2023-10-09 14:14:27,413][86121] Updated weights for policy 0, policy_version 51280 (0.0008) +[2023-10-09 14:14:27,694][86122] Updated weights for policy 1, policy_version 51500 (0.0009) +[2023-10-09 14:14:27,775][86121] Updated weights for policy 0, policy_version 51290 (0.0008) +[2023-10-09 14:14:28,048][86122] Updated weights for policy 1, policy_version 51510 (0.0009) +[2023-10-09 14:14:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 105250816. Throughput: 0: 1816.0, 1: 1812.6. Samples: 26317546. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) +[2023-10-09 14:14:28,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:28,406][86122] Updated weights for policy 1, policy_version 51520 (0.0011) +[2023-10-09 14:14:31,598][86121] Updated weights for policy 0, policy_version 51300 (0.0009) +[2023-10-09 14:14:31,966][86121] Updated weights for policy 0, policy_version 51310 (0.0008) +[2023-10-09 14:14:32,243][86122] Updated weights for policy 1, policy_version 51530 (0.0009) +[2023-10-09 14:14:32,332][86121] Updated weights for policy 0, policy_version 51320 (0.0010) +[2023-10-09 14:14:32,607][86122] Updated weights for policy 1, policy_version 51540 (0.0007) +[2023-10-09 14:14:32,966][86122] Updated weights for policy 1, policy_version 51550 (0.0009) +[2023-10-09 14:14:33,397][85186] Fps is (10 sec: 19661.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 105349120. Throughput: 0: 1818.2, 1: 1817.8. Samples: 26339774. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) +[2023-10-09 14:14:33,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:14:36,126][86121] Updated weights for policy 0, policy_version 51330 (0.0007) +[2023-10-09 14:14:36,488][86121] Updated weights for policy 0, policy_version 51340 (0.0010) +[2023-10-09 14:14:36,721][86122] Updated weights for policy 1, policy_version 51560 (0.0007) +[2023-10-09 14:14:36,851][86121] Updated weights for policy 0, policy_version 51350 (0.0007) +[2023-10-09 14:14:37,078][86122] Updated weights for policy 1, policy_version 51570 (0.0007) +[2023-10-09 14:14:37,215][86121] Updated weights for policy 0, policy_version 51360 (0.0008) +[2023-10-09 14:14:37,442][86122] Updated weights for policy 1, policy_version 51580 (0.0007) +[2023-10-09 14:14:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 105414656. Throughput: 0: 1814.0, 1: 1815.3. Samples: 26360060. Policy #0 lag: (min: 25.0, avg: 32.4, max: 57.0) +[2023-10-09 14:14:38,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:40,950][86121] Updated weights for policy 0, policy_version 51370 (0.0008) +[2023-10-09 14:14:41,015][86122] Updated weights for policy 1, policy_version 51590 (0.0008) +[2023-10-09 14:14:41,328][86121] Updated weights for policy 0, policy_version 51380 (0.0007) +[2023-10-09 14:14:41,365][86122] Updated weights for policy 1, policy_version 51600 (0.0008) +[2023-10-09 14:14:41,686][86121] Updated weights for policy 0, policy_version 51390 (0.0009) +[2023-10-09 14:14:41,726][86122] Updated weights for policy 1, policy_version 51610 (0.0009) +[2023-10-09 14:14:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 105480192. Throughput: 0: 1811.3, 1: 1820.4. Samples: 26372336. Policy #0 lag: (min: 9.0, avg: 12.4, max: 41.0) +[2023-10-09 14:14:43,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:45,313][86122] Updated weights for policy 1, policy_version 51620 (0.0008) +[2023-10-09 14:14:45,334][86121] Updated weights for policy 0, policy_version 51400 (0.0008) +[2023-10-09 14:14:45,681][86122] Updated weights for policy 1, policy_version 51630 (0.0010) +[2023-10-09 14:14:45,705][86121] Updated weights for policy 0, policy_version 51410 (0.0008) +[2023-10-09 14:14:46,037][86122] Updated weights for policy 1, policy_version 51640 (0.0007) +[2023-10-09 14:14:46,059][86121] Updated weights for policy 0, policy_version 51420 (0.0008) +[2023-10-09 14:14:48,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105545728. Throughput: 0: 1807.2, 1: 1823.4. Samples: 26393038. Policy #0 lag: (min: 9.0, avg: 12.4, max: 41.0) +[2023-10-09 14:14:48,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:49,723][86122] Updated weights for policy 1, policy_version 51650 (0.0007) +[2023-10-09 14:14:49,932][86121] Updated weights for policy 0, policy_version 51430 (0.0008) +[2023-10-09 14:14:50,087][86122] Updated weights for policy 1, policy_version 51660 (0.0008) +[2023-10-09 14:14:50,317][86121] Updated weights for policy 0, policy_version 51440 (0.0009) +[2023-10-09 14:14:50,454][86122] Updated weights for policy 1, policy_version 51670 (0.0007) +[2023-10-09 14:14:50,691][86121] Updated weights for policy 0, policy_version 51450 (0.0008) +[2023-10-09 14:14:50,816][86122] Updated weights for policy 1, policy_version 51680 (0.0009) +[2023-10-09 14:14:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105611264. Throughput: 0: 1804.9, 1: 1813.3. Samples: 26415442. Policy #0 lag: (min: 9.0, avg: 12.4, max: 41.0) +[2023-10-09 14:14:53,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:14:54,346][86121] Updated weights for policy 0, policy_version 51460 (0.0010) +[2023-10-09 14:14:54,640][86122] Updated weights for policy 1, policy_version 51690 (0.0009) +[2023-10-09 14:14:54,712][86121] Updated weights for policy 0, policy_version 51470 (0.0007) +[2023-10-09 14:14:55,004][86122] Updated weights for policy 1, policy_version 51700 (0.0008) +[2023-10-09 14:14:55,075][86121] Updated weights for policy 0, policy_version 51480 (0.0007) +[2023-10-09 14:14:55,364][86122] Updated weights for policy 1, policy_version 51710 (0.0007) +[2023-10-09 14:14:58,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105676800. Throughput: 0: 1805.4, 1: 1811.2. Samples: 26425290. Policy #0 lag: (min: 9.0, avg: 12.4, max: 41.0) +[2023-10-09 14:14:58,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:14:58,826][86121] Updated weights for policy 0, policy_version 51490 (0.0008) +[2023-10-09 14:14:59,180][86122] Updated weights for policy 1, policy_version 51720 (0.0008) +[2023-10-09 14:14:59,188][86121] Updated weights for policy 0, policy_version 51500 (0.0009) +[2023-10-09 14:14:59,539][86122] Updated weights for policy 1, policy_version 51730 (0.0008) +[2023-10-09 14:14:59,547][86121] Updated weights for policy 0, policy_version 51510 (0.0008) +[2023-10-09 14:14:59,906][86122] Updated weights for policy 1, policy_version 51740 (0.0008) +[2023-10-09 14:14:59,914][86121] Updated weights for policy 0, policy_version 51520 (0.0008) +[2023-10-09 14:15:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 105742336. Throughput: 0: 1801.6, 1: 1810.3. Samples: 26447740. Policy #0 lag: (min: 9.0, avg: 12.4, max: 41.0) +[2023-10-09 14:15:03,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:15:03,450][86121] Updated weights for policy 0, policy_version 51530 (0.0008) +[2023-10-09 14:15:03,480][86122] Updated weights for policy 1, policy_version 51750 (0.0008) +[2023-10-09 14:15:03,812][86121] Updated weights for policy 0, policy_version 51540 (0.0008) +[2023-10-09 14:15:03,844][86122] Updated weights for policy 1, policy_version 51760 (0.0007) +[2023-10-09 14:15:04,187][86121] Updated weights for policy 0, policy_version 51550 (0.0008) +[2023-10-09 14:15:04,202][86122] Updated weights for policy 1, policy_version 51770 (0.0008) +[2023-10-09 14:15:07,886][86122] Updated weights for policy 1, policy_version 51780 (0.0007) +[2023-10-09 14:15:07,951][86121] Updated weights for policy 0, policy_version 51560 (0.0009) +[2023-10-09 14:15:08,273][86122] Updated weights for policy 1, policy_version 51790 (0.0008) +[2023-10-09 14:15:08,316][86121] Updated weights for policy 0, policy_version 51570 (0.0008) +[2023-10-09 14:15:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 105807872. Throughput: 0: 1817.4, 1: 1812.7. Samples: 26470368. Policy #0 lag: (min: 9.0, avg: 12.4, max: 41.0) +[2023-10-09 14:15:08,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:15:08,634][86122] Updated weights for policy 1, policy_version 51800 (0.0007) +[2023-10-09 14:15:08,682][86121] Updated weights for policy 0, policy_version 51580 (0.0007) +[2023-10-09 14:15:12,296][86122] Updated weights for policy 1, policy_version 51810 (0.0008) +[2023-10-09 14:15:12,514][86121] Updated weights for policy 0, policy_version 51590 (0.0010) +[2023-10-09 14:15:12,655][86122] Updated weights for policy 1, policy_version 51820 (0.0008) +[2023-10-09 14:15:12,876][86121] Updated weights for policy 0, policy_version 51600 (0.0008) +[2023-10-09 14:15:13,015][86122] Updated weights for policy 1, policy_version 51830 (0.0009) +[2023-10-09 14:15:13,242][86121] Updated weights for policy 0, policy_version 51610 (0.0007) +[2023-10-09 14:15:13,371][86122] Updated weights for policy 1, policy_version 51840 (0.0009) +[2023-10-09 14:15:13,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 105906176. Throughput: 0: 1807.0, 1: 1814.6. Samples: 26480520. Policy #0 lag: (min: 9.0, avg: 12.4, max: 41.0) +[2023-10-09 14:15:13,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:15:16,844][86121] Updated weights for policy 0, policy_version 51620 (0.0007) +[2023-10-09 14:15:17,212][86121] Updated weights for policy 0, policy_version 51630 (0.0008) +[2023-10-09 14:15:17,221][86122] Updated weights for policy 1, policy_version 51850 (0.0007) +[2023-10-09 14:15:17,573][86121] Updated weights for policy 0, policy_version 51640 (0.0008) +[2023-10-09 14:15:17,573][86122] Updated weights for policy 1, policy_version 51860 (0.0007) +[2023-10-09 14:15:17,939][86122] Updated weights for policy 1, policy_version 51870 (0.0009) +[2023-10-09 14:15:18,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106004480. Throughput: 0: 1805.1, 1: 1814.0. Samples: 26502632. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:15:21,394][86121] Updated weights for policy 0, policy_version 51650 (0.0009) +[2023-10-09 14:15:21,569][86122] Updated weights for policy 1, policy_version 51880 (0.0008) +[2023-10-09 14:15:21,761][86121] Updated weights for policy 0, policy_version 51660 (0.0009) +[2023-10-09 14:15:21,927][86122] Updated weights for policy 1, policy_version 51890 (0.0008) +[2023-10-09 14:15:22,113][86121] Updated weights for policy 0, policy_version 51670 (0.0009) +[2023-10-09 14:15:22,291][86122] Updated weights for policy 1, policy_version 51900 (0.0009) +[2023-10-09 14:15:22,477][86121] Updated weights for policy 0, policy_version 51680 (0.0007) +[2023-10-09 14:15:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 106070016. Throughput: 0: 1797.7, 1: 1816.1. Samples: 26522682. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:15:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000051680_52920320.pth... +[2023-10-09 14:15:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000051904_53149696.pth... +[2023-10-09 14:15:23,453][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000049984_51183616.pth +[2023-10-09 14:15:23,453][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000050208_51412992.pth +[2023-10-09 14:15:26,092][86122] Updated weights for policy 1, policy_version 51910 (0.0008) +[2023-10-09 14:15:26,252][86121] Updated weights for policy 0, policy_version 51690 (0.0007) +[2023-10-09 14:15:26,462][86122] Updated weights for policy 1, policy_version 51920 (0.0008) +[2023-10-09 14:15:26,622][86121] Updated weights for policy 0, policy_version 51700 (0.0007) +[2023-10-09 14:15:26,817][86122] Updated weights for policy 1, policy_version 51930 (0.0009) +[2023-10-09 14:15:26,990][86121] Updated weights for policy 0, policy_version 51710 (0.0009) +[2023-10-09 14:15:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106135552. Throughput: 0: 1810.9, 1: 1812.0. Samples: 26535366. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:28,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:15:30,681][86122] Updated weights for policy 1, policy_version 51940 (0.0008) +[2023-10-09 14:15:30,877][86121] Updated weights for policy 0, policy_version 51720 (0.0009) +[2023-10-09 14:15:31,045][86122] Updated weights for policy 1, policy_version 51950 (0.0007) +[2023-10-09 14:15:31,245][86121] Updated weights for policy 0, policy_version 51730 (0.0008) +[2023-10-09 14:15:31,399][86122] Updated weights for policy 1, policy_version 51960 (0.0010) +[2023-10-09 14:15:31,605][86121] Updated weights for policy 0, policy_version 51740 (0.0008) +[2023-10-09 14:15:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 106201088. Throughput: 0: 1794.4, 1: 1802.6. Samples: 26554904. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:33,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:15:35,221][86122] Updated weights for policy 1, policy_version 51970 (0.0009) +[2023-10-09 14:15:35,406][86121] Updated weights for policy 0, policy_version 51750 (0.0007) +[2023-10-09 14:15:35,568][86122] Updated weights for policy 1, policy_version 51980 (0.0009) +[2023-10-09 14:15:35,798][86121] Updated weights for policy 0, policy_version 51760 (0.0008) +[2023-10-09 14:15:35,926][86122] Updated weights for policy 1, policy_version 51990 (0.0007) +[2023-10-09 14:15:36,161][86121] Updated weights for policy 0, policy_version 51770 (0.0009) +[2023-10-09 14:15:36,286][86122] Updated weights for policy 1, policy_version 52000 (0.0008) +[2023-10-09 14:15:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106266624. Throughput: 0: 1799.1, 1: 1805.0. Samples: 26577624. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:38,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:15:39,844][86121] Updated weights for policy 0, policy_version 51780 (0.0008) +[2023-10-09 14:15:40,070][86122] Updated weights for policy 1, policy_version 52010 (0.0008) +[2023-10-09 14:15:40,209][86121] Updated weights for policy 0, policy_version 51790 (0.0007) +[2023-10-09 14:15:40,420][86122] Updated weights for policy 1, policy_version 52020 (0.0008) +[2023-10-09 14:15:40,578][86121] Updated weights for policy 0, policy_version 51800 (0.0007) +[2023-10-09 14:15:40,788][86122] Updated weights for policy 1, policy_version 52030 (0.0009) +[2023-10-09 14:15:43,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106332160. Throughput: 0: 1796.3, 1: 1808.3. Samples: 26587498. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:43,398][85186] Avg episode reward: [(0, '9.870'), (1, '10.000')] +[2023-10-09 14:15:44,285][86121] Updated weights for policy 0, policy_version 51810 (0.0009) +[2023-10-09 14:15:44,568][86122] Updated weights for policy 1, policy_version 52040 (0.0007) +[2023-10-09 14:15:44,647][86121] Updated weights for policy 0, policy_version 51820 (0.0009) +[2023-10-09 14:15:44,936][86122] Updated weights for policy 1, policy_version 52050 (0.0008) +[2023-10-09 14:15:45,018][86121] Updated weights for policy 0, policy_version 51830 (0.0008) +[2023-10-09 14:15:45,292][86122] Updated weights for policy 1, policy_version 52060 (0.0008) +[2023-10-09 14:15:45,371][86121] Updated weights for policy 0, policy_version 51840 (0.0008) +[2023-10-09 14:15:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 106397696. Throughput: 0: 1796.0, 1: 1815.3. Samples: 26610250. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:48,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:15:48,997][86122] Updated weights for policy 1, policy_version 52070 (0.0008) +[2023-10-09 14:15:49,129][86121] Updated weights for policy 0, policy_version 51850 (0.0007) +[2023-10-09 14:15:49,348][86122] Updated weights for policy 1, policy_version 52080 (0.0009) +[2023-10-09 14:15:49,505][86121] Updated weights for policy 0, policy_version 51860 (0.0008) +[2023-10-09 14:15:49,711][86122] Updated weights for policy 1, policy_version 52090 (0.0007) +[2023-10-09 14:15:49,868][86121] Updated weights for policy 0, policy_version 51870 (0.0008) +[2023-10-09 14:15:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106463232. Throughput: 0: 1800.2, 1: 1812.4. Samples: 26632938. Policy #0 lag: (min: 1.0, avg: 18.2, max: 33.0) +[2023-10-09 14:15:53,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:15:53,430][86122] Updated weights for policy 1, policy_version 52100 (0.0007) +[2023-10-09 14:15:53,455][86121] Updated weights for policy 0, policy_version 51880 (0.0007) +[2023-10-09 14:15:53,821][86122] Updated weights for policy 1, policy_version 52110 (0.0008) +[2023-10-09 14:15:53,828][86121] Updated weights for policy 0, policy_version 51890 (0.0009) +[2023-10-09 14:15:54,174][86122] Updated weights for policy 1, policy_version 52120 (0.0007) +[2023-10-09 14:15:54,187][86121] Updated weights for policy 0, policy_version 51900 (0.0008) +[2023-10-09 14:15:57,807][86122] Updated weights for policy 1, policy_version 52130 (0.0008) +[2023-10-09 14:15:57,981][86121] Updated weights for policy 0, policy_version 51910 (0.0008) +[2023-10-09 14:15:58,170][86122] Updated weights for policy 1, policy_version 52140 (0.0008) +[2023-10-09 14:15:58,345][86121] Updated weights for policy 0, policy_version 51920 (0.0009) +[2023-10-09 14:15:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106528768. Throughput: 0: 1791.9, 1: 1806.3. Samples: 26642440. Policy #0 lag: (min: 6.0, avg: 10.7, max: 38.0) +[2023-10-09 14:15:58,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:15:58,530][86122] Updated weights for policy 1, policy_version 52150 (0.0008) +[2023-10-09 14:15:58,707][86121] Updated weights for policy 0, policy_version 51930 (0.0009) +[2023-10-09 14:15:58,892][86122] Updated weights for policy 1, policy_version 52160 (0.0009) +[2023-10-09 14:16:02,379][86121] Updated weights for policy 0, policy_version 51940 (0.0009) +[2023-10-09 14:16:02,517][86122] Updated weights for policy 1, policy_version 52170 (0.0009) +[2023-10-09 14:16:02,732][86121] Updated weights for policy 0, policy_version 51950 (0.0008) +[2023-10-09 14:16:02,876][86122] Updated weights for policy 1, policy_version 52180 (0.0008) +[2023-10-09 14:16:03,096][86121] Updated weights for policy 0, policy_version 51960 (0.0007) +[2023-10-09 14:16:03,231][86122] Updated weights for policy 1, policy_version 52190 (0.0007) +[2023-10-09 14:16:03,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106659840. Throughput: 0: 1803.9, 1: 1814.2. Samples: 26665448. Policy #0 lag: (min: 6.0, avg: 10.7, max: 38.0) +[2023-10-09 14:16:03,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:16:06,919][86121] Updated weights for policy 0, policy_version 51970 (0.0008) +[2023-10-09 14:16:07,114][86122] Updated weights for policy 1, policy_version 52200 (0.0007) +[2023-10-09 14:16:07,291][86121] Updated weights for policy 0, policy_version 51980 (0.0008) +[2023-10-09 14:16:07,472][86122] Updated weights for policy 1, policy_version 52210 (0.0008) +[2023-10-09 14:16:07,651][86121] Updated weights for policy 0, policy_version 51990 (0.0008) +[2023-10-09 14:16:07,833][86122] Updated weights for policy 1, policy_version 52220 (0.0007) +[2023-10-09 14:16:08,016][86121] Updated weights for policy 0, policy_version 52000 (0.0008) +[2023-10-09 14:16:08,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 106725376. Throughput: 0: 1800.2, 1: 1813.3. Samples: 26685292. Policy #0 lag: (min: 6.0, avg: 10.7, max: 38.0) +[2023-10-09 14:16:08,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:16:11,584][86122] Updated weights for policy 1, policy_version 52230 (0.0009) +[2023-10-09 14:16:11,903][86121] Updated weights for policy 0, policy_version 52010 (0.0008) +[2023-10-09 14:16:11,953][86122] Updated weights for policy 1, policy_version 52240 (0.0009) +[2023-10-09 14:16:12,269][86121] Updated weights for policy 0, policy_version 52020 (0.0007) +[2023-10-09 14:16:12,317][86122] Updated weights for policy 1, policy_version 52250 (0.0007) +[2023-10-09 14:16:12,633][86121] Updated weights for policy 0, policy_version 52030 (0.0007) +[2023-10-09 14:16:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 106790912. Throughput: 0: 1797.5, 1: 1807.2. Samples: 26697578. Policy #0 lag: (min: 6.0, avg: 10.7, max: 38.0) +[2023-10-09 14:16:13,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:16:16,078][86122] Updated weights for policy 1, policy_version 52260 (0.0009) +[2023-10-09 14:16:16,219][86121] Updated weights for policy 0, policy_version 52040 (0.0007) +[2023-10-09 14:16:16,442][86122] Updated weights for policy 1, policy_version 52270 (0.0008) +[2023-10-09 14:16:16,588][86121] Updated weights for policy 0, policy_version 52050 (0.0007) +[2023-10-09 14:16:16,810][86122] Updated weights for policy 1, policy_version 52280 (0.0009) +[2023-10-09 14:16:16,959][86121] Updated weights for policy 0, policy_version 52060 (0.0007) +[2023-10-09 14:16:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 106856448. Throughput: 0: 1811.8, 1: 1817.5. Samples: 26718222. Policy #0 lag: (min: 6.0, avg: 10.7, max: 38.0) +[2023-10-09 14:16:18,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:16:20,511][86122] Updated weights for policy 1, policy_version 52290 (0.0008) +[2023-10-09 14:16:20,690][86121] Updated weights for policy 0, policy_version 52070 (0.0008) +[2023-10-09 14:16:20,863][86122] Updated weights for policy 1, policy_version 52300 (0.0008) +[2023-10-09 14:16:21,069][86121] Updated weights for policy 0, policy_version 52080 (0.0009) +[2023-10-09 14:16:21,227][86122] Updated weights for policy 1, policy_version 52310 (0.0009) +[2023-10-09 14:16:21,438][86121] Updated weights for policy 0, policy_version 52090 (0.0007) +[2023-10-09 14:16:21,591][86122] Updated weights for policy 1, policy_version 52320 (0.0008) +[2023-10-09 14:16:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 106921984. Throughput: 0: 1805.0, 1: 1806.3. Samples: 26740130. Policy #0 lag: (min: 6.0, avg: 10.7, max: 38.0) +[2023-10-09 14:16:23,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:16:25,062][86121] Updated weights for policy 0, policy_version 52100 (0.0008) +[2023-10-09 14:16:25,302][86122] Updated weights for policy 1, policy_version 52330 (0.0007) +[2023-10-09 14:16:25,421][86121] Updated weights for policy 0, policy_version 52110 (0.0008) +[2023-10-09 14:16:25,656][86122] Updated weights for policy 1, policy_version 52340 (0.0008) +[2023-10-09 14:16:25,790][86121] Updated weights for policy 0, policy_version 52120 (0.0007) +[2023-10-09 14:16:26,021][86122] Updated weights for policy 1, policy_version 52350 (0.0010) +[2023-10-09 14:16:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 106987520. Throughput: 0: 1814.2, 1: 1814.8. Samples: 26750804. Policy #0 lag: (min: 6.0, avg: 10.7, max: 38.0) +[2023-10-09 14:16:28,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:16:29,529][86121] Updated weights for policy 0, policy_version 52130 (0.0007) +[2023-10-09 14:16:29,583][86122] Updated weights for policy 1, policy_version 52360 (0.0007) +[2023-10-09 14:16:29,893][86121] Updated weights for policy 0, policy_version 52140 (0.0007) +[2023-10-09 14:16:29,958][86122] Updated weights for policy 1, policy_version 52370 (0.0007) +[2023-10-09 14:16:30,253][86121] Updated weights for policy 0, policy_version 52150 (0.0009) +[2023-10-09 14:16:30,312][86122] Updated weights for policy 1, policy_version 52380 (0.0009) +[2023-10-09 14:16:30,626][86121] Updated weights for policy 0, policy_version 52160 (0.0010) +[2023-10-09 14:16:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107053056. Throughput: 0: 1806.7, 1: 1807.6. Samples: 26772896. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) +[2023-10-09 14:16:33,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:16:34,049][86122] Updated weights for policy 1, policy_version 52390 (0.0007) +[2023-10-09 14:16:34,279][86121] Updated weights for policy 0, policy_version 52170 (0.0009) +[2023-10-09 14:16:34,416][86122] Updated weights for policy 1, policy_version 52400 (0.0008) +[2023-10-09 14:16:34,640][86121] Updated weights for policy 0, policy_version 52180 (0.0007) +[2023-10-09 14:16:34,784][86122] Updated weights for policy 1, policy_version 52410 (0.0008) +[2023-10-09 14:16:35,007][86121] Updated weights for policy 0, policy_version 52190 (0.0009) +[2023-10-09 14:16:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107118592. Throughput: 0: 1807.6, 1: 1808.8. Samples: 26795676. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) +[2023-10-09 14:16:38,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 14:16:38,604][86122] Updated weights for policy 1, policy_version 52420 (0.0008) +[2023-10-09 14:16:38,842][86121] Updated weights for policy 0, policy_version 52200 (0.0008) +[2023-10-09 14:16:38,978][86122] Updated weights for policy 1, policy_version 52430 (0.0008) +[2023-10-09 14:16:39,205][86121] Updated weights for policy 0, policy_version 52210 (0.0008) +[2023-10-09 14:16:39,346][86122] Updated weights for policy 1, policy_version 52440 (0.0008) +[2023-10-09 14:16:39,574][86121] Updated weights for policy 0, policy_version 52220 (0.0008) +[2023-10-09 14:16:43,062][86122] Updated weights for policy 1, policy_version 52450 (0.0008) +[2023-10-09 14:16:43,205][86121] Updated weights for policy 0, policy_version 52230 (0.0009) +[2023-10-09 14:16:43,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 107184128. Throughput: 0: 1811.1, 1: 1813.5. Samples: 26805546. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) +[2023-10-09 14:16:43,399][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:16:43,421][86122] Updated weights for policy 1, policy_version 52460 (0.0009) +[2023-10-09 14:16:43,561][86121] Updated weights for policy 0, policy_version 52240 (0.0007) +[2023-10-09 14:16:43,786][86122] Updated weights for policy 1, policy_version 52470 (0.0007) +[2023-10-09 14:16:43,933][86121] Updated weights for policy 0, policy_version 52250 (0.0008) +[2023-10-09 14:16:44,146][86122] Updated weights for policy 1, policy_version 52480 (0.0009) +[2023-10-09 14:16:47,386][86121] Updated weights for policy 0, policy_version 52260 (0.0010) +[2023-10-09 14:16:47,759][86121] Updated weights for policy 0, policy_version 52270 (0.0007) +[2023-10-09 14:16:47,814][86122] Updated weights for policy 1, policy_version 52490 (0.0009) +[2023-10-09 14:16:48,120][86121] Updated weights for policy 0, policy_version 52280 (0.0009) +[2023-10-09 14:16:48,173][86122] Updated weights for policy 1, policy_version 52500 (0.0007) +[2023-10-09 14:16:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 107249664. Throughput: 0: 1819.4, 1: 1805.4. Samples: 26828564. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) +[2023-10-09 14:16:48,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:16:48,536][86122] Updated weights for policy 1, policy_version 52510 (0.0007) +[2023-10-09 14:16:51,811][86121] Updated weights for policy 0, policy_version 52290 (0.0008) +[2023-10-09 14:16:52,180][86121] Updated weights for policy 0, policy_version 52300 (0.0008) +[2023-10-09 14:16:52,207][86122] Updated weights for policy 1, policy_version 52520 (0.0009) +[2023-10-09 14:16:52,540][86121] Updated weights for policy 0, policy_version 52310 (0.0009) +[2023-10-09 14:16:52,567][86122] Updated weights for policy 1, policy_version 52530 (0.0009) +[2023-10-09 14:16:52,897][86121] Updated weights for policy 0, policy_version 52320 (0.0008) +[2023-10-09 14:16:52,919][86122] Updated weights for policy 1, policy_version 52540 (0.0007) +[2023-10-09 14:16:53,397][85186] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 107380736. Throughput: 0: 1820.4, 1: 1811.4. Samples: 26848722. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) +[2023-10-09 14:16:53,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:16:56,594][86121] Updated weights for policy 0, policy_version 52330 (0.0007) +[2023-10-09 14:16:56,675][86122] Updated weights for policy 1, policy_version 52550 (0.0008) +[2023-10-09 14:16:56,969][86121] Updated weights for policy 0, policy_version 52340 (0.0008) +[2023-10-09 14:16:57,035][86122] Updated weights for policy 1, policy_version 52560 (0.0007) +[2023-10-09 14:16:57,335][86121] Updated weights for policy 0, policy_version 52350 (0.0007) +[2023-10-09 14:16:57,395][86122] Updated weights for policy 1, policy_version 52570 (0.0007) +[2023-10-09 14:16:58,397][85186] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 107446272. Throughput: 0: 1827.1, 1: 1807.7. Samples: 26861146. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) +[2023-10-09 14:16:58,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 14:17:01,018][86121] Updated weights for policy 0, policy_version 52360 (0.0008) +[2023-10-09 14:17:01,218][86122] Updated weights for policy 1, policy_version 52580 (0.0008) +[2023-10-09 14:17:01,391][86121] Updated weights for policy 0, policy_version 52370 (0.0008) +[2023-10-09 14:17:01,573][86122] Updated weights for policy 1, policy_version 52590 (0.0008) +[2023-10-09 14:17:01,745][86121] Updated weights for policy 0, policy_version 52380 (0.0007) +[2023-10-09 14:17:01,934][86122] Updated weights for policy 1, policy_version 52600 (0.0009) +[2023-10-09 14:17:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 107511808. Throughput: 0: 1816.6, 1: 1808.3. Samples: 26881340. Policy #0 lag: (min: 22.0, avg: 24.2, max: 54.0) +[2023-10-09 14:17:03,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:17:05,418][86121] Updated weights for policy 0, policy_version 52390 (0.0008) +[2023-10-09 14:17:05,714][86122] Updated weights for policy 1, policy_version 52610 (0.0010) +[2023-10-09 14:17:05,798][86121] Updated weights for policy 0, policy_version 52400 (0.0008) +[2023-10-09 14:17:06,075][86122] Updated weights for policy 1, policy_version 52620 (0.0008) +[2023-10-09 14:17:06,158][86121] Updated weights for policy 0, policy_version 52410 (0.0007) +[2023-10-09 14:17:06,449][86122] Updated weights for policy 1, policy_version 52630 (0.0008) +[2023-10-09 14:17:06,813][86122] Updated weights for policy 1, policy_version 52640 (0.0010) +[2023-10-09 14:17:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107577344. Throughput: 0: 1824.5, 1: 1803.6. Samples: 26903396. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:17:08,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:17:09,859][86121] Updated weights for policy 0, policy_version 52420 (0.0008) +[2023-10-09 14:17:10,232][86121] Updated weights for policy 0, policy_version 52430 (0.0007) +[2023-10-09 14:17:10,587][86122] Updated weights for policy 1, policy_version 52650 (0.0008) +[2023-10-09 14:17:10,598][86121] Updated weights for policy 0, policy_version 52440 (0.0007) +[2023-10-09 14:17:10,944][86122] Updated weights for policy 1, policy_version 52660 (0.0007) +[2023-10-09 14:17:11,302][86122] Updated weights for policy 1, policy_version 52670 (0.0008) +[2023-10-09 14:17:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107642880. Throughput: 0: 1817.5, 1: 1808.8. Samples: 26913986. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:17:13,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:17:14,423][86121] Updated weights for policy 0, policy_version 52450 (0.0009) +[2023-10-09 14:17:14,786][86121] Updated weights for policy 0, policy_version 52460 (0.0010) +[2023-10-09 14:17:14,977][86122] Updated weights for policy 1, policy_version 52680 (0.0009) +[2023-10-09 14:17:15,154][86121] Updated weights for policy 0, policy_version 52470 (0.0008) +[2023-10-09 14:17:15,335][86122] Updated weights for policy 1, policy_version 52690 (0.0008) +[2023-10-09 14:17:15,515][86121] Updated weights for policy 0, policy_version 52480 (0.0008) +[2023-10-09 14:17:15,700][86122] Updated weights for policy 1, policy_version 52700 (0.0008) +[2023-10-09 14:17:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107708416. Throughput: 0: 1825.6, 1: 1800.7. Samples: 26936076. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:17:18,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:17:19,444][86122] Updated weights for policy 1, policy_version 52710 (0.0009) +[2023-10-09 14:17:19,463][86121] Updated weights for policy 0, policy_version 52490 (0.0010) +[2023-10-09 14:17:19,803][86122] Updated weights for policy 1, policy_version 52720 (0.0007) +[2023-10-09 14:17:19,816][86121] Updated weights for policy 0, policy_version 52500 (0.0008) +[2023-10-09 14:17:20,161][86122] Updated weights for policy 1, policy_version 52730 (0.0008) +[2023-10-09 14:17:20,177][86121] Updated weights for policy 0, policy_version 52510 (0.0009) +[2023-10-09 14:17:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107773952. Throughput: 0: 1823.0, 1: 1797.1. Samples: 26958580. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:17:23,399][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:17:23,414][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000052736_54001664.pth... +[2023-10-09 14:17:23,415][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000052512_53772288.pth... +[2023-10-09 14:17:23,453][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000050816_52035584.pth +[2023-10-09 14:17:23,455][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000051040_52264960.pth +[2023-10-09 14:17:23,889][86121] Updated weights for policy 0, policy_version 52520 (0.0008) +[2023-10-09 14:17:24,064][86122] Updated weights for policy 1, policy_version 52740 (0.0009) +[2023-10-09 14:17:24,245][86121] Updated weights for policy 0, policy_version 52530 (0.0008) +[2023-10-09 14:17:24,456][86122] Updated weights for policy 1, policy_version 52750 (0.0009) +[2023-10-09 14:17:24,608][86121] Updated weights for policy 0, policy_version 52540 (0.0007) +[2023-10-09 14:17:24,818][86122] Updated weights for policy 1, policy_version 52760 (0.0009) +[2023-10-09 14:17:28,385][86121] Updated weights for policy 0, policy_version 52550 (0.0009) +[2023-10-09 14:17:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107839488. Throughput: 0: 1822.0, 1: 1797.1. Samples: 26968402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:17:28,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 14:17:28,497][86122] Updated weights for policy 1, policy_version 52770 (0.0008) +[2023-10-09 14:17:28,744][86121] Updated weights for policy 0, policy_version 52560 (0.0010) +[2023-10-09 14:17:28,856][86122] Updated weights for policy 1, policy_version 52780 (0.0008) +[2023-10-09 14:17:29,110][86121] Updated weights for policy 0, policy_version 52570 (0.0007) +[2023-10-09 14:17:29,222][86122] Updated weights for policy 1, policy_version 52790 (0.0008) +[2023-10-09 14:17:29,585][86122] Updated weights for policy 1, policy_version 52800 (0.0009) +[2023-10-09 14:17:32,729][86121] Updated weights for policy 0, policy_version 52580 (0.0008) +[2023-10-09 14:17:33,088][86121] Updated weights for policy 0, policy_version 52590 (0.0009) +[2023-10-09 14:17:33,346][86122] Updated weights for policy 1, policy_version 52810 (0.0009) +[2023-10-09 14:17:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 107905024. Throughput: 0: 1816.8, 1: 1797.3. Samples: 26991200. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:17:33,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 14:17:33,453][86121] Updated weights for policy 0, policy_version 52600 (0.0007) +[2023-10-09 14:17:33,701][86122] Updated weights for policy 1, policy_version 52820 (0.0007) +[2023-10-09 14:17:34,065][86122] Updated weights for policy 1, policy_version 52830 (0.0008) +[2023-10-09 14:17:36,904][86121] Updated weights for policy 0, policy_version 52610 (0.0009) +[2023-10-09 14:17:37,272][86121] Updated weights for policy 0, policy_version 52620 (0.0009) +[2023-10-09 14:17:37,630][86121] Updated weights for policy 0, policy_version 52630 (0.0008) +[2023-10-09 14:17:37,690][86122] Updated weights for policy 1, policy_version 52840 (0.0007) +[2023-10-09 14:17:38,003][86121] Updated weights for policy 0, policy_version 52640 (0.0009) +[2023-10-09 14:17:38,058][86122] Updated weights for policy 1, policy_version 52850 (0.0008) +[2023-10-09 14:17:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108003328. Throughput: 0: 1825.5, 1: 1815.1. Samples: 27012548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:17:38,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.990')] +[2023-10-09 14:17:38,423][86122] Updated weights for policy 1, policy_version 52860 (0.0009) +[2023-10-09 14:17:41,731][86121] Updated weights for policy 0, policy_version 52650 (0.0008) +[2023-10-09 14:17:42,105][86121] Updated weights for policy 0, policy_version 52660 (0.0009) +[2023-10-09 14:17:42,159][86122] Updated weights for policy 1, policy_version 52870 (0.0007) +[2023-10-09 14:17:42,465][86121] Updated weights for policy 0, policy_version 52670 (0.0009) +[2023-10-09 14:17:42,523][86122] Updated weights for policy 1, policy_version 52880 (0.0007) +[2023-10-09 14:17:42,880][86122] Updated weights for policy 1, policy_version 52890 (0.0008) +[2023-10-09 14:17:43,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 108101632. Throughput: 0: 1820.4, 1: 1802.8. Samples: 27024194. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 14:17:43,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.990')] +[2023-10-09 14:17:46,086][86121] Updated weights for policy 0, policy_version 52680 (0.0008) +[2023-10-09 14:17:46,448][86121] Updated weights for policy 0, policy_version 52690 (0.0008) +[2023-10-09 14:17:46,481][86122] Updated weights for policy 1, policy_version 52900 (0.0008) +[2023-10-09 14:17:46,820][86121] Updated weights for policy 0, policy_version 52700 (0.0009) +[2023-10-09 14:17:46,838][86122] Updated weights for policy 1, policy_version 52910 (0.0008) +[2023-10-09 14:17:47,204][86122] Updated weights for policy 1, policy_version 52920 (0.0009) +[2023-10-09 14:17:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 108167168. Throughput: 0: 1826.8, 1: 1817.2. Samples: 27045316. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 14:17:48,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.990')] +[2023-10-09 14:17:50,419][86121] Updated weights for policy 0, policy_version 52710 (0.0008) +[2023-10-09 14:17:50,804][86121] Updated weights for policy 0, policy_version 52720 (0.0011) +[2023-10-09 14:17:51,016][86122] Updated weights for policy 1, policy_version 52930 (0.0009) +[2023-10-09 14:17:51,169][86121] Updated weights for policy 0, policy_version 52730 (0.0008) +[2023-10-09 14:17:51,371][86122] Updated weights for policy 1, policy_version 52940 (0.0007) +[2023-10-09 14:17:51,736][86122] Updated weights for policy 1, policy_version 52950 (0.0009) +[2023-10-09 14:17:52,096][86122] Updated weights for policy 1, policy_version 52960 (0.0008) +[2023-10-09 14:17:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108232704. Throughput: 0: 1822.3, 1: 1816.0. Samples: 27067120. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 14:17:53,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.990')] +[2023-10-09 14:17:54,851][86121] Updated weights for policy 0, policy_version 52740 (0.0007) +[2023-10-09 14:17:55,218][86121] Updated weights for policy 0, policy_version 52750 (0.0007) +[2023-10-09 14:17:55,589][86121] Updated weights for policy 0, policy_version 52760 (0.0008) +[2023-10-09 14:17:55,725][86122] Updated weights for policy 1, policy_version 52970 (0.0008) +[2023-10-09 14:17:56,092][86122] Updated weights for policy 1, policy_version 52980 (0.0011) +[2023-10-09 14:17:56,463][86122] Updated weights for policy 1, policy_version 52990 (0.0011) +[2023-10-09 14:17:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108298240. Throughput: 0: 1825.6, 1: 1821.8. Samples: 27078120. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 14:17:58,398][85186] Avg episode reward: [(0, '9.790'), (1, '9.990')] +[2023-10-09 14:17:59,224][86121] Updated weights for policy 0, policy_version 52770 (0.0009) +[2023-10-09 14:17:59,591][86121] Updated weights for policy 0, policy_version 52780 (0.0008) +[2023-10-09 14:17:59,953][86121] Updated weights for policy 0, policy_version 52790 (0.0008) +[2023-10-09 14:18:00,182][86122] Updated weights for policy 1, policy_version 53000 (0.0007) +[2023-10-09 14:18:00,316][86121] Updated weights for policy 0, policy_version 52800 (0.0007) +[2023-10-09 14:18:00,544][86122] Updated weights for policy 1, policy_version 53010 (0.0008) +[2023-10-09 14:18:00,904][86122] Updated weights for policy 1, policy_version 53020 (0.0010) +[2023-10-09 14:18:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 108363776. Throughput: 0: 1829.5, 1: 1816.0. Samples: 27100122. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 14:18:03,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.990')] +[2023-10-09 14:18:03,937][86121] Updated weights for policy 0, policy_version 52810 (0.0010) +[2023-10-09 14:18:04,304][86121] Updated weights for policy 0, policy_version 52820 (0.0009) +[2023-10-09 14:18:04,671][86121] Updated weights for policy 0, policy_version 52830 (0.0010) +[2023-10-09 14:18:04,739][86122] Updated weights for policy 1, policy_version 53030 (0.0008) +[2023-10-09 14:18:05,107][86122] Updated weights for policy 1, policy_version 53040 (0.0008) +[2023-10-09 14:18:05,471][86122] Updated weights for policy 1, policy_version 53050 (0.0008) +[2023-10-09 14:18:08,262][86121] Updated weights for policy 0, policy_version 52840 (0.0009) +[2023-10-09 14:18:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108429312. Throughput: 0: 1834.4, 1: 1813.5. Samples: 27122732. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 14:18:08,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.990')] +[2023-10-09 14:18:08,634][86121] Updated weights for policy 0, policy_version 52850 (0.0009) +[2023-10-09 14:18:08,995][86121] Updated weights for policy 0, policy_version 52860 (0.0008) +[2023-10-09 14:18:09,369][86122] Updated weights for policy 1, policy_version 53060 (0.0008) +[2023-10-09 14:18:09,763][86122] Updated weights for policy 1, policy_version 53070 (0.0008) +[2023-10-09 14:18:10,124][86122] Updated weights for policy 1, policy_version 53080 (0.0009) +[2023-10-09 14:18:12,820][86121] Updated weights for policy 0, policy_version 52870 (0.0007) +[2023-10-09 14:18:13,193][86121] Updated weights for policy 0, policy_version 52880 (0.0007) +[2023-10-09 14:18:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108494848. Throughput: 0: 1835.3, 1: 1814.0. Samples: 27132622. Policy #0 lag: (min: 17.0, avg: 28.5, max: 49.0) +[2023-10-09 14:18:13,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.990')] +[2023-10-09 14:18:13,556][86121] Updated weights for policy 0, policy_version 52890 (0.0007) +[2023-10-09 14:18:13,584][86122] Updated weights for policy 1, policy_version 53090 (0.0009) +[2023-10-09 14:18:13,949][86122] Updated weights for policy 1, policy_version 53100 (0.0007) +[2023-10-09 14:18:14,309][86122] Updated weights for policy 1, policy_version 53110 (0.0008) +[2023-10-09 14:18:14,673][86122] Updated weights for policy 1, policy_version 53120 (0.0008) +[2023-10-09 14:18:17,262][86121] Updated weights for policy 0, policy_version 52900 (0.0007) +[2023-10-09 14:18:17,622][86121] Updated weights for policy 0, policy_version 52910 (0.0007) +[2023-10-09 14:18:17,999][86121] Updated weights for policy 0, policy_version 52920 (0.0007) +[2023-10-09 14:18:18,241][86122] Updated weights for policy 1, policy_version 53130 (0.0008) +[2023-10-09 14:18:18,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 108593152. Throughput: 0: 1830.8, 1: 1825.3. Samples: 27155728. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:18,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.990')] +[2023-10-09 14:18:18,608][86122] Updated weights for policy 1, policy_version 53140 (0.0008) +[2023-10-09 14:18:18,963][86122] Updated weights for policy 1, policy_version 53150 (0.0010) +[2023-10-09 14:18:21,869][86121] Updated weights for policy 0, policy_version 52930 (0.0008) +[2023-10-09 14:18:22,241][86121] Updated weights for policy 0, policy_version 52940 (0.0009) +[2023-10-09 14:18:22,603][86121] Updated weights for policy 0, policy_version 52950 (0.0009) +[2023-10-09 14:18:22,683][86122] Updated weights for policy 1, policy_version 53160 (0.0010) +[2023-10-09 14:18:22,962][86121] Updated weights for policy 0, policy_version 52960 (0.0007) +[2023-10-09 14:18:23,043][86122] Updated weights for policy 1, policy_version 53170 (0.0008) +[2023-10-09 14:18:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 108658688. Throughput: 0: 1817.4, 1: 1827.7. Samples: 27176576. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:23,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 14:18:23,408][86122] Updated weights for policy 1, policy_version 53180 (0.0008) +[2023-10-09 14:18:26,801][86121] Updated weights for policy 0, policy_version 52970 (0.0009) +[2023-10-09 14:18:26,892][86122] Updated weights for policy 1, policy_version 53190 (0.0007) +[2023-10-09 14:18:27,169][86121] Updated weights for policy 0, policy_version 52980 (0.0008) +[2023-10-09 14:18:27,257][86122] Updated weights for policy 1, policy_version 53200 (0.0007) +[2023-10-09 14:18:27,535][86121] Updated weights for policy 0, policy_version 52990 (0.0007) +[2023-10-09 14:18:27,612][86122] Updated weights for policy 1, policy_version 53210 (0.0008) +[2023-10-09 14:18:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 108756992. Throughput: 0: 1812.9, 1: 1832.7. Samples: 27188244. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:28,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 14:18:31,274][86121] Updated weights for policy 0, policy_version 53000 (0.0008) +[2023-10-09 14:18:31,355][86122] Updated weights for policy 1, policy_version 53220 (0.0008) +[2023-10-09 14:18:31,646][86121] Updated weights for policy 0, policy_version 53010 (0.0007) +[2023-10-09 14:18:31,716][86122] Updated weights for policy 1, policy_version 53230 (0.0008) +[2023-10-09 14:18:32,001][86121] Updated weights for policy 0, policy_version 53020 (0.0008) +[2023-10-09 14:18:32,069][86122] Updated weights for policy 1, policy_version 53240 (0.0007) +[2023-10-09 14:18:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 108822528. Throughput: 0: 1814.5, 1: 1828.7. Samples: 27209258. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:33,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 14:18:35,696][86122] Updated weights for policy 1, policy_version 53250 (0.0007) +[2023-10-09 14:18:35,933][86121] Updated weights for policy 0, policy_version 53030 (0.0007) +[2023-10-09 14:18:36,053][86122] Updated weights for policy 1, policy_version 53260 (0.0007) +[2023-10-09 14:18:36,314][86121] Updated weights for policy 0, policy_version 53040 (0.0007) +[2023-10-09 14:18:36,411][86122] Updated weights for policy 1, policy_version 53270 (0.0007) +[2023-10-09 14:18:36,686][86121] Updated weights for policy 0, policy_version 53050 (0.0008) +[2023-10-09 14:18:36,778][86122] Updated weights for policy 1, policy_version 53280 (0.0007) +[2023-10-09 14:18:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 108888064. Throughput: 0: 1798.5, 1: 1839.7. Samples: 27230836. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:38,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.980')] +[2023-10-09 14:18:40,362][86122] Updated weights for policy 1, policy_version 53290 (0.0008) +[2023-10-09 14:18:40,537][86121] Updated weights for policy 0, policy_version 53060 (0.0007) +[2023-10-09 14:18:40,727][86122] Updated weights for policy 1, policy_version 53300 (0.0009) +[2023-10-09 14:18:40,895][86121] Updated weights for policy 0, policy_version 53070 (0.0007) +[2023-10-09 14:18:41,083][86122] Updated weights for policy 1, policy_version 53310 (0.0008) +[2023-10-09 14:18:41,257][86121] Updated weights for policy 0, policy_version 53080 (0.0007) +[2023-10-09 14:18:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 108953600. Throughput: 0: 1813.2, 1: 1830.1. Samples: 27242070. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:43,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.980')] +[2023-10-09 14:18:44,801][86122] Updated weights for policy 1, policy_version 53320 (0.0009) +[2023-10-09 14:18:45,006][86121] Updated weights for policy 0, policy_version 53090 (0.0009) +[2023-10-09 14:18:45,155][86122] Updated weights for policy 1, policy_version 53330 (0.0007) +[2023-10-09 14:18:45,371][86121] Updated weights for policy 0, policy_version 53100 (0.0008) +[2023-10-09 14:18:45,516][86122] Updated weights for policy 1, policy_version 53340 (0.0007) +[2023-10-09 14:18:45,739][86121] Updated weights for policy 0, policy_version 53110 (0.0008) +[2023-10-09 14:18:46,100][86121] Updated weights for policy 0, policy_version 53120 (0.0008) +[2023-10-09 14:18:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109019136. Throughput: 0: 1793.6, 1: 1841.2. Samples: 27263688. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:48,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 14:18:49,091][86122] Updated weights for policy 1, policy_version 53350 (0.0009) +[2023-10-09 14:18:49,453][86122] Updated weights for policy 1, policy_version 53360 (0.0009) +[2023-10-09 14:18:49,624][86121] Updated weights for policy 0, policy_version 53130 (0.0010) +[2023-10-09 14:18:49,817][86122] Updated weights for policy 1, policy_version 53370 (0.0009) +[2023-10-09 14:18:49,990][86121] Updated weights for policy 0, policy_version 53140 (0.0008) +[2023-10-09 14:18:50,352][86121] Updated weights for policy 0, policy_version 53150 (0.0008) +[2023-10-09 14:18:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109084672. Throughput: 0: 1790.6, 1: 1852.0. Samples: 27286652. Policy #0 lag: (min: 1.0, avg: 6.7, max: 33.0) +[2023-10-09 14:18:53,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 14:18:53,633][86122] Updated weights for policy 1, policy_version 53380 (0.0009) +[2023-10-09 14:18:54,001][86122] Updated weights for policy 1, policy_version 53390 (0.0008) +[2023-10-09 14:18:54,109][86121] Updated weights for policy 0, policy_version 53160 (0.0008) +[2023-10-09 14:18:54,360][86122] Updated weights for policy 1, policy_version 53400 (0.0009) +[2023-10-09 14:18:54,477][86121] Updated weights for policy 0, policy_version 53170 (0.0008) +[2023-10-09 14:18:54,853][86121] Updated weights for policy 0, policy_version 53180 (0.0008) +[2023-10-09 14:18:57,987][86122] Updated weights for policy 1, policy_version 53410 (0.0010) +[2023-10-09 14:18:58,350][86122] Updated weights for policy 1, policy_version 53420 (0.0010) +[2023-10-09 14:18:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109150208. Throughput: 0: 1788.6, 1: 1851.0. Samples: 27296404. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:18:58,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.980')] +[2023-10-09 14:18:58,505][86121] Updated weights for policy 0, policy_version 53190 (0.0008) +[2023-10-09 14:18:58,715][86122] Updated weights for policy 1, policy_version 53430 (0.0008) +[2023-10-09 14:18:58,877][86121] Updated weights for policy 0, policy_version 53200 (0.0007) +[2023-10-09 14:18:59,078][86122] Updated weights for policy 1, policy_version 53440 (0.0007) +[2023-10-09 14:18:59,238][86121] Updated weights for policy 0, policy_version 53210 (0.0008) +[2023-10-09 14:19:02,719][86122] Updated weights for policy 1, policy_version 53450 (0.0008) +[2023-10-09 14:19:02,815][86121] Updated weights for policy 0, policy_version 53220 (0.0008) +[2023-10-09 14:19:03,072][86122] Updated weights for policy 1, policy_version 53460 (0.0008) +[2023-10-09 14:19:03,189][86121] Updated weights for policy 0, policy_version 53230 (0.0008) +[2023-10-09 14:19:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109215744. Throughput: 0: 1792.3, 1: 1840.7. Samples: 27319212. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:19:03,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.980')] +[2023-10-09 14:19:03,426][86122] Updated weights for policy 1, policy_version 53470 (0.0007) +[2023-10-09 14:19:03,555][86121] Updated weights for policy 0, policy_version 53240 (0.0007) +[2023-10-09 14:19:07,170][86122] Updated weights for policy 1, policy_version 53480 (0.0008) +[2023-10-09 14:19:07,220][86121] Updated weights for policy 0, policy_version 53250 (0.0008) +[2023-10-09 14:19:07,534][86122] Updated weights for policy 1, policy_version 53490 (0.0009) +[2023-10-09 14:19:07,583][86121] Updated weights for policy 0, policy_version 53260 (0.0007) +[2023-10-09 14:19:07,886][86122] Updated weights for policy 1, policy_version 53500 (0.0007) +[2023-10-09 14:19:07,951][86121] Updated weights for policy 0, policy_version 53270 (0.0008) +[2023-10-09 14:19:08,317][86121] Updated weights for policy 0, policy_version 53280 (0.0009) +[2023-10-09 14:19:08,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 109346816. Throughput: 0: 1809.4, 1: 1819.2. Samples: 27339864. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:19:08,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.980')] +[2023-10-09 14:19:11,780][86122] Updated weights for policy 1, policy_version 53510 (0.0008) +[2023-10-09 14:19:12,028][86121] Updated weights for policy 0, policy_version 53290 (0.0009) +[2023-10-09 14:19:12,133][86122] Updated weights for policy 1, policy_version 53520 (0.0008) +[2023-10-09 14:19:12,390][86121] Updated weights for policy 0, policy_version 53300 (0.0008) +[2023-10-09 14:19:12,499][86122] Updated weights for policy 1, policy_version 53530 (0.0009) +[2023-10-09 14:19:12,758][86121] Updated weights for policy 0, policy_version 53310 (0.0010) +[2023-10-09 14:19:13,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 109412352. Throughput: 0: 1809.4, 1: 1828.8. Samples: 27351964. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:19:13,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.980')] +[2023-10-09 14:19:16,275][86122] Updated weights for policy 1, policy_version 53540 (0.0008) +[2023-10-09 14:19:16,532][86121] Updated weights for policy 0, policy_version 53320 (0.0009) +[2023-10-09 14:19:16,639][86122] Updated weights for policy 1, policy_version 53550 (0.0008) +[2023-10-09 14:19:16,908][86121] Updated weights for policy 0, policy_version 53330 (0.0007) +[2023-10-09 14:19:17,009][86122] Updated weights for policy 1, policy_version 53560 (0.0007) +[2023-10-09 14:19:17,266][86121] Updated weights for policy 0, policy_version 53340 (0.0008) +[2023-10-09 14:19:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 109477888. Throughput: 0: 1817.1, 1: 1820.0. Samples: 27372930. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:19:18,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 14:19:20,697][86122] Updated weights for policy 1, policy_version 53570 (0.0007) +[2023-10-09 14:19:21,008][86121] Updated weights for policy 0, policy_version 53350 (0.0009) +[2023-10-09 14:19:21,055][86122] Updated weights for policy 1, policy_version 53580 (0.0007) +[2023-10-09 14:19:21,369][86121] Updated weights for policy 0, policy_version 53360 (0.0008) +[2023-10-09 14:19:21,416][86122] Updated weights for policy 1, policy_version 53590 (0.0008) +[2023-10-09 14:19:21,734][86121] Updated weights for policy 0, policy_version 53370 (0.0007) +[2023-10-09 14:19:21,777][86122] Updated weights for policy 1, policy_version 53600 (0.0009) +[2023-10-09 14:19:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 109543424. Throughput: 0: 1820.7, 1: 1815.6. Samples: 27394468. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:19:23,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 14:19:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000053600_54886400.pth... +[2023-10-09 14:19:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000053376_54657024.pth... +[2023-10-09 14:19:23,437][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000051904_53149696.pth +[2023-10-09 14:19:23,437][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000051680_52920320.pth +[2023-10-09 14:19:25,288][86121] Updated weights for policy 0, policy_version 53380 (0.0008) +[2023-10-09 14:19:25,570][86122] Updated weights for policy 1, policy_version 53610 (0.0008) +[2023-10-09 14:19:25,653][86121] Updated weights for policy 0, policy_version 53390 (0.0008) +[2023-10-09 14:19:25,930][86122] Updated weights for policy 1, policy_version 53620 (0.0007) +[2023-10-09 14:19:26,019][86121] Updated weights for policy 0, policy_version 53400 (0.0007) +[2023-10-09 14:19:26,285][86122] Updated weights for policy 1, policy_version 53630 (0.0009) +[2023-10-09 14:19:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109608960. Throughput: 0: 1818.5, 1: 1820.4. Samples: 27405822. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:19:28,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.980')] +[2023-10-09 14:19:29,687][86121] Updated weights for policy 0, policy_version 53410 (0.0007) +[2023-10-09 14:19:29,936][86122] Updated weights for policy 1, policy_version 53640 (0.0008) +[2023-10-09 14:19:30,048][86121] Updated weights for policy 0, policy_version 53420 (0.0007) +[2023-10-09 14:19:30,298][86122] Updated weights for policy 1, policy_version 53650 (0.0009) +[2023-10-09 14:19:30,410][86121] Updated weights for policy 0, policy_version 53430 (0.0007) +[2023-10-09 14:19:30,658][86122] Updated weights for policy 1, policy_version 53660 (0.0008) +[2023-10-09 14:19:30,782][86121] Updated weights for policy 0, policy_version 53440 (0.0008) +[2023-10-09 14:19:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109674496. Throughput: 0: 1826.3, 1: 1817.5. Samples: 27427658. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:19:33,398][85186] Avg episode reward: [(0, '9.820'), (1, '9.980')] +[2023-10-09 14:19:34,247][86122] Updated weights for policy 1, policy_version 53670 (0.0008) +[2023-10-09 14:19:34,564][86121] Updated weights for policy 0, policy_version 53450 (0.0007) +[2023-10-09 14:19:34,607][86122] Updated weights for policy 1, policy_version 53680 (0.0008) +[2023-10-09 14:19:34,924][86121] Updated weights for policy 0, policy_version 53460 (0.0007) +[2023-10-09 14:19:34,965][86122] Updated weights for policy 1, policy_version 53690 (0.0009) +[2023-10-09 14:19:35,286][86121] Updated weights for policy 0, policy_version 53470 (0.0010) +[2023-10-09 14:19:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109740032. Throughput: 0: 1823.9, 1: 1810.6. Samples: 27450206. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:19:38,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.980')] +[2023-10-09 14:19:38,794][86122] Updated weights for policy 1, policy_version 53700 (0.0008) +[2023-10-09 14:19:39,058][86121] Updated weights for policy 0, policy_version 53480 (0.0009) +[2023-10-09 14:19:39,177][86122] Updated weights for policy 1, policy_version 53710 (0.0007) +[2023-10-09 14:19:39,422][86121] Updated weights for policy 0, policy_version 53490 (0.0008) +[2023-10-09 14:19:39,539][86122] Updated weights for policy 1, policy_version 53720 (0.0009) +[2023-10-09 14:19:39,783][86121] Updated weights for policy 0, policy_version 53500 (0.0009) +[2023-10-09 14:19:43,164][86122] Updated weights for policy 1, policy_version 53730 (0.0008) +[2023-10-09 14:19:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 109805568. Throughput: 0: 1825.0, 1: 1810.5. Samples: 27460004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:19:43,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.980')] +[2023-10-09 14:19:43,459][86121] Updated weights for policy 0, policy_version 53510 (0.0008) +[2023-10-09 14:19:43,519][86122] Updated weights for policy 1, policy_version 53740 (0.0008) +[2023-10-09 14:19:43,830][86121] Updated weights for policy 0, policy_version 53520 (0.0007) +[2023-10-09 14:19:43,879][86122] Updated weights for policy 1, policy_version 53750 (0.0008) +[2023-10-09 14:19:44,191][86121] Updated weights for policy 0, policy_version 53530 (0.0008) +[2023-10-09 14:19:44,240][86122] Updated weights for policy 1, policy_version 53760 (0.0008) +[2023-10-09 14:19:47,757][86122] Updated weights for policy 1, policy_version 53770 (0.0009) +[2023-10-09 14:19:47,840][86121] Updated weights for policy 0, policy_version 53540 (0.0009) +[2023-10-09 14:19:48,124][86122] Updated weights for policy 1, policy_version 53780 (0.0009) +[2023-10-09 14:19:48,206][86121] Updated weights for policy 0, policy_version 53550 (0.0008) +[2023-10-09 14:19:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 109871104. Throughput: 0: 1826.2, 1: 1813.5. Samples: 27482998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:19:48,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.980')] +[2023-10-09 14:19:48,480][86122] Updated weights for policy 1, policy_version 53790 (0.0008) +[2023-10-09 14:19:48,581][86121] Updated weights for policy 0, policy_version 53560 (0.0008) +[2023-10-09 14:19:52,228][86122] Updated weights for policy 1, policy_version 53800 (0.0009) +[2023-10-09 14:19:52,239][86121] Updated weights for policy 0, policy_version 53570 (0.0008) +[2023-10-09 14:19:52,590][86122] Updated weights for policy 1, policy_version 53810 (0.0009) +[2023-10-09 14:19:52,595][86121] Updated weights for policy 0, policy_version 53580 (0.0008) +[2023-10-09 14:19:52,947][86122] Updated weights for policy 1, policy_version 53820 (0.0008) +[2023-10-09 14:19:52,956][86121] Updated weights for policy 0, policy_version 53590 (0.0007) +[2023-10-09 14:19:53,325][86121] Updated weights for policy 0, policy_version 53600 (0.0008) +[2023-10-09 14:19:53,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 110002176. Throughput: 0: 1827.2, 1: 1820.2. Samples: 27503996. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:19:53,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 14:19:56,592][86122] Updated weights for policy 1, policy_version 53830 (0.0008) +[2023-10-09 14:19:56,959][86122] Updated weights for policy 1, policy_version 53840 (0.0007) +[2023-10-09 14:19:57,096][86121] Updated weights for policy 0, policy_version 53610 (0.0007) +[2023-10-09 14:19:57,321][86122] Updated weights for policy 1, policy_version 53850 (0.0009) +[2023-10-09 14:19:57,466][86121] Updated weights for policy 0, policy_version 53620 (0.0009) +[2023-10-09 14:19:57,834][86121] Updated weights for policy 0, policy_version 53630 (0.0010) +[2023-10-09 14:19:58,397][85186] Fps is (10 sec: 19661.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 110067712. Throughput: 0: 1823.3, 1: 1822.6. Samples: 27516030. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:19:58,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 14:20:01,167][86122] Updated weights for policy 1, policy_version 53860 (0.0009) +[2023-10-09 14:20:01,338][86121] Updated weights for policy 0, policy_version 53640 (0.0008) +[2023-10-09 14:20:01,530][86122] Updated weights for policy 1, policy_version 53870 (0.0009) +[2023-10-09 14:20:01,705][86121] Updated weights for policy 0, policy_version 53650 (0.0009) +[2023-10-09 14:20:01,892][86122] Updated weights for policy 1, policy_version 53880 (0.0008) +[2023-10-09 14:20:02,071][86121] Updated weights for policy 0, policy_version 53660 (0.0007) +[2023-10-09 14:20:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 110133248. Throughput: 0: 1820.1, 1: 1817.7. Samples: 27536632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:20:03,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 14:20:05,462][86122] Updated weights for policy 1, policy_version 53890 (0.0007) +[2023-10-09 14:20:05,747][86121] Updated weights for policy 0, policy_version 53670 (0.0009) +[2023-10-09 14:20:05,826][86122] Updated weights for policy 1, policy_version 53900 (0.0007) +[2023-10-09 14:20:06,122][86121] Updated weights for policy 0, policy_version 53680 (0.0008) +[2023-10-09 14:20:06,190][86122] Updated weights for policy 1, policy_version 53910 (0.0007) +[2023-10-09 14:20:06,481][86121] Updated weights for policy 0, policy_version 53690 (0.0008) +[2023-10-09 14:20:06,552][86122] Updated weights for policy 1, policy_version 53920 (0.0007) +[2023-10-09 14:20:08,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 110198784. Throughput: 0: 1827.9, 1: 1826.4. Samples: 27558916. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 14:20:08,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 14:20:10,114][86121] Updated weights for policy 0, policy_version 53700 (0.0009) +[2023-10-09 14:20:10,302][86122] Updated weights for policy 1, policy_version 53930 (0.0008) +[2023-10-09 14:20:10,480][86121] Updated weights for policy 0, policy_version 53710 (0.0008) +[2023-10-09 14:20:10,668][86122] Updated weights for policy 1, policy_version 53940 (0.0008) +[2023-10-09 14:20:10,844][86121] Updated weights for policy 0, policy_version 53720 (0.0009) +[2023-10-09 14:20:11,023][86122] Updated weights for policy 1, policy_version 53950 (0.0008) +[2023-10-09 14:20:13,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110264320. Throughput: 0: 1820.4, 1: 1818.5. Samples: 27569574. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 14:20:13,399][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 14:20:14,568][86121] Updated weights for policy 0, policy_version 53730 (0.0008) +[2023-10-09 14:20:14,746][86122] Updated weights for policy 1, policy_version 53960 (0.0009) +[2023-10-09 14:20:14,934][86121] Updated weights for policy 0, policy_version 53740 (0.0007) +[2023-10-09 14:20:15,109][86122] Updated weights for policy 1, policy_version 53970 (0.0008) +[2023-10-09 14:20:15,296][86121] Updated weights for policy 0, policy_version 53750 (0.0008) +[2023-10-09 14:20:15,468][86122] Updated weights for policy 1, policy_version 53980 (0.0008) +[2023-10-09 14:20:15,662][86121] Updated weights for policy 0, policy_version 53760 (0.0008) +[2023-10-09 14:20:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110329856. Throughput: 0: 1826.2, 1: 1826.7. Samples: 27592038. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 14:20:18,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.990')] +[2023-10-09 14:20:19,015][86122] Updated weights for policy 1, policy_version 53990 (0.0008) +[2023-10-09 14:20:19,313][86121] Updated weights for policy 0, policy_version 53770 (0.0010) +[2023-10-09 14:20:19,371][86122] Updated weights for policy 1, policy_version 54000 (0.0007) +[2023-10-09 14:20:19,681][86121] Updated weights for policy 0, policy_version 53780 (0.0008) +[2023-10-09 14:20:19,728][86122] Updated weights for policy 1, policy_version 54010 (0.0008) +[2023-10-09 14:20:20,047][86121] Updated weights for policy 0, policy_version 53790 (0.0008) +[2023-10-09 14:20:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110395392. Throughput: 0: 1828.8, 1: 1831.9. Samples: 27614938. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 14:20:23,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.990')] +[2023-10-09 14:20:23,621][86122] Updated weights for policy 1, policy_version 54020 (0.0009) +[2023-10-09 14:20:23,829][86121] Updated weights for policy 0, policy_version 53800 (0.0007) +[2023-10-09 14:20:24,008][86122] Updated weights for policy 1, policy_version 54030 (0.0009) +[2023-10-09 14:20:24,201][86121] Updated weights for policy 0, policy_version 53810 (0.0007) +[2023-10-09 14:20:24,381][86122] Updated weights for policy 1, policy_version 54040 (0.0007) +[2023-10-09 14:20:24,564][86121] Updated weights for policy 0, policy_version 53820 (0.0007) +[2023-10-09 14:20:27,998][86122] Updated weights for policy 1, policy_version 54050 (0.0008) +[2023-10-09 14:20:28,353][86122] Updated weights for policy 1, policy_version 54060 (0.0008) +[2023-10-09 14:20:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 110460928. Throughput: 0: 1828.7, 1: 1828.5. Samples: 27624580. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 14:20:28,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.990')] +[2023-10-09 14:20:28,408][86121] Updated weights for policy 0, policy_version 53830 (0.0008) +[2023-10-09 14:20:28,713][86122] Updated weights for policy 1, policy_version 54070 (0.0008) +[2023-10-09 14:20:28,771][86121] Updated weights for policy 0, policy_version 53840 (0.0007) +[2023-10-09 14:20:29,070][86122] Updated weights for policy 1, policy_version 54080 (0.0009) +[2023-10-09 14:20:29,124][86121] Updated weights for policy 0, policy_version 53850 (0.0010) +[2023-10-09 14:20:32,806][86121] Updated weights for policy 0, policy_version 53860 (0.0010) +[2023-10-09 14:20:32,828][86122] Updated weights for policy 1, policy_version 54090 (0.0007) +[2023-10-09 14:20:33,166][86121] Updated weights for policy 0, policy_version 53870 (0.0010) +[2023-10-09 14:20:33,194][86122] Updated weights for policy 1, policy_version 54100 (0.0007) +[2023-10-09 14:20:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110526464. Throughput: 0: 1820.3, 1: 1825.7. Samples: 27647068. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 14:20:33,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.990')] +[2023-10-09 14:20:33,535][86121] Updated weights for policy 0, policy_version 53880 (0.0007) +[2023-10-09 14:20:33,553][86122] Updated weights for policy 1, policy_version 54110 (0.0008) +[2023-10-09 14:20:37,261][86121] Updated weights for policy 0, policy_version 53890 (0.0008) +[2023-10-09 14:20:37,325][86122] Updated weights for policy 1, policy_version 54120 (0.0009) +[2023-10-09 14:20:37,630][86121] Updated weights for policy 0, policy_version 53900 (0.0007) +[2023-10-09 14:20:37,692][86122] Updated weights for policy 1, policy_version 54130 (0.0009) +[2023-10-09 14:20:37,999][86121] Updated weights for policy 0, policy_version 53910 (0.0007) +[2023-10-09 14:20:38,053][86122] Updated weights for policy 1, policy_version 54140 (0.0010) +[2023-10-09 14:20:38,367][86121] Updated weights for policy 0, policy_version 53920 (0.0009) +[2023-10-09 14:20:38,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 110657536. Throughput: 0: 1822.8, 1: 1825.9. Samples: 27668186. Policy #0 lag: (min: 31.0, avg: 36.0, max: 63.0) +[2023-10-09 14:20:38,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.990')] +[2023-10-09 14:20:41,748][86122] Updated weights for policy 1, policy_version 54150 (0.0007) +[2023-10-09 14:20:42,117][86122] Updated weights for policy 1, policy_version 54160 (0.0008) +[2023-10-09 14:20:42,178][86121] Updated weights for policy 0, policy_version 53930 (0.0009) +[2023-10-09 14:20:42,477][86122] Updated weights for policy 1, policy_version 54170 (0.0007) +[2023-10-09 14:20:42,546][86121] Updated weights for policy 0, policy_version 53940 (0.0007) +[2023-10-09 14:20:42,917][86121] Updated weights for policy 0, policy_version 53950 (0.0008) +[2023-10-09 14:20:43,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 110723072. Throughput: 0: 1819.1, 1: 1819.8. Samples: 27679782. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:20:43,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.970')] +[2023-10-09 14:20:46,214][86122] Updated weights for policy 1, policy_version 54180 (0.0007) +[2023-10-09 14:20:46,562][86121] Updated weights for policy 0, policy_version 53960 (0.0007) +[2023-10-09 14:20:46,579][86122] Updated weights for policy 1, policy_version 54190 (0.0008) +[2023-10-09 14:20:46,918][86121] Updated weights for policy 0, policy_version 53970 (0.0007) +[2023-10-09 14:20:46,938][86122] Updated weights for policy 1, policy_version 54200 (0.0008) +[2023-10-09 14:20:47,282][86121] Updated weights for policy 0, policy_version 53980 (0.0007) +[2023-10-09 14:20:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 110788608. Throughput: 0: 1823.2, 1: 1826.9. Samples: 27700890. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:20:48,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:20:50,757][86122] Updated weights for policy 1, policy_version 54210 (0.0008) +[2023-10-09 14:20:51,121][86122] Updated weights for policy 1, policy_version 54220 (0.0009) +[2023-10-09 14:20:51,130][86121] Updated weights for policy 0, policy_version 53990 (0.0007) +[2023-10-09 14:20:51,484][86122] Updated weights for policy 1, policy_version 54230 (0.0007) +[2023-10-09 14:20:51,506][86121] Updated weights for policy 0, policy_version 54000 (0.0007) +[2023-10-09 14:20:51,843][86122] Updated weights for policy 1, policy_version 54240 (0.0008) +[2023-10-09 14:20:51,869][86121] Updated weights for policy 0, policy_version 54010 (0.0009) +[2023-10-09 14:20:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 110854144. Throughput: 0: 1808.7, 1: 1817.6. Samples: 27722102. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:20:53,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:20:55,462][86122] Updated weights for policy 1, policy_version 54250 (0.0008) +[2023-10-09 14:20:55,540][86121] Updated weights for policy 0, policy_version 54020 (0.0008) +[2023-10-09 14:20:55,832][86122] Updated weights for policy 1, policy_version 54260 (0.0008) +[2023-10-09 14:20:55,906][86121] Updated weights for policy 0, policy_version 54030 (0.0007) +[2023-10-09 14:20:56,198][86122] Updated weights for policy 1, policy_version 54270 (0.0009) +[2023-10-09 14:20:56,277][86121] Updated weights for policy 0, policy_version 54040 (0.0009) +[2023-10-09 14:20:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110919680. Throughput: 0: 1822.6, 1: 1818.1. Samples: 27733404. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:20:58,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:20:59,860][86122] Updated weights for policy 1, policy_version 54280 (0.0010) +[2023-10-09 14:21:00,020][86121] Updated weights for policy 0, policy_version 54050 (0.0007) +[2023-10-09 14:21:00,223][86122] Updated weights for policy 1, policy_version 54290 (0.0009) +[2023-10-09 14:21:00,388][86121] Updated weights for policy 0, policy_version 54060 (0.0007) +[2023-10-09 14:21:00,581][86122] Updated weights for policy 1, policy_version 54300 (0.0007) +[2023-10-09 14:21:00,763][86121] Updated weights for policy 0, policy_version 54070 (0.0009) +[2023-10-09 14:21:01,119][86121] Updated weights for policy 0, policy_version 54080 (0.0008) +[2023-10-09 14:21:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 110985216. Throughput: 0: 1806.1, 1: 1807.7. Samples: 27754662. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:21:03,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:21:04,498][86122] Updated weights for policy 1, policy_version 54310 (0.0009) +[2023-10-09 14:21:04,767][86121] Updated weights for policy 0, policy_version 54090 (0.0008) +[2023-10-09 14:21:04,864][86122] Updated weights for policy 1, policy_version 54320 (0.0009) +[2023-10-09 14:21:05,128][86121] Updated weights for policy 0, policy_version 54100 (0.0008) +[2023-10-09 14:21:05,211][86122] Updated weights for policy 1, policy_version 54330 (0.0009) +[2023-10-09 14:21:05,495][86121] Updated weights for policy 0, policy_version 54110 (0.0008) +[2023-10-09 14:21:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111050752. Throughput: 0: 1803.1, 1: 1812.2. Samples: 27777626. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:21:08,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:21:08,772][86122] Updated weights for policy 1, policy_version 54340 (0.0008) +[2023-10-09 14:21:09,148][86122] Updated weights for policy 1, policy_version 54350 (0.0007) +[2023-10-09 14:21:09,179][86121] Updated weights for policy 0, policy_version 54120 (0.0008) +[2023-10-09 14:21:09,507][86122] Updated weights for policy 1, policy_version 54360 (0.0009) +[2023-10-09 14:21:09,545][86121] Updated weights for policy 0, policy_version 54130 (0.0009) +[2023-10-09 14:21:09,904][86121] Updated weights for policy 0, policy_version 54140 (0.0009) +[2023-10-09 14:21:13,084][86122] Updated weights for policy 1, policy_version 54370 (0.0008) +[2023-10-09 14:21:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111116288. Throughput: 0: 1801.9, 1: 1816.4. Samples: 27787402. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:21:13,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:21:13,445][86122] Updated weights for policy 1, policy_version 54380 (0.0007) +[2023-10-09 14:21:13,799][86122] Updated weights for policy 1, policy_version 54390 (0.0008) +[2023-10-09 14:21:13,801][86121] Updated weights for policy 0, policy_version 54150 (0.0008) +[2023-10-09 14:21:14,161][86122] Updated weights for policy 1, policy_version 54400 (0.0008) +[2023-10-09 14:21:14,166][86121] Updated weights for policy 0, policy_version 54160 (0.0007) +[2023-10-09 14:21:14,537][86121] Updated weights for policy 0, policy_version 54170 (0.0008) +[2023-10-09 14:21:17,862][86122] Updated weights for policy 1, policy_version 54410 (0.0011) +[2023-10-09 14:21:18,224][86122] Updated weights for policy 1, policy_version 54420 (0.0010) +[2023-10-09 14:21:18,251][86121] Updated weights for policy 0, policy_version 54180 (0.0009) +[2023-10-09 14:21:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 111181824. Throughput: 0: 1805.5, 1: 1819.7. Samples: 27810202. Policy #0 lag: (min: 8.0, avg: 34.7, max: 40.0) +[2023-10-09 14:21:18,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:21:18,594][86122] Updated weights for policy 1, policy_version 54430 (0.0008) +[2023-10-09 14:21:18,608][86121] Updated weights for policy 0, policy_version 54190 (0.0008) +[2023-10-09 14:21:18,970][86121] Updated weights for policy 0, policy_version 54200 (0.0007) +[2023-10-09 14:21:22,289][86122] Updated weights for policy 1, policy_version 54440 (0.0007) +[2023-10-09 14:21:22,648][86122] Updated weights for policy 1, policy_version 54450 (0.0007) +[2023-10-09 14:21:22,704][86121] Updated weights for policy 0, policy_version 54210 (0.0008) +[2023-10-09 14:21:23,015][86122] Updated weights for policy 1, policy_version 54460 (0.0007) +[2023-10-09 14:21:23,061][86121] Updated weights for policy 0, policy_version 54220 (0.0009) +[2023-10-09 14:21:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 111280128. Throughput: 0: 1810.8, 1: 1819.9. Samples: 27831566. Policy #0 lag: (min: 27.0, avg: 36.2, max: 59.0) +[2023-10-09 14:21:23,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.960')] +[2023-10-09 14:21:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000054464_55771136.pth... +[2023-10-09 14:21:23,425][86121] Updated weights for policy 0, policy_version 54230 (0.0009) +[2023-10-09 14:21:23,437][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000052736_54001664.pth +[2023-10-09 14:21:23,788][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000054240_55541760.pth... +[2023-10-09 14:21:23,790][86121] Updated weights for policy 0, policy_version 54240 (0.0010) +[2023-10-09 14:21:23,817][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000052512_53772288.pth +[2023-10-09 14:21:26,641][86122] Updated weights for policy 1, policy_version 54470 (0.0009) +[2023-10-09 14:21:27,002][86122] Updated weights for policy 1, policy_version 54480 (0.0010) +[2023-10-09 14:21:27,285][86121] Updated weights for policy 0, policy_version 54250 (0.0008) +[2023-10-09 14:21:27,370][86122] Updated weights for policy 1, policy_version 54490 (0.0009) +[2023-10-09 14:21:27,648][86121] Updated weights for policy 0, policy_version 54260 (0.0008) +[2023-10-09 14:21:28,022][86121] Updated weights for policy 0, policy_version 54270 (0.0008) +[2023-10-09 14:21:28,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 111378432. Throughput: 0: 1803.0, 1: 1822.2. Samples: 27842916. Policy #0 lag: (min: 27.0, avg: 36.2, max: 59.0) +[2023-10-09 14:21:28,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.970')] +[2023-10-09 14:21:31,061][86122] Updated weights for policy 1, policy_version 54500 (0.0010) +[2023-10-09 14:21:31,420][86122] Updated weights for policy 1, policy_version 54510 (0.0009) +[2023-10-09 14:21:31,712][86121] Updated weights for policy 0, policy_version 54280 (0.0008) +[2023-10-09 14:21:31,787][86122] Updated weights for policy 1, policy_version 54520 (0.0009) +[2023-10-09 14:21:32,075][86121] Updated weights for policy 0, policy_version 54290 (0.0007) +[2023-10-09 14:21:32,439][86121] Updated weights for policy 0, policy_version 54300 (0.0007) +[2023-10-09 14:21:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 111443968. Throughput: 0: 1807.1, 1: 1820.6. Samples: 27864136. Policy #0 lag: (min: 27.0, avg: 36.2, max: 59.0) +[2023-10-09 14:21:33,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:21:35,417][86122] Updated weights for policy 1, policy_version 54530 (0.0008) +[2023-10-09 14:21:35,772][86122] Updated weights for policy 1, policy_version 54540 (0.0009) +[2023-10-09 14:21:36,140][86122] Updated weights for policy 1, policy_version 54550 (0.0009) +[2023-10-09 14:21:36,442][86121] Updated weights for policy 0, policy_version 54310 (0.0008) +[2023-10-09 14:21:36,502][86122] Updated weights for policy 1, policy_version 54560 (0.0007) +[2023-10-09 14:21:36,828][86121] Updated weights for policy 0, policy_version 54320 (0.0008) +[2023-10-09 14:21:37,202][86121] Updated weights for policy 0, policy_version 54330 (0.0009) +[2023-10-09 14:21:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 111509504. Throughput: 0: 1802.5, 1: 1832.8. Samples: 27885690. Policy #0 lag: (min: 27.0, avg: 36.2, max: 59.0) +[2023-10-09 14:21:38,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:21:40,104][86122] Updated weights for policy 1, policy_version 54570 (0.0007) +[2023-10-09 14:21:40,466][86122] Updated weights for policy 1, policy_version 54580 (0.0007) +[2023-10-09 14:21:40,820][86122] Updated weights for policy 1, policy_version 54590 (0.0008) +[2023-10-09 14:21:41,003][86121] Updated weights for policy 0, policy_version 54340 (0.0008) +[2023-10-09 14:21:41,359][86121] Updated weights for policy 0, policy_version 54350 (0.0008) +[2023-10-09 14:21:41,727][86121] Updated weights for policy 0, policy_version 54360 (0.0009) +[2023-10-09 14:21:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 111575040. Throughput: 0: 1809.4, 1: 1827.3. Samples: 27897056. Policy #0 lag: (min: 27.0, avg: 36.2, max: 59.0) +[2023-10-09 14:21:43,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.960')] +[2023-10-09 14:21:44,495][86122] Updated weights for policy 1, policy_version 54600 (0.0008) +[2023-10-09 14:21:44,848][86122] Updated weights for policy 1, policy_version 54610 (0.0007) +[2023-10-09 14:21:45,207][86122] Updated weights for policy 1, policy_version 54620 (0.0008) +[2023-10-09 14:21:45,385][86121] Updated weights for policy 0, policy_version 54370 (0.0008) +[2023-10-09 14:21:45,760][86121] Updated weights for policy 0, policy_version 54380 (0.0008) +[2023-10-09 14:21:46,124][86121] Updated weights for policy 0, policy_version 54390 (0.0008) +[2023-10-09 14:21:46,490][86121] Updated weights for policy 0, policy_version 54400 (0.0008) +[2023-10-09 14:21:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 111640576. Throughput: 0: 1799.3, 1: 1842.9. Samples: 27918562. Policy #0 lag: (min: 27.0, avg: 36.2, max: 59.0) +[2023-10-09 14:21:48,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.960')] +[2023-10-09 14:21:48,768][86122] Updated weights for policy 1, policy_version 54630 (0.0009) +[2023-10-09 14:21:49,127][86122] Updated weights for policy 1, policy_version 54640 (0.0008) +[2023-10-09 14:21:49,485][86122] Updated weights for policy 1, policy_version 54650 (0.0009) +[2023-10-09 14:21:50,138][86121] Updated weights for policy 0, policy_version 54410 (0.0009) +[2023-10-09 14:21:50,514][86121] Updated weights for policy 0, policy_version 54420 (0.0012) +[2023-10-09 14:21:50,876][86121] Updated weights for policy 0, policy_version 54430 (0.0009) +[2023-10-09 14:21:53,116][86122] Updated weights for policy 1, policy_version 54660 (0.0009) +[2023-10-09 14:21:53,398][85186] Fps is (10 sec: 13106.6, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 111706112. Throughput: 0: 1803.6, 1: 1842.5. Samples: 27941702. Policy #0 lag: (min: 27.0, avg: 36.2, max: 59.0) +[2023-10-09 14:21:53,399][85186] Avg episode reward: [(0, '9.920'), (1, '9.960')] +[2023-10-09 14:21:53,485][86122] Updated weights for policy 1, policy_version 54670 (0.0007) +[2023-10-09 14:21:53,841][86122] Updated weights for policy 1, policy_version 54680 (0.0008) +[2023-10-09 14:21:54,377][86121] Updated weights for policy 0, policy_version 54440 (0.0008) +[2023-10-09 14:21:54,748][86121] Updated weights for policy 0, policy_version 54450 (0.0008) +[2023-10-09 14:21:55,118][86121] Updated weights for policy 0, policy_version 54460 (0.0007) +[2023-10-09 14:21:57,637][86122] Updated weights for policy 1, policy_version 54690 (0.0008) +[2023-10-09 14:21:58,056][86122] Updated weights for policy 1, policy_version 54700 (0.0008) +[2023-10-09 14:21:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 111771648. Throughput: 0: 1805.7, 1: 1843.6. Samples: 27951620. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 14:21:58,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.960')] +[2023-10-09 14:21:58,414][86122] Updated weights for policy 1, policy_version 54710 (0.0011) +[2023-10-09 14:21:58,788][86122] Updated weights for policy 1, policy_version 54720 (0.0008) +[2023-10-09 14:21:58,821][86121] Updated weights for policy 0, policy_version 54470 (0.0007) +[2023-10-09 14:21:59,182][86121] Updated weights for policy 0, policy_version 54480 (0.0009) +[2023-10-09 14:21:59,551][86121] Updated weights for policy 0, policy_version 54490 (0.0010) +[2023-10-09 14:22:02,606][86122] Updated weights for policy 1, policy_version 54730 (0.0009) +[2023-10-09 14:22:02,966][86122] Updated weights for policy 1, policy_version 54740 (0.0009) +[2023-10-09 14:22:03,146][86121] Updated weights for policy 0, policy_version 54500 (0.0009) +[2023-10-09 14:22:03,328][86122] Updated weights for policy 1, policy_version 54750 (0.0007) +[2023-10-09 14:22:03,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 111837184. Throughput: 0: 1809.2, 1: 1830.9. Samples: 27974008. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 14:22:03,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.960')] +[2023-10-09 14:22:03,517][86121] Updated weights for policy 0, policy_version 54510 (0.0007) +[2023-10-09 14:22:03,883][86121] Updated weights for policy 0, policy_version 54520 (0.0008) +[2023-10-09 14:22:06,885][86122] Updated weights for policy 1, policy_version 54760 (0.0007) +[2023-10-09 14:22:07,251][86122] Updated weights for policy 1, policy_version 54770 (0.0007) +[2023-10-09 14:22:07,609][86122] Updated weights for policy 1, policy_version 54780 (0.0008) +[2023-10-09 14:22:07,688][86121] Updated weights for policy 0, policy_version 54530 (0.0009) +[2023-10-09 14:22:08,063][86121] Updated weights for policy 0, policy_version 54540 (0.0007) +[2023-10-09 14:22:08,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 111935488. Throughput: 0: 1812.2, 1: 1824.6. Samples: 27995222. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 14:22:08,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.950')] +[2023-10-09 14:22:08,427][86121] Updated weights for policy 0, policy_version 54550 (0.0009) +[2023-10-09 14:22:08,788][86121] Updated weights for policy 0, policy_version 54560 (0.0011) +[2023-10-09 14:22:11,254][86122] Updated weights for policy 1, policy_version 54790 (0.0007) +[2023-10-09 14:22:11,623][86122] Updated weights for policy 1, policy_version 54800 (0.0007) +[2023-10-09 14:22:11,986][86122] Updated weights for policy 1, policy_version 54810 (0.0008) +[2023-10-09 14:22:12,559][86121] Updated weights for policy 0, policy_version 54570 (0.0008) +[2023-10-09 14:22:12,924][86121] Updated weights for policy 0, policy_version 54580 (0.0007) +[2023-10-09 14:22:13,303][86121] Updated weights for policy 0, policy_version 54590 (0.0007) +[2023-10-09 14:22:13,397][85186] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 112033792. Throughput: 0: 1804.4, 1: 1839.1. Samples: 28006870. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 14:22:13,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.950')] +[2023-10-09 14:22:15,867][86122] Updated weights for policy 1, policy_version 54820 (0.0008) +[2023-10-09 14:22:16,219][86122] Updated weights for policy 1, policy_version 54830 (0.0008) +[2023-10-09 14:22:16,584][86122] Updated weights for policy 1, policy_version 54840 (0.0008) +[2023-10-09 14:22:17,172][86121] Updated weights for policy 0, policy_version 54600 (0.0008) +[2023-10-09 14:22:17,545][86121] Updated weights for policy 0, policy_version 54610 (0.0009) +[2023-10-09 14:22:17,915][86121] Updated weights for policy 0, policy_version 54620 (0.0007) +[2023-10-09 14:22:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 112099328. Throughput: 0: 1814.9, 1: 1828.8. Samples: 28028104. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 14:22:18,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.950')] +[2023-10-09 14:22:20,251][86122] Updated weights for policy 1, policy_version 54850 (0.0009) +[2023-10-09 14:22:20,615][86122] Updated weights for policy 1, policy_version 54860 (0.0007) +[2023-10-09 14:22:20,986][86122] Updated weights for policy 1, policy_version 54870 (0.0009) +[2023-10-09 14:22:21,346][86122] Updated weights for policy 1, policy_version 54880 (0.0008) +[2023-10-09 14:22:21,701][86121] Updated weights for policy 0, policy_version 54630 (0.0008) +[2023-10-09 14:22:22,068][86121] Updated weights for policy 0, policy_version 54640 (0.0007) +[2023-10-09 14:22:22,435][86121] Updated weights for policy 0, policy_version 54650 (0.0008) +[2023-10-09 14:22:23,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112164864. Throughput: 0: 1809.4, 1: 1833.4. Samples: 28049616. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 14:22:23,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.940')] +[2023-10-09 14:22:24,903][86122] Updated weights for policy 1, policy_version 54890 (0.0008) +[2023-10-09 14:22:25,259][86122] Updated weights for policy 1, policy_version 54900 (0.0007) +[2023-10-09 14:22:25,621][86122] Updated weights for policy 1, policy_version 54910 (0.0009) +[2023-10-09 14:22:26,121][86121] Updated weights for policy 0, policy_version 54660 (0.0007) +[2023-10-09 14:22:26,484][86121] Updated weights for policy 0, policy_version 54670 (0.0007) +[2023-10-09 14:22:26,849][86121] Updated weights for policy 0, policy_version 54680 (0.0008) +[2023-10-09 14:22:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 112230400. Throughput: 0: 1817.3, 1: 1829.3. Samples: 28061154. Policy #0 lag: (min: 7.0, avg: 7.0, max: 7.0) +[2023-10-09 14:22:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.940')] +[2023-10-09 14:22:29,329][86122] Updated weights for policy 1, policy_version 54920 (0.0009) +[2023-10-09 14:22:29,702][86122] Updated weights for policy 1, policy_version 54930 (0.0007) +[2023-10-09 14:22:30,061][86122] Updated weights for policy 1, policy_version 54940 (0.0008) +[2023-10-09 14:22:30,565][86121] Updated weights for policy 0, policy_version 54690 (0.0010) +[2023-10-09 14:22:30,923][86121] Updated weights for policy 0, policy_version 54700 (0.0007) +[2023-10-09 14:22:31,297][86121] Updated weights for policy 0, policy_version 54710 (0.0008) +[2023-10-09 14:22:31,654][86121] Updated weights for policy 0, policy_version 54720 (0.0008) +[2023-10-09 14:22:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 112295936. Throughput: 0: 1812.7, 1: 1830.8. Samples: 28082522. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) +[2023-10-09 14:22:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.950')] +[2023-10-09 14:22:33,719][86122] Updated weights for policy 1, policy_version 54950 (0.0009) +[2023-10-09 14:22:34,089][86122] Updated weights for policy 1, policy_version 54960 (0.0010) +[2023-10-09 14:22:34,437][86122] Updated weights for policy 1, policy_version 54970 (0.0009) +[2023-10-09 14:22:35,286][86121] Updated weights for policy 0, policy_version 54730 (0.0008) +[2023-10-09 14:22:35,653][86121] Updated weights for policy 0, policy_version 54740 (0.0010) +[2023-10-09 14:22:36,022][86121] Updated weights for policy 0, policy_version 54750 (0.0008) +[2023-10-09 14:22:37,974][86122] Updated weights for policy 1, policy_version 54980 (0.0010) +[2023-10-09 14:22:38,337][86122] Updated weights for policy 1, policy_version 54990 (0.0009) +[2023-10-09 14:22:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 112361472. Throughput: 0: 1821.8, 1: 1827.7. Samples: 28105928. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) +[2023-10-09 14:22:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.960')] +[2023-10-09 14:22:38,702][86122] Updated weights for policy 1, policy_version 55000 (0.0007) +[2023-10-09 14:22:39,485][86121] Updated weights for policy 0, policy_version 54760 (0.0010) +[2023-10-09 14:22:39,862][86121] Updated weights for policy 0, policy_version 54770 (0.0009) +[2023-10-09 14:22:40,230][86121] Updated weights for policy 0, policy_version 54780 (0.0008) +[2023-10-09 14:22:42,343][86122] Updated weights for policy 1, policy_version 55010 (0.0010) +[2023-10-09 14:22:42,754][86122] Updated weights for policy 1, policy_version 55020 (0.0009) +[2023-10-09 14:22:43,107][86122] Updated weights for policy 1, policy_version 55030 (0.0008) +[2023-10-09 14:22:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 112427008. Throughput: 0: 1824.2, 1: 1827.6. Samples: 28115952. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) +[2023-10-09 14:22:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:22:43,468][86122] Updated weights for policy 1, policy_version 55040 (0.0007) +[2023-10-09 14:22:43,924][86121] Updated weights for policy 0, policy_version 54790 (0.0008) +[2023-10-09 14:22:44,289][86121] Updated weights for policy 0, policy_version 54800 (0.0010) +[2023-10-09 14:22:44,657][86121] Updated weights for policy 0, policy_version 54810 (0.0011) +[2023-10-09 14:22:47,256][86122] Updated weights for policy 1, policy_version 55050 (0.0009) +[2023-10-09 14:22:47,623][86122] Updated weights for policy 1, policy_version 55060 (0.0009) +[2023-10-09 14:22:47,982][86122] Updated weights for policy 1, policy_version 55070 (0.0009) +[2023-10-09 14:22:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 112525312. Throughput: 0: 1820.3, 1: 1836.0. Samples: 28138540. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) +[2023-10-09 14:22:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:22:48,564][86121] Updated weights for policy 0, policy_version 54820 (0.0011) +[2023-10-09 14:22:48,931][86121] Updated weights for policy 0, policy_version 54830 (0.0009) +[2023-10-09 14:22:49,297][86121] Updated weights for policy 0, policy_version 54840 (0.0007) +[2023-10-09 14:22:51,693][86122] Updated weights for policy 1, policy_version 55080 (0.0010) +[2023-10-09 14:22:52,060][86122] Updated weights for policy 1, policy_version 55090 (0.0010) +[2023-10-09 14:22:52,417][86122] Updated weights for policy 1, policy_version 55100 (0.0009) +[2023-10-09 14:22:52,836][86121] Updated weights for policy 0, policy_version 54850 (0.0007) +[2023-10-09 14:22:53,192][86121] Updated weights for policy 0, policy_version 54860 (0.0010) +[2023-10-09 14:22:53,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 112590848. Throughput: 0: 1826.8, 1: 1830.0. Samples: 28159778. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) +[2023-10-09 14:22:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:22:53,566][86121] Updated weights for policy 0, policy_version 54870 (0.0009) +[2023-10-09 14:22:53,926][86121] Updated weights for policy 0, policy_version 54880 (0.0007) +[2023-10-09 14:22:56,135][86122] Updated weights for policy 1, policy_version 55110 (0.0009) +[2023-10-09 14:22:56,497][86122] Updated weights for policy 1, policy_version 55120 (0.0009) +[2023-10-09 14:22:56,864][86122] Updated weights for policy 1, policy_version 55130 (0.0008) +[2023-10-09 14:22:57,572][86121] Updated weights for policy 0, policy_version 54890 (0.0010) +[2023-10-09 14:22:57,944][86121] Updated weights for policy 0, policy_version 54900 (0.0007) +[2023-10-09 14:22:58,313][86121] Updated weights for policy 0, policy_version 54910 (0.0011) +[2023-10-09 14:22:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 112689152. Throughput: 0: 1828.0, 1: 1827.9. Samples: 28171384. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) +[2023-10-09 14:22:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 14:23:00,630][86122] Updated weights for policy 1, policy_version 55140 (0.0009) +[2023-10-09 14:23:00,991][86122] Updated weights for policy 1, policy_version 55150 (0.0009) +[2023-10-09 14:23:01,354][86122] Updated weights for policy 1, policy_version 55160 (0.0008) +[2023-10-09 14:23:02,106][86121] Updated weights for policy 0, policy_version 54920 (0.0009) +[2023-10-09 14:23:02,477][86121] Updated weights for policy 0, policy_version 54930 (0.0010) +[2023-10-09 14:23:02,845][86121] Updated weights for policy 0, policy_version 54940 (0.0007) +[2023-10-09 14:23:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 112754688. Throughput: 0: 1823.2, 1: 1826.2. Samples: 28192324. Policy #0 lag: (min: 29.0, avg: 37.0, max: 61.0) +[2023-10-09 14:23:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 14:23:05,007][86122] Updated weights for policy 1, policy_version 55170 (0.0007) +[2023-10-09 14:23:05,376][86122] Updated weights for policy 1, policy_version 55180 (0.0007) +[2023-10-09 14:23:05,737][86122] Updated weights for policy 1, policy_version 55190 (0.0010) +[2023-10-09 14:23:06,095][86122] Updated weights for policy 1, policy_version 55200 (0.0008) +[2023-10-09 14:23:06,685][86121] Updated weights for policy 0, policy_version 54950 (0.0007) +[2023-10-09 14:23:07,073][86121] Updated weights for policy 0, policy_version 54960 (0.0008) +[2023-10-09 14:23:07,429][86121] Updated weights for policy 0, policy_version 54970 (0.0008) +[2023-10-09 14:23:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 112820224. Throughput: 0: 1815.4, 1: 1830.1. Samples: 28213662. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 14:23:09,767][86122] Updated weights for policy 1, policy_version 55210 (0.0007) +[2023-10-09 14:23:10,123][86122] Updated weights for policy 1, policy_version 55220 (0.0009) +[2023-10-09 14:23:10,481][86122] Updated weights for policy 1, policy_version 55230 (0.0008) +[2023-10-09 14:23:11,073][86121] Updated weights for policy 0, policy_version 54980 (0.0008) +[2023-10-09 14:23:11,442][86121] Updated weights for policy 0, policy_version 54990 (0.0009) +[2023-10-09 14:23:11,801][86121] Updated weights for policy 0, policy_version 55000 (0.0008) +[2023-10-09 14:23:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112885760. Throughput: 0: 1813.7, 1: 1827.6. Samples: 28225014. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 14:23:14,197][86122] Updated weights for policy 1, policy_version 55240 (0.0009) +[2023-10-09 14:23:14,557][86122] Updated weights for policy 1, policy_version 55250 (0.0010) +[2023-10-09 14:23:14,925][86122] Updated weights for policy 1, policy_version 55260 (0.0010) +[2023-10-09 14:23:15,485][86121] Updated weights for policy 0, policy_version 55010 (0.0008) +[2023-10-09 14:23:15,848][86121] Updated weights for policy 0, policy_version 55020 (0.0009) +[2023-10-09 14:23:16,210][86121] Updated weights for policy 0, policy_version 55030 (0.0010) +[2023-10-09 14:23:16,578][86121] Updated weights for policy 0, policy_version 55040 (0.0010) +[2023-10-09 14:23:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 112951296. Throughput: 0: 1817.7, 1: 1830.1. Samples: 28246674. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 14:23:18,497][86122] Updated weights for policy 1, policy_version 55270 (0.0009) +[2023-10-09 14:23:18,861][86122] Updated weights for policy 1, policy_version 55280 (0.0007) +[2023-10-09 14:23:19,221][86122] Updated weights for policy 1, policy_version 55290 (0.0007) +[2023-10-09 14:23:20,281][86121] Updated weights for policy 0, policy_version 55050 (0.0008) +[2023-10-09 14:23:20,637][86121] Updated weights for policy 0, policy_version 55060 (0.0008) +[2023-10-09 14:23:21,000][86121] Updated weights for policy 0, policy_version 55070 (0.0010) +[2023-10-09 14:23:22,818][86122] Updated weights for policy 1, policy_version 55300 (0.0007) +[2023-10-09 14:23:23,180][86122] Updated weights for policy 1, policy_version 55310 (0.0008) +[2023-10-09 14:23:23,398][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 113016832. Throughput: 0: 1807.1, 1: 1826.8. Samples: 28269452. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:23,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 14:23:23,411][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000055072_56393728.pth... +[2023-10-09 14:23:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000053376_54657024.pth +[2023-10-09 14:23:23,534][86122] Updated weights for policy 1, policy_version 55320 (0.0008) +[2023-10-09 14:23:23,819][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000055328_56655872.pth... +[2023-10-09 14:23:23,847][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000053600_54886400.pth +[2023-10-09 14:23:24,742][86121] Updated weights for policy 0, policy_version 55080 (0.0008) +[2023-10-09 14:23:25,108][86121] Updated weights for policy 0, policy_version 55090 (0.0007) +[2023-10-09 14:23:25,470][86121] Updated weights for policy 0, policy_version 55100 (0.0009) +[2023-10-09 14:23:27,390][86122] Updated weights for policy 1, policy_version 55330 (0.0009) +[2023-10-09 14:23:27,753][86122] Updated weights for policy 1, policy_version 55340 (0.0009) +[2023-10-09 14:23:28,118][86122] Updated weights for policy 1, policy_version 55350 (0.0010) +[2023-10-09 14:23:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113082368. Throughput: 0: 1803.9, 1: 1830.7. Samples: 28279508. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:23:28,476][86122] Updated weights for policy 1, policy_version 55360 (0.0009) +[2023-10-09 14:23:29,317][86121] Updated weights for policy 0, policy_version 55110 (0.0009) +[2023-10-09 14:23:29,681][86121] Updated weights for policy 0, policy_version 55120 (0.0011) +[2023-10-09 14:23:30,045][86121] Updated weights for policy 0, policy_version 55130 (0.0011) +[2023-10-09 14:23:32,217][86122] Updated weights for policy 1, policy_version 55370 (0.0009) +[2023-10-09 14:23:32,570][86122] Updated weights for policy 1, policy_version 55380 (0.0009) +[2023-10-09 14:23:32,936][86122] Updated weights for policy 1, policy_version 55390 (0.0008) +[2023-10-09 14:23:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113180672. Throughput: 0: 1803.4, 1: 1830.6. Samples: 28302072. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:23:33,674][86121] Updated weights for policy 0, policy_version 55140 (0.0008) +[2023-10-09 14:23:34,043][86121] Updated weights for policy 0, policy_version 55150 (0.0009) +[2023-10-09 14:23:34,405][86121] Updated weights for policy 0, policy_version 55160 (0.0009) +[2023-10-09 14:23:36,526][86122] Updated weights for policy 1, policy_version 55400 (0.0007) +[2023-10-09 14:23:36,889][86122] Updated weights for policy 1, policy_version 55410 (0.0007) +[2023-10-09 14:23:37,241][86122] Updated weights for policy 1, policy_version 55420 (0.0010) +[2023-10-09 14:23:38,070][86121] Updated weights for policy 0, policy_version 55170 (0.0010) +[2023-10-09 14:23:38,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113246208. Throughput: 0: 1812.0, 1: 1832.1. Samples: 28323762. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:23:38,430][86121] Updated weights for policy 0, policy_version 55180 (0.0010) +[2023-10-09 14:23:38,794][86121] Updated weights for policy 0, policy_version 55190 (0.0008) +[2023-10-09 14:23:39,162][86121] Updated weights for policy 0, policy_version 55200 (0.0009) +[2023-10-09 14:23:40,875][86122] Updated weights for policy 1, policy_version 55430 (0.0009) +[2023-10-09 14:23:41,243][86122] Updated weights for policy 1, policy_version 55440 (0.0008) +[2023-10-09 14:23:41,595][86122] Updated weights for policy 1, policy_version 55450 (0.0010) +[2023-10-09 14:23:42,932][86121] Updated weights for policy 0, policy_version 55210 (0.0008) +[2023-10-09 14:23:43,303][86121] Updated weights for policy 0, policy_version 55220 (0.0007) +[2023-10-09 14:23:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113311744. Throughput: 0: 1805.2, 1: 1824.9. Samples: 28334740. Policy #0 lag: (min: 15.0, avg: 18.1, max: 47.0) +[2023-10-09 14:23:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:23:43,670][86121] Updated weights for policy 0, policy_version 55230 (0.0011) +[2023-10-09 14:23:45,327][86122] Updated weights for policy 1, policy_version 55460 (0.0009) +[2023-10-09 14:23:45,680][86122] Updated weights for policy 1, policy_version 55470 (0.0008) +[2023-10-09 14:23:46,041][86122] Updated weights for policy 1, policy_version 55480 (0.0008) +[2023-10-09 14:23:47,321][86121] Updated weights for policy 0, policy_version 55240 (0.0008) +[2023-10-09 14:23:47,678][86121] Updated weights for policy 0, policy_version 55250 (0.0007) +[2023-10-09 14:23:48,056][86121] Updated weights for policy 0, policy_version 55260 (0.0009) +[2023-10-09 14:23:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113410048. Throughput: 0: 1814.7, 1: 1830.7. Samples: 28356364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:23:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:23:49,580][86122] Updated weights for policy 1, policy_version 55490 (0.0009) +[2023-10-09 14:23:49,936][86122] Updated weights for policy 1, policy_version 55500 (0.0011) +[2023-10-09 14:23:50,294][86122] Updated weights for policy 1, policy_version 55510 (0.0009) +[2023-10-09 14:23:50,659][86122] Updated weights for policy 1, policy_version 55520 (0.0008) +[2023-10-09 14:23:51,756][86121] Updated weights for policy 0, policy_version 55270 (0.0009) +[2023-10-09 14:23:52,119][86121] Updated weights for policy 0, policy_version 55280 (0.0010) +[2023-10-09 14:23:52,488][86121] Updated weights for policy 0, policy_version 55290 (0.0008) +[2023-10-09 14:23:53,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 113475584. Throughput: 0: 1821.3, 1: 1834.1. Samples: 28378156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:23:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:23:54,385][86122] Updated weights for policy 1, policy_version 55530 (0.0008) +[2023-10-09 14:23:54,756][86122] Updated weights for policy 1, policy_version 55540 (0.0008) +[2023-10-09 14:23:55,119][86122] Updated weights for policy 1, policy_version 55550 (0.0007) +[2023-10-09 14:23:56,153][86121] Updated weights for policy 0, policy_version 55300 (0.0007) +[2023-10-09 14:23:56,514][86121] Updated weights for policy 0, policy_version 55310 (0.0010) +[2023-10-09 14:23:56,882][86121] Updated weights for policy 0, policy_version 55320 (0.0010) +[2023-10-09 14:23:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 113541120. Throughput: 0: 1818.2, 1: 1831.6. Samples: 28389256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:23:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:23:58,854][86122] Updated weights for policy 1, policy_version 55560 (0.0007) +[2023-10-09 14:23:59,215][86122] Updated weights for policy 1, policy_version 55570 (0.0008) +[2023-10-09 14:23:59,581][86122] Updated weights for policy 1, policy_version 55580 (0.0009) +[2023-10-09 14:24:00,487][86121] Updated weights for policy 0, policy_version 55330 (0.0010) +[2023-10-09 14:24:00,860][86121] Updated weights for policy 0, policy_version 55340 (0.0008) +[2023-10-09 14:24:01,230][86121] Updated weights for policy 0, policy_version 55350 (0.0007) +[2023-10-09 14:24:01,591][86121] Updated weights for policy 0, policy_version 55360 (0.0007) +[2023-10-09 14:24:03,366][86122] Updated weights for policy 1, policy_version 55590 (0.0007) +[2023-10-09 14:24:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113606656. Throughput: 0: 1814.1, 1: 1827.8. Samples: 28410558. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:24:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:03,725][86122] Updated weights for policy 1, policy_version 55600 (0.0009) +[2023-10-09 14:24:04,094][86122] Updated weights for policy 1, policy_version 55610 (0.0007) +[2023-10-09 14:24:05,182][86121] Updated weights for policy 0, policy_version 55370 (0.0008) +[2023-10-09 14:24:05,548][86121] Updated weights for policy 0, policy_version 55380 (0.0010) +[2023-10-09 14:24:05,922][86121] Updated weights for policy 0, policy_version 55390 (0.0008) +[2023-10-09 14:24:07,824][86122] Updated weights for policy 1, policy_version 55620 (0.0008) +[2023-10-09 14:24:08,182][86122] Updated weights for policy 1, policy_version 55630 (0.0010) +[2023-10-09 14:24:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 113672192. Throughput: 0: 1823.6, 1: 1821.2. Samples: 28433470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:24:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:24:08,551][86122] Updated weights for policy 1, policy_version 55640 (0.0009) +[2023-10-09 14:24:09,596][86121] Updated weights for policy 0, policy_version 55400 (0.0008) +[2023-10-09 14:24:09,962][86121] Updated weights for policy 0, policy_version 55410 (0.0007) +[2023-10-09 14:24:10,332][86121] Updated weights for policy 0, policy_version 55420 (0.0010) +[2023-10-09 14:24:12,439][86122] Updated weights for policy 1, policy_version 55650 (0.0008) +[2023-10-09 14:24:12,841][86122] Updated weights for policy 1, policy_version 55660 (0.0011) +[2023-10-09 14:24:13,206][86122] Updated weights for policy 1, policy_version 55670 (0.0011) +[2023-10-09 14:24:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 113737728. Throughput: 0: 1825.4, 1: 1818.6. Samples: 28443490. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:24:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:24:13,566][86122] Updated weights for policy 1, policy_version 55680 (0.0011) +[2023-10-09 14:24:14,030][86121] Updated weights for policy 0, policy_version 55430 (0.0011) +[2023-10-09 14:24:14,401][86121] Updated weights for policy 0, policy_version 55440 (0.0009) +[2023-10-09 14:24:14,773][86121] Updated weights for policy 0, policy_version 55450 (0.0008) +[2023-10-09 14:24:17,258][86122] Updated weights for policy 1, policy_version 55690 (0.0007) +[2023-10-09 14:24:17,631][86122] Updated weights for policy 1, policy_version 55700 (0.0008) +[2023-10-09 14:24:17,995][86122] Updated weights for policy 1, policy_version 55710 (0.0008) +[2023-10-09 14:24:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113836032. Throughput: 0: 1830.6, 1: 1810.8. Samples: 28465938. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:24:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:18,495][86121] Updated weights for policy 0, policy_version 55460 (0.0008) +[2023-10-09 14:24:18,873][86121] Updated weights for policy 0, policy_version 55470 (0.0010) +[2023-10-09 14:24:19,241][86121] Updated weights for policy 0, policy_version 55480 (0.0008) +[2023-10-09 14:24:21,719][86122] Updated weights for policy 1, policy_version 55720 (0.0008) +[2023-10-09 14:24:22,081][86122] Updated weights for policy 1, policy_version 55730 (0.0007) +[2023-10-09 14:24:22,457][86122] Updated weights for policy 1, policy_version 55740 (0.0008) +[2023-10-09 14:24:22,868][86121] Updated weights for policy 0, policy_version 55490 (0.0009) +[2023-10-09 14:24:23,242][86121] Updated weights for policy 0, policy_version 55500 (0.0009) +[2023-10-09 14:24:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 113901568. Throughput: 0: 1820.8, 1: 1804.6. Samples: 28486904. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) +[2023-10-09 14:24:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:23,603][86121] Updated weights for policy 0, policy_version 55510 (0.0010) +[2023-10-09 14:24:23,971][86121] Updated weights for policy 0, policy_version 55520 (0.0010) +[2023-10-09 14:24:26,290][86122] Updated weights for policy 1, policy_version 55750 (0.0007) +[2023-10-09 14:24:26,652][86122] Updated weights for policy 1, policy_version 55760 (0.0007) +[2023-10-09 14:24:27,017][86122] Updated weights for policy 1, policy_version 55770 (0.0010) +[2023-10-09 14:24:27,630][86121] Updated weights for policy 0, policy_version 55530 (0.0009) +[2023-10-09 14:24:27,992][86121] Updated weights for policy 0, policy_version 55540 (0.0010) +[2023-10-09 14:24:28,368][86121] Updated weights for policy 0, policy_version 55550 (0.0010) +[2023-10-09 14:24:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 113967104. Throughput: 0: 1828.7, 1: 1812.1. Samples: 28498576. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) +[2023-10-09 14:24:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:30,830][86122] Updated weights for policy 1, policy_version 55780 (0.0008) +[2023-10-09 14:24:31,191][86122] Updated weights for policy 1, policy_version 55790 (0.0011) +[2023-10-09 14:24:31,552][86122] Updated weights for policy 1, policy_version 55800 (0.0010) +[2023-10-09 14:24:32,179][86121] Updated weights for policy 0, policy_version 55560 (0.0008) +[2023-10-09 14:24:32,536][86121] Updated weights for policy 0, policy_version 55570 (0.0011) +[2023-10-09 14:24:32,903][86121] Updated weights for policy 0, policy_version 55580 (0.0007) +[2023-10-09 14:24:33,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114065408. Throughput: 0: 1824.5, 1: 1805.2. Samples: 28519702. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) +[2023-10-09 14:24:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:35,317][86122] Updated weights for policy 1, policy_version 55810 (0.0009) +[2023-10-09 14:24:35,679][86122] Updated weights for policy 1, policy_version 55820 (0.0009) +[2023-10-09 14:24:36,049][86122] Updated weights for policy 1, policy_version 55830 (0.0009) +[2023-10-09 14:24:36,408][86122] Updated weights for policy 1, policy_version 55840 (0.0008) +[2023-10-09 14:24:36,602][86121] Updated weights for policy 0, policy_version 55590 (0.0007) +[2023-10-09 14:24:36,974][86121] Updated weights for policy 0, policy_version 55600 (0.0008) +[2023-10-09 14:24:37,335][86121] Updated weights for policy 0, policy_version 55610 (0.0007) +[2023-10-09 14:24:38,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 114130944. Throughput: 0: 1824.5, 1: 1795.1. Samples: 28541040. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) +[2023-10-09 14:24:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:40,203][86122] Updated weights for policy 1, policy_version 55850 (0.0009) +[2023-10-09 14:24:40,550][86122] Updated weights for policy 1, policy_version 55860 (0.0008) +[2023-10-09 14:24:40,853][86121] Updated weights for policy 0, policy_version 55620 (0.0009) +[2023-10-09 14:24:40,911][86122] Updated weights for policy 1, policy_version 55870 (0.0007) +[2023-10-09 14:24:41,228][86121] Updated weights for policy 0, policy_version 55630 (0.0008) +[2023-10-09 14:24:41,595][86121] Updated weights for policy 0, policy_version 55640 (0.0007) +[2023-10-09 14:24:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 114196480. Throughput: 0: 1822.7, 1: 1802.1. Samples: 28552374. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) +[2023-10-09 14:24:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:44,626][86122] Updated weights for policy 1, policy_version 55880 (0.0009) +[2023-10-09 14:24:44,990][86122] Updated weights for policy 1, policy_version 55890 (0.0009) +[2023-10-09 14:24:45,189][86121] Updated weights for policy 0, policy_version 55650 (0.0007) +[2023-10-09 14:24:45,361][86122] Updated weights for policy 1, policy_version 55900 (0.0007) +[2023-10-09 14:24:45,552][86121] Updated weights for policy 0, policy_version 55660 (0.0009) +[2023-10-09 14:24:45,926][86121] Updated weights for policy 0, policy_version 55670 (0.0008) +[2023-10-09 14:24:46,285][86121] Updated weights for policy 0, policy_version 55680 (0.0007) +[2023-10-09 14:24:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114262016. Throughput: 0: 1834.1, 1: 1797.4. Samples: 28573974. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) +[2023-10-09 14:24:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:24:49,083][86122] Updated weights for policy 1, policy_version 55910 (0.0009) +[2023-10-09 14:24:49,453][86122] Updated weights for policy 1, policy_version 55920 (0.0008) +[2023-10-09 14:24:49,809][86122] Updated weights for policy 1, policy_version 55930 (0.0009) +[2023-10-09 14:24:50,025][86121] Updated weights for policy 0, policy_version 55690 (0.0008) +[2023-10-09 14:24:50,388][86121] Updated weights for policy 0, policy_version 55700 (0.0009) +[2023-10-09 14:24:50,761][86121] Updated weights for policy 0, policy_version 55710 (0.0009) +[2023-10-09 14:24:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 114327552. Throughput: 0: 1829.3, 1: 1804.1. Samples: 28596976. Policy #0 lag: (min: 4.0, avg: 4.0, max: 4.0) +[2023-10-09 14:24:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:24:53,460][86122] Updated weights for policy 1, policy_version 55940 (0.0009) +[2023-10-09 14:24:53,809][86122] Updated weights for policy 1, policy_version 55950 (0.0009) +[2023-10-09 14:24:54,175][86122] Updated weights for policy 1, policy_version 55960 (0.0008) +[2023-10-09 14:24:54,435][86121] Updated weights for policy 0, policy_version 55720 (0.0008) +[2023-10-09 14:24:54,801][86121] Updated weights for policy 0, policy_version 55730 (0.0008) +[2023-10-09 14:24:55,164][86121] Updated weights for policy 0, policy_version 55740 (0.0007) +[2023-10-09 14:24:57,865][86122] Updated weights for policy 1, policy_version 55970 (0.0009) +[2023-10-09 14:24:58,258][86122] Updated weights for policy 1, policy_version 55980 (0.0009) +[2023-10-09 14:24:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114393088. Throughput: 0: 1825.0, 1: 1806.0. Samples: 28606882. Policy #0 lag: (min: 25.0, avg: 31.0, max: 57.0) +[2023-10-09 14:24:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:24:58,623][86122] Updated weights for policy 1, policy_version 55990 (0.0009) +[2023-10-09 14:24:58,949][86121] Updated weights for policy 0, policy_version 55750 (0.0008) +[2023-10-09 14:24:58,979][86122] Updated weights for policy 1, policy_version 56000 (0.0009) +[2023-10-09 14:24:59,317][86121] Updated weights for policy 0, policy_version 55760 (0.0009) +[2023-10-09 14:24:59,692][86121] Updated weights for policy 0, policy_version 55770 (0.0009) +[2023-10-09 14:25:02,590][86122] Updated weights for policy 1, policy_version 56010 (0.0007) +[2023-10-09 14:25:02,953][86122] Updated weights for policy 1, policy_version 56020 (0.0007) +[2023-10-09 14:25:03,314][86122] Updated weights for policy 1, policy_version 56030 (0.0007) +[2023-10-09 14:25:03,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 114491392. Throughput: 0: 1819.7, 1: 1811.3. Samples: 28629330. Policy #0 lag: (min: 25.0, avg: 31.0, max: 57.0) +[2023-10-09 14:25:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:25:03,608][86121] Updated weights for policy 0, policy_version 55780 (0.0009) +[2023-10-09 14:25:03,964][86121] Updated weights for policy 0, policy_version 55790 (0.0009) +[2023-10-09 14:25:04,338][86121] Updated weights for policy 0, policy_version 55800 (0.0009) +[2023-10-09 14:25:06,803][86122] Updated weights for policy 1, policy_version 56040 (0.0008) +[2023-10-09 14:25:07,160][86122] Updated weights for policy 1, policy_version 56050 (0.0009) +[2023-10-09 14:25:07,519][86122] Updated weights for policy 1, policy_version 56060 (0.0009) +[2023-10-09 14:25:08,061][86121] Updated weights for policy 0, policy_version 55810 (0.0009) +[2023-10-09 14:25:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 114556928. Throughput: 0: 1826.6, 1: 1821.6. Samples: 28651072. Policy #0 lag: (min: 25.0, avg: 31.0, max: 57.0) +[2023-10-09 14:25:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:25:08,426][86121] Updated weights for policy 0, policy_version 55820 (0.0008) +[2023-10-09 14:25:08,792][86121] Updated weights for policy 0, policy_version 55830 (0.0009) +[2023-10-09 14:25:09,166][86121] Updated weights for policy 0, policy_version 55840 (0.0007) +[2023-10-09 14:25:11,154][86122] Updated weights for policy 1, policy_version 56070 (0.0010) +[2023-10-09 14:25:11,524][86122] Updated weights for policy 1, policy_version 56080 (0.0007) +[2023-10-09 14:25:11,885][86122] Updated weights for policy 1, policy_version 56090 (0.0009) +[2023-10-09 14:25:12,682][86121] Updated weights for policy 0, policy_version 55850 (0.0010) +[2023-10-09 14:25:13,044][86121] Updated weights for policy 0, policy_version 55860 (0.0009) +[2023-10-09 14:25:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 114622464. Throughput: 0: 1825.2, 1: 1820.3. Samples: 28662624. Policy #0 lag: (min: 25.0, avg: 31.0, max: 57.0) +[2023-10-09 14:25:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:25:13,418][86121] Updated weights for policy 0, policy_version 55870 (0.0008) +[2023-10-09 14:25:15,632][86122] Updated weights for policy 1, policy_version 56100 (0.0010) +[2023-10-09 14:25:15,985][86122] Updated weights for policy 1, policy_version 56110 (0.0008) +[2023-10-09 14:25:16,355][86122] Updated weights for policy 1, policy_version 56120 (0.0008) +[2023-10-09 14:25:17,273][86121] Updated weights for policy 0, policy_version 55880 (0.0007) +[2023-10-09 14:25:17,634][86121] Updated weights for policy 0, policy_version 55890 (0.0008) +[2023-10-09 14:25:17,996][86121] Updated weights for policy 0, policy_version 55900 (0.0010) +[2023-10-09 14:25:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114720768. Throughput: 0: 1824.5, 1: 1827.3. Samples: 28684034. Policy #0 lag: (min: 25.0, avg: 31.0, max: 57.0) +[2023-10-09 14:25:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:25:20,024][86122] Updated weights for policy 1, policy_version 56130 (0.0007) +[2023-10-09 14:25:20,376][86122] Updated weights for policy 1, policy_version 56140 (0.0010) +[2023-10-09 14:25:20,744][86122] Updated weights for policy 1, policy_version 56150 (0.0010) +[2023-10-09 14:25:21,105][86122] Updated weights for policy 1, policy_version 56160 (0.0010) +[2023-10-09 14:25:21,743][86121] Updated weights for policy 0, policy_version 55910 (0.0010) +[2023-10-09 14:25:22,133][86121] Updated weights for policy 0, policy_version 55920 (0.0008) +[2023-10-09 14:25:22,503][86121] Updated weights for policy 0, policy_version 55930 (0.0008) +[2023-10-09 14:25:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114786304. Throughput: 0: 1817.7, 1: 1833.1. Samples: 28705324. Policy #0 lag: (min: 25.0, avg: 31.0, max: 57.0) +[2023-10-09 14:25:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:25:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000056160_57507840.pth... +[2023-10-09 14:25:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000055936_57278464.pth... +[2023-10-09 14:25:23,438][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000054464_55771136.pth +[2023-10-09 14:25:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000054240_55541760.pth +[2023-10-09 14:25:24,728][86122] Updated weights for policy 1, policy_version 56170 (0.0009) +[2023-10-09 14:25:25,096][86122] Updated weights for policy 1, policy_version 56180 (0.0009) +[2023-10-09 14:25:25,453][86122] Updated weights for policy 1, policy_version 56190 (0.0008) +[2023-10-09 14:25:25,952][86121] Updated weights for policy 0, policy_version 55940 (0.0008) +[2023-10-09 14:25:26,320][86121] Updated weights for policy 0, policy_version 55950 (0.0008) +[2023-10-09 14:25:26,684][86121] Updated weights for policy 0, policy_version 55960 (0.0008) +[2023-10-09 14:25:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 114851840. Throughput: 0: 1821.4, 1: 1830.3. Samples: 28716704. Policy #0 lag: (min: 25.0, avg: 31.0, max: 57.0) +[2023-10-09 14:25:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:25:29,145][86122] Updated weights for policy 1, policy_version 56200 (0.0008) +[2023-10-09 14:25:29,508][86122] Updated weights for policy 1, policy_version 56210 (0.0009) +[2023-10-09 14:25:29,869][86122] Updated weights for policy 1, policy_version 56220 (0.0007) +[2023-10-09 14:25:30,429][86121] Updated weights for policy 0, policy_version 55970 (0.0007) +[2023-10-09 14:25:30,796][86121] Updated weights for policy 0, policy_version 55980 (0.0007) +[2023-10-09 14:25:31,165][86121] Updated weights for policy 0, policy_version 55990 (0.0009) +[2023-10-09 14:25:31,534][86121] Updated weights for policy 0, policy_version 56000 (0.0007) +[2023-10-09 14:25:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114917376. Throughput: 0: 1809.4, 1: 1834.0. Samples: 28737924. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:25:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:25:33,592][86122] Updated weights for policy 1, policy_version 56230 (0.0008) +[2023-10-09 14:25:33,955][86122] Updated weights for policy 1, policy_version 56240 (0.0009) +[2023-10-09 14:25:34,315][86122] Updated weights for policy 1, policy_version 56250 (0.0009) +[2023-10-09 14:25:35,090][86121] Updated weights for policy 0, policy_version 56010 (0.0007) +[2023-10-09 14:25:35,459][86121] Updated weights for policy 0, policy_version 56020 (0.0010) +[2023-10-09 14:25:35,821][86121] Updated weights for policy 0, policy_version 56030 (0.0009) +[2023-10-09 14:25:37,972][86122] Updated weights for policy 1, policy_version 56260 (0.0008) +[2023-10-09 14:25:38,334][86122] Updated weights for policy 1, policy_version 56270 (0.0008) +[2023-10-09 14:25:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 114982912. Throughput: 0: 1811.9, 1: 1831.2. Samples: 28760918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:25:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:25:38,703][86122] Updated weights for policy 1, policy_version 56280 (0.0007) +[2023-10-09 14:25:39,594][86121] Updated weights for policy 0, policy_version 56040 (0.0007) +[2023-10-09 14:25:39,952][86121] Updated weights for policy 0, policy_version 56050 (0.0010) +[2023-10-09 14:25:40,312][86121] Updated weights for policy 0, policy_version 56060 (0.0007) +[2023-10-09 14:25:42,468][86122] Updated weights for policy 1, policy_version 56290 (0.0009) +[2023-10-09 14:25:42,882][86122] Updated weights for policy 1, policy_version 56300 (0.0011) +[2023-10-09 14:25:43,254][86122] Updated weights for policy 1, policy_version 56310 (0.0010) +[2023-10-09 14:25:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115048448. Throughput: 0: 1813.7, 1: 1829.8. Samples: 28770840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:25:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:25:43,610][86122] Updated weights for policy 1, policy_version 56320 (0.0009) +[2023-10-09 14:25:44,022][86121] Updated weights for policy 0, policy_version 56070 (0.0009) +[2023-10-09 14:25:44,393][86121] Updated weights for policy 0, policy_version 56080 (0.0008) +[2023-10-09 14:25:44,744][86121] Updated weights for policy 0, policy_version 56090 (0.0008) +[2023-10-09 14:25:47,390][86122] Updated weights for policy 1, policy_version 56330 (0.0009) +[2023-10-09 14:25:47,750][86122] Updated weights for policy 1, policy_version 56340 (0.0009) +[2023-10-09 14:25:48,113][86122] Updated weights for policy 1, policy_version 56350 (0.0008) +[2023-10-09 14:25:48,371][86121] Updated weights for policy 0, policy_version 56100 (0.0008) +[2023-10-09 14:25:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 115146752. Throughput: 0: 1815.9, 1: 1828.3. Samples: 28793324. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:25:48,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:25:48,733][86121] Updated weights for policy 0, policy_version 56110 (0.0008) +[2023-10-09 14:25:49,107][86121] Updated weights for policy 0, policy_version 56120 (0.0009) +[2023-10-09 14:25:51,707][86122] Updated weights for policy 1, policy_version 56360 (0.0008) +[2023-10-09 14:25:52,068][86122] Updated weights for policy 1, policy_version 56370 (0.0009) +[2023-10-09 14:25:52,437][86122] Updated weights for policy 1, policy_version 56380 (0.0010) +[2023-10-09 14:25:52,901][86121] Updated weights for policy 0, policy_version 56130 (0.0009) +[2023-10-09 14:25:53,268][86121] Updated weights for policy 0, policy_version 56140 (0.0007) +[2023-10-09 14:25:53,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 115212288. Throughput: 0: 1809.7, 1: 1824.0. Samples: 28814588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:25:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:25:53,643][86121] Updated weights for policy 0, policy_version 56150 (0.0008) +[2023-10-09 14:25:54,007][86121] Updated weights for policy 0, policy_version 56160 (0.0007) +[2023-10-09 14:25:56,049][86122] Updated weights for policy 1, policy_version 56390 (0.0008) +[2023-10-09 14:25:56,419][86122] Updated weights for policy 1, policy_version 56400 (0.0009) +[2023-10-09 14:25:56,779][86122] Updated weights for policy 1, policy_version 56410 (0.0009) +[2023-10-09 14:25:57,844][86121] Updated weights for policy 0, policy_version 56170 (0.0008) +[2023-10-09 14:25:58,209][86121] Updated weights for policy 0, policy_version 56180 (0.0007) +[2023-10-09 14:25:58,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115277824. Throughput: 0: 1807.2, 1: 1823.9. Samples: 28826024. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:25:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:25:58,573][86121] Updated weights for policy 0, policy_version 56190 (0.0008) +[2023-10-09 14:26:00,593][86122] Updated weights for policy 1, policy_version 56420 (0.0009) +[2023-10-09 14:26:00,960][86122] Updated weights for policy 1, policy_version 56430 (0.0010) +[2023-10-09 14:26:01,330][86122] Updated weights for policy 1, policy_version 56440 (0.0008) +[2023-10-09 14:26:02,226][86121] Updated weights for policy 0, policy_version 56200 (0.0007) +[2023-10-09 14:26:02,596][86121] Updated weights for policy 0, policy_version 56210 (0.0008) +[2023-10-09 14:26:02,975][86121] Updated weights for policy 0, policy_version 56220 (0.0007) +[2023-10-09 14:26:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115376128. Throughput: 0: 1811.4, 1: 1815.1. Samples: 28847226. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:26:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:04,926][86122] Updated weights for policy 1, policy_version 56450 (0.0009) +[2023-10-09 14:26:05,278][86122] Updated weights for policy 1, policy_version 56460 (0.0007) +[2023-10-09 14:26:05,646][86122] Updated weights for policy 1, policy_version 56470 (0.0008) +[2023-10-09 14:26:06,001][86122] Updated weights for policy 1, policy_version 56480 (0.0009) +[2023-10-09 14:26:06,682][86121] Updated weights for policy 0, policy_version 56230 (0.0007) +[2023-10-09 14:26:07,050][86121] Updated weights for policy 0, policy_version 56240 (0.0007) +[2023-10-09 14:26:07,408][86121] Updated weights for policy 0, policy_version 56250 (0.0008) +[2023-10-09 14:26:08,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115441664. Throughput: 0: 1818.8, 1: 1816.0. Samples: 28868892. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 14:26:08,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:09,780][86122] Updated weights for policy 1, policy_version 56490 (0.0010) +[2023-10-09 14:26:10,145][86122] Updated weights for policy 1, policy_version 56500 (0.0008) +[2023-10-09 14:26:10,508][86122] Updated weights for policy 1, policy_version 56510 (0.0007) +[2023-10-09 14:26:11,022][86121] Updated weights for policy 0, policy_version 56260 (0.0008) +[2023-10-09 14:26:11,385][86121] Updated weights for policy 0, policy_version 56270 (0.0008) +[2023-10-09 14:26:11,760][86121] Updated weights for policy 0, policy_version 56280 (0.0010) +[2023-10-09 14:26:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 115507200. Throughput: 0: 1819.6, 1: 1813.2. Samples: 28880176. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 14:26:13,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:14,250][86122] Updated weights for policy 1, policy_version 56520 (0.0008) +[2023-10-09 14:26:14,619][86122] Updated weights for policy 1, policy_version 56530 (0.0010) +[2023-10-09 14:26:14,978][86122] Updated weights for policy 1, policy_version 56540 (0.0007) +[2023-10-09 14:26:15,537][86121] Updated weights for policy 0, policy_version 56290 (0.0008) +[2023-10-09 14:26:15,907][86121] Updated weights for policy 0, policy_version 56300 (0.0007) +[2023-10-09 14:26:16,277][86121] Updated weights for policy 0, policy_version 56310 (0.0007) +[2023-10-09 14:26:16,640][86121] Updated weights for policy 0, policy_version 56320 (0.0008) +[2023-10-09 14:26:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 115572736. Throughput: 0: 1826.9, 1: 1814.2. Samples: 28901776. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 14:26:18,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:18,602][86122] Updated weights for policy 1, policy_version 56550 (0.0007) +[2023-10-09 14:26:18,966][86122] Updated weights for policy 1, policy_version 56560 (0.0007) +[2023-10-09 14:26:19,339][86122] Updated weights for policy 1, policy_version 56570 (0.0008) +[2023-10-09 14:26:20,260][86121] Updated weights for policy 0, policy_version 56330 (0.0008) +[2023-10-09 14:26:20,628][86121] Updated weights for policy 0, policy_version 56340 (0.0008) +[2023-10-09 14:26:20,989][86121] Updated weights for policy 0, policy_version 56350 (0.0007) +[2023-10-09 14:26:23,089][86122] Updated weights for policy 1, policy_version 56580 (0.0007) +[2023-10-09 14:26:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 115638272. Throughput: 0: 1824.6, 1: 1819.2. Samples: 28924890. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 14:26:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:23,451][86122] Updated weights for policy 1, policy_version 56590 (0.0007) +[2023-10-09 14:26:23,814][86122] Updated weights for policy 1, policy_version 56600 (0.0009) +[2023-10-09 14:26:24,607][86121] Updated weights for policy 0, policy_version 56360 (0.0010) +[2023-10-09 14:26:24,967][86121] Updated weights for policy 0, policy_version 56370 (0.0009) +[2023-10-09 14:26:25,342][86121] Updated weights for policy 0, policy_version 56380 (0.0009) +[2023-10-09 14:26:27,443][86122] Updated weights for policy 1, policy_version 56610 (0.0009) +[2023-10-09 14:26:27,822][86122] Updated weights for policy 1, policy_version 56620 (0.0008) +[2023-10-09 14:26:28,181][86122] Updated weights for policy 1, policy_version 56630 (0.0007) +[2023-10-09 14:26:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 115703808. Throughput: 0: 1825.2, 1: 1819.8. Samples: 28934866. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 14:26:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:28,546][86122] Updated weights for policy 1, policy_version 56640 (0.0009) +[2023-10-09 14:26:28,931][86121] Updated weights for policy 0, policy_version 56390 (0.0008) +[2023-10-09 14:26:29,302][86121] Updated weights for policy 0, policy_version 56400 (0.0009) +[2023-10-09 14:26:29,682][86121] Updated weights for policy 0, policy_version 56410 (0.0009) +[2023-10-09 14:26:32,306][86122] Updated weights for policy 1, policy_version 56650 (0.0008) +[2023-10-09 14:26:32,665][86122] Updated weights for policy 1, policy_version 56660 (0.0008) +[2023-10-09 14:26:33,032][86122] Updated weights for policy 1, policy_version 56670 (0.0009) +[2023-10-09 14:26:33,211][86121] Updated weights for policy 0, policy_version 56420 (0.0008) +[2023-10-09 14:26:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 115802112. Throughput: 0: 1829.0, 1: 1821.9. Samples: 28957614. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 14:26:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:33,577][86121] Updated weights for policy 0, policy_version 56430 (0.0007) +[2023-10-09 14:26:33,944][86121] Updated weights for policy 0, policy_version 56440 (0.0008) +[2023-10-09 14:26:36,683][86122] Updated weights for policy 1, policy_version 56680 (0.0007) +[2023-10-09 14:26:37,055][86122] Updated weights for policy 1, policy_version 56690 (0.0009) +[2023-10-09 14:26:37,424][86122] Updated weights for policy 1, policy_version 56700 (0.0010) +[2023-10-09 14:26:37,696][86121] Updated weights for policy 0, policy_version 56450 (0.0009) +[2023-10-09 14:26:38,062][86121] Updated weights for policy 0, policy_version 56460 (0.0010) +[2023-10-09 14:26:38,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 115867648. Throughput: 0: 1826.9, 1: 1817.6. Samples: 28978590. Policy #0 lag: (min: 12.0, avg: 12.0, max: 12.0) +[2023-10-09 14:26:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:26:38,432][86121] Updated weights for policy 0, policy_version 56470 (0.0008) +[2023-10-09 14:26:38,792][86121] Updated weights for policy 0, policy_version 56480 (0.0007) +[2023-10-09 14:26:41,120][86122] Updated weights for policy 1, policy_version 56710 (0.0009) +[2023-10-09 14:26:41,477][86122] Updated weights for policy 1, policy_version 56720 (0.0008) +[2023-10-09 14:26:41,842][86122] Updated weights for policy 1, policy_version 56730 (0.0008) +[2023-10-09 14:26:42,386][86121] Updated weights for policy 0, policy_version 56490 (0.0008) +[2023-10-09 14:26:42,748][86121] Updated weights for policy 0, policy_version 56500 (0.0008) +[2023-10-09 14:26:43,111][86121] Updated weights for policy 0, policy_version 56510 (0.0008) +[2023-10-09 14:26:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 115965952. Throughput: 0: 1836.2, 1: 1818.7. Samples: 28990492. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:26:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:26:45,551][86122] Updated weights for policy 1, policy_version 56740 (0.0009) +[2023-10-09 14:26:45,921][86122] Updated weights for policy 1, policy_version 56750 (0.0009) +[2023-10-09 14:26:46,284][86122] Updated weights for policy 1, policy_version 56760 (0.0009) +[2023-10-09 14:26:46,722][86121] Updated weights for policy 0, policy_version 56520 (0.0009) +[2023-10-09 14:26:47,080][86121] Updated weights for policy 0, policy_version 56530 (0.0008) +[2023-10-09 14:26:47,451][86121] Updated weights for policy 0, policy_version 56540 (0.0010) +[2023-10-09 14:26:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116031488. Throughput: 0: 1825.0, 1: 1822.3. Samples: 29011354. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:26:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 14:26:50,043][86122] Updated weights for policy 1, policy_version 56770 (0.0009) +[2023-10-09 14:26:50,405][86122] Updated weights for policy 1, policy_version 56780 (0.0010) +[2023-10-09 14:26:50,771][86122] Updated weights for policy 1, policy_version 56790 (0.0009) +[2023-10-09 14:26:51,128][86122] Updated weights for policy 1, policy_version 56800 (0.0009) +[2023-10-09 14:26:51,285][86121] Updated weights for policy 0, policy_version 56550 (0.0009) +[2023-10-09 14:26:51,649][86121] Updated weights for policy 0, policy_version 56560 (0.0009) +[2023-10-09 14:26:52,020][86121] Updated weights for policy 0, policy_version 56570 (0.0009) +[2023-10-09 14:26:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 116097024. Throughput: 0: 1836.4, 1: 1814.6. Samples: 29033186. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:26:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 14:26:54,848][86122] Updated weights for policy 1, policy_version 56810 (0.0009) +[2023-10-09 14:26:55,217][86122] Updated weights for policy 1, policy_version 56820 (0.0007) +[2023-10-09 14:26:55,582][86122] Updated weights for policy 1, policy_version 56830 (0.0008) +[2023-10-09 14:26:55,800][86121] Updated weights for policy 0, policy_version 56580 (0.0009) +[2023-10-09 14:26:56,203][86121] Updated weights for policy 0, policy_version 56590 (0.0008) +[2023-10-09 14:26:56,565][86121] Updated weights for policy 0, policy_version 56600 (0.0008) +[2023-10-09 14:26:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 116162560. Throughput: 0: 1826.7, 1: 1817.8. Samples: 29044176. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:26:58,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.970')] +[2023-10-09 14:26:59,252][86122] Updated weights for policy 1, policy_version 56840 (0.0008) +[2023-10-09 14:26:59,619][86122] Updated weights for policy 1, policy_version 56850 (0.0007) +[2023-10-09 14:26:59,977][86122] Updated weights for policy 1, policy_version 56860 (0.0007) +[2023-10-09 14:27:00,240][86121] Updated weights for policy 0, policy_version 56610 (0.0010) +[2023-10-09 14:27:00,604][86121] Updated weights for policy 0, policy_version 56620 (0.0010) +[2023-10-09 14:27:00,965][86121] Updated weights for policy 0, policy_version 56630 (0.0008) +[2023-10-09 14:27:01,327][86121] Updated weights for policy 0, policy_version 56640 (0.0009) +[2023-10-09 14:27:03,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116228096. Throughput: 0: 1828.7, 1: 1819.1. Samples: 29065926. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:27:03,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.970')] +[2023-10-09 14:27:03,645][86122] Updated weights for policy 1, policy_version 56870 (0.0009) +[2023-10-09 14:27:04,017][86122] Updated weights for policy 1, policy_version 56880 (0.0007) +[2023-10-09 14:27:04,385][86122] Updated weights for policy 1, policy_version 56890 (0.0008) +[2023-10-09 14:27:05,157][86121] Updated weights for policy 0, policy_version 56650 (0.0008) +[2023-10-09 14:27:05,520][86121] Updated weights for policy 0, policy_version 56660 (0.0010) +[2023-10-09 14:27:05,885][86121] Updated weights for policy 0, policy_version 56670 (0.0007) +[2023-10-09 14:27:08,103][86122] Updated weights for policy 1, policy_version 56900 (0.0007) +[2023-10-09 14:27:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 116293632. Throughput: 0: 1830.0, 1: 1810.0. Samples: 29088688. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:27:08,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.970')] +[2023-10-09 14:27:08,475][86122] Updated weights for policy 1, policy_version 56910 (0.0008) +[2023-10-09 14:27:08,846][86122] Updated weights for policy 1, policy_version 56920 (0.0008) +[2023-10-09 14:27:09,510][86121] Updated weights for policy 0, policy_version 56680 (0.0007) +[2023-10-09 14:27:09,880][86121] Updated weights for policy 0, policy_version 56690 (0.0008) +[2023-10-09 14:27:10,257][86121] Updated weights for policy 0, policy_version 56700 (0.0009) +[2023-10-09 14:27:12,560][86122] Updated weights for policy 1, policy_version 56930 (0.0009) +[2023-10-09 14:27:12,970][86122] Updated weights for policy 1, policy_version 56940 (0.0009) +[2023-10-09 14:27:13,337][86122] Updated weights for policy 1, policy_version 56950 (0.0010) +[2023-10-09 14:27:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116359168. Throughput: 0: 1831.2, 1: 1808.9. Samples: 29098670. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:27:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.970')] +[2023-10-09 14:27:13,696][86122] Updated weights for policy 1, policy_version 56960 (0.0009) +[2023-10-09 14:27:13,959][86121] Updated weights for policy 0, policy_version 56710 (0.0009) +[2023-10-09 14:27:14,334][86121] Updated weights for policy 0, policy_version 56720 (0.0011) +[2023-10-09 14:27:14,709][86121] Updated weights for policy 0, policy_version 56730 (0.0009) +[2023-10-09 14:27:17,493][86122] Updated weights for policy 1, policy_version 56970 (0.0010) +[2023-10-09 14:27:17,856][86122] Updated weights for policy 1, policy_version 56980 (0.0009) +[2023-10-09 14:27:18,223][86122] Updated weights for policy 1, policy_version 56990 (0.0008) +[2023-10-09 14:27:18,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116457472. Throughput: 0: 1828.2, 1: 1804.4. Samples: 29121080. Policy #0 lag: (min: 31.0, avg: 35.2, max: 63.0) +[2023-10-09 14:27:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 14:27:18,400][86121] Updated weights for policy 0, policy_version 56740 (0.0008) +[2023-10-09 14:27:18,778][86121] Updated weights for policy 0, policy_version 56750 (0.0008) +[2023-10-09 14:27:19,137][86121] Updated weights for policy 0, policy_version 56760 (0.0008) +[2023-10-09 14:27:21,998][86122] Updated weights for policy 1, policy_version 57000 (0.0008) +[2023-10-09 14:27:22,372][86122] Updated weights for policy 1, policy_version 57010 (0.0009) +[2023-10-09 14:27:22,732][86122] Updated weights for policy 1, policy_version 57020 (0.0007) +[2023-10-09 14:27:22,787][86121] Updated weights for policy 0, policy_version 56770 (0.0008) +[2023-10-09 14:27:23,156][86121] Updated weights for policy 0, policy_version 56780 (0.0008) +[2023-10-09 14:27:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 116523008. Throughput: 0: 1826.2, 1: 1811.8. Samples: 29142302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.970')] +[2023-10-09 14:27:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000057024_58392576.pth... +[2023-10-09 14:27:23,435][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000055328_56655872.pth +[2023-10-09 14:27:23,536][86121] Updated weights for policy 0, policy_version 56790 (0.0008) +[2023-10-09 14:27:23,897][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000056800_58163200.pth... +[2023-10-09 14:27:23,898][86121] Updated weights for policy 0, policy_version 56800 (0.0010) +[2023-10-09 14:27:23,926][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000055072_56393728.pth +[2023-10-09 14:27:26,547][86122] Updated weights for policy 1, policy_version 57030 (0.0007) +[2023-10-09 14:27:26,918][86122] Updated weights for policy 1, policy_version 57040 (0.0008) +[2023-10-09 14:27:27,268][86122] Updated weights for policy 1, policy_version 57050 (0.0008) +[2023-10-09 14:27:27,479][86121] Updated weights for policy 0, policy_version 56810 (0.0009) +[2023-10-09 14:27:27,857][86121] Updated weights for policy 0, policy_version 56820 (0.0007) +[2023-10-09 14:27:28,215][86121] Updated weights for policy 0, policy_version 56830 (0.0011) +[2023-10-09 14:27:28,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 116621312. Throughput: 0: 1821.4, 1: 1807.4. Samples: 29153788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:28,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.950')] +[2023-10-09 14:27:31,061][86122] Updated weights for policy 1, policy_version 57060 (0.0009) +[2023-10-09 14:27:31,428][86122] Updated weights for policy 1, policy_version 57070 (0.0007) +[2023-10-09 14:27:31,784][86122] Updated weights for policy 1, policy_version 57080 (0.0009) +[2023-10-09 14:27:31,842][86121] Updated weights for policy 0, policy_version 56840 (0.0008) +[2023-10-09 14:27:32,208][86121] Updated weights for policy 0, policy_version 56850 (0.0007) +[2023-10-09 14:27:32,576][86121] Updated weights for policy 0, policy_version 56860 (0.0007) +[2023-10-09 14:27:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 116686848. Throughput: 0: 1821.9, 1: 1818.4. Samples: 29175170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.950')] +[2023-10-09 14:27:35,551][86122] Updated weights for policy 1, policy_version 57090 (0.0010) +[2023-10-09 14:27:35,910][86122] Updated weights for policy 1, policy_version 57100 (0.0011) +[2023-10-09 14:27:36,253][86121] Updated weights for policy 0, policy_version 56870 (0.0009) +[2023-10-09 14:27:36,272][86122] Updated weights for policy 1, policy_version 57110 (0.0007) +[2023-10-09 14:27:36,622][86121] Updated weights for policy 0, policy_version 56880 (0.0009) +[2023-10-09 14:27:36,623][86122] Updated weights for policy 1, policy_version 57120 (0.0008) +[2023-10-09 14:27:36,990][86121] Updated weights for policy 0, policy_version 56890 (0.0008) +[2023-10-09 14:27:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 116752384. Throughput: 0: 1817.5, 1: 1811.1. Samples: 29196472. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.950')] +[2023-10-09 14:27:40,209][86122] Updated weights for policy 1, policy_version 57130 (0.0009) +[2023-10-09 14:27:40,576][86122] Updated weights for policy 1, policy_version 57140 (0.0008) +[2023-10-09 14:27:40,671][86121] Updated weights for policy 0, policy_version 56900 (0.0008) +[2023-10-09 14:27:40,943][86122] Updated weights for policy 1, policy_version 57150 (0.0010) +[2023-10-09 14:27:41,050][86121] Updated weights for policy 0, policy_version 56910 (0.0009) +[2023-10-09 14:27:41,423][86121] Updated weights for policy 0, policy_version 56920 (0.0007) +[2023-10-09 14:27:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116817920. Throughput: 0: 1814.7, 1: 1818.8. Samples: 29207686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 14:27:44,453][86122] Updated weights for policy 1, policy_version 57160 (0.0010) +[2023-10-09 14:27:44,823][86122] Updated weights for policy 1, policy_version 57170 (0.0008) +[2023-10-09 14:27:45,048][86121] Updated weights for policy 0, policy_version 56930 (0.0009) +[2023-10-09 14:27:45,186][86122] Updated weights for policy 1, policy_version 57180 (0.0008) +[2023-10-09 14:27:45,421][86121] Updated weights for policy 0, policy_version 56940 (0.0009) +[2023-10-09 14:27:45,777][86121] Updated weights for policy 0, policy_version 56950 (0.0009) +[2023-10-09 14:27:46,140][86121] Updated weights for policy 0, policy_version 56960 (0.0007) +[2023-10-09 14:27:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 116883456. Throughput: 0: 1815.5, 1: 1814.4. Samples: 29229272. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.950')] +[2023-10-09 14:27:48,990][86122] Updated weights for policy 1, policy_version 57190 (0.0009) +[2023-10-09 14:27:49,341][86122] Updated weights for policy 1, policy_version 57200 (0.0007) +[2023-10-09 14:27:49,708][86122] Updated weights for policy 1, policy_version 57210 (0.0008) +[2023-10-09 14:27:49,970][86121] Updated weights for policy 0, policy_version 56970 (0.0008) +[2023-10-09 14:27:50,348][86121] Updated weights for policy 0, policy_version 56980 (0.0009) +[2023-10-09 14:27:50,714][86121] Updated weights for policy 0, policy_version 56990 (0.0009) +[2023-10-09 14:27:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 116948992. Throughput: 0: 1815.1, 1: 1817.7. Samples: 29252160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.950')] +[2023-10-09 14:27:53,431][86122] Updated weights for policy 1, policy_version 57220 (0.0007) +[2023-10-09 14:27:53,793][86122] Updated weights for policy 1, policy_version 57230 (0.0011) +[2023-10-09 14:27:54,157][86122] Updated weights for policy 1, policy_version 57240 (0.0010) +[2023-10-09 14:27:54,332][86121] Updated weights for policy 0, policy_version 57000 (0.0009) +[2023-10-09 14:27:54,705][86121] Updated weights for policy 0, policy_version 57010 (0.0009) +[2023-10-09 14:27:55,073][86121] Updated weights for policy 0, policy_version 57020 (0.0010) +[2023-10-09 14:27:57,993][86122] Updated weights for policy 1, policy_version 57250 (0.0008) +[2023-10-09 14:27:58,380][86122] Updated weights for policy 1, policy_version 57260 (0.0009) +[2023-10-09 14:27:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117014528. Throughput: 0: 1819.7, 1: 1816.1. Samples: 29262280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:27:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:27:58,748][86122] Updated weights for policy 1, policy_version 57270 (0.0008) +[2023-10-09 14:27:58,863][86121] Updated weights for policy 0, policy_version 57030 (0.0008) +[2023-10-09 14:27:59,107][86122] Updated weights for policy 1, policy_version 57280 (0.0008) +[2023-10-09 14:27:59,222][86121] Updated weights for policy 0, policy_version 57040 (0.0011) +[2023-10-09 14:27:59,585][86121] Updated weights for policy 0, policy_version 57050 (0.0007) +[2023-10-09 14:28:02,772][86122] Updated weights for policy 1, policy_version 57290 (0.0009) +[2023-10-09 14:28:03,134][86122] Updated weights for policy 1, policy_version 57300 (0.0009) +[2023-10-09 14:28:03,210][86121] Updated weights for policy 0, policy_version 57060 (0.0008) +[2023-10-09 14:28:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117080064. Throughput: 0: 1818.8, 1: 1819.4. Samples: 29284796. Policy #0 lag: (min: 16.0, avg: 43.0, max: 48.0) +[2023-10-09 14:28:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:28:03,486][86122] Updated weights for policy 1, policy_version 57310 (0.0008) +[2023-10-09 14:28:03,567][86121] Updated weights for policy 0, policy_version 57070 (0.0008) +[2023-10-09 14:28:03,935][86121] Updated weights for policy 0, policy_version 57080 (0.0009) +[2023-10-09 14:28:07,184][86122] Updated weights for policy 1, policy_version 57320 (0.0008) +[2023-10-09 14:28:07,542][86122] Updated weights for policy 1, policy_version 57330 (0.0010) +[2023-10-09 14:28:07,730][86121] Updated weights for policy 0, policy_version 57090 (0.0009) +[2023-10-09 14:28:07,910][86122] Updated weights for policy 1, policy_version 57340 (0.0008) +[2023-10-09 14:28:08,097][86121] Updated weights for policy 0, policy_version 57100 (0.0008) +[2023-10-09 14:28:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117178368. Throughput: 0: 1817.3, 1: 1818.8. Samples: 29305928. Policy #0 lag: (min: 16.0, avg: 43.0, max: 48.0) +[2023-10-09 14:28:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:28:08,464][86121] Updated weights for policy 0, policy_version 57110 (0.0008) +[2023-10-09 14:28:08,820][86121] Updated weights for policy 0, policy_version 57120 (0.0007) +[2023-10-09 14:28:11,605][86122] Updated weights for policy 1, policy_version 57350 (0.0008) +[2023-10-09 14:28:11,965][86122] Updated weights for policy 1, policy_version 57360 (0.0010) +[2023-10-09 14:28:12,340][86122] Updated weights for policy 1, policy_version 57370 (0.0007) +[2023-10-09 14:28:12,635][86121] Updated weights for policy 0, policy_version 57130 (0.0007) +[2023-10-09 14:28:13,002][86121] Updated weights for policy 0, policy_version 57140 (0.0008) +[2023-10-09 14:28:13,359][86121] Updated weights for policy 0, policy_version 57150 (0.0007) +[2023-10-09 14:28:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117243904. Throughput: 0: 1818.7, 1: 1817.9. Samples: 29317432. Policy #0 lag: (min: 16.0, avg: 43.0, max: 48.0) +[2023-10-09 14:28:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:28:16,027][86122] Updated weights for policy 1, policy_version 57380 (0.0009) +[2023-10-09 14:28:16,400][86122] Updated weights for policy 1, policy_version 57390 (0.0011) +[2023-10-09 14:28:16,752][86122] Updated weights for policy 1, policy_version 57400 (0.0010) +[2023-10-09 14:28:17,153][86121] Updated weights for policy 0, policy_version 57160 (0.0007) +[2023-10-09 14:28:17,507][86121] Updated weights for policy 0, policy_version 57170 (0.0007) +[2023-10-09 14:28:17,883][86121] Updated weights for policy 0, policy_version 57180 (0.0007) +[2023-10-09 14:28:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117342208. Throughput: 0: 1823.0, 1: 1815.8. Samples: 29338916. Policy #0 lag: (min: 16.0, avg: 43.0, max: 48.0) +[2023-10-09 14:28:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:28:20,379][86122] Updated weights for policy 1, policy_version 57410 (0.0009) +[2023-10-09 14:28:20,743][86122] Updated weights for policy 1, policy_version 57420 (0.0008) +[2023-10-09 14:28:21,099][86122] Updated weights for policy 1, policy_version 57430 (0.0009) +[2023-10-09 14:28:21,461][86122] Updated weights for policy 1, policy_version 57440 (0.0007) +[2023-10-09 14:28:21,588][86121] Updated weights for policy 0, policy_version 57190 (0.0008) +[2023-10-09 14:28:21,958][86121] Updated weights for policy 0, policy_version 57200 (0.0008) +[2023-10-09 14:28:22,329][86121] Updated weights for policy 0, policy_version 57210 (0.0008) +[2023-10-09 14:28:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 117407744. Throughput: 0: 1813.3, 1: 1820.2. Samples: 29359980. Policy #0 lag: (min: 16.0, avg: 43.0, max: 48.0) +[2023-10-09 14:28:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:28:25,109][86122] Updated weights for policy 1, policy_version 57450 (0.0009) +[2023-10-09 14:28:25,474][86122] Updated weights for policy 1, policy_version 57460 (0.0009) +[2023-10-09 14:28:25,838][86122] Updated weights for policy 1, policy_version 57470 (0.0009) +[2023-10-09 14:28:25,997][86121] Updated weights for policy 0, policy_version 57220 (0.0009) +[2023-10-09 14:28:26,379][86121] Updated weights for policy 0, policy_version 57230 (0.0007) +[2023-10-09 14:28:26,740][86121] Updated weights for policy 0, policy_version 57240 (0.0008) +[2023-10-09 14:28:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117473280. Throughput: 0: 1819.2, 1: 1818.3. Samples: 29371372. Policy #0 lag: (min: 16.0, avg: 43.0, max: 48.0) +[2023-10-09 14:28:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:28:29,381][86122] Updated weights for policy 1, policy_version 57480 (0.0011) +[2023-10-09 14:28:29,738][86122] Updated weights for policy 1, policy_version 57490 (0.0010) +[2023-10-09 14:28:30,096][86122] Updated weights for policy 1, policy_version 57500 (0.0009) +[2023-10-09 14:28:30,380][86121] Updated weights for policy 0, policy_version 57250 (0.0009) +[2023-10-09 14:28:30,755][86121] Updated weights for policy 0, policy_version 57260 (0.0009) +[2023-10-09 14:28:31,113][86121] Updated weights for policy 0, policy_version 57270 (0.0008) +[2023-10-09 14:28:31,480][86121] Updated weights for policy 0, policy_version 57280 (0.0008) +[2023-10-09 14:28:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117538816. Throughput: 0: 1812.5, 1: 1823.4. Samples: 29392890. Policy #0 lag: (min: 16.0, avg: 43.0, max: 48.0) +[2023-10-09 14:28:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.940')] +[2023-10-09 14:28:33,850][86122] Updated weights for policy 1, policy_version 57510 (0.0009) +[2023-10-09 14:28:34,213][86122] Updated weights for policy 1, policy_version 57520 (0.0008) +[2023-10-09 14:28:34,576][86122] Updated weights for policy 1, policy_version 57530 (0.0009) +[2023-10-09 14:28:35,297][86121] Updated weights for policy 0, policy_version 57290 (0.0009) +[2023-10-09 14:28:35,665][86121] Updated weights for policy 0, policy_version 57300 (0.0009) +[2023-10-09 14:28:36,042][86121] Updated weights for policy 0, policy_version 57310 (0.0008) +[2023-10-09 14:28:38,198][86122] Updated weights for policy 1, policy_version 57540 (0.0009) +[2023-10-09 14:28:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 117604352. Throughput: 0: 1806.2, 1: 1827.4. Samples: 29415670. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:28:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.950')] +[2023-10-09 14:28:38,565][86122] Updated weights for policy 1, policy_version 57550 (0.0008) +[2023-10-09 14:28:38,923][86122] Updated weights for policy 1, policy_version 57560 (0.0008) +[2023-10-09 14:28:39,787][86121] Updated weights for policy 0, policy_version 57320 (0.0007) +[2023-10-09 14:28:40,151][86121] Updated weights for policy 0, policy_version 57330 (0.0007) +[2023-10-09 14:28:40,521][86121] Updated weights for policy 0, policy_version 57340 (0.0011) +[2023-10-09 14:28:42,692][86122] Updated weights for policy 1, policy_version 57570 (0.0009) +[2023-10-09 14:28:43,087][86122] Updated weights for policy 1, policy_version 57580 (0.0011) +[2023-10-09 14:28:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 117669888. Throughput: 0: 1801.7, 1: 1831.5. Samples: 29425774. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:28:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 14:28:43,441][86122] Updated weights for policy 1, policy_version 57590 (0.0009) +[2023-10-09 14:28:43,811][86122] Updated weights for policy 1, policy_version 57600 (0.0010) +[2023-10-09 14:28:44,395][86121] Updated weights for policy 0, policy_version 57350 (0.0011) +[2023-10-09 14:28:44,762][86121] Updated weights for policy 0, policy_version 57360 (0.0008) +[2023-10-09 14:28:45,130][86121] Updated weights for policy 0, policy_version 57370 (0.0008) +[2023-10-09 14:28:47,437][86122] Updated weights for policy 1, policy_version 57610 (0.0010) +[2023-10-09 14:28:47,795][86122] Updated weights for policy 1, policy_version 57620 (0.0009) +[2023-10-09 14:28:48,158][86122] Updated weights for policy 1, policy_version 57630 (0.0007) +[2023-10-09 14:28:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117768192. Throughput: 0: 1799.2, 1: 1835.0. Samples: 29448334. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:28:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.940')] +[2023-10-09 14:28:48,845][86121] Updated weights for policy 0, policy_version 57380 (0.0010) +[2023-10-09 14:28:49,210][86121] Updated weights for policy 0, policy_version 57390 (0.0008) +[2023-10-09 14:28:49,570][86121] Updated weights for policy 0, policy_version 57400 (0.0009) +[2023-10-09 14:28:51,818][86122] Updated weights for policy 1, policy_version 57640 (0.0009) +[2023-10-09 14:28:52,175][86122] Updated weights for policy 1, policy_version 57650 (0.0008) +[2023-10-09 14:28:52,533][86122] Updated weights for policy 1, policy_version 57660 (0.0008) +[2023-10-09 14:28:53,206][86121] Updated weights for policy 0, policy_version 57410 (0.0009) +[2023-10-09 14:28:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117833728. Throughput: 0: 1811.1, 1: 1833.6. Samples: 29469940. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:28:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 14:28:53,565][86121] Updated weights for policy 0, policy_version 57420 (0.0007) +[2023-10-09 14:28:53,933][86121] Updated weights for policy 0, policy_version 57430 (0.0007) +[2023-10-09 14:28:54,299][86121] Updated weights for policy 0, policy_version 57440 (0.0007) +[2023-10-09 14:28:56,223][86122] Updated weights for policy 1, policy_version 57670 (0.0008) +[2023-10-09 14:28:56,592][86122] Updated weights for policy 1, policy_version 57680 (0.0008) +[2023-10-09 14:28:56,954][86122] Updated weights for policy 1, policy_version 57690 (0.0009) +[2023-10-09 14:28:57,926][86121] Updated weights for policy 0, policy_version 57450 (0.0009) +[2023-10-09 14:28:58,280][86121] Updated weights for policy 0, policy_version 57460 (0.0010) +[2023-10-09 14:28:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 117899264. Throughput: 0: 1804.2, 1: 1834.6. Samples: 29481178. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:28:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.950')] +[2023-10-09 14:28:58,644][86121] Updated weights for policy 0, policy_version 57470 (0.0007) +[2023-10-09 14:29:00,645][86122] Updated weights for policy 1, policy_version 57700 (0.0010) +[2023-10-09 14:29:01,004][86122] Updated weights for policy 1, policy_version 57710 (0.0008) +[2023-10-09 14:29:01,370][86122] Updated weights for policy 1, policy_version 57720 (0.0009) +[2023-10-09 14:29:02,425][86121] Updated weights for policy 0, policy_version 57480 (0.0011) +[2023-10-09 14:29:02,788][86121] Updated weights for policy 0, policy_version 57490 (0.0012) +[2023-10-09 14:29:03,154][86121] Updated weights for policy 0, policy_version 57500 (0.0012) +[2023-10-09 14:29:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 117997568. Throughput: 0: 1807.7, 1: 1827.4. Samples: 29502494. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:29:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:29:05,245][86122] Updated weights for policy 1, policy_version 57730 (0.0009) +[2023-10-09 14:29:05,599][86122] Updated weights for policy 1, policy_version 57740 (0.0010) +[2023-10-09 14:29:05,966][86122] Updated weights for policy 1, policy_version 57750 (0.0009) +[2023-10-09 14:29:06,335][86122] Updated weights for policy 1, policy_version 57760 (0.0007) +[2023-10-09 14:29:06,962][86121] Updated weights for policy 0, policy_version 57510 (0.0011) +[2023-10-09 14:29:07,318][86121] Updated weights for policy 0, policy_version 57520 (0.0008) +[2023-10-09 14:29:07,684][86121] Updated weights for policy 0, policy_version 57530 (0.0009) +[2023-10-09 14:29:08,398][85186] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 118063104. Throughput: 0: 1807.1, 1: 1829.4. Samples: 29523622. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:29:08,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:29:09,876][86122] Updated weights for policy 1, policy_version 57770 (0.0007) +[2023-10-09 14:29:10,240][86122] Updated weights for policy 1, policy_version 57780 (0.0008) +[2023-10-09 14:29:10,598][86122] Updated weights for policy 1, policy_version 57790 (0.0008) +[2023-10-09 14:29:11,501][86121] Updated weights for policy 0, policy_version 57540 (0.0009) +[2023-10-09 14:29:11,891][86121] Updated weights for policy 0, policy_version 57550 (0.0009) +[2023-10-09 14:29:12,261][86121] Updated weights for policy 0, policy_version 57560 (0.0007) +[2023-10-09 14:29:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118128640. Throughput: 0: 1809.7, 1: 1824.2. Samples: 29534898. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:29:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.950')] +[2023-10-09 14:29:14,219][86122] Updated weights for policy 1, policy_version 57800 (0.0010) +[2023-10-09 14:29:14,591][86122] Updated weights for policy 1, policy_version 57810 (0.0008) +[2023-10-09 14:29:14,954][86122] Updated weights for policy 1, policy_version 57820 (0.0009) +[2023-10-09 14:29:16,063][86121] Updated weights for policy 0, policy_version 57570 (0.0008) +[2023-10-09 14:29:16,435][86121] Updated weights for policy 0, policy_version 57580 (0.0010) +[2023-10-09 14:29:16,797][86121] Updated weights for policy 0, policy_version 57590 (0.0010) +[2023-10-09 14:29:17,166][86121] Updated weights for policy 0, policy_version 57600 (0.0011) +[2023-10-09 14:29:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 118194176. Throughput: 0: 1807.0, 1: 1825.5. Samples: 29556354. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:29:18,551][86122] Updated weights for policy 1, policy_version 57830 (0.0008) +[2023-10-09 14:29:18,906][86122] Updated weights for policy 1, policy_version 57840 (0.0008) +[2023-10-09 14:29:19,267][86122] Updated weights for policy 1, policy_version 57850 (0.0009) +[2023-10-09 14:29:20,810][86121] Updated weights for policy 0, policy_version 57610 (0.0008) +[2023-10-09 14:29:21,178][86121] Updated weights for policy 0, policy_version 57620 (0.0009) +[2023-10-09 14:29:21,551][86121] Updated weights for policy 0, policy_version 57630 (0.0011) +[2023-10-09 14:29:22,995][86122] Updated weights for policy 1, policy_version 57860 (0.0009) +[2023-10-09 14:29:23,361][86122] Updated weights for policy 1, policy_version 57870 (0.0009) +[2023-10-09 14:29:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 118259712. Throughput: 0: 1798.5, 1: 1827.5. Samples: 29578842. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:23,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:29:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000057632_59015168.pth... +[2023-10-09 14:29:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000055936_57278464.pth +[2023-10-09 14:29:23,728][86122] Updated weights for policy 1, policy_version 57880 (0.0008) +[2023-10-09 14:29:24,009][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000057888_59277312.pth... +[2023-10-09 14:29:24,048][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000056160_57507840.pth +[2023-10-09 14:29:25,111][86121] Updated weights for policy 0, policy_version 57640 (0.0008) +[2023-10-09 14:29:25,472][86121] Updated weights for policy 0, policy_version 57650 (0.0008) +[2023-10-09 14:29:25,837][86121] Updated weights for policy 0, policy_version 57660 (0.0009) +[2023-10-09 14:29:27,441][86122] Updated weights for policy 1, policy_version 57890 (0.0009) +[2023-10-09 14:29:27,826][86122] Updated weights for policy 1, policy_version 57900 (0.0007) +[2023-10-09 14:29:28,191][86122] Updated weights for policy 1, policy_version 57910 (0.0008) +[2023-10-09 14:29:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 118325248. Throughput: 0: 1805.0, 1: 1824.5. Samples: 29589102. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:29:28,546][86122] Updated weights for policy 1, policy_version 57920 (0.0010) +[2023-10-09 14:29:29,465][86121] Updated weights for policy 0, policy_version 57670 (0.0008) +[2023-10-09 14:29:29,832][86121] Updated weights for policy 0, policy_version 57680 (0.0009) +[2023-10-09 14:29:30,190][86121] Updated weights for policy 0, policy_version 57690 (0.0008) +[2023-10-09 14:29:32,187][86122] Updated weights for policy 1, policy_version 57930 (0.0008) +[2023-10-09 14:29:32,556][86122] Updated weights for policy 1, policy_version 57940 (0.0009) +[2023-10-09 14:29:32,916][86122] Updated weights for policy 1, policy_version 57950 (0.0008) +[2023-10-09 14:29:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 118423552. Throughput: 0: 1806.7, 1: 1824.3. Samples: 29611730. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:29:34,185][86121] Updated weights for policy 0, policy_version 57700 (0.0008) +[2023-10-09 14:29:34,560][86121] Updated weights for policy 0, policy_version 57710 (0.0009) +[2023-10-09 14:29:34,925][86121] Updated weights for policy 0, policy_version 57720 (0.0009) +[2023-10-09 14:29:36,546][86122] Updated weights for policy 1, policy_version 57960 (0.0008) +[2023-10-09 14:29:36,910][86122] Updated weights for policy 1, policy_version 57970 (0.0009) +[2023-10-09 14:29:37,277][86122] Updated weights for policy 1, policy_version 57980 (0.0009) +[2023-10-09 14:29:38,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118489088. Throughput: 0: 1794.8, 1: 1826.1. Samples: 29632884. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:29:38,722][86121] Updated weights for policy 0, policy_version 57730 (0.0008) +[2023-10-09 14:29:39,090][86121] Updated weights for policy 0, policy_version 57740 (0.0008) +[2023-10-09 14:29:39,458][86121] Updated weights for policy 0, policy_version 57750 (0.0008) +[2023-10-09 14:29:39,820][86121] Updated weights for policy 0, policy_version 57760 (0.0008) +[2023-10-09 14:29:40,965][86122] Updated weights for policy 1, policy_version 57990 (0.0010) +[2023-10-09 14:29:41,328][86122] Updated weights for policy 1, policy_version 58000 (0.0009) +[2023-10-09 14:29:41,692][86122] Updated weights for policy 1, policy_version 58010 (0.0008) +[2023-10-09 14:29:43,368][86121] Updated weights for policy 0, policy_version 57770 (0.0009) +[2023-10-09 14:29:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118554624. Throughput: 0: 1794.7, 1: 1831.4. Samples: 29644352. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:29:43,722][86121] Updated weights for policy 0, policy_version 57780 (0.0010) +[2023-10-09 14:29:44,095][86121] Updated weights for policy 0, policy_version 57790 (0.0009) +[2023-10-09 14:29:45,098][86122] Updated weights for policy 1, policy_version 58020 (0.0008) +[2023-10-09 14:29:45,455][86122] Updated weights for policy 1, policy_version 58030 (0.0010) +[2023-10-09 14:29:45,824][86122] Updated weights for policy 1, policy_version 58040 (0.0009) +[2023-10-09 14:29:47,564][86121] Updated weights for policy 0, policy_version 57800 (0.0007) +[2023-10-09 14:29:47,927][86121] Updated weights for policy 0, policy_version 57810 (0.0009) +[2023-10-09 14:29:48,289][86121] Updated weights for policy 0, policy_version 57820 (0.0009) +[2023-10-09 14:29:48,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 118620160. Throughput: 0: 1802.8, 1: 1840.8. Samples: 29666454. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:29:49,541][86122] Updated weights for policy 1, policy_version 58050 (0.0010) +[2023-10-09 14:29:49,902][86122] Updated weights for policy 1, policy_version 58060 (0.0008) +[2023-10-09 14:29:50,254][86122] Updated weights for policy 1, policy_version 58070 (0.0010) +[2023-10-09 14:29:50,623][86122] Updated weights for policy 1, policy_version 58080 (0.0008) +[2023-10-09 14:29:51,907][86121] Updated weights for policy 0, policy_version 57830 (0.0007) +[2023-10-09 14:29:52,273][86121] Updated weights for policy 0, policy_version 57840 (0.0007) +[2023-10-09 14:29:52,638][86121] Updated weights for policy 0, policy_version 57850 (0.0008) +[2023-10-09 14:29:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 118718464. Throughput: 0: 1810.3, 1: 1842.7. Samples: 29688008. Policy #0 lag: (min: 31.0, avg: 36.4, max: 63.0) +[2023-10-09 14:29:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:29:54,150][86122] Updated weights for policy 1, policy_version 58090 (0.0008) +[2023-10-09 14:29:54,510][86122] Updated weights for policy 1, policy_version 58100 (0.0007) +[2023-10-09 14:29:54,866][86122] Updated weights for policy 1, policy_version 58110 (0.0007) +[2023-10-09 14:29:56,309][86121] Updated weights for policy 0, policy_version 57860 (0.0008) +[2023-10-09 14:29:56,677][86121] Updated weights for policy 0, policy_version 57870 (0.0009) +[2023-10-09 14:29:57,044][86121] Updated weights for policy 0, policy_version 57880 (0.0010) +[2023-10-09 14:29:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 118784000. Throughput: 0: 1816.8, 1: 1845.4. Samples: 29699698. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:29:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:29:58,575][86122] Updated weights for policy 1, policy_version 58120 (0.0009) +[2023-10-09 14:29:58,928][86122] Updated weights for policy 1, policy_version 58130 (0.0008) +[2023-10-09 14:29:59,284][86122] Updated weights for policy 1, policy_version 58140 (0.0011) +[2023-10-09 14:30:00,760][86121] Updated weights for policy 0, policy_version 57890 (0.0007) +[2023-10-09 14:30:01,128][86121] Updated weights for policy 0, policy_version 57900 (0.0008) +[2023-10-09 14:30:01,492][86121] Updated weights for policy 0, policy_version 57910 (0.0008) +[2023-10-09 14:30:01,862][86121] Updated weights for policy 0, policy_version 57920 (0.0007) +[2023-10-09 14:30:02,926][86122] Updated weights for policy 1, policy_version 58150 (0.0008) +[2023-10-09 14:30:03,288][86122] Updated weights for policy 1, policy_version 58160 (0.0008) +[2023-10-09 14:30:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118849536. Throughput: 0: 1818.2, 1: 1843.4. Samples: 29721128. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:30:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:03,647][86122] Updated weights for policy 1, policy_version 58170 (0.0009) +[2023-10-09 14:30:05,549][86121] Updated weights for policy 0, policy_version 57930 (0.0007) +[2023-10-09 14:30:05,925][86121] Updated weights for policy 0, policy_version 57940 (0.0007) +[2023-10-09 14:30:06,291][86121] Updated weights for policy 0, policy_version 57950 (0.0009) +[2023-10-09 14:30:07,488][86122] Updated weights for policy 1, policy_version 58180 (0.0007) +[2023-10-09 14:30:07,842][86122] Updated weights for policy 1, policy_version 58190 (0.0008) +[2023-10-09 14:30:08,200][86122] Updated weights for policy 1, policy_version 58200 (0.0007) +[2023-10-09 14:30:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 118915072. Throughput: 0: 1828.1, 1: 1827.8. Samples: 29743360. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:30:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:10,079][86121] Updated weights for policy 0, policy_version 57960 (0.0008) +[2023-10-09 14:30:10,440][86121] Updated weights for policy 0, policy_version 57970 (0.0009) +[2023-10-09 14:30:10,801][86121] Updated weights for policy 0, policy_version 57980 (0.0009) +[2023-10-09 14:30:11,848][86122] Updated weights for policy 1, policy_version 58210 (0.0010) +[2023-10-09 14:30:12,210][86122] Updated weights for policy 1, policy_version 58220 (0.0008) +[2023-10-09 14:30:12,562][86122] Updated weights for policy 1, policy_version 58230 (0.0009) +[2023-10-09 14:30:12,923][86122] Updated weights for policy 1, policy_version 58240 (0.0009) +[2023-10-09 14:30:13,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119013376. Throughput: 0: 1826.2, 1: 1841.6. Samples: 29754150. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:30:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:14,407][86121] Updated weights for policy 0, policy_version 57990 (0.0007) +[2023-10-09 14:30:14,762][86121] Updated weights for policy 0, policy_version 58000 (0.0007) +[2023-10-09 14:30:15,137][86121] Updated weights for policy 0, policy_version 58010 (0.0008) +[2023-10-09 14:30:16,642][86122] Updated weights for policy 1, policy_version 58250 (0.0010) +[2023-10-09 14:30:17,003][86122] Updated weights for policy 1, policy_version 58260 (0.0007) +[2023-10-09 14:30:17,363][86122] Updated weights for policy 1, policy_version 58270 (0.0008) +[2023-10-09 14:30:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119078912. Throughput: 0: 1832.5, 1: 1830.5. Samples: 29776562. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:30:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:18,673][86121] Updated weights for policy 0, policy_version 58020 (0.0010) +[2023-10-09 14:30:19,037][86121] Updated weights for policy 0, policy_version 58030 (0.0011) +[2023-10-09 14:30:19,402][86121] Updated weights for policy 0, policy_version 58040 (0.0009) +[2023-10-09 14:30:21,161][86122] Updated weights for policy 1, policy_version 58280 (0.0008) +[2023-10-09 14:30:21,528][86122] Updated weights for policy 1, policy_version 58290 (0.0008) +[2023-10-09 14:30:21,883][86122] Updated weights for policy 1, policy_version 58300 (0.0009) +[2023-10-09 14:30:23,053][86121] Updated weights for policy 0, policy_version 58050 (0.0008) +[2023-10-09 14:30:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119144448. Throughput: 0: 1847.7, 1: 1838.1. Samples: 29798744. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:30:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:23,418][86121] Updated weights for policy 0, policy_version 58060 (0.0008) +[2023-10-09 14:30:23,797][86121] Updated weights for policy 0, policy_version 58070 (0.0009) +[2023-10-09 14:30:24,157][86121] Updated weights for policy 0, policy_version 58080 (0.0011) +[2023-10-09 14:30:25,642][86122] Updated weights for policy 1, policy_version 58310 (0.0011) +[2023-10-09 14:30:26,009][86122] Updated weights for policy 1, policy_version 58320 (0.0009) +[2023-10-09 14:30:26,381][86122] Updated weights for policy 1, policy_version 58330 (0.0009) +[2023-10-09 14:30:27,851][86121] Updated weights for policy 0, policy_version 58090 (0.0009) +[2023-10-09 14:30:28,224][86121] Updated weights for policy 0, policy_version 58100 (0.0010) +[2023-10-09 14:30:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 119209984. Throughput: 0: 1847.4, 1: 1828.4. Samples: 29809760. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:30:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:30:28,592][86121] Updated weights for policy 0, policy_version 58110 (0.0009) +[2023-10-09 14:30:30,179][86122] Updated weights for policy 1, policy_version 58340 (0.0009) +[2023-10-09 14:30:30,536][86122] Updated weights for policy 1, policy_version 58350 (0.0008) +[2023-10-09 14:30:30,904][86122] Updated weights for policy 1, policy_version 58360 (0.0008) +[2023-10-09 14:30:32,183][86121] Updated weights for policy 0, policy_version 58120 (0.0010) +[2023-10-09 14:30:32,543][86121] Updated weights for policy 0, policy_version 58130 (0.0010) +[2023-10-09 14:30:32,916][86121] Updated weights for policy 0, policy_version 58140 (0.0008) +[2023-10-09 14:30:33,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119308288. Throughput: 0: 1839.5, 1: 1826.8. Samples: 29831434. Policy #0 lag: (min: 14.0, avg: 20.1, max: 46.0) +[2023-10-09 14:30:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:30:34,626][86122] Updated weights for policy 1, policy_version 58370 (0.0008) +[2023-10-09 14:30:34,999][86122] Updated weights for policy 1, policy_version 58380 (0.0009) +[2023-10-09 14:30:35,351][86122] Updated weights for policy 1, policy_version 58390 (0.0010) +[2023-10-09 14:30:35,711][86122] Updated weights for policy 1, policy_version 58400 (0.0009) +[2023-10-09 14:30:36,674][86121] Updated weights for policy 0, policy_version 58150 (0.0008) +[2023-10-09 14:30:37,047][86121] Updated weights for policy 0, policy_version 58160 (0.0011) +[2023-10-09 14:30:37,410][86121] Updated weights for policy 0, policy_version 58170 (0.0009) +[2023-10-09 14:30:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119373824. Throughput: 0: 1832.2, 1: 1823.3. Samples: 29852504. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:30:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:39,464][86122] Updated weights for policy 1, policy_version 58410 (0.0007) +[2023-10-09 14:30:39,817][86122] Updated weights for policy 1, policy_version 58420 (0.0007) +[2023-10-09 14:30:40,177][86122] Updated weights for policy 1, policy_version 58430 (0.0008) +[2023-10-09 14:30:41,075][86121] Updated weights for policy 0, policy_version 58180 (0.0009) +[2023-10-09 14:30:41,449][86121] Updated weights for policy 0, policy_version 58190 (0.0010) +[2023-10-09 14:30:41,822][86121] Updated weights for policy 0, policy_version 58200 (0.0008) +[2023-10-09 14:30:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 119439360. Throughput: 0: 1829.1, 1: 1821.4. Samples: 29863972. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:30:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:43,756][86122] Updated weights for policy 1, policy_version 58440 (0.0009) +[2023-10-09 14:30:44,122][86122] Updated weights for policy 1, policy_version 58450 (0.0008) +[2023-10-09 14:30:44,483][86122] Updated weights for policy 1, policy_version 58460 (0.0009) +[2023-10-09 14:30:45,513][86121] Updated weights for policy 0, policy_version 58210 (0.0009) +[2023-10-09 14:30:45,924][86121] Updated weights for policy 0, policy_version 58220 (0.0010) +[2023-10-09 14:30:46,299][86121] Updated weights for policy 0, policy_version 58230 (0.0010) +[2023-10-09 14:30:46,673][86121] Updated weights for policy 0, policy_version 58240 (0.0010) +[2023-10-09 14:30:48,115][86122] Updated weights for policy 1, policy_version 58470 (0.0008) +[2023-10-09 14:30:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119504896. Throughput: 0: 1824.2, 1: 1826.6. Samples: 29885414. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:30:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:48,471][86122] Updated weights for policy 1, policy_version 58480 (0.0008) +[2023-10-09 14:30:48,833][86122] Updated weights for policy 1, policy_version 58490 (0.0008) +[2023-10-09 14:30:50,367][86121] Updated weights for policy 0, policy_version 58250 (0.0008) +[2023-10-09 14:30:50,733][86121] Updated weights for policy 0, policy_version 58260 (0.0007) +[2023-10-09 14:30:51,098][86121] Updated weights for policy 0, policy_version 58270 (0.0009) +[2023-10-09 14:30:52,492][86122] Updated weights for policy 1, policy_version 58500 (0.0007) +[2023-10-09 14:30:52,849][86122] Updated weights for policy 1, policy_version 58510 (0.0008) +[2023-10-09 14:30:53,212][86122] Updated weights for policy 1, policy_version 58520 (0.0008) +[2023-10-09 14:30:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119570432. Throughput: 0: 1827.1, 1: 1829.0. Samples: 29907884. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:30:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:54,815][86121] Updated weights for policy 0, policy_version 58280 (0.0008) +[2023-10-09 14:30:55,192][86121] Updated weights for policy 0, policy_version 58290 (0.0009) +[2023-10-09 14:30:55,557][86121] Updated weights for policy 0, policy_version 58300 (0.0008) +[2023-10-09 14:30:56,872][86122] Updated weights for policy 1, policy_version 58530 (0.0010) +[2023-10-09 14:30:57,235][86122] Updated weights for policy 1, policy_version 58540 (0.0011) +[2023-10-09 14:30:57,590][86122] Updated weights for policy 1, policy_version 58550 (0.0011) +[2023-10-09 14:30:57,950][86122] Updated weights for policy 1, policy_version 58560 (0.0010) +[2023-10-09 14:30:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119668736. Throughput: 0: 1819.2, 1: 1831.4. Samples: 29918430. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:30:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:30:59,335][86121] Updated weights for policy 0, policy_version 58310 (0.0008) +[2023-10-09 14:30:59,699][86121] Updated weights for policy 0, policy_version 58320 (0.0008) +[2023-10-09 14:31:00,063][86121] Updated weights for policy 0, policy_version 58330 (0.0009) +[2023-10-09 14:31:01,758][86122] Updated weights for policy 1, policy_version 58570 (0.0008) +[2023-10-09 14:31:02,123][86122] Updated weights for policy 1, policy_version 58580 (0.0009) +[2023-10-09 14:31:02,491][86122] Updated weights for policy 1, policy_version 58590 (0.0011) +[2023-10-09 14:31:03,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119734272. Throughput: 0: 1818.0, 1: 1830.4. Samples: 29940740. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:31:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:31:03,851][86121] Updated weights for policy 0, policy_version 58340 (0.0008) +[2023-10-09 14:31:04,207][86121] Updated weights for policy 0, policy_version 58350 (0.0010) +[2023-10-09 14:31:04,576][86121] Updated weights for policy 0, policy_version 58360 (0.0007) +[2023-10-09 14:31:05,953][86122] Updated weights for policy 1, policy_version 58600 (0.0008) +[2023-10-09 14:31:06,313][86122] Updated weights for policy 1, policy_version 58610 (0.0007) +[2023-10-09 14:31:06,686][86122] Updated weights for policy 1, policy_version 58620 (0.0010) +[2023-10-09 14:31:08,188][86121] Updated weights for policy 0, policy_version 58370 (0.0007) +[2023-10-09 14:31:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 119799808. Throughput: 0: 1813.6, 1: 1831.6. Samples: 29962776. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:31:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:31:08,562][86121] Updated weights for policy 0, policy_version 58380 (0.0009) +[2023-10-09 14:31:08,930][86121] Updated weights for policy 0, policy_version 58390 (0.0007) +[2023-10-09 14:31:09,292][86121] Updated weights for policy 0, policy_version 58400 (0.0008) +[2023-10-09 14:31:10,425][86122] Updated weights for policy 1, policy_version 58630 (0.0009) +[2023-10-09 14:31:10,790][86122] Updated weights for policy 1, policy_version 58640 (0.0009) +[2023-10-09 14:31:11,156][86122] Updated weights for policy 1, policy_version 58650 (0.0008) +[2023-10-09 14:31:12,945][86121] Updated weights for policy 0, policy_version 58410 (0.0008) +[2023-10-09 14:31:13,308][86121] Updated weights for policy 0, policy_version 58420 (0.0008) +[2023-10-09 14:31:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 119865344. Throughput: 0: 1815.0, 1: 1822.5. Samples: 29973446. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 14:31:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 14:31:13,671][86121] Updated weights for policy 0, policy_version 58430 (0.0007) +[2023-10-09 14:31:14,809][86122] Updated weights for policy 1, policy_version 58660 (0.0008) +[2023-10-09 14:31:15,183][86122] Updated weights for policy 1, policy_version 58670 (0.0007) +[2023-10-09 14:31:15,542][86122] Updated weights for policy 1, policy_version 58680 (0.0007) +[2023-10-09 14:31:17,372][86121] Updated weights for policy 0, policy_version 58440 (0.0010) +[2023-10-09 14:31:17,740][86121] Updated weights for policy 0, policy_version 58450 (0.0010) +[2023-10-09 14:31:18,109][86121] Updated weights for policy 0, policy_version 58460 (0.0008) +[2023-10-09 14:31:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 119963648. Throughput: 0: 1817.2, 1: 1834.5. Samples: 29995760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 14:31:19,180][86122] Updated weights for policy 1, policy_version 58690 (0.0008) +[2023-10-09 14:31:19,533][86122] Updated weights for policy 1, policy_version 58700 (0.0008) +[2023-10-09 14:31:19,896][86122] Updated weights for policy 1, policy_version 58710 (0.0007) +[2023-10-09 14:31:20,252][86122] Updated weights for policy 1, policy_version 58720 (0.0008) +[2023-10-09 14:31:21,711][86121] Updated weights for policy 0, policy_version 58470 (0.0009) +[2023-10-09 14:31:22,089][86121] Updated weights for policy 0, policy_version 58480 (0.0010) +[2023-10-09 14:31:22,451][86121] Updated weights for policy 0, policy_version 58490 (0.0011) +[2023-10-09 14:31:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 120029184. Throughput: 0: 1821.9, 1: 1847.5. Samples: 30017626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:31:23,405][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000058496_59899904.pth... +[2023-10-09 14:31:23,437][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000056800_58163200.pth +[2023-10-09 14:31:23,741][86122] Updated weights for policy 1, policy_version 58730 (0.0008) +[2023-10-09 14:31:24,103][86122] Updated weights for policy 1, policy_version 58740 (0.0009) +[2023-10-09 14:31:24,463][86122] Updated weights for policy 1, policy_version 58750 (0.0010) +[2023-10-09 14:31:24,534][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000058752_60162048.pth... +[2023-10-09 14:31:24,574][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000057024_58392576.pth +[2023-10-09 14:31:26,112][86121] Updated weights for policy 0, policy_version 58500 (0.0010) +[2023-10-09 14:31:26,474][86121] Updated weights for policy 0, policy_version 58510 (0.0009) +[2023-10-09 14:31:26,846][86121] Updated weights for policy 0, policy_version 58520 (0.0010) +[2023-10-09 14:31:28,090][86122] Updated weights for policy 1, policy_version 58760 (0.0008) +[2023-10-09 14:31:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120094720. Throughput: 0: 1828.2, 1: 1845.6. Samples: 30029292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:31:28,449][86122] Updated weights for policy 1, policy_version 58770 (0.0011) +[2023-10-09 14:31:28,806][86122] Updated weights for policy 1, policy_version 58780 (0.0009) +[2023-10-09 14:31:30,464][86121] Updated weights for policy 0, policy_version 58530 (0.0008) +[2023-10-09 14:31:30,829][86121] Updated weights for policy 0, policy_version 58540 (0.0009) +[2023-10-09 14:31:31,196][86121] Updated weights for policy 0, policy_version 58550 (0.0011) +[2023-10-09 14:31:31,549][86121] Updated weights for policy 0, policy_version 58560 (0.0008) +[2023-10-09 14:31:32,319][86122] Updated weights for policy 1, policy_version 58790 (0.0007) +[2023-10-09 14:31:32,682][86122] Updated weights for policy 1, policy_version 58800 (0.0007) +[2023-10-09 14:31:33,044][86122] Updated weights for policy 1, policy_version 58810 (0.0007) +[2023-10-09 14:31:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120193024. Throughput: 0: 1833.0, 1: 1845.7. Samples: 30050954. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.960')] +[2023-10-09 14:31:35,147][86121] Updated weights for policy 0, policy_version 58570 (0.0009) +[2023-10-09 14:31:35,522][86121] Updated weights for policy 0, policy_version 58580 (0.0007) +[2023-10-09 14:31:35,885][86121] Updated weights for policy 0, policy_version 58590 (0.0007) +[2023-10-09 14:31:36,787][86122] Updated weights for policy 1, policy_version 58820 (0.0008) +[2023-10-09 14:31:37,147][86122] Updated weights for policy 1, policy_version 58830 (0.0009) +[2023-10-09 14:31:37,511][86122] Updated weights for policy 1, policy_version 58840 (0.0007) +[2023-10-09 14:31:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120258560. Throughput: 0: 1839.2, 1: 1828.0. Samples: 30072908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.960')] +[2023-10-09 14:31:39,636][86121] Updated weights for policy 0, policy_version 58600 (0.0007) +[2023-10-09 14:31:40,002][86121] Updated weights for policy 0, policy_version 58610 (0.0008) +[2023-10-09 14:31:40,364][86121] Updated weights for policy 0, policy_version 58620 (0.0009) +[2023-10-09 14:31:41,201][86122] Updated weights for policy 1, policy_version 58850 (0.0008) +[2023-10-09 14:31:41,558][86122] Updated weights for policy 1, policy_version 58860 (0.0008) +[2023-10-09 14:31:41,923][86122] Updated weights for policy 1, policy_version 58870 (0.0009) +[2023-10-09 14:31:42,274][86122] Updated weights for policy 1, policy_version 58880 (0.0008) +[2023-10-09 14:31:43,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120324096. Throughput: 0: 1836.3, 1: 1845.9. Samples: 30084132. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:43,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.960')] +[2023-10-09 14:31:44,102][86121] Updated weights for policy 0, policy_version 58630 (0.0009) +[2023-10-09 14:31:44,461][86121] Updated weights for policy 0, policy_version 58640 (0.0008) +[2023-10-09 14:31:44,837][86121] Updated weights for policy 0, policy_version 58650 (0.0010) +[2023-10-09 14:31:45,925][86122] Updated weights for policy 1, policy_version 58890 (0.0009) +[2023-10-09 14:31:46,274][86122] Updated weights for policy 1, policy_version 58900 (0.0010) +[2023-10-09 14:31:46,629][86122] Updated weights for policy 1, policy_version 58910 (0.0010) +[2023-10-09 14:31:48,398][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 120389632. Throughput: 0: 1840.2, 1: 1828.3. Samples: 30105824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:48,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.960')] +[2023-10-09 14:31:48,511][86121] Updated weights for policy 0, policy_version 58660 (0.0009) +[2023-10-09 14:31:48,882][86121] Updated weights for policy 0, policy_version 58670 (0.0007) +[2023-10-09 14:31:49,249][86121] Updated weights for policy 0, policy_version 58680 (0.0008) +[2023-10-09 14:31:50,374][86122] Updated weights for policy 1, policy_version 58920 (0.0010) +[2023-10-09 14:31:50,735][86122] Updated weights for policy 1, policy_version 58930 (0.0009) +[2023-10-09 14:31:51,094][86122] Updated weights for policy 1, policy_version 58940 (0.0007) +[2023-10-09 14:31:52,941][86121] Updated weights for policy 0, policy_version 58690 (0.0007) +[2023-10-09 14:31:53,314][86121] Updated weights for policy 0, policy_version 58700 (0.0008) +[2023-10-09 14:31:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120455168. Throughput: 0: 1835.8, 1: 1848.7. Samples: 30128578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 14:31:53,679][86121] Updated weights for policy 0, policy_version 58710 (0.0009) +[2023-10-09 14:31:54,050][86121] Updated weights for policy 0, policy_version 58720 (0.0011) +[2023-10-09 14:31:54,651][86122] Updated weights for policy 1, policy_version 58950 (0.0007) +[2023-10-09 14:31:55,015][86122] Updated weights for policy 1, policy_version 58960 (0.0008) +[2023-10-09 14:31:55,376][86122] Updated weights for policy 1, policy_version 58970 (0.0008) +[2023-10-09 14:31:57,781][86121] Updated weights for policy 0, policy_version 58730 (0.0008) +[2023-10-09 14:31:58,145][86121] Updated weights for policy 0, policy_version 58740 (0.0008) +[2023-10-09 14:31:58,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 120520704. Throughput: 0: 1838.8, 1: 1836.1. Samples: 30138816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:31:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 14:31:58,508][86121] Updated weights for policy 0, policy_version 58750 (0.0007) +[2023-10-09 14:31:58,986][86122] Updated weights for policy 1, policy_version 58980 (0.0008) +[2023-10-09 14:31:59,350][86122] Updated weights for policy 1, policy_version 58990 (0.0009) +[2023-10-09 14:31:59,705][86122] Updated weights for policy 1, policy_version 59000 (0.0011) +[2023-10-09 14:32:02,196][86121] Updated weights for policy 0, policy_version 58760 (0.0009) +[2023-10-09 14:32:02,557][86121] Updated weights for policy 0, policy_version 58770 (0.0010) +[2023-10-09 14:32:02,927][86121] Updated weights for policy 0, policy_version 58780 (0.0010) +[2023-10-09 14:32:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 120619008. Throughput: 0: 1834.8, 1: 1846.6. Samples: 30161424. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 14:32:03,507][86122] Updated weights for policy 1, policy_version 59010 (0.0009) +[2023-10-09 14:32:03,863][86122] Updated weights for policy 1, policy_version 59020 (0.0009) +[2023-10-09 14:32:04,214][86122] Updated weights for policy 1, policy_version 59030 (0.0007) +[2023-10-09 14:32:04,581][86122] Updated weights for policy 1, policy_version 59040 (0.0007) +[2023-10-09 14:32:06,624][86121] Updated weights for policy 0, policy_version 58790 (0.0010) +[2023-10-09 14:32:07,003][86121] Updated weights for policy 0, policy_version 58800 (0.0009) +[2023-10-09 14:32:07,370][86121] Updated weights for policy 0, policy_version 58810 (0.0009) +[2023-10-09 14:32:08,243][86122] Updated weights for policy 1, policy_version 59050 (0.0010) +[2023-10-09 14:32:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120684544. Throughput: 0: 1827.6, 1: 1841.4. Samples: 30182730. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 14:32:08,608][86122] Updated weights for policy 1, policy_version 59060 (0.0008) +[2023-10-09 14:32:08,969][86122] Updated weights for policy 1, policy_version 59070 (0.0009) +[2023-10-09 14:32:11,124][86121] Updated weights for policy 0, policy_version 58820 (0.0008) +[2023-10-09 14:32:11,488][86121] Updated weights for policy 0, policy_version 58830 (0.0007) +[2023-10-09 14:32:11,859][86121] Updated weights for policy 0, policy_version 58840 (0.0007) +[2023-10-09 14:32:12,621][86122] Updated weights for policy 1, policy_version 59080 (0.0007) +[2023-10-09 14:32:12,987][86122] Updated weights for policy 1, policy_version 59090 (0.0008) +[2023-10-09 14:32:13,349][86122] Updated weights for policy 1, policy_version 59100 (0.0008) +[2023-10-09 14:32:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120750080. Throughput: 0: 1819.2, 1: 1841.8. Samples: 30194034. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.970')] +[2023-10-09 14:32:15,429][86121] Updated weights for policy 0, policy_version 58850 (0.0008) +[2023-10-09 14:32:15,799][86121] Updated weights for policy 0, policy_version 58860 (0.0010) +[2023-10-09 14:32:16,158][86121] Updated weights for policy 0, policy_version 58870 (0.0009) +[2023-10-09 14:32:16,526][86121] Updated weights for policy 0, policy_version 58880 (0.0008) +[2023-10-09 14:32:17,079][86122] Updated weights for policy 1, policy_version 59110 (0.0009) +[2023-10-09 14:32:17,438][86122] Updated weights for policy 1, policy_version 59120 (0.0008) +[2023-10-09 14:32:17,801][86122] Updated weights for policy 1, policy_version 59130 (0.0007) +[2023-10-09 14:32:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 120848384. Throughput: 0: 1822.3, 1: 1839.5. Samples: 30215732. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:18,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.970')] +[2023-10-09 14:32:20,186][86121] Updated weights for policy 0, policy_version 58890 (0.0009) +[2023-10-09 14:32:20,551][86121] Updated weights for policy 0, policy_version 58900 (0.0008) +[2023-10-09 14:32:20,913][86121] Updated weights for policy 0, policy_version 58910 (0.0010) +[2023-10-09 14:32:21,549][86122] Updated weights for policy 1, policy_version 59140 (0.0008) +[2023-10-09 14:32:21,918][86122] Updated weights for policy 1, policy_version 59150 (0.0007) +[2023-10-09 14:32:22,284][86122] Updated weights for policy 1, policy_version 59160 (0.0008) +[2023-10-09 14:32:23,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 120913920. Throughput: 0: 1816.8, 1: 1832.0. Samples: 30237104. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.970')] +[2023-10-09 14:32:24,649][86121] Updated weights for policy 0, policy_version 58920 (0.0009) +[2023-10-09 14:32:25,013][86121] Updated weights for policy 0, policy_version 58930 (0.0010) +[2023-10-09 14:32:25,382][86121] Updated weights for policy 0, policy_version 58940 (0.0009) +[2023-10-09 14:32:25,982][86122] Updated weights for policy 1, policy_version 59170 (0.0009) +[2023-10-09 14:32:26,351][86122] Updated weights for policy 1, policy_version 59180 (0.0009) +[2023-10-09 14:32:26,727][86122] Updated weights for policy 1, policy_version 59190 (0.0008) +[2023-10-09 14:32:27,085][86122] Updated weights for policy 1, policy_version 59200 (0.0007) +[2023-10-09 14:32:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 120979456. Throughput: 0: 1821.6, 1: 1832.9. Samples: 30248584. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:28,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 14:32:28,957][86121] Updated weights for policy 0, policy_version 58950 (0.0009) +[2023-10-09 14:32:29,325][86121] Updated weights for policy 0, policy_version 58960 (0.0008) +[2023-10-09 14:32:29,687][86121] Updated weights for policy 0, policy_version 58970 (0.0010) +[2023-10-09 14:32:30,824][86122] Updated weights for policy 1, policy_version 59210 (0.0008) +[2023-10-09 14:32:31,181][86122] Updated weights for policy 1, policy_version 59220 (0.0007) +[2023-10-09 14:32:31,535][86122] Updated weights for policy 1, policy_version 59230 (0.0010) +[2023-10-09 14:32:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 121044992. Throughput: 0: 1815.8, 1: 1827.3. Samples: 30269762. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:33,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 14:32:33,688][86121] Updated weights for policy 0, policy_version 58980 (0.0009) +[2023-10-09 14:32:34,048][86121] Updated weights for policy 0, policy_version 58990 (0.0008) +[2023-10-09 14:32:34,416][86121] Updated weights for policy 0, policy_version 59000 (0.0007) +[2023-10-09 14:32:35,207][86122] Updated weights for policy 1, policy_version 59240 (0.0010) +[2023-10-09 14:32:35,572][86122] Updated weights for policy 1, policy_version 59250 (0.0011) +[2023-10-09 14:32:35,939][86122] Updated weights for policy 1, policy_version 59260 (0.0007) +[2023-10-09 14:32:37,984][86121] Updated weights for policy 0, policy_version 59010 (0.0008) +[2023-10-09 14:32:38,363][86121] Updated weights for policy 0, policy_version 59020 (0.0008) +[2023-10-09 14:32:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121110528. Throughput: 0: 1817.3, 1: 1833.8. Samples: 30292876. Policy #0 lag: (min: 2.0, avg: 4.3, max: 33.0) +[2023-10-09 14:32:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 14:32:38,723][86121] Updated weights for policy 0, policy_version 59030 (0.0008) +[2023-10-09 14:32:39,094][86121] Updated weights for policy 0, policy_version 59040 (0.0009) +[2023-10-09 14:32:39,373][86122] Updated weights for policy 1, policy_version 59270 (0.0009) +[2023-10-09 14:32:39,722][86122] Updated weights for policy 1, policy_version 59280 (0.0009) +[2023-10-09 14:32:40,083][86122] Updated weights for policy 1, policy_version 59290 (0.0008) +[2023-10-09 14:32:42,722][86121] Updated weights for policy 0, policy_version 59050 (0.0008) +[2023-10-09 14:32:43,095][86121] Updated weights for policy 0, policy_version 59060 (0.0010) +[2023-10-09 14:32:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121176064. Throughput: 0: 1814.9, 1: 1833.2. Samples: 30302982. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:32:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 14:32:43,452][86121] Updated weights for policy 0, policy_version 59070 (0.0010) +[2023-10-09 14:32:43,739][86122] Updated weights for policy 1, policy_version 59300 (0.0009) +[2023-10-09 14:32:44,107][86122] Updated weights for policy 1, policy_version 59310 (0.0009) +[2023-10-09 14:32:44,467][86122] Updated weights for policy 1, policy_version 59320 (0.0007) +[2023-10-09 14:32:47,190][86121] Updated weights for policy 0, policy_version 59080 (0.0009) +[2023-10-09 14:32:47,559][86121] Updated weights for policy 0, policy_version 59090 (0.0008) +[2023-10-09 14:32:47,921][86121] Updated weights for policy 0, policy_version 59100 (0.0008) +[2023-10-09 14:32:48,234][86122] Updated weights for policy 1, policy_version 59330 (0.0007) +[2023-10-09 14:32:48,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121274368. Throughput: 0: 1818.0, 1: 1833.6. Samples: 30325748. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:32:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 14:32:48,594][86122] Updated weights for policy 1, policy_version 59340 (0.0011) +[2023-10-09 14:32:48,961][86122] Updated weights for policy 1, policy_version 59350 (0.0009) +[2023-10-09 14:32:49,320][86122] Updated weights for policy 1, policy_version 59360 (0.0009) +[2023-10-09 14:32:51,569][86121] Updated weights for policy 0, policy_version 59110 (0.0007) +[2023-10-09 14:32:51,927][86121] Updated weights for policy 0, policy_version 59120 (0.0008) +[2023-10-09 14:32:52,296][86121] Updated weights for policy 0, policy_version 59130 (0.0008) +[2023-10-09 14:32:53,063][86122] Updated weights for policy 1, policy_version 59370 (0.0008) +[2023-10-09 14:32:53,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121339904. Throughput: 0: 1820.8, 1: 1827.5. Samples: 30346902. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:32:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 14:32:53,435][86122] Updated weights for policy 1, policy_version 59380 (0.0009) +[2023-10-09 14:32:53,799][86122] Updated weights for policy 1, policy_version 59390 (0.0008) +[2023-10-09 14:32:56,023][86121] Updated weights for policy 0, policy_version 59140 (0.0007) +[2023-10-09 14:32:56,389][86121] Updated weights for policy 0, policy_version 59150 (0.0008) +[2023-10-09 14:32:56,759][86121] Updated weights for policy 0, policy_version 59160 (0.0007) +[2023-10-09 14:32:57,401][86122] Updated weights for policy 1, policy_version 59400 (0.0008) +[2023-10-09 14:32:57,770][86122] Updated weights for policy 1, policy_version 59410 (0.0009) +[2023-10-09 14:32:58,129][86122] Updated weights for policy 1, policy_version 59420 (0.0007) +[2023-10-09 14:32:58,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 121438208. Throughput: 0: 1821.8, 1: 1831.5. Samples: 30358432. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:32:58,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 14:33:00,267][86121] Updated weights for policy 0, policy_version 59170 (0.0007) +[2023-10-09 14:33:00,636][86121] Updated weights for policy 0, policy_version 59180 (0.0008) +[2023-10-09 14:33:00,996][86121] Updated weights for policy 0, policy_version 59190 (0.0008) +[2023-10-09 14:33:01,356][86121] Updated weights for policy 0, policy_version 59200 (0.0009) +[2023-10-09 14:33:01,721][86122] Updated weights for policy 1, policy_version 59430 (0.0009) +[2023-10-09 14:33:02,078][86122] Updated weights for policy 1, policy_version 59440 (0.0010) +[2023-10-09 14:33:02,441][86122] Updated weights for policy 1, policy_version 59450 (0.0008) +[2023-10-09 14:33:03,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121503744. Throughput: 0: 1824.4, 1: 1824.7. Samples: 30379942. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:33:03,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.970')] +[2023-10-09 14:33:05,248][86121] Updated weights for policy 0, policy_version 59210 (0.0010) +[2023-10-09 14:33:05,614][86121] Updated weights for policy 0, policy_version 59220 (0.0007) +[2023-10-09 14:33:05,977][86121] Updated weights for policy 0, policy_version 59230 (0.0007) +[2023-10-09 14:33:06,219][86122] Updated weights for policy 1, policy_version 59460 (0.0009) +[2023-10-09 14:33:06,580][86122] Updated weights for policy 1, policy_version 59470 (0.0008) +[2023-10-09 14:33:06,946][86122] Updated weights for policy 1, policy_version 59480 (0.0009) +[2023-10-09 14:33:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 121569280. Throughput: 0: 1821.1, 1: 1835.9. Samples: 30401668. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:33:08,399][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 14:33:09,518][86121] Updated weights for policy 0, policy_version 59240 (0.0008) +[2023-10-09 14:33:09,889][86121] Updated weights for policy 0, policy_version 59250 (0.0009) +[2023-10-09 14:33:10,254][86121] Updated weights for policy 0, policy_version 59260 (0.0010) +[2023-10-09 14:33:10,653][86122] Updated weights for policy 1, policy_version 59490 (0.0008) +[2023-10-09 14:33:11,020][86122] Updated weights for policy 1, policy_version 59500 (0.0008) +[2023-10-09 14:33:11,381][86122] Updated weights for policy 1, policy_version 59510 (0.0007) +[2023-10-09 14:33:11,734][86122] Updated weights for policy 1, policy_version 59520 (0.0008) +[2023-10-09 14:33:13,398][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 121634816. Throughput: 0: 1821.6, 1: 1827.1. Samples: 30412774. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:33:13,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:33:13,918][86121] Updated weights for policy 0, policy_version 59270 (0.0007) +[2023-10-09 14:33:14,279][86121] Updated weights for policy 0, policy_version 59280 (0.0007) +[2023-10-09 14:33:14,647][86121] Updated weights for policy 0, policy_version 59290 (0.0007) +[2023-10-09 14:33:15,315][86122] Updated weights for policy 1, policy_version 59530 (0.0008) +[2023-10-09 14:33:15,675][86122] Updated weights for policy 1, policy_version 59540 (0.0008) +[2023-10-09 14:33:16,034][86122] Updated weights for policy 1, policy_version 59550 (0.0007) +[2023-10-09 14:33:18,248][86121] Updated weights for policy 0, policy_version 59300 (0.0009) +[2023-10-09 14:33:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 121700352. Throughput: 0: 1830.8, 1: 1841.6. Samples: 30435022. Policy #0 lag: (min: 5.0, avg: 5.0, max: 7.0) +[2023-10-09 14:33:18,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:33:18,618][86121] Updated weights for policy 0, policy_version 59310 (0.0009) +[2023-10-09 14:33:18,977][86121] Updated weights for policy 0, policy_version 59320 (0.0010) +[2023-10-09 14:33:19,792][86122] Updated weights for policy 1, policy_version 59560 (0.0008) +[2023-10-09 14:33:20,174][86122] Updated weights for policy 1, policy_version 59570 (0.0008) +[2023-10-09 14:33:20,528][86122] Updated weights for policy 1, policy_version 59580 (0.0008) +[2023-10-09 14:33:22,714][86121] Updated weights for policy 0, policy_version 59330 (0.0007) +[2023-10-09 14:33:23,084][86121] Updated weights for policy 0, policy_version 59340 (0.0007) +[2023-10-09 14:33:23,398][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 121765888. Throughput: 0: 1827.1, 1: 1836.5. Samples: 30457740. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:23,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:33:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000059584_61014016.pth... +[2023-10-09 14:33:23,448][86121] Updated weights for policy 0, policy_version 59350 (0.0008) +[2023-10-09 14:33:23,451][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000057888_59277312.pth +[2023-10-09 14:33:23,456][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000059584_61014016.pth +[2023-10-09 14:33:23,811][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000059360_60784640.pth... +[2023-10-09 14:33:23,811][86121] Updated weights for policy 0, policy_version 59360 (0.0007) +[2023-10-09 14:33:23,850][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000057632_59015168.pth +[2023-10-09 14:33:23,854][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000059360_60784640.pth +[2023-10-09 14:33:23,923][86122] Updated weights for policy 1, policy_version 59590 (0.0007) +[2023-10-09 14:33:24,280][86122] Updated weights for policy 1, policy_version 59600 (0.0009) +[2023-10-09 14:33:24,645][86122] Updated weights for policy 1, policy_version 59610 (0.0010) +[2023-10-09 14:33:27,509][86121] Updated weights for policy 0, policy_version 59370 (0.0008) +[2023-10-09 14:33:27,885][86121] Updated weights for policy 0, policy_version 59380 (0.0009) +[2023-10-09 14:33:28,256][86121] Updated weights for policy 0, policy_version 59390 (0.0009) +[2023-10-09 14:33:28,357][86122] Updated weights for policy 1, policy_version 59620 (0.0010) +[2023-10-09 14:33:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121864192. Throughput: 0: 1833.7, 1: 1833.0. Samples: 30467984. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 14:33:28,714][86122] Updated weights for policy 1, policy_version 59630 (0.0011) +[2023-10-09 14:33:29,065][86122] Updated weights for policy 1, policy_version 59640 (0.0010) +[2023-10-09 14:33:31,893][86121] Updated weights for policy 0, policy_version 59400 (0.0009) +[2023-10-09 14:33:32,263][86121] Updated weights for policy 0, policy_version 59410 (0.0008) +[2023-10-09 14:33:32,632][86121] Updated weights for policy 0, policy_version 59420 (0.0008) +[2023-10-09 14:33:32,787][86122] Updated weights for policy 1, policy_version 59650 (0.0009) +[2023-10-09 14:33:33,149][86122] Updated weights for policy 1, policy_version 59660 (0.0008) +[2023-10-09 14:33:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 121929728. Throughput: 0: 1825.4, 1: 1838.3. Samples: 30490614. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:33:33,519][86122] Updated weights for policy 1, policy_version 59670 (0.0007) +[2023-10-09 14:33:33,883][86122] Updated weights for policy 1, policy_version 59680 (0.0008) +[2023-10-09 14:33:36,223][86121] Updated weights for policy 0, policy_version 59430 (0.0010) +[2023-10-09 14:33:36,588][86121] Updated weights for policy 0, policy_version 59440 (0.0009) +[2023-10-09 14:33:36,955][86121] Updated weights for policy 0, policy_version 59450 (0.0011) +[2023-10-09 14:33:37,497][86122] Updated weights for policy 1, policy_version 59690 (0.0010) +[2023-10-09 14:33:37,859][86122] Updated weights for policy 1, policy_version 59700 (0.0008) +[2023-10-09 14:33:38,214][86122] Updated weights for policy 1, policy_version 59710 (0.0009) +[2023-10-09 14:33:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 122028032. Throughput: 0: 1841.0, 1: 1830.0. Samples: 30512094. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:33:40,725][86121] Updated weights for policy 0, policy_version 59460 (0.0010) +[2023-10-09 14:33:41,098][86121] Updated weights for policy 0, policy_version 59470 (0.0011) +[2023-10-09 14:33:41,463][86121] Updated weights for policy 0, policy_version 59480 (0.0008) +[2023-10-09 14:33:41,942][86122] Updated weights for policy 1, policy_version 59720 (0.0010) +[2023-10-09 14:33:42,303][86122] Updated weights for policy 1, policy_version 59730 (0.0011) +[2023-10-09 14:33:42,668][86122] Updated weights for policy 1, policy_version 59740 (0.0011) +[2023-10-09 14:33:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 122093568. Throughput: 0: 1830.3, 1: 1842.4. Samples: 30523702. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:33:45,102][86121] Updated weights for policy 0, policy_version 59490 (0.0007) +[2023-10-09 14:33:45,468][86121] Updated weights for policy 0, policy_version 59500 (0.0009) +[2023-10-09 14:33:45,833][86121] Updated weights for policy 0, policy_version 59510 (0.0010) +[2023-10-09 14:33:46,195][86121] Updated weights for policy 0, policy_version 59520 (0.0008) +[2023-10-09 14:33:46,282][86122] Updated weights for policy 1, policy_version 59750 (0.0009) +[2023-10-09 14:33:46,638][86122] Updated weights for policy 1, policy_version 59760 (0.0011) +[2023-10-09 14:33:46,998][86122] Updated weights for policy 1, policy_version 59770 (0.0011) +[2023-10-09 14:33:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 122159104. Throughput: 0: 1835.3, 1: 1825.9. Samples: 30544694. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:33:50,059][86121] Updated weights for policy 0, policy_version 59530 (0.0007) +[2023-10-09 14:33:50,424][86121] Updated weights for policy 0, policy_version 59540 (0.0008) +[2023-10-09 14:33:50,717][86122] Updated weights for policy 1, policy_version 59780 (0.0009) +[2023-10-09 14:33:50,785][86121] Updated weights for policy 0, policy_version 59550 (0.0008) +[2023-10-09 14:33:51,078][86122] Updated weights for policy 1, policy_version 59790 (0.0010) +[2023-10-09 14:33:51,452][86122] Updated weights for policy 1, policy_version 59800 (0.0008) +[2023-10-09 14:33:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122224640. Throughput: 0: 1841.3, 1: 1838.7. Samples: 30567268. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:33:54,316][86121] Updated weights for policy 0, policy_version 59560 (0.0009) +[2023-10-09 14:33:54,675][86121] Updated weights for policy 0, policy_version 59570 (0.0009) +[2023-10-09 14:33:55,034][86121] Updated weights for policy 0, policy_version 59580 (0.0009) +[2023-10-09 14:33:55,093][86122] Updated weights for policy 1, policy_version 59810 (0.0009) +[2023-10-09 14:33:55,456][86122] Updated weights for policy 1, policy_version 59820 (0.0009) +[2023-10-09 14:33:55,810][86122] Updated weights for policy 1, policy_version 59830 (0.0011) +[2023-10-09 14:33:56,173][86122] Updated weights for policy 1, policy_version 59840 (0.0010) +[2023-10-09 14:33:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122290176. Throughput: 0: 1843.1, 1: 1828.1. Samples: 30577978. Policy #0 lag: (min: 24.0, avg: 52.0, max: 56.0) +[2023-10-09 14:33:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:33:58,659][86121] Updated weights for policy 0, policy_version 59590 (0.0008) +[2023-10-09 14:33:59,029][86121] Updated weights for policy 0, policy_version 59600 (0.0009) +[2023-10-09 14:33:59,389][86121] Updated weights for policy 0, policy_version 59610 (0.0009) +[2023-10-09 14:33:59,956][86122] Updated weights for policy 1, policy_version 59850 (0.0008) +[2023-10-09 14:34:00,311][86122] Updated weights for policy 1, policy_version 59860 (0.0009) +[2023-10-09 14:34:00,667][86122] Updated weights for policy 1, policy_version 59870 (0.0010) +[2023-10-09 14:34:03,089][86121] Updated weights for policy 0, policy_version 59620 (0.0008) +[2023-10-09 14:34:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122355712. Throughput: 0: 1835.7, 1: 1841.7. Samples: 30600506. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.960')] +[2023-10-09 14:34:03,460][86121] Updated weights for policy 0, policy_version 59630 (0.0007) +[2023-10-09 14:34:03,816][86121] Updated weights for policy 0, policy_version 59640 (0.0008) +[2023-10-09 14:34:04,300][86122] Updated weights for policy 1, policy_version 59880 (0.0008) +[2023-10-09 14:34:04,664][86122] Updated weights for policy 1, policy_version 59890 (0.0007) +[2023-10-09 14:34:05,021][86122] Updated weights for policy 1, policy_version 59900 (0.0007) +[2023-10-09 14:34:07,432][86121] Updated weights for policy 0, policy_version 59650 (0.0009) +[2023-10-09 14:34:07,791][86121] Updated weights for policy 0, policy_version 59660 (0.0007) +[2023-10-09 14:34:08,154][86121] Updated weights for policy 0, policy_version 59670 (0.0008) +[2023-10-09 14:34:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 122421248. Throughput: 0: 1829.3, 1: 1840.9. Samples: 30622894. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 14:34:08,522][86121] Updated weights for policy 0, policy_version 59680 (0.0009) +[2023-10-09 14:34:08,773][86122] Updated weights for policy 1, policy_version 59910 (0.0009) +[2023-10-09 14:34:09,152][86122] Updated weights for policy 1, policy_version 59920 (0.0009) +[2023-10-09 14:34:09,519][86122] Updated weights for policy 1, policy_version 59930 (0.0008) +[2023-10-09 14:34:12,183][86121] Updated weights for policy 0, policy_version 59690 (0.0009) +[2023-10-09 14:34:12,553][86121] Updated weights for policy 0, policy_version 59700 (0.0010) +[2023-10-09 14:34:12,912][86121] Updated weights for policy 0, policy_version 59710 (0.0007) +[2023-10-09 14:34:13,107][86122] Updated weights for policy 1, policy_version 59940 (0.0007) +[2023-10-09 14:34:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 122519552. Throughput: 0: 1838.5, 1: 1839.5. Samples: 30633496. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 14:34:13,479][86122] Updated weights for policy 1, policy_version 59950 (0.0008) +[2023-10-09 14:34:13,837][86122] Updated weights for policy 1, policy_version 59960 (0.0010) +[2023-10-09 14:34:16,600][86121] Updated weights for policy 0, policy_version 59720 (0.0007) +[2023-10-09 14:34:16,971][86121] Updated weights for policy 0, policy_version 59730 (0.0009) +[2023-10-09 14:34:17,344][86121] Updated weights for policy 0, policy_version 59740 (0.0007) +[2023-10-09 14:34:17,395][86122] Updated weights for policy 1, policy_version 59970 (0.0009) +[2023-10-09 14:34:17,766][86122] Updated weights for policy 1, policy_version 59980 (0.0007) +[2023-10-09 14:34:18,121][86122] Updated weights for policy 1, policy_version 59990 (0.0010) +[2023-10-09 14:34:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122585088. Throughput: 0: 1831.7, 1: 1840.8. Samples: 30655876. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.960')] +[2023-10-09 14:34:18,481][86122] Updated weights for policy 1, policy_version 60000 (0.0009) +[2023-10-09 14:34:20,971][86121] Updated weights for policy 0, policy_version 59750 (0.0009) +[2023-10-09 14:34:21,331][86121] Updated weights for policy 0, policy_version 59760 (0.0010) +[2023-10-09 14:34:21,691][86121] Updated weights for policy 0, policy_version 59770 (0.0009) +[2023-10-09 14:34:22,127][86122] Updated weights for policy 1, policy_version 60010 (0.0009) +[2023-10-09 14:34:22,500][86122] Updated weights for policy 1, policy_version 60020 (0.0011) +[2023-10-09 14:34:22,870][86122] Updated weights for policy 1, policy_version 60030 (0.0010) +[2023-10-09 14:34:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14773.4). Total num frames: 122683392. Throughput: 0: 1842.4, 1: 1826.7. Samples: 30677202. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:34:25,311][86121] Updated weights for policy 0, policy_version 59780 (0.0008) +[2023-10-09 14:34:25,673][86121] Updated weights for policy 0, policy_version 59790 (0.0009) +[2023-10-09 14:34:26,040][86121] Updated weights for policy 0, policy_version 59800 (0.0010) +[2023-10-09 14:34:26,493][86122] Updated weights for policy 1, policy_version 60040 (0.0008) +[2023-10-09 14:34:26,856][86122] Updated weights for policy 1, policy_version 60050 (0.0010) +[2023-10-09 14:34:27,222][86122] Updated weights for policy 1, policy_version 60060 (0.0008) +[2023-10-09 14:34:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122748928. Throughput: 0: 1833.4, 1: 1837.6. Samples: 30688900. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:34:29,641][86121] Updated weights for policy 0, policy_version 59810 (0.0010) +[2023-10-09 14:34:29,999][86121] Updated weights for policy 0, policy_version 59820 (0.0007) +[2023-10-09 14:34:30,369][86121] Updated weights for policy 0, policy_version 59830 (0.0009) +[2023-10-09 14:34:30,736][86121] Updated weights for policy 0, policy_version 59840 (0.0007) +[2023-10-09 14:34:31,002][86122] Updated weights for policy 1, policy_version 60070 (0.0007) +[2023-10-09 14:34:31,371][86122] Updated weights for policy 1, policy_version 60080 (0.0009) +[2023-10-09 14:34:31,745][86122] Updated weights for policy 1, policy_version 60090 (0.0011) +[2023-10-09 14:34:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 122814464. Throughput: 0: 1845.4, 1: 1828.4. Samples: 30710016. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:34:34,256][86121] Updated weights for policy 0, policy_version 59850 (0.0009) +[2023-10-09 14:34:34,623][86121] Updated weights for policy 0, policy_version 59860 (0.0009) +[2023-10-09 14:34:34,977][86121] Updated weights for policy 0, policy_version 59870 (0.0007) +[2023-10-09 14:34:35,443][86122] Updated weights for policy 1, policy_version 60100 (0.0010) +[2023-10-09 14:34:35,814][86122] Updated weights for policy 1, policy_version 60110 (0.0009) +[2023-10-09 14:34:36,175][86122] Updated weights for policy 1, policy_version 60120 (0.0007) +[2023-10-09 14:34:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 122880000. Throughput: 0: 1842.3, 1: 1841.2. Samples: 30733028. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:34:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:34:38,738][86121] Updated weights for policy 0, policy_version 59880 (0.0008) +[2023-10-09 14:34:39,109][86121] Updated weights for policy 0, policy_version 59890 (0.0010) +[2023-10-09 14:34:39,475][86121] Updated weights for policy 0, policy_version 59900 (0.0008) +[2023-10-09 14:34:39,865][86122] Updated weights for policy 1, policy_version 60130 (0.0010) +[2023-10-09 14:34:40,219][86122] Updated weights for policy 1, policy_version 60140 (0.0008) +[2023-10-09 14:34:40,579][86122] Updated weights for policy 1, policy_version 60150 (0.0009) +[2023-10-09 14:34:40,941][86122] Updated weights for policy 1, policy_version 60160 (0.0007) +[2023-10-09 14:34:43,229][86121] Updated weights for policy 0, policy_version 59910 (0.0007) +[2023-10-09 14:34:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 122945536. Throughput: 0: 1838.1, 1: 1833.4. Samples: 30743198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:34:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 14:34:43,589][86121] Updated weights for policy 0, policy_version 59920 (0.0007) +[2023-10-09 14:34:43,959][86121] Updated weights for policy 0, policy_version 59930 (0.0009) +[2023-10-09 14:34:44,782][86122] Updated weights for policy 1, policy_version 60170 (0.0009) +[2023-10-09 14:34:45,144][86122] Updated weights for policy 1, policy_version 60180 (0.0009) +[2023-10-09 14:34:45,505][86122] Updated weights for policy 1, policy_version 60190 (0.0008) +[2023-10-09 14:34:47,724][86121] Updated weights for policy 0, policy_version 59940 (0.0008) +[2023-10-09 14:34:48,092][86121] Updated weights for policy 0, policy_version 59950 (0.0010) +[2023-10-09 14:34:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123011072. Throughput: 0: 1835.0, 1: 1834.4. Samples: 30765632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:34:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:34:48,457][86121] Updated weights for policy 0, policy_version 59960 (0.0009) +[2023-10-09 14:34:49,181][86122] Updated weights for policy 1, policy_version 60200 (0.0008) +[2023-10-09 14:34:49,540][86122] Updated weights for policy 1, policy_version 60210 (0.0009) +[2023-10-09 14:34:49,905][86122] Updated weights for policy 1, policy_version 60220 (0.0008) +[2023-10-09 14:34:52,180][86121] Updated weights for policy 0, policy_version 59970 (0.0008) +[2023-10-09 14:34:52,532][86121] Updated weights for policy 0, policy_version 59980 (0.0007) +[2023-10-09 14:34:52,893][86121] Updated weights for policy 0, policy_version 59990 (0.0007) +[2023-10-09 14:34:53,258][86121] Updated weights for policy 0, policy_version 60000 (0.0009) +[2023-10-09 14:34:53,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123109376. Throughput: 0: 1823.5, 1: 1835.9. Samples: 30787566. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:34:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:34:53,623][86122] Updated weights for policy 1, policy_version 60230 (0.0008) +[2023-10-09 14:34:54,003][86122] Updated weights for policy 1, policy_version 60240 (0.0010) +[2023-10-09 14:34:54,364][86122] Updated weights for policy 1, policy_version 60250 (0.0007) +[2023-10-09 14:34:56,768][86121] Updated weights for policy 0, policy_version 60010 (0.0010) +[2023-10-09 14:34:57,126][86121] Updated weights for policy 0, policy_version 60020 (0.0009) +[2023-10-09 14:34:57,495][86121] Updated weights for policy 0, policy_version 60030 (0.0007) +[2023-10-09 14:34:57,883][86122] Updated weights for policy 1, policy_version 60260 (0.0007) +[2023-10-09 14:34:58,238][86122] Updated weights for policy 1, policy_version 60270 (0.0011) +[2023-10-09 14:34:58,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123174912. Throughput: 0: 1831.5, 1: 1836.9. Samples: 30798576. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:34:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 14:34:58,606][86122] Updated weights for policy 1, policy_version 60280 (0.0011) +[2023-10-09 14:35:01,306][86121] Updated weights for policy 0, policy_version 60040 (0.0008) +[2023-10-09 14:35:01,668][86121] Updated weights for policy 0, policy_version 60050 (0.0009) +[2023-10-09 14:35:02,036][86121] Updated weights for policy 0, policy_version 60060 (0.0010) +[2023-10-09 14:35:02,226][86122] Updated weights for policy 1, policy_version 60290 (0.0009) +[2023-10-09 14:35:02,584][86122] Updated weights for policy 1, policy_version 60300 (0.0009) +[2023-10-09 14:35:02,943][86122] Updated weights for policy 1, policy_version 60310 (0.0010) +[2023-10-09 14:35:03,298][86122] Updated weights for policy 1, policy_version 60320 (0.0011) +[2023-10-09 14:35:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 123273216. Throughput: 0: 1819.1, 1: 1838.3. Samples: 30820458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:35:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 14:35:05,576][86121] Updated weights for policy 0, policy_version 60070 (0.0008) +[2023-10-09 14:35:05,946][86121] Updated weights for policy 0, policy_version 60080 (0.0007) +[2023-10-09 14:35:06,316][86121] Updated weights for policy 0, policy_version 60090 (0.0007) +[2023-10-09 14:35:07,048][86122] Updated weights for policy 1, policy_version 60330 (0.0007) +[2023-10-09 14:35:07,411][86122] Updated weights for policy 1, policy_version 60340 (0.0007) +[2023-10-09 14:35:07,769][86122] Updated weights for policy 1, policy_version 60350 (0.0007) +[2023-10-09 14:35:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 123338752. Throughput: 0: 1826.4, 1: 1834.2. Samples: 30841932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:35:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 14:35:10,168][86121] Updated weights for policy 0, policy_version 60100 (0.0009) +[2023-10-09 14:35:10,537][86121] Updated weights for policy 0, policy_version 60110 (0.0010) +[2023-10-09 14:35:10,902][86121] Updated weights for policy 0, policy_version 60120 (0.0009) +[2023-10-09 14:35:11,452][86122] Updated weights for policy 1, policy_version 60360 (0.0009) +[2023-10-09 14:35:11,811][86122] Updated weights for policy 1, policy_version 60370 (0.0008) +[2023-10-09 14:35:12,177][86122] Updated weights for policy 1, policy_version 60380 (0.0008) +[2023-10-09 14:35:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 123404288. Throughput: 0: 1818.4, 1: 1838.4. Samples: 30853458. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:35:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 14:35:14,614][86121] Updated weights for policy 0, policy_version 60130 (0.0009) +[2023-10-09 14:35:14,969][86121] Updated weights for policy 0, policy_version 60140 (0.0007) +[2023-10-09 14:35:15,341][86121] Updated weights for policy 0, policy_version 60150 (0.0007) +[2023-10-09 14:35:15,703][86121] Updated weights for policy 0, policy_version 60160 (0.0007) +[2023-10-09 14:35:15,979][86122] Updated weights for policy 1, policy_version 60390 (0.0009) +[2023-10-09 14:35:16,340][86122] Updated weights for policy 1, policy_version 60400 (0.0010) +[2023-10-09 14:35:16,711][86122] Updated weights for policy 1, policy_version 60410 (0.0007) +[2023-10-09 14:35:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123469824. Throughput: 0: 1823.0, 1: 1832.8. Samples: 30874524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:35:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 14:35:19,550][86121] Updated weights for policy 0, policy_version 60170 (0.0010) +[2023-10-09 14:35:19,920][86121] Updated weights for policy 0, policy_version 60180 (0.0009) +[2023-10-09 14:35:20,242][86122] Updated weights for policy 1, policy_version 60420 (0.0008) +[2023-10-09 14:35:20,286][86121] Updated weights for policy 0, policy_version 60190 (0.0008) +[2023-10-09 14:35:20,594][86122] Updated weights for policy 1, policy_version 60430 (0.0010) +[2023-10-09 14:35:20,949][86122] Updated weights for policy 1, policy_version 60440 (0.0009) +[2023-10-09 14:35:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 123535360. Throughput: 0: 1814.8, 1: 1832.6. Samples: 30897164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:23,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.950')] +[2023-10-09 14:35:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000060192_61636608.pth... +[2023-10-09 14:35:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000060448_61898752.pth... +[2023-10-09 14:35:23,441][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000058496_59899904.pth +[2023-10-09 14:35:23,451][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000058752_60162048.pth +[2023-10-09 14:35:24,021][86121] Updated weights for policy 0, policy_version 60200 (0.0008) +[2023-10-09 14:35:24,396][86121] Updated weights for policy 0, policy_version 60210 (0.0007) +[2023-10-09 14:35:24,622][86122] Updated weights for policy 1, policy_version 60450 (0.0010) +[2023-10-09 14:35:24,754][86121] Updated weights for policy 0, policy_version 60220 (0.0007) +[2023-10-09 14:35:24,984][86122] Updated weights for policy 1, policy_version 60460 (0.0009) +[2023-10-09 14:35:25,347][86122] Updated weights for policy 1, policy_version 60470 (0.0008) +[2023-10-09 14:35:25,707][86122] Updated weights for policy 1, policy_version 60480 (0.0007) +[2023-10-09 14:35:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 123600896. Throughput: 0: 1813.0, 1: 1827.0. Samples: 30906998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.940')] +[2023-10-09 14:35:28,449][86121] Updated weights for policy 0, policy_version 60230 (0.0007) +[2023-10-09 14:35:28,828][86121] Updated weights for policy 0, policy_version 60240 (0.0009) +[2023-10-09 14:35:29,199][86121] Updated weights for policy 0, policy_version 60250 (0.0010) +[2023-10-09 14:35:29,331][86122] Updated weights for policy 1, policy_version 60490 (0.0007) +[2023-10-09 14:35:29,700][86122] Updated weights for policy 1, policy_version 60500 (0.0010) +[2023-10-09 14:35:30,065][86122] Updated weights for policy 1, policy_version 60510 (0.0007) +[2023-10-09 14:35:33,011][86121] Updated weights for policy 0, policy_version 60260 (0.0010) +[2023-10-09 14:35:33,377][86121] Updated weights for policy 0, policy_version 60270 (0.0009) +[2023-10-09 14:35:33,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123666432. Throughput: 0: 1810.1, 1: 1834.0. Samples: 30929620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.940')] +[2023-10-09 14:35:33,729][86122] Updated weights for policy 1, policy_version 60520 (0.0008) +[2023-10-09 14:35:33,739][86121] Updated weights for policy 0, policy_version 60280 (0.0007) +[2023-10-09 14:35:34,083][86122] Updated weights for policy 1, policy_version 60530 (0.0009) +[2023-10-09 14:35:34,453][86122] Updated weights for policy 1, policy_version 60540 (0.0010) +[2023-10-09 14:35:37,397][86121] Updated weights for policy 0, policy_version 60290 (0.0007) +[2023-10-09 14:35:37,770][86121] Updated weights for policy 0, policy_version 60300 (0.0012) +[2023-10-09 14:35:37,982][86122] Updated weights for policy 1, policy_version 60550 (0.0008) +[2023-10-09 14:35:38,137][86121] Updated weights for policy 0, policy_version 60310 (0.0007) +[2023-10-09 14:35:38,340][86122] Updated weights for policy 1, policy_version 60560 (0.0009) +[2023-10-09 14:35:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 123731968. Throughput: 0: 1810.8, 1: 1835.9. Samples: 30951666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.940')] +[2023-10-09 14:35:38,502][86121] Updated weights for policy 0, policy_version 60320 (0.0008) +[2023-10-09 14:35:38,706][86122] Updated weights for policy 1, policy_version 60570 (0.0008) +[2023-10-09 14:35:42,195][86121] Updated weights for policy 0, policy_version 60330 (0.0008) +[2023-10-09 14:35:42,484][86122] Updated weights for policy 1, policy_version 60580 (0.0010) +[2023-10-09 14:35:42,559][86121] Updated weights for policy 0, policy_version 60340 (0.0007) +[2023-10-09 14:35:42,845][86122] Updated weights for policy 1, policy_version 60590 (0.0007) +[2023-10-09 14:35:42,925][86121] Updated weights for policy 0, policy_version 60350 (0.0007) +[2023-10-09 14:35:43,210][86122] Updated weights for policy 1, policy_version 60600 (0.0007) +[2023-10-09 14:35:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 123830272. Throughput: 0: 1803.7, 1: 1837.0. Samples: 30962410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:35:46,627][86121] Updated weights for policy 0, policy_version 60360 (0.0009) +[2023-10-09 14:35:46,945][86122] Updated weights for policy 1, policy_version 60610 (0.0008) +[2023-10-09 14:35:46,994][86121] Updated weights for policy 0, policy_version 60370 (0.0007) +[2023-10-09 14:35:47,304][86122] Updated weights for policy 1, policy_version 60620 (0.0008) +[2023-10-09 14:35:47,363][86121] Updated weights for policy 0, policy_version 60380 (0.0007) +[2023-10-09 14:35:47,669][86122] Updated weights for policy 1, policy_version 60630 (0.0008) +[2023-10-09 14:35:48,027][86122] Updated weights for policy 1, policy_version 60640 (0.0008) +[2023-10-09 14:35:48,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 123928576. Throughput: 0: 1813.1, 1: 1831.2. Samples: 30984452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:35:51,107][86121] Updated weights for policy 0, policy_version 60390 (0.0008) +[2023-10-09 14:35:51,463][86121] Updated weights for policy 0, policy_version 60400 (0.0008) +[2023-10-09 14:35:51,735][86122] Updated weights for policy 1, policy_version 60650 (0.0009) +[2023-10-09 14:35:51,827][86121] Updated weights for policy 0, policy_version 60410 (0.0008) +[2023-10-09 14:35:52,098][86122] Updated weights for policy 1, policy_version 60660 (0.0009) +[2023-10-09 14:35:52,456][86122] Updated weights for policy 1, policy_version 60670 (0.0008) +[2023-10-09 14:35:53,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 123994112. Throughput: 0: 1800.7, 1: 1820.7. Samples: 31004892. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 14:35:55,472][86121] Updated weights for policy 0, policy_version 60420 (0.0009) +[2023-10-09 14:35:55,851][86121] Updated weights for policy 0, policy_version 60430 (0.0010) +[2023-10-09 14:35:56,194][86122] Updated weights for policy 1, policy_version 60680 (0.0008) +[2023-10-09 14:35:56,218][86121] Updated weights for policy 0, policy_version 60440 (0.0007) +[2023-10-09 14:35:56,558][86122] Updated weights for policy 1, policy_version 60690 (0.0010) +[2023-10-09 14:35:56,920][86122] Updated weights for policy 1, policy_version 60700 (0.0007) +[2023-10-09 14:35:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 124059648. Throughput: 0: 1813.3, 1: 1822.8. Samples: 31017078. Policy #0 lag: (min: 31.0, avg: 31.0, max: 36.0) +[2023-10-09 14:35:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:35:59,946][86121] Updated weights for policy 0, policy_version 60450 (0.0007) +[2023-10-09 14:36:00,311][86121] Updated weights for policy 0, policy_version 60460 (0.0008) +[2023-10-09 14:36:00,403][86122] Updated weights for policy 1, policy_version 60710 (0.0008) +[2023-10-09 14:36:00,675][86121] Updated weights for policy 0, policy_version 60470 (0.0009) +[2023-10-09 14:36:00,765][86122] Updated weights for policy 1, policy_version 60720 (0.0008) +[2023-10-09 14:36:01,035][86121] Updated weights for policy 0, policy_version 60480 (0.0008) +[2023-10-09 14:36:01,128][86122] Updated weights for policy 1, policy_version 60730 (0.0007) +[2023-10-09 14:36:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 124125184. Throughput: 0: 1798.4, 1: 1825.5. Samples: 31037600. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:36:04,637][86121] Updated weights for policy 0, policy_version 60490 (0.0007) +[2023-10-09 14:36:04,957][86122] Updated weights for policy 1, policy_version 60740 (0.0009) +[2023-10-09 14:36:04,989][86121] Updated weights for policy 0, policy_version 60500 (0.0007) +[2023-10-09 14:36:05,312][86122] Updated weights for policy 1, policy_version 60750 (0.0009) +[2023-10-09 14:36:05,356][86121] Updated weights for policy 0, policy_version 60510 (0.0007) +[2023-10-09 14:36:05,673][86122] Updated weights for policy 1, policy_version 60760 (0.0008) +[2023-10-09 14:36:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124190720. Throughput: 0: 1810.4, 1: 1827.2. Samples: 31060854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 14:36:09,083][86121] Updated weights for policy 0, policy_version 60520 (0.0008) +[2023-10-09 14:36:09,433][86122] Updated weights for policy 1, policy_version 60770 (0.0008) +[2023-10-09 14:36:09,458][86121] Updated weights for policy 0, policy_version 60530 (0.0008) +[2023-10-09 14:36:09,795][86122] Updated weights for policy 1, policy_version 60780 (0.0009) +[2023-10-09 14:36:09,832][86121] Updated weights for policy 0, policy_version 60540 (0.0007) +[2023-10-09 14:36:10,152][86122] Updated weights for policy 1, policy_version 60790 (0.0007) +[2023-10-09 14:36:10,507][86122] Updated weights for policy 1, policy_version 60800 (0.0010) +[2023-10-09 14:36:13,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124256256. Throughput: 0: 1811.7, 1: 1821.4. Samples: 31070486. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 14:36:13,568][86121] Updated weights for policy 0, policy_version 60550 (0.0007) +[2023-10-09 14:36:13,932][86121] Updated weights for policy 0, policy_version 60560 (0.0007) +[2023-10-09 14:36:14,124][86122] Updated weights for policy 1, policy_version 60810 (0.0008) +[2023-10-09 14:36:14,285][86121] Updated weights for policy 0, policy_version 60570 (0.0007) +[2023-10-09 14:36:14,481][86122] Updated weights for policy 1, policy_version 60820 (0.0007) +[2023-10-09 14:36:14,844][86122] Updated weights for policy 1, policy_version 60830 (0.0008) +[2023-10-09 14:36:17,975][86121] Updated weights for policy 0, policy_version 60580 (0.0008) +[2023-10-09 14:36:18,336][86121] Updated weights for policy 0, policy_version 60590 (0.0008) +[2023-10-09 14:36:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124321792. Throughput: 0: 1817.0, 1: 1828.2. Samples: 31093652. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:36:18,588][86122] Updated weights for policy 1, policy_version 60840 (0.0007) +[2023-10-09 14:36:18,708][86121] Updated weights for policy 0, policy_version 60600 (0.0009) +[2023-10-09 14:36:18,949][86122] Updated weights for policy 1, policy_version 60850 (0.0008) +[2023-10-09 14:36:19,316][86122] Updated weights for policy 1, policy_version 60860 (0.0009) +[2023-10-09 14:36:22,385][86121] Updated weights for policy 0, policy_version 60610 (0.0007) +[2023-10-09 14:36:22,758][86121] Updated weights for policy 0, policy_version 60620 (0.0007) +[2023-10-09 14:36:23,119][86121] Updated weights for policy 0, policy_version 60630 (0.0007) +[2023-10-09 14:36:23,320][86122] Updated weights for policy 1, policy_version 60870 (0.0007) +[2023-10-09 14:36:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124387328. Throughput: 0: 1827.6, 1: 1813.5. Samples: 31115518. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:36:23,488][86121] Updated weights for policy 0, policy_version 60640 (0.0008) +[2023-10-09 14:36:23,675][86122] Updated weights for policy 1, policy_version 60880 (0.0007) +[2023-10-09 14:36:24,049][86122] Updated weights for policy 1, policy_version 60890 (0.0008) +[2023-10-09 14:36:27,115][86121] Updated weights for policy 0, policy_version 60650 (0.0010) +[2023-10-09 14:36:27,482][86121] Updated weights for policy 0, policy_version 60660 (0.0012) +[2023-10-09 14:36:27,841][86121] Updated weights for policy 0, policy_version 60670 (0.0007) +[2023-10-09 14:36:27,856][86122] Updated weights for policy 1, policy_version 60900 (0.0009) +[2023-10-09 14:36:28,245][86122] Updated weights for policy 1, policy_version 60910 (0.0007) +[2023-10-09 14:36:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124485632. Throughput: 0: 1825.5, 1: 1812.6. Samples: 31126122. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:36:28,607][86122] Updated weights for policy 1, policy_version 60920 (0.0007) +[2023-10-09 14:36:31,582][86121] Updated weights for policy 0, policy_version 60680 (0.0007) +[2023-10-09 14:36:31,951][86121] Updated weights for policy 0, policy_version 60690 (0.0010) +[2023-10-09 14:36:32,314][86121] Updated weights for policy 0, policy_version 60700 (0.0010) +[2023-10-09 14:36:32,336][86122] Updated weights for policy 1, policy_version 60930 (0.0008) +[2023-10-09 14:36:32,696][86122] Updated weights for policy 1, policy_version 60940 (0.0008) +[2023-10-09 14:36:33,072][86122] Updated weights for policy 1, policy_version 60950 (0.0008) +[2023-10-09 14:36:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 124551168. Throughput: 0: 1826.4, 1: 1807.4. Samples: 31147972. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:36:33,430][86122] Updated weights for policy 1, policy_version 60960 (0.0008) +[2023-10-09 14:36:35,916][86121] Updated weights for policy 0, policy_version 60710 (0.0010) +[2023-10-09 14:36:36,283][86121] Updated weights for policy 0, policy_version 60720 (0.0009) +[2023-10-09 14:36:36,654][86121] Updated weights for policy 0, policy_version 60730 (0.0008) +[2023-10-09 14:36:37,138][86122] Updated weights for policy 1, policy_version 60970 (0.0007) +[2023-10-09 14:36:37,501][86122] Updated weights for policy 1, policy_version 60980 (0.0008) +[2023-10-09 14:36:37,856][86122] Updated weights for policy 1, policy_version 60990 (0.0008) +[2023-10-09 14:36:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 124649472. Throughput: 0: 1828.3, 1: 1817.0. Samples: 31168928. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:36:40,235][86121] Updated weights for policy 0, policy_version 60740 (0.0007) +[2023-10-09 14:36:40,595][86121] Updated weights for policy 0, policy_version 60750 (0.0008) +[2023-10-09 14:36:40,960][86121] Updated weights for policy 0, policy_version 60760 (0.0009) +[2023-10-09 14:36:41,562][86122] Updated weights for policy 1, policy_version 61000 (0.0008) +[2023-10-09 14:36:41,921][86122] Updated weights for policy 1, policy_version 61010 (0.0009) +[2023-10-09 14:36:42,287][86122] Updated weights for policy 1, policy_version 61020 (0.0007) +[2023-10-09 14:36:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 124715008. Throughput: 0: 1827.6, 1: 1813.3. Samples: 31180920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 14:36:44,568][86121] Updated weights for policy 0, policy_version 60770 (0.0009) +[2023-10-09 14:36:44,928][86121] Updated weights for policy 0, policy_version 60780 (0.0007) +[2023-10-09 14:36:45,288][86121] Updated weights for policy 0, policy_version 60790 (0.0010) +[2023-10-09 14:36:45,647][86121] Updated weights for policy 0, policy_version 60800 (0.0007) +[2023-10-09 14:36:45,960][86122] Updated weights for policy 1, policy_version 61030 (0.0008) +[2023-10-09 14:36:46,324][86122] Updated weights for policy 1, policy_version 61040 (0.0008) +[2023-10-09 14:36:46,676][86122] Updated weights for policy 1, policy_version 61050 (0.0008) +[2023-10-09 14:36:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124780544. Throughput: 0: 1840.1, 1: 1815.1. Samples: 31202084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 14:36:49,343][86121] Updated weights for policy 0, policy_version 60810 (0.0008) +[2023-10-09 14:36:49,707][86121] Updated weights for policy 0, policy_version 60820 (0.0010) +[2023-10-09 14:36:50,078][86121] Updated weights for policy 0, policy_version 60830 (0.0009) +[2023-10-09 14:36:50,380][86122] Updated weights for policy 1, policy_version 61060 (0.0009) +[2023-10-09 14:36:50,738][86122] Updated weights for policy 1, policy_version 61070 (0.0009) +[2023-10-09 14:36:51,109][86122] Updated weights for policy 1, policy_version 61080 (0.0010) +[2023-10-09 14:36:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 124846080. Throughput: 0: 1831.2, 1: 1807.7. Samples: 31224604. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 14:36:53,838][86121] Updated weights for policy 0, policy_version 60840 (0.0010) +[2023-10-09 14:36:54,210][86121] Updated weights for policy 0, policy_version 60850 (0.0010) +[2023-10-09 14:36:54,578][86121] Updated weights for policy 0, policy_version 60860 (0.0007) +[2023-10-09 14:36:54,848][86122] Updated weights for policy 1, policy_version 61090 (0.0010) +[2023-10-09 14:36:55,214][86122] Updated weights for policy 1, policy_version 61100 (0.0009) +[2023-10-09 14:36:55,568][86122] Updated weights for policy 1, policy_version 61110 (0.0009) +[2023-10-09 14:36:55,929][86122] Updated weights for policy 1, policy_version 61120 (0.0008) +[2023-10-09 14:36:58,367][86121] Updated weights for policy 0, policy_version 60870 (0.0007) +[2023-10-09 14:36:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124911616. Throughput: 0: 1830.7, 1: 1818.7. Samples: 31234706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:36:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:36:58,742][86121] Updated weights for policy 0, policy_version 60880 (0.0007) +[2023-10-09 14:36:59,116][86121] Updated weights for policy 0, policy_version 60890 (0.0007) +[2023-10-09 14:36:59,713][86122] Updated weights for policy 1, policy_version 61130 (0.0010) +[2023-10-09 14:37:00,073][86122] Updated weights for policy 1, policy_version 61140 (0.0010) +[2023-10-09 14:37:00,430][86122] Updated weights for policy 1, policy_version 61150 (0.0009) +[2023-10-09 14:37:02,887][86121] Updated weights for policy 0, policy_version 60900 (0.0008) +[2023-10-09 14:37:03,249][86121] Updated weights for policy 0, policy_version 60910 (0.0009) +[2023-10-09 14:37:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 124977152. Throughput: 0: 1827.7, 1: 1806.3. Samples: 31257180. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:37:03,610][86121] Updated weights for policy 0, policy_version 60920 (0.0008) +[2023-10-09 14:37:03,959][86122] Updated weights for policy 1, policy_version 61160 (0.0008) +[2023-10-09 14:37:04,327][86122] Updated weights for policy 1, policy_version 61170 (0.0008) +[2023-10-09 14:37:04,681][86122] Updated weights for policy 1, policy_version 61180 (0.0010) +[2023-10-09 14:37:07,264][86121] Updated weights for policy 0, policy_version 60930 (0.0009) +[2023-10-09 14:37:07,617][86121] Updated weights for policy 0, policy_version 60940 (0.0010) +[2023-10-09 14:37:07,986][86121] Updated weights for policy 0, policy_version 60950 (0.0010) +[2023-10-09 14:37:08,262][86122] Updated weights for policy 1, policy_version 61190 (0.0009) +[2023-10-09 14:37:08,356][86121] Updated weights for policy 0, policy_version 60960 (0.0008) +[2023-10-09 14:37:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125075456. Throughput: 0: 1820.9, 1: 1824.7. Samples: 31279570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:37:08,622][86122] Updated weights for policy 1, policy_version 61200 (0.0008) +[2023-10-09 14:37:08,982][86122] Updated weights for policy 1, policy_version 61210 (0.0007) +[2023-10-09 14:37:12,032][86121] Updated weights for policy 0, policy_version 60970 (0.0008) +[2023-10-09 14:37:12,398][86121] Updated weights for policy 0, policy_version 60980 (0.0008) +[2023-10-09 14:37:12,745][86122] Updated weights for policy 1, policy_version 61220 (0.0008) +[2023-10-09 14:37:12,766][86121] Updated weights for policy 0, policy_version 60990 (0.0007) +[2023-10-09 14:37:13,107][86122] Updated weights for policy 1, policy_version 61230 (0.0008) +[2023-10-09 14:37:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125140992. Throughput: 0: 1825.9, 1: 1824.6. Samples: 31290392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:37:13,471][86122] Updated weights for policy 1, policy_version 61240 (0.0012) +[2023-10-09 14:37:16,523][86121] Updated weights for policy 0, policy_version 61000 (0.0008) +[2023-10-09 14:37:16,891][86121] Updated weights for policy 0, policy_version 61010 (0.0007) +[2023-10-09 14:37:17,124][86122] Updated weights for policy 1, policy_version 61250 (0.0009) +[2023-10-09 14:37:17,250][86121] Updated weights for policy 0, policy_version 61020 (0.0008) +[2023-10-09 14:37:17,484][86122] Updated weights for policy 1, policy_version 61260 (0.0008) +[2023-10-09 14:37:17,846][86122] Updated weights for policy 1, policy_version 61270 (0.0010) +[2023-10-09 14:37:18,209][86122] Updated weights for policy 1, policy_version 61280 (0.0009) +[2023-10-09 14:37:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125239296. Throughput: 0: 1821.8, 1: 1832.8. Samples: 31312428. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:37:20,937][86121] Updated weights for policy 0, policy_version 61030 (0.0010) +[2023-10-09 14:37:21,302][86121] Updated weights for policy 0, policy_version 61040 (0.0008) +[2023-10-09 14:37:21,667][86121] Updated weights for policy 0, policy_version 61050 (0.0009) +[2023-10-09 14:37:21,763][86122] Updated weights for policy 1, policy_version 61290 (0.0008) +[2023-10-09 14:37:22,128][86122] Updated weights for policy 1, policy_version 61300 (0.0007) +[2023-10-09 14:37:22,480][86122] Updated weights for policy 1, policy_version 61310 (0.0007) +[2023-10-09 14:37:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 125304832. Throughput: 0: 1818.9, 1: 1833.9. Samples: 31333304. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:37:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000061312_62783488.pth... +[2023-10-09 14:37:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000061056_62521344.pth... +[2023-10-09 14:37:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000059584_61014016.pth +[2023-10-09 14:37:23,450][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000059360_60784640.pth +[2023-10-09 14:37:25,500][86121] Updated weights for policy 0, policy_version 61060 (0.0009) +[2023-10-09 14:37:25,874][86121] Updated weights for policy 0, policy_version 61070 (0.0007) +[2023-10-09 14:37:26,116][86122] Updated weights for policy 1, policy_version 61320 (0.0007) +[2023-10-09 14:37:26,237][86121] Updated weights for policy 0, policy_version 61080 (0.0007) +[2023-10-09 14:37:26,475][86122] Updated weights for policy 1, policy_version 61330 (0.0008) +[2023-10-09 14:37:26,852][86122] Updated weights for policy 1, policy_version 61340 (0.0011) +[2023-10-09 14:37:28,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125370368. Throughput: 0: 1818.7, 1: 1842.3. Samples: 31345662. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:37:29,987][86121] Updated weights for policy 0, policy_version 61090 (0.0007) +[2023-10-09 14:37:30,353][86121] Updated weights for policy 0, policy_version 61100 (0.0007) +[2023-10-09 14:37:30,621][86122] Updated weights for policy 1, policy_version 61350 (0.0008) +[2023-10-09 14:37:30,718][86121] Updated weights for policy 0, policy_version 61110 (0.0008) +[2023-10-09 14:37:30,987][86122] Updated weights for policy 1, policy_version 61360 (0.0008) +[2023-10-09 14:37:31,088][86121] Updated weights for policy 0, policy_version 61120 (0.0007) +[2023-10-09 14:37:31,363][86122] Updated weights for policy 1, policy_version 61370 (0.0008) +[2023-10-09 14:37:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125435904. Throughput: 0: 1804.9, 1: 1836.0. Samples: 31365926. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:37:34,686][86121] Updated weights for policy 0, policy_version 61130 (0.0010) +[2023-10-09 14:37:34,936][86122] Updated weights for policy 1, policy_version 61380 (0.0009) +[2023-10-09 14:37:35,058][86121] Updated weights for policy 0, policy_version 61140 (0.0008) +[2023-10-09 14:37:35,293][86122] Updated weights for policy 1, policy_version 61390 (0.0008) +[2023-10-09 14:37:35,416][86121] Updated weights for policy 0, policy_version 61150 (0.0010) +[2023-10-09 14:37:35,657][86122] Updated weights for policy 1, policy_version 61400 (0.0009) +[2023-10-09 14:37:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 125501440. Throughput: 0: 1809.4, 1: 1843.9. Samples: 31389000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:37:39,132][86121] Updated weights for policy 0, policy_version 61160 (0.0010) +[2023-10-09 14:37:39,337][86122] Updated weights for policy 1, policy_version 61410 (0.0009) +[2023-10-09 14:37:39,500][86121] Updated weights for policy 0, policy_version 61170 (0.0007) +[2023-10-09 14:37:39,693][86122] Updated weights for policy 1, policy_version 61420 (0.0009) +[2023-10-09 14:37:39,857][86121] Updated weights for policy 0, policy_version 61180 (0.0007) +[2023-10-09 14:37:40,059][86122] Updated weights for policy 1, policy_version 61430 (0.0008) +[2023-10-09 14:37:40,417][86122] Updated weights for policy 1, policy_version 61440 (0.0011) +[2023-10-09 14:37:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125566976. Throughput: 0: 1811.1, 1: 1838.1. Samples: 31398920. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:37:43,583][86121] Updated weights for policy 0, policy_version 61190 (0.0008) +[2023-10-09 14:37:43,941][86121] Updated weights for policy 0, policy_version 61200 (0.0008) +[2023-10-09 14:37:44,027][86122] Updated weights for policy 1, policy_version 61450 (0.0009) +[2023-10-09 14:37:44,305][86121] Updated weights for policy 0, policy_version 61210 (0.0007) +[2023-10-09 14:37:44,393][86122] Updated weights for policy 1, policy_version 61460 (0.0008) +[2023-10-09 14:37:44,753][86122] Updated weights for policy 1, policy_version 61470 (0.0009) +[2023-10-09 14:37:48,048][86121] Updated weights for policy 0, policy_version 61220 (0.0010) +[2023-10-09 14:37:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 125632512. Throughput: 0: 1811.8, 1: 1845.4. Samples: 31421756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:37:48,408][86121] Updated weights for policy 0, policy_version 61230 (0.0008) +[2023-10-09 14:37:48,718][86122] Updated weights for policy 1, policy_version 61480 (0.0008) +[2023-10-09 14:37:48,772][86121] Updated weights for policy 0, policy_version 61240 (0.0008) +[2023-10-09 14:37:49,080][86122] Updated weights for policy 1, policy_version 61490 (0.0008) +[2023-10-09 14:37:49,443][86122] Updated weights for policy 1, policy_version 61500 (0.0007) +[2023-10-09 14:37:52,427][86121] Updated weights for policy 0, policy_version 61250 (0.0008) +[2023-10-09 14:37:52,798][86121] Updated weights for policy 0, policy_version 61260 (0.0008) +[2023-10-09 14:37:53,127][86122] Updated weights for policy 1, policy_version 61510 (0.0008) +[2023-10-09 14:37:53,175][86121] Updated weights for policy 0, policy_version 61270 (0.0007) +[2023-10-09 14:37:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 125698048. Throughput: 0: 1815.8, 1: 1830.0. Samples: 31443632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:37:53,489][86122] Updated weights for policy 1, policy_version 61520 (0.0007) +[2023-10-09 14:37:53,531][86121] Updated weights for policy 0, policy_version 61280 (0.0008) +[2023-10-09 14:37:53,846][86122] Updated weights for policy 1, policy_version 61530 (0.0008) +[2023-10-09 14:37:57,265][86121] Updated weights for policy 0, policy_version 61290 (0.0008) +[2023-10-09 14:37:57,459][86122] Updated weights for policy 1, policy_version 61540 (0.0010) +[2023-10-09 14:37:57,631][86121] Updated weights for policy 0, policy_version 61300 (0.0008) +[2023-10-09 14:37:57,844][86122] Updated weights for policy 1, policy_version 61550 (0.0008) +[2023-10-09 14:37:57,997][86121] Updated weights for policy 0, policy_version 61310 (0.0008) +[2023-10-09 14:37:58,199][86122] Updated weights for policy 1, policy_version 61560 (0.0008) +[2023-10-09 14:37:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 125796352. Throughput: 0: 1807.0, 1: 1828.6. Samples: 31453994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:37:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.980')] +[2023-10-09 14:38:01,756][86121] Updated weights for policy 0, policy_version 61320 (0.0009) +[2023-10-09 14:38:01,820][86122] Updated weights for policy 1, policy_version 61570 (0.0008) +[2023-10-09 14:38:02,123][86121] Updated weights for policy 0, policy_version 61330 (0.0009) +[2023-10-09 14:38:02,183][86122] Updated weights for policy 1, policy_version 61580 (0.0008) +[2023-10-09 14:38:02,493][86121] Updated weights for policy 0, policy_version 61340 (0.0008) +[2023-10-09 14:38:02,544][86122] Updated weights for policy 1, policy_version 61590 (0.0007) +[2023-10-09 14:38:02,903][86122] Updated weights for policy 1, policy_version 61600 (0.0007) +[2023-10-09 14:38:03,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 125894656. Throughput: 0: 1814.0, 1: 1823.0. Samples: 31476094. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 14:38:06,141][86121] Updated weights for policy 0, policy_version 61350 (0.0009) +[2023-10-09 14:38:06,506][86121] Updated weights for policy 0, policy_version 61360 (0.0008) +[2023-10-09 14:38:06,583][86122] Updated weights for policy 1, policy_version 61610 (0.0008) +[2023-10-09 14:38:06,876][86121] Updated weights for policy 0, policy_version 61370 (0.0007) +[2023-10-09 14:38:06,942][86122] Updated weights for policy 1, policy_version 61620 (0.0010) +[2023-10-09 14:38:07,311][86122] Updated weights for policy 1, policy_version 61630 (0.0008) +[2023-10-09 14:38:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 125960192. Throughput: 0: 1810.8, 1: 1821.6. Samples: 31496760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 14:38:10,451][86121] Updated weights for policy 0, policy_version 61380 (0.0009) +[2023-10-09 14:38:10,821][86121] Updated weights for policy 0, policy_version 61390 (0.0008) +[2023-10-09 14:38:11,050][86122] Updated weights for policy 1, policy_version 61640 (0.0008) +[2023-10-09 14:38:11,185][86121] Updated weights for policy 0, policy_version 61400 (0.0008) +[2023-10-09 14:38:11,412][86122] Updated weights for policy 1, policy_version 61650 (0.0008) +[2023-10-09 14:38:11,779][86122] Updated weights for policy 1, policy_version 61660 (0.0009) +[2023-10-09 14:38:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126025728. Throughput: 0: 1814.2, 1: 1809.3. Samples: 31508718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 14:38:14,891][86121] Updated weights for policy 0, policy_version 61410 (0.0009) +[2023-10-09 14:38:15,258][86121] Updated weights for policy 0, policy_version 61420 (0.0011) +[2023-10-09 14:38:15,445][86122] Updated weights for policy 1, policy_version 61670 (0.0009) +[2023-10-09 14:38:15,620][86121] Updated weights for policy 0, policy_version 61430 (0.0010) +[2023-10-09 14:38:15,811][86122] Updated weights for policy 1, policy_version 61680 (0.0009) +[2023-10-09 14:38:15,988][86121] Updated weights for policy 0, policy_version 61440 (0.0009) +[2023-10-09 14:38:16,173][86122] Updated weights for policy 1, policy_version 61690 (0.0008) +[2023-10-09 14:38:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 126091264. Throughput: 0: 1824.6, 1: 1812.9. Samples: 31529612. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:38:19,877][86121] Updated weights for policy 0, policy_version 61450 (0.0007) +[2023-10-09 14:38:20,056][86122] Updated weights for policy 1, policy_version 61700 (0.0008) +[2023-10-09 14:38:20,254][86121] Updated weights for policy 0, policy_version 61460 (0.0008) +[2023-10-09 14:38:20,407][86122] Updated weights for policy 1, policy_version 61710 (0.0007) +[2023-10-09 14:38:20,607][86121] Updated weights for policy 0, policy_version 61470 (0.0007) +[2023-10-09 14:38:20,776][86122] Updated weights for policy 1, policy_version 61720 (0.0009) +[2023-10-09 14:38:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126156800. Throughput: 0: 1818.0, 1: 1813.2. Samples: 31552404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:38:24,403][86121] Updated weights for policy 0, policy_version 61480 (0.0009) +[2023-10-09 14:38:24,583][86122] Updated weights for policy 1, policy_version 61730 (0.0009) +[2023-10-09 14:38:24,770][86121] Updated weights for policy 0, policy_version 61490 (0.0009) +[2023-10-09 14:38:24,946][86122] Updated weights for policy 1, policy_version 61740 (0.0009) +[2023-10-09 14:38:25,140][86121] Updated weights for policy 0, policy_version 61500 (0.0008) +[2023-10-09 14:38:25,309][86122] Updated weights for policy 1, policy_version 61750 (0.0008) +[2023-10-09 14:38:25,672][86122] Updated weights for policy 1, policy_version 61760 (0.0010) +[2023-10-09 14:38:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126222336. Throughput: 0: 1814.6, 1: 1812.5. Samples: 31562138. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:38:28,846][86121] Updated weights for policy 0, policy_version 61510 (0.0009) +[2023-10-09 14:38:29,219][86121] Updated weights for policy 0, policy_version 61520 (0.0010) +[2023-10-09 14:38:29,238][86122] Updated weights for policy 1, policy_version 61770 (0.0008) +[2023-10-09 14:38:29,580][86121] Updated weights for policy 0, policy_version 61530 (0.0008) +[2023-10-09 14:38:29,598][86122] Updated weights for policy 1, policy_version 61780 (0.0008) +[2023-10-09 14:38:29,968][86122] Updated weights for policy 1, policy_version 61790 (0.0010) +[2023-10-09 14:38:33,301][86121] Updated weights for policy 0, policy_version 61540 (0.0009) +[2023-10-09 14:38:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126287872. Throughput: 0: 1812.4, 1: 1811.3. Samples: 31584826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:38:33,666][86121] Updated weights for policy 0, policy_version 61550 (0.0007) +[2023-10-09 14:38:33,671][86122] Updated weights for policy 1, policy_version 61800 (0.0008) +[2023-10-09 14:38:34,033][86122] Updated weights for policy 1, policy_version 61810 (0.0009) +[2023-10-09 14:38:34,035][86121] Updated weights for policy 0, policy_version 61560 (0.0007) +[2023-10-09 14:38:34,393][86122] Updated weights for policy 1, policy_version 61820 (0.0008) +[2023-10-09 14:38:37,867][86121] Updated weights for policy 0, policy_version 61570 (0.0007) +[2023-10-09 14:38:38,031][86122] Updated weights for policy 1, policy_version 61830 (0.0009) +[2023-10-09 14:38:38,228][86121] Updated weights for policy 0, policy_version 61580 (0.0008) +[2023-10-09 14:38:38,393][86122] Updated weights for policy 1, policy_version 61840 (0.0007) +[2023-10-09 14:38:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126353408. Throughput: 0: 1817.1, 1: 1818.4. Samples: 31607228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:38:38,587][86121] Updated weights for policy 0, policy_version 61590 (0.0007) +[2023-10-09 14:38:38,746][86122] Updated weights for policy 1, policy_version 61850 (0.0007) +[2023-10-09 14:38:38,956][86121] Updated weights for policy 0, policy_version 61600 (0.0007) +[2023-10-09 14:38:42,629][86122] Updated weights for policy 1, policy_version 61860 (0.0010) +[2023-10-09 14:38:42,719][86121] Updated weights for policy 0, policy_version 61610 (0.0008) +[2023-10-09 14:38:43,015][86122] Updated weights for policy 1, policy_version 61870 (0.0007) +[2023-10-09 14:38:43,074][86121] Updated weights for policy 0, policy_version 61620 (0.0008) +[2023-10-09 14:38:43,372][86122] Updated weights for policy 1, policy_version 61880 (0.0007) +[2023-10-09 14:38:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126418944. Throughput: 0: 1806.0, 1: 1819.7. Samples: 31617152. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:38:43,444][86121] Updated weights for policy 0, policy_version 61630 (0.0008) +[2023-10-09 14:38:47,118][86122] Updated weights for policy 1, policy_version 61890 (0.0008) +[2023-10-09 14:38:47,322][86121] Updated weights for policy 0, policy_version 61640 (0.0008) +[2023-10-09 14:38:47,478][86122] Updated weights for policy 1, policy_version 61900 (0.0007) +[2023-10-09 14:38:47,691][86121] Updated weights for policy 0, policy_version 61650 (0.0008) +[2023-10-09 14:38:47,842][86122] Updated weights for policy 1, policy_version 61910 (0.0009) +[2023-10-09 14:38:48,055][86121] Updated weights for policy 0, policy_version 61660 (0.0007) +[2023-10-09 14:38:48,207][86122] Updated weights for policy 1, policy_version 61920 (0.0008) +[2023-10-09 14:38:48,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 126550016. Throughput: 0: 1814.5, 1: 1813.4. Samples: 31639350. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:38:51,806][86121] Updated weights for policy 0, policy_version 61670 (0.0009) +[2023-10-09 14:38:51,954][86122] Updated weights for policy 1, policy_version 61930 (0.0008) +[2023-10-09 14:38:52,167][86121] Updated weights for policy 0, policy_version 61680 (0.0007) +[2023-10-09 14:38:52,311][86122] Updated weights for policy 1, policy_version 61940 (0.0007) +[2023-10-09 14:38:52,546][86121] Updated weights for policy 0, policy_version 61690 (0.0008) +[2023-10-09 14:38:52,670][86122] Updated weights for policy 1, policy_version 61950 (0.0007) +[2023-10-09 14:38:53,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 126615552. Throughput: 0: 1794.7, 1: 1806.3. Samples: 31658804. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:38:56,225][86121] Updated weights for policy 0, policy_version 61700 (0.0008) +[2023-10-09 14:38:56,434][86122] Updated weights for policy 1, policy_version 61960 (0.0008) +[2023-10-09 14:38:56,585][86121] Updated weights for policy 0, policy_version 61710 (0.0009) +[2023-10-09 14:38:56,794][86122] Updated weights for policy 1, policy_version 61970 (0.0009) +[2023-10-09 14:38:56,944][86121] Updated weights for policy 0, policy_version 61720 (0.0007) +[2023-10-09 14:38:57,167][86122] Updated weights for policy 1, policy_version 61980 (0.0007) +[2023-10-09 14:38:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 126681088. Throughput: 0: 1807.4, 1: 1810.0. Samples: 31671502. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:38:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:39:00,690][86121] Updated weights for policy 0, policy_version 61730 (0.0008) +[2023-10-09 14:39:00,921][86122] Updated weights for policy 1, policy_version 61990 (0.0008) +[2023-10-09 14:39:01,055][86121] Updated weights for policy 0, policy_version 61740 (0.0007) +[2023-10-09 14:39:01,282][86122] Updated weights for policy 1, policy_version 62000 (0.0010) +[2023-10-09 14:39:01,421][86121] Updated weights for policy 0, policy_version 61750 (0.0008) +[2023-10-09 14:39:01,646][86122] Updated weights for policy 1, policy_version 62010 (0.0007) +[2023-10-09 14:39:01,786][86121] Updated weights for policy 0, policy_version 61760 (0.0009) +[2023-10-09 14:39:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 126746624. Throughput: 0: 1779.6, 1: 1808.1. Samples: 31691060. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:39:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:05,203][86122] Updated weights for policy 1, policy_version 62020 (0.0007) +[2023-10-09 14:39:05,516][86121] Updated weights for policy 0, policy_version 61770 (0.0009) +[2023-10-09 14:39:05,562][86122] Updated weights for policy 1, policy_version 62030 (0.0008) +[2023-10-09 14:39:05,876][86121] Updated weights for policy 0, policy_version 61780 (0.0007) +[2023-10-09 14:39:05,930][86122] Updated weights for policy 1, policy_version 62040 (0.0007) +[2023-10-09 14:39:06,251][86121] Updated weights for policy 0, policy_version 61790 (0.0007) +[2023-10-09 14:39:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126812160. Throughput: 0: 1780.1, 1: 1804.0. Samples: 31713690. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:39:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:09,698][86122] Updated weights for policy 1, policy_version 62050 (0.0008) +[2023-10-09 14:39:10,057][86122] Updated weights for policy 1, policy_version 62060 (0.0008) +[2023-10-09 14:39:10,070][86121] Updated weights for policy 0, policy_version 61800 (0.0008) +[2023-10-09 14:39:10,427][86122] Updated weights for policy 1, policy_version 62070 (0.0008) +[2023-10-09 14:39:10,446][86121] Updated weights for policy 0, policy_version 61810 (0.0007) +[2023-10-09 14:39:10,780][86122] Updated weights for policy 1, policy_version 62080 (0.0008) +[2023-10-09 14:39:10,811][86121] Updated weights for policy 0, policy_version 61820 (0.0009) +[2023-10-09 14:39:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 126877696. Throughput: 0: 1787.8, 1: 1802.0. Samples: 31723682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:39:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:14,496][86121] Updated weights for policy 0, policy_version 61830 (0.0009) +[2023-10-09 14:39:14,510][86122] Updated weights for policy 1, policy_version 62090 (0.0007) +[2023-10-09 14:39:14,863][86122] Updated weights for policy 1, policy_version 62100 (0.0008) +[2023-10-09 14:39:14,863][86121] Updated weights for policy 0, policy_version 61840 (0.0008) +[2023-10-09 14:39:15,223][86121] Updated weights for policy 0, policy_version 61850 (0.0008) +[2023-10-09 14:39:15,233][86122] Updated weights for policy 1, policy_version 62110 (0.0008) +[2023-10-09 14:39:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 126943232. Throughput: 0: 1790.9, 1: 1796.8. Samples: 31746270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:39:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:18,792][86121] Updated weights for policy 0, policy_version 61860 (0.0008) +[2023-10-09 14:39:19,023][86122] Updated weights for policy 1, policy_version 62120 (0.0009) +[2023-10-09 14:39:19,155][86121] Updated weights for policy 0, policy_version 61870 (0.0008) +[2023-10-09 14:39:19,386][86122] Updated weights for policy 1, policy_version 62130 (0.0008) +[2023-10-09 14:39:19,525][86121] Updated weights for policy 0, policy_version 61880 (0.0007) +[2023-10-09 14:39:19,743][86122] Updated weights for policy 1, policy_version 62140 (0.0007) +[2023-10-09 14:39:23,045][86121] Updated weights for policy 0, policy_version 61890 (0.0007) +[2023-10-09 14:39:23,359][86122] Updated weights for policy 1, policy_version 62150 (0.0010) +[2023-10-09 14:39:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 127008768. Throughput: 0: 1805.7, 1: 1797.2. Samples: 31769360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:39:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:23,407][86121] Updated weights for policy 0, policy_version 61900 (0.0008) +[2023-10-09 14:39:23,720][86122] Updated weights for policy 1, policy_version 62160 (0.0008) +[2023-10-09 14:39:23,775][86121] Updated weights for policy 0, policy_version 61910 (0.0008) +[2023-10-09 14:39:24,075][86122] Updated weights for policy 1, policy_version 62170 (0.0009) +[2023-10-09 14:39:24,136][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000061920_63406080.pth... +[2023-10-09 14:39:24,139][86121] Updated weights for policy 0, policy_version 61920 (0.0008) +[2023-10-09 14:39:24,165][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000060192_61636608.pth +[2023-10-09 14:39:24,296][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000062176_63668224.pth... +[2023-10-09 14:39:24,333][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000060448_61898752.pth +[2023-10-09 14:39:27,883][86121] Updated weights for policy 0, policy_version 61930 (0.0007) +[2023-10-09 14:39:27,922][86122] Updated weights for policy 1, policy_version 62180 (0.0009) +[2023-10-09 14:39:28,250][86121] Updated weights for policy 0, policy_version 61940 (0.0009) +[2023-10-09 14:39:28,323][86122] Updated weights for policy 1, policy_version 62190 (0.0008) +[2023-10-09 14:39:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 127074304. Throughput: 0: 1807.2, 1: 1797.6. Samples: 31779364. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:39:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:28,619][86121] Updated weights for policy 0, policy_version 61950 (0.0009) +[2023-10-09 14:39:28,682][86122] Updated weights for policy 1, policy_version 62200 (0.0008) +[2023-10-09 14:39:32,402][86121] Updated weights for policy 0, policy_version 61960 (0.0007) +[2023-10-09 14:39:32,479][86122] Updated weights for policy 1, policy_version 62210 (0.0008) +[2023-10-09 14:39:32,768][86121] Updated weights for policy 0, policy_version 61970 (0.0008) +[2023-10-09 14:39:32,840][86122] Updated weights for policy 1, policy_version 62220 (0.0007) +[2023-10-09 14:39:33,141][86121] Updated weights for policy 0, policy_version 61980 (0.0007) +[2023-10-09 14:39:33,198][86122] Updated weights for policy 1, policy_version 62230 (0.0008) +[2023-10-09 14:39:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 127172608. Throughput: 0: 1815.0, 1: 1797.5. Samples: 31801912. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:39:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:33,561][86122] Updated weights for policy 1, policy_version 62240 (0.0010) +[2023-10-09 14:39:36,781][86121] Updated weights for policy 0, policy_version 61990 (0.0007) +[2023-10-09 14:39:37,154][86121] Updated weights for policy 0, policy_version 62000 (0.0008) +[2023-10-09 14:39:37,296][86122] Updated weights for policy 1, policy_version 62250 (0.0007) +[2023-10-09 14:39:37,525][86121] Updated weights for policy 0, policy_version 62010 (0.0009) +[2023-10-09 14:39:37,655][86122] Updated weights for policy 1, policy_version 62260 (0.0008) +[2023-10-09 14:39:38,016][86122] Updated weights for policy 1, policy_version 62270 (0.0008) +[2023-10-09 14:39:38,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127270912. Throughput: 0: 1823.6, 1: 1810.9. Samples: 31822358. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:39:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:41,142][86121] Updated weights for policy 0, policy_version 62020 (0.0009) +[2023-10-09 14:39:41,508][86121] Updated weights for policy 0, policy_version 62030 (0.0007) +[2023-10-09 14:39:41,714][86122] Updated weights for policy 1, policy_version 62280 (0.0009) +[2023-10-09 14:39:41,875][86121] Updated weights for policy 0, policy_version 62040 (0.0008) +[2023-10-09 14:39:42,071][86122] Updated weights for policy 1, policy_version 62290 (0.0009) +[2023-10-09 14:39:42,432][86122] Updated weights for policy 1, policy_version 62300 (0.0008) +[2023-10-09 14:39:43,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127336448. Throughput: 0: 1826.7, 1: 1808.1. Samples: 31835072. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:39:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:45,582][86121] Updated weights for policy 0, policy_version 62050 (0.0007) +[2023-10-09 14:39:45,944][86121] Updated weights for policy 0, policy_version 62060 (0.0008) +[2023-10-09 14:39:46,245][86122] Updated weights for policy 1, policy_version 62310 (0.0008) +[2023-10-09 14:39:46,308][86121] Updated weights for policy 0, policy_version 62070 (0.0008) +[2023-10-09 14:39:46,616][86122] Updated weights for policy 1, policy_version 62320 (0.0008) +[2023-10-09 14:39:46,667][86121] Updated weights for policy 0, policy_version 62080 (0.0007) +[2023-10-09 14:39:46,968][86122] Updated weights for policy 1, policy_version 62330 (0.0008) +[2023-10-09 14:39:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 127401984. Throughput: 0: 1834.1, 1: 1813.0. Samples: 31855182. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:39:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:39:50,261][86121] Updated weights for policy 0, policy_version 62090 (0.0008) +[2023-10-09 14:39:50,632][86121] Updated weights for policy 0, policy_version 62100 (0.0010) +[2023-10-09 14:39:50,694][86122] Updated weights for policy 1, policy_version 62340 (0.0009) +[2023-10-09 14:39:50,997][86121] Updated weights for policy 0, policy_version 62110 (0.0009) +[2023-10-09 14:39:51,055][86122] Updated weights for policy 1, policy_version 62350 (0.0007) +[2023-10-09 14:39:51,417][86122] Updated weights for policy 1, policy_version 62360 (0.0007) +[2023-10-09 14:39:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 127467520. Throughput: 0: 1837.6, 1: 1797.9. Samples: 31877288. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:39:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:39:54,769][86121] Updated weights for policy 0, policy_version 62120 (0.0009) +[2023-10-09 14:39:55,139][86121] Updated weights for policy 0, policy_version 62130 (0.0007) +[2023-10-09 14:39:55,202][86122] Updated weights for policy 1, policy_version 62370 (0.0009) +[2023-10-09 14:39:55,498][86121] Updated weights for policy 0, policy_version 62140 (0.0009) +[2023-10-09 14:39:55,568][86122] Updated weights for policy 1, policy_version 62380 (0.0008) +[2023-10-09 14:39:55,929][86122] Updated weights for policy 1, policy_version 62390 (0.0007) +[2023-10-09 14:39:56,292][86122] Updated weights for policy 1, policy_version 62400 (0.0007) +[2023-10-09 14:39:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 127533056. Throughput: 0: 1833.0, 1: 1813.4. Samples: 31887772. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:39:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:39:59,048][86121] Updated weights for policy 0, policy_version 62150 (0.0009) +[2023-10-09 14:39:59,417][86121] Updated weights for policy 0, policy_version 62160 (0.0008) +[2023-10-09 14:39:59,780][86121] Updated weights for policy 0, policy_version 62170 (0.0007) +[2023-10-09 14:39:59,974][86122] Updated weights for policy 1, policy_version 62410 (0.0007) +[2023-10-09 14:40:00,332][86122] Updated weights for policy 1, policy_version 62420 (0.0008) +[2023-10-09 14:40:00,699][86122] Updated weights for policy 1, policy_version 62430 (0.0007) +[2023-10-09 14:40:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 127598592. Throughput: 0: 1839.5, 1: 1804.6. Samples: 31910254. Policy #0 lag: (min: 17.0, avg: 33.8, max: 49.0) +[2023-10-09 14:40:03,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:40:03,407][86121] Updated weights for policy 0, policy_version 62180 (0.0008) +[2023-10-09 14:40:03,773][86121] Updated weights for policy 0, policy_version 62190 (0.0008) +[2023-10-09 14:40:04,149][86121] Updated weights for policy 0, policy_version 62200 (0.0007) +[2023-10-09 14:40:04,411][86122] Updated weights for policy 1, policy_version 62440 (0.0008) +[2023-10-09 14:40:04,785][86122] Updated weights for policy 1, policy_version 62450 (0.0010) +[2023-10-09 14:40:05,147][86122] Updated weights for policy 1, policy_version 62460 (0.0009) +[2023-10-09 14:40:07,923][86121] Updated weights for policy 0, policy_version 62210 (0.0007) +[2023-10-09 14:40:08,282][86121] Updated weights for policy 0, policy_version 62220 (0.0010) +[2023-10-09 14:40:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 127664128. Throughput: 0: 1826.4, 1: 1805.6. Samples: 31932802. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:40:08,636][86121] Updated weights for policy 0, policy_version 62230 (0.0007) +[2023-10-09 14:40:08,778][86122] Updated weights for policy 1, policy_version 62470 (0.0009) +[2023-10-09 14:40:08,995][86121] Updated weights for policy 0, policy_version 62240 (0.0007) +[2023-10-09 14:40:09,142][86122] Updated weights for policy 1, policy_version 62480 (0.0008) +[2023-10-09 14:40:09,505][86122] Updated weights for policy 1, policy_version 62490 (0.0007) +[2023-10-09 14:40:12,685][86121] Updated weights for policy 0, policy_version 62250 (0.0007) +[2023-10-09 14:40:13,043][86121] Updated weights for policy 0, policy_version 62260 (0.0008) +[2023-10-09 14:40:13,226][86122] Updated weights for policy 1, policy_version 62500 (0.0007) +[2023-10-09 14:40:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 127729664. Throughput: 0: 1827.7, 1: 1805.6. Samples: 31942864. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:40:13,411][86121] Updated weights for policy 0, policy_version 62270 (0.0008) +[2023-10-09 14:40:13,605][86122] Updated weights for policy 1, policy_version 62510 (0.0009) +[2023-10-09 14:40:13,967][86122] Updated weights for policy 1, policy_version 62520 (0.0007) +[2023-10-09 14:40:17,186][86121] Updated weights for policy 0, policy_version 62280 (0.0008) +[2023-10-09 14:40:17,557][86121] Updated weights for policy 0, policy_version 62290 (0.0008) +[2023-10-09 14:40:17,600][86122] Updated weights for policy 1, policy_version 62530 (0.0007) +[2023-10-09 14:40:17,918][86121] Updated weights for policy 0, policy_version 62300 (0.0008) +[2023-10-09 14:40:17,965][86122] Updated weights for policy 1, policy_version 62540 (0.0010) +[2023-10-09 14:40:18,334][86122] Updated weights for policy 1, policy_version 62550 (0.0009) +[2023-10-09 14:40:18,397][85186] Fps is (10 sec: 16383.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 127827968. Throughput: 0: 1825.5, 1: 1816.8. Samples: 31965816. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:18,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:40:18,692][86122] Updated weights for policy 1, policy_version 62560 (0.0008) +[2023-10-09 14:40:21,735][86121] Updated weights for policy 0, policy_version 62310 (0.0009) +[2023-10-09 14:40:22,098][86121] Updated weights for policy 0, policy_version 62320 (0.0009) +[2023-10-09 14:40:22,343][86122] Updated weights for policy 1, policy_version 62570 (0.0008) +[2023-10-09 14:40:22,460][86121] Updated weights for policy 0, policy_version 62330 (0.0008) +[2023-10-09 14:40:22,689][86122] Updated weights for policy 1, policy_version 62580 (0.0008) +[2023-10-09 14:40:23,058][86122] Updated weights for policy 1, policy_version 62590 (0.0009) +[2023-10-09 14:40:23,397][85186] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127926272. Throughput: 0: 1820.0, 1: 1818.2. Samples: 31986080. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.960')] +[2023-10-09 14:40:26,054][86121] Updated weights for policy 0, policy_version 62340 (0.0008) +[2023-10-09 14:40:26,417][86121] Updated weights for policy 0, policy_version 62350 (0.0009) +[2023-10-09 14:40:26,777][86121] Updated weights for policy 0, policy_version 62360 (0.0008) +[2023-10-09 14:40:26,812][86122] Updated weights for policy 1, policy_version 62600 (0.0007) +[2023-10-09 14:40:27,177][86122] Updated weights for policy 1, policy_version 62610 (0.0009) +[2023-10-09 14:40:27,539][86122] Updated weights for policy 1, policy_version 62620 (0.0010) +[2023-10-09 14:40:28,397][85186] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 127991808. Throughput: 0: 1818.2, 1: 1810.4. Samples: 31998358. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.950')] +[2023-10-09 14:40:30,448][86121] Updated weights for policy 0, policy_version 62370 (0.0008) +[2023-10-09 14:40:30,820][86121] Updated weights for policy 0, policy_version 62380 (0.0008) +[2023-10-09 14:40:31,180][86121] Updated weights for policy 0, policy_version 62390 (0.0008) +[2023-10-09 14:40:31,309][86122] Updated weights for policy 1, policy_version 62630 (0.0009) +[2023-10-09 14:40:31,546][86121] Updated weights for policy 0, policy_version 62400 (0.0007) +[2023-10-09 14:40:31,664][86122] Updated weights for policy 1, policy_version 62640 (0.0009) +[2023-10-09 14:40:32,029][86122] Updated weights for policy 1, policy_version 62650 (0.0009) +[2023-10-09 14:40:33,398][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 128057344. Throughput: 0: 1820.4, 1: 1816.8. Samples: 32018856. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:33,399][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:40:35,116][86121] Updated weights for policy 0, policy_version 62410 (0.0007) +[2023-10-09 14:40:35,481][86121] Updated weights for policy 0, policy_version 62420 (0.0008) +[2023-10-09 14:40:35,598][86122] Updated weights for policy 1, policy_version 62660 (0.0008) +[2023-10-09 14:40:35,839][86121] Updated weights for policy 0, policy_version 62430 (0.0008) +[2023-10-09 14:40:35,964][86122] Updated weights for policy 1, policy_version 62670 (0.0009) +[2023-10-09 14:40:36,330][86122] Updated weights for policy 1, policy_version 62680 (0.0008) +[2023-10-09 14:40:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128122880. Throughput: 0: 1824.8, 1: 1826.8. Samples: 32041608. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:40:39,454][86121] Updated weights for policy 0, policy_version 62440 (0.0009) +[2023-10-09 14:40:39,818][86121] Updated weights for policy 0, policy_version 62450 (0.0008) +[2023-10-09 14:40:39,951][86122] Updated weights for policy 1, policy_version 62690 (0.0009) +[2023-10-09 14:40:40,189][86121] Updated weights for policy 0, policy_version 62460 (0.0008) +[2023-10-09 14:40:40,302][86122] Updated weights for policy 1, policy_version 62700 (0.0008) +[2023-10-09 14:40:40,663][86122] Updated weights for policy 1, policy_version 62710 (0.0007) +[2023-10-09 14:40:41,026][86122] Updated weights for policy 1, policy_version 62720 (0.0010) +[2023-10-09 14:40:43,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128188416. Throughput: 0: 1822.8, 1: 1821.1. Samples: 32051746. Policy #0 lag: (min: 21.0, avg: 26.3, max: 53.0) +[2023-10-09 14:40:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:40:43,987][86121] Updated weights for policy 0, policy_version 62470 (0.0008) +[2023-10-09 14:40:44,361][86121] Updated weights for policy 0, policy_version 62480 (0.0008) +[2023-10-09 14:40:44,729][86121] Updated weights for policy 0, policy_version 62490 (0.0007) +[2023-10-09 14:40:44,774][86122] Updated weights for policy 1, policy_version 62730 (0.0009) +[2023-10-09 14:40:45,130][86122] Updated weights for policy 1, policy_version 62740 (0.0009) +[2023-10-09 14:40:45,491][86122] Updated weights for policy 1, policy_version 62750 (0.0008) +[2023-10-09 14:40:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 128253952. Throughput: 0: 1811.7, 1: 1825.6. Samples: 32073932. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:40:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:40:48,481][86121] Updated weights for policy 0, policy_version 62500 (0.0008) +[2023-10-09 14:40:48,843][86121] Updated weights for policy 0, policy_version 62510 (0.0007) +[2023-10-09 14:40:49,208][86121] Updated weights for policy 0, policy_version 62520 (0.0007) +[2023-10-09 14:40:49,290][86122] Updated weights for policy 1, policy_version 62760 (0.0008) +[2023-10-09 14:40:49,649][86122] Updated weights for policy 1, policy_version 62770 (0.0009) +[2023-10-09 14:40:50,008][86122] Updated weights for policy 1, policy_version 62780 (0.0012) +[2023-10-09 14:40:52,950][86121] Updated weights for policy 0, policy_version 62530 (0.0010) +[2023-10-09 14:40:53,314][86121] Updated weights for policy 0, policy_version 62540 (0.0008) +[2023-10-09 14:40:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128319488. Throughput: 0: 1816.0, 1: 1818.1. Samples: 32096336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:40:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.970')] +[2023-10-09 14:40:53,679][86121] Updated weights for policy 0, policy_version 62550 (0.0009) +[2023-10-09 14:40:53,699][86122] Updated weights for policy 1, policy_version 62790 (0.0008) +[2023-10-09 14:40:54,054][86122] Updated weights for policy 1, policy_version 62800 (0.0008) +[2023-10-09 14:40:54,054][86121] Updated weights for policy 0, policy_version 62560 (0.0009) +[2023-10-09 14:40:54,415][86122] Updated weights for policy 1, policy_version 62810 (0.0008) +[2023-10-09 14:40:57,754][86121] Updated weights for policy 0, policy_version 62570 (0.0007) +[2023-10-09 14:40:58,107][86122] Updated weights for policy 1, policy_version 62820 (0.0009) +[2023-10-09 14:40:58,121][86121] Updated weights for policy 0, policy_version 62580 (0.0007) +[2023-10-09 14:40:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 128385024. Throughput: 0: 1813.5, 1: 1817.8. Samples: 32106270. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:40:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.970')] +[2023-10-09 14:40:58,489][86122] Updated weights for policy 1, policy_version 62830 (0.0008) +[2023-10-09 14:40:58,491][86121] Updated weights for policy 0, policy_version 62590 (0.0007) +[2023-10-09 14:40:58,847][86122] Updated weights for policy 1, policy_version 62840 (0.0010) +[2023-10-09 14:41:02,185][86121] Updated weights for policy 0, policy_version 62600 (0.0007) +[2023-10-09 14:41:02,530][86122] Updated weights for policy 1, policy_version 62850 (0.0008) +[2023-10-09 14:41:02,554][86121] Updated weights for policy 0, policy_version 62610 (0.0008) +[2023-10-09 14:41:02,896][86122] Updated weights for policy 1, policy_version 62860 (0.0008) +[2023-10-09 14:41:02,929][86121] Updated weights for policy 0, policy_version 62620 (0.0007) +[2023-10-09 14:41:03,259][86122] Updated weights for policy 1, policy_version 62870 (0.0008) +[2023-10-09 14:41:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 128483328. Throughput: 0: 1811.0, 1: 1812.0. Samples: 32128850. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:41:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:41:03,618][86122] Updated weights for policy 1, policy_version 62880 (0.0007) +[2023-10-09 14:41:06,685][86121] Updated weights for policy 0, policy_version 62630 (0.0007) +[2023-10-09 14:41:07,046][86121] Updated weights for policy 0, policy_version 62640 (0.0007) +[2023-10-09 14:41:07,344][86122] Updated weights for policy 1, policy_version 62890 (0.0008) +[2023-10-09 14:41:07,408][86121] Updated weights for policy 0, policy_version 62650 (0.0007) +[2023-10-09 14:41:07,695][86122] Updated weights for policy 1, policy_version 62900 (0.0008) +[2023-10-09 14:41:08,061][86122] Updated weights for policy 1, policy_version 62910 (0.0009) +[2023-10-09 14:41:08,397][85186] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 128581632. Throughput: 0: 1811.1, 1: 1814.4. Samples: 32149228. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:41:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 14:41:11,149][86121] Updated weights for policy 0, policy_version 62660 (0.0009) +[2023-10-09 14:41:11,523][86121] Updated weights for policy 0, policy_version 62670 (0.0008) +[2023-10-09 14:41:11,780][86122] Updated weights for policy 1, policy_version 62920 (0.0008) +[2023-10-09 14:41:11,889][86121] Updated weights for policy 0, policy_version 62680 (0.0009) +[2023-10-09 14:41:12,130][86122] Updated weights for policy 1, policy_version 62930 (0.0008) +[2023-10-09 14:41:12,490][86122] Updated weights for policy 1, policy_version 62940 (0.0007) +[2023-10-09 14:41:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 128647168. Throughput: 0: 1808.6, 1: 1818.1. Samples: 32161562. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:41:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:41:15,631][86121] Updated weights for policy 0, policy_version 62690 (0.0007) +[2023-10-09 14:41:16,009][86121] Updated weights for policy 0, policy_version 62700 (0.0009) +[2023-10-09 14:41:16,180][86122] Updated weights for policy 1, policy_version 62950 (0.0009) +[2023-10-09 14:41:16,369][86121] Updated weights for policy 0, policy_version 62710 (0.0007) +[2023-10-09 14:41:16,545][86122] Updated weights for policy 1, policy_version 62960 (0.0010) +[2023-10-09 14:41:16,735][86121] Updated weights for policy 0, policy_version 62720 (0.0007) +[2023-10-09 14:41:16,912][86122] Updated weights for policy 1, policy_version 62970 (0.0010) +[2023-10-09 14:41:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 128712704. Throughput: 0: 1805.6, 1: 1821.3. Samples: 32182066. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:41:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:41:20,430][86122] Updated weights for policy 1, policy_version 62980 (0.0008) +[2023-10-09 14:41:20,605][86121] Updated weights for policy 0, policy_version 62730 (0.0007) +[2023-10-09 14:41:20,786][86122] Updated weights for policy 1, policy_version 62990 (0.0009) +[2023-10-09 14:41:20,963][86121] Updated weights for policy 0, policy_version 62740 (0.0007) +[2023-10-09 14:41:21,146][86122] Updated weights for policy 1, policy_version 63000 (0.0009) +[2023-10-09 14:41:21,336][86121] Updated weights for policy 0, policy_version 62750 (0.0007) +[2023-10-09 14:41:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 128778240. Throughput: 0: 1796.4, 1: 1826.4. Samples: 32204632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:41:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 14:41:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000062752_64258048.pth... +[2023-10-09 14:41:23,408][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000063008_64520192.pth... +[2023-10-09 14:41:23,437][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000061056_62521344.pth +[2023-10-09 14:41:23,439][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000061312_62783488.pth +[2023-10-09 14:41:24,768][86122] Updated weights for policy 1, policy_version 63010 (0.0007) +[2023-10-09 14:41:25,081][86121] Updated weights for policy 0, policy_version 62760 (0.0007) +[2023-10-09 14:41:25,125][86122] Updated weights for policy 1, policy_version 63020 (0.0007) +[2023-10-09 14:41:25,447][86121] Updated weights for policy 0, policy_version 62770 (0.0009) +[2023-10-09 14:41:25,484][86122] Updated weights for policy 1, policy_version 63030 (0.0009) +[2023-10-09 14:41:25,814][86121] Updated weights for policy 0, policy_version 62780 (0.0009) +[2023-10-09 14:41:25,843][86122] Updated weights for policy 1, policy_version 63040 (0.0008) +[2023-10-09 14:41:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 128843776. Throughput: 0: 1800.7, 1: 1822.2. Samples: 32214778. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:41:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:41:29,473][86121] Updated weights for policy 0, policy_version 62790 (0.0008) +[2023-10-09 14:41:29,546][86122] Updated weights for policy 1, policy_version 63050 (0.0008) +[2023-10-09 14:41:29,842][86121] Updated weights for policy 0, policy_version 62800 (0.0008) +[2023-10-09 14:41:29,905][86122] Updated weights for policy 1, policy_version 63060 (0.0011) +[2023-10-09 14:41:30,200][86121] Updated weights for policy 0, policy_version 62810 (0.0008) +[2023-10-09 14:41:30,276][86122] Updated weights for policy 1, policy_version 63070 (0.0008) +[2023-10-09 14:41:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128909312. Throughput: 0: 1801.6, 1: 1832.2. Samples: 32237454. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:41:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:41:33,918][86122] Updated weights for policy 1, policy_version 63080 (0.0008) +[2023-10-09 14:41:34,052][86121] Updated weights for policy 0, policy_version 62820 (0.0008) +[2023-10-09 14:41:34,281][86122] Updated weights for policy 1, policy_version 63090 (0.0008) +[2023-10-09 14:41:34,435][86121] Updated weights for policy 0, policy_version 62830 (0.0007) +[2023-10-09 14:41:34,640][86122] Updated weights for policy 1, policy_version 63100 (0.0009) +[2023-10-09 14:41:34,802][86121] Updated weights for policy 0, policy_version 62840 (0.0007) +[2023-10-09 14:41:38,382][86121] Updated weights for policy 0, policy_version 62850 (0.0008) +[2023-10-09 14:41:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 128974848. Throughput: 0: 1807.4, 1: 1839.5. Samples: 32260446. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:41:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:41:38,490][86122] Updated weights for policy 1, policy_version 63110 (0.0009) +[2023-10-09 14:41:38,747][86121] Updated weights for policy 0, policy_version 62860 (0.0008) +[2023-10-09 14:41:38,849][86122] Updated weights for policy 1, policy_version 63120 (0.0008) +[2023-10-09 14:41:39,108][86121] Updated weights for policy 0, policy_version 62870 (0.0009) +[2023-10-09 14:41:39,203][86122] Updated weights for policy 1, policy_version 63130 (0.0008) +[2023-10-09 14:41:39,476][86121] Updated weights for policy 0, policy_version 62880 (0.0007) +[2023-10-09 14:41:43,038][86122] Updated weights for policy 1, policy_version 63140 (0.0007) +[2023-10-09 14:41:43,092][86121] Updated weights for policy 0, policy_version 62890 (0.0007) +[2023-10-09 14:41:43,397][86122] Updated weights for policy 1, policy_version 63150 (0.0009) +[2023-10-09 14:41:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129040384. Throughput: 0: 1804.4, 1: 1836.8. Samples: 32270124. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:41:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:41:43,467][86121] Updated weights for policy 0, policy_version 62900 (0.0007) +[2023-10-09 14:41:43,763][86122] Updated weights for policy 1, policy_version 63160 (0.0008) +[2023-10-09 14:41:43,831][86121] Updated weights for policy 0, policy_version 62910 (0.0009) +[2023-10-09 14:41:47,294][86122] Updated weights for policy 1, policy_version 63170 (0.0008) +[2023-10-09 14:41:47,653][86122] Updated weights for policy 1, policy_version 63180 (0.0007) +[2023-10-09 14:41:47,703][86121] Updated weights for policy 0, policy_version 62920 (0.0010) +[2023-10-09 14:41:48,011][86122] Updated weights for policy 1, policy_version 63190 (0.0008) +[2023-10-09 14:41:48,067][86121] Updated weights for policy 0, policy_version 62930 (0.0008) +[2023-10-09 14:41:48,374][86122] Updated weights for policy 1, policy_version 63200 (0.0009) +[2023-10-09 14:41:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129138688. Throughput: 0: 1808.6, 1: 1844.2. Samples: 32293226. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:41:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:41:48,430][86121] Updated weights for policy 0, policy_version 62940 (0.0009) +[2023-10-09 14:41:51,947][86122] Updated weights for policy 1, policy_version 63210 (0.0010) +[2023-10-09 14:41:52,182][86121] Updated weights for policy 0, policy_version 62950 (0.0007) +[2023-10-09 14:41:52,313][86122] Updated weights for policy 1, policy_version 63220 (0.0009) +[2023-10-09 14:41:52,545][86121] Updated weights for policy 0, policy_version 62960 (0.0009) +[2023-10-09 14:41:52,675][86122] Updated weights for policy 1, policy_version 63230 (0.0007) +[2023-10-09 14:41:52,905][86121] Updated weights for policy 0, policy_version 62970 (0.0009) +[2023-10-09 14:41:53,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 129236992. Throughput: 0: 1811.2, 1: 1834.5. Samples: 32313282. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:41:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:41:56,377][86122] Updated weights for policy 1, policy_version 63240 (0.0009) +[2023-10-09 14:41:56,549][86121] Updated weights for policy 0, policy_version 62980 (0.0009) +[2023-10-09 14:41:56,745][86122] Updated weights for policy 1, policy_version 63250 (0.0009) +[2023-10-09 14:41:56,909][86121] Updated weights for policy 0, policy_version 62990 (0.0007) +[2023-10-09 14:41:57,094][86122] Updated weights for policy 1, policy_version 63260 (0.0009) +[2023-10-09 14:41:57,279][86121] Updated weights for policy 0, policy_version 63000 (0.0007) +[2023-10-09 14:41:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 129302528. Throughput: 0: 1807.7, 1: 1846.3. Samples: 32325990. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:41:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:42:00,695][86122] Updated weights for policy 1, policy_version 63270 (0.0009) +[2023-10-09 14:42:00,909][86121] Updated weights for policy 0, policy_version 63010 (0.0007) +[2023-10-09 14:42:01,054][86122] Updated weights for policy 1, policy_version 63280 (0.0008) +[2023-10-09 14:42:01,268][86121] Updated weights for policy 0, policy_version 63020 (0.0008) +[2023-10-09 14:42:01,424][86122] Updated weights for policy 1, policy_version 63290 (0.0009) +[2023-10-09 14:42:01,632][86121] Updated weights for policy 0, policy_version 63030 (0.0008) +[2023-10-09 14:42:02,001][86121] Updated weights for policy 0, policy_version 63040 (0.0009) +[2023-10-09 14:42:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129368064. Throughput: 0: 1815.6, 1: 1834.8. Samples: 32346330. Policy #0 lag: (min: 31.0, avg: 34.1, max: 63.0) +[2023-10-09 14:42:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:42:05,046][86122] Updated weights for policy 1, policy_version 63300 (0.0009) +[2023-10-09 14:42:05,413][86122] Updated weights for policy 1, policy_version 63310 (0.0008) +[2023-10-09 14:42:05,720][86121] Updated weights for policy 0, policy_version 63050 (0.0008) +[2023-10-09 14:42:05,770][86122] Updated weights for policy 1, policy_version 63320 (0.0008) +[2023-10-09 14:42:06,087][86121] Updated weights for policy 0, policy_version 63060 (0.0009) +[2023-10-09 14:42:06,471][86121] Updated weights for policy 0, policy_version 63070 (0.0009) +[2023-10-09 14:42:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 129433600. Throughput: 0: 1814.6, 1: 1843.9. Samples: 32369262. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:42:09,403][86122] Updated weights for policy 1, policy_version 63330 (0.0007) +[2023-10-09 14:42:09,759][86122] Updated weights for policy 1, policy_version 63340 (0.0007) +[2023-10-09 14:42:10,120][86122] Updated weights for policy 1, policy_version 63350 (0.0007) +[2023-10-09 14:42:10,129][86121] Updated weights for policy 0, policy_version 63080 (0.0008) +[2023-10-09 14:42:10,483][86122] Updated weights for policy 1, policy_version 63360 (0.0010) +[2023-10-09 14:42:10,493][86121] Updated weights for policy 0, policy_version 63090 (0.0010) +[2023-10-09 14:42:10,863][86121] Updated weights for policy 0, policy_version 63100 (0.0010) +[2023-10-09 14:42:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129499136. Throughput: 0: 1818.9, 1: 1838.7. Samples: 32379372. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:42:14,047][86122] Updated weights for policy 1, policy_version 63370 (0.0011) +[2023-10-09 14:42:14,398][86122] Updated weights for policy 1, policy_version 63380 (0.0009) +[2023-10-09 14:42:14,603][86121] Updated weights for policy 0, policy_version 63110 (0.0008) +[2023-10-09 14:42:14,760][86122] Updated weights for policy 1, policy_version 63390 (0.0008) +[2023-10-09 14:42:14,964][86121] Updated weights for policy 0, policy_version 63120 (0.0007) +[2023-10-09 14:42:15,324][86121] Updated weights for policy 0, policy_version 63130 (0.0009) +[2023-10-09 14:42:18,341][86122] Updated weights for policy 1, policy_version 63400 (0.0009) +[2023-10-09 14:42:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129564672. Throughput: 0: 1818.9, 1: 1853.1. Samples: 32402692. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:42:18,702][86122] Updated weights for policy 1, policy_version 63410 (0.0007) +[2023-10-09 14:42:18,917][86121] Updated weights for policy 0, policy_version 63140 (0.0010) +[2023-10-09 14:42:19,067][86122] Updated weights for policy 1, policy_version 63420 (0.0008) +[2023-10-09 14:42:19,309][86121] Updated weights for policy 0, policy_version 63150 (0.0010) +[2023-10-09 14:42:19,667][86121] Updated weights for policy 0, policy_version 63160 (0.0008) +[2023-10-09 14:42:22,826][86122] Updated weights for policy 1, policy_version 63430 (0.0009) +[2023-10-09 14:42:23,198][86122] Updated weights for policy 1, policy_version 63440 (0.0010) +[2023-10-09 14:42:23,208][86121] Updated weights for policy 0, policy_version 63170 (0.0008) +[2023-10-09 14:42:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 129630208. Throughput: 0: 1822.1, 1: 1841.5. Samples: 32425312. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:42:23,557][86122] Updated weights for policy 1, policy_version 63450 (0.0009) +[2023-10-09 14:42:23,578][86121] Updated weights for policy 0, policy_version 63180 (0.0008) +[2023-10-09 14:42:23,942][86121] Updated weights for policy 0, policy_version 63190 (0.0008) +[2023-10-09 14:42:24,309][86121] Updated weights for policy 0, policy_version 63200 (0.0010) +[2023-10-09 14:42:27,420][86122] Updated weights for policy 1, policy_version 63460 (0.0010) +[2023-10-09 14:42:27,790][86122] Updated weights for policy 1, policy_version 63470 (0.0009) +[2023-10-09 14:42:28,073][86121] Updated weights for policy 0, policy_version 63210 (0.0009) +[2023-10-09 14:42:28,152][86122] Updated weights for policy 1, policy_version 63480 (0.0009) +[2023-10-09 14:42:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 129695744. Throughput: 0: 1819.2, 1: 1847.3. Samples: 32435112. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:42:28,444][86121] Updated weights for policy 0, policy_version 63220 (0.0009) +[2023-10-09 14:42:28,804][86121] Updated weights for policy 0, policy_version 63230 (0.0008) +[2023-10-09 14:42:31,905][86122] Updated weights for policy 1, policy_version 63490 (0.0008) +[2023-10-09 14:42:32,309][86122] Updated weights for policy 1, policy_version 63500 (0.0009) +[2023-10-09 14:42:32,678][86122] Updated weights for policy 1, policy_version 63510 (0.0008) +[2023-10-09 14:42:32,685][86121] Updated weights for policy 0, policy_version 63240 (0.0008) +[2023-10-09 14:42:33,031][86122] Updated weights for policy 1, policy_version 63520 (0.0008) +[2023-10-09 14:42:33,054][86121] Updated weights for policy 0, policy_version 63250 (0.0007) +[2023-10-09 14:42:33,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 129794048. Throughput: 0: 1811.6, 1: 1837.9. Samples: 32457454. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:42:33,423][86121] Updated weights for policy 0, policy_version 63260 (0.0008) +[2023-10-09 14:42:36,738][86122] Updated weights for policy 1, policy_version 63530 (0.0008) +[2023-10-09 14:42:37,100][86122] Updated weights for policy 1, policy_version 63540 (0.0008) +[2023-10-09 14:42:37,244][86121] Updated weights for policy 0, policy_version 63270 (0.0008) +[2023-10-09 14:42:37,457][86122] Updated weights for policy 1, policy_version 63550 (0.0007) +[2023-10-09 14:42:37,620][86121] Updated weights for policy 0, policy_version 63280 (0.0009) +[2023-10-09 14:42:37,993][86121] Updated weights for policy 0, policy_version 63290 (0.0009) +[2023-10-09 14:42:38,397][85186] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 129892352. Throughput: 0: 1817.2, 1: 1830.8. Samples: 32477438. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:42:41,107][86122] Updated weights for policy 1, policy_version 63560 (0.0009) +[2023-10-09 14:42:41,476][86122] Updated weights for policy 1, policy_version 63570 (0.0008) +[2023-10-09 14:42:41,778][86121] Updated weights for policy 0, policy_version 63300 (0.0008) +[2023-10-09 14:42:41,832][86122] Updated weights for policy 1, policy_version 63580 (0.0007) +[2023-10-09 14:42:42,148][86121] Updated weights for policy 0, policy_version 63310 (0.0008) +[2023-10-09 14:42:42,514][86121] Updated weights for policy 0, policy_version 63320 (0.0008) +[2023-10-09 14:42:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 129957888. Throughput: 0: 1808.8, 1: 1830.9. Samples: 32489776. Policy #0 lag: (min: 28.0, avg: 35.5, max: 60.0) +[2023-10-09 14:42:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:42:45,506][86122] Updated weights for policy 1, policy_version 63590 (0.0009) +[2023-10-09 14:42:45,865][86122] Updated weights for policy 1, policy_version 63600 (0.0009) +[2023-10-09 14:42:46,225][86122] Updated weights for policy 1, policy_version 63610 (0.0008) +[2023-10-09 14:42:46,243][86121] Updated weights for policy 0, policy_version 63330 (0.0009) +[2023-10-09 14:42:46,610][86121] Updated weights for policy 0, policy_version 63340 (0.0009) +[2023-10-09 14:42:46,972][86121] Updated weights for policy 0, policy_version 63350 (0.0009) +[2023-10-09 14:42:47,345][86121] Updated weights for policy 0, policy_version 63360 (0.0008) +[2023-10-09 14:42:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 130023424. Throughput: 0: 1811.0, 1: 1827.9. Samples: 32510084. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:42:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:42:50,034][86122] Updated weights for policy 1, policy_version 63620 (0.0009) +[2023-10-09 14:42:50,389][86122] Updated weights for policy 1, policy_version 63630 (0.0011) +[2023-10-09 14:42:50,747][86122] Updated weights for policy 1, policy_version 63640 (0.0009) +[2023-10-09 14:42:51,093][86121] Updated weights for policy 0, policy_version 63370 (0.0007) +[2023-10-09 14:42:51,466][86121] Updated weights for policy 0, policy_version 63380 (0.0007) +[2023-10-09 14:42:51,826][86121] Updated weights for policy 0, policy_version 63390 (0.0007) +[2023-10-09 14:42:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 130088960. Throughput: 0: 1804.4, 1: 1815.5. Samples: 32532156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:42:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:42:54,536][86122] Updated weights for policy 1, policy_version 63650 (0.0009) +[2023-10-09 14:42:54,898][86122] Updated weights for policy 1, policy_version 63660 (0.0010) +[2023-10-09 14:42:55,259][86122] Updated weights for policy 1, policy_version 63670 (0.0010) +[2023-10-09 14:42:55,489][86121] Updated weights for policy 0, policy_version 63400 (0.0008) +[2023-10-09 14:42:55,627][86122] Updated weights for policy 1, policy_version 63680 (0.0007) +[2023-10-09 14:42:55,864][86121] Updated weights for policy 0, policy_version 63410 (0.0010) +[2023-10-09 14:42:56,223][86121] Updated weights for policy 0, policy_version 63420 (0.0009) +[2023-10-09 14:42:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 130154496. Throughput: 0: 1812.8, 1: 1814.1. Samples: 32542584. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:42:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:42:59,416][86122] Updated weights for policy 1, policy_version 63690 (0.0008) +[2023-10-09 14:42:59,784][86122] Updated weights for policy 1, policy_version 63700 (0.0008) +[2023-10-09 14:43:00,028][86121] Updated weights for policy 0, policy_version 63430 (0.0007) +[2023-10-09 14:43:00,136][86122] Updated weights for policy 1, policy_version 63710 (0.0007) +[2023-10-09 14:43:00,396][86121] Updated weights for policy 0, policy_version 63440 (0.0008) +[2023-10-09 14:43:00,777][86121] Updated weights for policy 0, policy_version 63450 (0.0008) +[2023-10-09 14:43:03,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 130220032. Throughput: 0: 1798.6, 1: 1802.9. Samples: 32564760. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:43:03,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:43:03,786][86122] Updated weights for policy 1, policy_version 63720 (0.0008) +[2023-10-09 14:43:04,148][86122] Updated weights for policy 1, policy_version 63730 (0.0010) +[2023-10-09 14:43:04,511][86122] Updated weights for policy 1, policy_version 63740 (0.0009) +[2023-10-09 14:43:04,523][86121] Updated weights for policy 0, policy_version 63460 (0.0009) +[2023-10-09 14:43:04,912][86121] Updated weights for policy 0, policy_version 63470 (0.0008) +[2023-10-09 14:43:05,269][86121] Updated weights for policy 0, policy_version 63480 (0.0008) +[2023-10-09 14:43:08,115][86122] Updated weights for policy 1, policy_version 63750 (0.0008) +[2023-10-09 14:43:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 130285568. Throughput: 0: 1789.2, 1: 1813.6. Samples: 32587434. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:43:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:43:08,486][86122] Updated weights for policy 1, policy_version 63760 (0.0009) +[2023-10-09 14:43:08,849][86122] Updated weights for policy 1, policy_version 63770 (0.0009) +[2023-10-09 14:43:09,010][86121] Updated weights for policy 0, policy_version 63490 (0.0008) +[2023-10-09 14:43:09,381][86121] Updated weights for policy 0, policy_version 63500 (0.0009) +[2023-10-09 14:43:09,754][86121] Updated weights for policy 0, policy_version 63510 (0.0008) +[2023-10-09 14:43:10,113][86121] Updated weights for policy 0, policy_version 63520 (0.0007) +[2023-10-09 14:43:12,485][86122] Updated weights for policy 1, policy_version 63780 (0.0010) +[2023-10-09 14:43:12,850][86122] Updated weights for policy 1, policy_version 63790 (0.0011) +[2023-10-09 14:43:13,216][86122] Updated weights for policy 1, policy_version 63800 (0.0007) +[2023-10-09 14:43:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 130351104. Throughput: 0: 1792.7, 1: 1811.0. Samples: 32597276. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:43:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:43:13,904][86121] Updated weights for policy 0, policy_version 63530 (0.0009) +[2023-10-09 14:43:14,268][86121] Updated weights for policy 0, policy_version 63540 (0.0007) +[2023-10-09 14:43:14,642][86121] Updated weights for policy 0, policy_version 63550 (0.0009) +[2023-10-09 14:43:16,942][86122] Updated weights for policy 1, policy_version 63810 (0.0008) +[2023-10-09 14:43:17,337][86122] Updated weights for policy 1, policy_version 63820 (0.0008) +[2023-10-09 14:43:17,697][86122] Updated weights for policy 1, policy_version 63830 (0.0007) +[2023-10-09 14:43:18,057][86122] Updated weights for policy 1, policy_version 63840 (0.0008) +[2023-10-09 14:43:18,322][86121] Updated weights for policy 0, policy_version 63560 (0.0009) +[2023-10-09 14:43:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130449408. Throughput: 0: 1791.8, 1: 1817.7. Samples: 32619880. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:43:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:43:18,689][86121] Updated weights for policy 0, policy_version 63570 (0.0010) +[2023-10-09 14:43:19,065][86121] Updated weights for policy 0, policy_version 63580 (0.0009) +[2023-10-09 14:43:21,617][86122] Updated weights for policy 1, policy_version 63850 (0.0009) +[2023-10-09 14:43:21,991][86122] Updated weights for policy 1, policy_version 63860 (0.0010) +[2023-10-09 14:43:22,347][86122] Updated weights for policy 1, policy_version 63870 (0.0007) +[2023-10-09 14:43:22,846][86121] Updated weights for policy 0, policy_version 63590 (0.0007) +[2023-10-09 14:43:23,208][86121] Updated weights for policy 0, policy_version 63600 (0.0008) +[2023-10-09 14:43:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 130514944. Throughput: 0: 1809.4, 1: 1823.8. Samples: 32640930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:43:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:43:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000063872_65404928.pth... +[2023-10-09 14:43:23,446][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000062176_63668224.pth +[2023-10-09 14:43:23,585][86121] Updated weights for policy 0, policy_version 63610 (0.0010) +[2023-10-09 14:43:23,798][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000063616_65142784.pth... +[2023-10-09 14:43:23,837][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000061920_63406080.pth +[2023-10-09 14:43:26,111][86122] Updated weights for policy 1, policy_version 63880 (0.0010) +[2023-10-09 14:43:26,477][86122] Updated weights for policy 1, policy_version 63890 (0.0010) +[2023-10-09 14:43:26,826][86122] Updated weights for policy 1, policy_version 63900 (0.0010) +[2023-10-09 14:43:27,226][86121] Updated weights for policy 0, policy_version 63620 (0.0008) +[2023-10-09 14:43:27,599][86121] Updated weights for policy 0, policy_version 63630 (0.0008) +[2023-10-09 14:43:27,958][86121] Updated weights for policy 0, policy_version 63640 (0.0007) +[2023-10-09 14:43:28,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 130613248. Throughput: 0: 1796.0, 1: 1823.6. Samples: 32652660. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:43:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:43:30,508][86122] Updated weights for policy 1, policy_version 63910 (0.0009) +[2023-10-09 14:43:30,865][86122] Updated weights for policy 1, policy_version 63920 (0.0010) +[2023-10-09 14:43:31,233][86122] Updated weights for policy 1, policy_version 63930 (0.0010) +[2023-10-09 14:43:31,673][86121] Updated weights for policy 0, policy_version 63650 (0.0007) +[2023-10-09 14:43:32,045][86121] Updated weights for policy 0, policy_version 63660 (0.0008) +[2023-10-09 14:43:32,404][86121] Updated weights for policy 0, policy_version 63670 (0.0009) +[2023-10-09 14:43:32,768][86121] Updated weights for policy 0, policy_version 63680 (0.0010) +[2023-10-09 14:43:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 130678784. Throughput: 0: 1812.4, 1: 1827.8. Samples: 32673892. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:43:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:43:34,872][86122] Updated weights for policy 1, policy_version 63940 (0.0009) +[2023-10-09 14:43:35,227][86122] Updated weights for policy 1, policy_version 63950 (0.0009) +[2023-10-09 14:43:35,588][86122] Updated weights for policy 1, policy_version 63960 (0.0011) +[2023-10-09 14:43:36,678][86121] Updated weights for policy 0, policy_version 63690 (0.0009) +[2023-10-09 14:43:37,032][86121] Updated weights for policy 0, policy_version 63700 (0.0008) +[2023-10-09 14:43:37,401][86121] Updated weights for policy 0, policy_version 63710 (0.0008) +[2023-10-09 14:43:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 130744320. Throughput: 0: 1797.3, 1: 1836.8. Samples: 32695690. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:43:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:43:39,280][86122] Updated weights for policy 1, policy_version 63970 (0.0010) +[2023-10-09 14:43:39,648][86122] Updated weights for policy 1, policy_version 63980 (0.0007) +[2023-10-09 14:43:40,007][86122] Updated weights for policy 1, policy_version 63990 (0.0007) +[2023-10-09 14:43:40,369][86122] Updated weights for policy 1, policy_version 64000 (0.0010) +[2023-10-09 14:43:41,100][86121] Updated weights for policy 0, policy_version 63720 (0.0008) +[2023-10-09 14:43:41,474][86121] Updated weights for policy 0, policy_version 63730 (0.0007) +[2023-10-09 14:43:41,838][86121] Updated weights for policy 0, policy_version 63740 (0.0009) +[2023-10-09 14:43:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 130809856. Throughput: 0: 1813.4, 1: 1838.0. Samples: 32706898. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:43:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:43:43,910][86122] Updated weights for policy 1, policy_version 64010 (0.0008) +[2023-10-09 14:43:44,267][86122] Updated weights for policy 1, policy_version 64020 (0.0009) +[2023-10-09 14:43:44,636][86122] Updated weights for policy 1, policy_version 64030 (0.0008) +[2023-10-09 14:43:45,442][86121] Updated weights for policy 0, policy_version 63750 (0.0008) +[2023-10-09 14:43:45,804][86121] Updated weights for policy 0, policy_version 63760 (0.0008) +[2023-10-09 14:43:46,173][86121] Updated weights for policy 0, policy_version 63770 (0.0009) +[2023-10-09 14:43:48,216][86122] Updated weights for policy 1, policy_version 64040 (0.0009) +[2023-10-09 14:43:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 130875392. Throughput: 0: 1802.4, 1: 1845.2. Samples: 32728900. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:43:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:43:48,578][86122] Updated weights for policy 1, policy_version 64050 (0.0009) +[2023-10-09 14:43:48,936][86122] Updated weights for policy 1, policy_version 64060 (0.0009) +[2023-10-09 14:43:49,887][86121] Updated weights for policy 0, policy_version 63780 (0.0008) +[2023-10-09 14:43:50,273][86121] Updated weights for policy 0, policy_version 63790 (0.0009) +[2023-10-09 14:43:50,638][86121] Updated weights for policy 0, policy_version 63800 (0.0008) +[2023-10-09 14:43:52,713][86122] Updated weights for policy 1, policy_version 64070 (0.0010) +[2023-10-09 14:43:53,078][86122] Updated weights for policy 1, policy_version 64080 (0.0010) +[2023-10-09 14:43:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 130940928. Throughput: 0: 1807.3, 1: 1833.3. Samples: 32751262. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:43:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:43:53,430][86122] Updated weights for policy 1, policy_version 64090 (0.0009) +[2023-10-09 14:43:54,115][86121] Updated weights for policy 0, policy_version 63810 (0.0008) +[2023-10-09 14:43:54,479][86121] Updated weights for policy 0, policy_version 63820 (0.0010) +[2023-10-09 14:43:54,847][86121] Updated weights for policy 0, policy_version 63830 (0.0010) +[2023-10-09 14:43:55,214][86121] Updated weights for policy 0, policy_version 63840 (0.0007) +[2023-10-09 14:43:57,186][86122] Updated weights for policy 1, policy_version 64100 (0.0008) +[2023-10-09 14:43:57,540][86122] Updated weights for policy 1, policy_version 64110 (0.0008) +[2023-10-09 14:43:57,901][86122] Updated weights for policy 1, policy_version 64120 (0.0009) +[2023-10-09 14:43:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131039232. Throughput: 0: 1809.4, 1: 1840.6. Samples: 32761526. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:43:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:43:59,117][86121] Updated weights for policy 0, policy_version 63850 (0.0011) +[2023-10-09 14:43:59,481][86121] Updated weights for policy 0, policy_version 63860 (0.0009) +[2023-10-09 14:43:59,853][86121] Updated weights for policy 0, policy_version 63870 (0.0007) +[2023-10-09 14:44:01,637][86122] Updated weights for policy 1, policy_version 64130 (0.0007) +[2023-10-09 14:44:02,031][86122] Updated weights for policy 1, policy_version 64140 (0.0009) +[2023-10-09 14:44:02,392][86122] Updated weights for policy 1, policy_version 64150 (0.0009) +[2023-10-09 14:44:02,755][86122] Updated weights for policy 1, policy_version 64160 (0.0007) +[2023-10-09 14:44:03,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 131104768. Throughput: 0: 1808.9, 1: 1830.4. Samples: 32783650. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:44:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:44:03,576][86121] Updated weights for policy 0, policy_version 63880 (0.0010) +[2023-10-09 14:44:03,940][86121] Updated weights for policy 0, policy_version 63890 (0.0011) +[2023-10-09 14:44:04,299][86121] Updated weights for policy 0, policy_version 63900 (0.0009) +[2023-10-09 14:44:06,358][86122] Updated weights for policy 1, policy_version 64170 (0.0007) +[2023-10-09 14:44:06,729][86122] Updated weights for policy 1, policy_version 64180 (0.0009) +[2023-10-09 14:44:07,079][86122] Updated weights for policy 1, policy_version 64190 (0.0008) +[2023-10-09 14:44:08,143][86121] Updated weights for policy 0, policy_version 63910 (0.0008) +[2023-10-09 14:44:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131170304. Throughput: 0: 1820.0, 1: 1831.1. Samples: 32805228. Policy #0 lag: (min: 2.0, avg: 9.6, max: 34.0) +[2023-10-09 14:44:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:44:08,505][86121] Updated weights for policy 0, policy_version 63920 (0.0009) +[2023-10-09 14:44:08,880][86121] Updated weights for policy 0, policy_version 63930 (0.0008) +[2023-10-09 14:44:10,743][86122] Updated weights for policy 1, policy_version 64200 (0.0007) +[2023-10-09 14:44:11,113][86122] Updated weights for policy 1, policy_version 64210 (0.0007) +[2023-10-09 14:44:11,476][86122] Updated weights for policy 1, policy_version 64220 (0.0009) +[2023-10-09 14:44:12,585][86121] Updated weights for policy 0, policy_version 63940 (0.0008) +[2023-10-09 14:44:12,954][86121] Updated weights for policy 0, policy_version 63950 (0.0009) +[2023-10-09 14:44:13,315][86121] Updated weights for policy 0, policy_version 63960 (0.0008) +[2023-10-09 14:44:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131235840. Throughput: 0: 1811.6, 1: 1822.4. Samples: 32816192. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:44:15,239][86122] Updated weights for policy 1, policy_version 64230 (0.0007) +[2023-10-09 14:44:15,602][86122] Updated weights for policy 1, policy_version 64240 (0.0008) +[2023-10-09 14:44:15,966][86122] Updated weights for policy 1, policy_version 64250 (0.0010) +[2023-10-09 14:44:17,059][86121] Updated weights for policy 0, policy_version 63970 (0.0008) +[2023-10-09 14:44:17,424][86121] Updated weights for policy 0, policy_version 63980 (0.0010) +[2023-10-09 14:44:17,791][86121] Updated weights for policy 0, policy_version 63990 (0.0009) +[2023-10-09 14:44:18,146][86121] Updated weights for policy 0, policy_version 64000 (0.0007) +[2023-10-09 14:44:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131334144. Throughput: 0: 1816.6, 1: 1830.8. Samples: 32838022. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:44:19,614][86122] Updated weights for policy 1, policy_version 64260 (0.0010) +[2023-10-09 14:44:19,978][86122] Updated weights for policy 1, policy_version 64270 (0.0010) +[2023-10-09 14:44:20,337][86122] Updated weights for policy 1, policy_version 64280 (0.0008) +[2023-10-09 14:44:21,751][86121] Updated weights for policy 0, policy_version 64010 (0.0009) +[2023-10-09 14:44:22,129][86121] Updated weights for policy 0, policy_version 64020 (0.0010) +[2023-10-09 14:44:22,493][86121] Updated weights for policy 0, policy_version 64030 (0.0011) +[2023-10-09 14:44:23,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 131399680. Throughput: 0: 1809.7, 1: 1827.3. Samples: 32859356. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:44:24,111][86122] Updated weights for policy 1, policy_version 64290 (0.0008) +[2023-10-09 14:44:24,475][86122] Updated weights for policy 1, policy_version 64300 (0.0007) +[2023-10-09 14:44:24,835][86122] Updated weights for policy 1, policy_version 64310 (0.0008) +[2023-10-09 14:44:25,199][86122] Updated weights for policy 1, policy_version 64320 (0.0007) +[2023-10-09 14:44:26,114][86121] Updated weights for policy 0, policy_version 64040 (0.0008) +[2023-10-09 14:44:26,480][86121] Updated weights for policy 0, policy_version 64050 (0.0007) +[2023-10-09 14:44:26,845][86121] Updated weights for policy 0, policy_version 64060 (0.0007) +[2023-10-09 14:44:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131465216. Throughput: 0: 1811.4, 1: 1828.0. Samples: 32870670. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:44:28,880][86122] Updated weights for policy 1, policy_version 64330 (0.0008) +[2023-10-09 14:44:29,240][86122] Updated weights for policy 1, policy_version 64340 (0.0008) +[2023-10-09 14:44:29,605][86122] Updated weights for policy 1, policy_version 64350 (0.0009) +[2023-10-09 14:44:30,657][86121] Updated weights for policy 0, policy_version 64070 (0.0008) +[2023-10-09 14:44:31,021][86121] Updated weights for policy 0, policy_version 64080 (0.0007) +[2023-10-09 14:44:31,387][86121] Updated weights for policy 0, policy_version 64090 (0.0008) +[2023-10-09 14:44:33,279][86122] Updated weights for policy 1, policy_version 64360 (0.0007) +[2023-10-09 14:44:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 131530752. Throughput: 0: 1813.6, 1: 1820.1. Samples: 32892418. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:44:33,645][86122] Updated weights for policy 1, policy_version 64370 (0.0007) +[2023-10-09 14:44:34,005][86122] Updated weights for policy 1, policy_version 64380 (0.0008) +[2023-10-09 14:44:34,774][86121] Updated weights for policy 0, policy_version 64100 (0.0008) +[2023-10-09 14:44:35,151][86121] Updated weights for policy 0, policy_version 64110 (0.0009) +[2023-10-09 14:44:35,522][86121] Updated weights for policy 0, policy_version 64120 (0.0009) +[2023-10-09 14:44:37,755][86122] Updated weights for policy 1, policy_version 64390 (0.0008) +[2023-10-09 14:44:38,119][86122] Updated weights for policy 1, policy_version 64400 (0.0009) +[2023-10-09 14:44:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 131596288. Throughput: 0: 1818.4, 1: 1820.5. Samples: 32915016. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:44:38,486][86122] Updated weights for policy 1, policy_version 64410 (0.0008) +[2023-10-09 14:44:39,219][86121] Updated weights for policy 0, policy_version 64130 (0.0010) +[2023-10-09 14:44:39,586][86121] Updated weights for policy 0, policy_version 64140 (0.0009) +[2023-10-09 14:44:39,943][86121] Updated weights for policy 0, policy_version 64150 (0.0010) +[2023-10-09 14:44:40,306][86121] Updated weights for policy 0, policy_version 64160 (0.0011) +[2023-10-09 14:44:42,168][86122] Updated weights for policy 1, policy_version 64420 (0.0010) +[2023-10-09 14:44:42,533][86122] Updated weights for policy 1, policy_version 64430 (0.0007) +[2023-10-09 14:44:42,893][86122] Updated weights for policy 1, policy_version 64440 (0.0010) +[2023-10-09 14:44:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131694592. Throughput: 0: 1817.0, 1: 1820.6. Samples: 32925218. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:44:43,977][86121] Updated weights for policy 0, policy_version 64170 (0.0007) +[2023-10-09 14:44:44,342][86121] Updated weights for policy 0, policy_version 64180 (0.0009) +[2023-10-09 14:44:44,707][86121] Updated weights for policy 0, policy_version 64190 (0.0007) +[2023-10-09 14:44:46,595][86122] Updated weights for policy 1, policy_version 64450 (0.0007) +[2023-10-09 14:44:46,996][86122] Updated weights for policy 1, policy_version 64460 (0.0009) +[2023-10-09 14:44:47,354][86122] Updated weights for policy 1, policy_version 64470 (0.0009) +[2023-10-09 14:44:47,709][86122] Updated weights for policy 1, policy_version 64480 (0.0008) +[2023-10-09 14:44:48,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131760128. Throughput: 0: 1830.7, 1: 1820.1. Samples: 32947936. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:44:48,499][86121] Updated weights for policy 0, policy_version 64200 (0.0009) +[2023-10-09 14:44:48,861][86121] Updated weights for policy 0, policy_version 64210 (0.0009) +[2023-10-09 14:44:49,228][86121] Updated weights for policy 0, policy_version 64220 (0.0007) +[2023-10-09 14:44:51,580][86122] Updated weights for policy 1, policy_version 64490 (0.0008) +[2023-10-09 14:44:51,937][86122] Updated weights for policy 1, policy_version 64500 (0.0008) +[2023-10-09 14:44:52,293][86122] Updated weights for policy 1, policy_version 64510 (0.0008) +[2023-10-09 14:44:53,007][86121] Updated weights for policy 0, policy_version 64230 (0.0008) +[2023-10-09 14:44:53,374][86121] Updated weights for policy 0, policy_version 64240 (0.0009) +[2023-10-09 14:44:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 131825664. Throughput: 0: 1821.9, 1: 1820.3. Samples: 32969128. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 14:44:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:44:53,731][86121] Updated weights for policy 0, policy_version 64250 (0.0009) +[2023-10-09 14:44:55,928][86122] Updated weights for policy 1, policy_version 64520 (0.0008) +[2023-10-09 14:44:56,297][86122] Updated weights for policy 1, policy_version 64530 (0.0008) +[2023-10-09 14:44:56,659][86122] Updated weights for policy 1, policy_version 64540 (0.0007) +[2023-10-09 14:44:57,386][86121] Updated weights for policy 0, policy_version 64260 (0.0011) +[2023-10-09 14:44:57,756][86121] Updated weights for policy 0, policy_version 64270 (0.0008) +[2023-10-09 14:44:58,120][86121] Updated weights for policy 0, policy_version 64280 (0.0009) +[2023-10-09 14:44:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 131891200. Throughput: 0: 1825.6, 1: 1824.7. Samples: 32980456. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:44:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:45:00,251][86122] Updated weights for policy 1, policy_version 64550 (0.0008) +[2023-10-09 14:45:00,621][86122] Updated weights for policy 1, policy_version 64560 (0.0007) +[2023-10-09 14:45:00,984][86122] Updated weights for policy 1, policy_version 64570 (0.0008) +[2023-10-09 14:45:01,859][86121] Updated weights for policy 0, policy_version 64290 (0.0007) +[2023-10-09 14:45:02,221][86121] Updated weights for policy 0, policy_version 64300 (0.0010) +[2023-10-09 14:45:02,593][86121] Updated weights for policy 0, policy_version 64310 (0.0007) +[2023-10-09 14:45:02,954][86121] Updated weights for policy 0, policy_version 64320 (0.0010) +[2023-10-09 14:45:03,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 131989504. Throughput: 0: 1825.6, 1: 1817.9. Samples: 33001980. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:45:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:45:04,721][86122] Updated weights for policy 1, policy_version 64580 (0.0008) +[2023-10-09 14:45:05,085][86122] Updated weights for policy 1, policy_version 64590 (0.0009) +[2023-10-09 14:45:05,451][86122] Updated weights for policy 1, policy_version 64600 (0.0008) +[2023-10-09 14:45:06,708][86121] Updated weights for policy 0, policy_version 64330 (0.0007) +[2023-10-09 14:45:07,079][86121] Updated weights for policy 0, policy_version 64340 (0.0008) +[2023-10-09 14:45:07,439][86121] Updated weights for policy 0, policy_version 64350 (0.0009) +[2023-10-09 14:45:08,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 132055040. Throughput: 0: 1832.3, 1: 1818.6. Samples: 33023648. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:45:08,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:45:09,128][86122] Updated weights for policy 1, policy_version 64610 (0.0007) +[2023-10-09 14:45:09,486][86122] Updated weights for policy 1, policy_version 64620 (0.0007) +[2023-10-09 14:45:09,855][86122] Updated weights for policy 1, policy_version 64630 (0.0009) +[2023-10-09 14:45:10,218][86122] Updated weights for policy 1, policy_version 64640 (0.0008) +[2023-10-09 14:45:11,184][86121] Updated weights for policy 0, policy_version 64360 (0.0010) +[2023-10-09 14:45:11,547][86121] Updated weights for policy 0, policy_version 64370 (0.0009) +[2023-10-09 14:45:11,909][86121] Updated weights for policy 0, policy_version 64380 (0.0008) +[2023-10-09 14:45:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132120576. Throughput: 0: 1830.1, 1: 1820.0. Samples: 33034924. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:45:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:45:13,760][86122] Updated weights for policy 1, policy_version 64650 (0.0008) +[2023-10-09 14:45:14,124][86122] Updated weights for policy 1, policy_version 64660 (0.0008) +[2023-10-09 14:45:14,483][86122] Updated weights for policy 1, policy_version 64670 (0.0007) +[2023-10-09 14:45:15,529][86121] Updated weights for policy 0, policy_version 64390 (0.0009) +[2023-10-09 14:45:15,898][86121] Updated weights for policy 0, policy_version 64400 (0.0007) +[2023-10-09 14:45:16,254][86121] Updated weights for policy 0, policy_version 64410 (0.0008) +[2023-10-09 14:45:18,130][86122] Updated weights for policy 1, policy_version 64680 (0.0010) +[2023-10-09 14:45:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 132186112. Throughput: 0: 1826.4, 1: 1824.0. Samples: 33056686. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:45:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:45:18,493][86122] Updated weights for policy 1, policy_version 64690 (0.0008) +[2023-10-09 14:45:18,867][86122] Updated weights for policy 1, policy_version 64700 (0.0009) +[2023-10-09 14:45:19,767][86121] Updated weights for policy 0, policy_version 64420 (0.0007) +[2023-10-09 14:45:20,162][86121] Updated weights for policy 0, policy_version 64430 (0.0010) +[2023-10-09 14:45:20,524][86121] Updated weights for policy 0, policy_version 64440 (0.0008) +[2023-10-09 14:45:22,753][86122] Updated weights for policy 1, policy_version 64710 (0.0008) +[2023-10-09 14:45:23,100][86122] Updated weights for policy 1, policy_version 64720 (0.0007) +[2023-10-09 14:45:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132251648. Throughput: 0: 1818.3, 1: 1820.6. Samples: 33078768. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:45:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:45:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000064448_65994752.pth... +[2023-10-09 14:45:23,436][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000062752_64258048.pth +[2023-10-09 14:45:23,459][86122] Updated weights for policy 1, policy_version 64730 (0.0007) +[2023-10-09 14:45:23,676][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000064736_66289664.pth... +[2023-10-09 14:45:23,716][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000063008_64520192.pth +[2023-10-09 14:45:24,137][86121] Updated weights for policy 0, policy_version 64450 (0.0008) +[2023-10-09 14:45:24,507][86121] Updated weights for policy 0, policy_version 64460 (0.0008) +[2023-10-09 14:45:24,872][86121] Updated weights for policy 0, policy_version 64470 (0.0008) +[2023-10-09 14:45:25,244][86121] Updated weights for policy 0, policy_version 64480 (0.0008) +[2023-10-09 14:45:27,155][86122] Updated weights for policy 1, policy_version 64740 (0.0009) +[2023-10-09 14:45:27,518][86122] Updated weights for policy 1, policy_version 64750 (0.0011) +[2023-10-09 14:45:27,884][86122] Updated weights for policy 1, policy_version 64760 (0.0011) +[2023-10-09 14:45:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132349952. Throughput: 0: 1819.9, 1: 1819.2. Samples: 33088976. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:45:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:45:29,025][86121] Updated weights for policy 0, policy_version 64490 (0.0009) +[2023-10-09 14:45:29,385][86121] Updated weights for policy 0, policy_version 64500 (0.0010) +[2023-10-09 14:45:29,743][86121] Updated weights for policy 0, policy_version 64510 (0.0010) +[2023-10-09 14:45:31,778][86122] Updated weights for policy 1, policy_version 64770 (0.0011) +[2023-10-09 14:45:32,161][86122] Updated weights for policy 1, policy_version 64780 (0.0010) +[2023-10-09 14:45:32,524][86122] Updated weights for policy 1, policy_version 64790 (0.0010) +[2023-10-09 14:45:32,885][86122] Updated weights for policy 1, policy_version 64800 (0.0009) +[2023-10-09 14:45:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 132415488. Throughput: 0: 1811.5, 1: 1814.9. Samples: 33111124. Policy #0 lag: (min: 25.0, avg: 42.6, max: 57.0) +[2023-10-09 14:45:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:45:33,489][86121] Updated weights for policy 0, policy_version 64520 (0.0009) +[2023-10-09 14:45:33,857][86121] Updated weights for policy 0, policy_version 64530 (0.0009) +[2023-10-09 14:45:34,229][86121] Updated weights for policy 0, policy_version 64540 (0.0008) +[2023-10-09 14:45:36,519][86122] Updated weights for policy 1, policy_version 64810 (0.0008) +[2023-10-09 14:45:36,886][86122] Updated weights for policy 1, policy_version 64820 (0.0007) +[2023-10-09 14:45:37,244][86122] Updated weights for policy 1, policy_version 64830 (0.0007) +[2023-10-09 14:45:37,681][86121] Updated weights for policy 0, policy_version 64550 (0.0008) +[2023-10-09 14:45:38,043][86121] Updated weights for policy 0, policy_version 64560 (0.0007) +[2023-10-09 14:45:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132481024. Throughput: 0: 1811.0, 1: 1816.3. Samples: 33132358. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:45:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:45:38,406][86121] Updated weights for policy 0, policy_version 64570 (0.0007) +[2023-10-09 14:45:40,872][86122] Updated weights for policy 1, policy_version 64840 (0.0009) +[2023-10-09 14:45:41,232][86122] Updated weights for policy 1, policy_version 64850 (0.0009) +[2023-10-09 14:45:41,594][86122] Updated weights for policy 1, policy_version 64860 (0.0012) +[2023-10-09 14:45:41,976][86121] Updated weights for policy 0, policy_version 64580 (0.0008) +[2023-10-09 14:45:42,341][86121] Updated weights for policy 0, policy_version 64590 (0.0010) +[2023-10-09 14:45:42,718][86121] Updated weights for policy 0, policy_version 64600 (0.0010) +[2023-10-09 14:45:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132579328. Throughput: 0: 1825.4, 1: 1817.7. Samples: 33144396. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:45:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:45:45,393][86122] Updated weights for policy 1, policy_version 64870 (0.0010) +[2023-10-09 14:45:45,753][86122] Updated weights for policy 1, policy_version 64880 (0.0007) +[2023-10-09 14:45:46,113][86122] Updated weights for policy 1, policy_version 64890 (0.0007) +[2023-10-09 14:45:46,567][86121] Updated weights for policy 0, policy_version 64610 (0.0010) +[2023-10-09 14:45:46,922][86121] Updated weights for policy 0, policy_version 64620 (0.0010) +[2023-10-09 14:45:47,294][86121] Updated weights for policy 0, policy_version 64630 (0.0009) +[2023-10-09 14:45:47,656][86121] Updated weights for policy 0, policy_version 64640 (0.0009) +[2023-10-09 14:45:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132644864. Throughput: 0: 1816.1, 1: 1821.2. Samples: 33165660. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:45:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:45:49,658][86122] Updated weights for policy 1, policy_version 64900 (0.0007) +[2023-10-09 14:45:50,015][86122] Updated weights for policy 1, policy_version 64910 (0.0007) +[2023-10-09 14:45:50,381][86122] Updated weights for policy 1, policy_version 64920 (0.0008) +[2023-10-09 14:45:51,463][86121] Updated weights for policy 0, policy_version 64650 (0.0008) +[2023-10-09 14:45:51,831][86121] Updated weights for policy 0, policy_version 64660 (0.0009) +[2023-10-09 14:45:52,201][86121] Updated weights for policy 0, policy_version 64670 (0.0007) +[2023-10-09 14:45:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 132710400. Throughput: 0: 1822.5, 1: 1819.3. Samples: 33187528. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:45:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:45:54,147][86122] Updated weights for policy 1, policy_version 64930 (0.0007) +[2023-10-09 14:45:54,507][86122] Updated weights for policy 1, policy_version 64940 (0.0007) +[2023-10-09 14:45:54,872][86122] Updated weights for policy 1, policy_version 64950 (0.0008) +[2023-10-09 14:45:55,232][86122] Updated weights for policy 1, policy_version 64960 (0.0009) +[2023-10-09 14:45:56,017][86121] Updated weights for policy 0, policy_version 64680 (0.0008) +[2023-10-09 14:45:56,374][86121] Updated weights for policy 0, policy_version 64690 (0.0008) +[2023-10-09 14:45:56,735][86121] Updated weights for policy 0, policy_version 64700 (0.0008) +[2023-10-09 14:45:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 132775936. Throughput: 0: 1821.3, 1: 1818.1. Samples: 33198698. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:45:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:45:58,915][86122] Updated weights for policy 1, policy_version 64970 (0.0009) +[2023-10-09 14:45:59,270][86122] Updated weights for policy 1, policy_version 64980 (0.0009) +[2023-10-09 14:45:59,632][86122] Updated weights for policy 1, policy_version 64990 (0.0007) +[2023-10-09 14:46:00,293][86121] Updated weights for policy 0, policy_version 64710 (0.0007) +[2023-10-09 14:46:00,657][86121] Updated weights for policy 0, policy_version 64720 (0.0007) +[2023-10-09 14:46:01,017][86121] Updated weights for policy 0, policy_version 64730 (0.0008) +[2023-10-09 14:46:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 132841472. Throughput: 0: 1826.1, 1: 1817.9. Samples: 33220666. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:46:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:46:03,485][86122] Updated weights for policy 1, policy_version 65000 (0.0007) +[2023-10-09 14:46:03,848][86122] Updated weights for policy 1, policy_version 65010 (0.0007) +[2023-10-09 14:46:04,212][86122] Updated weights for policy 1, policy_version 65020 (0.0007) +[2023-10-09 14:46:04,670][86121] Updated weights for policy 0, policy_version 64740 (0.0008) +[2023-10-09 14:46:05,032][86121] Updated weights for policy 0, policy_version 64750 (0.0009) +[2023-10-09 14:46:05,399][86121] Updated weights for policy 0, policy_version 64760 (0.0009) +[2023-10-09 14:46:07,885][86122] Updated weights for policy 1, policy_version 65030 (0.0008) +[2023-10-09 14:46:08,247][86122] Updated weights for policy 1, policy_version 65040 (0.0008) +[2023-10-09 14:46:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 132907008. Throughput: 0: 1833.2, 1: 1827.5. Samples: 33243500. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:46:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:46:08,618][86122] Updated weights for policy 1, policy_version 65050 (0.0010) +[2023-10-09 14:46:09,105][86121] Updated weights for policy 0, policy_version 64770 (0.0009) +[2023-10-09 14:46:09,471][86121] Updated weights for policy 0, policy_version 64780 (0.0009) +[2023-10-09 14:46:09,832][86121] Updated weights for policy 0, policy_version 64790 (0.0009) +[2023-10-09 14:46:10,198][86121] Updated weights for policy 0, policy_version 64800 (0.0007) +[2023-10-09 14:46:12,261][86122] Updated weights for policy 1, policy_version 65060 (0.0008) +[2023-10-09 14:46:12,627][86122] Updated weights for policy 1, policy_version 65070 (0.0009) +[2023-10-09 14:46:12,992][86122] Updated weights for policy 1, policy_version 65080 (0.0009) +[2023-10-09 14:46:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133005312. Throughput: 0: 1830.9, 1: 1827.2. Samples: 33253590. Policy #0 lag: (min: 10.0, avg: 10.0, max: 10.0) +[2023-10-09 14:46:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:46:13,931][86121] Updated weights for policy 0, policy_version 64810 (0.0010) +[2023-10-09 14:46:14,303][86121] Updated weights for policy 0, policy_version 64820 (0.0007) +[2023-10-09 14:46:14,667][86121] Updated weights for policy 0, policy_version 64830 (0.0007) +[2023-10-09 14:46:16,769][86122] Updated weights for policy 1, policy_version 65090 (0.0009) +[2023-10-09 14:46:17,164][86122] Updated weights for policy 1, policy_version 65100 (0.0008) +[2023-10-09 14:46:17,526][86122] Updated weights for policy 1, policy_version 65110 (0.0010) +[2023-10-09 14:46:17,889][86122] Updated weights for policy 1, policy_version 65120 (0.0009) +[2023-10-09 14:46:18,357][86121] Updated weights for policy 0, policy_version 64840 (0.0009) +[2023-10-09 14:46:18,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133070848. Throughput: 0: 1834.1, 1: 1835.9. Samples: 33276274. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:46:18,727][86121] Updated weights for policy 0, policy_version 64850 (0.0008) +[2023-10-09 14:46:19,092][86121] Updated weights for policy 0, policy_version 64860 (0.0009) +[2023-10-09 14:46:21,500][86122] Updated weights for policy 1, policy_version 65130 (0.0009) +[2023-10-09 14:46:21,862][86122] Updated weights for policy 1, policy_version 65140 (0.0011) +[2023-10-09 14:46:22,226][86122] Updated weights for policy 1, policy_version 65150 (0.0009) +[2023-10-09 14:46:22,746][86121] Updated weights for policy 0, policy_version 64870 (0.0009) +[2023-10-09 14:46:23,109][86121] Updated weights for policy 0, policy_version 64880 (0.0009) +[2023-10-09 14:46:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133136384. Throughput: 0: 1831.6, 1: 1835.1. Samples: 33297358. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:46:23,470][86121] Updated weights for policy 0, policy_version 64890 (0.0009) +[2023-10-09 14:46:25,735][86122] Updated weights for policy 1, policy_version 65160 (0.0009) +[2023-10-09 14:46:26,098][86122] Updated weights for policy 1, policy_version 65170 (0.0010) +[2023-10-09 14:46:26,451][86122] Updated weights for policy 1, policy_version 65180 (0.0007) +[2023-10-09 14:46:27,107][86121] Updated weights for policy 0, policy_version 64900 (0.0008) +[2023-10-09 14:46:27,474][86121] Updated weights for policy 0, policy_version 64910 (0.0008) +[2023-10-09 14:46:27,841][86121] Updated weights for policy 0, policy_version 64920 (0.0008) +[2023-10-09 14:46:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 133234688. Throughput: 0: 1829.7, 1: 1831.0. Samples: 33309126. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:46:29,994][86122] Updated weights for policy 1, policy_version 65190 (0.0008) +[2023-10-09 14:46:30,350][86122] Updated weights for policy 1, policy_version 65200 (0.0007) +[2023-10-09 14:46:30,714][86122] Updated weights for policy 1, policy_version 65210 (0.0008) +[2023-10-09 14:46:31,542][86121] Updated weights for policy 0, policy_version 64930 (0.0009) +[2023-10-09 14:46:31,906][86121] Updated weights for policy 0, policy_version 64940 (0.0009) +[2023-10-09 14:46:32,280][86121] Updated weights for policy 0, policy_version 64950 (0.0011) +[2023-10-09 14:46:32,653][86121] Updated weights for policy 0, policy_version 64960 (0.0010) +[2023-10-09 14:46:33,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133300224. Throughput: 0: 1824.0, 1: 1835.7. Samples: 33330348. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 14:46:34,259][86122] Updated weights for policy 1, policy_version 65220 (0.0008) +[2023-10-09 14:46:34,628][86122] Updated weights for policy 1, policy_version 65230 (0.0007) +[2023-10-09 14:46:34,994][86122] Updated weights for policy 1, policy_version 65240 (0.0008) +[2023-10-09 14:46:36,479][86121] Updated weights for policy 0, policy_version 64970 (0.0009) +[2023-10-09 14:46:36,840][86121] Updated weights for policy 0, policy_version 64980 (0.0010) +[2023-10-09 14:46:37,199][86121] Updated weights for policy 0, policy_version 64990 (0.0011) +[2023-10-09 14:46:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133365760. Throughput: 0: 1817.6, 1: 1843.6. Samples: 33352286. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 14:46:38,658][86122] Updated weights for policy 1, policy_version 65250 (0.0007) +[2023-10-09 14:46:39,026][86122] Updated weights for policy 1, policy_version 65260 (0.0008) +[2023-10-09 14:46:39,389][86122] Updated weights for policy 1, policy_version 65270 (0.0007) +[2023-10-09 14:46:39,745][86122] Updated weights for policy 1, policy_version 65280 (0.0007) +[2023-10-09 14:46:40,933][86121] Updated weights for policy 0, policy_version 65000 (0.0010) +[2023-10-09 14:46:41,292][86121] Updated weights for policy 0, policy_version 65010 (0.0009) +[2023-10-09 14:46:41,666][86121] Updated weights for policy 0, policy_version 65020 (0.0009) +[2023-10-09 14:46:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 133431296. Throughput: 0: 1817.2, 1: 1843.8. Samples: 33363444. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 14:46:43,422][86122] Updated weights for policy 1, policy_version 65290 (0.0007) +[2023-10-09 14:46:43,779][86122] Updated weights for policy 1, policy_version 65300 (0.0010) +[2023-10-09 14:46:44,139][86122] Updated weights for policy 1, policy_version 65310 (0.0011) +[2023-10-09 14:46:45,253][86121] Updated weights for policy 0, policy_version 65030 (0.0008) +[2023-10-09 14:46:45,616][86121] Updated weights for policy 0, policy_version 65040 (0.0009) +[2023-10-09 14:46:45,983][86121] Updated weights for policy 0, policy_version 65050 (0.0009) +[2023-10-09 14:46:47,759][86122] Updated weights for policy 1, policy_version 65320 (0.0011) +[2023-10-09 14:46:48,116][86122] Updated weights for policy 1, policy_version 65330 (0.0008) +[2023-10-09 14:46:48,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 133496832. Throughput: 0: 1820.5, 1: 1841.6. Samples: 33385460. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:48,399][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:46:48,482][86122] Updated weights for policy 1, policy_version 65340 (0.0007) +[2023-10-09 14:46:49,712][86121] Updated weights for policy 0, policy_version 65060 (0.0008) +[2023-10-09 14:46:50,099][86121] Updated weights for policy 0, policy_version 65070 (0.0007) +[2023-10-09 14:46:50,464][86121] Updated weights for policy 0, policy_version 65080 (0.0010) +[2023-10-09 14:46:52,171][86122] Updated weights for policy 1, policy_version 65350 (0.0009) +[2023-10-09 14:46:52,518][86122] Updated weights for policy 1, policy_version 65360 (0.0009) +[2023-10-09 14:46:52,875][86122] Updated weights for policy 1, policy_version 65370 (0.0007) +[2023-10-09 14:46:53,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133595136. Throughput: 0: 1812.1, 1: 1826.5. Samples: 33407238. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:46:54,296][86121] Updated weights for policy 0, policy_version 65090 (0.0009) +[2023-10-09 14:46:54,668][86121] Updated weights for policy 0, policy_version 65100 (0.0010) +[2023-10-09 14:46:55,037][86121] Updated weights for policy 0, policy_version 65110 (0.0008) +[2023-10-09 14:46:55,407][86121] Updated weights for policy 0, policy_version 65120 (0.0008) +[2023-10-09 14:46:56,531][86122] Updated weights for policy 1, policy_version 65380 (0.0008) +[2023-10-09 14:46:56,886][86122] Updated weights for policy 1, policy_version 65390 (0.0008) +[2023-10-09 14:46:57,252][86122] Updated weights for policy 1, policy_version 65400 (0.0008) +[2023-10-09 14:46:58,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133660672. Throughput: 0: 1812.6, 1: 1849.2. Samples: 33418370. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 14:46:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:46:59,044][86121] Updated weights for policy 0, policy_version 65130 (0.0009) +[2023-10-09 14:46:59,413][86121] Updated weights for policy 0, policy_version 65140 (0.0008) +[2023-10-09 14:46:59,773][86121] Updated weights for policy 0, policy_version 65150 (0.0009) +[2023-10-09 14:47:01,137][86122] Updated weights for policy 1, policy_version 65410 (0.0011) +[2023-10-09 14:47:01,527][86122] Updated weights for policy 1, policy_version 65420 (0.0009) +[2023-10-09 14:47:01,889][86122] Updated weights for policy 1, policy_version 65430 (0.0010) +[2023-10-09 14:47:02,245][86122] Updated weights for policy 1, policy_version 65440 (0.0007) +[2023-10-09 14:47:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 133726208. Throughput: 0: 1812.7, 1: 1828.9. Samples: 33440146. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:47:03,435][86121] Updated weights for policy 0, policy_version 65160 (0.0007) +[2023-10-09 14:47:03,800][86121] Updated weights for policy 0, policy_version 65170 (0.0009) +[2023-10-09 14:47:04,169][86121] Updated weights for policy 0, policy_version 65180 (0.0011) +[2023-10-09 14:47:05,672][86122] Updated weights for policy 1, policy_version 65450 (0.0008) +[2023-10-09 14:47:06,034][86122] Updated weights for policy 1, policy_version 65460 (0.0008) +[2023-10-09 14:47:06,402][86122] Updated weights for policy 1, policy_version 65470 (0.0008) +[2023-10-09 14:47:07,840][86121] Updated weights for policy 0, policy_version 65190 (0.0010) +[2023-10-09 14:47:08,211][86121] Updated weights for policy 0, policy_version 65200 (0.0010) +[2023-10-09 14:47:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 133791744. Throughput: 0: 1814.9, 1: 1850.3. Samples: 33462292. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:47:08,574][86121] Updated weights for policy 0, policy_version 65210 (0.0008) +[2023-10-09 14:47:10,018][86122] Updated weights for policy 1, policy_version 65480 (0.0010) +[2023-10-09 14:47:10,376][86122] Updated weights for policy 1, policy_version 65490 (0.0009) +[2023-10-09 14:47:10,738][86122] Updated weights for policy 1, policy_version 65500 (0.0011) +[2023-10-09 14:47:12,302][86121] Updated weights for policy 0, policy_version 65220 (0.0009) +[2023-10-09 14:47:12,662][86121] Updated weights for policy 0, policy_version 65230 (0.0009) +[2023-10-09 14:47:13,032][86121] Updated weights for policy 0, policy_version 65240 (0.0008) +[2023-10-09 14:47:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133890048. Throughput: 0: 1811.2, 1: 1828.7. Samples: 33472918. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 14:47:14,478][86122] Updated weights for policy 1, policy_version 65510 (0.0008) +[2023-10-09 14:47:14,837][86122] Updated weights for policy 1, policy_version 65520 (0.0008) +[2023-10-09 14:47:15,209][86122] Updated weights for policy 1, policy_version 65530 (0.0010) +[2023-10-09 14:47:16,792][86121] Updated weights for policy 0, policy_version 65250 (0.0009) +[2023-10-09 14:47:17,162][86121] Updated weights for policy 0, policy_version 65260 (0.0010) +[2023-10-09 14:47:17,522][86121] Updated weights for policy 0, policy_version 65270 (0.0011) +[2023-10-09 14:47:17,893][86121] Updated weights for policy 0, policy_version 65280 (0.0011) +[2023-10-09 14:47:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 133955584. Throughput: 0: 1820.4, 1: 1844.8. Samples: 33495282. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:18,735][86122] Updated weights for policy 1, policy_version 65540 (0.0010) +[2023-10-09 14:47:19,101][86122] Updated weights for policy 1, policy_version 65550 (0.0008) +[2023-10-09 14:47:19,457][86122] Updated weights for policy 1, policy_version 65560 (0.0007) +[2023-10-09 14:47:21,645][86121] Updated weights for policy 0, policy_version 65290 (0.0007) +[2023-10-09 14:47:22,016][86121] Updated weights for policy 0, policy_version 65300 (0.0008) +[2023-10-09 14:47:22,387][86121] Updated weights for policy 0, policy_version 65310 (0.0008) +[2023-10-09 14:47:23,021][86122] Updated weights for policy 1, policy_version 65570 (0.0007) +[2023-10-09 14:47:23,380][86122] Updated weights for policy 1, policy_version 65580 (0.0007) +[2023-10-09 14:47:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134021120. Throughput: 0: 1819.5, 1: 1843.7. Samples: 33517130. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:23,405][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000065312_66879488.pth... +[2023-10-09 14:47:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000063616_65142784.pth +[2023-10-09 14:47:23,746][86122] Updated weights for policy 1, policy_version 65590 (0.0007) +[2023-10-09 14:47:24,105][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000065600_67174400.pth... +[2023-10-09 14:47:24,108][86122] Updated weights for policy 1, policy_version 65600 (0.0008) +[2023-10-09 14:47:24,134][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000063872_65404928.pth +[2023-10-09 14:47:26,113][86121] Updated weights for policy 0, policy_version 65320 (0.0007) +[2023-10-09 14:47:26,486][86121] Updated weights for policy 0, policy_version 65330 (0.0007) +[2023-10-09 14:47:26,842][86121] Updated weights for policy 0, policy_version 65340 (0.0007) +[2023-10-09 14:47:27,827][86122] Updated weights for policy 1, policy_version 65610 (0.0008) +[2023-10-09 14:47:28,190][86122] Updated weights for policy 1, policy_version 65620 (0.0009) +[2023-10-09 14:47:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 134086656. Throughput: 0: 1822.1, 1: 1848.0. Samples: 33528596. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:28,545][86122] Updated weights for policy 1, policy_version 65630 (0.0009) +[2023-10-09 14:47:30,528][86121] Updated weights for policy 0, policy_version 65350 (0.0009) +[2023-10-09 14:47:30,896][86121] Updated weights for policy 0, policy_version 65360 (0.0007) +[2023-10-09 14:47:31,269][86121] Updated weights for policy 0, policy_version 65370 (0.0008) +[2023-10-09 14:47:32,283][86122] Updated weights for policy 1, policy_version 65640 (0.0009) +[2023-10-09 14:47:32,652][86122] Updated weights for policy 1, policy_version 65650 (0.0010) +[2023-10-09 14:47:33,013][86122] Updated weights for policy 1, policy_version 65660 (0.0009) +[2023-10-09 14:47:33,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134184960. Throughput: 0: 1820.4, 1: 1845.8. Samples: 33550438. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:35,048][86121] Updated weights for policy 0, policy_version 65380 (0.0010) +[2023-10-09 14:47:35,437][86121] Updated weights for policy 0, policy_version 65390 (0.0010) +[2023-10-09 14:47:35,806][86121] Updated weights for policy 0, policy_version 65400 (0.0010) +[2023-10-09 14:47:36,660][86122] Updated weights for policy 1, policy_version 65670 (0.0008) +[2023-10-09 14:47:37,020][86122] Updated weights for policy 1, policy_version 65680 (0.0008) +[2023-10-09 14:47:37,377][86122] Updated weights for policy 1, policy_version 65690 (0.0008) +[2023-10-09 14:47:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134250496. Throughput: 0: 1820.9, 1: 1833.7. Samples: 33571696. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:39,356][86121] Updated weights for policy 0, policy_version 65410 (0.0009) +[2023-10-09 14:47:39,718][86121] Updated weights for policy 0, policy_version 65420 (0.0007) +[2023-10-09 14:47:40,087][86121] Updated weights for policy 0, policy_version 65430 (0.0009) +[2023-10-09 14:47:40,452][86121] Updated weights for policy 0, policy_version 65440 (0.0011) +[2023-10-09 14:47:41,237][86122] Updated weights for policy 1, policy_version 65700 (0.0008) +[2023-10-09 14:47:41,590][86122] Updated weights for policy 1, policy_version 65710 (0.0009) +[2023-10-09 14:47:41,959][86122] Updated weights for policy 1, policy_version 65720 (0.0009) +[2023-10-09 14:47:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134316032. Throughput: 0: 1821.0, 1: 1835.3. Samples: 33582904. Policy #0 lag: (min: 31.0, avg: 38.6, max: 63.0) +[2023-10-09 14:47:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:44,014][86121] Updated weights for policy 0, policy_version 65450 (0.0009) +[2023-10-09 14:47:44,369][86121] Updated weights for policy 0, policy_version 65460 (0.0010) +[2023-10-09 14:47:44,741][86121] Updated weights for policy 0, policy_version 65470 (0.0008) +[2023-10-09 14:47:45,756][86122] Updated weights for policy 1, policy_version 65730 (0.0008) +[2023-10-09 14:47:46,116][86122] Updated weights for policy 1, policy_version 65740 (0.0009) +[2023-10-09 14:47:46,482][86122] Updated weights for policy 1, policy_version 65750 (0.0009) +[2023-10-09 14:47:46,846][86122] Updated weights for policy 1, policy_version 65760 (0.0008) +[2023-10-09 14:47:48,317][86121] Updated weights for policy 0, policy_version 65480 (0.0008) +[2023-10-09 14:47:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 134381568. Throughput: 0: 1825.7, 1: 1825.0. Samples: 33604430. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:47:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:48,688][86121] Updated weights for policy 0, policy_version 65490 (0.0008) +[2023-10-09 14:47:49,057][86121] Updated weights for policy 0, policy_version 65500 (0.0008) +[2023-10-09 14:47:50,616][86122] Updated weights for policy 1, policy_version 65770 (0.0008) +[2023-10-09 14:47:50,981][86122] Updated weights for policy 1, policy_version 65780 (0.0009) +[2023-10-09 14:47:51,342][86122] Updated weights for policy 1, policy_version 65790 (0.0009) +[2023-10-09 14:47:52,790][86121] Updated weights for policy 0, policy_version 65510 (0.0010) +[2023-10-09 14:47:53,164][86121] Updated weights for policy 0, policy_version 65520 (0.0009) +[2023-10-09 14:47:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 134447104. Throughput: 0: 1825.4, 1: 1823.9. Samples: 33626508. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:47:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:53,525][86121] Updated weights for policy 0, policy_version 65530 (0.0007) +[2023-10-09 14:47:54,991][86122] Updated weights for policy 1, policy_version 65800 (0.0009) +[2023-10-09 14:47:55,349][86122] Updated weights for policy 1, policy_version 65810 (0.0009) +[2023-10-09 14:47:55,710][86122] Updated weights for policy 1, policy_version 65820 (0.0008) +[2023-10-09 14:47:57,299][86121] Updated weights for policy 0, policy_version 65540 (0.0007) +[2023-10-09 14:47:57,664][86121] Updated weights for policy 0, policy_version 65550 (0.0008) +[2023-10-09 14:47:58,034][86121] Updated weights for policy 0, policy_version 65560 (0.0009) +[2023-10-09 14:47:58,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134545408. Throughput: 0: 1825.1, 1: 1825.1. Samples: 33637176. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:47:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:47:59,518][86122] Updated weights for policy 1, policy_version 65830 (0.0008) +[2023-10-09 14:47:59,878][86122] Updated weights for policy 1, policy_version 65840 (0.0008) +[2023-10-09 14:48:00,233][86122] Updated weights for policy 1, policy_version 65850 (0.0008) +[2023-10-09 14:48:01,731][86121] Updated weights for policy 0, policy_version 65570 (0.0009) +[2023-10-09 14:48:02,104][86121] Updated weights for policy 0, policy_version 65580 (0.0007) +[2023-10-09 14:48:02,474][86121] Updated weights for policy 0, policy_version 65590 (0.0007) +[2023-10-09 14:48:02,833][86121] Updated weights for policy 0, policy_version 65600 (0.0007) +[2023-10-09 14:48:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134610944. Throughput: 0: 1822.0, 1: 1823.6. Samples: 33659334. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:48:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:03,979][86122] Updated weights for policy 1, policy_version 65860 (0.0007) +[2023-10-09 14:48:04,337][86122] Updated weights for policy 1, policy_version 65870 (0.0007) +[2023-10-09 14:48:04,704][86122] Updated weights for policy 1, policy_version 65880 (0.0007) +[2023-10-09 14:48:06,406][86121] Updated weights for policy 0, policy_version 65610 (0.0011) +[2023-10-09 14:48:06,784][86121] Updated weights for policy 0, policy_version 65620 (0.0010) +[2023-10-09 14:48:07,146][86121] Updated weights for policy 0, policy_version 65630 (0.0008) +[2023-10-09 14:48:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 134676480. Throughput: 0: 1825.3, 1: 1814.6. Samples: 33680928. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:48:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:08,474][86122] Updated weights for policy 1, policy_version 65890 (0.0007) +[2023-10-09 14:48:08,840][86122] Updated weights for policy 1, policy_version 65900 (0.0007) +[2023-10-09 14:48:09,198][86122] Updated weights for policy 1, policy_version 65910 (0.0009) +[2023-10-09 14:48:09,554][86122] Updated weights for policy 1, policy_version 65920 (0.0007) +[2023-10-09 14:48:10,826][86121] Updated weights for policy 0, policy_version 65640 (0.0011) +[2023-10-09 14:48:11,195][86121] Updated weights for policy 0, policy_version 65650 (0.0008) +[2023-10-09 14:48:11,561][86121] Updated weights for policy 0, policy_version 65660 (0.0007) +[2023-10-09 14:48:13,237][86122] Updated weights for policy 1, policy_version 65930 (0.0008) +[2023-10-09 14:48:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 134742016. Throughput: 0: 1818.3, 1: 1809.4. Samples: 33691844. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:48:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:13,611][86122] Updated weights for policy 1, policy_version 65940 (0.0007) +[2023-10-09 14:48:13,963][86122] Updated weights for policy 1, policy_version 65950 (0.0010) +[2023-10-09 14:48:15,136][86121] Updated weights for policy 0, policy_version 65670 (0.0007) +[2023-10-09 14:48:15,503][86121] Updated weights for policy 0, policy_version 65680 (0.0009) +[2023-10-09 14:48:15,869][86121] Updated weights for policy 0, policy_version 65690 (0.0007) +[2023-10-09 14:48:17,578][86122] Updated weights for policy 1, policy_version 65960 (0.0009) +[2023-10-09 14:48:17,936][86122] Updated weights for policy 1, policy_version 65970 (0.0009) +[2023-10-09 14:48:18,295][86122] Updated weights for policy 1, policy_version 65980 (0.0008) +[2023-10-09 14:48:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 134807552. Throughput: 0: 1827.8, 1: 1809.2. Samples: 33714106. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:48:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:19,593][86121] Updated weights for policy 0, policy_version 65700 (0.0008) +[2023-10-09 14:48:19,986][86121] Updated weights for policy 0, policy_version 65710 (0.0007) +[2023-10-09 14:48:20,349][86121] Updated weights for policy 0, policy_version 65720 (0.0008) +[2023-10-09 14:48:21,859][86122] Updated weights for policy 1, policy_version 65990 (0.0008) +[2023-10-09 14:48:22,219][86122] Updated weights for policy 1, policy_version 66000 (0.0007) +[2023-10-09 14:48:22,574][86122] Updated weights for policy 1, policy_version 66010 (0.0007) +[2023-10-09 14:48:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134905856. Throughput: 0: 1827.1, 1: 1817.9. Samples: 33735718. Policy #0 lag: (min: 0.0, avg: 26.6, max: 32.0) +[2023-10-09 14:48:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:23,972][86121] Updated weights for policy 0, policy_version 65730 (0.0008) +[2023-10-09 14:48:24,341][86121] Updated weights for policy 0, policy_version 65740 (0.0007) +[2023-10-09 14:48:24,704][86121] Updated weights for policy 0, policy_version 65750 (0.0008) +[2023-10-09 14:48:25,076][86121] Updated weights for policy 0, policy_version 65760 (0.0010) +[2023-10-09 14:48:26,271][86122] Updated weights for policy 1, policy_version 66020 (0.0007) +[2023-10-09 14:48:26,629][86122] Updated weights for policy 1, policy_version 66030 (0.0007) +[2023-10-09 14:48:26,997][86122] Updated weights for policy 1, policy_version 66040 (0.0007) +[2023-10-09 14:48:28,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 134971392. Throughput: 0: 1824.9, 1: 1820.9. Samples: 33746966. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:48:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:28,936][86121] Updated weights for policy 0, policy_version 65770 (0.0010) +[2023-10-09 14:48:29,300][86121] Updated weights for policy 0, policy_version 65780 (0.0008) +[2023-10-09 14:48:29,667][86121] Updated weights for policy 0, policy_version 65790 (0.0008) +[2023-10-09 14:48:30,698][86122] Updated weights for policy 1, policy_version 66050 (0.0008) +[2023-10-09 14:48:31,060][86122] Updated weights for policy 1, policy_version 66060 (0.0008) +[2023-10-09 14:48:31,423][86122] Updated weights for policy 1, policy_version 66070 (0.0007) +[2023-10-09 14:48:31,788][86122] Updated weights for policy 1, policy_version 66080 (0.0009) +[2023-10-09 14:48:33,306][86121] Updated weights for policy 0, policy_version 65800 (0.0007) +[2023-10-09 14:48:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 135036928. Throughput: 0: 1818.7, 1: 1825.2. Samples: 33768402. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:48:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:33,671][86121] Updated weights for policy 0, policy_version 65810 (0.0010) +[2023-10-09 14:48:34,035][86121] Updated weights for policy 0, policy_version 65820 (0.0010) +[2023-10-09 14:48:35,329][86122] Updated weights for policy 1, policy_version 66090 (0.0008) +[2023-10-09 14:48:35,695][86122] Updated weights for policy 1, policy_version 66100 (0.0010) +[2023-10-09 14:48:36,051][86122] Updated weights for policy 1, policy_version 66110 (0.0008) +[2023-10-09 14:48:37,626][86121] Updated weights for policy 0, policy_version 65830 (0.0008) +[2023-10-09 14:48:37,990][86121] Updated weights for policy 0, policy_version 65840 (0.0008) +[2023-10-09 14:48:38,357][86121] Updated weights for policy 0, policy_version 65850 (0.0010) +[2023-10-09 14:48:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 135102464. Throughput: 0: 1817.7, 1: 1832.3. Samples: 33790762. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:48:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:39,756][86122] Updated weights for policy 1, policy_version 66120 (0.0008) +[2023-10-09 14:48:40,126][86122] Updated weights for policy 1, policy_version 66130 (0.0008) +[2023-10-09 14:48:40,488][86122] Updated weights for policy 1, policy_version 66140 (0.0011) +[2023-10-09 14:48:42,170][86121] Updated weights for policy 0, policy_version 65860 (0.0009) +[2023-10-09 14:48:42,539][86121] Updated weights for policy 0, policy_version 65870 (0.0008) +[2023-10-09 14:48:42,905][86121] Updated weights for policy 0, policy_version 65880 (0.0008) +[2023-10-09 14:48:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135200768. Throughput: 0: 1820.4, 1: 1825.9. Samples: 33801258. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:48:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:44,225][86122] Updated weights for policy 1, policy_version 66150 (0.0008) +[2023-10-09 14:48:44,585][86122] Updated weights for policy 1, policy_version 66160 (0.0007) +[2023-10-09 14:48:44,945][86122] Updated weights for policy 1, policy_version 66170 (0.0007) +[2023-10-09 14:48:46,772][86121] Updated weights for policy 0, policy_version 65890 (0.0008) +[2023-10-09 14:48:47,134][86121] Updated weights for policy 0, policy_version 65900 (0.0008) +[2023-10-09 14:48:47,503][86121] Updated weights for policy 0, policy_version 65910 (0.0009) +[2023-10-09 14:48:47,869][86121] Updated weights for policy 0, policy_version 65920 (0.0008) +[2023-10-09 14:48:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135266304. Throughput: 0: 1823.8, 1: 1831.3. Samples: 33823814. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:48:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:48,726][86122] Updated weights for policy 1, policy_version 66180 (0.0009) +[2023-10-09 14:48:49,083][86122] Updated weights for policy 1, policy_version 66190 (0.0007) +[2023-10-09 14:48:49,452][86122] Updated weights for policy 1, policy_version 66200 (0.0007) +[2023-10-09 14:48:51,653][86121] Updated weights for policy 0, policy_version 65930 (0.0010) +[2023-10-09 14:48:52,015][86121] Updated weights for policy 0, policy_version 65940 (0.0009) +[2023-10-09 14:48:52,380][86121] Updated weights for policy 0, policy_version 65950 (0.0008) +[2023-10-09 14:48:52,949][86122] Updated weights for policy 1, policy_version 66210 (0.0007) +[2023-10-09 14:48:53,310][86122] Updated weights for policy 1, policy_version 66220 (0.0008) +[2023-10-09 14:48:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135331840. Throughput: 0: 1817.6, 1: 1841.1. Samples: 33845570. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:48:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:48:53,668][86122] Updated weights for policy 1, policy_version 66230 (0.0008) +[2023-10-09 14:48:54,021][86122] Updated weights for policy 1, policy_version 66240 (0.0009) +[2023-10-09 14:48:55,870][86121] Updated weights for policy 0, policy_version 65960 (0.0008) +[2023-10-09 14:48:56,237][86121] Updated weights for policy 0, policy_version 65970 (0.0010) +[2023-10-09 14:48:56,602][86121] Updated weights for policy 0, policy_version 65980 (0.0007) +[2023-10-09 14:48:57,555][86122] Updated weights for policy 1, policy_version 66250 (0.0008) +[2023-10-09 14:48:57,905][86122] Updated weights for policy 1, policy_version 66260 (0.0010) +[2023-10-09 14:48:58,265][86122] Updated weights for policy 1, policy_version 66270 (0.0009) +[2023-10-09 14:48:58,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 135430144. Throughput: 0: 1822.0, 1: 1843.5. Samples: 33856792. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:48:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 14:49:00,151][86121] Updated weights for policy 0, policy_version 65990 (0.0007) +[2023-10-09 14:49:00,513][86121] Updated weights for policy 0, policy_version 66000 (0.0008) +[2023-10-09 14:49:00,879][86121] Updated weights for policy 0, policy_version 66010 (0.0009) +[2023-10-09 14:49:01,932][86122] Updated weights for policy 1, policy_version 66280 (0.0010) +[2023-10-09 14:49:02,296][86122] Updated weights for policy 1, policy_version 66290 (0.0008) +[2023-10-09 14:49:02,665][86122] Updated weights for policy 1, policy_version 66300 (0.0008) +[2023-10-09 14:49:03,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135495680. Throughput: 0: 1815.4, 1: 1840.3. Samples: 33878614. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:49:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:04,675][86121] Updated weights for policy 0, policy_version 66020 (0.0009) +[2023-10-09 14:49:05,053][86121] Updated weights for policy 0, policy_version 66030 (0.0009) +[2023-10-09 14:49:05,421][86121] Updated weights for policy 0, policy_version 66040 (0.0009) +[2023-10-09 14:49:06,501][86122] Updated weights for policy 1, policy_version 66310 (0.0008) +[2023-10-09 14:49:06,863][86122] Updated weights for policy 1, policy_version 66320 (0.0009) +[2023-10-09 14:49:07,220][86122] Updated weights for policy 1, policy_version 66330 (0.0009) +[2023-10-09 14:49:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135561216. Throughput: 0: 1822.3, 1: 1829.4. Samples: 33900046. Policy #0 lag: (min: 31.0, avg: 31.8, max: 51.0) +[2023-10-09 14:49:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:09,095][86121] Updated weights for policy 0, policy_version 66050 (0.0009) +[2023-10-09 14:49:09,465][86121] Updated weights for policy 0, policy_version 66060 (0.0009) +[2023-10-09 14:49:09,832][86121] Updated weights for policy 0, policy_version 66070 (0.0007) +[2023-10-09 14:49:10,191][86121] Updated weights for policy 0, policy_version 66080 (0.0008) +[2023-10-09 14:49:10,962][86122] Updated weights for policy 1, policy_version 66340 (0.0008) +[2023-10-09 14:49:11,327][86122] Updated weights for policy 1, policy_version 66350 (0.0007) +[2023-10-09 14:49:11,694][86122] Updated weights for policy 1, policy_version 66360 (0.0008) +[2023-10-09 14:49:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135626752. Throughput: 0: 1825.9, 1: 1827.1. Samples: 33911352. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:13,707][86121] Updated weights for policy 0, policy_version 66090 (0.0008) +[2023-10-09 14:49:14,074][86121] Updated weights for policy 0, policy_version 66100 (0.0011) +[2023-10-09 14:49:14,438][86121] Updated weights for policy 0, policy_version 66110 (0.0008) +[2023-10-09 14:49:15,456][86122] Updated weights for policy 1, policy_version 66370 (0.0009) +[2023-10-09 14:49:15,803][86122] Updated weights for policy 1, policy_version 66380 (0.0010) +[2023-10-09 14:49:16,163][86122] Updated weights for policy 1, policy_version 66390 (0.0008) +[2023-10-09 14:49:16,530][86122] Updated weights for policy 1, policy_version 66400 (0.0010) +[2023-10-09 14:49:18,139][86121] Updated weights for policy 0, policy_version 66120 (0.0011) +[2023-10-09 14:49:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 135692288. Throughput: 0: 1829.1, 1: 1828.9. Samples: 33933010. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:18,492][86121] Updated weights for policy 0, policy_version 66130 (0.0009) +[2023-10-09 14:49:18,865][86121] Updated weights for policy 0, policy_version 66140 (0.0010) +[2023-10-09 14:49:20,397][86122] Updated weights for policy 1, policy_version 66410 (0.0009) +[2023-10-09 14:49:20,752][86122] Updated weights for policy 1, policy_version 66420 (0.0009) +[2023-10-09 14:49:21,117][86122] Updated weights for policy 1, policy_version 66430 (0.0012) +[2023-10-09 14:49:22,754][86121] Updated weights for policy 0, policy_version 66150 (0.0007) +[2023-10-09 14:49:23,107][86121] Updated weights for policy 0, policy_version 66160 (0.0007) +[2023-10-09 14:49:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 135757824. Throughput: 0: 1828.3, 1: 1825.1. Samples: 33955162. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000066432_68026368.pth... +[2023-10-09 14:49:23,439][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000064736_66289664.pth +[2023-10-09 14:49:23,478][86121] Updated weights for policy 0, policy_version 66170 (0.0007) +[2023-10-09 14:49:23,691][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000066176_67764224.pth... +[2023-10-09 14:49:23,729][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000064448_65994752.pth +[2023-10-09 14:49:24,797][86122] Updated weights for policy 1, policy_version 66440 (0.0010) +[2023-10-09 14:49:25,151][86122] Updated weights for policy 1, policy_version 66450 (0.0009) +[2023-10-09 14:49:25,504][86122] Updated weights for policy 1, policy_version 66460 (0.0010) +[2023-10-09 14:49:27,145][86121] Updated weights for policy 0, policy_version 66180 (0.0008) +[2023-10-09 14:49:27,512][86121] Updated weights for policy 0, policy_version 66190 (0.0008) +[2023-10-09 14:49:27,878][86121] Updated weights for policy 0, policy_version 66200 (0.0009) +[2023-10-09 14:49:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135856128. Throughput: 0: 1826.7, 1: 1823.3. Samples: 33965508. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:29,291][86122] Updated weights for policy 1, policy_version 66470 (0.0010) +[2023-10-09 14:49:29,652][86122] Updated weights for policy 1, policy_version 66480 (0.0008) +[2023-10-09 14:49:30,015][86122] Updated weights for policy 1, policy_version 66490 (0.0007) +[2023-10-09 14:49:31,568][86121] Updated weights for policy 0, policy_version 66210 (0.0008) +[2023-10-09 14:49:31,928][86121] Updated weights for policy 0, policy_version 66220 (0.0008) +[2023-10-09 14:49:32,302][86121] Updated weights for policy 0, policy_version 66230 (0.0007) +[2023-10-09 14:49:32,665][86121] Updated weights for policy 0, policy_version 66240 (0.0009) +[2023-10-09 14:49:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 135921664. Throughput: 0: 1823.5, 1: 1820.7. Samples: 33987804. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:33,552][86122] Updated weights for policy 1, policy_version 66500 (0.0009) +[2023-10-09 14:49:33,913][86122] Updated weights for policy 1, policy_version 66510 (0.0011) +[2023-10-09 14:49:34,271][86122] Updated weights for policy 1, policy_version 66520 (0.0008) +[2023-10-09 14:49:36,417][86121] Updated weights for policy 0, policy_version 66250 (0.0010) +[2023-10-09 14:49:36,784][86121] Updated weights for policy 0, policy_version 66260 (0.0008) +[2023-10-09 14:49:37,150][86121] Updated weights for policy 0, policy_version 66270 (0.0009) +[2023-10-09 14:49:37,905][86122] Updated weights for policy 1, policy_version 66530 (0.0008) +[2023-10-09 14:49:38,270][86122] Updated weights for policy 1, policy_version 66540 (0.0008) +[2023-10-09 14:49:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 135987200. Throughput: 0: 1832.8, 1: 1817.4. Samples: 34009832. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:38,630][86122] Updated weights for policy 1, policy_version 66550 (0.0007) +[2023-10-09 14:49:38,991][86122] Updated weights for policy 1, policy_version 66560 (0.0007) +[2023-10-09 14:49:40,694][86121] Updated weights for policy 0, policy_version 66280 (0.0009) +[2023-10-09 14:49:41,060][86121] Updated weights for policy 0, policy_version 66290 (0.0010) +[2023-10-09 14:49:41,424][86121] Updated weights for policy 0, policy_version 66300 (0.0008) +[2023-10-09 14:49:42,630][86122] Updated weights for policy 1, policy_version 66570 (0.0007) +[2023-10-09 14:49:42,994][86122] Updated weights for policy 1, policy_version 66580 (0.0007) +[2023-10-09 14:49:43,360][86122] Updated weights for policy 1, policy_version 66590 (0.0009) +[2023-10-09 14:49:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136052736. Throughput: 0: 1827.6, 1: 1813.7. Samples: 34020654. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:45,144][86121] Updated weights for policy 0, policy_version 66310 (0.0010) +[2023-10-09 14:49:45,505][86121] Updated weights for policy 0, policy_version 66320 (0.0010) +[2023-10-09 14:49:45,867][86121] Updated weights for policy 0, policy_version 66330 (0.0007) +[2023-10-09 14:49:47,131][86122] Updated weights for policy 1, policy_version 66600 (0.0008) +[2023-10-09 14:49:47,502][86122] Updated weights for policy 1, policy_version 66610 (0.0008) +[2023-10-09 14:49:47,856][86122] Updated weights for policy 1, policy_version 66620 (0.0007) +[2023-10-09 14:49:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136151040. Throughput: 0: 1832.8, 1: 1808.1. Samples: 34042452. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:49,536][86121] Updated weights for policy 0, policy_version 66340 (0.0007) +[2023-10-09 14:49:49,913][86121] Updated weights for policy 0, policy_version 66350 (0.0008) +[2023-10-09 14:49:50,278][86121] Updated weights for policy 0, policy_version 66360 (0.0008) +[2023-10-09 14:49:51,748][86122] Updated weights for policy 1, policy_version 66630 (0.0010) +[2023-10-09 14:49:52,109][86122] Updated weights for policy 1, policy_version 66640 (0.0009) +[2023-10-09 14:49:52,473][86122] Updated weights for policy 1, policy_version 66650 (0.0007) +[2023-10-09 14:49:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136216576. Throughput: 0: 1827.8, 1: 1805.5. Samples: 34063546. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 14:49:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:53,979][86121] Updated weights for policy 0, policy_version 66370 (0.0010) +[2023-10-09 14:49:54,339][86121] Updated weights for policy 0, policy_version 66380 (0.0008) +[2023-10-09 14:49:54,708][86121] Updated weights for policy 0, policy_version 66390 (0.0007) +[2023-10-09 14:49:55,084][86121] Updated weights for policy 0, policy_version 66400 (0.0009) +[2023-10-09 14:49:56,255][86122] Updated weights for policy 1, policy_version 66660 (0.0008) +[2023-10-09 14:49:56,623][86122] Updated weights for policy 1, policy_version 66670 (0.0008) +[2023-10-09 14:49:56,987][86122] Updated weights for policy 1, policy_version 66680 (0.0008) +[2023-10-09 14:49:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136282112. Throughput: 0: 1826.3, 1: 1810.3. Samples: 34074998. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:49:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:49:58,629][86121] Updated weights for policy 0, policy_version 66410 (0.0009) +[2023-10-09 14:49:58,996][86121] Updated weights for policy 0, policy_version 66420 (0.0008) +[2023-10-09 14:49:59,367][86121] Updated weights for policy 0, policy_version 66430 (0.0008) +[2023-10-09 14:50:00,603][86122] Updated weights for policy 1, policy_version 66690 (0.0008) +[2023-10-09 14:50:00,968][86122] Updated weights for policy 1, policy_version 66700 (0.0010) +[2023-10-09 14:50:01,334][86122] Updated weights for policy 1, policy_version 66710 (0.0011) +[2023-10-09 14:50:01,701][86122] Updated weights for policy 1, policy_version 66720 (0.0009) +[2023-10-09 14:50:03,083][86121] Updated weights for policy 0, policy_version 66440 (0.0008) +[2023-10-09 14:50:03,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 136347648. Throughput: 0: 1827.1, 1: 1808.7. Samples: 34096622. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:03,448][86121] Updated weights for policy 0, policy_version 66450 (0.0009) +[2023-10-09 14:50:03,823][86121] Updated weights for policy 0, policy_version 66460 (0.0008) +[2023-10-09 14:50:05,574][86122] Updated weights for policy 1, policy_version 66730 (0.0008) +[2023-10-09 14:50:05,936][86122] Updated weights for policy 1, policy_version 66740 (0.0009) +[2023-10-09 14:50:06,302][86122] Updated weights for policy 1, policy_version 66750 (0.0008) +[2023-10-09 14:50:07,484][86121] Updated weights for policy 0, policy_version 66470 (0.0007) +[2023-10-09 14:50:07,856][86121] Updated weights for policy 0, policy_version 66480 (0.0008) +[2023-10-09 14:50:08,219][86121] Updated weights for policy 0, policy_version 66490 (0.0007) +[2023-10-09 14:50:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136413184. Throughput: 0: 1821.6, 1: 1807.2. Samples: 34118454. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:10,003][86122] Updated weights for policy 1, policy_version 66760 (0.0009) +[2023-10-09 14:50:10,374][86122] Updated weights for policy 1, policy_version 66770 (0.0009) +[2023-10-09 14:50:10,733][86122] Updated weights for policy 1, policy_version 66780 (0.0011) +[2023-10-09 14:50:12,040][86121] Updated weights for policy 0, policy_version 66500 (0.0009) +[2023-10-09 14:50:12,412][86121] Updated weights for policy 0, policy_version 66510 (0.0009) +[2023-10-09 14:50:12,766][86121] Updated weights for policy 0, policy_version 66520 (0.0007) +[2023-10-09 14:50:13,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136511488. Throughput: 0: 1827.8, 1: 1811.1. Samples: 34129262. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:14,318][86122] Updated weights for policy 1, policy_version 66790 (0.0010) +[2023-10-09 14:50:14,681][86122] Updated weights for policy 1, policy_version 66800 (0.0009) +[2023-10-09 14:50:15,045][86122] Updated weights for policy 1, policy_version 66810 (0.0007) +[2023-10-09 14:50:16,400][86121] Updated weights for policy 0, policy_version 66530 (0.0007) +[2023-10-09 14:50:16,766][86121] Updated weights for policy 0, policy_version 66540 (0.0008) +[2023-10-09 14:50:17,131][86121] Updated weights for policy 0, policy_version 66550 (0.0009) +[2023-10-09 14:50:17,497][86121] Updated weights for policy 0, policy_version 66560 (0.0009) +[2023-10-09 14:50:18,398][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 136577024. Throughput: 0: 1820.3, 1: 1816.9. Samples: 34151478. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:18,399][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:18,783][86122] Updated weights for policy 1, policy_version 66820 (0.0008) +[2023-10-09 14:50:19,152][86122] Updated weights for policy 1, policy_version 66830 (0.0008) +[2023-10-09 14:50:19,517][86122] Updated weights for policy 1, policy_version 66840 (0.0009) +[2023-10-09 14:50:21,253][86121] Updated weights for policy 0, policy_version 66570 (0.0007) +[2023-10-09 14:50:21,611][86121] Updated weights for policy 0, policy_version 66580 (0.0010) +[2023-10-09 14:50:21,984][86121] Updated weights for policy 0, policy_version 66590 (0.0008) +[2023-10-09 14:50:23,234][86122] Updated weights for policy 1, policy_version 66850 (0.0010) +[2023-10-09 14:50:23,398][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 136642560. Throughput: 0: 1822.2, 1: 1810.1. Samples: 34173288. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:23,399][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:23,598][86122] Updated weights for policy 1, policy_version 66860 (0.0009) +[2023-10-09 14:50:23,964][86122] Updated weights for policy 1, policy_version 66870 (0.0010) +[2023-10-09 14:50:24,324][86122] Updated weights for policy 1, policy_version 66880 (0.0008) +[2023-10-09 14:50:25,712][86121] Updated weights for policy 0, policy_version 66600 (0.0009) +[2023-10-09 14:50:26,071][86121] Updated weights for policy 0, policy_version 66610 (0.0010) +[2023-10-09 14:50:26,453][86121] Updated weights for policy 0, policy_version 66620 (0.0009) +[2023-10-09 14:50:27,880][86122] Updated weights for policy 1, policy_version 66890 (0.0007) +[2023-10-09 14:50:28,250][86122] Updated weights for policy 1, policy_version 66900 (0.0008) +[2023-10-09 14:50:28,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 136708096. Throughput: 0: 1823.6, 1: 1813.5. Samples: 34184320. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:28,606][86122] Updated weights for policy 1, policy_version 66910 (0.0009) +[2023-10-09 14:50:30,105][86121] Updated weights for policy 0, policy_version 66630 (0.0010) +[2023-10-09 14:50:30,466][86121] Updated weights for policy 0, policy_version 66640 (0.0008) +[2023-10-09 14:50:30,831][86121] Updated weights for policy 0, policy_version 66650 (0.0010) +[2023-10-09 14:50:32,212][86122] Updated weights for policy 1, policy_version 66920 (0.0009) +[2023-10-09 14:50:32,575][86122] Updated weights for policy 1, policy_version 66930 (0.0008) +[2023-10-09 14:50:32,944][86122] Updated weights for policy 1, policy_version 66940 (0.0009) +[2023-10-09 14:50:33,397][85186] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 136806400. Throughput: 0: 1821.9, 1: 1824.8. Samples: 34206550. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:34,509][86121] Updated weights for policy 0, policy_version 66660 (0.0009) +[2023-10-09 14:50:34,893][86121] Updated weights for policy 0, policy_version 66670 (0.0008) +[2023-10-09 14:50:35,270][86121] Updated weights for policy 0, policy_version 66680 (0.0009) +[2023-10-09 14:50:36,599][86122] Updated weights for policy 1, policy_version 66950 (0.0008) +[2023-10-09 14:50:36,972][86122] Updated weights for policy 1, policy_version 66960 (0.0007) +[2023-10-09 14:50:37,324][86122] Updated weights for policy 1, policy_version 66970 (0.0008) +[2023-10-09 14:50:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136871936. Throughput: 0: 1825.6, 1: 1830.4. Samples: 34228064. Policy #0 lag: (min: 28.0, avg: 28.1, max: 36.0) +[2023-10-09 14:50:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:38,764][86121] Updated weights for policy 0, policy_version 66690 (0.0010) +[2023-10-09 14:50:39,135][86121] Updated weights for policy 0, policy_version 66700 (0.0010) +[2023-10-09 14:50:39,498][86121] Updated weights for policy 0, policy_version 66710 (0.0010) +[2023-10-09 14:50:39,863][86121] Updated weights for policy 0, policy_version 66720 (0.0008) +[2023-10-09 14:50:41,108][86122] Updated weights for policy 1, policy_version 66980 (0.0009) +[2023-10-09 14:50:41,465][86122] Updated weights for policy 1, policy_version 66990 (0.0007) +[2023-10-09 14:50:41,834][86122] Updated weights for policy 1, policy_version 67000 (0.0009) +[2023-10-09 14:50:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 136937472. Throughput: 0: 1829.6, 1: 1826.1. Samples: 34239504. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:50:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:43,554][86121] Updated weights for policy 0, policy_version 66730 (0.0010) +[2023-10-09 14:50:43,907][86121] Updated weights for policy 0, policy_version 66740 (0.0009) +[2023-10-09 14:50:44,275][86121] Updated weights for policy 0, policy_version 66750 (0.0009) +[2023-10-09 14:50:45,496][86122] Updated weights for policy 1, policy_version 67010 (0.0008) +[2023-10-09 14:50:45,857][86122] Updated weights for policy 1, policy_version 67020 (0.0008) +[2023-10-09 14:50:46,234][86122] Updated weights for policy 1, policy_version 67030 (0.0008) +[2023-10-09 14:50:46,584][86122] Updated weights for policy 1, policy_version 67040 (0.0007) +[2023-10-09 14:50:48,037][86121] Updated weights for policy 0, policy_version 66760 (0.0008) +[2023-10-09 14:50:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137003008. Throughput: 0: 1823.1, 1: 1823.6. Samples: 34260722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:50:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:50:48,408][86121] Updated weights for policy 0, policy_version 66770 (0.0009) +[2023-10-09 14:50:48,776][86121] Updated weights for policy 0, policy_version 66780 (0.0008) +[2023-10-09 14:50:50,421][86122] Updated weights for policy 1, policy_version 67050 (0.0007) +[2023-10-09 14:50:50,791][86122] Updated weights for policy 1, policy_version 67060 (0.0007) +[2023-10-09 14:50:51,155][86122] Updated weights for policy 1, policy_version 67070 (0.0007) +[2023-10-09 14:50:52,498][86121] Updated weights for policy 0, policy_version 66790 (0.0008) +[2023-10-09 14:50:52,860][86121] Updated weights for policy 0, policy_version 66800 (0.0008) +[2023-10-09 14:50:53,226][86121] Updated weights for policy 0, policy_version 66810 (0.0008) +[2023-10-09 14:50:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137068544. Throughput: 0: 1825.7, 1: 1821.5. Samples: 34282578. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:50:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:50:54,920][86122] Updated weights for policy 1, policy_version 67080 (0.0008) +[2023-10-09 14:50:55,288][86122] Updated weights for policy 1, policy_version 67090 (0.0011) +[2023-10-09 14:50:55,654][86122] Updated weights for policy 1, policy_version 67100 (0.0009) +[2023-10-09 14:50:57,001][86121] Updated weights for policy 0, policy_version 66820 (0.0008) +[2023-10-09 14:50:57,367][86121] Updated weights for policy 0, policy_version 66830 (0.0007) +[2023-10-09 14:50:57,730][86121] Updated weights for policy 0, policy_version 66840 (0.0007) +[2023-10-09 14:50:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 137166848. Throughput: 0: 1824.1, 1: 1819.5. Samples: 34293222. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:50:58,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:50:59,207][86122] Updated weights for policy 1, policy_version 67110 (0.0007) +[2023-10-09 14:50:59,574][86122] Updated weights for policy 1, policy_version 67120 (0.0009) +[2023-10-09 14:50:59,933][86122] Updated weights for policy 1, policy_version 67130 (0.0007) +[2023-10-09 14:51:01,334][86121] Updated weights for policy 0, policy_version 66850 (0.0009) +[2023-10-09 14:51:01,700][86121] Updated weights for policy 0, policy_version 66860 (0.0010) +[2023-10-09 14:51:02,070][86121] Updated weights for policy 0, policy_version 66870 (0.0009) +[2023-10-09 14:51:02,437][86121] Updated weights for policy 0, policy_version 66880 (0.0007) +[2023-10-09 14:51:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137232384. Throughput: 0: 1824.0, 1: 1816.7. Samples: 34315310. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:51:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:03,705][86122] Updated weights for policy 1, policy_version 67140 (0.0008) +[2023-10-09 14:51:04,070][86122] Updated weights for policy 1, policy_version 67150 (0.0007) +[2023-10-09 14:51:04,427][86122] Updated weights for policy 1, policy_version 67160 (0.0008) +[2023-10-09 14:51:05,981][86121] Updated weights for policy 0, policy_version 66890 (0.0010) +[2023-10-09 14:51:06,352][86121] Updated weights for policy 0, policy_version 66900 (0.0008) +[2023-10-09 14:51:06,721][86121] Updated weights for policy 0, policy_version 66910 (0.0007) +[2023-10-09 14:51:08,201][86122] Updated weights for policy 1, policy_version 67170 (0.0008) +[2023-10-09 14:51:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 137297920. Throughput: 0: 1833.3, 1: 1819.9. Samples: 34337680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:51:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:08,568][86122] Updated weights for policy 1, policy_version 67180 (0.0007) +[2023-10-09 14:51:08,927][86122] Updated weights for policy 1, policy_version 67190 (0.0010) +[2023-10-09 14:51:09,283][86122] Updated weights for policy 1, policy_version 67200 (0.0012) +[2023-10-09 14:51:10,260][86121] Updated weights for policy 0, policy_version 66920 (0.0007) +[2023-10-09 14:51:10,627][86121] Updated weights for policy 0, policy_version 66930 (0.0010) +[2023-10-09 14:51:10,984][86121] Updated weights for policy 0, policy_version 66940 (0.0011) +[2023-10-09 14:51:13,036][86122] Updated weights for policy 1, policy_version 67210 (0.0007) +[2023-10-09 14:51:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 137363456. Throughput: 0: 1822.5, 1: 1817.1. Samples: 34348102. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:51:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:13,410][86122] Updated weights for policy 1, policy_version 67220 (0.0010) +[2023-10-09 14:51:13,760][86122] Updated weights for policy 1, policy_version 67230 (0.0008) +[2023-10-09 14:51:14,577][86121] Updated weights for policy 0, policy_version 66950 (0.0010) +[2023-10-09 14:51:14,941][86121] Updated weights for policy 0, policy_version 66960 (0.0008) +[2023-10-09 14:51:15,310][86121] Updated weights for policy 0, policy_version 66970 (0.0007) +[2023-10-09 14:51:17,358][86122] Updated weights for policy 1, policy_version 67240 (0.0008) +[2023-10-09 14:51:17,729][86122] Updated weights for policy 1, policy_version 67250 (0.0008) +[2023-10-09 14:51:18,084][86122] Updated weights for policy 1, policy_version 67260 (0.0008) +[2023-10-09 14:51:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137461760. Throughput: 0: 1833.8, 1: 1817.4. Samples: 34370852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:51:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:18,964][86121] Updated weights for policy 0, policy_version 66980 (0.0008) +[2023-10-09 14:51:19,333][86121] Updated weights for policy 0, policy_version 66990 (0.0009) +[2023-10-09 14:51:19,701][86121] Updated weights for policy 0, policy_version 67000 (0.0008) +[2023-10-09 14:51:21,663][86122] Updated weights for policy 1, policy_version 67270 (0.0008) +[2023-10-09 14:51:22,014][86122] Updated weights for policy 1, policy_version 67280 (0.0010) +[2023-10-09 14:51:22,382][86122] Updated weights for policy 1, policy_version 67290 (0.0009) +[2023-10-09 14:51:23,357][86121] Updated weights for policy 0, policy_version 67010 (0.0008) +[2023-10-09 14:51:23,398][85186] Fps is (10 sec: 16383.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137527296. Throughput: 0: 1838.6, 1: 1820.5. Samples: 34392722. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:51:23,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000067296_68911104.pth... +[2023-10-09 14:51:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000065600_67174400.pth +[2023-10-09 14:51:23,748][86121] Updated weights for policy 0, policy_version 67020 (0.0009) +[2023-10-09 14:51:24,108][86121] Updated weights for policy 0, policy_version 67030 (0.0010) +[2023-10-09 14:51:24,482][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000067040_68648960.pth... +[2023-10-09 14:51:24,483][86121] Updated weights for policy 0, policy_version 67040 (0.0011) +[2023-10-09 14:51:24,510][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000065312_66879488.pth +[2023-10-09 14:51:25,944][86122] Updated weights for policy 1, policy_version 67300 (0.0009) +[2023-10-09 14:51:26,302][86122] Updated weights for policy 1, policy_version 67310 (0.0008) +[2023-10-09 14:51:26,661][86122] Updated weights for policy 1, policy_version 67320 (0.0009) +[2023-10-09 14:51:28,055][86121] Updated weights for policy 0, policy_version 67050 (0.0007) +[2023-10-09 14:51:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137592832. Throughput: 0: 1833.5, 1: 1826.5. Samples: 34404202. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:51:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:28,415][86121] Updated weights for policy 0, policy_version 67060 (0.0007) +[2023-10-09 14:51:28,779][86121] Updated weights for policy 0, policy_version 67070 (0.0007) +[2023-10-09 14:51:30,547][86122] Updated weights for policy 1, policy_version 67330 (0.0009) +[2023-10-09 14:51:30,918][86122] Updated weights for policy 1, policy_version 67340 (0.0009) +[2023-10-09 14:51:31,276][86122] Updated weights for policy 1, policy_version 67350 (0.0010) +[2023-10-09 14:51:31,646][86122] Updated weights for policy 1, policy_version 67360 (0.0009) +[2023-10-09 14:51:32,522][86121] Updated weights for policy 0, policy_version 67080 (0.0009) +[2023-10-09 14:51:32,884][86121] Updated weights for policy 0, policy_version 67090 (0.0010) +[2023-10-09 14:51:33,250][86121] Updated weights for policy 0, policy_version 67100 (0.0007) +[2023-10-09 14:51:33,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137691136. Throughput: 0: 1839.7, 1: 1823.4. Samples: 34425562. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:51:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:35,369][86122] Updated weights for policy 1, policy_version 67370 (0.0008) +[2023-10-09 14:51:35,727][86122] Updated weights for policy 1, policy_version 67380 (0.0008) +[2023-10-09 14:51:36,093][86122] Updated weights for policy 1, policy_version 67390 (0.0010) +[2023-10-09 14:51:36,909][86121] Updated weights for policy 0, policy_version 67110 (0.0009) +[2023-10-09 14:51:37,279][86121] Updated weights for policy 0, policy_version 67120 (0.0011) +[2023-10-09 14:51:37,641][86121] Updated weights for policy 0, policy_version 67130 (0.0008) +[2023-10-09 14:51:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137756672. Throughput: 0: 1819.9, 1: 1838.2. Samples: 34447194. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:51:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:39,669][86122] Updated weights for policy 1, policy_version 67400 (0.0009) +[2023-10-09 14:51:40,034][86122] Updated weights for policy 1, policy_version 67410 (0.0008) +[2023-10-09 14:51:40,405][86122] Updated weights for policy 1, policy_version 67420 (0.0008) +[2023-10-09 14:51:41,305][86121] Updated weights for policy 0, policy_version 67140 (0.0008) +[2023-10-09 14:51:41,671][86121] Updated weights for policy 0, policy_version 67150 (0.0007) +[2023-10-09 14:51:42,034][86121] Updated weights for policy 0, policy_version 67160 (0.0007) +[2023-10-09 14:51:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 137822208. Throughput: 0: 1836.4, 1: 1836.7. Samples: 34458512. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:51:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:43,926][86122] Updated weights for policy 1, policy_version 67430 (0.0008) +[2023-10-09 14:51:44,294][86122] Updated weights for policy 1, policy_version 67440 (0.0009) +[2023-10-09 14:51:44,651][86122] Updated weights for policy 1, policy_version 67450 (0.0010) +[2023-10-09 14:51:45,666][86121] Updated weights for policy 0, policy_version 67170 (0.0007) +[2023-10-09 14:51:46,034][86121] Updated weights for policy 0, policy_version 67180 (0.0007) +[2023-10-09 14:51:46,395][86121] Updated weights for policy 0, policy_version 67190 (0.0007) +[2023-10-09 14:51:46,759][86121] Updated weights for policy 0, policy_version 67200 (0.0007) +[2023-10-09 14:51:48,281][86122] Updated weights for policy 1, policy_version 67460 (0.0008) +[2023-10-09 14:51:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137887744. Throughput: 0: 1826.7, 1: 1842.1. Samples: 34480406. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:51:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:48,640][86122] Updated weights for policy 1, policy_version 67470 (0.0007) +[2023-10-09 14:51:49,005][86122] Updated weights for policy 1, policy_version 67480 (0.0008) +[2023-10-09 14:51:50,519][86121] Updated weights for policy 0, policy_version 67210 (0.0008) +[2023-10-09 14:51:50,872][86121] Updated weights for policy 0, policy_version 67220 (0.0009) +[2023-10-09 14:51:51,234][86121] Updated weights for policy 0, policy_version 67230 (0.0008) +[2023-10-09 14:51:52,630][86122] Updated weights for policy 1, policy_version 67490 (0.0008) +[2023-10-09 14:51:52,992][86122] Updated weights for policy 1, policy_version 67500 (0.0008) +[2023-10-09 14:51:53,358][86122] Updated weights for policy 1, policy_version 67510 (0.0010) +[2023-10-09 14:51:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 137953280. Throughput: 0: 1836.1, 1: 1838.7. Samples: 34503046. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:51:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:53,721][86122] Updated weights for policy 1, policy_version 67520 (0.0007) +[2023-10-09 14:51:54,947][86121] Updated weights for policy 0, policy_version 67240 (0.0009) +[2023-10-09 14:51:55,328][86121] Updated weights for policy 0, policy_version 67250 (0.0011) +[2023-10-09 14:51:55,693][86121] Updated weights for policy 0, policy_version 67260 (0.0009) +[2023-10-09 14:51:57,313][86122] Updated weights for policy 1, policy_version 67530 (0.0007) +[2023-10-09 14:51:57,669][86122] Updated weights for policy 1, policy_version 67540 (0.0008) +[2023-10-09 14:51:58,028][86122] Updated weights for policy 1, policy_version 67550 (0.0011) +[2023-10-09 14:51:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 138051584. Throughput: 0: 1821.8, 1: 1852.2. Samples: 34513432. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:51:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:51:59,506][86121] Updated weights for policy 0, policy_version 67270 (0.0007) +[2023-10-09 14:51:59,874][86121] Updated weights for policy 0, policy_version 67280 (0.0009) +[2023-10-09 14:52:00,247][86121] Updated weights for policy 0, policy_version 67290 (0.0009) +[2023-10-09 14:52:01,819][86122] Updated weights for policy 1, policy_version 67560 (0.0011) +[2023-10-09 14:52:02,188][86122] Updated weights for policy 1, policy_version 67570 (0.0010) +[2023-10-09 14:52:02,547][86122] Updated weights for policy 1, policy_version 67580 (0.0008) +[2023-10-09 14:52:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138117120. Throughput: 0: 1830.1, 1: 1839.2. Samples: 34535974. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:52:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:03,918][86121] Updated weights for policy 0, policy_version 67300 (0.0009) +[2023-10-09 14:52:04,284][86121] Updated weights for policy 0, policy_version 67310 (0.0009) +[2023-10-09 14:52:04,653][86121] Updated weights for policy 0, policy_version 67320 (0.0008) +[2023-10-09 14:52:06,267][86122] Updated weights for policy 1, policy_version 67590 (0.0008) +[2023-10-09 14:52:06,624][86122] Updated weights for policy 1, policy_version 67600 (0.0010) +[2023-10-09 14:52:06,992][86122] Updated weights for policy 1, policy_version 67610 (0.0010) +[2023-10-09 14:52:08,145][86121] Updated weights for policy 0, policy_version 67330 (0.0009) +[2023-10-09 14:52:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138182656. Throughput: 0: 1830.4, 1: 1843.8. Samples: 34558060. Policy #0 lag: (min: 10.0, avg: 20.8, max: 42.0) +[2023-10-09 14:52:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:08,517][86121] Updated weights for policy 0, policy_version 67340 (0.0007) +[2023-10-09 14:52:08,878][86121] Updated weights for policy 0, policy_version 67350 (0.0008) +[2023-10-09 14:52:09,246][86121] Updated weights for policy 0, policy_version 67360 (0.0008) +[2023-10-09 14:52:10,541][86122] Updated weights for policy 1, policy_version 67620 (0.0008) +[2023-10-09 14:52:10,908][86122] Updated weights for policy 1, policy_version 67630 (0.0007) +[2023-10-09 14:52:11,269][86122] Updated weights for policy 1, policy_version 67640 (0.0008) +[2023-10-09 14:52:12,997][86121] Updated weights for policy 0, policy_version 67370 (0.0008) +[2023-10-09 14:52:13,371][86121] Updated weights for policy 0, policy_version 67380 (0.0007) +[2023-10-09 14:52:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138248192. Throughput: 0: 1830.4, 1: 1828.8. Samples: 34568866. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:13,736][86121] Updated weights for policy 0, policy_version 67390 (0.0008) +[2023-10-09 14:52:14,851][86122] Updated weights for policy 1, policy_version 67650 (0.0009) +[2023-10-09 14:52:15,217][86122] Updated weights for policy 1, policy_version 67660 (0.0008) +[2023-10-09 14:52:15,573][86122] Updated weights for policy 1, policy_version 67670 (0.0008) +[2023-10-09 14:52:15,938][86122] Updated weights for policy 1, policy_version 67680 (0.0007) +[2023-10-09 14:52:17,361][86121] Updated weights for policy 0, policy_version 67400 (0.0011) +[2023-10-09 14:52:17,725][86121] Updated weights for policy 0, policy_version 67410 (0.0008) +[2023-10-09 14:52:18,087][86121] Updated weights for policy 0, policy_version 67420 (0.0008) +[2023-10-09 14:52:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 138346496. Throughput: 0: 1829.7, 1: 1851.2. Samples: 34591204. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:19,636][86122] Updated weights for policy 1, policy_version 67690 (0.0007) +[2023-10-09 14:52:20,007][86122] Updated weights for policy 1, policy_version 67700 (0.0007) +[2023-10-09 14:52:20,367][86122] Updated weights for policy 1, policy_version 67710 (0.0007) +[2023-10-09 14:52:21,725][86121] Updated weights for policy 0, policy_version 67430 (0.0008) +[2023-10-09 14:52:22,091][86121] Updated weights for policy 0, policy_version 67440 (0.0008) +[2023-10-09 14:52:22,460][86121] Updated weights for policy 0, policy_version 67450 (0.0007) +[2023-10-09 14:52:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138412032. Throughput: 0: 1829.8, 1: 1847.4. Samples: 34612670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:23,399][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:23,993][86122] Updated weights for policy 1, policy_version 67720 (0.0008) +[2023-10-09 14:52:24,355][86122] Updated weights for policy 1, policy_version 67730 (0.0007) +[2023-10-09 14:52:24,722][86122] Updated weights for policy 1, policy_version 67740 (0.0008) +[2023-10-09 14:52:26,122][86121] Updated weights for policy 0, policy_version 67460 (0.0009) +[2023-10-09 14:52:26,481][86121] Updated weights for policy 0, policy_version 67470 (0.0007) +[2023-10-09 14:52:26,847][86121] Updated weights for policy 0, policy_version 67480 (0.0011) +[2023-10-09 14:52:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138477568. Throughput: 0: 1831.7, 1: 1850.8. Samples: 34624224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:28,423][86122] Updated weights for policy 1, policy_version 67750 (0.0008) +[2023-10-09 14:52:28,790][86122] Updated weights for policy 1, policy_version 67760 (0.0008) +[2023-10-09 14:52:29,162][86122] Updated weights for policy 1, policy_version 67770 (0.0007) +[2023-10-09 14:52:30,470][86121] Updated weights for policy 0, policy_version 67490 (0.0007) +[2023-10-09 14:52:30,843][86121] Updated weights for policy 0, policy_version 67500 (0.0009) +[2023-10-09 14:52:31,209][86121] Updated weights for policy 0, policy_version 67510 (0.0008) +[2023-10-09 14:52:31,581][86121] Updated weights for policy 0, policy_version 67520 (0.0007) +[2023-10-09 14:52:32,693][86122] Updated weights for policy 1, policy_version 67780 (0.0007) +[2023-10-09 14:52:33,048][86122] Updated weights for policy 1, policy_version 67790 (0.0008) +[2023-10-09 14:52:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 138543104. Throughput: 0: 1825.5, 1: 1851.5. Samples: 34645868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:33,418][86122] Updated weights for policy 1, policy_version 67800 (0.0007) +[2023-10-09 14:52:35,258][86121] Updated weights for policy 0, policy_version 67530 (0.0008) +[2023-10-09 14:52:35,633][86121] Updated weights for policy 0, policy_version 67540 (0.0012) +[2023-10-09 14:52:35,995][86121] Updated weights for policy 0, policy_version 67550 (0.0010) +[2023-10-09 14:52:37,034][86122] Updated weights for policy 1, policy_version 67810 (0.0008) +[2023-10-09 14:52:37,383][86122] Updated weights for policy 1, policy_version 67820 (0.0008) +[2023-10-09 14:52:37,744][86122] Updated weights for policy 1, policy_version 67830 (0.0007) +[2023-10-09 14:52:38,106][86122] Updated weights for policy 1, policy_version 67840 (0.0008) +[2023-10-09 14:52:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138641408. Throughput: 0: 1828.9, 1: 1831.2. Samples: 34667754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:39,828][86121] Updated weights for policy 0, policy_version 67560 (0.0009) +[2023-10-09 14:52:40,195][86121] Updated weights for policy 0, policy_version 67570 (0.0008) +[2023-10-09 14:52:40,559][86121] Updated weights for policy 0, policy_version 67580 (0.0008) +[2023-10-09 14:52:41,804][86122] Updated weights for policy 1, policy_version 67850 (0.0010) +[2023-10-09 14:52:42,168][86122] Updated weights for policy 1, policy_version 67860 (0.0009) +[2023-10-09 14:52:42,528][86122] Updated weights for policy 1, policy_version 67870 (0.0007) +[2023-10-09 14:52:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 138706944. Throughput: 0: 1828.0, 1: 1846.8. Samples: 34678800. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:44,371][86121] Updated weights for policy 0, policy_version 67590 (0.0007) +[2023-10-09 14:52:44,738][86121] Updated weights for policy 0, policy_version 67600 (0.0008) +[2023-10-09 14:52:45,104][86121] Updated weights for policy 0, policy_version 67610 (0.0008) +[2023-10-09 14:52:46,264][86122] Updated weights for policy 1, policy_version 67880 (0.0007) +[2023-10-09 14:52:46,642][86122] Updated weights for policy 1, policy_version 67890 (0.0007) +[2023-10-09 14:52:46,999][86122] Updated weights for policy 1, policy_version 67900 (0.0007) +[2023-10-09 14:52:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 138772480. Throughput: 0: 1829.7, 1: 1831.1. Samples: 34700708. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:48,713][86121] Updated weights for policy 0, policy_version 67620 (0.0009) +[2023-10-09 14:52:49,078][86121] Updated weights for policy 0, policy_version 67630 (0.0008) +[2023-10-09 14:52:49,444][86121] Updated weights for policy 0, policy_version 67640 (0.0008) +[2023-10-09 14:52:50,669][86122] Updated weights for policy 1, policy_version 67910 (0.0008) +[2023-10-09 14:52:51,028][86122] Updated weights for policy 1, policy_version 67920 (0.0011) +[2023-10-09 14:52:51,388][86122] Updated weights for policy 1, policy_version 67930 (0.0007) +[2023-10-09 14:52:53,207][86121] Updated weights for policy 0, policy_version 67650 (0.0009) +[2023-10-09 14:52:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 138838016. Throughput: 0: 1819.7, 1: 1844.7. Samples: 34722956. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:53,620][86121] Updated weights for policy 0, policy_version 67660 (0.0007) +[2023-10-09 14:52:53,988][86121] Updated weights for policy 0, policy_version 67670 (0.0008) +[2023-10-09 14:52:54,357][86121] Updated weights for policy 0, policy_version 67680 (0.0009) +[2023-10-09 14:52:55,212][86122] Updated weights for policy 1, policy_version 67940 (0.0007) +[2023-10-09 14:52:55,567][86122] Updated weights for policy 1, policy_version 67950 (0.0008) +[2023-10-09 14:52:55,928][86122] Updated weights for policy 1, policy_version 67960 (0.0009) +[2023-10-09 14:52:58,010][86121] Updated weights for policy 0, policy_version 67690 (0.0009) +[2023-10-09 14:52:58,374][86121] Updated weights for policy 0, policy_version 67700 (0.0009) +[2023-10-09 14:52:58,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 138903552. Throughput: 0: 1818.9, 1: 1833.9. Samples: 34733244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:52:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:52:58,734][86121] Updated weights for policy 0, policy_version 67710 (0.0007) +[2023-10-09 14:52:59,741][86122] Updated weights for policy 1, policy_version 67970 (0.0008) +[2023-10-09 14:53:00,103][86122] Updated weights for policy 1, policy_version 67980 (0.0007) +[2023-10-09 14:53:00,463][86122] Updated weights for policy 1, policy_version 67990 (0.0009) +[2023-10-09 14:53:00,822][86122] Updated weights for policy 1, policy_version 68000 (0.0009) +[2023-10-09 14:53:02,481][86121] Updated weights for policy 0, policy_version 67720 (0.0009) +[2023-10-09 14:53:02,848][86121] Updated weights for policy 0, policy_version 67730 (0.0008) +[2023-10-09 14:53:03,214][86121] Updated weights for policy 0, policy_version 67740 (0.0007) +[2023-10-09 14:53:03,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139001856. Throughput: 0: 1817.2, 1: 1831.0. Samples: 34755376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:53:04,515][86122] Updated weights for policy 1, policy_version 68010 (0.0008) +[2023-10-09 14:53:04,887][86122] Updated weights for policy 1, policy_version 68020 (0.0008) +[2023-10-09 14:53:05,254][86122] Updated weights for policy 1, policy_version 68030 (0.0008) +[2023-10-09 14:53:06,934][86121] Updated weights for policy 0, policy_version 67750 (0.0008) +[2023-10-09 14:53:07,294][86121] Updated weights for policy 0, policy_version 67760 (0.0007) +[2023-10-09 14:53:07,676][86121] Updated weights for policy 0, policy_version 67770 (0.0007) +[2023-10-09 14:53:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139067392. Throughput: 0: 1821.3, 1: 1826.9. Samples: 34776838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:08,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:53:08,924][86122] Updated weights for policy 1, policy_version 68040 (0.0008) +[2023-10-09 14:53:09,286][86122] Updated weights for policy 1, policy_version 68050 (0.0010) +[2023-10-09 14:53:09,638][86122] Updated weights for policy 1, policy_version 68060 (0.0010) +[2023-10-09 14:53:11,359][86121] Updated weights for policy 0, policy_version 67780 (0.0008) +[2023-10-09 14:53:11,729][86121] Updated weights for policy 0, policy_version 67790 (0.0012) +[2023-10-09 14:53:12,100][86121] Updated weights for policy 0, policy_version 67800 (0.0008) +[2023-10-09 14:53:13,152][86122] Updated weights for policy 1, policy_version 68070 (0.0008) +[2023-10-09 14:53:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139132928. Throughput: 0: 1817.1, 1: 1824.9. Samples: 34788114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:13,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:53:13,520][86122] Updated weights for policy 1, policy_version 68080 (0.0008) +[2023-10-09 14:53:13,875][86122] Updated weights for policy 1, policy_version 68090 (0.0009) +[2023-10-09 14:53:15,745][86121] Updated weights for policy 0, policy_version 67810 (0.0009) +[2023-10-09 14:53:16,113][86121] Updated weights for policy 0, policy_version 67820 (0.0009) +[2023-10-09 14:53:16,482][86121] Updated weights for policy 0, policy_version 67830 (0.0009) +[2023-10-09 14:53:16,839][86121] Updated weights for policy 0, policy_version 67840 (0.0007) +[2023-10-09 14:53:17,527][86122] Updated weights for policy 1, policy_version 68100 (0.0007) +[2023-10-09 14:53:17,891][86122] Updated weights for policy 1, policy_version 68110 (0.0008) +[2023-10-09 14:53:18,257][86122] Updated weights for policy 1, policy_version 68120 (0.0010) +[2023-10-09 14:53:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 139198464. Throughput: 0: 1816.6, 1: 1823.5. Samples: 34809674. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:53:20,597][86121] Updated weights for policy 0, policy_version 67850 (0.0011) +[2023-10-09 14:53:20,963][86121] Updated weights for policy 0, policy_version 67860 (0.0009) +[2023-10-09 14:53:21,337][86121] Updated weights for policy 0, policy_version 67870 (0.0008) +[2023-10-09 14:53:22,029][86122] Updated weights for policy 1, policy_version 68130 (0.0009) +[2023-10-09 14:53:22,389][86122] Updated weights for policy 1, policy_version 68140 (0.0009) +[2023-10-09 14:53:22,760][86122] Updated weights for policy 1, policy_version 68150 (0.0010) +[2023-10-09 14:53:23,128][86122] Updated weights for policy 1, policy_version 68160 (0.0008) +[2023-10-09 14:53:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139296768. Throughput: 0: 1820.8, 1: 1819.9. Samples: 34831586. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:53:23,412][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000068160_69795840.pth... +[2023-10-09 14:53:23,412][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000067872_69500928.pth... +[2023-10-09 14:53:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000066176_67764224.pth +[2023-10-09 14:53:23,451][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000067872_69500928.pth +[2023-10-09 14:53:23,451][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000066432_68026368.pth +[2023-10-09 14:53:23,456][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000068160_69795840.pth +[2023-10-09 14:53:24,964][86121] Updated weights for policy 0, policy_version 67880 (0.0009) +[2023-10-09 14:53:25,330][86121] Updated weights for policy 0, policy_version 67890 (0.0008) +[2023-10-09 14:53:25,704][86121] Updated weights for policy 0, policy_version 67900 (0.0007) +[2023-10-09 14:53:26,791][86122] Updated weights for policy 1, policy_version 68170 (0.0010) +[2023-10-09 14:53:27,158][86122] Updated weights for policy 1, policy_version 68180 (0.0009) +[2023-10-09 14:53:27,521][86122] Updated weights for policy 1, policy_version 68190 (0.0008) +[2023-10-09 14:53:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139362304. Throughput: 0: 1825.1, 1: 1815.6. Samples: 34842628. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:53:29,210][86121] Updated weights for policy 0, policy_version 67910 (0.0009) +[2023-10-09 14:53:29,590][86121] Updated weights for policy 0, policy_version 67920 (0.0008) +[2023-10-09 14:53:29,959][86121] Updated weights for policy 0, policy_version 67930 (0.0007) +[2023-10-09 14:53:31,299][86122] Updated weights for policy 1, policy_version 68200 (0.0009) +[2023-10-09 14:53:31,652][86122] Updated weights for policy 1, policy_version 68210 (0.0008) +[2023-10-09 14:53:32,023][86122] Updated weights for policy 1, policy_version 68220 (0.0009) +[2023-10-09 14:53:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 139427840. Throughput: 0: 1824.7, 1: 1820.6. Samples: 34864746. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 14:53:33,657][86121] Updated weights for policy 0, policy_version 67940 (0.0008) +[2023-10-09 14:53:34,019][86121] Updated weights for policy 0, policy_version 67950 (0.0010) +[2023-10-09 14:53:34,389][86121] Updated weights for policy 0, policy_version 67960 (0.0009) +[2023-10-09 14:53:35,664][86122] Updated weights for policy 1, policy_version 68230 (0.0008) +[2023-10-09 14:53:36,019][86122] Updated weights for policy 1, policy_version 68240 (0.0007) +[2023-10-09 14:53:36,389][86122] Updated weights for policy 1, policy_version 68250 (0.0007) +[2023-10-09 14:53:37,949][86121] Updated weights for policy 0, policy_version 67970 (0.0010) +[2023-10-09 14:53:38,345][86121] Updated weights for policy 0, policy_version 67980 (0.0008) +[2023-10-09 14:53:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 139493376. Throughput: 0: 1828.6, 1: 1817.8. Samples: 34887044. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 14:53:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 14:53:38,709][86121] Updated weights for policy 0, policy_version 67990 (0.0007) +[2023-10-09 14:53:39,070][86121] Updated weights for policy 0, policy_version 68000 (0.0007) +[2023-10-09 14:53:40,053][86122] Updated weights for policy 1, policy_version 68260 (0.0007) +[2023-10-09 14:53:40,416][86122] Updated weights for policy 1, policy_version 68270 (0.0008) +[2023-10-09 14:53:40,779][86122] Updated weights for policy 1, policy_version 68280 (0.0010) +[2023-10-09 14:53:42,690][86121] Updated weights for policy 0, policy_version 68010 (0.0010) +[2023-10-09 14:53:43,059][86121] Updated weights for policy 0, policy_version 68020 (0.0008) +[2023-10-09 14:53:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 139558912. Throughput: 0: 1834.3, 1: 1813.1. Samples: 34897376. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:53:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 14:53:43,430][86121] Updated weights for policy 0, policy_version 68030 (0.0010) +[2023-10-09 14:53:44,297][86122] Updated weights for policy 1, policy_version 68290 (0.0010) +[2023-10-09 14:53:44,656][86122] Updated weights for policy 1, policy_version 68300 (0.0007) +[2023-10-09 14:53:45,013][86122] Updated weights for policy 1, policy_version 68310 (0.0009) +[2023-10-09 14:53:45,368][86122] Updated weights for policy 1, policy_version 68320 (0.0008) +[2023-10-09 14:53:47,140][86121] Updated weights for policy 0, policy_version 68040 (0.0009) +[2023-10-09 14:53:47,508][86121] Updated weights for policy 0, policy_version 68050 (0.0010) +[2023-10-09 14:53:47,878][86121] Updated weights for policy 0, policy_version 68060 (0.0009) +[2023-10-09 14:53:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 139657216. Throughput: 0: 1831.6, 1: 1825.1. Samples: 34919928. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:53:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:53:49,074][86122] Updated weights for policy 1, policy_version 68330 (0.0011) +[2023-10-09 14:53:49,446][86122] Updated weights for policy 1, policy_version 68340 (0.0011) +[2023-10-09 14:53:49,798][86122] Updated weights for policy 1, policy_version 68350 (0.0011) +[2023-10-09 14:53:51,748][86121] Updated weights for policy 0, policy_version 68070 (0.0009) +[2023-10-09 14:53:52,106][86121] Updated weights for policy 0, policy_version 68080 (0.0008) +[2023-10-09 14:53:52,473][86121] Updated weights for policy 0, policy_version 68090 (0.0008) +[2023-10-09 14:53:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 139722752. Throughput: 0: 1826.3, 1: 1830.5. Samples: 34941394. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:53:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:53:53,523][86122] Updated weights for policy 1, policy_version 68360 (0.0008) +[2023-10-09 14:53:53,880][86122] Updated weights for policy 1, policy_version 68370 (0.0009) +[2023-10-09 14:53:54,243][86122] Updated weights for policy 1, policy_version 68380 (0.0007) +[2023-10-09 14:53:56,053][86121] Updated weights for policy 0, policy_version 68100 (0.0008) +[2023-10-09 14:53:56,428][86121] Updated weights for policy 0, policy_version 68110 (0.0011) +[2023-10-09 14:53:56,796][86121] Updated weights for policy 0, policy_version 68120 (0.0010) +[2023-10-09 14:53:58,024][86122] Updated weights for policy 1, policy_version 68390 (0.0008) +[2023-10-09 14:53:58,378][86122] Updated weights for policy 1, policy_version 68400 (0.0009) +[2023-10-09 14:53:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 139788288. Throughput: 0: 1831.6, 1: 1828.5. Samples: 34952820. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:53:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:53:58,741][86122] Updated weights for policy 1, policy_version 68410 (0.0010) +[2023-10-09 14:54:00,513][86121] Updated weights for policy 0, policy_version 68130 (0.0008) +[2023-10-09 14:54:00,876][86121] Updated weights for policy 0, policy_version 68140 (0.0007) +[2023-10-09 14:54:01,233][86121] Updated weights for policy 0, policy_version 68150 (0.0009) +[2023-10-09 14:54:01,594][86121] Updated weights for policy 0, policy_version 68160 (0.0008) +[2023-10-09 14:54:02,421][86122] Updated weights for policy 1, policy_version 68420 (0.0010) +[2023-10-09 14:54:02,785][86122] Updated weights for policy 1, policy_version 68430 (0.0008) +[2023-10-09 14:54:03,141][86122] Updated weights for policy 1, policy_version 68440 (0.0007) +[2023-10-09 14:54:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 139853824. Throughput: 0: 1828.8, 1: 1823.6. Samples: 34974032. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:54:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:54:05,199][86121] Updated weights for policy 0, policy_version 68170 (0.0008) +[2023-10-09 14:54:05,554][86121] Updated weights for policy 0, policy_version 68180 (0.0007) +[2023-10-09 14:54:05,917][86121] Updated weights for policy 0, policy_version 68190 (0.0009) +[2023-10-09 14:54:06,847][86122] Updated weights for policy 1, policy_version 68450 (0.0008) +[2023-10-09 14:54:07,203][86122] Updated weights for policy 1, policy_version 68460 (0.0007) +[2023-10-09 14:54:07,564][86122] Updated weights for policy 1, policy_version 68470 (0.0007) +[2023-10-09 14:54:07,924][86122] Updated weights for policy 1, policy_version 68480 (0.0007) +[2023-10-09 14:54:08,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 139952128. Throughput: 0: 1826.3, 1: 1825.0. Samples: 34995892. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:54:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:54:09,702][86121] Updated weights for policy 0, policy_version 68200 (0.0009) +[2023-10-09 14:54:10,066][86121] Updated weights for policy 0, policy_version 68210 (0.0007) +[2023-10-09 14:54:10,434][86121] Updated weights for policy 0, policy_version 68220 (0.0008) +[2023-10-09 14:54:11,541][86122] Updated weights for policy 1, policy_version 68490 (0.0009) +[2023-10-09 14:54:11,895][86122] Updated weights for policy 1, policy_version 68500 (0.0008) +[2023-10-09 14:54:12,265][86122] Updated weights for policy 1, policy_version 68510 (0.0009) +[2023-10-09 14:54:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140017664. Throughput: 0: 1824.8, 1: 1834.9. Samples: 35007316. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:54:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:14,008][86121] Updated weights for policy 0, policy_version 68230 (0.0009) +[2023-10-09 14:54:14,374][86121] Updated weights for policy 0, policy_version 68240 (0.0011) +[2023-10-09 14:54:14,747][86121] Updated weights for policy 0, policy_version 68250 (0.0011) +[2023-10-09 14:54:16,031][86122] Updated weights for policy 1, policy_version 68520 (0.0012) +[2023-10-09 14:54:16,402][86122] Updated weights for policy 1, policy_version 68530 (0.0010) +[2023-10-09 14:54:16,756][86122] Updated weights for policy 1, policy_version 68540 (0.0010) +[2023-10-09 14:54:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140083200. Throughput: 0: 1827.4, 1: 1821.7. Samples: 35028956. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:54:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:18,504][86121] Updated weights for policy 0, policy_version 68260 (0.0009) +[2023-10-09 14:54:18,870][86121] Updated weights for policy 0, policy_version 68270 (0.0009) +[2023-10-09 14:54:19,246][86121] Updated weights for policy 0, policy_version 68280 (0.0009) +[2023-10-09 14:54:20,420][86122] Updated weights for policy 1, policy_version 68550 (0.0008) +[2023-10-09 14:54:20,789][86122] Updated weights for policy 1, policy_version 68560 (0.0007) +[2023-10-09 14:54:21,151][86122] Updated weights for policy 1, policy_version 68570 (0.0007) +[2023-10-09 14:54:23,076][86121] Updated weights for policy 0, policy_version 68290 (0.0008) +[2023-10-09 14:54:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 140148736. Throughput: 0: 1818.5, 1: 1837.6. Samples: 35051570. Policy #0 lag: (min: 31.0, avg: 44.0, max: 63.0) +[2023-10-09 14:54:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:23,455][86121] Updated weights for policy 0, policy_version 68300 (0.0008) +[2023-10-09 14:54:23,817][86121] Updated weights for policy 0, policy_version 68310 (0.0008) +[2023-10-09 14:54:24,189][86121] Updated weights for policy 0, policy_version 68320 (0.0009) +[2023-10-09 14:54:24,655][86122] Updated weights for policy 1, policy_version 68580 (0.0009) +[2023-10-09 14:54:25,012][86122] Updated weights for policy 1, policy_version 68590 (0.0010) +[2023-10-09 14:54:25,382][86122] Updated weights for policy 1, policy_version 68600 (0.0012) +[2023-10-09 14:54:27,693][86121] Updated weights for policy 0, policy_version 68330 (0.0009) +[2023-10-09 14:54:28,064][86121] Updated weights for policy 0, policy_version 68340 (0.0010) +[2023-10-09 14:54:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 140214272. Throughput: 0: 1815.3, 1: 1831.0. Samples: 35061462. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:54:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:28,433][86121] Updated weights for policy 0, policy_version 68350 (0.0009) +[2023-10-09 14:54:29,196][86122] Updated weights for policy 1, policy_version 68610 (0.0010) +[2023-10-09 14:54:29,559][86122] Updated weights for policy 1, policy_version 68620 (0.0007) +[2023-10-09 14:54:29,925][86122] Updated weights for policy 1, policy_version 68630 (0.0010) +[2023-10-09 14:54:30,283][86122] Updated weights for policy 1, policy_version 68640 (0.0010) +[2023-10-09 14:54:32,259][86121] Updated weights for policy 0, policy_version 68360 (0.0008) +[2023-10-09 14:54:32,629][86121] Updated weights for policy 0, policy_version 68370 (0.0007) +[2023-10-09 14:54:32,991][86121] Updated weights for policy 0, policy_version 68380 (0.0009) +[2023-10-09 14:54:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140312576. Throughput: 0: 1817.3, 1: 1832.9. Samples: 35084186. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:54:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:33,918][86122] Updated weights for policy 1, policy_version 68650 (0.0007) +[2023-10-09 14:54:34,275][86122] Updated weights for policy 1, policy_version 68660 (0.0009) +[2023-10-09 14:54:34,633][86122] Updated weights for policy 1, policy_version 68670 (0.0011) +[2023-10-09 14:54:36,670][86121] Updated weights for policy 0, policy_version 68390 (0.0008) +[2023-10-09 14:54:37,031][86121] Updated weights for policy 0, policy_version 68400 (0.0008) +[2023-10-09 14:54:37,397][86121] Updated weights for policy 0, policy_version 68410 (0.0007) +[2023-10-09 14:54:38,389][86122] Updated weights for policy 1, policy_version 68680 (0.0009) +[2023-10-09 14:54:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140378112. Throughput: 0: 1819.9, 1: 1834.0. Samples: 35105818. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:54:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:38,760][86122] Updated weights for policy 1, policy_version 68690 (0.0009) +[2023-10-09 14:54:39,113][86122] Updated weights for policy 1, policy_version 68700 (0.0009) +[2023-10-09 14:54:41,197][86121] Updated weights for policy 0, policy_version 68420 (0.0008) +[2023-10-09 14:54:41,571][86121] Updated weights for policy 0, policy_version 68430 (0.0009) +[2023-10-09 14:54:41,931][86121] Updated weights for policy 0, policy_version 68440 (0.0010) +[2023-10-09 14:54:42,663][86122] Updated weights for policy 1, policy_version 68710 (0.0007) +[2023-10-09 14:54:43,030][86122] Updated weights for policy 1, policy_version 68720 (0.0007) +[2023-10-09 14:54:43,387][86122] Updated weights for policy 1, policy_version 68730 (0.0008) +[2023-10-09 14:54:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 140443648. Throughput: 0: 1815.1, 1: 1835.6. Samples: 35117098. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:54:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:45,680][86121] Updated weights for policy 0, policy_version 68450 (0.0009) +[2023-10-09 14:54:46,040][86121] Updated weights for policy 0, policy_version 68460 (0.0007) +[2023-10-09 14:54:46,412][86121] Updated weights for policy 0, policy_version 68470 (0.0009) +[2023-10-09 14:54:46,769][86121] Updated weights for policy 0, policy_version 68480 (0.0008) +[2023-10-09 14:54:46,972][86122] Updated weights for policy 1, policy_version 68740 (0.0008) +[2023-10-09 14:54:47,325][86122] Updated weights for policy 1, policy_version 68750 (0.0007) +[2023-10-09 14:54:47,682][86122] Updated weights for policy 1, policy_version 68760 (0.0009) +[2023-10-09 14:54:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140541952. Throughput: 0: 1815.9, 1: 1842.4. Samples: 35138654. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:54:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 14:54:50,224][86121] Updated weights for policy 0, policy_version 68490 (0.0009) +[2023-10-09 14:54:50,587][86121] Updated weights for policy 0, policy_version 68500 (0.0007) +[2023-10-09 14:54:50,955][86121] Updated weights for policy 0, policy_version 68510 (0.0007) +[2023-10-09 14:54:51,397][86122] Updated weights for policy 1, policy_version 68770 (0.0012) +[2023-10-09 14:54:51,765][86122] Updated weights for policy 1, policy_version 68780 (0.0008) +[2023-10-09 14:54:52,125][86122] Updated weights for policy 1, policy_version 68790 (0.0007) +[2023-10-09 14:54:52,487][86122] Updated weights for policy 1, policy_version 68800 (0.0009) +[2023-10-09 14:54:53,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140607488. Throughput: 0: 1824.2, 1: 1833.4. Samples: 35160486. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:54:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 14:54:54,492][86121] Updated weights for policy 0, policy_version 68520 (0.0008) +[2023-10-09 14:54:54,851][86121] Updated weights for policy 0, policy_version 68530 (0.0008) +[2023-10-09 14:54:55,212][86121] Updated weights for policy 0, policy_version 68540 (0.0007) +[2023-10-09 14:54:56,216][86122] Updated weights for policy 1, policy_version 68810 (0.0007) +[2023-10-09 14:54:56,588][86122] Updated weights for policy 1, policy_version 68820 (0.0007) +[2023-10-09 14:54:56,945][86122] Updated weights for policy 1, policy_version 68830 (0.0011) +[2023-10-09 14:54:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 140673024. Throughput: 0: 1824.3, 1: 1829.3. Samples: 35171728. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:54:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 14:54:59,116][86121] Updated weights for policy 0, policy_version 68550 (0.0011) +[2023-10-09 14:54:59,484][86121] Updated weights for policy 0, policy_version 68560 (0.0008) +[2023-10-09 14:54:59,854][86121] Updated weights for policy 0, policy_version 68570 (0.0008) +[2023-10-09 14:55:00,645][86122] Updated weights for policy 1, policy_version 68840 (0.0008) +[2023-10-09 14:55:01,005][86122] Updated weights for policy 1, policy_version 68850 (0.0010) +[2023-10-09 14:55:01,363][86122] Updated weights for policy 1, policy_version 68860 (0.0009) +[2023-10-09 14:55:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140738560. Throughput: 0: 1821.4, 1: 1827.4. Samples: 35193152. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:55:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 14:55:03,508][86121] Updated weights for policy 0, policy_version 68580 (0.0009) +[2023-10-09 14:55:03,883][86121] Updated weights for policy 0, policy_version 68590 (0.0008) +[2023-10-09 14:55:04,255][86121] Updated weights for policy 0, policy_version 68600 (0.0008) +[2023-10-09 14:55:04,923][86122] Updated weights for policy 1, policy_version 68870 (0.0007) +[2023-10-09 14:55:05,284][86122] Updated weights for policy 1, policy_version 68880 (0.0009) +[2023-10-09 14:55:05,650][86122] Updated weights for policy 1, policy_version 68890 (0.0008) +[2023-10-09 14:55:07,941][86121] Updated weights for policy 0, policy_version 68610 (0.0009) +[2023-10-09 14:55:08,350][86121] Updated weights for policy 0, policy_version 68620 (0.0007) +[2023-10-09 14:55:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 140804096. Throughput: 0: 1826.1, 1: 1832.4. Samples: 35216204. Policy #0 lag: (min: 31.0, avg: 37.3, max: 63.0) +[2023-10-09 14:55:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 14:55:08,716][86121] Updated weights for policy 0, policy_version 68630 (0.0007) +[2023-10-09 14:55:09,084][86121] Updated weights for policy 0, policy_version 68640 (0.0009) +[2023-10-09 14:55:09,258][86122] Updated weights for policy 1, policy_version 68900 (0.0008) +[2023-10-09 14:55:09,609][86122] Updated weights for policy 1, policy_version 68910 (0.0008) +[2023-10-09 14:55:09,970][86122] Updated weights for policy 1, policy_version 68920 (0.0007) +[2023-10-09 14:55:12,817][86121] Updated weights for policy 0, policy_version 68650 (0.0008) +[2023-10-09 14:55:13,182][86121] Updated weights for policy 0, policy_version 68660 (0.0008) +[2023-10-09 14:55:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 140869632. Throughput: 0: 1828.9, 1: 1832.9. Samples: 35226242. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 14:55:13,535][86121] Updated weights for policy 0, policy_version 68670 (0.0009) +[2023-10-09 14:55:13,675][86122] Updated weights for policy 1, policy_version 68930 (0.0010) +[2023-10-09 14:55:14,036][86122] Updated weights for policy 1, policy_version 68940 (0.0011) +[2023-10-09 14:55:14,397][86122] Updated weights for policy 1, policy_version 68950 (0.0011) +[2023-10-09 14:55:14,762][86122] Updated weights for policy 1, policy_version 68960 (0.0012) +[2023-10-09 14:55:17,304][86121] Updated weights for policy 0, policy_version 68680 (0.0009) +[2023-10-09 14:55:17,672][86121] Updated weights for policy 0, policy_version 68690 (0.0007) +[2023-10-09 14:55:18,049][86121] Updated weights for policy 0, policy_version 68700 (0.0007) +[2023-10-09 14:55:18,378][86122] Updated weights for policy 1, policy_version 68970 (0.0009) +[2023-10-09 14:55:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 140967936. Throughput: 0: 1827.6, 1: 1846.0. Samples: 35249498. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 14:55:18,750][86122] Updated weights for policy 1, policy_version 68980 (0.0007) +[2023-10-09 14:55:19,103][86122] Updated weights for policy 1, policy_version 68990 (0.0008) +[2023-10-09 14:55:21,680][86121] Updated weights for policy 0, policy_version 68710 (0.0009) +[2023-10-09 14:55:22,054][86121] Updated weights for policy 0, policy_version 68720 (0.0009) +[2023-10-09 14:55:22,427][86121] Updated weights for policy 0, policy_version 68730 (0.0010) +[2023-10-09 14:55:22,792][86122] Updated weights for policy 1, policy_version 69000 (0.0009) +[2023-10-09 14:55:23,161][86122] Updated weights for policy 1, policy_version 69010 (0.0010) +[2023-10-09 14:55:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141033472. Throughput: 0: 1826.8, 1: 1832.2. Samples: 35270476. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:55:23,405][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000068736_70385664.pth... +[2023-10-09 14:55:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000067040_68648960.pth +[2023-10-09 14:55:23,524][86122] Updated weights for policy 1, policy_version 69020 (0.0007) +[2023-10-09 14:55:23,666][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000069024_70680576.pth... +[2023-10-09 14:55:23,704][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000067296_68911104.pth +[2023-10-09 14:55:26,076][86121] Updated weights for policy 0, policy_version 68740 (0.0008) +[2023-10-09 14:55:26,444][86121] Updated weights for policy 0, policy_version 68750 (0.0010) +[2023-10-09 14:55:26,820][86121] Updated weights for policy 0, policy_version 68760 (0.0011) +[2023-10-09 14:55:27,367][86122] Updated weights for policy 1, policy_version 69030 (0.0009) +[2023-10-09 14:55:27,726][86122] Updated weights for policy 1, policy_version 69040 (0.0010) +[2023-10-09 14:55:28,091][86122] Updated weights for policy 1, policy_version 69050 (0.0010) +[2023-10-09 14:55:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 141131776. Throughput: 0: 1828.7, 1: 1837.6. Samples: 35282082. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:55:30,480][86121] Updated weights for policy 0, policy_version 68770 (0.0009) +[2023-10-09 14:55:30,846][86121] Updated weights for policy 0, policy_version 68780 (0.0008) +[2023-10-09 14:55:31,213][86121] Updated weights for policy 0, policy_version 68790 (0.0008) +[2023-10-09 14:55:31,587][86121] Updated weights for policy 0, policy_version 68800 (0.0008) +[2023-10-09 14:55:31,784][86122] Updated weights for policy 1, policy_version 69060 (0.0007) +[2023-10-09 14:55:32,146][86122] Updated weights for policy 1, policy_version 69070 (0.0007) +[2023-10-09 14:55:32,510][86122] Updated weights for policy 1, policy_version 69080 (0.0008) +[2023-10-09 14:55:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141197312. Throughput: 0: 1831.8, 1: 1826.4. Samples: 35303274. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 14:55:35,207][86121] Updated weights for policy 0, policy_version 68810 (0.0008) +[2023-10-09 14:55:35,565][86121] Updated weights for policy 0, policy_version 68820 (0.0008) +[2023-10-09 14:55:35,923][86121] Updated weights for policy 0, policy_version 68830 (0.0008) +[2023-10-09 14:55:36,188][86122] Updated weights for policy 1, policy_version 69090 (0.0008) +[2023-10-09 14:55:36,556][86122] Updated weights for policy 1, policy_version 69100 (0.0008) +[2023-10-09 14:55:36,919][86122] Updated weights for policy 1, policy_version 69110 (0.0008) +[2023-10-09 14:55:37,278][86122] Updated weights for policy 1, policy_version 69120 (0.0007) +[2023-10-09 14:55:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141262848. Throughput: 0: 1829.6, 1: 1831.2. Samples: 35325218. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 14:55:39,367][86121] Updated weights for policy 0, policy_version 68840 (0.0008) +[2023-10-09 14:55:39,741][86121] Updated weights for policy 0, policy_version 68850 (0.0007) +[2023-10-09 14:55:40,107][86121] Updated weights for policy 0, policy_version 68860 (0.0007) +[2023-10-09 14:55:41,044][86122] Updated weights for policy 1, policy_version 69130 (0.0008) +[2023-10-09 14:55:41,400][86122] Updated weights for policy 1, policy_version 69140 (0.0011) +[2023-10-09 14:55:41,763][86122] Updated weights for policy 1, policy_version 69150 (0.0010) +[2023-10-09 14:55:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141328384. Throughput: 0: 1827.5, 1: 1833.2. Samples: 35336456. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 14:55:44,013][86121] Updated weights for policy 0, policy_version 68870 (0.0008) +[2023-10-09 14:55:44,379][86121] Updated weights for policy 0, policy_version 68880 (0.0008) +[2023-10-09 14:55:44,751][86121] Updated weights for policy 0, policy_version 68890 (0.0009) +[2023-10-09 14:55:45,460][86122] Updated weights for policy 1, policy_version 69160 (0.0007) +[2023-10-09 14:55:45,820][86122] Updated weights for policy 1, policy_version 69170 (0.0007) +[2023-10-09 14:55:46,190][86122] Updated weights for policy 1, policy_version 69180 (0.0008) +[2023-10-09 14:55:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 141393920. Throughput: 0: 1825.1, 1: 1840.1. Samples: 35358086. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 14:55:48,554][86121] Updated weights for policy 0, policy_version 68900 (0.0008) +[2023-10-09 14:55:48,923][86121] Updated weights for policy 0, policy_version 68910 (0.0009) +[2023-10-09 14:55:49,291][86121] Updated weights for policy 0, policy_version 68920 (0.0007) +[2023-10-09 14:55:50,052][86122] Updated weights for policy 1, policy_version 69190 (0.0009) +[2023-10-09 14:55:50,409][86122] Updated weights for policy 1, policy_version 69200 (0.0011) +[2023-10-09 14:55:50,771][86122] Updated weights for policy 1, policy_version 69210 (0.0010) +[2023-10-09 14:55:52,910][86121] Updated weights for policy 0, policy_version 68930 (0.0007) +[2023-10-09 14:55:53,303][86121] Updated weights for policy 0, policy_version 68940 (0.0008) +[2023-10-09 14:55:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141459456. Throughput: 0: 1821.0, 1: 1831.0. Samples: 35380542. Policy #0 lag: (min: 31.0, avg: 31.1, max: 36.0) +[2023-10-09 14:55:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:55:53,665][86121] Updated weights for policy 0, policy_version 68950 (0.0009) +[2023-10-09 14:55:54,035][86121] Updated weights for policy 0, policy_version 68960 (0.0008) +[2023-10-09 14:55:54,344][86122] Updated weights for policy 1, policy_version 69220 (0.0010) +[2023-10-09 14:55:54,705][86122] Updated weights for policy 1, policy_version 69230 (0.0009) +[2023-10-09 14:55:55,070][86122] Updated weights for policy 1, policy_version 69240 (0.0008) +[2023-10-09 14:55:57,585][86121] Updated weights for policy 0, policy_version 68970 (0.0008) +[2023-10-09 14:55:57,950][86121] Updated weights for policy 0, policy_version 68980 (0.0009) +[2023-10-09 14:55:58,315][86121] Updated weights for policy 0, policy_version 68990 (0.0009) +[2023-10-09 14:55:58,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 141557760. Throughput: 0: 1826.4, 1: 1832.8. Samples: 35390908. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:55:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:55:58,707][86122] Updated weights for policy 1, policy_version 69250 (0.0009) +[2023-10-09 14:55:59,071][86122] Updated weights for policy 1, policy_version 69260 (0.0009) +[2023-10-09 14:55:59,434][86122] Updated weights for policy 1, policy_version 69270 (0.0009) +[2023-10-09 14:55:59,804][86122] Updated weights for policy 1, policy_version 69280 (0.0009) +[2023-10-09 14:56:01,966][86121] Updated weights for policy 0, policy_version 69000 (0.0008) +[2023-10-09 14:56:02,333][86121] Updated weights for policy 0, policy_version 69010 (0.0007) +[2023-10-09 14:56:02,700][86121] Updated weights for policy 0, policy_version 69020 (0.0007) +[2023-10-09 14:56:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141623296. Throughput: 0: 1823.6, 1: 1822.7. Samples: 35413584. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:56:03,635][86122] Updated weights for policy 1, policy_version 69290 (0.0009) +[2023-10-09 14:56:04,010][86122] Updated weights for policy 1, policy_version 69300 (0.0008) +[2023-10-09 14:56:04,378][86122] Updated weights for policy 1, policy_version 69310 (0.0008) +[2023-10-09 14:56:06,418][86121] Updated weights for policy 0, policy_version 69030 (0.0008) +[2023-10-09 14:56:06,790][86121] Updated weights for policy 0, policy_version 69040 (0.0008) +[2023-10-09 14:56:07,150][86121] Updated weights for policy 0, policy_version 69050 (0.0008) +[2023-10-09 14:56:07,951][86122] Updated weights for policy 1, policy_version 69320 (0.0010) +[2023-10-09 14:56:08,333][86122] Updated weights for policy 1, policy_version 69330 (0.0009) +[2023-10-09 14:56:08,397][85186] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 141688832. Throughput: 0: 1827.0, 1: 1832.3. Samples: 35435146. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:08,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 14:56:08,693][86122] Updated weights for policy 1, policy_version 69340 (0.0011) +[2023-10-09 14:56:10,971][86121] Updated weights for policy 0, policy_version 69060 (0.0008) +[2023-10-09 14:56:11,335][86121] Updated weights for policy 0, policy_version 69070 (0.0010) +[2023-10-09 14:56:11,694][86121] Updated weights for policy 0, policy_version 69080 (0.0010) +[2023-10-09 14:56:12,411][86122] Updated weights for policy 1, policy_version 69350 (0.0008) +[2023-10-09 14:56:12,781][86122] Updated weights for policy 1, policy_version 69360 (0.0008) +[2023-10-09 14:56:13,133][86122] Updated weights for policy 1, policy_version 69370 (0.0010) +[2023-10-09 14:56:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 141787136. Throughput: 0: 1821.3, 1: 1828.7. Samples: 35446332. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:56:15,392][86121] Updated weights for policy 0, policy_version 69090 (0.0010) +[2023-10-09 14:56:15,758][86121] Updated weights for policy 0, policy_version 69100 (0.0009) +[2023-10-09 14:56:16,116][86121] Updated weights for policy 0, policy_version 69110 (0.0009) +[2023-10-09 14:56:16,485][86121] Updated weights for policy 0, policy_version 69120 (0.0011) +[2023-10-09 14:56:16,992][86122] Updated weights for policy 1, policy_version 69380 (0.0009) +[2023-10-09 14:56:17,349][86122] Updated weights for policy 1, policy_version 69390 (0.0009) +[2023-10-09 14:56:17,709][86122] Updated weights for policy 1, policy_version 69400 (0.0008) +[2023-10-09 14:56:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 141852672. Throughput: 0: 1823.0, 1: 1832.1. Samples: 35467754. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:56:20,245][86121] Updated weights for policy 0, policy_version 69130 (0.0007) +[2023-10-09 14:56:20,612][86121] Updated weights for policy 0, policy_version 69140 (0.0007) +[2023-10-09 14:56:20,981][86121] Updated weights for policy 0, policy_version 69150 (0.0008) +[2023-10-09 14:56:21,372][86122] Updated weights for policy 1, policy_version 69410 (0.0008) +[2023-10-09 14:56:21,733][86122] Updated weights for policy 1, policy_version 69420 (0.0008) +[2023-10-09 14:56:22,098][86122] Updated weights for policy 1, policy_version 69430 (0.0007) +[2023-10-09 14:56:22,457][86122] Updated weights for policy 1, policy_version 69440 (0.0007) +[2023-10-09 14:56:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 141918208. Throughput: 0: 1824.7, 1: 1825.2. Samples: 35489466. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:56:24,542][86121] Updated weights for policy 0, policy_version 69160 (0.0009) +[2023-10-09 14:56:24,903][86121] Updated weights for policy 0, policy_version 69170 (0.0009) +[2023-10-09 14:56:25,259][86121] Updated weights for policy 0, policy_version 69180 (0.0008) +[2023-10-09 14:56:26,072][86122] Updated weights for policy 1, policy_version 69450 (0.0007) +[2023-10-09 14:56:26,441][86122] Updated weights for policy 1, policy_version 69460 (0.0009) +[2023-10-09 14:56:26,794][86122] Updated weights for policy 1, policy_version 69470 (0.0008) +[2023-10-09 14:56:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 141983744. Throughput: 0: 1824.1, 1: 1826.2. Samples: 35500720. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:56:28,910][86121] Updated weights for policy 0, policy_version 69190 (0.0009) +[2023-10-09 14:56:29,282][86121] Updated weights for policy 0, policy_version 69200 (0.0009) +[2023-10-09 14:56:29,645][86121] Updated weights for policy 0, policy_version 69210 (0.0008) +[2023-10-09 14:56:30,524][86122] Updated weights for policy 1, policy_version 69480 (0.0007) +[2023-10-09 14:56:30,890][86122] Updated weights for policy 1, policy_version 69490 (0.0008) +[2023-10-09 14:56:31,244][86122] Updated weights for policy 1, policy_version 69500 (0.0007) +[2023-10-09 14:56:33,377][86121] Updated weights for policy 0, policy_version 69220 (0.0010) +[2023-10-09 14:56:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142049280. Throughput: 0: 1826.3, 1: 1823.1. Samples: 35522308. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:56:33,742][86121] Updated weights for policy 0, policy_version 69230 (0.0011) +[2023-10-09 14:56:34,103][86121] Updated weights for policy 0, policy_version 69240 (0.0009) +[2023-10-09 14:56:34,789][86122] Updated weights for policy 1, policy_version 69510 (0.0008) +[2023-10-09 14:56:35,140][86122] Updated weights for policy 1, policy_version 69520 (0.0011) +[2023-10-09 14:56:35,505][86122] Updated weights for policy 1, policy_version 69530 (0.0010) +[2023-10-09 14:56:37,890][86121] Updated weights for policy 0, policy_version 69250 (0.0010) +[2023-10-09 14:56:38,290][86121] Updated weights for policy 0, policy_version 69260 (0.0010) +[2023-10-09 14:56:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 142114816. Throughput: 0: 1827.6, 1: 1831.5. Samples: 35545202. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 14:56:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:56:38,659][86121] Updated weights for policy 0, policy_version 69270 (0.0008) +[2023-10-09 14:56:39,023][86121] Updated weights for policy 0, policy_version 69280 (0.0008) +[2023-10-09 14:56:39,108][86122] Updated weights for policy 1, policy_version 69540 (0.0009) +[2023-10-09 14:56:39,463][86122] Updated weights for policy 1, policy_version 69550 (0.0007) +[2023-10-09 14:56:39,818][86122] Updated weights for policy 1, policy_version 69560 (0.0007) +[2023-10-09 14:56:42,674][86121] Updated weights for policy 0, policy_version 69290 (0.0009) +[2023-10-09 14:56:43,038][86121] Updated weights for policy 0, policy_version 69300 (0.0008) +[2023-10-09 14:56:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142180352. Throughput: 0: 1824.3, 1: 1827.5. Samples: 35555240. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:56:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 14:56:43,410][86121] Updated weights for policy 0, policy_version 69310 (0.0008) +[2023-10-09 14:56:43,626][86122] Updated weights for policy 1, policy_version 69570 (0.0009) +[2023-10-09 14:56:43,986][86122] Updated weights for policy 1, policy_version 69580 (0.0007) +[2023-10-09 14:56:44,339][86122] Updated weights for policy 1, policy_version 69590 (0.0008) +[2023-10-09 14:56:44,700][86122] Updated weights for policy 1, policy_version 69600 (0.0007) +[2023-10-09 14:56:46,996][86121] Updated weights for policy 0, policy_version 69320 (0.0007) +[2023-10-09 14:56:47,362][86121] Updated weights for policy 0, policy_version 69330 (0.0010) +[2023-10-09 14:56:47,733][86121] Updated weights for policy 0, policy_version 69340 (0.0009) +[2023-10-09 14:56:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142278656. Throughput: 0: 1820.5, 1: 1818.9. Samples: 35577360. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:56:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 14:56:48,550][86122] Updated weights for policy 1, policy_version 69610 (0.0008) +[2023-10-09 14:56:48,917][86122] Updated weights for policy 1, policy_version 69620 (0.0010) +[2023-10-09 14:56:49,275][86122] Updated weights for policy 1, policy_version 69630 (0.0008) +[2023-10-09 14:56:51,263][86121] Updated weights for policy 0, policy_version 69350 (0.0008) +[2023-10-09 14:56:51,633][86121] Updated weights for policy 0, policy_version 69360 (0.0009) +[2023-10-09 14:56:52,004][86121] Updated weights for policy 0, policy_version 69370 (0.0011) +[2023-10-09 14:56:52,967][86122] Updated weights for policy 1, policy_version 69640 (0.0009) +[2023-10-09 14:56:53,328][86122] Updated weights for policy 1, policy_version 69650 (0.0009) +[2023-10-09 14:56:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 142344192. Throughput: 0: 1826.0, 1: 1818.6. Samples: 35599156. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:56:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:56:53,681][86122] Updated weights for policy 1, policy_version 69660 (0.0008) +[2023-10-09 14:56:55,717][86121] Updated weights for policy 0, policy_version 69380 (0.0007) +[2023-10-09 14:56:56,081][86121] Updated weights for policy 0, policy_version 69390 (0.0009) +[2023-10-09 14:56:56,448][86121] Updated weights for policy 0, policy_version 69400 (0.0011) +[2023-10-09 14:56:57,238][86122] Updated weights for policy 1, policy_version 69670 (0.0008) +[2023-10-09 14:56:57,593][86122] Updated weights for policy 1, policy_version 69680 (0.0007) +[2023-10-09 14:56:57,957][86122] Updated weights for policy 1, policy_version 69690 (0.0007) +[2023-10-09 14:56:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142442496. Throughput: 0: 1819.4, 1: 1823.8. Samples: 35610278. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:56:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:57:00,085][86121] Updated weights for policy 0, policy_version 69410 (0.0010) +[2023-10-09 14:57:00,447][86121] Updated weights for policy 0, policy_version 69420 (0.0010) +[2023-10-09 14:57:00,808][86121] Updated weights for policy 0, policy_version 69430 (0.0010) +[2023-10-09 14:57:01,176][86121] Updated weights for policy 0, policy_version 69440 (0.0011) +[2023-10-09 14:57:01,632][86122] Updated weights for policy 1, policy_version 69700 (0.0009) +[2023-10-09 14:57:01,992][86122] Updated weights for policy 1, policy_version 69710 (0.0010) +[2023-10-09 14:57:02,356][86122] Updated weights for policy 1, policy_version 69720 (0.0009) +[2023-10-09 14:57:03,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142508032. Throughput: 0: 1826.8, 1: 1816.6. Samples: 35631706. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:57:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:57:04,825][86121] Updated weights for policy 0, policy_version 69450 (0.0009) +[2023-10-09 14:57:05,195][86121] Updated weights for policy 0, policy_version 69460 (0.0007) +[2023-10-09 14:57:05,555][86121] Updated weights for policy 0, policy_version 69470 (0.0009) +[2023-10-09 14:57:06,094][86122] Updated weights for policy 1, policy_version 69730 (0.0009) +[2023-10-09 14:57:06,456][86122] Updated weights for policy 1, policy_version 69740 (0.0008) +[2023-10-09 14:57:06,822][86122] Updated weights for policy 1, policy_version 69750 (0.0009) +[2023-10-09 14:57:07,186][86122] Updated weights for policy 1, policy_version 69760 (0.0008) +[2023-10-09 14:57:08,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 142573568. Throughput: 0: 1822.5, 1: 1831.1. Samples: 35653878. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:57:08,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:57:09,348][86121] Updated weights for policy 0, policy_version 69480 (0.0010) +[2023-10-09 14:57:09,724][86121] Updated weights for policy 0, policy_version 69490 (0.0009) +[2023-10-09 14:57:10,095][86121] Updated weights for policy 0, policy_version 69500 (0.0009) +[2023-10-09 14:57:10,812][86122] Updated weights for policy 1, policy_version 69770 (0.0010) +[2023-10-09 14:57:11,180][86122] Updated weights for policy 1, policy_version 69780 (0.0008) +[2023-10-09 14:57:11,527][86122] Updated weights for policy 1, policy_version 69790 (0.0008) +[2023-10-09 14:57:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142639104. Throughput: 0: 1824.0, 1: 1820.8. Samples: 35664732. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:57:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:57:13,668][86121] Updated weights for policy 0, policy_version 69510 (0.0008) +[2023-10-09 14:57:14,035][86121] Updated weights for policy 0, policy_version 69520 (0.0008) +[2023-10-09 14:57:14,408][86121] Updated weights for policy 0, policy_version 69530 (0.0008) +[2023-10-09 14:57:15,215][86122] Updated weights for policy 1, policy_version 69800 (0.0008) +[2023-10-09 14:57:15,576][86122] Updated weights for policy 1, policy_version 69810 (0.0009) +[2023-10-09 14:57:15,945][86122] Updated weights for policy 1, policy_version 69820 (0.0008) +[2023-10-09 14:57:18,073][86121] Updated weights for policy 0, policy_version 69540 (0.0007) +[2023-10-09 14:57:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142704640. Throughput: 0: 1828.4, 1: 1829.1. Samples: 35686898. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:57:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:57:18,429][86121] Updated weights for policy 0, policy_version 69550 (0.0010) +[2023-10-09 14:57:18,803][86121] Updated weights for policy 0, policy_version 69560 (0.0009) +[2023-10-09 14:57:19,584][86122] Updated weights for policy 1, policy_version 69830 (0.0009) +[2023-10-09 14:57:19,948][86122] Updated weights for policy 1, policy_version 69840 (0.0008) +[2023-10-09 14:57:20,315][86122] Updated weights for policy 1, policy_version 69850 (0.0010) +[2023-10-09 14:57:22,572][86121] Updated weights for policy 0, policy_version 69570 (0.0008) +[2023-10-09 14:57:22,968][86121] Updated weights for policy 0, policy_version 69580 (0.0008) +[2023-10-09 14:57:23,332][86121] Updated weights for policy 0, policy_version 69590 (0.0008) +[2023-10-09 14:57:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 142770176. Throughput: 0: 1823.5, 1: 1822.5. Samples: 35709268. Policy #0 lag: (min: 31.0, avg: 38.9, max: 63.0) +[2023-10-09 14:57:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:57:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000069856_71532544.pth... +[2023-10-09 14:57:23,442][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000068160_69795840.pth +[2023-10-09 14:57:23,696][86121] Updated weights for policy 0, policy_version 69600 (0.0010) +[2023-10-09 14:57:23,697][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000069600_71270400.pth... +[2023-10-09 14:57:23,726][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000067872_69500928.pth +[2023-10-09 14:57:24,087][86122] Updated weights for policy 1, policy_version 69860 (0.0010) +[2023-10-09 14:57:24,445][86122] Updated weights for policy 1, policy_version 69870 (0.0011) +[2023-10-09 14:57:24,807][86122] Updated weights for policy 1, policy_version 69880 (0.0010) +[2023-10-09 14:57:27,173][86121] Updated weights for policy 0, policy_version 69610 (0.0010) +[2023-10-09 14:57:27,539][86121] Updated weights for policy 0, policy_version 69620 (0.0007) +[2023-10-09 14:57:27,904][86121] Updated weights for policy 0, policy_version 69630 (0.0007) +[2023-10-09 14:57:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 142868480. Throughput: 0: 1834.7, 1: 1825.0. Samples: 35719926. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:57:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:57:28,497][86122] Updated weights for policy 1, policy_version 69890 (0.0009) +[2023-10-09 14:57:28,860][86122] Updated weights for policy 1, policy_version 69900 (0.0008) +[2023-10-09 14:57:29,219][86122] Updated weights for policy 1, policy_version 69910 (0.0010) +[2023-10-09 14:57:29,586][86122] Updated weights for policy 1, policy_version 69920 (0.0009) +[2023-10-09 14:57:31,666][86121] Updated weights for policy 0, policy_version 69640 (0.0009) +[2023-10-09 14:57:32,031][86121] Updated weights for policy 0, policy_version 69650 (0.0009) +[2023-10-09 14:57:32,397][86121] Updated weights for policy 0, policy_version 69660 (0.0009) +[2023-10-09 14:57:33,246][86122] Updated weights for policy 1, policy_version 69930 (0.0008) +[2023-10-09 14:57:33,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 142934016. Throughput: 0: 1829.2, 1: 1832.6. Samples: 35742142. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:57:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:57:33,612][86122] Updated weights for policy 1, policy_version 69940 (0.0010) +[2023-10-09 14:57:33,972][86122] Updated weights for policy 1, policy_version 69950 (0.0010) +[2023-10-09 14:57:36,059][86121] Updated weights for policy 0, policy_version 69670 (0.0009) +[2023-10-09 14:57:36,421][86121] Updated weights for policy 0, policy_version 69680 (0.0007) +[2023-10-09 14:57:36,783][86121] Updated weights for policy 0, policy_version 69690 (0.0007) +[2023-10-09 14:57:37,661][86122] Updated weights for policy 1, policy_version 69960 (0.0009) +[2023-10-09 14:57:38,036][86122] Updated weights for policy 1, policy_version 69970 (0.0008) +[2023-10-09 14:57:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 142999552. Throughput: 0: 1833.2, 1: 1822.1. Samples: 35763648. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:57:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:57:38,400][86122] Updated weights for policy 1, policy_version 69980 (0.0008) +[2023-10-09 14:57:40,548][86121] Updated weights for policy 0, policy_version 69700 (0.0008) +[2023-10-09 14:57:40,921][86121] Updated weights for policy 0, policy_version 69710 (0.0009) +[2023-10-09 14:57:41,278][86121] Updated weights for policy 0, policy_version 69720 (0.0008) +[2023-10-09 14:57:42,035][86122] Updated weights for policy 1, policy_version 69990 (0.0009) +[2023-10-09 14:57:42,388][86122] Updated weights for policy 1, policy_version 70000 (0.0011) +[2023-10-09 14:57:42,750][86122] Updated weights for policy 1, policy_version 70010 (0.0011) +[2023-10-09 14:57:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 143097856. Throughput: 0: 1831.0, 1: 1827.5. Samples: 35774910. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:57:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:57:44,891][86121] Updated weights for policy 0, policy_version 69730 (0.0008) +[2023-10-09 14:57:45,252][86121] Updated weights for policy 0, policy_version 69740 (0.0009) +[2023-10-09 14:57:45,617][86121] Updated weights for policy 0, policy_version 69750 (0.0008) +[2023-10-09 14:57:45,980][86121] Updated weights for policy 0, policy_version 69760 (0.0007) +[2023-10-09 14:57:46,410][86122] Updated weights for policy 1, policy_version 70020 (0.0007) +[2023-10-09 14:57:46,773][86122] Updated weights for policy 1, policy_version 70030 (0.0009) +[2023-10-09 14:57:47,138][86122] Updated weights for policy 1, policy_version 70040 (0.0008) +[2023-10-09 14:57:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143163392. Throughput: 0: 1839.1, 1: 1825.9. Samples: 35796634. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:57:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:57:49,584][86121] Updated weights for policy 0, policy_version 69770 (0.0010) +[2023-10-09 14:57:49,954][86121] Updated weights for policy 0, policy_version 69780 (0.0009) +[2023-10-09 14:57:50,316][86121] Updated weights for policy 0, policy_version 69790 (0.0011) +[2023-10-09 14:57:50,741][86122] Updated weights for policy 1, policy_version 70050 (0.0009) +[2023-10-09 14:57:51,100][86122] Updated weights for policy 1, policy_version 70060 (0.0007) +[2023-10-09 14:57:51,452][86122] Updated weights for policy 1, policy_version 70070 (0.0007) +[2023-10-09 14:57:51,815][86122] Updated weights for policy 1, policy_version 70080 (0.0008) +[2023-10-09 14:57:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143228928. Throughput: 0: 1832.1, 1: 1833.6. Samples: 35818834. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:57:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:57:54,182][86121] Updated weights for policy 0, policy_version 69800 (0.0009) +[2023-10-09 14:57:54,538][86121] Updated weights for policy 0, policy_version 69810 (0.0008) +[2023-10-09 14:57:54,906][86121] Updated weights for policy 0, policy_version 69820 (0.0008) +[2023-10-09 14:57:55,442][86122] Updated weights for policy 1, policy_version 70090 (0.0008) +[2023-10-09 14:57:55,802][86122] Updated weights for policy 1, policy_version 70100 (0.0009) +[2023-10-09 14:57:56,170][86122] Updated weights for policy 1, policy_version 70110 (0.0009) +[2023-10-09 14:57:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 143294464. Throughput: 0: 1834.1, 1: 1827.3. Samples: 35829496. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:57:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:57:58,571][86121] Updated weights for policy 0, policy_version 69830 (0.0008) +[2023-10-09 14:57:58,936][86121] Updated weights for policy 0, policy_version 69840 (0.0007) +[2023-10-09 14:57:59,301][86121] Updated weights for policy 0, policy_version 69850 (0.0009) +[2023-10-09 14:57:59,820][86122] Updated weights for policy 1, policy_version 70120 (0.0008) +[2023-10-09 14:58:00,185][86122] Updated weights for policy 1, policy_version 70130 (0.0008) +[2023-10-09 14:58:00,548][86122] Updated weights for policy 1, policy_version 70140 (0.0008) +[2023-10-09 14:58:02,906][86121] Updated weights for policy 0, policy_version 69860 (0.0009) +[2023-10-09 14:58:03,277][86121] Updated weights for policy 0, policy_version 69870 (0.0009) +[2023-10-09 14:58:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 143360000. Throughput: 0: 1824.5, 1: 1838.2. Samples: 35851720. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:58:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:58:03,640][86121] Updated weights for policy 0, policy_version 69880 (0.0007) +[2023-10-09 14:58:04,258][86122] Updated weights for policy 1, policy_version 70150 (0.0008) +[2023-10-09 14:58:04,619][86122] Updated weights for policy 1, policy_version 70160 (0.0009) +[2023-10-09 14:58:04,986][86122] Updated weights for policy 1, policy_version 70170 (0.0009) +[2023-10-09 14:58:07,298][86121] Updated weights for policy 0, policy_version 69890 (0.0008) +[2023-10-09 14:58:07,665][86121] Updated weights for policy 0, policy_version 69900 (0.0008) +[2023-10-09 14:58:08,050][86121] Updated weights for policy 0, policy_version 69910 (0.0010) +[2023-10-09 14:58:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 143425536. Throughput: 0: 1817.9, 1: 1833.7. Samples: 35873590. Policy #0 lag: (min: 31.0, avg: 38.2, max: 63.0) +[2023-10-09 14:58:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:58:08,408][86121] Updated weights for policy 0, policy_version 69920 (0.0010) +[2023-10-09 14:58:08,708][86122] Updated weights for policy 1, policy_version 70180 (0.0008) +[2023-10-09 14:58:09,065][86122] Updated weights for policy 1, policy_version 70190 (0.0008) +[2023-10-09 14:58:09,421][86122] Updated weights for policy 1, policy_version 70200 (0.0007) +[2023-10-09 14:58:12,111][86121] Updated weights for policy 0, policy_version 69930 (0.0010) +[2023-10-09 14:58:12,475][86121] Updated weights for policy 0, policy_version 69940 (0.0010) +[2023-10-09 14:58:12,850][86121] Updated weights for policy 0, policy_version 69950 (0.0008) +[2023-10-09 14:58:13,092][86122] Updated weights for policy 1, policy_version 70210 (0.0009) +[2023-10-09 14:58:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143523840. Throughput: 0: 1823.6, 1: 1831.7. Samples: 35884418. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:58:13,458][86122] Updated weights for policy 1, policy_version 70220 (0.0008) +[2023-10-09 14:58:13,823][86122] Updated weights for policy 1, policy_version 70230 (0.0009) +[2023-10-09 14:58:14,179][86122] Updated weights for policy 1, policy_version 70240 (0.0010) +[2023-10-09 14:58:16,540][86121] Updated weights for policy 0, policy_version 69960 (0.0007) +[2023-10-09 14:58:16,908][86121] Updated weights for policy 0, policy_version 69970 (0.0009) +[2023-10-09 14:58:17,271][86121] Updated weights for policy 0, policy_version 69980 (0.0009) +[2023-10-09 14:58:17,821][86122] Updated weights for policy 1, policy_version 70250 (0.0008) +[2023-10-09 14:58:18,181][86122] Updated weights for policy 1, policy_version 70260 (0.0010) +[2023-10-09 14:58:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 143589376. Throughput: 0: 1814.4, 1: 1834.9. Samples: 35906360. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:58:18,550][86122] Updated weights for policy 1, policy_version 70270 (0.0009) +[2023-10-09 14:58:20,906][86121] Updated weights for policy 0, policy_version 69990 (0.0010) +[2023-10-09 14:58:21,271][86121] Updated weights for policy 0, policy_version 70000 (0.0008) +[2023-10-09 14:58:21,635][86121] Updated weights for policy 0, policy_version 70010 (0.0007) +[2023-10-09 14:58:22,413][86122] Updated weights for policy 1, policy_version 70280 (0.0009) +[2023-10-09 14:58:22,774][86122] Updated weights for policy 1, policy_version 70290 (0.0008) +[2023-10-09 14:58:23,145][86122] Updated weights for policy 1, policy_version 70300 (0.0009) +[2023-10-09 14:58:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 143687680. Throughput: 0: 1818.2, 1: 1821.4. Samples: 35927430. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:58:25,560][86121] Updated weights for policy 0, policy_version 70020 (0.0009) +[2023-10-09 14:58:25,930][86121] Updated weights for policy 0, policy_version 70030 (0.0007) +[2023-10-09 14:58:26,298][86121] Updated weights for policy 0, policy_version 70040 (0.0007) +[2023-10-09 14:58:27,061][86122] Updated weights for policy 1, policy_version 70310 (0.0008) +[2023-10-09 14:58:27,448][86122] Updated weights for policy 1, policy_version 70320 (0.0007) +[2023-10-09 14:58:27,805][86122] Updated weights for policy 1, policy_version 70330 (0.0008) +[2023-10-09 14:58:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143753216. Throughput: 0: 1816.1, 1: 1824.4. Samples: 35938734. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.960')] +[2023-10-09 14:58:29,979][86121] Updated weights for policy 0, policy_version 70050 (0.0008) +[2023-10-09 14:58:30,349][86121] Updated weights for policy 0, policy_version 70060 (0.0008) +[2023-10-09 14:58:30,707][86121] Updated weights for policy 0, policy_version 70070 (0.0007) +[2023-10-09 14:58:31,085][86121] Updated weights for policy 0, policy_version 70080 (0.0007) +[2023-10-09 14:58:31,473][86122] Updated weights for policy 1, policy_version 70340 (0.0007) +[2023-10-09 14:58:31,837][86122] Updated weights for policy 1, policy_version 70350 (0.0010) +[2023-10-09 14:58:32,190][86122] Updated weights for policy 1, policy_version 70360 (0.0007) +[2023-10-09 14:58:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143818752. Throughput: 0: 1813.2, 1: 1816.8. Samples: 35959984. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:58:34,898][86121] Updated weights for policy 0, policy_version 70090 (0.0009) +[2023-10-09 14:58:35,277][86121] Updated weights for policy 0, policy_version 70100 (0.0010) +[2023-10-09 14:58:35,641][86121] Updated weights for policy 0, policy_version 70110 (0.0011) +[2023-10-09 14:58:35,911][86122] Updated weights for policy 1, policy_version 70370 (0.0007) +[2023-10-09 14:58:36,268][86122] Updated weights for policy 1, policy_version 70380 (0.0008) +[2023-10-09 14:58:36,637][86122] Updated weights for policy 1, policy_version 70390 (0.0011) +[2023-10-09 14:58:36,984][86122] Updated weights for policy 1, policy_version 70400 (0.0010) +[2023-10-09 14:58:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 143884288. Throughput: 0: 1819.1, 1: 1815.1. Samples: 35982372. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.960')] +[2023-10-09 14:58:39,270][86121] Updated weights for policy 0, policy_version 70120 (0.0010) +[2023-10-09 14:58:39,638][86121] Updated weights for policy 0, policy_version 70130 (0.0008) +[2023-10-09 14:58:40,006][86121] Updated weights for policy 0, policy_version 70140 (0.0007) +[2023-10-09 14:58:40,638][86122] Updated weights for policy 1, policy_version 70410 (0.0008) +[2023-10-09 14:58:40,998][86122] Updated weights for policy 1, policy_version 70420 (0.0010) +[2023-10-09 14:58:41,361][86122] Updated weights for policy 1, policy_version 70430 (0.0009) +[2023-10-09 14:58:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 143949824. Throughput: 0: 1815.5, 1: 1821.1. Samples: 35993142. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:58:43,669][86121] Updated weights for policy 0, policy_version 70150 (0.0008) +[2023-10-09 14:58:44,037][86121] Updated weights for policy 0, policy_version 70160 (0.0007) +[2023-10-09 14:58:44,399][86121] Updated weights for policy 0, policy_version 70170 (0.0007) +[2023-10-09 14:58:45,014][86122] Updated weights for policy 1, policy_version 70440 (0.0010) +[2023-10-09 14:58:45,373][86122] Updated weights for policy 1, policy_version 70450 (0.0010) +[2023-10-09 14:58:45,738][86122] Updated weights for policy 1, policy_version 70460 (0.0011) +[2023-10-09 14:58:47,925][86121] Updated weights for policy 0, policy_version 70180 (0.0008) +[2023-10-09 14:58:48,286][86121] Updated weights for policy 0, policy_version 70190 (0.0009) +[2023-10-09 14:58:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144015360. Throughput: 0: 1822.7, 1: 1810.7. Samples: 36015222. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:58:48,655][86121] Updated weights for policy 0, policy_version 70200 (0.0007) +[2023-10-09 14:58:49,425][86122] Updated weights for policy 1, policy_version 70470 (0.0009) +[2023-10-09 14:58:49,783][86122] Updated weights for policy 1, policy_version 70480 (0.0009) +[2023-10-09 14:58:50,142][86122] Updated weights for policy 1, policy_version 70490 (0.0008) +[2023-10-09 14:58:52,374][86121] Updated weights for policy 0, policy_version 70210 (0.0007) +[2023-10-09 14:58:52,733][86121] Updated weights for policy 0, policy_version 70220 (0.0008) +[2023-10-09 14:58:53,096][86121] Updated weights for policy 0, policy_version 70230 (0.0007) +[2023-10-09 14:58:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 144080896. Throughput: 0: 1824.1, 1: 1809.9. Samples: 36037120. Policy #0 lag: (min: 14.0, avg: 14.4, max: 29.0) +[2023-10-09 14:58:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:58:53,460][86121] Updated weights for policy 0, policy_version 70240 (0.0008) +[2023-10-09 14:58:53,851][86122] Updated weights for policy 1, policy_version 70500 (0.0009) +[2023-10-09 14:58:54,219][86122] Updated weights for policy 1, policy_version 70510 (0.0008) +[2023-10-09 14:58:54,585][86122] Updated weights for policy 1, policy_version 70520 (0.0007) +[2023-10-09 14:58:57,160][86121] Updated weights for policy 0, policy_version 70250 (0.0007) +[2023-10-09 14:58:57,531][86121] Updated weights for policy 0, policy_version 70260 (0.0008) +[2023-10-09 14:58:57,897][86121] Updated weights for policy 0, policy_version 70270 (0.0008) +[2023-10-09 14:58:58,300][86122] Updated weights for policy 1, policy_version 70530 (0.0009) +[2023-10-09 14:58:58,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144179200. Throughput: 0: 1824.1, 1: 1812.3. Samples: 36048056. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:58:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:58:58,661][86122] Updated weights for policy 1, policy_version 70540 (0.0007) +[2023-10-09 14:58:59,029][86122] Updated weights for policy 1, policy_version 70550 (0.0007) +[2023-10-09 14:58:59,384][86122] Updated weights for policy 1, policy_version 70560 (0.0007) +[2023-10-09 14:59:01,670][86121] Updated weights for policy 0, policy_version 70280 (0.0009) +[2023-10-09 14:59:02,045][86121] Updated weights for policy 0, policy_version 70290 (0.0009) +[2023-10-09 14:59:02,405][86121] Updated weights for policy 0, policy_version 70300 (0.0011) +[2023-10-09 14:59:03,032][86122] Updated weights for policy 1, policy_version 70570 (0.0007) +[2023-10-09 14:59:03,395][86122] Updated weights for policy 1, policy_version 70580 (0.0007) +[2023-10-09 14:59:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144244736. Throughput: 0: 1829.2, 1: 1812.1. Samples: 36070222. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:59:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:59:03,756][86122] Updated weights for policy 1, policy_version 70590 (0.0008) +[2023-10-09 14:59:05,966][86121] Updated weights for policy 0, policy_version 70310 (0.0011) +[2023-10-09 14:59:06,330][86121] Updated weights for policy 0, policy_version 70320 (0.0010) +[2023-10-09 14:59:06,701][86121] Updated weights for policy 0, policy_version 70330 (0.0009) +[2023-10-09 14:59:07,298][86122] Updated weights for policy 1, policy_version 70600 (0.0009) +[2023-10-09 14:59:07,657][86122] Updated weights for policy 1, policy_version 70610 (0.0008) +[2023-10-09 14:59:08,028][86122] Updated weights for policy 1, policy_version 70620 (0.0007) +[2023-10-09 14:59:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 144343040. Throughput: 0: 1830.5, 1: 1818.0. Samples: 36091616. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:59:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:59:10,190][86121] Updated weights for policy 0, policy_version 70340 (0.0007) +[2023-10-09 14:59:10,551][86121] Updated weights for policy 0, policy_version 70350 (0.0007) +[2023-10-09 14:59:10,922][86121] Updated weights for policy 0, policy_version 70360 (0.0008) +[2023-10-09 14:59:11,947][86122] Updated weights for policy 1, policy_version 70630 (0.0009) +[2023-10-09 14:59:12,320][86122] Updated weights for policy 1, policy_version 70640 (0.0007) +[2023-10-09 14:59:12,671][86122] Updated weights for policy 1, policy_version 70650 (0.0007) +[2023-10-09 14:59:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144408576. Throughput: 0: 1828.7, 1: 1825.9. Samples: 36103188. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:59:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:59:14,764][86121] Updated weights for policy 0, policy_version 70370 (0.0009) +[2023-10-09 14:59:15,139][86121] Updated weights for policy 0, policy_version 70380 (0.0008) +[2023-10-09 14:59:15,509][86121] Updated weights for policy 0, policy_version 70390 (0.0009) +[2023-10-09 14:59:15,874][86121] Updated weights for policy 0, policy_version 70400 (0.0008) +[2023-10-09 14:59:16,351][86122] Updated weights for policy 1, policy_version 70660 (0.0008) +[2023-10-09 14:59:16,713][86122] Updated weights for policy 1, policy_version 70670 (0.0007) +[2023-10-09 14:59:17,069][86122] Updated weights for policy 1, policy_version 70680 (0.0007) +[2023-10-09 14:59:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144474112. Throughput: 0: 1838.2, 1: 1827.0. Samples: 36124920. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:59:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 14:59:19,189][86121] Updated weights for policy 0, policy_version 70410 (0.0007) +[2023-10-09 14:59:19,563][86121] Updated weights for policy 0, policy_version 70420 (0.0008) +[2023-10-09 14:59:19,925][86121] Updated weights for policy 0, policy_version 70430 (0.0009) +[2023-10-09 14:59:20,691][86122] Updated weights for policy 1, policy_version 70690 (0.0009) +[2023-10-09 14:59:21,054][86122] Updated weights for policy 1, policy_version 70700 (0.0009) +[2023-10-09 14:59:21,424][86122] Updated weights for policy 1, policy_version 70710 (0.0008) +[2023-10-09 14:59:21,781][86122] Updated weights for policy 1, policy_version 70720 (0.0008) +[2023-10-09 14:59:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 144539648. Throughput: 0: 1836.6, 1: 1829.3. Samples: 36147336. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:59:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:59:23,406][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000070720_72417280.pth... +[2023-10-09 14:59:23,443][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000069024_70680576.pth +[2023-10-09 14:59:23,674][86121] Updated weights for policy 0, policy_version 70440 (0.0008) +[2023-10-09 14:59:24,029][86121] Updated weights for policy 0, policy_version 70450 (0.0009) +[2023-10-09 14:59:24,394][86121] Updated weights for policy 0, policy_version 70460 (0.0008) +[2023-10-09 14:59:24,539][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000070464_72155136.pth... +[2023-10-09 14:59:24,569][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000068736_70385664.pth +[2023-10-09 14:59:25,383][86122] Updated weights for policy 1, policy_version 70730 (0.0010) +[2023-10-09 14:59:25,736][86122] Updated weights for policy 1, policy_version 70740 (0.0009) +[2023-10-09 14:59:26,100][86122] Updated weights for policy 1, policy_version 70750 (0.0008) +[2023-10-09 14:59:28,259][86121] Updated weights for policy 0, policy_version 70470 (0.0007) +[2023-10-09 14:59:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 144605184. Throughput: 0: 1837.9, 1: 1822.9. Samples: 36157878. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:59:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:59:28,624][86121] Updated weights for policy 0, policy_version 70480 (0.0008) +[2023-10-09 14:59:28,991][86121] Updated weights for policy 0, policy_version 70490 (0.0011) +[2023-10-09 14:59:29,838][86122] Updated weights for policy 1, policy_version 70760 (0.0010) +[2023-10-09 14:59:30,206][86122] Updated weights for policy 1, policy_version 70770 (0.0008) +[2023-10-09 14:59:30,574][86122] Updated weights for policy 1, policy_version 70780 (0.0008) +[2023-10-09 14:59:32,551][86121] Updated weights for policy 0, policy_version 70500 (0.0009) +[2023-10-09 14:59:32,916][86121] Updated weights for policy 0, policy_version 70510 (0.0010) +[2023-10-09 14:59:33,281][86121] Updated weights for policy 0, policy_version 70520 (0.0007) +[2023-10-09 14:59:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 144670720. Throughput: 0: 1831.1, 1: 1832.4. Samples: 36180080. Policy #0 lag: (min: 31.0, avg: 38.7, max: 63.0) +[2023-10-09 14:59:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:59:34,194][86122] Updated weights for policy 1, policy_version 70790 (0.0008) +[2023-10-09 14:59:34,550][86122] Updated weights for policy 1, policy_version 70800 (0.0007) +[2023-10-09 14:59:34,910][86122] Updated weights for policy 1, policy_version 70810 (0.0009) +[2023-10-09 14:59:37,095][86121] Updated weights for policy 0, policy_version 70530 (0.0009) +[2023-10-09 14:59:37,458][86121] Updated weights for policy 0, policy_version 70540 (0.0009) +[2023-10-09 14:59:37,827][86121] Updated weights for policy 0, policy_version 70550 (0.0008) +[2023-10-09 14:59:38,196][86121] Updated weights for policy 0, policy_version 70560 (0.0008) +[2023-10-09 14:59:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 144769024. Throughput: 0: 1824.3, 1: 1842.3. Samples: 36202118. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 14:59:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:59:38,474][86122] Updated weights for policy 1, policy_version 70820 (0.0010) +[2023-10-09 14:59:38,840][86122] Updated weights for policy 1, policy_version 70830 (0.0009) +[2023-10-09 14:59:39,207][86122] Updated weights for policy 1, policy_version 70840 (0.0010) +[2023-10-09 14:59:41,916][86121] Updated weights for policy 0, policy_version 70570 (0.0008) +[2023-10-09 14:59:42,275][86121] Updated weights for policy 0, policy_version 70580 (0.0009) +[2023-10-09 14:59:42,640][86121] Updated weights for policy 0, policy_version 70590 (0.0009) +[2023-10-09 14:59:42,874][86122] Updated weights for policy 1, policy_version 70850 (0.0010) +[2023-10-09 14:59:43,235][86122] Updated weights for policy 1, policy_version 70860 (0.0008) +[2023-10-09 14:59:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 144834560. Throughput: 0: 1833.6, 1: 1838.3. Samples: 36213290. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 14:59:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:59:43,604][86122] Updated weights for policy 1, policy_version 70870 (0.0009) +[2023-10-09 14:59:43,967][86122] Updated weights for policy 1, policy_version 70880 (0.0007) +[2023-10-09 14:59:46,328][86121] Updated weights for policy 0, policy_version 70600 (0.0008) +[2023-10-09 14:59:46,688][86121] Updated weights for policy 0, policy_version 70610 (0.0009) +[2023-10-09 14:59:47,058][86121] Updated weights for policy 0, policy_version 70620 (0.0007) +[2023-10-09 14:59:47,545][86122] Updated weights for policy 1, policy_version 70890 (0.0008) +[2023-10-09 14:59:47,905][86122] Updated weights for policy 1, policy_version 70900 (0.0008) +[2023-10-09 14:59:48,267][86122] Updated weights for policy 1, policy_version 70910 (0.0009) +[2023-10-09 14:59:48,397][85186] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 144932864. Throughput: 0: 1824.2, 1: 1844.2. Samples: 36235302. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 14:59:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 14:59:50,736][86121] Updated weights for policy 0, policy_version 70630 (0.0008) +[2023-10-09 14:59:51,102][86121] Updated weights for policy 0, policy_version 70640 (0.0009) +[2023-10-09 14:59:51,479][86121] Updated weights for policy 0, policy_version 70650 (0.0008) +[2023-10-09 14:59:52,106][86122] Updated weights for policy 1, policy_version 70920 (0.0010) +[2023-10-09 14:59:52,477][86122] Updated weights for policy 1, policy_version 70930 (0.0008) +[2023-10-09 14:59:52,837][86122] Updated weights for policy 1, policy_version 70940 (0.0008) +[2023-10-09 14:59:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 144998400. Throughput: 0: 1834.5, 1: 1832.3. Samples: 36256620. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 14:59:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:59:54,952][86121] Updated weights for policy 0, policy_version 70660 (0.0009) +[2023-10-09 14:59:55,322][86121] Updated weights for policy 0, policy_version 70670 (0.0008) +[2023-10-09 14:59:55,680][86121] Updated weights for policy 0, policy_version 70680 (0.0008) +[2023-10-09 14:59:56,577][86122] Updated weights for policy 1, policy_version 70950 (0.0007) +[2023-10-09 14:59:56,962][86122] Updated weights for policy 1, policy_version 70960 (0.0009) +[2023-10-09 14:59:57,322][86122] Updated weights for policy 1, policy_version 70970 (0.0012) +[2023-10-09 14:59:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145063936. Throughput: 0: 1830.9, 1: 1839.5. Samples: 36268354. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 14:59:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 14:59:59,417][86121] Updated weights for policy 0, policy_version 70690 (0.0007) +[2023-10-09 14:59:59,788][86121] Updated weights for policy 0, policy_version 70700 (0.0007) +[2023-10-09 15:00:00,153][86121] Updated weights for policy 0, policy_version 70710 (0.0008) +[2023-10-09 15:00:00,514][86121] Updated weights for policy 0, policy_version 70720 (0.0009) +[2023-10-09 15:00:01,059][86122] Updated weights for policy 1, policy_version 70980 (0.0012) +[2023-10-09 15:00:01,423][86122] Updated weights for policy 1, policy_version 70990 (0.0009) +[2023-10-09 15:00:01,785][86122] Updated weights for policy 1, policy_version 71000 (0.0007) +[2023-10-09 15:00:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 145129472. Throughput: 0: 1837.4, 1: 1827.4. Samples: 36289838. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 15:00:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:00:04,154][86121] Updated weights for policy 0, policy_version 70730 (0.0009) +[2023-10-09 15:00:04,527][86121] Updated weights for policy 0, policy_version 70740 (0.0008) +[2023-10-09 15:00:04,891][86121] Updated weights for policy 0, policy_version 70750 (0.0009) +[2023-10-09 15:00:05,346][86122] Updated weights for policy 1, policy_version 71010 (0.0009) +[2023-10-09 15:00:05,701][86122] Updated weights for policy 1, policy_version 71020 (0.0010) +[2023-10-09 15:00:06,064][86122] Updated weights for policy 1, policy_version 71030 (0.0011) +[2023-10-09 15:00:06,421][86122] Updated weights for policy 1, policy_version 71040 (0.0008) +[2023-10-09 15:00:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 145195008. Throughput: 0: 1840.1, 1: 1833.8. Samples: 36312664. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 15:00:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:00:08,451][86121] Updated weights for policy 0, policy_version 70760 (0.0009) +[2023-10-09 15:00:08,822][86121] Updated weights for policy 0, policy_version 70770 (0.0008) +[2023-10-09 15:00:09,197][86121] Updated weights for policy 0, policy_version 70780 (0.0009) +[2023-10-09 15:00:09,904][86122] Updated weights for policy 1, policy_version 71050 (0.0007) +[2023-10-09 15:00:10,276][86122] Updated weights for policy 1, policy_version 71060 (0.0009) +[2023-10-09 15:00:10,635][86122] Updated weights for policy 1, policy_version 71070 (0.0009) +[2023-10-09 15:00:12,998][86121] Updated weights for policy 0, policy_version 70790 (0.0008) +[2023-10-09 15:00:13,367][86121] Updated weights for policy 0, policy_version 70800 (0.0009) +[2023-10-09 15:00:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 145260544. Throughput: 0: 1840.8, 1: 1825.5. Samples: 36322858. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 15:00:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:00:13,742][86121] Updated weights for policy 0, policy_version 70810 (0.0009) +[2023-10-09 15:00:14,186][86122] Updated weights for policy 1, policy_version 71080 (0.0008) +[2023-10-09 15:00:14,555][86122] Updated weights for policy 1, policy_version 71090 (0.0007) +[2023-10-09 15:00:14,926][86122] Updated weights for policy 1, policy_version 71100 (0.0009) +[2023-10-09 15:00:17,297][86121] Updated weights for policy 0, policy_version 70820 (0.0010) +[2023-10-09 15:00:17,668][86121] Updated weights for policy 0, policy_version 70830 (0.0010) +[2023-10-09 15:00:18,028][86121] Updated weights for policy 0, policy_version 70840 (0.0007) +[2023-10-09 15:00:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145358848. Throughput: 0: 1844.5, 1: 1841.3. Samples: 36345940. Policy #0 lag: (min: 31.0, avg: 34.4, max: 63.0) +[2023-10-09 15:00:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:00:18,770][86122] Updated weights for policy 1, policy_version 71110 (0.0010) +[2023-10-09 15:00:19,132][86122] Updated weights for policy 1, policy_version 71120 (0.0007) +[2023-10-09 15:00:19,493][86122] Updated weights for policy 1, policy_version 71130 (0.0008) +[2023-10-09 15:00:21,676][86121] Updated weights for policy 0, policy_version 70850 (0.0008) +[2023-10-09 15:00:22,048][86121] Updated weights for policy 0, policy_version 70860 (0.0009) +[2023-10-09 15:00:22,415][86121] Updated weights for policy 0, policy_version 70870 (0.0008) +[2023-10-09 15:00:22,786][86121] Updated weights for policy 0, policy_version 70880 (0.0008) +[2023-10-09 15:00:23,194][86122] Updated weights for policy 1, policy_version 71140 (0.0008) +[2023-10-09 15:00:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145424384. Throughput: 0: 1834.7, 1: 1835.7. Samples: 36367286. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:00:23,557][86122] Updated weights for policy 1, policy_version 71150 (0.0008) +[2023-10-09 15:00:23,920][86122] Updated weights for policy 1, policy_version 71160 (0.0009) +[2023-10-09 15:00:26,415][86121] Updated weights for policy 0, policy_version 70890 (0.0009) +[2023-10-09 15:00:26,785][86121] Updated weights for policy 0, policy_version 70900 (0.0008) +[2023-10-09 15:00:27,150][86121] Updated weights for policy 0, policy_version 70910 (0.0008) +[2023-10-09 15:00:27,630][86122] Updated weights for policy 1, policy_version 71170 (0.0007) +[2023-10-09 15:00:27,996][86122] Updated weights for policy 1, policy_version 71180 (0.0008) +[2023-10-09 15:00:28,361][86122] Updated weights for policy 1, policy_version 71190 (0.0008) +[2023-10-09 15:00:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 145489920. Throughput: 0: 1839.6, 1: 1838.1. Samples: 36378784. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:00:28,715][86122] Updated weights for policy 1, policy_version 71200 (0.0009) +[2023-10-09 15:00:30,816][86121] Updated weights for policy 0, policy_version 70920 (0.0010) +[2023-10-09 15:00:31,177][86121] Updated weights for policy 0, policy_version 70930 (0.0009) +[2023-10-09 15:00:31,547][86121] Updated weights for policy 0, policy_version 70940 (0.0009) +[2023-10-09 15:00:32,526][86122] Updated weights for policy 1, policy_version 71210 (0.0008) +[2023-10-09 15:00:32,883][86122] Updated weights for policy 1, policy_version 71220 (0.0008) +[2023-10-09 15:00:33,245][86122] Updated weights for policy 1, policy_version 71230 (0.0009) +[2023-10-09 15:00:33,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 145588224. Throughput: 0: 1833.8, 1: 1825.6. Samples: 36399976. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:00:35,255][86121] Updated weights for policy 0, policy_version 70950 (0.0007) +[2023-10-09 15:00:35,621][86121] Updated weights for policy 0, policy_version 70960 (0.0009) +[2023-10-09 15:00:35,987][86121] Updated weights for policy 0, policy_version 70970 (0.0008) +[2023-10-09 15:00:36,995][86122] Updated weights for policy 1, policy_version 71240 (0.0010) +[2023-10-09 15:00:37,354][86122] Updated weights for policy 1, policy_version 71250 (0.0011) +[2023-10-09 15:00:37,722][86122] Updated weights for policy 1, policy_version 71260 (0.0009) +[2023-10-09 15:00:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 145653760. Throughput: 0: 1839.7, 1: 1822.9. Samples: 36421440. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:00:39,712][86121] Updated weights for policy 0, policy_version 70980 (0.0007) +[2023-10-09 15:00:40,080][86121] Updated weights for policy 0, policy_version 70990 (0.0009) +[2023-10-09 15:00:40,448][86121] Updated weights for policy 0, policy_version 71000 (0.0009) +[2023-10-09 15:00:41,290][86122] Updated weights for policy 1, policy_version 71270 (0.0008) +[2023-10-09 15:00:41,656][86122] Updated weights for policy 1, policy_version 71280 (0.0008) +[2023-10-09 15:00:42,017][86122] Updated weights for policy 1, policy_version 71290 (0.0008) +[2023-10-09 15:00:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 145719296. Throughput: 0: 1825.5, 1: 1826.6. Samples: 36432698. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:00:44,114][86121] Updated weights for policy 0, policy_version 71010 (0.0010) +[2023-10-09 15:00:44,478][86121] Updated weights for policy 0, policy_version 71020 (0.0009) +[2023-10-09 15:00:44,835][86121] Updated weights for policy 0, policy_version 71030 (0.0009) +[2023-10-09 15:00:45,207][86121] Updated weights for policy 0, policy_version 71040 (0.0008) +[2023-10-09 15:00:45,730][86122] Updated weights for policy 1, policy_version 71300 (0.0009) +[2023-10-09 15:00:46,118][86122] Updated weights for policy 1, policy_version 71310 (0.0008) +[2023-10-09 15:00:46,481][86122] Updated weights for policy 1, policy_version 71320 (0.0009) +[2023-10-09 15:00:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 145784832. Throughput: 0: 1829.4, 1: 1820.6. Samples: 36454088. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:00:48,975][86121] Updated weights for policy 0, policy_version 71050 (0.0008) +[2023-10-09 15:00:49,338][86121] Updated weights for policy 0, policy_version 71060 (0.0008) +[2023-10-09 15:00:49,706][86121] Updated weights for policy 0, policy_version 71070 (0.0007) +[2023-10-09 15:00:50,203][86122] Updated weights for policy 1, policy_version 71330 (0.0010) +[2023-10-09 15:00:50,561][86122] Updated weights for policy 1, policy_version 71340 (0.0011) +[2023-10-09 15:00:50,932][86122] Updated weights for policy 1, policy_version 71350 (0.0011) +[2023-10-09 15:00:51,288][86122] Updated weights for policy 1, policy_version 71360 (0.0010) +[2023-10-09 15:00:53,353][86121] Updated weights for policy 0, policy_version 71080 (0.0007) +[2023-10-09 15:00:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 145850368. Throughput: 0: 1824.0, 1: 1823.2. Samples: 36476788. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:00:53,718][86121] Updated weights for policy 0, policy_version 71090 (0.0008) +[2023-10-09 15:00:54,086][86121] Updated weights for policy 0, policy_version 71100 (0.0010) +[2023-10-09 15:00:55,093][86122] Updated weights for policy 1, policy_version 71370 (0.0010) +[2023-10-09 15:00:55,457][86122] Updated weights for policy 1, policy_version 71380 (0.0008) +[2023-10-09 15:00:55,833][86122] Updated weights for policy 1, policy_version 71390 (0.0009) +[2023-10-09 15:00:57,761][86121] Updated weights for policy 0, policy_version 71110 (0.0008) +[2023-10-09 15:00:58,123][86121] Updated weights for policy 0, policy_version 71120 (0.0008) +[2023-10-09 15:00:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 145915904. Throughput: 0: 1825.9, 1: 1821.0. Samples: 36486970. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:00:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:00:58,480][86121] Updated weights for policy 0, policy_version 71130 (0.0007) +[2023-10-09 15:00:59,457][86122] Updated weights for policy 1, policy_version 71400 (0.0008) +[2023-10-09 15:00:59,821][86122] Updated weights for policy 1, policy_version 71410 (0.0007) +[2023-10-09 15:01:00,172][86122] Updated weights for policy 1, policy_version 71420 (0.0007) +[2023-10-09 15:01:02,198][86121] Updated weights for policy 0, policy_version 71140 (0.0008) +[2023-10-09 15:01:02,569][86121] Updated weights for policy 0, policy_version 71150 (0.0008) +[2023-10-09 15:01:02,936][86121] Updated weights for policy 0, policy_version 71160 (0.0008) +[2023-10-09 15:01:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146014208. Throughput: 0: 1822.1, 1: 1818.2. Samples: 36509756. Policy #0 lag: (min: 2.0, avg: 4.5, max: 22.0) +[2023-10-09 15:01:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:01:03,617][86122] Updated weights for policy 1, policy_version 71430 (0.0008) +[2023-10-09 15:01:03,974][86122] Updated weights for policy 1, policy_version 71440 (0.0007) +[2023-10-09 15:01:04,331][86122] Updated weights for policy 1, policy_version 71450 (0.0008) +[2023-10-09 15:01:06,524][86121] Updated weights for policy 0, policy_version 71170 (0.0012) +[2023-10-09 15:01:06,900][86121] Updated weights for policy 0, policy_version 71180 (0.0010) +[2023-10-09 15:01:07,263][86121] Updated weights for policy 0, policy_version 71190 (0.0008) +[2023-10-09 15:01:07,633][86121] Updated weights for policy 0, policy_version 71200 (0.0008) +[2023-10-09 15:01:08,060][86122] Updated weights for policy 1, policy_version 71460 (0.0008) +[2023-10-09 15:01:08,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 146079744. Throughput: 0: 1819.1, 1: 1825.3. Samples: 36531286. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:01:08,421][86122] Updated weights for policy 1, policy_version 71470 (0.0008) +[2023-10-09 15:01:08,773][86122] Updated weights for policy 1, policy_version 71480 (0.0009) +[2023-10-09 15:01:11,166][86121] Updated weights for policy 0, policy_version 71210 (0.0007) +[2023-10-09 15:01:11,541][86121] Updated weights for policy 0, policy_version 71220 (0.0007) +[2023-10-09 15:01:11,910][86121] Updated weights for policy 0, policy_version 71230 (0.0011) +[2023-10-09 15:01:12,499][86122] Updated weights for policy 1, policy_version 71490 (0.0009) +[2023-10-09 15:01:12,872][86122] Updated weights for policy 1, policy_version 71500 (0.0009) +[2023-10-09 15:01:13,232][86122] Updated weights for policy 1, policy_version 71510 (0.0008) +[2023-10-09 15:01:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146145280. Throughput: 0: 1817.9, 1: 1824.3. Samples: 36542686. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:01:13,603][86122] Updated weights for policy 1, policy_version 71520 (0.0009) +[2023-10-09 15:01:15,749][86121] Updated weights for policy 0, policy_version 71240 (0.0008) +[2023-10-09 15:01:16,114][86121] Updated weights for policy 0, policy_version 71250 (0.0010) +[2023-10-09 15:01:16,494][86121] Updated weights for policy 0, policy_version 71260 (0.0009) +[2023-10-09 15:01:17,289][86122] Updated weights for policy 1, policy_version 71530 (0.0009) +[2023-10-09 15:01:17,654][86122] Updated weights for policy 1, policy_version 71540 (0.0008) +[2023-10-09 15:01:18,015][86122] Updated weights for policy 1, policy_version 71550 (0.0008) +[2023-10-09 15:01:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146243584. Throughput: 0: 1822.0, 1: 1827.2. Samples: 36564188. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:01:20,314][86121] Updated weights for policy 0, policy_version 71270 (0.0009) +[2023-10-09 15:01:20,694][86121] Updated weights for policy 0, policy_version 71280 (0.0009) +[2023-10-09 15:01:21,057][86121] Updated weights for policy 0, policy_version 71290 (0.0008) +[2023-10-09 15:01:21,836][86122] Updated weights for policy 1, policy_version 71560 (0.0008) +[2023-10-09 15:01:22,199][86122] Updated weights for policy 1, policy_version 71570 (0.0008) +[2023-10-09 15:01:22,562][86122] Updated weights for policy 1, policy_version 71580 (0.0010) +[2023-10-09 15:01:23,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146309120. Throughput: 0: 1818.6, 1: 1826.9. Samples: 36585488. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:01:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000071296_73007104.pth... +[2023-10-09 15:01:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000071584_73302016.pth... +[2023-10-09 15:01:23,449][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000069600_71270400.pth +[2023-10-09 15:01:23,450][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000069856_71532544.pth +[2023-10-09 15:01:24,887][86121] Updated weights for policy 0, policy_version 71300 (0.0007) +[2023-10-09 15:01:25,260][86121] Updated weights for policy 0, policy_version 71310 (0.0008) +[2023-10-09 15:01:25,628][86121] Updated weights for policy 0, policy_version 71320 (0.0009) +[2023-10-09 15:01:26,328][86122] Updated weights for policy 1, policy_version 71590 (0.0009) +[2023-10-09 15:01:26,685][86122] Updated weights for policy 1, policy_version 71600 (0.0007) +[2023-10-09 15:01:27,041][86122] Updated weights for policy 1, policy_version 71610 (0.0008) +[2023-10-09 15:01:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146374656. Throughput: 0: 1821.1, 1: 1824.1. Samples: 36596732. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:01:29,207][86121] Updated weights for policy 0, policy_version 71330 (0.0010) +[2023-10-09 15:01:29,573][86121] Updated weights for policy 0, policy_version 71340 (0.0008) +[2023-10-09 15:01:29,945][86121] Updated weights for policy 0, policy_version 71350 (0.0008) +[2023-10-09 15:01:30,311][86121] Updated weights for policy 0, policy_version 71360 (0.0008) +[2023-10-09 15:01:30,680][86122] Updated weights for policy 1, policy_version 71620 (0.0008) +[2023-10-09 15:01:31,040][86122] Updated weights for policy 1, policy_version 71630 (0.0007) +[2023-10-09 15:01:31,409][86122] Updated weights for policy 1, policy_version 71640 (0.0008) +[2023-10-09 15:01:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 146440192. Throughput: 0: 1820.5, 1: 1826.5. Samples: 36618200. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:01:34,064][86121] Updated weights for policy 0, policy_version 71370 (0.0007) +[2023-10-09 15:01:34,424][86121] Updated weights for policy 0, policy_version 71380 (0.0007) +[2023-10-09 15:01:34,800][86121] Updated weights for policy 0, policy_version 71390 (0.0009) +[2023-10-09 15:01:35,118][86122] Updated weights for policy 1, policy_version 71650 (0.0008) +[2023-10-09 15:01:35,497][86122] Updated weights for policy 1, policy_version 71660 (0.0009) +[2023-10-09 15:01:35,860][86122] Updated weights for policy 1, policy_version 71670 (0.0007) +[2023-10-09 15:01:36,227][86122] Updated weights for policy 1, policy_version 71680 (0.0007) +[2023-10-09 15:01:38,383][86121] Updated weights for policy 0, policy_version 71400 (0.0008) +[2023-10-09 15:01:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 146505728. Throughput: 0: 1824.7, 1: 1827.3. Samples: 36641130. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:01:38,749][86121] Updated weights for policy 0, policy_version 71410 (0.0008) +[2023-10-09 15:01:39,118][86121] Updated weights for policy 0, policy_version 71420 (0.0009) +[2023-10-09 15:01:39,806][86122] Updated weights for policy 1, policy_version 71690 (0.0010) +[2023-10-09 15:01:40,166][86122] Updated weights for policy 1, policy_version 71700 (0.0009) +[2023-10-09 15:01:40,522][86122] Updated weights for policy 1, policy_version 71710 (0.0010) +[2023-10-09 15:01:42,842][86121] Updated weights for policy 0, policy_version 71430 (0.0008) +[2023-10-09 15:01:43,208][86121] Updated weights for policy 0, policy_version 71440 (0.0010) +[2023-10-09 15:01:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 146571264. Throughput: 0: 1824.2, 1: 1821.8. Samples: 36651040. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:01:43,568][86121] Updated weights for policy 0, policy_version 71450 (0.0009) +[2023-10-09 15:01:44,218][86122] Updated weights for policy 1, policy_version 71720 (0.0010) +[2023-10-09 15:01:44,570][86122] Updated weights for policy 1, policy_version 71730 (0.0011) +[2023-10-09 15:01:44,937][86122] Updated weights for policy 1, policy_version 71740 (0.0009) +[2023-10-09 15:01:47,055][86121] Updated weights for policy 0, policy_version 71460 (0.0008) +[2023-10-09 15:01:47,418][86121] Updated weights for policy 0, policy_version 71470 (0.0011) +[2023-10-09 15:01:47,779][86121] Updated weights for policy 0, policy_version 71480 (0.0010) +[2023-10-09 15:01:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146669568. Throughput: 0: 1830.0, 1: 1826.6. Samples: 36674300. Policy #0 lag: (min: 0.0, avg: 27.7, max: 32.0) +[2023-10-09 15:01:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:01:48,594][86122] Updated weights for policy 1, policy_version 71750 (0.0008) +[2023-10-09 15:01:48,956][86122] Updated weights for policy 1, policy_version 71760 (0.0008) +[2023-10-09 15:01:49,316][86122] Updated weights for policy 1, policy_version 71770 (0.0008) +[2023-10-09 15:01:51,381][86121] Updated weights for policy 0, policy_version 71490 (0.0010) +[2023-10-09 15:01:51,737][86121] Updated weights for policy 0, policy_version 71500 (0.0008) +[2023-10-09 15:01:52,105][86121] Updated weights for policy 0, policy_version 71510 (0.0007) +[2023-10-09 15:01:52,463][86121] Updated weights for policy 0, policy_version 71520 (0.0007) +[2023-10-09 15:01:53,101][86122] Updated weights for policy 1, policy_version 71780 (0.0007) +[2023-10-09 15:01:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146735104. Throughput: 0: 1834.3, 1: 1822.6. Samples: 36695844. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:01:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:01:53,466][86122] Updated weights for policy 1, policy_version 71790 (0.0008) +[2023-10-09 15:01:53,821][86122] Updated weights for policy 1, policy_version 71800 (0.0007) +[2023-10-09 15:01:56,100][86121] Updated weights for policy 0, policy_version 71530 (0.0008) +[2023-10-09 15:01:56,461][86121] Updated weights for policy 0, policy_version 71540 (0.0007) +[2023-10-09 15:01:56,828][86121] Updated weights for policy 0, policy_version 71550 (0.0007) +[2023-10-09 15:01:57,564][86122] Updated weights for policy 1, policy_version 71810 (0.0008) +[2023-10-09 15:01:57,929][86122] Updated weights for policy 1, policy_version 71820 (0.0007) +[2023-10-09 15:01:58,285][86122] Updated weights for policy 1, policy_version 71830 (0.0008) +[2023-10-09 15:01:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 146800640. Throughput: 0: 1828.3, 1: 1820.0. Samples: 36706856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:01:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:01:58,650][86122] Updated weights for policy 1, policy_version 71840 (0.0007) +[2023-10-09 15:02:00,605][86121] Updated weights for policy 0, policy_version 71560 (0.0009) +[2023-10-09 15:02:00,964][86121] Updated weights for policy 0, policy_version 71570 (0.0009) +[2023-10-09 15:02:01,330][86121] Updated weights for policy 0, policy_version 71580 (0.0008) +[2023-10-09 15:02:02,266][86122] Updated weights for policy 1, policy_version 71850 (0.0009) +[2023-10-09 15:02:02,626][86122] Updated weights for policy 1, policy_version 71860 (0.0008) +[2023-10-09 15:02:02,982][86122] Updated weights for policy 1, policy_version 71870 (0.0008) +[2023-10-09 15:02:03,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 146898944. Throughput: 0: 1827.7, 1: 1826.4. Samples: 36728624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:02:05,021][86121] Updated weights for policy 0, policy_version 71590 (0.0009) +[2023-10-09 15:02:05,396][86121] Updated weights for policy 0, policy_version 71600 (0.0008) +[2023-10-09 15:02:05,762][86121] Updated weights for policy 0, policy_version 71610 (0.0008) +[2023-10-09 15:02:06,705][86122] Updated weights for policy 1, policy_version 71880 (0.0010) +[2023-10-09 15:02:07,065][86122] Updated weights for policy 1, policy_version 71890 (0.0010) +[2023-10-09 15:02:07,432][86122] Updated weights for policy 1, policy_version 71900 (0.0007) +[2023-10-09 15:02:08,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 146964480. Throughput: 0: 1832.8, 1: 1826.9. Samples: 36750176. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:02:09,466][86121] Updated weights for policy 0, policy_version 71620 (0.0008) +[2023-10-09 15:02:09,829][86121] Updated weights for policy 0, policy_version 71630 (0.0007) +[2023-10-09 15:02:10,196][86121] Updated weights for policy 0, policy_version 71640 (0.0008) +[2023-10-09 15:02:11,047][86122] Updated weights for policy 1, policy_version 71910 (0.0009) +[2023-10-09 15:02:11,405][86122] Updated weights for policy 1, policy_version 71920 (0.0011) +[2023-10-09 15:02:11,780][86122] Updated weights for policy 1, policy_version 71930 (0.0009) +[2023-10-09 15:02:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147030016. Throughput: 0: 1833.3, 1: 1831.0. Samples: 36761626. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:02:13,686][86121] Updated weights for policy 0, policy_version 71650 (0.0010) +[2023-10-09 15:02:14,056][86121] Updated weights for policy 0, policy_version 71660 (0.0011) +[2023-10-09 15:02:14,424][86121] Updated weights for policy 0, policy_version 71670 (0.0007) +[2023-10-09 15:02:14,783][86121] Updated weights for policy 0, policy_version 71680 (0.0008) +[2023-10-09 15:02:15,548][86122] Updated weights for policy 1, policy_version 71940 (0.0009) +[2023-10-09 15:02:15,914][86122] Updated weights for policy 1, policy_version 71950 (0.0009) +[2023-10-09 15:02:16,283][86122] Updated weights for policy 1, policy_version 71960 (0.0008) +[2023-10-09 15:02:18,343][86121] Updated weights for policy 0, policy_version 71690 (0.0008) +[2023-10-09 15:02:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 147095552. Throughput: 0: 1845.1, 1: 1825.2. Samples: 36783362. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:02:18,715][86121] Updated weights for policy 0, policy_version 71700 (0.0008) +[2023-10-09 15:02:19,076][86121] Updated weights for policy 0, policy_version 71710 (0.0008) +[2023-10-09 15:02:20,033][86122] Updated weights for policy 1, policy_version 71970 (0.0009) +[2023-10-09 15:02:20,444][86122] Updated weights for policy 1, policy_version 71980 (0.0009) +[2023-10-09 15:02:20,806][86122] Updated weights for policy 1, policy_version 71990 (0.0008) +[2023-10-09 15:02:21,171][86122] Updated weights for policy 1, policy_version 72000 (0.0007) +[2023-10-09 15:02:22,729][86121] Updated weights for policy 0, policy_version 71720 (0.0008) +[2023-10-09 15:02:23,095][86121] Updated weights for policy 0, policy_version 71730 (0.0007) +[2023-10-09 15:02:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147161088. Throughput: 0: 1832.3, 1: 1827.4. Samples: 36805816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:02:23,456][86121] Updated weights for policy 0, policy_version 71740 (0.0007) +[2023-10-09 15:02:24,827][86122] Updated weights for policy 1, policy_version 72010 (0.0008) +[2023-10-09 15:02:25,188][86122] Updated weights for policy 1, policy_version 72020 (0.0008) +[2023-10-09 15:02:25,547][86122] Updated weights for policy 1, policy_version 72030 (0.0010) +[2023-10-09 15:02:27,197][86121] Updated weights for policy 0, policy_version 71750 (0.0012) +[2023-10-09 15:02:27,577][86121] Updated weights for policy 0, policy_version 71760 (0.0007) +[2023-10-09 15:02:27,942][86121] Updated weights for policy 0, policy_version 71770 (0.0008) +[2023-10-09 15:02:28,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147259392. Throughput: 0: 1846.0, 1: 1828.4. Samples: 36816390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:02:29,081][86122] Updated weights for policy 1, policy_version 72040 (0.0009) +[2023-10-09 15:02:29,437][86122] Updated weights for policy 1, policy_version 72050 (0.0010) +[2023-10-09 15:02:29,800][86122] Updated weights for policy 1, policy_version 72060 (0.0008) +[2023-10-09 15:02:31,695][86121] Updated weights for policy 0, policy_version 71780 (0.0009) +[2023-10-09 15:02:32,062][86121] Updated weights for policy 0, policy_version 71790 (0.0010) +[2023-10-09 15:02:32,431][86121] Updated weights for policy 0, policy_version 71800 (0.0009) +[2023-10-09 15:02:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147324928. Throughput: 0: 1828.0, 1: 1824.6. Samples: 36838666. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:02:33,420][86122] Updated weights for policy 1, policy_version 72070 (0.0008) +[2023-10-09 15:02:33,780][86122] Updated weights for policy 1, policy_version 72080 (0.0008) +[2023-10-09 15:02:34,139][86122] Updated weights for policy 1, policy_version 72090 (0.0007) +[2023-10-09 15:02:36,067][86121] Updated weights for policy 0, policy_version 71810 (0.0008) +[2023-10-09 15:02:36,434][86121] Updated weights for policy 0, policy_version 71820 (0.0008) +[2023-10-09 15:02:36,797][86121] Updated weights for policy 0, policy_version 71830 (0.0009) +[2023-10-09 15:02:37,163][86121] Updated weights for policy 0, policy_version 71840 (0.0009) +[2023-10-09 15:02:37,834][86122] Updated weights for policy 1, policy_version 72100 (0.0007) +[2023-10-09 15:02:38,195][86122] Updated weights for policy 1, policy_version 72110 (0.0009) +[2023-10-09 15:02:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147390464. Throughput: 0: 1831.8, 1: 1825.9. Samples: 36860440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:02:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:02:38,564][86122] Updated weights for policy 1, policy_version 72120 (0.0007) +[2023-10-09 15:02:40,893][86121] Updated weights for policy 0, policy_version 71850 (0.0008) +[2023-10-09 15:02:41,254][86121] Updated weights for policy 0, policy_version 71860 (0.0007) +[2023-10-09 15:02:41,613][86121] Updated weights for policy 0, policy_version 71870 (0.0008) +[2023-10-09 15:02:42,290][86122] Updated weights for policy 1, policy_version 72130 (0.0008) +[2023-10-09 15:02:42,652][86122] Updated weights for policy 1, policy_version 72140 (0.0010) +[2023-10-09 15:02:43,015][86122] Updated weights for policy 1, policy_version 72150 (0.0010) +[2023-10-09 15:02:43,385][86122] Updated weights for policy 1, policy_version 72160 (0.0008) +[2023-10-09 15:02:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 147488768. Throughput: 0: 1829.6, 1: 1833.3. Samples: 36871688. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:02:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:02:45,240][86121] Updated weights for policy 0, policy_version 71880 (0.0008) +[2023-10-09 15:02:45,603][86121] Updated weights for policy 0, policy_version 71890 (0.0010) +[2023-10-09 15:02:45,970][86121] Updated weights for policy 0, policy_version 71900 (0.0009) +[2023-10-09 15:02:47,145][86122] Updated weights for policy 1, policy_version 72170 (0.0007) +[2023-10-09 15:02:47,507][86122] Updated weights for policy 1, policy_version 72180 (0.0007) +[2023-10-09 15:02:47,872][86122] Updated weights for policy 1, policy_version 72190 (0.0009) +[2023-10-09 15:02:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147554304. Throughput: 0: 1838.5, 1: 1823.9. Samples: 36893430. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:02:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:02:49,715][86121] Updated weights for policy 0, policy_version 71910 (0.0007) +[2023-10-09 15:02:50,072][86121] Updated weights for policy 0, policy_version 71920 (0.0009) +[2023-10-09 15:02:50,443][86121] Updated weights for policy 0, policy_version 71930 (0.0009) +[2023-10-09 15:02:51,461][86122] Updated weights for policy 1, policy_version 72200 (0.0009) +[2023-10-09 15:02:51,835][86122] Updated weights for policy 1, policy_version 72210 (0.0010) +[2023-10-09 15:02:52,190][86122] Updated weights for policy 1, policy_version 72220 (0.0011) +[2023-10-09 15:02:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 147619840. Throughput: 0: 1833.3, 1: 1823.8. Samples: 36914748. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:02:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:02:54,161][86121] Updated weights for policy 0, policy_version 71940 (0.0010) +[2023-10-09 15:02:54,553][86121] Updated weights for policy 0, policy_version 71950 (0.0009) +[2023-10-09 15:02:54,922][86121] Updated weights for policy 0, policy_version 71960 (0.0009) +[2023-10-09 15:02:56,022][86122] Updated weights for policy 1, policy_version 72230 (0.0008) +[2023-10-09 15:02:56,379][86122] Updated weights for policy 1, policy_version 72240 (0.0007) +[2023-10-09 15:02:56,747][86122] Updated weights for policy 1, policy_version 72250 (0.0008) +[2023-10-09 15:02:58,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147685376. Throughput: 0: 1831.1, 1: 1817.2. Samples: 36925802. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:02:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:02:58,680][86121] Updated weights for policy 0, policy_version 71970 (0.0008) +[2023-10-09 15:02:59,040][86121] Updated weights for policy 0, policy_version 71980 (0.0008) +[2023-10-09 15:02:59,403][86121] Updated weights for policy 0, policy_version 71990 (0.0009) +[2023-10-09 15:02:59,777][86121] Updated weights for policy 0, policy_version 72000 (0.0007) +[2023-10-09 15:03:00,378][86122] Updated weights for policy 1, policy_version 72260 (0.0009) +[2023-10-09 15:03:00,741][86122] Updated weights for policy 1, policy_version 72270 (0.0008) +[2023-10-09 15:03:01,099][86122] Updated weights for policy 1, policy_version 72280 (0.0007) +[2023-10-09 15:03:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 147750912. Throughput: 0: 1819.1, 1: 1824.5. Samples: 36947326. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:03:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:03:03,429][86121] Updated weights for policy 0, policy_version 72010 (0.0010) +[2023-10-09 15:03:03,798][86121] Updated weights for policy 0, policy_version 72020 (0.0009) +[2023-10-09 15:03:04,166][86121] Updated weights for policy 0, policy_version 72030 (0.0007) +[2023-10-09 15:03:04,802][86122] Updated weights for policy 1, policy_version 72290 (0.0008) +[2023-10-09 15:03:05,207][86122] Updated weights for policy 1, policy_version 72300 (0.0008) +[2023-10-09 15:03:05,568][86122] Updated weights for policy 1, policy_version 72310 (0.0008) +[2023-10-09 15:03:05,932][86122] Updated weights for policy 1, policy_version 72320 (0.0007) +[2023-10-09 15:03:07,751][86121] Updated weights for policy 0, policy_version 72040 (0.0009) +[2023-10-09 15:03:08,119][86121] Updated weights for policy 0, policy_version 72050 (0.0010) +[2023-10-09 15:03:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 147816448. Throughput: 0: 1816.9, 1: 1826.8. Samples: 36969780. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:03:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:03:08,485][86121] Updated weights for policy 0, policy_version 72060 (0.0009) +[2023-10-09 15:03:09,566][86122] Updated weights for policy 1, policy_version 72330 (0.0007) +[2023-10-09 15:03:09,929][86122] Updated weights for policy 1, policy_version 72340 (0.0007) +[2023-10-09 15:03:10,285][86122] Updated weights for policy 1, policy_version 72350 (0.0008) +[2023-10-09 15:03:12,238][86121] Updated weights for policy 0, policy_version 72070 (0.0008) +[2023-10-09 15:03:12,602][86121] Updated weights for policy 0, policy_version 72080 (0.0009) +[2023-10-09 15:03:12,975][86121] Updated weights for policy 0, policy_version 72090 (0.0009) +[2023-10-09 15:03:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 147914752. Throughput: 0: 1816.1, 1: 1829.6. Samples: 36980450. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:03:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:03:13,798][86122] Updated weights for policy 1, policy_version 72360 (0.0007) +[2023-10-09 15:03:14,154][86122] Updated weights for policy 1, policy_version 72370 (0.0009) +[2023-10-09 15:03:14,516][86122] Updated weights for policy 1, policy_version 72380 (0.0007) +[2023-10-09 15:03:16,563][86121] Updated weights for policy 0, policy_version 72100 (0.0007) +[2023-10-09 15:03:16,929][86121] Updated weights for policy 0, policy_version 72110 (0.0009) +[2023-10-09 15:03:17,294][86121] Updated weights for policy 0, policy_version 72120 (0.0009) +[2023-10-09 15:03:18,243][86122] Updated weights for policy 1, policy_version 72390 (0.0009) +[2023-10-09 15:03:18,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 147980288. Throughput: 0: 1821.7, 1: 1833.0. Samples: 37003128. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:03:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:03:18,612][86122] Updated weights for policy 1, policy_version 72400 (0.0010) +[2023-10-09 15:03:18,971][86122] Updated weights for policy 1, policy_version 72410 (0.0010) +[2023-10-09 15:03:20,994][86121] Updated weights for policy 0, policy_version 72130 (0.0011) +[2023-10-09 15:03:21,358][86121] Updated weights for policy 0, policy_version 72140 (0.0008) +[2023-10-09 15:03:21,720][86121] Updated weights for policy 0, policy_version 72150 (0.0007) +[2023-10-09 15:03:22,095][86121] Updated weights for policy 0, policy_version 72160 (0.0009) +[2023-10-09 15:03:22,725][86122] Updated weights for policy 1, policy_version 72420 (0.0008) +[2023-10-09 15:03:23,084][86122] Updated weights for policy 1, policy_version 72430 (0.0007) +[2023-10-09 15:03:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 148045824. Throughput: 0: 1828.4, 1: 1826.9. Samples: 37024930. Policy #0 lag: (min: 26.0, avg: 32.1, max: 58.0) +[2023-10-09 15:03:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:03:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000072160_73891840.pth... +[2023-10-09 15:03:23,443][86122] Updated weights for policy 1, policy_version 72440 (0.0008) +[2023-10-09 15:03:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000070464_72155136.pth +[2023-10-09 15:03:23,735][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000072448_74186752.pth... +[2023-10-09 15:03:23,765][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000070720_72417280.pth +[2023-10-09 15:03:25,793][86121] Updated weights for policy 0, policy_version 72170 (0.0010) +[2023-10-09 15:03:26,168][86121] Updated weights for policy 0, policy_version 72180 (0.0009) +[2023-10-09 15:03:26,537][86121] Updated weights for policy 0, policy_version 72190 (0.0010) +[2023-10-09 15:03:27,155][86122] Updated weights for policy 1, policy_version 72450 (0.0008) +[2023-10-09 15:03:27,521][86122] Updated weights for policy 1, policy_version 72460 (0.0009) +[2023-10-09 15:03:27,877][86122] Updated weights for policy 1, policy_version 72470 (0.0010) +[2023-10-09 15:03:28,241][86122] Updated weights for policy 1, policy_version 72480 (0.0007) +[2023-10-09 15:03:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148144128. Throughput: 0: 1821.5, 1: 1827.1. Samples: 37035874. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:03:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:03:30,260][86121] Updated weights for policy 0, policy_version 72200 (0.0008) +[2023-10-09 15:03:30,623][86121] Updated weights for policy 0, policy_version 72210 (0.0009) +[2023-10-09 15:03:30,993][86121] Updated weights for policy 0, policy_version 72220 (0.0008) +[2023-10-09 15:03:31,970][86122] Updated weights for policy 1, policy_version 72490 (0.0012) +[2023-10-09 15:03:32,328][86122] Updated weights for policy 1, policy_version 72500 (0.0009) +[2023-10-09 15:03:32,687][86122] Updated weights for policy 1, policy_version 72510 (0.0007) +[2023-10-09 15:03:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148209664. Throughput: 0: 1821.9, 1: 1826.9. Samples: 37057624. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:03:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:03:34,695][86121] Updated weights for policy 0, policy_version 72230 (0.0010) +[2023-10-09 15:03:35,058][86121] Updated weights for policy 0, policy_version 72240 (0.0010) +[2023-10-09 15:03:35,425][86121] Updated weights for policy 0, policy_version 72250 (0.0009) +[2023-10-09 15:03:36,402][86122] Updated weights for policy 1, policy_version 72520 (0.0008) +[2023-10-09 15:03:36,755][86122] Updated weights for policy 1, policy_version 72530 (0.0007) +[2023-10-09 15:03:37,121][86122] Updated weights for policy 1, policy_version 72540 (0.0008) +[2023-10-09 15:03:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 148275200. Throughput: 0: 1828.5, 1: 1834.3. Samples: 37079574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:03:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:03:39,209][86121] Updated weights for policy 0, policy_version 72260 (0.0008) +[2023-10-09 15:03:39,594][86121] Updated weights for policy 0, policy_version 72270 (0.0009) +[2023-10-09 15:03:39,955][86121] Updated weights for policy 0, policy_version 72280 (0.0008) +[2023-10-09 15:03:40,783][86122] Updated weights for policy 1, policy_version 72550 (0.0008) +[2023-10-09 15:03:41,140][86122] Updated weights for policy 1, policy_version 72560 (0.0007) +[2023-10-09 15:03:41,512][86122] Updated weights for policy 1, policy_version 72570 (0.0008) +[2023-10-09 15:03:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 148340736. Throughput: 0: 1827.0, 1: 1831.7. Samples: 37090444. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:03:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:03:43,552][86121] Updated weights for policy 0, policy_version 72290 (0.0008) +[2023-10-09 15:03:43,928][86121] Updated weights for policy 0, policy_version 72300 (0.0009) +[2023-10-09 15:03:44,288][86121] Updated weights for policy 0, policy_version 72310 (0.0010) +[2023-10-09 15:03:44,652][86121] Updated weights for policy 0, policy_version 72320 (0.0009) +[2023-10-09 15:03:45,071][86122] Updated weights for policy 1, policy_version 72580 (0.0008) +[2023-10-09 15:03:45,435][86122] Updated weights for policy 1, policy_version 72590 (0.0009) +[2023-10-09 15:03:45,787][86122] Updated weights for policy 1, policy_version 72600 (0.0009) +[2023-10-09 15:03:48,220][86121] Updated weights for policy 0, policy_version 72330 (0.0008) +[2023-10-09 15:03:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 148406272. Throughput: 0: 1830.5, 1: 1841.0. Samples: 37112544. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:03:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:03:48,591][86121] Updated weights for policy 0, policy_version 72340 (0.0008) +[2023-10-09 15:03:48,957][86121] Updated weights for policy 0, policy_version 72350 (0.0008) +[2023-10-09 15:03:49,460][86122] Updated weights for policy 1, policy_version 72610 (0.0009) +[2023-10-09 15:03:49,816][86122] Updated weights for policy 1, policy_version 72620 (0.0011) +[2023-10-09 15:03:50,183][86122] Updated weights for policy 1, policy_version 72630 (0.0010) +[2023-10-09 15:03:50,529][86122] Updated weights for policy 1, policy_version 72640 (0.0009) +[2023-10-09 15:03:52,716][86121] Updated weights for policy 0, policy_version 72360 (0.0008) +[2023-10-09 15:03:53,084][86121] Updated weights for policy 0, policy_version 72370 (0.0007) +[2023-10-09 15:03:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 148471808. Throughput: 0: 1823.5, 1: 1843.7. Samples: 37134802. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:03:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:03:53,447][86121] Updated weights for policy 0, policy_version 72380 (0.0007) +[2023-10-09 15:03:54,176][86122] Updated weights for policy 1, policy_version 72650 (0.0010) +[2023-10-09 15:03:54,540][86122] Updated weights for policy 1, policy_version 72660 (0.0008) +[2023-10-09 15:03:54,895][86122] Updated weights for policy 1, policy_version 72670 (0.0007) +[2023-10-09 15:03:57,124][86121] Updated weights for policy 0, policy_version 72390 (0.0009) +[2023-10-09 15:03:57,490][86121] Updated weights for policy 0, policy_version 72400 (0.0008) +[2023-10-09 15:03:57,850][86121] Updated weights for policy 0, policy_version 72410 (0.0008) +[2023-10-09 15:03:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148570112. Throughput: 0: 1823.2, 1: 1842.8. Samples: 37145420. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:03:58,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:03:58,478][86122] Updated weights for policy 1, policy_version 72680 (0.0008) +[2023-10-09 15:03:58,845][86122] Updated weights for policy 1, policy_version 72690 (0.0008) +[2023-10-09 15:03:59,201][86122] Updated weights for policy 1, policy_version 72700 (0.0008) +[2023-10-09 15:04:01,602][86121] Updated weights for policy 0, policy_version 72420 (0.0009) +[2023-10-09 15:04:01,972][86121] Updated weights for policy 0, policy_version 72430 (0.0008) +[2023-10-09 15:04:02,330][86121] Updated weights for policy 0, policy_version 72440 (0.0008) +[2023-10-09 15:04:02,831][86122] Updated weights for policy 1, policy_version 72710 (0.0008) +[2023-10-09 15:04:03,196][86122] Updated weights for policy 1, policy_version 72720 (0.0009) +[2023-10-09 15:04:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 148635648. Throughput: 0: 1818.4, 1: 1843.6. Samples: 37167918. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:03,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:04:03,560][86122] Updated weights for policy 1, policy_version 72730 (0.0009) +[2023-10-09 15:04:05,854][86121] Updated weights for policy 0, policy_version 72450 (0.0007) +[2023-10-09 15:04:06,217][86121] Updated weights for policy 0, policy_version 72460 (0.0011) +[2023-10-09 15:04:06,595][86121] Updated weights for policy 0, policy_version 72470 (0.0010) +[2023-10-09 15:04:06,968][86121] Updated weights for policy 0, policy_version 72480 (0.0009) +[2023-10-09 15:04:07,260][86122] Updated weights for policy 1, policy_version 72740 (0.0008) +[2023-10-09 15:04:07,612][86122] Updated weights for policy 1, policy_version 72750 (0.0009) +[2023-10-09 15:04:07,967][86122] Updated weights for policy 1, policy_version 72760 (0.0011) +[2023-10-09 15:04:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 148733952. Throughput: 0: 1827.4, 1: 1828.5. Samples: 37189448. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:04:10,607][86121] Updated weights for policy 0, policy_version 72490 (0.0008) +[2023-10-09 15:04:10,965][86121] Updated weights for policy 0, policy_version 72500 (0.0007) +[2023-10-09 15:04:11,328][86121] Updated weights for policy 0, policy_version 72510 (0.0007) +[2023-10-09 15:04:11,653][86122] Updated weights for policy 1, policy_version 72770 (0.0010) +[2023-10-09 15:04:12,012][86122] Updated weights for policy 1, policy_version 72780 (0.0011) +[2023-10-09 15:04:12,378][86122] Updated weights for policy 1, policy_version 72790 (0.0007) +[2023-10-09 15:04:12,734][86122] Updated weights for policy 1, policy_version 72800 (0.0008) +[2023-10-09 15:04:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148799488. Throughput: 0: 1820.4, 1: 1842.5. Samples: 37200706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:04:14,923][86121] Updated weights for policy 0, policy_version 72520 (0.0010) +[2023-10-09 15:04:15,287][86121] Updated weights for policy 0, policy_version 72530 (0.0007) +[2023-10-09 15:04:15,653][86121] Updated weights for policy 0, policy_version 72540 (0.0008) +[2023-10-09 15:04:16,281][86122] Updated weights for policy 1, policy_version 72810 (0.0010) +[2023-10-09 15:04:16,640][86122] Updated weights for policy 1, policy_version 72820 (0.0008) +[2023-10-09 15:04:17,006][86122] Updated weights for policy 1, policy_version 72830 (0.0010) +[2023-10-09 15:04:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148865024. Throughput: 0: 1828.3, 1: 1828.8. Samples: 37222190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:04:19,417][86121] Updated weights for policy 0, policy_version 72550 (0.0009) +[2023-10-09 15:04:19,778][86121] Updated weights for policy 0, policy_version 72560 (0.0009) +[2023-10-09 15:04:20,150][86121] Updated weights for policy 0, policy_version 72570 (0.0008) +[2023-10-09 15:04:20,667][86122] Updated weights for policy 1, policy_version 72840 (0.0011) +[2023-10-09 15:04:21,029][86122] Updated weights for policy 1, policy_version 72850 (0.0009) +[2023-10-09 15:04:21,395][86122] Updated weights for policy 1, policy_version 72860 (0.0008) +[2023-10-09 15:04:23,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 148930560. Throughput: 0: 1824.7, 1: 1850.2. Samples: 37244944. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:04:23,866][86121] Updated weights for policy 0, policy_version 72580 (0.0010) +[2023-10-09 15:04:24,246][86121] Updated weights for policy 0, policy_version 72590 (0.0010) +[2023-10-09 15:04:24,615][86121] Updated weights for policy 0, policy_version 72600 (0.0008) +[2023-10-09 15:04:25,099][86122] Updated weights for policy 1, policy_version 72870 (0.0008) +[2023-10-09 15:04:25,465][86122] Updated weights for policy 1, policy_version 72880 (0.0008) +[2023-10-09 15:04:25,826][86122] Updated weights for policy 1, policy_version 72890 (0.0009) +[2023-10-09 15:04:28,280][86121] Updated weights for policy 0, policy_version 72610 (0.0008) +[2023-10-09 15:04:28,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 148996096. Throughput: 0: 1827.7, 1: 1832.6. Samples: 37255156. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:04:28,647][86121] Updated weights for policy 0, policy_version 72620 (0.0008) +[2023-10-09 15:04:29,025][86121] Updated weights for policy 0, policy_version 72630 (0.0008) +[2023-10-09 15:04:29,383][86121] Updated weights for policy 0, policy_version 72640 (0.0009) +[2023-10-09 15:04:29,506][86122] Updated weights for policy 1, policy_version 72900 (0.0008) +[2023-10-09 15:04:29,873][86122] Updated weights for policy 1, policy_version 72910 (0.0009) +[2023-10-09 15:04:30,228][86122] Updated weights for policy 1, policy_version 72920 (0.0009) +[2023-10-09 15:04:32,979][86121] Updated weights for policy 0, policy_version 72650 (0.0007) +[2023-10-09 15:04:33,344][86121] Updated weights for policy 0, policy_version 72660 (0.0007) +[2023-10-09 15:04:33,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 149061632. Throughput: 0: 1827.5, 1: 1848.0. Samples: 37277940. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:04:33,704][86121] Updated weights for policy 0, policy_version 72670 (0.0008) +[2023-10-09 15:04:33,754][86122] Updated weights for policy 1, policy_version 72930 (0.0010) +[2023-10-09 15:04:34,116][86122] Updated weights for policy 1, policy_version 72940 (0.0007) +[2023-10-09 15:04:34,486][86122] Updated weights for policy 1, policy_version 72950 (0.0007) +[2023-10-09 15:04:34,841][86122] Updated weights for policy 1, policy_version 72960 (0.0007) +[2023-10-09 15:04:37,320][86121] Updated weights for policy 0, policy_version 72680 (0.0007) +[2023-10-09 15:04:37,687][86121] Updated weights for policy 0, policy_version 72690 (0.0007) +[2023-10-09 15:04:38,060][86121] Updated weights for policy 0, policy_version 72700 (0.0008) +[2023-10-09 15:04:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 149159936. Throughput: 0: 1823.2, 1: 1848.0. Samples: 37300004. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:04:38,662][86122] Updated weights for policy 1, policy_version 72970 (0.0007) +[2023-10-09 15:04:39,033][86122] Updated weights for policy 1, policy_version 72980 (0.0008) +[2023-10-09 15:04:39,383][86122] Updated weights for policy 1, policy_version 72990 (0.0010) +[2023-10-09 15:04:41,669][86121] Updated weights for policy 0, policy_version 72710 (0.0008) +[2023-10-09 15:04:42,038][86121] Updated weights for policy 0, policy_version 72720 (0.0010) +[2023-10-09 15:04:42,402][86121] Updated weights for policy 0, policy_version 72730 (0.0007) +[2023-10-09 15:04:42,936][86122] Updated weights for policy 1, policy_version 73000 (0.0008) +[2023-10-09 15:04:43,296][86122] Updated weights for policy 1, policy_version 73010 (0.0009) +[2023-10-09 15:04:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149225472. Throughput: 0: 1840.1, 1: 1839.3. Samples: 37310994. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:04:43,667][86122] Updated weights for policy 1, policy_version 73020 (0.0008) +[2023-10-09 15:04:46,150][86121] Updated weights for policy 0, policy_version 72740 (0.0007) +[2023-10-09 15:04:46,525][86121] Updated weights for policy 0, policy_version 72750 (0.0008) +[2023-10-09 15:04:46,898][86121] Updated weights for policy 0, policy_version 72760 (0.0010) +[2023-10-09 15:04:47,345][86122] Updated weights for policy 1, policy_version 73030 (0.0008) +[2023-10-09 15:04:47,705][86122] Updated weights for policy 1, policy_version 73040 (0.0012) +[2023-10-09 15:04:48,074][86122] Updated weights for policy 1, policy_version 73050 (0.0010) +[2023-10-09 15:04:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 149323776. Throughput: 0: 1827.1, 1: 1841.2. Samples: 37332990. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:04:50,576][86121] Updated weights for policy 0, policy_version 72770 (0.0008) +[2023-10-09 15:04:50,934][86121] Updated weights for policy 0, policy_version 72780 (0.0009) +[2023-10-09 15:04:51,301][86121] Updated weights for policy 0, policy_version 72790 (0.0008) +[2023-10-09 15:04:51,667][86121] Updated weights for policy 0, policy_version 72800 (0.0009) +[2023-10-09 15:04:51,763][86122] Updated weights for policy 1, policy_version 73060 (0.0010) +[2023-10-09 15:04:52,130][86122] Updated weights for policy 1, policy_version 73070 (0.0009) +[2023-10-09 15:04:52,495][86122] Updated weights for policy 1, policy_version 73080 (0.0007) +[2023-10-09 15:04:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 149389312. Throughput: 0: 1830.0, 1: 1825.5. Samples: 37353948. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:04:55,498][86121] Updated weights for policy 0, policy_version 72810 (0.0008) +[2023-10-09 15:04:55,868][86121] Updated weights for policy 0, policy_version 72820 (0.0007) +[2023-10-09 15:04:56,232][86121] Updated weights for policy 0, policy_version 72830 (0.0008) +[2023-10-09 15:04:56,277][86122] Updated weights for policy 1, policy_version 73090 (0.0008) +[2023-10-09 15:04:56,636][86122] Updated weights for policy 1, policy_version 73100 (0.0010) +[2023-10-09 15:04:56,998][86122] Updated weights for policy 1, policy_version 73110 (0.0010) +[2023-10-09 15:04:57,359][86122] Updated weights for policy 1, policy_version 73120 (0.0008) +[2023-10-09 15:04:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149454848. Throughput: 0: 1829.6, 1: 1839.9. Samples: 37365830. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:04:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:04:59,964][86121] Updated weights for policy 0, policy_version 72840 (0.0008) +[2023-10-09 15:05:00,339][86121] Updated weights for policy 0, policy_version 72850 (0.0010) +[2023-10-09 15:05:00,697][86121] Updated weights for policy 0, policy_version 72860 (0.0009) +[2023-10-09 15:05:01,040][86122] Updated weights for policy 1, policy_version 73130 (0.0010) +[2023-10-09 15:05:01,402][86122] Updated weights for policy 1, policy_version 73140 (0.0010) +[2023-10-09 15:05:01,762][86122] Updated weights for policy 1, policy_version 73150 (0.0009) +[2023-10-09 15:05:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149520384. Throughput: 0: 1828.2, 1: 1827.6. Samples: 37386700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:05:04,399][86121] Updated weights for policy 0, policy_version 72870 (0.0008) +[2023-10-09 15:05:04,773][86121] Updated weights for policy 0, policy_version 72880 (0.0007) +[2023-10-09 15:05:05,136][86121] Updated weights for policy 0, policy_version 72890 (0.0007) +[2023-10-09 15:05:05,490][86122] Updated weights for policy 1, policy_version 73160 (0.0011) +[2023-10-09 15:05:05,856][86122] Updated weights for policy 1, policy_version 73170 (0.0011) +[2023-10-09 15:05:06,209][86122] Updated weights for policy 1, policy_version 73180 (0.0010) +[2023-10-09 15:05:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 149585920. Throughput: 0: 1826.9, 1: 1828.0. Samples: 37409416. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:05:08,945][86121] Updated weights for policy 0, policy_version 72900 (0.0009) +[2023-10-09 15:05:09,334][86121] Updated weights for policy 0, policy_version 72910 (0.0010) +[2023-10-09 15:05:09,702][86121] Updated weights for policy 0, policy_version 72920 (0.0010) +[2023-10-09 15:05:09,942][86122] Updated weights for policy 1, policy_version 73190 (0.0008) +[2023-10-09 15:05:10,307][86122] Updated weights for policy 1, policy_version 73200 (0.0008) +[2023-10-09 15:05:10,665][86122] Updated weights for policy 1, policy_version 73210 (0.0009) +[2023-10-09 15:05:13,228][86121] Updated weights for policy 0, policy_version 72930 (0.0010) +[2023-10-09 15:05:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149651456. Throughput: 0: 1826.0, 1: 1824.0. Samples: 37419402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:05:13,589][86121] Updated weights for policy 0, policy_version 72940 (0.0008) +[2023-10-09 15:05:13,958][86121] Updated weights for policy 0, policy_version 72950 (0.0007) +[2023-10-09 15:05:14,236][86122] Updated weights for policy 1, policy_version 73220 (0.0009) +[2023-10-09 15:05:14,315][86121] Updated weights for policy 0, policy_version 72960 (0.0008) +[2023-10-09 15:05:14,605][86122] Updated weights for policy 1, policy_version 73230 (0.0008) +[2023-10-09 15:05:14,968][86122] Updated weights for policy 1, policy_version 73240 (0.0008) +[2023-10-09 15:05:17,934][86121] Updated weights for policy 0, policy_version 72970 (0.0008) +[2023-10-09 15:05:18,289][86121] Updated weights for policy 0, policy_version 72980 (0.0008) +[2023-10-09 15:05:18,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 149716992. Throughput: 0: 1824.7, 1: 1828.1. Samples: 37442316. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:05:18,658][86121] Updated weights for policy 0, policy_version 72990 (0.0007) +[2023-10-09 15:05:18,667][86122] Updated weights for policy 1, policy_version 73250 (0.0009) +[2023-10-09 15:05:19,027][86122] Updated weights for policy 1, policy_version 73260 (0.0010) +[2023-10-09 15:05:19,390][86122] Updated weights for policy 1, policy_version 73270 (0.0008) +[2023-10-09 15:05:19,753][86122] Updated weights for policy 1, policy_version 73280 (0.0009) +[2023-10-09 15:05:22,344][86121] Updated weights for policy 0, policy_version 73000 (0.0008) +[2023-10-09 15:05:22,714][86121] Updated weights for policy 0, policy_version 73010 (0.0008) +[2023-10-09 15:05:23,083][86121] Updated weights for policy 0, policy_version 73020 (0.0007) +[2023-10-09 15:05:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 149815296. Throughput: 0: 1828.2, 1: 1824.6. Samples: 37464382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:05:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000073024_74776576.pth... +[2023-10-09 15:05:23,444][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000071296_73007104.pth +[2023-10-09 15:05:23,702][86122] Updated weights for policy 1, policy_version 73290 (0.0011) +[2023-10-09 15:05:24,071][86122] Updated weights for policy 1, policy_version 73300 (0.0010) +[2023-10-09 15:05:24,438][86122] Updated weights for policy 1, policy_version 73310 (0.0009) +[2023-10-09 15:05:24,502][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000073312_75071488.pth... +[2023-10-09 15:05:24,531][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000071584_73302016.pth +[2023-10-09 15:05:26,701][86121] Updated weights for policy 0, policy_version 73030 (0.0008) +[2023-10-09 15:05:27,075][86121] Updated weights for policy 0, policy_version 73040 (0.0007) +[2023-10-09 15:05:27,450][86121] Updated weights for policy 0, policy_version 73050 (0.0007) +[2023-10-09 15:05:28,025][86122] Updated weights for policy 1, policy_version 73320 (0.0008) +[2023-10-09 15:05:28,374][86122] Updated weights for policy 1, policy_version 73330 (0.0009) +[2023-10-09 15:05:28,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149880832. Throughput: 0: 1822.6, 1: 1830.0. Samples: 37475360. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:05:28,737][86122] Updated weights for policy 1, policy_version 73340 (0.0007) +[2023-10-09 15:05:31,133][86121] Updated weights for policy 0, policy_version 73060 (0.0008) +[2023-10-09 15:05:31,499][86121] Updated weights for policy 0, policy_version 73070 (0.0008) +[2023-10-09 15:05:31,864][86121] Updated weights for policy 0, policy_version 73080 (0.0007) +[2023-10-09 15:05:32,610][86122] Updated weights for policy 1, policy_version 73350 (0.0008) +[2023-10-09 15:05:32,978][86122] Updated weights for policy 1, policy_version 73360 (0.0009) +[2023-10-09 15:05:33,342][86122] Updated weights for policy 1, policy_version 73370 (0.0008) +[2023-10-09 15:05:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 149946368. Throughput: 0: 1822.8, 1: 1822.0. Samples: 37497008. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:05:35,447][86121] Updated weights for policy 0, policy_version 73090 (0.0008) +[2023-10-09 15:05:35,816][86121] Updated weights for policy 0, policy_version 73100 (0.0008) +[2023-10-09 15:05:36,184][86121] Updated weights for policy 0, policy_version 73110 (0.0008) +[2023-10-09 15:05:36,550][86121] Updated weights for policy 0, policy_version 73120 (0.0008) +[2023-10-09 15:05:37,116][86122] Updated weights for policy 1, policy_version 73380 (0.0007) +[2023-10-09 15:05:37,486][86122] Updated weights for policy 1, policy_version 73390 (0.0009) +[2023-10-09 15:05:37,842][86122] Updated weights for policy 1, policy_version 73400 (0.0009) +[2023-10-09 15:05:38,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 150044672. Throughput: 0: 1831.1, 1: 1824.9. Samples: 37518468. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:05:40,175][86121] Updated weights for policy 0, policy_version 73130 (0.0009) +[2023-10-09 15:05:40,543][86121] Updated weights for policy 0, policy_version 73140 (0.0008) +[2023-10-09 15:05:40,909][86121] Updated weights for policy 0, policy_version 73150 (0.0008) +[2023-10-09 15:05:41,516][86122] Updated weights for policy 1, policy_version 73410 (0.0008) +[2023-10-09 15:05:41,878][86122] Updated weights for policy 1, policy_version 73420 (0.0008) +[2023-10-09 15:05:42,240][86122] Updated weights for policy 1, policy_version 73430 (0.0009) +[2023-10-09 15:05:42,598][86122] Updated weights for policy 1, policy_version 73440 (0.0007) +[2023-10-09 15:05:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150110208. Throughput: 0: 1822.8, 1: 1813.6. Samples: 37529470. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:05:44,539][86121] Updated weights for policy 0, policy_version 73160 (0.0008) +[2023-10-09 15:05:44,898][86121] Updated weights for policy 0, policy_version 73170 (0.0007) +[2023-10-09 15:05:45,268][86121] Updated weights for policy 0, policy_version 73180 (0.0008) +[2023-10-09 15:05:46,187][86122] Updated weights for policy 1, policy_version 73450 (0.0007) +[2023-10-09 15:05:46,549][86122] Updated weights for policy 1, policy_version 73460 (0.0008) +[2023-10-09 15:05:46,914][86122] Updated weights for policy 1, policy_version 73470 (0.0008) +[2023-10-09 15:05:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 150175744. Throughput: 0: 1836.2, 1: 1825.6. Samples: 37551482. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:05:48,789][86121] Updated weights for policy 0, policy_version 73190 (0.0007) +[2023-10-09 15:05:49,153][86121] Updated weights for policy 0, policy_version 73200 (0.0008) +[2023-10-09 15:05:49,519][86121] Updated weights for policy 0, policy_version 73210 (0.0008) +[2023-10-09 15:05:50,685][86122] Updated weights for policy 1, policy_version 73480 (0.0010) +[2023-10-09 15:05:51,043][86122] Updated weights for policy 1, policy_version 73490 (0.0009) +[2023-10-09 15:05:51,415][86122] Updated weights for policy 1, policy_version 73500 (0.0008) +[2023-10-09 15:05:53,215][86121] Updated weights for policy 0, policy_version 73220 (0.0010) +[2023-10-09 15:05:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 150241280. Throughput: 0: 1839.1, 1: 1820.5. Samples: 37574098. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:05:53,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:05:53,578][86121] Updated weights for policy 0, policy_version 73230 (0.0009) +[2023-10-09 15:05:53,940][86121] Updated weights for policy 0, policy_version 73240 (0.0007) +[2023-10-09 15:05:55,139][86122] Updated weights for policy 1, policy_version 73510 (0.0009) +[2023-10-09 15:05:55,499][86122] Updated weights for policy 1, policy_version 73520 (0.0010) +[2023-10-09 15:05:55,859][86122] Updated weights for policy 1, policy_version 73530 (0.0011) +[2023-10-09 15:05:57,668][86121] Updated weights for policy 0, policy_version 73250 (0.0008) +[2023-10-09 15:05:58,077][86121] Updated weights for policy 0, policy_version 73260 (0.0008) +[2023-10-09 15:05:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150306816. Throughput: 0: 1843.8, 1: 1824.8. Samples: 37584490. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:05:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:05:58,438][86121] Updated weights for policy 0, policy_version 73270 (0.0009) +[2023-10-09 15:05:58,811][86121] Updated weights for policy 0, policy_version 73280 (0.0009) +[2023-10-09 15:05:59,415][86122] Updated weights for policy 1, policy_version 73540 (0.0009) +[2023-10-09 15:05:59,781][86122] Updated weights for policy 1, policy_version 73550 (0.0008) +[2023-10-09 15:06:00,146][86122] Updated weights for policy 1, policy_version 73560 (0.0008) +[2023-10-09 15:06:02,662][86121] Updated weights for policy 0, policy_version 73290 (0.0008) +[2023-10-09 15:06:03,033][86121] Updated weights for policy 0, policy_version 73300 (0.0009) +[2023-10-09 15:06:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150372352. Throughput: 0: 1836.8, 1: 1816.8. Samples: 37606728. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:06:03,401][86121] Updated weights for policy 0, policy_version 73310 (0.0007) +[2023-10-09 15:06:03,795][86122] Updated weights for policy 1, policy_version 73570 (0.0008) +[2023-10-09 15:06:04,155][86122] Updated weights for policy 1, policy_version 73580 (0.0010) +[2023-10-09 15:06:04,522][86122] Updated weights for policy 1, policy_version 73590 (0.0007) +[2023-10-09 15:06:04,881][86122] Updated weights for policy 1, policy_version 73600 (0.0007) +[2023-10-09 15:06:07,020][86121] Updated weights for policy 0, policy_version 73320 (0.0010) +[2023-10-09 15:06:07,386][86121] Updated weights for policy 0, policy_version 73330 (0.0010) +[2023-10-09 15:06:07,760][86121] Updated weights for policy 0, policy_version 73340 (0.0011) +[2023-10-09 15:06:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150470656. Throughput: 0: 1821.6, 1: 1821.7. Samples: 37628332. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:06:08,777][86122] Updated weights for policy 1, policy_version 73610 (0.0008) +[2023-10-09 15:06:09,147][86122] Updated weights for policy 1, policy_version 73620 (0.0009) +[2023-10-09 15:06:09,510][86122] Updated weights for policy 1, policy_version 73630 (0.0008) +[2023-10-09 15:06:11,547][86121] Updated weights for policy 0, policy_version 73350 (0.0009) +[2023-10-09 15:06:11,909][86121] Updated weights for policy 0, policy_version 73360 (0.0008) +[2023-10-09 15:06:12,279][86121] Updated weights for policy 0, policy_version 73370 (0.0007) +[2023-10-09 15:06:13,011][86122] Updated weights for policy 1, policy_version 73640 (0.0008) +[2023-10-09 15:06:13,375][86122] Updated weights for policy 1, policy_version 73650 (0.0008) +[2023-10-09 15:06:13,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 150536192. Throughput: 0: 1828.9, 1: 1818.3. Samples: 37639482. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:06:13,744][86122] Updated weights for policy 1, policy_version 73660 (0.0009) +[2023-10-09 15:06:16,123][86121] Updated weights for policy 0, policy_version 73380 (0.0008) +[2023-10-09 15:06:16,486][86121] Updated weights for policy 0, policy_version 73390 (0.0008) +[2023-10-09 15:06:16,852][86121] Updated weights for policy 0, policy_version 73400 (0.0007) +[2023-10-09 15:06:17,322][86122] Updated weights for policy 1, policy_version 73670 (0.0008) +[2023-10-09 15:06:17,686][86122] Updated weights for policy 1, policy_version 73680 (0.0009) +[2023-10-09 15:06:18,062][86122] Updated weights for policy 1, policy_version 73690 (0.0010) +[2023-10-09 15:06:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 150634496. Throughput: 0: 1825.7, 1: 1830.6. Samples: 37661540. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:06:20,492][86121] Updated weights for policy 0, policy_version 73410 (0.0008) +[2023-10-09 15:06:20,858][86121] Updated weights for policy 0, policy_version 73420 (0.0011) +[2023-10-09 15:06:21,230][86121] Updated weights for policy 0, policy_version 73430 (0.0008) +[2023-10-09 15:06:21,593][86121] Updated weights for policy 0, policy_version 73440 (0.0009) +[2023-10-09 15:06:21,664][86122] Updated weights for policy 1, policy_version 73700 (0.0009) +[2023-10-09 15:06:22,024][86122] Updated weights for policy 1, policy_version 73710 (0.0009) +[2023-10-09 15:06:22,383][86122] Updated weights for policy 1, policy_version 73720 (0.0009) +[2023-10-09 15:06:23,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 150700032. Throughput: 0: 1820.1, 1: 1832.1. Samples: 37682816. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:06:25,252][86121] Updated weights for policy 0, policy_version 73450 (0.0010) +[2023-10-09 15:06:25,616][86121] Updated weights for policy 0, policy_version 73460 (0.0011) +[2023-10-09 15:06:25,985][86121] Updated weights for policy 0, policy_version 73470 (0.0011) +[2023-10-09 15:06:26,084][86122] Updated weights for policy 1, policy_version 73730 (0.0008) +[2023-10-09 15:06:26,442][86122] Updated weights for policy 1, policy_version 73740 (0.0008) +[2023-10-09 15:06:26,806][86122] Updated weights for policy 1, policy_version 73750 (0.0007) +[2023-10-09 15:06:27,166][86122] Updated weights for policy 1, policy_version 73760 (0.0008) +[2023-10-09 15:06:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150765568. Throughput: 0: 1822.5, 1: 1842.6. Samples: 37694400. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:06:29,675][86121] Updated weights for policy 0, policy_version 73480 (0.0008) +[2023-10-09 15:06:30,048][86121] Updated weights for policy 0, policy_version 73490 (0.0008) +[2023-10-09 15:06:30,413][86121] Updated weights for policy 0, policy_version 73500 (0.0007) +[2023-10-09 15:06:30,865][86122] Updated weights for policy 1, policy_version 73770 (0.0009) +[2023-10-09 15:06:31,232][86122] Updated weights for policy 1, policy_version 73780 (0.0007) +[2023-10-09 15:06:31,591][86122] Updated weights for policy 1, policy_version 73790 (0.0008) +[2023-10-09 15:06:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 150831104. Throughput: 0: 1814.6, 1: 1831.6. Samples: 37715562. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:06:34,081][86121] Updated weights for policy 0, policy_version 73510 (0.0007) +[2023-10-09 15:06:34,453][86121] Updated weights for policy 0, policy_version 73520 (0.0007) +[2023-10-09 15:06:34,815][86121] Updated weights for policy 0, policy_version 73530 (0.0007) +[2023-10-09 15:06:35,227][86122] Updated weights for policy 1, policy_version 73800 (0.0008) +[2023-10-09 15:06:35,593][86122] Updated weights for policy 1, policy_version 73810 (0.0008) +[2023-10-09 15:06:35,957][86122] Updated weights for policy 1, policy_version 73820 (0.0008) +[2023-10-09 15:06:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 150896640. Throughput: 0: 1815.7, 1: 1838.3. Samples: 37738526. Policy #0 lag: (min: 10.0, avg: 10.3, max: 21.0) +[2023-10-09 15:06:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:06:38,484][86121] Updated weights for policy 0, policy_version 73540 (0.0008) +[2023-10-09 15:06:38,849][86121] Updated weights for policy 0, policy_version 73550 (0.0007) +[2023-10-09 15:06:39,219][86121] Updated weights for policy 0, policy_version 73560 (0.0007) +[2023-10-09 15:06:39,636][86122] Updated weights for policy 1, policy_version 73830 (0.0009) +[2023-10-09 15:06:39,991][86122] Updated weights for policy 1, policy_version 73840 (0.0011) +[2023-10-09 15:06:40,347][86122] Updated weights for policy 1, policy_version 73850 (0.0010) +[2023-10-09 15:06:42,882][86121] Updated weights for policy 0, policy_version 73570 (0.0009) +[2023-10-09 15:06:43,275][86121] Updated weights for policy 0, policy_version 73580 (0.0007) +[2023-10-09 15:06:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 150962176. Throughput: 0: 1814.2, 1: 1831.6. Samples: 37748552. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:06:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:06:43,643][86121] Updated weights for policy 0, policy_version 73590 (0.0008) +[2023-10-09 15:06:43,911][86122] Updated weights for policy 1, policy_version 73860 (0.0008) +[2023-10-09 15:06:44,007][86121] Updated weights for policy 0, policy_version 73600 (0.0007) +[2023-10-09 15:06:44,265][86122] Updated weights for policy 1, policy_version 73870 (0.0007) +[2023-10-09 15:06:44,619][86122] Updated weights for policy 1, policy_version 73880 (0.0007) +[2023-10-09 15:06:47,682][86121] Updated weights for policy 0, policy_version 73610 (0.0009) +[2023-10-09 15:06:48,055][86121] Updated weights for policy 0, policy_version 73620 (0.0008) +[2023-10-09 15:06:48,335][86122] Updated weights for policy 1, policy_version 73890 (0.0009) +[2023-10-09 15:06:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151027712. Throughput: 0: 1818.7, 1: 1842.4. Samples: 37771480. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:06:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:06:48,418][86121] Updated weights for policy 0, policy_version 73630 (0.0007) +[2023-10-09 15:06:48,696][86122] Updated weights for policy 1, policy_version 73900 (0.0009) +[2023-10-09 15:06:49,061][86122] Updated weights for policy 1, policy_version 73910 (0.0007) +[2023-10-09 15:06:49,411][86122] Updated weights for policy 1, policy_version 73920 (0.0009) +[2023-10-09 15:06:52,089][86121] Updated weights for policy 0, policy_version 73640 (0.0008) +[2023-10-09 15:06:52,448][86121] Updated weights for policy 0, policy_version 73650 (0.0008) +[2023-10-09 15:06:52,821][86121] Updated weights for policy 0, policy_version 73660 (0.0008) +[2023-10-09 15:06:53,192][86122] Updated weights for policy 1, policy_version 73930 (0.0009) +[2023-10-09 15:06:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 151126016. Throughput: 0: 1821.4, 1: 1836.0. Samples: 37792916. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:06:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:06:53,558][86122] Updated weights for policy 1, policy_version 73940 (0.0009) +[2023-10-09 15:06:53,918][86122] Updated weights for policy 1, policy_version 73950 (0.0008) +[2023-10-09 15:06:56,358][86121] Updated weights for policy 0, policy_version 73670 (0.0009) +[2023-10-09 15:06:56,718][86121] Updated weights for policy 0, policy_version 73680 (0.0008) +[2023-10-09 15:06:57,074][86121] Updated weights for policy 0, policy_version 73690 (0.0008) +[2023-10-09 15:06:57,615][86122] Updated weights for policy 1, policy_version 73960 (0.0007) +[2023-10-09 15:06:57,983][86122] Updated weights for policy 1, policy_version 73970 (0.0007) +[2023-10-09 15:06:58,342][86122] Updated weights for policy 1, policy_version 73980 (0.0008) +[2023-10-09 15:06:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 151191552. Throughput: 0: 1820.6, 1: 1843.1. Samples: 37804346. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:06:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:07:00,752][86121] Updated weights for policy 0, policy_version 73700 (0.0008) +[2023-10-09 15:07:01,118][86121] Updated weights for policy 0, policy_version 73710 (0.0008) +[2023-10-09 15:07:01,488][86121] Updated weights for policy 0, policy_version 73720 (0.0008) +[2023-10-09 15:07:02,003][86122] Updated weights for policy 1, policy_version 73990 (0.0008) +[2023-10-09 15:07:02,371][86122] Updated weights for policy 1, policy_version 74000 (0.0009) +[2023-10-09 15:07:02,741][86122] Updated weights for policy 1, policy_version 74010 (0.0007) +[2023-10-09 15:07:03,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 151289856. Throughput: 0: 1815.5, 1: 1833.2. Samples: 37825736. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:07:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:07:05,217][86121] Updated weights for policy 0, policy_version 73730 (0.0008) +[2023-10-09 15:07:05,584][86121] Updated weights for policy 0, policy_version 73740 (0.0008) +[2023-10-09 15:07:05,955][86121] Updated weights for policy 0, policy_version 73750 (0.0008) +[2023-10-09 15:07:06,314][86121] Updated weights for policy 0, policy_version 73760 (0.0008) +[2023-10-09 15:07:06,357][86122] Updated weights for policy 1, policy_version 74020 (0.0007) +[2023-10-09 15:07:06,723][86122] Updated weights for policy 1, policy_version 74030 (0.0008) +[2023-10-09 15:07:07,076][86122] Updated weights for policy 1, policy_version 74040 (0.0008) +[2023-10-09 15:07:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 151355392. Throughput: 0: 1824.5, 1: 1832.5. Samples: 37847382. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:07:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:07:09,943][86121] Updated weights for policy 0, policy_version 73770 (0.0008) +[2023-10-09 15:07:10,308][86121] Updated weights for policy 0, policy_version 73780 (0.0007) +[2023-10-09 15:07:10,545][86122] Updated weights for policy 1, policy_version 74050 (0.0008) +[2023-10-09 15:07:10,674][86121] Updated weights for policy 0, policy_version 73790 (0.0010) +[2023-10-09 15:07:10,912][86122] Updated weights for policy 1, policy_version 74060 (0.0008) +[2023-10-09 15:07:11,274][86122] Updated weights for policy 1, policy_version 74070 (0.0009) +[2023-10-09 15:07:11,639][86122] Updated weights for policy 1, policy_version 74080 (0.0010) +[2023-10-09 15:07:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 151420928. Throughput: 0: 1821.8, 1: 1827.3. Samples: 37858612. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:07:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:07:14,352][86121] Updated weights for policy 0, policy_version 73800 (0.0009) +[2023-10-09 15:07:14,728][86121] Updated weights for policy 0, policy_version 73810 (0.0009) +[2023-10-09 15:07:15,098][86121] Updated weights for policy 0, policy_version 73820 (0.0009) +[2023-10-09 15:07:15,273][86122] Updated weights for policy 1, policy_version 74090 (0.0009) +[2023-10-09 15:07:15,632][86122] Updated weights for policy 1, policy_version 74100 (0.0009) +[2023-10-09 15:07:15,996][86122] Updated weights for policy 1, policy_version 74110 (0.0009) +[2023-10-09 15:07:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 151486464. Throughput: 0: 1833.4, 1: 1836.4. Samples: 37880702. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:07:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:07:18,968][86121] Updated weights for policy 0, policy_version 73830 (0.0009) +[2023-10-09 15:07:19,329][86121] Updated weights for policy 0, policy_version 73840 (0.0009) +[2023-10-09 15:07:19,435][86122] Updated weights for policy 1, policy_version 74120 (0.0007) +[2023-10-09 15:07:19,698][86121] Updated weights for policy 0, policy_version 73850 (0.0009) +[2023-10-09 15:07:19,794][86122] Updated weights for policy 1, policy_version 74130 (0.0007) +[2023-10-09 15:07:20,160][86122] Updated weights for policy 1, policy_version 74140 (0.0008) +[2023-10-09 15:07:23,363][86121] Updated weights for policy 0, policy_version 73860 (0.0008) +[2023-10-09 15:07:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151552000. Throughput: 0: 1821.5, 1: 1846.7. Samples: 37903594. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:07:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:07:23,405][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000074144_75923456.pth... +[2023-10-09 15:07:23,438][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000072448_74186752.pth +[2023-10-09 15:07:23,727][86121] Updated weights for policy 0, policy_version 73870 (0.0007) +[2023-10-09 15:07:23,778][86122] Updated weights for policy 1, policy_version 74150 (0.0008) +[2023-10-09 15:07:24,095][86121] Updated weights for policy 0, policy_version 73880 (0.0009) +[2023-10-09 15:07:24,145][86122] Updated weights for policy 1, policy_version 74160 (0.0009) +[2023-10-09 15:07:24,382][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000073888_75661312.pth... +[2023-10-09 15:07:24,421][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000072160_73891840.pth +[2023-10-09 15:07:24,495][86122] Updated weights for policy 1, policy_version 74170 (0.0008) +[2023-10-09 15:07:27,936][86121] Updated weights for policy 0, policy_version 73890 (0.0008) +[2023-10-09 15:07:28,229][86122] Updated weights for policy 1, policy_version 74180 (0.0008) +[2023-10-09 15:07:28,333][86121] Updated weights for policy 0, policy_version 73900 (0.0008) +[2023-10-09 15:07:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151617536. Throughput: 0: 1824.6, 1: 1847.0. Samples: 37913772. Policy #0 lag: (min: 1.0, avg: 18.4, max: 33.0) +[2023-10-09 15:07:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:07:28,593][86122] Updated weights for policy 1, policy_version 74190 (0.0010) +[2023-10-09 15:07:28,705][86121] Updated weights for policy 0, policy_version 73910 (0.0007) +[2023-10-09 15:07:28,957][86122] Updated weights for policy 1, policy_version 74200 (0.0008) +[2023-10-09 15:07:29,072][86121] Updated weights for policy 0, policy_version 73920 (0.0008) +[2023-10-09 15:07:32,760][86121] Updated weights for policy 0, policy_version 73930 (0.0008) +[2023-10-09 15:07:32,805][86122] Updated weights for policy 1, policy_version 74210 (0.0010) +[2023-10-09 15:07:33,125][86121] Updated weights for policy 0, policy_version 73940 (0.0009) +[2023-10-09 15:07:33,172][86122] Updated weights for policy 1, policy_version 74220 (0.0008) +[2023-10-09 15:07:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 151683072. Throughput: 0: 1820.2, 1: 1842.0. Samples: 37936280. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:07:33,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:07:33,488][86121] Updated weights for policy 0, policy_version 73950 (0.0009) +[2023-10-09 15:07:33,533][86122] Updated weights for policy 1, policy_version 74230 (0.0008) +[2023-10-09 15:07:33,899][86122] Updated weights for policy 1, policy_version 74240 (0.0010) +[2023-10-09 15:07:37,135][86121] Updated weights for policy 0, policy_version 73960 (0.0010) +[2023-10-09 15:07:37,501][86121] Updated weights for policy 0, policy_version 73970 (0.0009) +[2023-10-09 15:07:37,564][86122] Updated weights for policy 1, policy_version 74250 (0.0010) +[2023-10-09 15:07:37,857][86121] Updated weights for policy 0, policy_version 73980 (0.0008) +[2023-10-09 15:07:37,925][86122] Updated weights for policy 1, policy_version 74260 (0.0010) +[2023-10-09 15:07:38,293][86122] Updated weights for policy 1, policy_version 74270 (0.0009) +[2023-10-09 15:07:38,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 151814144. Throughput: 0: 1816.1, 1: 1826.9. Samples: 37956854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:07:38,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:07:41,584][86121] Updated weights for policy 0, policy_version 73990 (0.0008) +[2023-10-09 15:07:41,946][86121] Updated weights for policy 0, policy_version 74000 (0.0008) +[2023-10-09 15:07:42,195][86122] Updated weights for policy 1, policy_version 74280 (0.0009) +[2023-10-09 15:07:42,305][86121] Updated weights for policy 0, policy_version 74010 (0.0010) +[2023-10-09 15:07:42,556][86122] Updated weights for policy 1, policy_version 74290 (0.0009) +[2023-10-09 15:07:42,923][86122] Updated weights for policy 1, policy_version 74300 (0.0007) +[2023-10-09 15:07:43,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 151879680. Throughput: 0: 1814.4, 1: 1839.5. Samples: 37968772. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:07:43,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:07:46,030][86121] Updated weights for policy 0, policy_version 74020 (0.0009) +[2023-10-09 15:07:46,398][86121] Updated weights for policy 0, policy_version 74030 (0.0009) +[2023-10-09 15:07:46,631][86122] Updated weights for policy 1, policy_version 74310 (0.0007) +[2023-10-09 15:07:46,767][86121] Updated weights for policy 0, policy_version 74040 (0.0008) +[2023-10-09 15:07:46,987][86122] Updated weights for policy 1, policy_version 74320 (0.0008) +[2023-10-09 15:07:47,354][86122] Updated weights for policy 1, policy_version 74330 (0.0007) +[2023-10-09 15:07:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 151945216. Throughput: 0: 1818.9, 1: 1821.0. Samples: 37989534. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:07:48,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:07:50,423][86121] Updated weights for policy 0, policy_version 74050 (0.0009) +[2023-10-09 15:07:50,788][86121] Updated weights for policy 0, policy_version 74060 (0.0008) +[2023-10-09 15:07:51,151][86121] Updated weights for policy 0, policy_version 74070 (0.0009) +[2023-10-09 15:07:51,171][86122] Updated weights for policy 1, policy_version 74340 (0.0007) +[2023-10-09 15:07:51,527][86121] Updated weights for policy 0, policy_version 74080 (0.0007) +[2023-10-09 15:07:51,541][86122] Updated weights for policy 1, policy_version 74350 (0.0008) +[2023-10-09 15:07:51,914][86122] Updated weights for policy 1, policy_version 74360 (0.0010) +[2023-10-09 15:07:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152010752. Throughput: 0: 1816.3, 1: 1823.8. Samples: 38011188. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:07:53,398][85186] Avg episode reward: [(0, '10.000'), (1, '9.990')] +[2023-10-09 15:07:55,277][86121] Updated weights for policy 0, policy_version 74090 (0.0008) +[2023-10-09 15:07:55,563][86122] Updated weights for policy 1, policy_version 74370 (0.0009) +[2023-10-09 15:07:55,640][86121] Updated weights for policy 0, policy_version 74100 (0.0009) +[2023-10-09 15:07:55,919][86122] Updated weights for policy 1, policy_version 74380 (0.0007) +[2023-10-09 15:07:56,012][86121] Updated weights for policy 0, policy_version 74110 (0.0008) +[2023-10-09 15:07:56,273][86122] Updated weights for policy 1, policy_version 74390 (0.0009) +[2023-10-09 15:07:56,637][86122] Updated weights for policy 1, policy_version 74400 (0.0008) +[2023-10-09 15:07:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 152076288. Throughput: 0: 1817.4, 1: 1822.2. Samples: 38022394. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:07:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:07:59,579][86121] Updated weights for policy 0, policy_version 74120 (0.0008) +[2023-10-09 15:07:59,943][86121] Updated weights for policy 0, policy_version 74130 (0.0010) +[2023-10-09 15:08:00,315][86121] Updated weights for policy 0, policy_version 74140 (0.0009) +[2023-10-09 15:08:00,345][86122] Updated weights for policy 1, policy_version 74410 (0.0008) +[2023-10-09 15:08:00,698][86122] Updated weights for policy 1, policy_version 74420 (0.0008) +[2023-10-09 15:08:01,071][86122] Updated weights for policy 1, policy_version 74430 (0.0009) +[2023-10-09 15:08:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152141824. Throughput: 0: 1809.9, 1: 1818.9. Samples: 38044000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:08:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:08:04,059][86121] Updated weights for policy 0, policy_version 74150 (0.0007) +[2023-10-09 15:08:04,433][86121] Updated weights for policy 0, policy_version 74160 (0.0008) +[2023-10-09 15:08:04,755][86122] Updated weights for policy 1, policy_version 74440 (0.0008) +[2023-10-09 15:08:04,792][86121] Updated weights for policy 0, policy_version 74170 (0.0007) +[2023-10-09 15:08:05,122][86122] Updated weights for policy 1, policy_version 74450 (0.0008) +[2023-10-09 15:08:05,485][86122] Updated weights for policy 1, policy_version 74460 (0.0009) +[2023-10-09 15:08:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152207360. Throughput: 0: 1818.9, 1: 1809.1. Samples: 38066852. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:08:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:08:08,521][86121] Updated weights for policy 0, policy_version 74180 (0.0009) +[2023-10-09 15:08:08,885][86121] Updated weights for policy 0, policy_version 74190 (0.0008) +[2023-10-09 15:08:09,250][86121] Updated weights for policy 0, policy_version 74200 (0.0008) +[2023-10-09 15:08:09,328][86122] Updated weights for policy 1, policy_version 74470 (0.0009) +[2023-10-09 15:08:09,693][86122] Updated weights for policy 1, policy_version 74480 (0.0007) +[2023-10-09 15:08:10,057][86122] Updated weights for policy 1, policy_version 74490 (0.0007) +[2023-10-09 15:08:13,081][86121] Updated weights for policy 0, policy_version 74210 (0.0007) +[2023-10-09 15:08:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152272896. Throughput: 0: 1814.7, 1: 1807.5. Samples: 38076770. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:08:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:08:13,478][86121] Updated weights for policy 0, policy_version 74220 (0.0007) +[2023-10-09 15:08:13,553][86122] Updated weights for policy 1, policy_version 74500 (0.0008) +[2023-10-09 15:08:13,843][86121] Updated weights for policy 0, policy_version 74230 (0.0008) +[2023-10-09 15:08:13,906][86122] Updated weights for policy 1, policy_version 74510 (0.0007) +[2023-10-09 15:08:14,210][86121] Updated weights for policy 0, policy_version 74240 (0.0008) +[2023-10-09 15:08:14,276][86122] Updated weights for policy 1, policy_version 74520 (0.0009) +[2023-10-09 15:08:17,974][86121] Updated weights for policy 0, policy_version 74250 (0.0009) +[2023-10-09 15:08:18,050][86122] Updated weights for policy 1, policy_version 74530 (0.0008) +[2023-10-09 15:08:18,350][86121] Updated weights for policy 0, policy_version 74260 (0.0007) +[2023-10-09 15:08:18,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 152338432. Throughput: 0: 1811.9, 1: 1811.1. Samples: 38099314. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:08:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:08:18,410][86122] Updated weights for policy 1, policy_version 74540 (0.0009) +[2023-10-09 15:08:18,708][86121] Updated weights for policy 0, policy_version 74270 (0.0007) +[2023-10-09 15:08:18,782][86122] Updated weights for policy 1, policy_version 74550 (0.0007) +[2023-10-09 15:08:19,143][86122] Updated weights for policy 1, policy_version 74560 (0.0011) +[2023-10-09 15:08:22,319][86121] Updated weights for policy 0, policy_version 74280 (0.0010) +[2023-10-09 15:08:22,689][86121] Updated weights for policy 0, policy_version 74290 (0.0007) +[2023-10-09 15:08:22,733][86122] Updated weights for policy 1, policy_version 74570 (0.0008) +[2023-10-09 15:08:23,061][86121] Updated weights for policy 0, policy_version 74300 (0.0008) +[2023-10-09 15:08:23,100][86122] Updated weights for policy 1, policy_version 74580 (0.0007) +[2023-10-09 15:08:23,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 152436736. Throughput: 0: 1827.2, 1: 1814.9. Samples: 38120748. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:08:23,452][86122] Updated weights for policy 1, policy_version 74590 (0.0007) +[2023-10-09 15:08:26,860][86121] Updated weights for policy 0, policy_version 74310 (0.0007) +[2023-10-09 15:08:27,206][86122] Updated weights for policy 1, policy_version 74600 (0.0008) +[2023-10-09 15:08:27,233][86121] Updated weights for policy 0, policy_version 74320 (0.0009) +[2023-10-09 15:08:27,558][86122] Updated weights for policy 1, policy_version 74610 (0.0008) +[2023-10-09 15:08:27,589][86121] Updated weights for policy 0, policy_version 74330 (0.0008) +[2023-10-09 15:08:27,930][86122] Updated weights for policy 1, policy_version 74620 (0.0008) +[2023-10-09 15:08:28,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 152535040. Throughput: 0: 1819.8, 1: 1814.5. Samples: 38132318. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:08:31,204][86121] Updated weights for policy 0, policy_version 74340 (0.0008) +[2023-10-09 15:08:31,573][86121] Updated weights for policy 0, policy_version 74350 (0.0009) +[2023-10-09 15:08:31,584][86122] Updated weights for policy 1, policy_version 74630 (0.0008) +[2023-10-09 15:08:31,929][86121] Updated weights for policy 0, policy_version 74360 (0.0007) +[2023-10-09 15:08:31,941][86122] Updated weights for policy 1, policy_version 74640 (0.0007) +[2023-10-09 15:08:32,295][86122] Updated weights for policy 1, policy_version 74650 (0.0009) +[2023-10-09 15:08:33,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 152600576. Throughput: 0: 1826.2, 1: 1817.5. Samples: 38153500. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:08:35,456][86121] Updated weights for policy 0, policy_version 74370 (0.0008) +[2023-10-09 15:08:35,826][86121] Updated weights for policy 0, policy_version 74380 (0.0008) +[2023-10-09 15:08:36,021][86122] Updated weights for policy 1, policy_version 74660 (0.0008) +[2023-10-09 15:08:36,206][86121] Updated weights for policy 0, policy_version 74390 (0.0008) +[2023-10-09 15:08:36,381][86122] Updated weights for policy 1, policy_version 74670 (0.0008) +[2023-10-09 15:08:36,565][86121] Updated weights for policy 0, policy_version 74400 (0.0010) +[2023-10-09 15:08:36,741][86122] Updated weights for policy 1, policy_version 74680 (0.0008) +[2023-10-09 15:08:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152666112. Throughput: 0: 1814.1, 1: 1822.4. Samples: 38174832. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:08:40,068][86121] Updated weights for policy 0, policy_version 74410 (0.0008) +[2023-10-09 15:08:40,444][86121] Updated weights for policy 0, policy_version 74420 (0.0008) +[2023-10-09 15:08:40,468][86122] Updated weights for policy 1, policy_version 74690 (0.0010) +[2023-10-09 15:08:40,819][86121] Updated weights for policy 0, policy_version 74430 (0.0008) +[2023-10-09 15:08:40,822][86122] Updated weights for policy 1, policy_version 74700 (0.0009) +[2023-10-09 15:08:41,184][86122] Updated weights for policy 1, policy_version 74710 (0.0011) +[2023-10-09 15:08:41,541][86122] Updated weights for policy 1, policy_version 74720 (0.0008) +[2023-10-09 15:08:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 152731648. Throughput: 0: 1810.2, 1: 1822.5. Samples: 38185866. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:43,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:08:44,523][86121] Updated weights for policy 0, policy_version 74440 (0.0011) +[2023-10-09 15:08:44,890][86121] Updated weights for policy 0, policy_version 74450 (0.0011) +[2023-10-09 15:08:45,261][86121] Updated weights for policy 0, policy_version 74460 (0.0009) +[2023-10-09 15:08:45,413][86122] Updated weights for policy 1, policy_version 74730 (0.0011) +[2023-10-09 15:08:45,775][86122] Updated weights for policy 1, policy_version 74740 (0.0009) +[2023-10-09 15:08:46,142][86122] Updated weights for policy 1, policy_version 74750 (0.0010) +[2023-10-09 15:08:48,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 152797184. Throughput: 0: 1816.7, 1: 1821.4. Samples: 38207714. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:48,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:08:49,088][86121] Updated weights for policy 0, policy_version 74470 (0.0009) +[2023-10-09 15:08:49,457][86121] Updated weights for policy 0, policy_version 74480 (0.0011) +[2023-10-09 15:08:49,706][86122] Updated weights for policy 1, policy_version 74760 (0.0008) +[2023-10-09 15:08:49,831][86121] Updated weights for policy 0, policy_version 74490 (0.0008) +[2023-10-09 15:08:50,067][86122] Updated weights for policy 1, policy_version 74770 (0.0007) +[2023-10-09 15:08:50,426][86122] Updated weights for policy 1, policy_version 74780 (0.0009) +[2023-10-09 15:08:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152862720. Throughput: 0: 1809.1, 1: 1820.6. Samples: 38230190. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:53,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.980')] +[2023-10-09 15:08:53,647][86121] Updated weights for policy 0, policy_version 74500 (0.0009) +[2023-10-09 15:08:54,009][86121] Updated weights for policy 0, policy_version 74510 (0.0007) +[2023-10-09 15:08:54,253][86122] Updated weights for policy 1, policy_version 74790 (0.0009) +[2023-10-09 15:08:54,368][86121] Updated weights for policy 0, policy_version 74520 (0.0007) +[2023-10-09 15:08:54,621][86122] Updated weights for policy 1, policy_version 74800 (0.0007) +[2023-10-09 15:08:54,986][86122] Updated weights for policy 1, policy_version 74810 (0.0008) +[2023-10-09 15:08:58,148][86121] Updated weights for policy 0, policy_version 74530 (0.0007) +[2023-10-09 15:08:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 152928256. Throughput: 0: 1813.7, 1: 1820.5. Samples: 38240310. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:08:58,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.980')] +[2023-10-09 15:08:58,535][86121] Updated weights for policy 0, policy_version 74540 (0.0008) +[2023-10-09 15:08:58,588][86122] Updated weights for policy 1, policy_version 74820 (0.0007) +[2023-10-09 15:08:58,899][86121] Updated weights for policy 0, policy_version 74550 (0.0007) +[2023-10-09 15:08:58,947][86122] Updated weights for policy 1, policy_version 74830 (0.0010) +[2023-10-09 15:08:59,268][86121] Updated weights for policy 0, policy_version 74560 (0.0009) +[2023-10-09 15:08:59,309][86122] Updated weights for policy 1, policy_version 74840 (0.0009) +[2023-10-09 15:09:03,020][86122] Updated weights for policy 1, policy_version 74850 (0.0009) +[2023-10-09 15:09:03,062][86121] Updated weights for policy 0, policy_version 74570 (0.0008) +[2023-10-09 15:09:03,384][86122] Updated weights for policy 1, policy_version 74860 (0.0008) +[2023-10-09 15:09:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 152993792. Throughput: 0: 1815.3, 1: 1817.0. Samples: 38262766. Policy #0 lag: (min: 11.0, avg: 17.9, max: 43.0) +[2023-10-09 15:09:03,398][85186] Avg episode reward: [(0, '9.830'), (1, '9.980')] +[2023-10-09 15:09:03,439][86121] Updated weights for policy 0, policy_version 74580 (0.0008) +[2023-10-09 15:09:03,748][86122] Updated weights for policy 1, policy_version 74870 (0.0008) +[2023-10-09 15:09:03,797][86121] Updated weights for policy 0, policy_version 74590 (0.0009) +[2023-10-09 15:09:04,113][86122] Updated weights for policy 1, policy_version 74880 (0.0009) +[2023-10-09 15:09:07,518][86121] Updated weights for policy 0, policy_version 74600 (0.0007) +[2023-10-09 15:09:07,876][86122] Updated weights for policy 1, policy_version 74890 (0.0008) +[2023-10-09 15:09:07,886][86121] Updated weights for policy 0, policy_version 74610 (0.0008) +[2023-10-09 15:09:08,243][86122] Updated weights for policy 1, policy_version 74900 (0.0008) +[2023-10-09 15:09:08,256][86121] Updated weights for policy 0, policy_version 74620 (0.0009) +[2023-10-09 15:09:08,398][85186] Fps is (10 sec: 16383.3, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 153092096. Throughput: 0: 1813.7, 1: 1818.0. Samples: 38284176. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:08,399][85186] Avg episode reward: [(0, '9.810'), (1, '9.980')] +[2023-10-09 15:09:08,596][86122] Updated weights for policy 1, policy_version 74910 (0.0010) +[2023-10-09 15:09:11,917][86121] Updated weights for policy 0, policy_version 74630 (0.0009) +[2023-10-09 15:09:12,278][86121] Updated weights for policy 0, policy_version 74640 (0.0008) +[2023-10-09 15:09:12,368][86122] Updated weights for policy 1, policy_version 74920 (0.0008) +[2023-10-09 15:09:12,640][86121] Updated weights for policy 0, policy_version 74650 (0.0009) +[2023-10-09 15:09:12,736][86122] Updated weights for policy 1, policy_version 74930 (0.0008) +[2023-10-09 15:09:13,087][86122] Updated weights for policy 1, policy_version 74940 (0.0007) +[2023-10-09 15:09:13,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 153190400. Throughput: 0: 1811.6, 1: 1812.4. Samples: 38295394. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:13,398][85186] Avg episode reward: [(0, '9.800'), (1, '9.980')] +[2023-10-09 15:09:16,415][86121] Updated weights for policy 0, policy_version 74660 (0.0009) +[2023-10-09 15:09:16,784][86121] Updated weights for policy 0, policy_version 74670 (0.0010) +[2023-10-09 15:09:17,016][86122] Updated weights for policy 1, policy_version 74950 (0.0008) +[2023-10-09 15:09:17,150][86121] Updated weights for policy 0, policy_version 74680 (0.0007) +[2023-10-09 15:09:17,372][86122] Updated weights for policy 1, policy_version 74960 (0.0007) +[2023-10-09 15:09:17,732][86122] Updated weights for policy 1, policy_version 74970 (0.0008) +[2023-10-09 15:09:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 153255936. Throughput: 0: 1814.8, 1: 1816.7. Samples: 38316916. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:18,398][85186] Avg episode reward: [(0, '9.770'), (1, '9.980')] +[2023-10-09 15:09:20,844][86121] Updated weights for policy 0, policy_version 74690 (0.0008) +[2023-10-09 15:09:21,217][86121] Updated weights for policy 0, policy_version 74700 (0.0010) +[2023-10-09 15:09:21,513][86122] Updated weights for policy 1, policy_version 74980 (0.0009) +[2023-10-09 15:09:21,584][86121] Updated weights for policy 0, policy_version 74710 (0.0009) +[2023-10-09 15:09:21,868][86122] Updated weights for policy 1, policy_version 74990 (0.0010) +[2023-10-09 15:09:21,942][86121] Updated weights for policy 0, policy_version 74720 (0.0008) +[2023-10-09 15:09:22,231][86122] Updated weights for policy 1, policy_version 75000 (0.0007) +[2023-10-09 15:09:23,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 153321472. Throughput: 0: 1808.4, 1: 1804.9. Samples: 38337430. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:23,399][85186] Avg episode reward: [(0, '9.760'), (1, '9.970')] +[2023-10-09 15:09:23,411][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000075008_76808192.pth... +[2023-10-09 15:09:23,412][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000074720_76513280.pth... +[2023-10-09 15:09:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000073024_74776576.pth +[2023-10-09 15:09:23,449][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000073312_75071488.pth +[2023-10-09 15:09:25,731][86121] Updated weights for policy 0, policy_version 74730 (0.0009) +[2023-10-09 15:09:25,927][86122] Updated weights for policy 1, policy_version 75010 (0.0009) +[2023-10-09 15:09:26,101][86121] Updated weights for policy 0, policy_version 74740 (0.0008) +[2023-10-09 15:09:26,286][86122] Updated weights for policy 1, policy_version 75020 (0.0008) +[2023-10-09 15:09:26,461][86121] Updated weights for policy 0, policy_version 74750 (0.0009) +[2023-10-09 15:09:26,646][86122] Updated weights for policy 1, policy_version 75030 (0.0007) +[2023-10-09 15:09:27,007][86122] Updated weights for policy 1, policy_version 75040 (0.0007) +[2023-10-09 15:09:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 153387008. Throughput: 0: 1825.9, 1: 1807.9. Samples: 38349386. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:28,398][85186] Avg episode reward: [(0, '9.760'), (1, '9.970')] +[2023-10-09 15:09:30,205][86121] Updated weights for policy 0, policy_version 74760 (0.0007) +[2023-10-09 15:09:30,573][86121] Updated weights for policy 0, policy_version 74770 (0.0007) +[2023-10-09 15:09:30,767][86122] Updated weights for policy 1, policy_version 75050 (0.0008) +[2023-10-09 15:09:30,935][86121] Updated weights for policy 0, policy_version 74780 (0.0007) +[2023-10-09 15:09:31,122][86122] Updated weights for policy 1, policy_version 75060 (0.0008) +[2023-10-09 15:09:31,485][86122] Updated weights for policy 1, policy_version 75070 (0.0008) +[2023-10-09 15:09:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153452544. Throughput: 0: 1798.2, 1: 1799.5. Samples: 38369612. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:33,398][85186] Avg episode reward: [(0, '9.720'), (1, '9.970')] +[2023-10-09 15:09:34,419][86121] Updated weights for policy 0, policy_version 74790 (0.0008) +[2023-10-09 15:09:34,790][86121] Updated weights for policy 0, policy_version 74800 (0.0009) +[2023-10-09 15:09:35,153][86121] Updated weights for policy 0, policy_version 74810 (0.0008) +[2023-10-09 15:09:35,216][86122] Updated weights for policy 1, policy_version 75080 (0.0008) +[2023-10-09 15:09:35,571][86122] Updated weights for policy 1, policy_version 75090 (0.0009) +[2023-10-09 15:09:35,937][86122] Updated weights for policy 1, policy_version 75100 (0.0008) +[2023-10-09 15:09:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 153518080. Throughput: 0: 1800.7, 1: 1806.0. Samples: 38392492. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:38,398][85186] Avg episode reward: [(0, '9.700'), (1, '9.970')] +[2023-10-09 15:09:39,028][86121] Updated weights for policy 0, policy_version 74820 (0.0010) +[2023-10-09 15:09:39,397][86121] Updated weights for policy 0, policy_version 74830 (0.0008) +[2023-10-09 15:09:39,584][86122] Updated weights for policy 1, policy_version 75110 (0.0010) +[2023-10-09 15:09:39,766][86121] Updated weights for policy 0, policy_version 74840 (0.0008) +[2023-10-09 15:09:39,954][86122] Updated weights for policy 1, policy_version 75120 (0.0007) +[2023-10-09 15:09:40,312][86122] Updated weights for policy 1, policy_version 75130 (0.0009) +[2023-10-09 15:09:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153583616. Throughput: 0: 1793.8, 1: 1805.6. Samples: 38402284. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:43,398][85186] Avg episode reward: [(0, '9.700'), (1, '9.970')] +[2023-10-09 15:09:43,544][86121] Updated weights for policy 0, policy_version 74850 (0.0009) +[2023-10-09 15:09:43,839][86122] Updated weights for policy 1, policy_version 75140 (0.0008) +[2023-10-09 15:09:43,941][86121] Updated weights for policy 0, policy_version 74860 (0.0008) +[2023-10-09 15:09:44,195][86122] Updated weights for policy 1, policy_version 75150 (0.0008) +[2023-10-09 15:09:44,309][86121] Updated weights for policy 0, policy_version 74870 (0.0008) +[2023-10-09 15:09:44,557][86122] Updated weights for policy 1, policy_version 75160 (0.0008) +[2023-10-09 15:09:44,675][86121] Updated weights for policy 0, policy_version 74880 (0.0008) +[2023-10-09 15:09:48,208][86122] Updated weights for policy 1, policy_version 75170 (0.0007) +[2023-10-09 15:09:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153649152. Throughput: 0: 1791.7, 1: 1812.1. Samples: 38424936. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:48,398][85186] Avg episode reward: [(0, '9.680'), (1, '9.970')] +[2023-10-09 15:09:48,487][86121] Updated weights for policy 0, policy_version 74890 (0.0007) +[2023-10-09 15:09:48,567][86122] Updated weights for policy 1, policy_version 75180 (0.0007) +[2023-10-09 15:09:48,854][86121] Updated weights for policy 0, policy_version 74900 (0.0008) +[2023-10-09 15:09:48,921][86122] Updated weights for policy 1, policy_version 75190 (0.0008) +[2023-10-09 15:09:49,207][86121] Updated weights for policy 0, policy_version 74910 (0.0008) +[2023-10-09 15:09:49,282][86122] Updated weights for policy 1, policy_version 75200 (0.0008) +[2023-10-09 15:09:52,917][86121] Updated weights for policy 0, policy_version 74920 (0.0007) +[2023-10-09 15:09:53,015][86122] Updated weights for policy 1, policy_version 75210 (0.0009) +[2023-10-09 15:09:53,280][86121] Updated weights for policy 0, policy_version 74930 (0.0008) +[2023-10-09 15:09:53,375][86122] Updated weights for policy 1, policy_version 75220 (0.0007) +[2023-10-09 15:09:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 153714688. Throughput: 0: 1804.8, 1: 1822.1. Samples: 38447384. Policy #0 lag: (min: 10.0, avg: 10.7, max: 23.0) +[2023-10-09 15:09:53,398][85186] Avg episode reward: [(0, '9.670'), (1, '9.970')] +[2023-10-09 15:09:53,650][86121] Updated weights for policy 0, policy_version 74940 (0.0008) +[2023-10-09 15:09:53,740][86122] Updated weights for policy 1, policy_version 75230 (0.0008) +[2023-10-09 15:09:57,423][86121] Updated weights for policy 0, policy_version 74950 (0.0009) +[2023-10-09 15:09:57,588][86122] Updated weights for policy 1, policy_version 75240 (0.0009) +[2023-10-09 15:09:57,794][86121] Updated weights for policy 0, policy_version 74960 (0.0007) +[2023-10-09 15:09:57,956][86122] Updated weights for policy 1, policy_version 75250 (0.0010) +[2023-10-09 15:09:58,155][86121] Updated weights for policy 0, policy_version 74970 (0.0008) +[2023-10-09 15:09:58,318][86122] Updated weights for policy 1, policy_version 75260 (0.0008) +[2023-10-09 15:09:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 153812992. Throughput: 0: 1792.8, 1: 1819.1. Samples: 38457930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:09:58,398][85186] Avg episode reward: [(0, '9.670'), (1, '9.970')] +[2023-10-09 15:10:01,905][86121] Updated weights for policy 0, policy_version 74980 (0.0009) +[2023-10-09 15:10:01,940][86122] Updated weights for policy 1, policy_version 75270 (0.0009) +[2023-10-09 15:10:02,264][86121] Updated weights for policy 0, policy_version 74990 (0.0009) +[2023-10-09 15:10:02,293][86122] Updated weights for policy 1, policy_version 75280 (0.0007) +[2023-10-09 15:10:02,629][86121] Updated weights for policy 0, policy_version 75000 (0.0009) +[2023-10-09 15:10:02,656][86122] Updated weights for policy 1, policy_version 75290 (0.0007) +[2023-10-09 15:10:03,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 153911296. Throughput: 0: 1803.1, 1: 1815.5. Samples: 38479754. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:03,398][85186] Avg episode reward: [(0, '9.670'), (1, '9.970')] +[2023-10-09 15:10:06,397][86122] Updated weights for policy 1, policy_version 75300 (0.0009) +[2023-10-09 15:10:06,471][86121] Updated weights for policy 0, policy_version 75010 (0.0007) +[2023-10-09 15:10:06,752][86122] Updated weights for policy 1, policy_version 75310 (0.0008) +[2023-10-09 15:10:06,834][86121] Updated weights for policy 0, policy_version 75020 (0.0007) +[2023-10-09 15:10:07,109][86122] Updated weights for policy 1, policy_version 75320 (0.0008) +[2023-10-09 15:10:07,203][86121] Updated weights for policy 0, policy_version 75030 (0.0008) +[2023-10-09 15:10:07,565][86121] Updated weights for policy 0, policy_version 75040 (0.0009) +[2023-10-09 15:10:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 153976832. Throughput: 0: 1789.3, 1: 1820.9. Samples: 38499890. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:08,398][85186] Avg episode reward: [(0, '9.670'), (1, '9.970')] +[2023-10-09 15:10:10,708][86122] Updated weights for policy 1, policy_version 75330 (0.0007) +[2023-10-09 15:10:11,072][86122] Updated weights for policy 1, policy_version 75340 (0.0008) +[2023-10-09 15:10:11,387][86121] Updated weights for policy 0, policy_version 75050 (0.0007) +[2023-10-09 15:10:11,441][86122] Updated weights for policy 1, policy_version 75350 (0.0007) +[2023-10-09 15:10:11,751][86121] Updated weights for policy 0, policy_version 75060 (0.0008) +[2023-10-09 15:10:11,801][86122] Updated weights for policy 1, policy_version 75360 (0.0008) +[2023-10-09 15:10:12,115][86121] Updated weights for policy 0, policy_version 75070 (0.0010) +[2023-10-09 15:10:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 154042368. Throughput: 0: 1801.5, 1: 1823.9. Samples: 38512526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:13,398][85186] Avg episode reward: [(0, '9.660'), (1, '9.970')] +[2023-10-09 15:10:15,662][86122] Updated weights for policy 1, policy_version 75370 (0.0010) +[2023-10-09 15:10:16,011][86121] Updated weights for policy 0, policy_version 75080 (0.0008) +[2023-10-09 15:10:16,030][86122] Updated weights for policy 1, policy_version 75380 (0.0009) +[2023-10-09 15:10:16,378][86121] Updated weights for policy 0, policy_version 75090 (0.0009) +[2023-10-09 15:10:16,392][86122] Updated weights for policy 1, policy_version 75390 (0.0008) +[2023-10-09 15:10:16,749][86121] Updated weights for policy 0, policy_version 75100 (0.0009) +[2023-10-09 15:10:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154107904. Throughput: 0: 1791.4, 1: 1822.9. Samples: 38532256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:18,398][85186] Avg episode reward: [(0, '9.670'), (1, '9.970')] +[2023-10-09 15:10:20,295][86122] Updated weights for policy 1, policy_version 75400 (0.0008) +[2023-10-09 15:10:20,366][86121] Updated weights for policy 0, policy_version 75110 (0.0011) +[2023-10-09 15:10:20,655][86122] Updated weights for policy 1, policy_version 75410 (0.0008) +[2023-10-09 15:10:20,736][86121] Updated weights for policy 0, policy_version 75120 (0.0009) +[2023-10-09 15:10:21,017][86122] Updated weights for policy 1, policy_version 75420 (0.0009) +[2023-10-09 15:10:21,100][86121] Updated weights for policy 0, policy_version 75130 (0.0008) +[2023-10-09 15:10:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154173440. Throughput: 0: 1793.6, 1: 1811.4. Samples: 38554716. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:23,398][85186] Avg episode reward: [(0, '9.670'), (1, '9.980')] +[2023-10-09 15:10:24,818][86121] Updated weights for policy 0, policy_version 75140 (0.0008) +[2023-10-09 15:10:24,830][86122] Updated weights for policy 1, policy_version 75430 (0.0008) +[2023-10-09 15:10:25,179][86121] Updated weights for policy 0, policy_version 75150 (0.0007) +[2023-10-09 15:10:25,196][86122] Updated weights for policy 1, policy_version 75440 (0.0008) +[2023-10-09 15:10:25,539][86121] Updated weights for policy 0, policy_version 75160 (0.0007) +[2023-10-09 15:10:25,553][86122] Updated weights for policy 1, policy_version 75450 (0.0007) +[2023-10-09 15:10:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154238976. Throughput: 0: 1794.4, 1: 1809.0. Samples: 38564440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:28,398][85186] Avg episode reward: [(0, '9.670'), (1, '9.980')] +[2023-10-09 15:10:29,099][86121] Updated weights for policy 0, policy_version 75170 (0.0008) +[2023-10-09 15:10:29,234][86122] Updated weights for policy 1, policy_version 75460 (0.0007) +[2023-10-09 15:10:29,463][86121] Updated weights for policy 0, policy_version 75180 (0.0007) +[2023-10-09 15:10:29,599][86122] Updated weights for policy 1, policy_version 75470 (0.0008) +[2023-10-09 15:10:29,826][86121] Updated weights for policy 0, policy_version 75190 (0.0008) +[2023-10-09 15:10:29,973][86122] Updated weights for policy 1, policy_version 75480 (0.0008) +[2023-10-09 15:10:30,187][86121] Updated weights for policy 0, policy_version 75200 (0.0008) +[2023-10-09 15:10:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154304512. Throughput: 0: 1802.8, 1: 1810.2. Samples: 38587524. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:33,398][85186] Avg episode reward: [(0, '9.700'), (1, '9.980')] +[2023-10-09 15:10:33,665][86122] Updated weights for policy 1, policy_version 75490 (0.0007) +[2023-10-09 15:10:33,992][86121] Updated weights for policy 0, policy_version 75210 (0.0008) +[2023-10-09 15:10:34,026][86122] Updated weights for policy 1, policy_version 75500 (0.0009) +[2023-10-09 15:10:34,355][86121] Updated weights for policy 0, policy_version 75220 (0.0009) +[2023-10-09 15:10:34,396][86122] Updated weights for policy 1, policy_version 75510 (0.0009) +[2023-10-09 15:10:34,714][86121] Updated weights for policy 0, policy_version 75230 (0.0010) +[2023-10-09 15:10:34,762][86122] Updated weights for policy 1, policy_version 75520 (0.0010) +[2023-10-09 15:10:38,308][86122] Updated weights for policy 1, policy_version 75530 (0.0008) +[2023-10-09 15:10:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154370048. Throughput: 0: 1802.7, 1: 1813.4. Samples: 38610108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:38,398][85186] Avg episode reward: [(0, '9.710'), (1, '9.990')] +[2023-10-09 15:10:38,475][86121] Updated weights for policy 0, policy_version 75240 (0.0009) +[2023-10-09 15:10:38,667][86122] Updated weights for policy 1, policy_version 75540 (0.0008) +[2023-10-09 15:10:38,835][86121] Updated weights for policy 0, policy_version 75250 (0.0009) +[2023-10-09 15:10:39,022][86122] Updated weights for policy 1, policy_version 75550 (0.0007) +[2023-10-09 15:10:39,209][86121] Updated weights for policy 0, policy_version 75260 (0.0008) +[2023-10-09 15:10:42,769][86122] Updated weights for policy 1, policy_version 75560 (0.0007) +[2023-10-09 15:10:42,907][86121] Updated weights for policy 0, policy_version 75270 (0.0007) +[2023-10-09 15:10:43,141][86122] Updated weights for policy 1, policy_version 75570 (0.0008) +[2023-10-09 15:10:43,280][86121] Updated weights for policy 0, policy_version 75280 (0.0008) +[2023-10-09 15:10:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154435584. Throughput: 0: 1793.8, 1: 1809.1. Samples: 38620062. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:43,398][85186] Avg episode reward: [(0, '9.710'), (1, '9.990')] +[2023-10-09 15:10:43,502][86122] Updated weights for policy 1, policy_version 75580 (0.0007) +[2023-10-09 15:10:43,655][86121] Updated weights for policy 0, policy_version 75290 (0.0008) +[2023-10-09 15:10:47,070][86122] Updated weights for policy 1, policy_version 75590 (0.0007) +[2023-10-09 15:10:47,379][86121] Updated weights for policy 0, policy_version 75300 (0.0010) +[2023-10-09 15:10:47,428][86122] Updated weights for policy 1, policy_version 75600 (0.0008) +[2023-10-09 15:10:47,748][86121] Updated weights for policy 0, policy_version 75310 (0.0007) +[2023-10-09 15:10:47,791][86122] Updated weights for policy 1, policy_version 75610 (0.0009) +[2023-10-09 15:10:48,109][86121] Updated weights for policy 0, policy_version 75320 (0.0008) +[2023-10-09 15:10:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 154533888. Throughput: 0: 1801.0, 1: 1820.6. Samples: 38642728. Policy #0 lag: (min: 31.0, avg: 31.0, max: 33.0) +[2023-10-09 15:10:48,398][85186] Avg episode reward: [(0, '9.710'), (1, '9.990')] +[2023-10-09 15:10:51,587][86122] Updated weights for policy 1, policy_version 75620 (0.0009) +[2023-10-09 15:10:51,882][86121] Updated weights for policy 0, policy_version 75330 (0.0007) +[2023-10-09 15:10:51,948][86122] Updated weights for policy 1, policy_version 75630 (0.0009) +[2023-10-09 15:10:52,246][86121] Updated weights for policy 0, policy_version 75340 (0.0008) +[2023-10-09 15:10:52,318][86122] Updated weights for policy 1, policy_version 75640 (0.0009) +[2023-10-09 15:10:52,611][86121] Updated weights for policy 0, policy_version 75350 (0.0008) +[2023-10-09 15:10:52,971][86121] Updated weights for policy 0, policy_version 75360 (0.0009) +[2023-10-09 15:10:53,397][85186] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 154632192. Throughput: 0: 1803.6, 1: 1812.8. Samples: 38662630. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:10:53,399][85186] Avg episode reward: [(0, '9.750'), (1, '9.990')] +[2023-10-09 15:10:56,095][86122] Updated weights for policy 1, policy_version 75650 (0.0007) +[2023-10-09 15:10:56,450][86122] Updated weights for policy 1, policy_version 75660 (0.0007) +[2023-10-09 15:10:56,678][86121] Updated weights for policy 0, policy_version 75370 (0.0007) +[2023-10-09 15:10:56,816][86122] Updated weights for policy 1, policy_version 75670 (0.0008) +[2023-10-09 15:10:57,040][86121] Updated weights for policy 0, policy_version 75380 (0.0008) +[2023-10-09 15:10:57,172][86122] Updated weights for policy 1, policy_version 75680 (0.0008) +[2023-10-09 15:10:57,398][86121] Updated weights for policy 0, policy_version 75390 (0.0009) +[2023-10-09 15:10:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 154697728. Throughput: 0: 1799.9, 1: 1816.9. Samples: 38675280. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:10:58,398][85186] Avg episode reward: [(0, '9.780'), (1, '9.990')] +[2023-10-09 15:11:00,927][86122] Updated weights for policy 1, policy_version 75690 (0.0011) +[2023-10-09 15:11:01,287][86122] Updated weights for policy 1, policy_version 75700 (0.0007) +[2023-10-09 15:11:01,312][86121] Updated weights for policy 0, policy_version 75400 (0.0010) +[2023-10-09 15:11:01,645][86122] Updated weights for policy 1, policy_version 75710 (0.0008) +[2023-10-09 15:11:01,676][86121] Updated weights for policy 0, policy_version 75410 (0.0008) +[2023-10-09 15:11:02,040][86121] Updated weights for policy 0, policy_version 75420 (0.0007) +[2023-10-09 15:11:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154763264. Throughput: 0: 1810.4, 1: 1814.0. Samples: 38695352. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:11:03,398][85186] Avg episode reward: [(0, '9.780'), (1, '9.990')] +[2023-10-09 15:11:05,317][86122] Updated weights for policy 1, policy_version 75720 (0.0009) +[2023-10-09 15:11:05,666][86121] Updated weights for policy 0, policy_version 75430 (0.0007) +[2023-10-09 15:11:05,672][86122] Updated weights for policy 1, policy_version 75730 (0.0008) +[2023-10-09 15:11:06,026][86121] Updated weights for policy 0, policy_version 75440 (0.0007) +[2023-10-09 15:11:06,034][86122] Updated weights for policy 1, policy_version 75740 (0.0007) +[2023-10-09 15:11:06,393][86121] Updated weights for policy 0, policy_version 75450 (0.0008) +[2023-10-09 15:11:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 154828800. Throughput: 0: 1801.5, 1: 1825.5. Samples: 38717928. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:11:08,399][85186] Avg episode reward: [(0, '9.780'), (1, '9.990')] +[2023-10-09 15:11:09,765][86122] Updated weights for policy 1, policy_version 75750 (0.0009) +[2023-10-09 15:11:10,131][86122] Updated weights for policy 1, policy_version 75760 (0.0008) +[2023-10-09 15:11:10,152][86121] Updated weights for policy 0, policy_version 75460 (0.0007) +[2023-10-09 15:11:10,488][86122] Updated weights for policy 1, policy_version 75770 (0.0009) +[2023-10-09 15:11:10,517][86121] Updated weights for policy 0, policy_version 75470 (0.0008) +[2023-10-09 15:11:10,891][86121] Updated weights for policy 0, policy_version 75480 (0.0010) +[2023-10-09 15:11:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 154894336. Throughput: 0: 1812.0, 1: 1826.0. Samples: 38728152. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:11:13,398][85186] Avg episode reward: [(0, '9.810'), (1, '9.990')] +[2023-10-09 15:11:13,954][86122] Updated weights for policy 1, policy_version 75780 (0.0007) +[2023-10-09 15:11:14,312][86122] Updated weights for policy 1, policy_version 75790 (0.0008) +[2023-10-09 15:11:14,535][86121] Updated weights for policy 0, policy_version 75490 (0.0007) +[2023-10-09 15:11:14,679][86122] Updated weights for policy 1, policy_version 75800 (0.0007) +[2023-10-09 15:11:14,902][86121] Updated weights for policy 0, policy_version 75500 (0.0007) +[2023-10-09 15:11:15,282][86121] Updated weights for policy 0, policy_version 75510 (0.0008) +[2023-10-09 15:11:15,635][86121] Updated weights for policy 0, policy_version 75520 (0.0011) +[2023-10-09 15:11:18,297][86122] Updated weights for policy 1, policy_version 75810 (0.0009) +[2023-10-09 15:11:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 154959872. Throughput: 0: 1801.9, 1: 1832.8. Samples: 38751084. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:11:18,398][85186] Avg episode reward: [(0, '9.800'), (1, '10.000')] +[2023-10-09 15:11:18,653][86122] Updated weights for policy 1, policy_version 75820 (0.0008) +[2023-10-09 15:11:19,016][86122] Updated weights for policy 1, policy_version 75830 (0.0009) +[2023-10-09 15:11:19,380][86122] Updated weights for policy 1, policy_version 75840 (0.0010) +[2023-10-09 15:11:19,413][86121] Updated weights for policy 0, policy_version 75530 (0.0009) +[2023-10-09 15:11:19,783][86121] Updated weights for policy 0, policy_version 75540 (0.0008) +[2023-10-09 15:11:20,143][86121] Updated weights for policy 0, policy_version 75550 (0.0009) +[2023-10-09 15:11:23,023][86122] Updated weights for policy 1, policy_version 75850 (0.0010) +[2023-10-09 15:11:23,370][86122] Updated weights for policy 1, policy_version 75860 (0.0007) +[2023-10-09 15:11:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155025408. Throughput: 0: 1806.4, 1: 1830.9. Samples: 38773788. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:11:23,398][85186] Avg episode reward: [(0, '9.800'), (1, '10.000')] +[2023-10-09 15:11:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000075552_77365248.pth... +[2023-10-09 15:11:23,443][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000073888_75661312.pth +[2023-10-09 15:11:23,735][86122] Updated weights for policy 1, policy_version 75870 (0.0008) +[2023-10-09 15:11:23,797][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000075872_77692928.pth... +[2023-10-09 15:11:23,827][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000074144_75923456.pth +[2023-10-09 15:11:23,909][86121] Updated weights for policy 0, policy_version 75560 (0.0008) +[2023-10-09 15:11:24,280][86121] Updated weights for policy 0, policy_version 75570 (0.0007) +[2023-10-09 15:11:24,642][86121] Updated weights for policy 0, policy_version 75580 (0.0007) +[2023-10-09 15:11:27,236][86122] Updated weights for policy 1, policy_version 75880 (0.0008) +[2023-10-09 15:11:27,595][86122] Updated weights for policy 1, policy_version 75890 (0.0008) +[2023-10-09 15:11:27,959][86122] Updated weights for policy 1, policy_version 75900 (0.0009) +[2023-10-09 15:11:28,308][86121] Updated weights for policy 0, policy_version 75590 (0.0008) +[2023-10-09 15:11:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155123712. Throughput: 0: 1804.3, 1: 1837.4. Samples: 38783940. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:11:28,398][85186] Avg episode reward: [(0, '9.810'), (1, '10.000')] +[2023-10-09 15:11:28,668][86121] Updated weights for policy 0, policy_version 75600 (0.0007) +[2023-10-09 15:11:29,032][86121] Updated weights for policy 0, policy_version 75610 (0.0007) +[2023-10-09 15:11:31,832][86122] Updated weights for policy 1, policy_version 75910 (0.0009) +[2023-10-09 15:11:32,206][86122] Updated weights for policy 1, policy_version 75920 (0.0007) +[2023-10-09 15:11:32,563][86122] Updated weights for policy 1, policy_version 75930 (0.0007) +[2023-10-09 15:11:32,713][86121] Updated weights for policy 0, policy_version 75620 (0.0007) +[2023-10-09 15:11:33,082][86121] Updated weights for policy 0, policy_version 75630 (0.0008) +[2023-10-09 15:11:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155189248. Throughput: 0: 1806.9, 1: 1830.3. Samples: 38806402. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:11:33,398][85186] Avg episode reward: [(0, '9.850'), (1, '10.000')] +[2023-10-09 15:11:33,449][86121] Updated weights for policy 0, policy_version 75640 (0.0008) +[2023-10-09 15:11:36,129][86122] Updated weights for policy 1, policy_version 75940 (0.0007) +[2023-10-09 15:11:36,486][86122] Updated weights for policy 1, policy_version 75950 (0.0010) +[2023-10-09 15:11:36,854][86122] Updated weights for policy 1, policy_version 75960 (0.0009) +[2023-10-09 15:11:37,207][86121] Updated weights for policy 0, policy_version 75650 (0.0007) +[2023-10-09 15:11:37,579][86121] Updated weights for policy 0, policy_version 75660 (0.0008) +[2023-10-09 15:11:37,952][86121] Updated weights for policy 0, policy_version 75670 (0.0010) +[2023-10-09 15:11:38,326][86121] Updated weights for policy 0, policy_version 75680 (0.0009) +[2023-10-09 15:11:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155287552. Throughput: 0: 1812.4, 1: 1841.8. Samples: 38827068. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:11:38,398][85186] Avg episode reward: [(0, '9.870'), (1, '10.000')] +[2023-10-09 15:11:40,558][86122] Updated weights for policy 1, policy_version 75970 (0.0009) +[2023-10-09 15:11:40,923][86122] Updated weights for policy 1, policy_version 75980 (0.0009) +[2023-10-09 15:11:41,276][86122] Updated weights for policy 1, policy_version 75990 (0.0008) +[2023-10-09 15:11:41,643][86122] Updated weights for policy 1, policy_version 76000 (0.0009) +[2023-10-09 15:11:41,931][86121] Updated weights for policy 0, policy_version 75690 (0.0009) +[2023-10-09 15:11:42,305][86121] Updated weights for policy 0, policy_version 75700 (0.0009) +[2023-10-09 15:11:42,680][86121] Updated weights for policy 0, policy_version 75710 (0.0009) +[2023-10-09 15:11:43,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155353088. Throughput: 0: 1804.0, 1: 1826.8. Samples: 38838670. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:11:43,398][85186] Avg episode reward: [(0, '9.870'), (1, '10.000')] +[2023-10-09 15:11:45,489][86122] Updated weights for policy 1, policy_version 76010 (0.0010) +[2023-10-09 15:11:45,851][86122] Updated weights for policy 1, policy_version 76020 (0.0008) +[2023-10-09 15:11:46,211][86122] Updated weights for policy 1, policy_version 76030 (0.0008) +[2023-10-09 15:11:46,412][86121] Updated weights for policy 0, policy_version 75720 (0.0009) +[2023-10-09 15:11:46,778][86121] Updated weights for policy 0, policy_version 75730 (0.0008) +[2023-10-09 15:11:47,148][86121] Updated weights for policy 0, policy_version 75740 (0.0007) +[2023-10-09 15:11:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155418624. Throughput: 0: 1811.3, 1: 1840.0. Samples: 38859664. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:11:48,398][85186] Avg episode reward: [(0, '9.870'), (1, '10.000')] +[2023-10-09 15:11:49,794][86122] Updated weights for policy 1, policy_version 76040 (0.0008) +[2023-10-09 15:11:50,165][86122] Updated weights for policy 1, policy_version 76050 (0.0008) +[2023-10-09 15:11:50,524][86122] Updated weights for policy 1, policy_version 76060 (0.0008) +[2023-10-09 15:11:50,797][86121] Updated weights for policy 0, policy_version 75750 (0.0009) +[2023-10-09 15:11:51,167][86121] Updated weights for policy 0, policy_version 75760 (0.0010) +[2023-10-09 15:11:51,548][86121] Updated weights for policy 0, policy_version 75770 (0.0011) +[2023-10-09 15:11:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 155484160. Throughput: 0: 1813.2, 1: 1838.0. Samples: 38882232. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:11:53,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 15:11:54,247][86122] Updated weights for policy 1, policy_version 76070 (0.0008) +[2023-10-09 15:11:54,606][86122] Updated weights for policy 1, policy_version 76080 (0.0008) +[2023-10-09 15:11:54,970][86122] Updated weights for policy 1, policy_version 76090 (0.0007) +[2023-10-09 15:11:55,197][86121] Updated weights for policy 0, policy_version 75780 (0.0008) +[2023-10-09 15:11:55,561][86121] Updated weights for policy 0, policy_version 75790 (0.0009) +[2023-10-09 15:11:55,927][86121] Updated weights for policy 0, policy_version 75800 (0.0008) +[2023-10-09 15:11:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 155549696. Throughput: 0: 1815.1, 1: 1841.4. Samples: 38892694. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:11:58,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 15:11:58,648][86122] Updated weights for policy 1, policy_version 76100 (0.0007) +[2023-10-09 15:11:59,011][86122] Updated weights for policy 1, policy_version 76110 (0.0008) +[2023-10-09 15:11:59,382][86122] Updated weights for policy 1, policy_version 76120 (0.0008) +[2023-10-09 15:11:59,549][86121] Updated weights for policy 0, policy_version 75810 (0.0008) +[2023-10-09 15:11:59,919][86121] Updated weights for policy 0, policy_version 75820 (0.0007) +[2023-10-09 15:12:00,285][86121] Updated weights for policy 0, policy_version 75830 (0.0011) +[2023-10-09 15:12:00,656][86121] Updated weights for policy 0, policy_version 75840 (0.0008) +[2023-10-09 15:12:03,051][86122] Updated weights for policy 1, policy_version 76130 (0.0008) +[2023-10-09 15:12:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155615232. Throughput: 0: 1818.7, 1: 1834.8. Samples: 38915494. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:12:03,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 15:12:03,418][86122] Updated weights for policy 1, policy_version 76140 (0.0009) +[2023-10-09 15:12:03,779][86122] Updated weights for policy 1, policy_version 76150 (0.0007) +[2023-10-09 15:12:04,135][86122] Updated weights for policy 1, policy_version 76160 (0.0007) +[2023-10-09 15:12:04,408][86121] Updated weights for policy 0, policy_version 75850 (0.0010) +[2023-10-09 15:12:04,768][86121] Updated weights for policy 0, policy_version 75860 (0.0010) +[2023-10-09 15:12:05,133][86121] Updated weights for policy 0, policy_version 75870 (0.0008) +[2023-10-09 15:12:07,809][86122] Updated weights for policy 1, policy_version 76170 (0.0008) +[2023-10-09 15:12:08,169][86122] Updated weights for policy 1, policy_version 76180 (0.0008) +[2023-10-09 15:12:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 155680768. Throughput: 0: 1814.5, 1: 1827.4. Samples: 38937676. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:12:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 15:12:08,528][86122] Updated weights for policy 1, policy_version 76190 (0.0008) +[2023-10-09 15:12:08,816][86121] Updated weights for policy 0, policy_version 75880 (0.0008) +[2023-10-09 15:12:09,179][86121] Updated weights for policy 0, policy_version 75890 (0.0008) +[2023-10-09 15:12:09,544][86121] Updated weights for policy 0, policy_version 75900 (0.0009) +[2023-10-09 15:12:12,188][86122] Updated weights for policy 1, policy_version 76200 (0.0009) +[2023-10-09 15:12:12,553][86122] Updated weights for policy 1, policy_version 76210 (0.0010) +[2023-10-09 15:12:12,925][86122] Updated weights for policy 1, policy_version 76220 (0.0010) +[2023-10-09 15:12:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155779072. Throughput: 0: 1818.5, 1: 1830.6. Samples: 38948148. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:12:13,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 15:12:13,400][86121] Updated weights for policy 0, policy_version 75910 (0.0008) +[2023-10-09 15:12:13,766][86121] Updated weights for policy 0, policy_version 75920 (0.0007) +[2023-10-09 15:12:14,133][86121] Updated weights for policy 0, policy_version 75930 (0.0007) +[2023-10-09 15:12:16,587][86122] Updated weights for policy 1, policy_version 76230 (0.0010) +[2023-10-09 15:12:16,955][86122] Updated weights for policy 1, policy_version 76240 (0.0010) +[2023-10-09 15:12:17,307][86122] Updated weights for policy 1, policy_version 76250 (0.0007) +[2023-10-09 15:12:17,757][86121] Updated weights for policy 0, policy_version 75940 (0.0008) +[2023-10-09 15:12:18,120][86121] Updated weights for policy 0, policy_version 75950 (0.0009) +[2023-10-09 15:12:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 155844608. Throughput: 0: 1821.4, 1: 1827.0. Samples: 38970578. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:12:18,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 15:12:18,482][86121] Updated weights for policy 0, policy_version 75960 (0.0010) +[2023-10-09 15:12:20,975][86122] Updated weights for policy 1, policy_version 76260 (0.0009) +[2023-10-09 15:12:21,362][86122] Updated weights for policy 1, policy_version 76270 (0.0008) +[2023-10-09 15:12:21,735][86122] Updated weights for policy 1, policy_version 76280 (0.0009) +[2023-10-09 15:12:22,098][86121] Updated weights for policy 0, policy_version 75970 (0.0007) +[2023-10-09 15:12:22,466][86121] Updated weights for policy 0, policy_version 75980 (0.0007) +[2023-10-09 15:12:22,834][86121] Updated weights for policy 0, policy_version 75990 (0.0008) +[2023-10-09 15:12:23,195][86121] Updated weights for policy 0, policy_version 76000 (0.0007) +[2023-10-09 15:12:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 155942912. Throughput: 0: 1821.8, 1: 1833.7. Samples: 38991566. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:12:23,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 15:12:25,266][86122] Updated weights for policy 1, policy_version 76290 (0.0007) +[2023-10-09 15:12:25,617][86122] Updated weights for policy 1, policy_version 76300 (0.0009) +[2023-10-09 15:12:25,988][86122] Updated weights for policy 1, policy_version 76310 (0.0008) +[2023-10-09 15:12:26,349][86122] Updated weights for policy 1, policy_version 76320 (0.0008) +[2023-10-09 15:12:26,867][86121] Updated weights for policy 0, policy_version 76010 (0.0007) +[2023-10-09 15:12:27,226][86121] Updated weights for policy 0, policy_version 76020 (0.0010) +[2023-10-09 15:12:27,586][86121] Updated weights for policy 0, policy_version 76030 (0.0007) +[2023-10-09 15:12:28,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156008448. Throughput: 0: 1827.1, 1: 1833.2. Samples: 39003382. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:12:28,399][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 15:12:29,975][86122] Updated weights for policy 1, policy_version 76330 (0.0008) +[2023-10-09 15:12:30,330][86122] Updated weights for policy 1, policy_version 76340 (0.0010) +[2023-10-09 15:12:30,690][86122] Updated weights for policy 1, policy_version 76350 (0.0010) +[2023-10-09 15:12:31,346][86121] Updated weights for policy 0, policy_version 76040 (0.0008) +[2023-10-09 15:12:31,710][86121] Updated weights for policy 0, policy_version 76050 (0.0009) +[2023-10-09 15:12:32,069][86121] Updated weights for policy 0, policy_version 76060 (0.0008) +[2023-10-09 15:12:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 156073984. Throughput: 0: 1823.3, 1: 1841.9. Samples: 39024594. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:12:33,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 15:12:34,395][86122] Updated weights for policy 1, policy_version 76360 (0.0008) +[2023-10-09 15:12:34,759][86122] Updated weights for policy 1, policy_version 76370 (0.0008) +[2023-10-09 15:12:35,117][86122] Updated weights for policy 1, policy_version 76380 (0.0007) +[2023-10-09 15:12:35,888][86121] Updated weights for policy 0, policy_version 76070 (0.0008) +[2023-10-09 15:12:36,250][86121] Updated weights for policy 0, policy_version 76080 (0.0008) +[2023-10-09 15:12:36,613][86121] Updated weights for policy 0, policy_version 76090 (0.0010) +[2023-10-09 15:12:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156139520. Throughput: 0: 1817.3, 1: 1843.0. Samples: 39046948. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:12:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:12:38,839][86122] Updated weights for policy 1, policy_version 76390 (0.0009) +[2023-10-09 15:12:39,201][86122] Updated weights for policy 1, policy_version 76400 (0.0011) +[2023-10-09 15:12:39,569][86122] Updated weights for policy 1, policy_version 76410 (0.0010) +[2023-10-09 15:12:40,255][86121] Updated weights for policy 0, policy_version 76100 (0.0010) +[2023-10-09 15:12:40,610][86121] Updated weights for policy 0, policy_version 76110 (0.0009) +[2023-10-09 15:12:40,981][86121] Updated weights for policy 0, policy_version 76120 (0.0011) +[2023-10-09 15:12:43,169][86122] Updated weights for policy 1, policy_version 76420 (0.0008) +[2023-10-09 15:12:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156205056. Throughput: 0: 1822.4, 1: 1837.4. Samples: 39057386. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:12:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:12:43,528][86122] Updated weights for policy 1, policy_version 76430 (0.0007) +[2023-10-09 15:12:43,889][86122] Updated weights for policy 1, policy_version 76440 (0.0008) +[2023-10-09 15:12:44,773][86121] Updated weights for policy 0, policy_version 76130 (0.0008) +[2023-10-09 15:12:45,136][86121] Updated weights for policy 0, policy_version 76140 (0.0007) +[2023-10-09 15:12:45,506][86121] Updated weights for policy 0, policy_version 76150 (0.0009) +[2023-10-09 15:12:45,864][86121] Updated weights for policy 0, policy_version 76160 (0.0008) +[2023-10-09 15:12:47,747][86122] Updated weights for policy 1, policy_version 76450 (0.0008) +[2023-10-09 15:12:48,111][86122] Updated weights for policy 1, policy_version 76460 (0.0007) +[2023-10-09 15:12:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156270592. Throughput: 0: 1810.1, 1: 1836.9. Samples: 39079610. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:12:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:12:48,470][86122] Updated weights for policy 1, policy_version 76470 (0.0008) +[2023-10-09 15:12:48,836][86122] Updated weights for policy 1, policy_version 76480 (0.0007) +[2023-10-09 15:12:49,707][86121] Updated weights for policy 0, policy_version 76170 (0.0007) +[2023-10-09 15:12:50,084][86121] Updated weights for policy 0, policy_version 76180 (0.0009) +[2023-10-09 15:12:50,441][86121] Updated weights for policy 0, policy_version 76190 (0.0008) +[2023-10-09 15:12:52,422][86122] Updated weights for policy 1, policy_version 76490 (0.0007) +[2023-10-09 15:12:52,789][86122] Updated weights for policy 1, policy_version 76500 (0.0008) +[2023-10-09 15:12:53,150][86122] Updated weights for policy 1, policy_version 76510 (0.0008) +[2023-10-09 15:12:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156368896. Throughput: 0: 1816.2, 1: 1824.4. Samples: 39101502. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:12:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:12:53,974][86121] Updated weights for policy 0, policy_version 76200 (0.0010) +[2023-10-09 15:12:54,334][86121] Updated weights for policy 0, policy_version 76210 (0.0007) +[2023-10-09 15:12:54,696][86121] Updated weights for policy 0, policy_version 76220 (0.0007) +[2023-10-09 15:12:56,919][86122] Updated weights for policy 1, policy_version 76520 (0.0009) +[2023-10-09 15:12:57,286][86122] Updated weights for policy 1, policy_version 76530 (0.0008) +[2023-10-09 15:12:57,647][86122] Updated weights for policy 1, policy_version 76540 (0.0009) +[2023-10-09 15:12:58,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156434432. Throughput: 0: 1812.0, 1: 1833.4. Samples: 39112190. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:12:58,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:12:58,579][86121] Updated weights for policy 0, policy_version 76230 (0.0010) +[2023-10-09 15:12:58,941][86121] Updated weights for policy 0, policy_version 76240 (0.0008) +[2023-10-09 15:12:59,307][86121] Updated weights for policy 0, policy_version 76250 (0.0008) +[2023-10-09 15:13:01,235][86122] Updated weights for policy 1, policy_version 76550 (0.0008) +[2023-10-09 15:13:01,592][86122] Updated weights for policy 1, policy_version 76560 (0.0010) +[2023-10-09 15:13:01,963][86122] Updated weights for policy 1, policy_version 76570 (0.0010) +[2023-10-09 15:13:03,001][86121] Updated weights for policy 0, policy_version 76260 (0.0007) +[2023-10-09 15:13:03,362][86121] Updated weights for policy 0, policy_version 76270 (0.0008) +[2023-10-09 15:13:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156499968. Throughput: 0: 1805.1, 1: 1825.5. Samples: 39133952. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:13:03,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:13:03,731][86121] Updated weights for policy 0, policy_version 76280 (0.0007) +[2023-10-09 15:13:05,664][86122] Updated weights for policy 1, policy_version 76580 (0.0010) +[2023-10-09 15:13:06,039][86122] Updated weights for policy 1, policy_version 76590 (0.0009) +[2023-10-09 15:13:06,402][86122] Updated weights for policy 1, policy_version 76600 (0.0008) +[2023-10-09 15:13:07,349][86121] Updated weights for policy 0, policy_version 76290 (0.0009) +[2023-10-09 15:13:07,720][86121] Updated weights for policy 0, policy_version 76300 (0.0011) +[2023-10-09 15:13:08,091][86121] Updated weights for policy 0, policy_version 76310 (0.0010) +[2023-10-09 15:13:08,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156565504. Throughput: 0: 1815.7, 1: 1837.9. Samples: 39155978. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:13:08,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:13:08,457][86121] Updated weights for policy 0, policy_version 76320 (0.0011) +[2023-10-09 15:13:09,835][86122] Updated weights for policy 1, policy_version 76610 (0.0009) +[2023-10-09 15:13:10,200][86122] Updated weights for policy 1, policy_version 76620 (0.0008) +[2023-10-09 15:13:10,570][86122] Updated weights for policy 1, policy_version 76630 (0.0009) +[2023-10-09 15:13:10,924][86122] Updated weights for policy 1, policy_version 76640 (0.0008) +[2023-10-09 15:13:12,373][86121] Updated weights for policy 0, policy_version 76330 (0.0009) +[2023-10-09 15:13:12,741][86121] Updated weights for policy 0, policy_version 76340 (0.0009) +[2023-10-09 15:13:13,105][86121] Updated weights for policy 0, policy_version 76350 (0.0009) +[2023-10-09 15:13:13,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 156663808. Throughput: 0: 1807.2, 1: 1826.9. Samples: 39166918. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:13:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:13:14,611][86122] Updated weights for policy 1, policy_version 76650 (0.0008) +[2023-10-09 15:13:14,978][86122] Updated weights for policy 1, policy_version 76660 (0.0010) +[2023-10-09 15:13:15,338][86122] Updated weights for policy 1, policy_version 76670 (0.0011) +[2023-10-09 15:13:16,881][86121] Updated weights for policy 0, policy_version 76360 (0.0007) +[2023-10-09 15:13:17,240][86121] Updated weights for policy 0, policy_version 76370 (0.0007) +[2023-10-09 15:13:17,613][86121] Updated weights for policy 0, policy_version 76380 (0.0008) +[2023-10-09 15:13:18,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156729344. Throughput: 0: 1813.8, 1: 1838.0. Samples: 39188922. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:13:18,892][86122] Updated weights for policy 1, policy_version 76680 (0.0008) +[2023-10-09 15:13:19,249][86122] Updated weights for policy 1, policy_version 76690 (0.0008) +[2023-10-09 15:13:19,616][86122] Updated weights for policy 1, policy_version 76700 (0.0009) +[2023-10-09 15:13:21,365][86121] Updated weights for policy 0, policy_version 76390 (0.0009) +[2023-10-09 15:13:21,730][86121] Updated weights for policy 0, policy_version 76400 (0.0007) +[2023-10-09 15:13:22,101][86121] Updated weights for policy 0, policy_version 76410 (0.0010) +[2023-10-09 15:13:23,341][86122] Updated weights for policy 1, policy_version 76710 (0.0008) +[2023-10-09 15:13:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156794880. Throughput: 0: 1800.4, 1: 1839.2. Samples: 39210734. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:13:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000076416_78249984.pth... +[2023-10-09 15:13:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000074720_76513280.pth +[2023-10-09 15:13:23,452][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000076416_78249984.pth +[2023-10-09 15:13:23,705][86122] Updated weights for policy 1, policy_version 76720 (0.0009) +[2023-10-09 15:13:24,064][86122] Updated weights for policy 1, policy_version 76730 (0.0007) +[2023-10-09 15:13:24,277][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000076736_78577664.pth... +[2023-10-09 15:13:24,306][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000075008_76808192.pth +[2023-10-09 15:13:24,310][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000076736_78577664.pth +[2023-10-09 15:13:25,795][86121] Updated weights for policy 0, policy_version 76420 (0.0010) +[2023-10-09 15:13:26,157][86121] Updated weights for policy 0, policy_version 76430 (0.0009) +[2023-10-09 15:13:26,521][86121] Updated weights for policy 0, policy_version 76440 (0.0008) +[2023-10-09 15:13:27,653][86122] Updated weights for policy 1, policy_version 76740 (0.0007) +[2023-10-09 15:13:28,019][86122] Updated weights for policy 1, policy_version 76750 (0.0008) +[2023-10-09 15:13:28,375][86122] Updated weights for policy 1, policy_version 76760 (0.0009) +[2023-10-09 15:13:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 156860416. Throughput: 0: 1810.4, 1: 1844.4. Samples: 39221852. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:13:30,191][86121] Updated weights for policy 0, policy_version 76450 (0.0008) +[2023-10-09 15:13:30,566][86121] Updated weights for policy 0, policy_version 76460 (0.0008) +[2023-10-09 15:13:30,931][86121] Updated weights for policy 0, policy_version 76470 (0.0008) +[2023-10-09 15:13:31,294][86121] Updated weights for policy 0, policy_version 76480 (0.0008) +[2023-10-09 15:13:32,197][86122] Updated weights for policy 1, policy_version 76770 (0.0008) +[2023-10-09 15:13:32,562][86122] Updated weights for policy 1, policy_version 76780 (0.0008) +[2023-10-09 15:13:32,937][86122] Updated weights for policy 1, policy_version 76790 (0.0008) +[2023-10-09 15:13:33,291][86122] Updated weights for policy 1, policy_version 76800 (0.0011) +[2023-10-09 15:13:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 156958720. Throughput: 0: 1802.5, 1: 1845.7. Samples: 39243780. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:13:34,953][86121] Updated weights for policy 0, policy_version 76490 (0.0007) +[2023-10-09 15:13:35,329][86121] Updated weights for policy 0, policy_version 76500 (0.0008) +[2023-10-09 15:13:35,695][86121] Updated weights for policy 0, policy_version 76510 (0.0009) +[2023-10-09 15:13:36,885][86122] Updated weights for policy 1, policy_version 76810 (0.0007) +[2023-10-09 15:13:37,253][86122] Updated weights for policy 1, policy_version 76820 (0.0007) +[2023-10-09 15:13:37,621][86122] Updated weights for policy 1, policy_version 76830 (0.0007) +[2023-10-09 15:13:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157024256. Throughput: 0: 1801.8, 1: 1836.0. Samples: 39265200. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:13:39,353][86121] Updated weights for policy 0, policy_version 76520 (0.0009) +[2023-10-09 15:13:39,724][86121] Updated weights for policy 0, policy_version 76530 (0.0010) +[2023-10-09 15:13:40,100][86121] Updated weights for policy 0, policy_version 76540 (0.0010) +[2023-10-09 15:13:41,125][86122] Updated weights for policy 1, policy_version 76840 (0.0008) +[2023-10-09 15:13:41,476][86122] Updated weights for policy 1, policy_version 76850 (0.0007) +[2023-10-09 15:13:41,841][86122] Updated weights for policy 1, policy_version 76860 (0.0010) +[2023-10-09 15:13:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157089792. Throughput: 0: 1802.4, 1: 1852.0. Samples: 39276638. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:13:43,894][86121] Updated weights for policy 0, policy_version 76550 (0.0009) +[2023-10-09 15:13:44,258][86121] Updated weights for policy 0, policy_version 76560 (0.0008) +[2023-10-09 15:13:44,626][86121] Updated weights for policy 0, policy_version 76570 (0.0008) +[2023-10-09 15:13:45,508][86122] Updated weights for policy 1, policy_version 76870 (0.0008) +[2023-10-09 15:13:45,874][86122] Updated weights for policy 1, policy_version 76880 (0.0009) +[2023-10-09 15:13:46,232][86122] Updated weights for policy 1, policy_version 76890 (0.0008) +[2023-10-09 15:13:48,287][86121] Updated weights for policy 0, policy_version 76580 (0.0008) +[2023-10-09 15:13:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157155328. Throughput: 0: 1807.4, 1: 1838.5. Samples: 39298018. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:13:48,653][86121] Updated weights for policy 0, policy_version 76590 (0.0009) +[2023-10-09 15:13:49,022][86121] Updated weights for policy 0, policy_version 76600 (0.0007) +[2023-10-09 15:13:49,716][86122] Updated weights for policy 1, policy_version 76900 (0.0007) +[2023-10-09 15:13:50,080][86122] Updated weights for policy 1, policy_version 76910 (0.0008) +[2023-10-09 15:13:50,436][86122] Updated weights for policy 1, policy_version 76920 (0.0010) +[2023-10-09 15:13:52,669][86121] Updated weights for policy 0, policy_version 76610 (0.0008) +[2023-10-09 15:13:53,030][86121] Updated weights for policy 0, policy_version 76620 (0.0007) +[2023-10-09 15:13:53,397][86121] Updated weights for policy 0, policy_version 76630 (0.0008) +[2023-10-09 15:13:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 157220864. Throughput: 0: 1812.6, 1: 1854.9. Samples: 39321014. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:13:53,757][86121] Updated weights for policy 0, policy_version 76640 (0.0008) +[2023-10-09 15:13:53,986][86122] Updated weights for policy 1, policy_version 76930 (0.0007) +[2023-10-09 15:13:54,387][86122] Updated weights for policy 1, policy_version 76940 (0.0010) +[2023-10-09 15:13:54,746][86122] Updated weights for policy 1, policy_version 76950 (0.0007) +[2023-10-09 15:13:55,113][86122] Updated weights for policy 1, policy_version 76960 (0.0007) +[2023-10-09 15:13:57,547][86121] Updated weights for policy 0, policy_version 76650 (0.0008) +[2023-10-09 15:13:57,910][86121] Updated weights for policy 0, policy_version 76660 (0.0008) +[2023-10-09 15:13:58,277][86121] Updated weights for policy 0, policy_version 76670 (0.0007) +[2023-10-09 15:13:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 157319168. Throughput: 0: 1805.6, 1: 1847.9. Samples: 39331326. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:13:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:13:58,753][86122] Updated weights for policy 1, policy_version 76970 (0.0008) +[2023-10-09 15:13:59,114][86122] Updated weights for policy 1, policy_version 76980 (0.0008) +[2023-10-09 15:13:59,469][86122] Updated weights for policy 1, policy_version 76990 (0.0009) +[2023-10-09 15:14:02,058][86121] Updated weights for policy 0, policy_version 76680 (0.0007) +[2023-10-09 15:14:02,421][86121] Updated weights for policy 0, policy_version 76690 (0.0007) +[2023-10-09 15:14:02,790][86121] Updated weights for policy 0, policy_version 76700 (0.0007) +[2023-10-09 15:14:03,364][86122] Updated weights for policy 1, policy_version 77000 (0.0008) +[2023-10-09 15:14:03,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157384704. Throughput: 0: 1814.9, 1: 1849.8. Samples: 39353834. Policy #0 lag: (min: 31.0, avg: 33.9, max: 63.0) +[2023-10-09 15:14:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:14:03,741][86122] Updated weights for policy 1, policy_version 77010 (0.0010) +[2023-10-09 15:14:04,108][86122] Updated weights for policy 1, policy_version 77020 (0.0009) +[2023-10-09 15:14:06,387][86121] Updated weights for policy 0, policy_version 76710 (0.0008) +[2023-10-09 15:14:06,748][86121] Updated weights for policy 0, policy_version 76720 (0.0008) +[2023-10-09 15:14:07,128][86121] Updated weights for policy 0, policy_version 76730 (0.0007) +[2023-10-09 15:14:07,752][86122] Updated weights for policy 1, policy_version 77030 (0.0008) +[2023-10-09 15:14:08,115][86122] Updated weights for policy 1, policy_version 77040 (0.0010) +[2023-10-09 15:14:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 157450240. Throughput: 0: 1818.5, 1: 1843.2. Samples: 39375510. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:14:08,487][86122] Updated weights for policy 1, policy_version 77050 (0.0008) +[2023-10-09 15:14:10,805][86121] Updated weights for policy 0, policy_version 76740 (0.0008) +[2023-10-09 15:14:11,182][86121] Updated weights for policy 0, policy_version 76750 (0.0010) +[2023-10-09 15:14:11,551][86121] Updated weights for policy 0, policy_version 76760 (0.0010) +[2023-10-09 15:14:12,145][86122] Updated weights for policy 1, policy_version 77060 (0.0008) +[2023-10-09 15:14:12,498][86122] Updated weights for policy 1, policy_version 77070 (0.0007) +[2023-10-09 15:14:12,866][86122] Updated weights for policy 1, policy_version 77080 (0.0007) +[2023-10-09 15:14:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157548544. Throughput: 0: 1817.4, 1: 1849.3. Samples: 39386854. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:14:15,255][86121] Updated weights for policy 0, policy_version 76770 (0.0008) +[2023-10-09 15:14:15,618][86121] Updated weights for policy 0, policy_version 76780 (0.0008) +[2023-10-09 15:14:15,978][86121] Updated weights for policy 0, policy_version 76790 (0.0008) +[2023-10-09 15:14:16,343][86121] Updated weights for policy 0, policy_version 76800 (0.0007) +[2023-10-09 15:14:16,372][86122] Updated weights for policy 1, policy_version 77090 (0.0008) +[2023-10-09 15:14:16,734][86122] Updated weights for policy 1, policy_version 77100 (0.0010) +[2023-10-09 15:14:17,103][86122] Updated weights for policy 1, policy_version 77110 (0.0009) +[2023-10-09 15:14:17,455][86122] Updated weights for policy 1, policy_version 77120 (0.0007) +[2023-10-09 15:14:18,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157614080. Throughput: 0: 1813.9, 1: 1838.2. Samples: 39408124. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:14:20,271][86121] Updated weights for policy 0, policy_version 76810 (0.0008) +[2023-10-09 15:14:20,648][86121] Updated weights for policy 0, policy_version 76820 (0.0009) +[2023-10-09 15:14:21,021][86121] Updated weights for policy 0, policy_version 76830 (0.0007) +[2023-10-09 15:14:21,156][86122] Updated weights for policy 1, policy_version 77130 (0.0007) +[2023-10-09 15:14:21,517][86122] Updated weights for policy 1, policy_version 77140 (0.0008) +[2023-10-09 15:14:21,879][86122] Updated weights for policy 1, policy_version 77150 (0.0009) +[2023-10-09 15:14:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 157679616. Throughput: 0: 1813.8, 1: 1850.7. Samples: 39430102. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:23,399][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:14:24,481][86121] Updated weights for policy 0, policy_version 76840 (0.0008) +[2023-10-09 15:14:24,846][86121] Updated weights for policy 0, policy_version 76850 (0.0008) +[2023-10-09 15:14:25,215][86121] Updated weights for policy 0, policy_version 76860 (0.0009) +[2023-10-09 15:14:25,475][86122] Updated weights for policy 1, policy_version 77160 (0.0009) +[2023-10-09 15:14:25,841][86122] Updated weights for policy 1, policy_version 77170 (0.0007) +[2023-10-09 15:14:26,202][86122] Updated weights for policy 1, policy_version 77180 (0.0007) +[2023-10-09 15:14:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 157745152. Throughput: 0: 1817.7, 1: 1831.1. Samples: 39440834. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:14:28,953][86121] Updated weights for policy 0, policy_version 76870 (0.0008) +[2023-10-09 15:14:29,314][86121] Updated weights for policy 0, policy_version 76880 (0.0007) +[2023-10-09 15:14:29,668][86121] Updated weights for policy 0, policy_version 76890 (0.0008) +[2023-10-09 15:14:29,956][86122] Updated weights for policy 1, policy_version 77190 (0.0009) +[2023-10-09 15:14:30,324][86122] Updated weights for policy 1, policy_version 77200 (0.0008) +[2023-10-09 15:14:30,689][86122] Updated weights for policy 1, policy_version 77210 (0.0008) +[2023-10-09 15:14:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 157810688. Throughput: 0: 1815.8, 1: 1852.4. Samples: 39463086. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:14:33,506][86121] Updated weights for policy 0, policy_version 76900 (0.0007) +[2023-10-09 15:14:33,874][86121] Updated weights for policy 0, policy_version 76910 (0.0010) +[2023-10-09 15:14:34,239][86121] Updated weights for policy 0, policy_version 76920 (0.0009) +[2023-10-09 15:14:34,265][86122] Updated weights for policy 1, policy_version 77220 (0.0010) +[2023-10-09 15:14:34,631][86122] Updated weights for policy 1, policy_version 77230 (0.0009) +[2023-10-09 15:14:34,998][86122] Updated weights for policy 1, policy_version 77240 (0.0011) +[2023-10-09 15:14:37,948][86121] Updated weights for policy 0, policy_version 76930 (0.0008) +[2023-10-09 15:14:38,317][86121] Updated weights for policy 0, policy_version 76940 (0.0010) +[2023-10-09 15:14:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 157876224. Throughput: 0: 1821.2, 1: 1843.8. Samples: 39485940. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:14:38,680][86121] Updated weights for policy 0, policy_version 76950 (0.0007) +[2023-10-09 15:14:38,844][86122] Updated weights for policy 1, policy_version 77250 (0.0008) +[2023-10-09 15:14:39,049][86121] Updated weights for policy 0, policy_version 76960 (0.0009) +[2023-10-09 15:14:39,236][86122] Updated weights for policy 1, policy_version 77260 (0.0007) +[2023-10-09 15:14:39,597][86122] Updated weights for policy 1, policy_version 77270 (0.0010) +[2023-10-09 15:14:39,955][86122] Updated weights for policy 1, policy_version 77280 (0.0008) +[2023-10-09 15:14:42,702][86121] Updated weights for policy 0, policy_version 76970 (0.0008) +[2023-10-09 15:14:43,075][86121] Updated weights for policy 0, policy_version 76980 (0.0009) +[2023-10-09 15:14:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 157941760. Throughput: 0: 1816.3, 1: 1837.1. Samples: 39495728. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:14:43,442][86121] Updated weights for policy 0, policy_version 76990 (0.0008) +[2023-10-09 15:14:43,758][86122] Updated weights for policy 1, policy_version 77290 (0.0008) +[2023-10-09 15:14:44,134][86122] Updated weights for policy 1, policy_version 77300 (0.0009) +[2023-10-09 15:14:44,494][86122] Updated weights for policy 1, policy_version 77310 (0.0008) +[2023-10-09 15:14:47,172][86121] Updated weights for policy 0, policy_version 77000 (0.0008) +[2023-10-09 15:14:47,534][86121] Updated weights for policy 0, policy_version 77010 (0.0009) +[2023-10-09 15:14:47,905][86121] Updated weights for policy 0, policy_version 77020 (0.0008) +[2023-10-09 15:14:48,208][86122] Updated weights for policy 1, policy_version 77320 (0.0010) +[2023-10-09 15:14:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 158040064. Throughput: 0: 1819.7, 1: 1837.7. Samples: 39518418. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:14:48,567][86122] Updated weights for policy 1, policy_version 77330 (0.0009) +[2023-10-09 15:14:48,934][86122] Updated weights for policy 1, policy_version 77340 (0.0008) +[2023-10-09 15:14:51,491][86121] Updated weights for policy 0, policy_version 77030 (0.0009) +[2023-10-09 15:14:51,853][86121] Updated weights for policy 0, policy_version 77040 (0.0008) +[2023-10-09 15:14:52,228][86121] Updated weights for policy 0, policy_version 77050 (0.0009) +[2023-10-09 15:14:52,628][86122] Updated weights for policy 1, policy_version 77350 (0.0008) +[2023-10-09 15:14:52,989][86122] Updated weights for policy 1, policy_version 77360 (0.0008) +[2023-10-09 15:14:53,360][86122] Updated weights for policy 1, policy_version 77370 (0.0009) +[2023-10-09 15:14:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158105600. Throughput: 0: 1815.0, 1: 1830.3. Samples: 39539548. Policy #0 lag: (min: 31.0, avg: 31.3, max: 42.0) +[2023-10-09 15:14:53,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 15:14:55,623][86121] Updated weights for policy 0, policy_version 77060 (0.0009) +[2023-10-09 15:14:55,990][86121] Updated weights for policy 0, policy_version 77070 (0.0011) +[2023-10-09 15:14:56,365][86121] Updated weights for policy 0, policy_version 77080 (0.0009) +[2023-10-09 15:14:56,904][86122] Updated weights for policy 1, policy_version 77380 (0.0010) +[2023-10-09 15:14:57,261][86122] Updated weights for policy 1, policy_version 77390 (0.0008) +[2023-10-09 15:14:57,623][86122] Updated weights for policy 1, policy_version 77400 (0.0008) +[2023-10-09 15:14:58,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158203904. Throughput: 0: 1818.5, 1: 1832.4. Samples: 39551142. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:14:58,398][85186] Avg episode reward: [(0, '9.900'), (1, '10.000')] +[2023-10-09 15:15:00,052][86121] Updated weights for policy 0, policy_version 77090 (0.0008) +[2023-10-09 15:15:00,428][86121] Updated weights for policy 0, policy_version 77100 (0.0009) +[2023-10-09 15:15:00,803][86121] Updated weights for policy 0, policy_version 77110 (0.0009) +[2023-10-09 15:15:01,155][86121] Updated weights for policy 0, policy_version 77120 (0.0008) +[2023-10-09 15:15:01,336][86122] Updated weights for policy 1, policy_version 77410 (0.0007) +[2023-10-09 15:15:01,695][86122] Updated weights for policy 1, policy_version 77420 (0.0007) +[2023-10-09 15:15:02,058][86122] Updated weights for policy 1, policy_version 77430 (0.0007) +[2023-10-09 15:15:02,422][86122] Updated weights for policy 1, policy_version 77440 (0.0008) +[2023-10-09 15:15:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158269440. Throughput: 0: 1828.7, 1: 1826.2. Samples: 39572594. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:03,398][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 15:15:04,979][86121] Updated weights for policy 0, policy_version 77130 (0.0007) +[2023-10-09 15:15:05,349][86121] Updated weights for policy 0, policy_version 77140 (0.0007) +[2023-10-09 15:15:05,716][86121] Updated weights for policy 0, policy_version 77150 (0.0010) +[2023-10-09 15:15:06,134][86122] Updated weights for policy 1, policy_version 77450 (0.0011) +[2023-10-09 15:15:06,509][86122] Updated weights for policy 1, policy_version 77460 (0.0011) +[2023-10-09 15:15:06,872][86122] Updated weights for policy 1, policy_version 77470 (0.0009) +[2023-10-09 15:15:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158334976. Throughput: 0: 1828.1, 1: 1826.9. Samples: 39594580. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:08,399][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 15:15:09,497][86121] Updated weights for policy 0, policy_version 77160 (0.0008) +[2023-10-09 15:15:09,869][86121] Updated weights for policy 0, policy_version 77170 (0.0010) +[2023-10-09 15:15:10,228][86121] Updated weights for policy 0, policy_version 77180 (0.0008) +[2023-10-09 15:15:10,553][86122] Updated weights for policy 1, policy_version 77480 (0.0008) +[2023-10-09 15:15:10,912][86122] Updated weights for policy 1, policy_version 77490 (0.0008) +[2023-10-09 15:15:11,273][86122] Updated weights for policy 1, policy_version 77500 (0.0008) +[2023-10-09 15:15:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 158400512. Throughput: 0: 1824.0, 1: 1832.9. Samples: 39605396. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:13,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 15:15:13,858][86121] Updated weights for policy 0, policy_version 77190 (0.0008) +[2023-10-09 15:15:14,228][86121] Updated weights for policy 0, policy_version 77200 (0.0009) +[2023-10-09 15:15:14,593][86121] Updated weights for policy 0, policy_version 77210 (0.0009) +[2023-10-09 15:15:14,795][86122] Updated weights for policy 1, policy_version 77510 (0.0008) +[2023-10-09 15:15:15,150][86122] Updated weights for policy 1, policy_version 77520 (0.0008) +[2023-10-09 15:15:15,507][86122] Updated weights for policy 1, policy_version 77530 (0.0008) +[2023-10-09 15:15:18,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158466048. Throughput: 0: 1829.9, 1: 1831.6. Samples: 39627852. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:18,398][85186] Avg episode reward: [(0, '9.880'), (1, '10.000')] +[2023-10-09 15:15:18,470][86121] Updated weights for policy 0, policy_version 77220 (0.0007) +[2023-10-09 15:15:18,827][86121] Updated weights for policy 0, policy_version 77230 (0.0010) +[2023-10-09 15:15:19,125][86122] Updated weights for policy 1, policy_version 77540 (0.0008) +[2023-10-09 15:15:19,197][86121] Updated weights for policy 0, policy_version 77240 (0.0009) +[2023-10-09 15:15:19,487][86122] Updated weights for policy 1, policy_version 77550 (0.0009) +[2023-10-09 15:15:19,850][86122] Updated weights for policy 1, policy_version 77560 (0.0008) +[2023-10-09 15:15:22,687][86121] Updated weights for policy 0, policy_version 77250 (0.0008) +[2023-10-09 15:15:23,049][86121] Updated weights for policy 0, policy_version 77260 (0.0010) +[2023-10-09 15:15:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 158531584. Throughput: 0: 1824.5, 1: 1835.3. Samples: 39650632. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:23,398][85186] Avg episode reward: [(0, '9.870'), (1, '10.000')] +[2023-10-09 15:15:23,421][86121] Updated weights for policy 0, policy_version 77270 (0.0009) +[2023-10-09 15:15:23,454][86122] Updated weights for policy 1, policy_version 77570 (0.0007) +[2023-10-09 15:15:23,786][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000077280_79134720.pth... +[2023-10-09 15:15:23,789][86121] Updated weights for policy 0, policy_version 77280 (0.0008) +[2023-10-09 15:15:23,826][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000075552_77365248.pth +[2023-10-09 15:15:23,857][86122] Updated weights for policy 1, policy_version 77580 (0.0009) +[2023-10-09 15:15:24,223][86122] Updated weights for policy 1, policy_version 77590 (0.0011) +[2023-10-09 15:15:24,577][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000077600_79462400.pth... +[2023-10-09 15:15:24,580][86122] Updated weights for policy 1, policy_version 77600 (0.0010) +[2023-10-09 15:15:24,607][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000075872_77692928.pth +[2023-10-09 15:15:27,476][86121] Updated weights for policy 0, policy_version 77290 (0.0007) +[2023-10-09 15:15:27,834][86121] Updated weights for policy 0, policy_version 77300 (0.0007) +[2023-10-09 15:15:28,197][86121] Updated weights for policy 0, policy_version 77310 (0.0009) +[2023-10-09 15:15:28,277][86122] Updated weights for policy 1, policy_version 77610 (0.0008) +[2023-10-09 15:15:28,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158629888. Throughput: 0: 1830.5, 1: 1836.7. Samples: 39660756. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:28,399][85186] Avg episode reward: [(0, '9.870'), (1, '10.000')] +[2023-10-09 15:15:28,646][86122] Updated weights for policy 1, policy_version 77620 (0.0008) +[2023-10-09 15:15:29,002][86122] Updated weights for policy 1, policy_version 77630 (0.0009) +[2023-10-09 15:15:31,820][86121] Updated weights for policy 0, policy_version 77320 (0.0007) +[2023-10-09 15:15:32,190][86121] Updated weights for policy 0, policy_version 77330 (0.0007) +[2023-10-09 15:15:32,550][86121] Updated weights for policy 0, policy_version 77340 (0.0007) +[2023-10-09 15:15:32,691][86122] Updated weights for policy 1, policy_version 77640 (0.0008) +[2023-10-09 15:15:33,051][86122] Updated weights for policy 1, policy_version 77650 (0.0007) +[2023-10-09 15:15:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 158695424. Throughput: 0: 1823.1, 1: 1838.7. Samples: 39683198. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:33,398][85186] Avg episode reward: [(0, '9.860'), (1, '10.000')] +[2023-10-09 15:15:33,416][86122] Updated weights for policy 1, policy_version 77660 (0.0009) +[2023-10-09 15:15:36,179][86121] Updated weights for policy 0, policy_version 77350 (0.0009) +[2023-10-09 15:15:36,555][86121] Updated weights for policy 0, policy_version 77360 (0.0010) +[2023-10-09 15:15:36,922][86121] Updated weights for policy 0, policy_version 77370 (0.0007) +[2023-10-09 15:15:37,056][86122] Updated weights for policy 1, policy_version 77670 (0.0007) +[2023-10-09 15:15:37,421][86122] Updated weights for policy 1, policy_version 77680 (0.0007) +[2023-10-09 15:15:37,770][86122] Updated weights for policy 1, policy_version 77690 (0.0007) +[2023-10-09 15:15:38,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 158793728. Throughput: 0: 1832.3, 1: 1822.8. Samples: 39704028. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:38,398][85186] Avg episode reward: [(0, '9.850'), (1, '10.000')] +[2023-10-09 15:15:40,657][86121] Updated weights for policy 0, policy_version 77380 (0.0007) +[2023-10-09 15:15:41,027][86121] Updated weights for policy 0, policy_version 77390 (0.0008) +[2023-10-09 15:15:41,389][86121] Updated weights for policy 0, policy_version 77400 (0.0010) +[2023-10-09 15:15:41,515][86122] Updated weights for policy 1, policy_version 77700 (0.0009) +[2023-10-09 15:15:41,871][86122] Updated weights for policy 1, policy_version 77710 (0.0009) +[2023-10-09 15:15:42,238][86122] Updated weights for policy 1, policy_version 77720 (0.0008) +[2023-10-09 15:15:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 158859264. Throughput: 0: 1822.7, 1: 1840.7. Samples: 39715994. Policy #0 lag: (min: 17.0, avg: 29.6, max: 49.0) +[2023-10-09 15:15:43,398][85186] Avg episode reward: [(0, '9.840'), (1, '10.000')] +[2023-10-09 15:15:44,993][86121] Updated weights for policy 0, policy_version 77410 (0.0009) +[2023-10-09 15:15:45,363][86121] Updated weights for policy 0, policy_version 77420 (0.0010) +[2023-10-09 15:15:45,728][86121] Updated weights for policy 0, policy_version 77430 (0.0010) +[2023-10-09 15:15:45,831][86122] Updated weights for policy 1, policy_version 77730 (0.0010) +[2023-10-09 15:15:46,096][86121] Updated weights for policy 0, policy_version 77440 (0.0008) +[2023-10-09 15:15:46,187][86122] Updated weights for policy 1, policy_version 77740 (0.0008) +[2023-10-09 15:15:46,557][86122] Updated weights for policy 1, policy_version 77750 (0.0008) +[2023-10-09 15:15:46,922][86122] Updated weights for policy 1, policy_version 77760 (0.0007) +[2023-10-09 15:15:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158924800. Throughput: 0: 1824.5, 1: 1829.6. Samples: 39737032. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:15:48,398][85186] Avg episode reward: [(0, '9.860'), (1, '10.000')] +[2023-10-09 15:15:49,801][86121] Updated weights for policy 0, policy_version 77450 (0.0007) +[2023-10-09 15:15:50,172][86121] Updated weights for policy 0, policy_version 77460 (0.0007) +[2023-10-09 15:15:50,532][86121] Updated weights for policy 0, policy_version 77470 (0.0009) +[2023-10-09 15:15:50,712][86122] Updated weights for policy 1, policy_version 77770 (0.0009) +[2023-10-09 15:15:51,080][86122] Updated weights for policy 1, policy_version 77780 (0.0009) +[2023-10-09 15:15:51,439][86122] Updated weights for policy 1, policy_version 77790 (0.0009) +[2023-10-09 15:15:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 158990336. Throughput: 0: 1826.4, 1: 1839.3. Samples: 39759532. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:15:53,398][85186] Avg episode reward: [(0, '9.850'), (1, '10.000')] +[2023-10-09 15:15:54,349][86121] Updated weights for policy 0, policy_version 77480 (0.0008) +[2023-10-09 15:15:54,721][86121] Updated weights for policy 0, policy_version 77490 (0.0007) +[2023-10-09 15:15:55,089][86121] Updated weights for policy 0, policy_version 77500 (0.0007) +[2023-10-09 15:15:55,110][86122] Updated weights for policy 1, policy_version 77800 (0.0008) +[2023-10-09 15:15:55,469][86122] Updated weights for policy 1, policy_version 77810 (0.0008) +[2023-10-09 15:15:55,829][86122] Updated weights for policy 1, policy_version 77820 (0.0007) +[2023-10-09 15:15:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159055872. Throughput: 0: 1825.4, 1: 1825.9. Samples: 39769706. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:15:58,398][85186] Avg episode reward: [(0, '9.840'), (1, '10.000')] +[2023-10-09 15:15:58,840][86121] Updated weights for policy 0, policy_version 77510 (0.0008) +[2023-10-09 15:15:59,210][86121] Updated weights for policy 0, policy_version 77520 (0.0008) +[2023-10-09 15:15:59,388][86122] Updated weights for policy 1, policy_version 77830 (0.0009) +[2023-10-09 15:15:59,570][86121] Updated weights for policy 0, policy_version 77530 (0.0007) +[2023-10-09 15:15:59,743][86122] Updated weights for policy 1, policy_version 77840 (0.0007) +[2023-10-09 15:16:00,104][86122] Updated weights for policy 1, policy_version 77850 (0.0009) +[2023-10-09 15:16:03,168][86121] Updated weights for policy 0, policy_version 77540 (0.0008) +[2023-10-09 15:16:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159121408. Throughput: 0: 1826.9, 1: 1832.2. Samples: 39792510. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:16:03,398][85186] Avg episode reward: [(0, '9.840'), (1, '10.000')] +[2023-10-09 15:16:03,527][86121] Updated weights for policy 0, policy_version 77550 (0.0007) +[2023-10-09 15:16:03,724][86122] Updated weights for policy 1, policy_version 77860 (0.0009) +[2023-10-09 15:16:03,891][86121] Updated weights for policy 0, policy_version 77560 (0.0009) +[2023-10-09 15:16:04,082][86122] Updated weights for policy 1, policy_version 77870 (0.0007) +[2023-10-09 15:16:04,446][86122] Updated weights for policy 1, policy_version 77880 (0.0007) +[2023-10-09 15:16:07,582][86121] Updated weights for policy 0, policy_version 77570 (0.0008) +[2023-10-09 15:16:07,953][86121] Updated weights for policy 0, policy_version 77580 (0.0007) +[2023-10-09 15:16:08,048][86122] Updated weights for policy 1, policy_version 77890 (0.0007) +[2023-10-09 15:16:08,329][86121] Updated weights for policy 0, policy_version 77590 (0.0007) +[2023-10-09 15:16:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159186944. Throughput: 0: 1824.4, 1: 1834.7. Samples: 39815294. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:16:08,398][85186] Avg episode reward: [(0, '9.840'), (1, '10.000')] +[2023-10-09 15:16:08,409][86122] Updated weights for policy 1, policy_version 77900 (0.0007) +[2023-10-09 15:16:08,689][86121] Updated weights for policy 0, policy_version 77600 (0.0007) +[2023-10-09 15:16:08,780][86122] Updated weights for policy 1, policy_version 77910 (0.0008) +[2023-10-09 15:16:09,135][86122] Updated weights for policy 1, policy_version 77920 (0.0009) +[2023-10-09 15:16:12,179][86121] Updated weights for policy 0, policy_version 77610 (0.0008) +[2023-10-09 15:16:12,547][86121] Updated weights for policy 0, policy_version 77620 (0.0008) +[2023-10-09 15:16:12,918][86121] Updated weights for policy 0, policy_version 77630 (0.0007) +[2023-10-09 15:16:13,095][86122] Updated weights for policy 1, policy_version 77930 (0.0010) +[2023-10-09 15:16:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 159285248. Throughput: 0: 1828.9, 1: 1835.8. Samples: 39825666. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:16:13,398][85186] Avg episode reward: [(0, '9.850'), (1, '10.000')] +[2023-10-09 15:16:13,466][86122] Updated weights for policy 1, policy_version 77940 (0.0010) +[2023-10-09 15:16:13,823][86122] Updated weights for policy 1, policy_version 77950 (0.0010) +[2023-10-09 15:16:16,648][86121] Updated weights for policy 0, policy_version 77640 (0.0007) +[2023-10-09 15:16:17,019][86121] Updated weights for policy 0, policy_version 77650 (0.0007) +[2023-10-09 15:16:17,375][86121] Updated weights for policy 0, policy_version 77660 (0.0007) +[2023-10-09 15:16:17,628][86122] Updated weights for policy 1, policy_version 77960 (0.0011) +[2023-10-09 15:16:17,993][86122] Updated weights for policy 1, policy_version 77970 (0.0010) +[2023-10-09 15:16:18,349][86122] Updated weights for policy 1, policy_version 77980 (0.0009) +[2023-10-09 15:16:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 159350784. Throughput: 0: 1822.8, 1: 1832.0. Samples: 39847668. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:16:18,398][85186] Avg episode reward: [(0, '9.840'), (1, '10.000')] +[2023-10-09 15:16:21,212][86121] Updated weights for policy 0, policy_version 77670 (0.0008) +[2023-10-09 15:16:21,576][86121] Updated weights for policy 0, policy_version 77680 (0.0008) +[2023-10-09 15:16:21,943][86121] Updated weights for policy 0, policy_version 77690 (0.0009) +[2023-10-09 15:16:21,971][86122] Updated weights for policy 1, policy_version 77990 (0.0010) +[2023-10-09 15:16:22,336][86122] Updated weights for policy 1, policy_version 78000 (0.0008) +[2023-10-09 15:16:22,696][86122] Updated weights for policy 1, policy_version 78010 (0.0007) +[2023-10-09 15:16:23,397][85186] Fps is (10 sec: 16383.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 159449088. Throughput: 0: 1823.0, 1: 1828.9. Samples: 39868364. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:16:23,398][85186] Avg episode reward: [(0, '9.820'), (1, '10.000')] +[2023-10-09 15:16:25,694][86121] Updated weights for policy 0, policy_version 77700 (0.0008) +[2023-10-09 15:16:26,067][86121] Updated weights for policy 0, policy_version 77710 (0.0007) +[2023-10-09 15:16:26,421][86122] Updated weights for policy 1, policy_version 78020 (0.0008) +[2023-10-09 15:16:26,441][86121] Updated weights for policy 0, policy_version 77720 (0.0008) +[2023-10-09 15:16:26,788][86122] Updated weights for policy 1, policy_version 78030 (0.0008) +[2023-10-09 15:16:27,147][86122] Updated weights for policy 1, policy_version 78040 (0.0011) +[2023-10-09 15:16:28,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 159514624. Throughput: 0: 1826.0, 1: 1832.5. Samples: 39880628. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:16:28,398][85186] Avg episode reward: [(0, '9.820'), (1, '10.000')] +[2023-10-09 15:16:30,039][86121] Updated weights for policy 0, policy_version 77730 (0.0007) +[2023-10-09 15:16:30,412][86121] Updated weights for policy 0, policy_version 77740 (0.0008) +[2023-10-09 15:16:30,769][86121] Updated weights for policy 0, policy_version 77750 (0.0008) +[2023-10-09 15:16:30,800][86122] Updated weights for policy 1, policy_version 78050 (0.0009) +[2023-10-09 15:16:31,138][86121] Updated weights for policy 0, policy_version 77760 (0.0008) +[2023-10-09 15:16:31,152][86122] Updated weights for policy 1, policy_version 78060 (0.0009) +[2023-10-09 15:16:31,517][86122] Updated weights for policy 1, policy_version 78070 (0.0010) +[2023-10-09 15:16:31,875][86122] Updated weights for policy 1, policy_version 78080 (0.0009) +[2023-10-09 15:16:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 159580160. Throughput: 0: 1818.6, 1: 1825.6. Samples: 39901018. Policy #0 lag: (min: 30.0, avg: 30.0, max: 30.0) +[2023-10-09 15:16:33,398][85186] Avg episode reward: [(0, '9.820'), (1, '10.000')] +[2023-10-09 15:16:34,667][86121] Updated weights for policy 0, policy_version 77770 (0.0009) +[2023-10-09 15:16:35,039][86121] Updated weights for policy 0, policy_version 77780 (0.0007) +[2023-10-09 15:16:35,407][86121] Updated weights for policy 0, policy_version 77790 (0.0009) +[2023-10-09 15:16:35,442][86122] Updated weights for policy 1, policy_version 78090 (0.0009) +[2023-10-09 15:16:35,802][86122] Updated weights for policy 1, policy_version 78100 (0.0008) +[2023-10-09 15:16:36,169][86122] Updated weights for policy 1, policy_version 78110 (0.0009) +[2023-10-09 15:16:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159645696. Throughput: 0: 1819.9, 1: 1831.1. Samples: 39923826. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:16:38,398][85186] Avg episode reward: [(0, '9.810'), (1, '10.000')] +[2023-10-09 15:16:39,178][86121] Updated weights for policy 0, policy_version 77800 (0.0009) +[2023-10-09 15:16:39,552][86121] Updated weights for policy 0, policy_version 77810 (0.0008) +[2023-10-09 15:16:39,817][86122] Updated weights for policy 1, policy_version 78120 (0.0008) +[2023-10-09 15:16:39,918][86121] Updated weights for policy 0, policy_version 77820 (0.0008) +[2023-10-09 15:16:40,168][86122] Updated weights for policy 1, policy_version 78130 (0.0009) +[2023-10-09 15:16:40,528][86122] Updated weights for policy 1, policy_version 78140 (0.0008) +[2023-10-09 15:16:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 159711232. Throughput: 0: 1819.6, 1: 1821.3. Samples: 39933550. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:16:43,398][85186] Avg episode reward: [(0, '9.810'), (1, '10.000')] +[2023-10-09 15:16:43,678][86121] Updated weights for policy 0, policy_version 77830 (0.0007) +[2023-10-09 15:16:44,044][86121] Updated weights for policy 0, policy_version 77840 (0.0009) +[2023-10-09 15:16:44,182][86122] Updated weights for policy 1, policy_version 78150 (0.0007) +[2023-10-09 15:16:44,414][86121] Updated weights for policy 0, policy_version 77850 (0.0009) +[2023-10-09 15:16:44,548][86122] Updated weights for policy 1, policy_version 78160 (0.0009) +[2023-10-09 15:16:44,910][86122] Updated weights for policy 1, policy_version 78170 (0.0010) +[2023-10-09 15:16:48,044][86121] Updated weights for policy 0, policy_version 77860 (0.0009) +[2023-10-09 15:16:48,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159776768. Throughput: 0: 1811.3, 1: 1830.9. Samples: 39956412. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:16:48,398][85186] Avg episode reward: [(0, '9.830'), (1, '10.000')] +[2023-10-09 15:16:48,409][86121] Updated weights for policy 0, policy_version 77870 (0.0008) +[2023-10-09 15:16:48,712][86122] Updated weights for policy 1, policy_version 78180 (0.0009) +[2023-10-09 15:16:48,783][86121] Updated weights for policy 0, policy_version 77880 (0.0008) +[2023-10-09 15:16:49,073][86122] Updated weights for policy 1, policy_version 78190 (0.0009) +[2023-10-09 15:16:49,433][86122] Updated weights for policy 1, policy_version 78200 (0.0009) +[2023-10-09 15:16:52,671][86121] Updated weights for policy 0, policy_version 77890 (0.0007) +[2023-10-09 15:16:53,029][86121] Updated weights for policy 0, policy_version 77900 (0.0007) +[2023-10-09 15:16:53,072][86122] Updated weights for policy 1, policy_version 78210 (0.0009) +[2023-10-09 15:16:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159842304. Throughput: 0: 1809.2, 1: 1820.5. Samples: 39978630. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:16:53,398][85186] Avg episode reward: [(0, '9.820'), (1, '10.000')] +[2023-10-09 15:16:53,401][86121] Updated weights for policy 0, policy_version 77910 (0.0007) +[2023-10-09 15:16:53,442][86122] Updated weights for policy 1, policy_version 78220 (0.0008) +[2023-10-09 15:16:53,771][86121] Updated weights for policy 0, policy_version 77920 (0.0007) +[2023-10-09 15:16:53,811][86122] Updated weights for policy 1, policy_version 78230 (0.0007) +[2023-10-09 15:16:54,182][86122] Updated weights for policy 1, policy_version 78240 (0.0010) +[2023-10-09 15:16:57,619][86121] Updated weights for policy 0, policy_version 77930 (0.0009) +[2023-10-09 15:16:57,979][86121] Updated weights for policy 0, policy_version 77940 (0.0008) +[2023-10-09 15:16:58,044][86122] Updated weights for policy 1, policy_version 78250 (0.0007) +[2023-10-09 15:16:58,345][86121] Updated weights for policy 0, policy_version 77950 (0.0008) +[2023-10-09 15:16:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 159907840. Throughput: 0: 1802.5, 1: 1820.3. Samples: 39988692. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:16:58,398][85186] Avg episode reward: [(0, '9.840'), (1, '10.000')] +[2023-10-09 15:16:58,408][86122] Updated weights for policy 1, policy_version 78260 (0.0009) +[2023-10-09 15:16:58,773][86122] Updated weights for policy 1, policy_version 78270 (0.0008) +[2023-10-09 15:17:02,148][86121] Updated weights for policy 0, policy_version 77960 (0.0009) +[2023-10-09 15:17:02,502][86122] Updated weights for policy 1, policy_version 78280 (0.0007) +[2023-10-09 15:17:02,519][86121] Updated weights for policy 0, policy_version 77970 (0.0007) +[2023-10-09 15:17:02,862][86122] Updated weights for policy 1, policy_version 78290 (0.0007) +[2023-10-09 15:17:02,886][86121] Updated weights for policy 0, policy_version 77980 (0.0008) +[2023-10-09 15:17:03,230][86122] Updated weights for policy 1, policy_version 78300 (0.0008) +[2023-10-09 15:17:03,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14773.4). Total num frames: 160038912. Throughput: 0: 1812.0, 1: 1821.6. Samples: 40011176. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:17:03,398][85186] Avg episode reward: [(0, '9.840'), (1, '10.000')] +[2023-10-09 15:17:06,545][86121] Updated weights for policy 0, policy_version 77990 (0.0009) +[2023-10-09 15:17:06,911][86121] Updated weights for policy 0, policy_version 78000 (0.0008) +[2023-10-09 15:17:07,056][86122] Updated weights for policy 1, policy_version 78310 (0.0009) +[2023-10-09 15:17:07,277][86121] Updated weights for policy 0, policy_version 78010 (0.0007) +[2023-10-09 15:17:07,417][86122] Updated weights for policy 1, policy_version 78320 (0.0008) +[2023-10-09 15:17:07,770][86122] Updated weights for policy 1, policy_version 78330 (0.0007) +[2023-10-09 15:17:08,397][85186] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160104448. Throughput: 0: 1797.9, 1: 1824.7. Samples: 40031380. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:17:08,398][85186] Avg episode reward: [(0, '9.850'), (1, '10.000')] +[2023-10-09 15:17:10,826][86121] Updated weights for policy 0, policy_version 78020 (0.0007) +[2023-10-09 15:17:11,196][86121] Updated weights for policy 0, policy_version 78030 (0.0007) +[2023-10-09 15:17:11,564][86121] Updated weights for policy 0, policy_version 78040 (0.0008) +[2023-10-09 15:17:11,574][86122] Updated weights for policy 1, policy_version 78340 (0.0009) +[2023-10-09 15:17:11,937][86122] Updated weights for policy 1, policy_version 78350 (0.0009) +[2023-10-09 15:17:12,295][86122] Updated weights for policy 1, policy_version 78360 (0.0007) +[2023-10-09 15:17:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 160169984. Throughput: 0: 1808.3, 1: 1815.5. Samples: 40043700. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:17:13,398][85186] Avg episode reward: [(0, '9.850'), (1, '10.000')] +[2023-10-09 15:17:15,254][86121] Updated weights for policy 0, policy_version 78050 (0.0007) +[2023-10-09 15:17:15,630][86121] Updated weights for policy 0, policy_version 78060 (0.0010) +[2023-10-09 15:17:15,826][86122] Updated weights for policy 1, policy_version 78370 (0.0008) +[2023-10-09 15:17:15,995][86121] Updated weights for policy 0, policy_version 78070 (0.0008) +[2023-10-09 15:17:16,188][86122] Updated weights for policy 1, policy_version 78380 (0.0010) +[2023-10-09 15:17:16,350][86121] Updated weights for policy 0, policy_version 78080 (0.0008) +[2023-10-09 15:17:16,551][86122] Updated weights for policy 1, policy_version 78390 (0.0009) +[2023-10-09 15:17:16,914][86122] Updated weights for policy 1, policy_version 78400 (0.0010) +[2023-10-09 15:17:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 160235520. Throughput: 0: 1806.9, 1: 1823.2. Samples: 40064372. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:17:18,398][85186] Avg episode reward: [(0, '9.860'), (1, '10.000')] +[2023-10-09 15:17:19,905][86121] Updated weights for policy 0, policy_version 78090 (0.0010) +[2023-10-09 15:17:20,267][86121] Updated weights for policy 0, policy_version 78100 (0.0007) +[2023-10-09 15:17:20,591][86122] Updated weights for policy 1, policy_version 78410 (0.0009) +[2023-10-09 15:17:20,638][86121] Updated weights for policy 0, policy_version 78110 (0.0008) +[2023-10-09 15:17:20,961][86122] Updated weights for policy 1, policy_version 78420 (0.0011) +[2023-10-09 15:17:21,309][86122] Updated weights for policy 1, policy_version 78430 (0.0010) +[2023-10-09 15:17:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160301056. Throughput: 0: 1807.0, 1: 1813.8. Samples: 40086762. Policy #0 lag: (min: 20.0, avg: 23.0, max: 52.0) +[2023-10-09 15:17:23,399][85186] Avg episode reward: [(0, '9.860'), (1, '10.000')] +[2023-10-09 15:17:23,413][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000078112_79986688.pth... +[2023-10-09 15:17:23,414][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000078432_80314368.pth... +[2023-10-09 15:17:23,448][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000076416_78249984.pth +[2023-10-09 15:17:23,452][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000076736_78577664.pth +[2023-10-09 15:17:24,498][86121] Updated weights for policy 0, policy_version 78120 (0.0008) +[2023-10-09 15:17:24,855][86121] Updated weights for policy 0, policy_version 78130 (0.0008) +[2023-10-09 15:17:25,110][86122] Updated weights for policy 1, policy_version 78440 (0.0008) +[2023-10-09 15:17:25,219][86121] Updated weights for policy 0, policy_version 78140 (0.0007) +[2023-10-09 15:17:25,461][86122] Updated weights for policy 1, policy_version 78450 (0.0011) +[2023-10-09 15:17:25,834][86122] Updated weights for policy 1, policy_version 78460 (0.0008) +[2023-10-09 15:17:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 160366592. Throughput: 0: 1810.6, 1: 1823.0. Samples: 40097064. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:17:28,399][85186] Avg episode reward: [(0, '9.890'), (1, '10.000')] +[2023-10-09 15:17:29,009][86121] Updated weights for policy 0, policy_version 78150 (0.0010) +[2023-10-09 15:17:29,378][86121] Updated weights for policy 0, policy_version 78160 (0.0010) +[2023-10-09 15:17:29,441][86122] Updated weights for policy 1, policy_version 78470 (0.0007) +[2023-10-09 15:17:29,743][86121] Updated weights for policy 0, policy_version 78170 (0.0007) +[2023-10-09 15:17:29,792][86122] Updated weights for policy 1, policy_version 78480 (0.0007) +[2023-10-09 15:17:30,149][86122] Updated weights for policy 1, policy_version 78490 (0.0007) +[2023-10-09 15:17:33,376][86121] Updated weights for policy 0, policy_version 78180 (0.0008) +[2023-10-09 15:17:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160432128. Throughput: 0: 1813.7, 1: 1814.9. Samples: 40119698. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:17:33,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.980')] +[2023-10-09 15:17:33,741][86121] Updated weights for policy 0, policy_version 78190 (0.0008) +[2023-10-09 15:17:33,779][86122] Updated weights for policy 1, policy_version 78500 (0.0008) +[2023-10-09 15:17:34,111][86121] Updated weights for policy 0, policy_version 78200 (0.0007) +[2023-10-09 15:17:34,143][86122] Updated weights for policy 1, policy_version 78510 (0.0008) +[2023-10-09 15:17:34,492][86122] Updated weights for policy 1, policy_version 78520 (0.0007) +[2023-10-09 15:17:37,807][86121] Updated weights for policy 0, policy_version 78210 (0.0008) +[2023-10-09 15:17:38,024][86122] Updated weights for policy 1, policy_version 78530 (0.0009) +[2023-10-09 15:17:38,169][86121] Updated weights for policy 0, policy_version 78220 (0.0007) +[2023-10-09 15:17:38,386][86122] Updated weights for policy 1, policy_version 78540 (0.0007) +[2023-10-09 15:17:38,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160497664. Throughput: 0: 1822.2, 1: 1823.6. Samples: 40142690. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:17:38,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 15:17:38,545][86121] Updated weights for policy 0, policy_version 78230 (0.0009) +[2023-10-09 15:17:38,746][86122] Updated weights for policy 1, policy_version 78550 (0.0008) +[2023-10-09 15:17:38,913][86121] Updated weights for policy 0, policy_version 78240 (0.0010) +[2023-10-09 15:17:39,112][86122] Updated weights for policy 1, policy_version 78560 (0.0009) +[2023-10-09 15:17:42,648][86121] Updated weights for policy 0, policy_version 78250 (0.0007) +[2023-10-09 15:17:43,001][86122] Updated weights for policy 1, policy_version 78570 (0.0007) +[2023-10-09 15:17:43,009][86121] Updated weights for policy 0, policy_version 78260 (0.0007) +[2023-10-09 15:17:43,352][86122] Updated weights for policy 1, policy_version 78580 (0.0007) +[2023-10-09 15:17:43,372][86121] Updated weights for policy 0, policy_version 78270 (0.0007) +[2023-10-09 15:17:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160563200. Throughput: 0: 1822.7, 1: 1826.4. Samples: 40152900. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:17:43,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 15:17:43,717][86122] Updated weights for policy 1, policy_version 78590 (0.0008) +[2023-10-09 15:17:47,027][86121] Updated weights for policy 0, policy_version 78280 (0.0008) +[2023-10-09 15:17:47,389][86121] Updated weights for policy 0, policy_version 78290 (0.0007) +[2023-10-09 15:17:47,446][86122] Updated weights for policy 1, policy_version 78600 (0.0008) +[2023-10-09 15:17:47,759][86121] Updated weights for policy 0, policy_version 78300 (0.0008) +[2023-10-09 15:17:47,817][86122] Updated weights for policy 1, policy_version 78610 (0.0010) +[2023-10-09 15:17:48,172][86122] Updated weights for policy 1, policy_version 78620 (0.0009) +[2023-10-09 15:17:48,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 160694272. Throughput: 0: 1821.4, 1: 1824.6. Samples: 40175246. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:17:48,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 15:17:51,404][86121] Updated weights for policy 0, policy_version 78310 (0.0007) +[2023-10-09 15:17:51,771][86121] Updated weights for policy 0, policy_version 78320 (0.0010) +[2023-10-09 15:17:51,832][86122] Updated weights for policy 1, policy_version 78630 (0.0009) +[2023-10-09 15:17:52,137][86121] Updated weights for policy 0, policy_version 78330 (0.0008) +[2023-10-09 15:17:52,189][86122] Updated weights for policy 1, policy_version 78640 (0.0008) +[2023-10-09 15:17:52,555][86122] Updated weights for policy 1, policy_version 78650 (0.0008) +[2023-10-09 15:17:53,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160759808. Throughput: 0: 1822.6, 1: 1819.2. Samples: 40195258. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:17:53,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 15:17:55,960][86121] Updated weights for policy 0, policy_version 78340 (0.0007) +[2023-10-09 15:17:56,224][86122] Updated weights for policy 1, policy_version 78660 (0.0008) +[2023-10-09 15:17:56,315][86121] Updated weights for policy 0, policy_version 78350 (0.0009) +[2023-10-09 15:17:56,576][86122] Updated weights for policy 1, policy_version 78670 (0.0007) +[2023-10-09 15:17:56,678][86121] Updated weights for policy 0, policy_version 78360 (0.0009) +[2023-10-09 15:17:56,941][86122] Updated weights for policy 1, policy_version 78680 (0.0007) +[2023-10-09 15:17:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 160825344. Throughput: 0: 1815.0, 1: 1828.7. Samples: 40207668. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:17:58,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 15:18:00,418][86121] Updated weights for policy 0, policy_version 78370 (0.0008) +[2023-10-09 15:18:00,772][86122] Updated weights for policy 1, policy_version 78690 (0.0009) +[2023-10-09 15:18:00,788][86121] Updated weights for policy 0, policy_version 78380 (0.0009) +[2023-10-09 15:18:01,133][86122] Updated weights for policy 1, policy_version 78700 (0.0007) +[2023-10-09 15:18:01,147][86121] Updated weights for policy 0, policy_version 78390 (0.0008) +[2023-10-09 15:18:01,496][86122] Updated weights for policy 1, policy_version 78710 (0.0010) +[2023-10-09 15:18:01,504][86121] Updated weights for policy 0, policy_version 78400 (0.0007) +[2023-10-09 15:18:01,851][86122] Updated weights for policy 1, policy_version 78720 (0.0009) +[2023-10-09 15:18:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 160890880. Throughput: 0: 1810.2, 1: 1818.0. Samples: 40227640. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:18:03,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:18:05,427][86121] Updated weights for policy 0, policy_version 78410 (0.0009) +[2023-10-09 15:18:05,467][86122] Updated weights for policy 1, policy_version 78730 (0.0010) +[2023-10-09 15:18:05,791][86121] Updated weights for policy 0, policy_version 78420 (0.0008) +[2023-10-09 15:18:05,824][86122] Updated weights for policy 1, policy_version 78740 (0.0007) +[2023-10-09 15:18:06,143][86121] Updated weights for policy 0, policy_version 78430 (0.0009) +[2023-10-09 15:18:06,181][86122] Updated weights for policy 1, policy_version 78750 (0.0008) +[2023-10-09 15:18:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 160956416. Throughput: 0: 1803.9, 1: 1826.4. Samples: 40250122. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:18:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:18:09,898][86122] Updated weights for policy 1, policy_version 78760 (0.0008) +[2023-10-09 15:18:10,046][86121] Updated weights for policy 0, policy_version 78440 (0.0008) +[2023-10-09 15:18:10,269][86122] Updated weights for policy 1, policy_version 78770 (0.0008) +[2023-10-09 15:18:10,423][86121] Updated weights for policy 0, policy_version 78450 (0.0008) +[2023-10-09 15:18:10,630][86122] Updated weights for policy 1, policy_version 78780 (0.0008) +[2023-10-09 15:18:10,780][86121] Updated weights for policy 0, policy_version 78460 (0.0009) +[2023-10-09 15:18:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161021952. Throughput: 0: 1800.8, 1: 1818.1. Samples: 40259912. Policy #0 lag: (min: 31.0, avg: 32.9, max: 63.0) +[2023-10-09 15:18:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 15:18:14,400][86121] Updated weights for policy 0, policy_version 78470 (0.0009) +[2023-10-09 15:18:14,415][86122] Updated weights for policy 1, policy_version 78790 (0.0007) +[2023-10-09 15:18:14,764][86121] Updated weights for policy 0, policy_version 78480 (0.0007) +[2023-10-09 15:18:14,774][86122] Updated weights for policy 1, policy_version 78800 (0.0008) +[2023-10-09 15:18:15,134][86121] Updated weights for policy 0, policy_version 78490 (0.0007) +[2023-10-09 15:18:15,141][86122] Updated weights for policy 1, policy_version 78810 (0.0008) +[2023-10-09 15:18:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161087488. Throughput: 0: 1803.3, 1: 1821.3. Samples: 40282806. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:18,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:18:18,609][86121] Updated weights for policy 0, policy_version 78500 (0.0008) +[2023-10-09 15:18:18,737][86122] Updated weights for policy 1, policy_version 78820 (0.0007) +[2023-10-09 15:18:18,976][86121] Updated weights for policy 0, policy_version 78510 (0.0008) +[2023-10-09 15:18:19,097][86122] Updated weights for policy 1, policy_version 78830 (0.0008) +[2023-10-09 15:18:19,336][86121] Updated weights for policy 0, policy_version 78520 (0.0007) +[2023-10-09 15:18:19,466][86122] Updated weights for policy 1, policy_version 78840 (0.0008) +[2023-10-09 15:18:23,090][86121] Updated weights for policy 0, policy_version 78530 (0.0007) +[2023-10-09 15:18:23,138][86122] Updated weights for policy 1, policy_version 78850 (0.0008) +[2023-10-09 15:18:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161153024. Throughput: 0: 1805.0, 1: 1816.2. Samples: 40305644. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 15:18:23,451][86121] Updated weights for policy 0, policy_version 78540 (0.0007) +[2023-10-09 15:18:23,496][86122] Updated weights for policy 1, policy_version 78860 (0.0008) +[2023-10-09 15:18:23,820][86121] Updated weights for policy 0, policy_version 78550 (0.0009) +[2023-10-09 15:18:23,856][86122] Updated weights for policy 1, policy_version 78870 (0.0009) +[2023-10-09 15:18:24,188][86121] Updated weights for policy 0, policy_version 78560 (0.0008) +[2023-10-09 15:18:24,227][86122] Updated weights for policy 1, policy_version 78880 (0.0009) +[2023-10-09 15:18:27,826][86121] Updated weights for policy 0, policy_version 78570 (0.0007) +[2023-10-09 15:18:28,173][86122] Updated weights for policy 1, policy_version 78890 (0.0008) +[2023-10-09 15:18:28,190][86121] Updated weights for policy 0, policy_version 78580 (0.0010) +[2023-10-09 15:18:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161218560. Throughput: 0: 1799.3, 1: 1812.5. Samples: 40315430. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 15:18:28,545][86122] Updated weights for policy 1, policy_version 78900 (0.0007) +[2023-10-09 15:18:28,556][86121] Updated weights for policy 0, policy_version 78590 (0.0009) +[2023-10-09 15:18:28,902][86122] Updated weights for policy 1, policy_version 78910 (0.0008) +[2023-10-09 15:18:32,298][86121] Updated weights for policy 0, policy_version 78600 (0.0008) +[2023-10-09 15:18:32,566][86122] Updated weights for policy 1, policy_version 78920 (0.0007) +[2023-10-09 15:18:32,669][86121] Updated weights for policy 0, policy_version 78610 (0.0008) +[2023-10-09 15:18:32,939][86122] Updated weights for policy 1, policy_version 78930 (0.0008) +[2023-10-09 15:18:33,042][86121] Updated weights for policy 0, policy_version 78620 (0.0009) +[2023-10-09 15:18:33,291][86122] Updated weights for policy 1, policy_version 78940 (0.0007) +[2023-10-09 15:18:33,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161316864. Throughput: 0: 1813.5, 1: 1808.5. Samples: 40338240. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 15:18:36,863][86121] Updated weights for policy 0, policy_version 78630 (0.0007) +[2023-10-09 15:18:37,088][86122] Updated weights for policy 1, policy_version 78950 (0.0007) +[2023-10-09 15:18:37,232][86121] Updated weights for policy 0, policy_version 78640 (0.0009) +[2023-10-09 15:18:37,446][86122] Updated weights for policy 1, policy_version 78960 (0.0007) +[2023-10-09 15:18:37,602][86121] Updated weights for policy 0, policy_version 78650 (0.0008) +[2023-10-09 15:18:37,804][86122] Updated weights for policy 1, policy_version 78970 (0.0007) +[2023-10-09 15:18:38,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 161415168. Throughput: 0: 1812.8, 1: 1810.0. Samples: 40358282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 15:18:41,157][86121] Updated weights for policy 0, policy_version 78660 (0.0009) +[2023-10-09 15:18:41,366][86122] Updated weights for policy 1, policy_version 78980 (0.0007) +[2023-10-09 15:18:41,521][86121] Updated weights for policy 0, policy_version 78670 (0.0007) +[2023-10-09 15:18:41,729][86122] Updated weights for policy 1, policy_version 78990 (0.0007) +[2023-10-09 15:18:41,888][86121] Updated weights for policy 0, policy_version 78680 (0.0007) +[2023-10-09 15:18:42,083][86122] Updated weights for policy 1, policy_version 79000 (0.0008) +[2023-10-09 15:18:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 161480704. Throughput: 0: 1820.2, 1: 1809.0. Samples: 40370984. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:43,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 15:18:45,656][86121] Updated weights for policy 0, policy_version 78690 (0.0008) +[2023-10-09 15:18:45,908][86122] Updated weights for policy 1, policy_version 79010 (0.0008) +[2023-10-09 15:18:46,019][86121] Updated weights for policy 0, policy_version 78700 (0.0008) +[2023-10-09 15:18:46,264][86122] Updated weights for policy 1, policy_version 79020 (0.0008) +[2023-10-09 15:18:46,387][86121] Updated weights for policy 0, policy_version 78710 (0.0008) +[2023-10-09 15:18:46,629][86122] Updated weights for policy 1, policy_version 79030 (0.0009) +[2023-10-09 15:18:46,749][86121] Updated weights for policy 0, policy_version 78720 (0.0008) +[2023-10-09 15:18:46,982][86122] Updated weights for policy 1, policy_version 79040 (0.0009) +[2023-10-09 15:18:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 161546240. Throughput: 0: 1815.5, 1: 1811.1. Samples: 40390838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 15:18:50,495][86121] Updated weights for policy 0, policy_version 78730 (0.0010) +[2023-10-09 15:18:50,620][86122] Updated weights for policy 1, policy_version 79050 (0.0008) +[2023-10-09 15:18:50,873][86121] Updated weights for policy 0, policy_version 78740 (0.0008) +[2023-10-09 15:18:50,979][86122] Updated weights for policy 1, policy_version 79060 (0.0008) +[2023-10-09 15:18:51,227][86121] Updated weights for policy 0, policy_version 78750 (0.0008) +[2023-10-09 15:18:51,342][86122] Updated weights for policy 1, policy_version 79070 (0.0009) +[2023-10-09 15:18:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161611776. Throughput: 0: 1826.1, 1: 1804.0. Samples: 40413478. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 15:18:54,766][86121] Updated weights for policy 0, policy_version 78760 (0.0007) +[2023-10-09 15:18:55,134][86121] Updated weights for policy 0, policy_version 78770 (0.0008) +[2023-10-09 15:18:55,294][86122] Updated weights for policy 1, policy_version 79080 (0.0009) +[2023-10-09 15:18:55,500][86121] Updated weights for policy 0, policy_version 78780 (0.0009) +[2023-10-09 15:18:55,648][86122] Updated weights for policy 1, policy_version 79090 (0.0010) +[2023-10-09 15:18:56,014][86122] Updated weights for policy 1, policy_version 79100 (0.0007) +[2023-10-09 15:18:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161677312. Throughput: 0: 1830.2, 1: 1813.5. Samples: 40423876. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:18:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 15:18:59,311][86121] Updated weights for policy 0, policy_version 78790 (0.0009) +[2023-10-09 15:18:59,674][86121] Updated weights for policy 0, policy_version 78800 (0.0007) +[2023-10-09 15:18:59,753][86122] Updated weights for policy 1, policy_version 79110 (0.0007) +[2023-10-09 15:19:00,039][86121] Updated weights for policy 0, policy_version 78810 (0.0007) +[2023-10-09 15:19:00,108][86122] Updated weights for policy 1, policy_version 79120 (0.0009) +[2023-10-09 15:19:00,470][86122] Updated weights for policy 1, policy_version 79130 (0.0010) +[2023-10-09 15:19:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 161742848. Throughput: 0: 1822.1, 1: 1801.3. Samples: 40445860. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:03,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 15:19:03,688][86121] Updated weights for policy 0, policy_version 78820 (0.0007) +[2023-10-09 15:19:04,026][86122] Updated weights for policy 1, policy_version 79140 (0.0009) +[2023-10-09 15:19:04,046][86121] Updated weights for policy 0, policy_version 78830 (0.0009) +[2023-10-09 15:19:04,390][86122] Updated weights for policy 1, policy_version 79150 (0.0007) +[2023-10-09 15:19:04,416][86121] Updated weights for policy 0, policy_version 78840 (0.0007) +[2023-10-09 15:19:04,742][86122] Updated weights for policy 1, policy_version 79160 (0.0008) +[2023-10-09 15:19:08,217][86121] Updated weights for policy 0, policy_version 78850 (0.0008) +[2023-10-09 15:19:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 161808384. Throughput: 0: 1826.4, 1: 1797.8. Samples: 40468732. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:08,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 15:19:08,509][86122] Updated weights for policy 1, policy_version 79170 (0.0010) +[2023-10-09 15:19:08,582][86121] Updated weights for policy 0, policy_version 78860 (0.0008) +[2023-10-09 15:19:08,872][86122] Updated weights for policy 1, policy_version 79180 (0.0008) +[2023-10-09 15:19:08,953][86121] Updated weights for policy 0, policy_version 78870 (0.0009) +[2023-10-09 15:19:09,234][86122] Updated weights for policy 1, policy_version 79190 (0.0009) +[2023-10-09 15:19:09,318][86121] Updated weights for policy 0, policy_version 78880 (0.0007) +[2023-10-09 15:19:09,595][86122] Updated weights for policy 1, policy_version 79200 (0.0008) +[2023-10-09 15:19:12,968][86121] Updated weights for policy 0, policy_version 78890 (0.0008) +[2023-10-09 15:19:13,324][86121] Updated weights for policy 0, policy_version 78900 (0.0007) +[2023-10-09 15:19:13,363][86122] Updated weights for policy 1, policy_version 79210 (0.0009) +[2023-10-09 15:19:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 161873920. Throughput: 0: 1821.7, 1: 1803.9. Samples: 40478582. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 15:19:13,685][86121] Updated weights for policy 0, policy_version 78910 (0.0008) +[2023-10-09 15:19:13,724][86122] Updated weights for policy 1, policy_version 79220 (0.0008) +[2023-10-09 15:19:14,088][86122] Updated weights for policy 1, policy_version 79230 (0.0008) +[2023-10-09 15:19:17,382][86121] Updated weights for policy 0, policy_version 78920 (0.0009) +[2023-10-09 15:19:17,703][86122] Updated weights for policy 1, policy_version 79240 (0.0007) +[2023-10-09 15:19:17,747][86121] Updated weights for policy 0, policy_version 78930 (0.0009) +[2023-10-09 15:19:18,064][86122] Updated weights for policy 1, policy_version 79250 (0.0008) +[2023-10-09 15:19:18,109][86121] Updated weights for policy 0, policy_version 78940 (0.0008) +[2023-10-09 15:19:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 161972224. Throughput: 0: 1815.0, 1: 1809.8. Samples: 40501358. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 15:19:18,425][86122] Updated weights for policy 1, policy_version 79260 (0.0008) +[2023-10-09 15:19:21,836][86121] Updated weights for policy 0, policy_version 78950 (0.0010) +[2023-10-09 15:19:22,204][86121] Updated weights for policy 0, policy_version 78960 (0.0008) +[2023-10-09 15:19:22,231][86122] Updated weights for policy 1, policy_version 79270 (0.0009) +[2023-10-09 15:19:22,572][86121] Updated weights for policy 0, policy_version 78970 (0.0010) +[2023-10-09 15:19:22,598][86122] Updated weights for policy 1, policy_version 79280 (0.0007) +[2023-10-09 15:19:22,956][86122] Updated weights for policy 1, policy_version 79290 (0.0009) +[2023-10-09 15:19:23,397][85186] Fps is (10 sec: 19661.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 162070528. Throughput: 0: 1812.7, 1: 1816.0. Samples: 40521574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 15:19:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000078976_80871424.pth... +[2023-10-09 15:19:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000079296_81199104.pth... +[2023-10-09 15:19:23,439][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000077600_79462400.pth +[2023-10-09 15:19:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000077280_79134720.pth +[2023-10-09 15:19:26,260][86121] Updated weights for policy 0, policy_version 78980 (0.0009) +[2023-10-09 15:19:26,621][86121] Updated weights for policy 0, policy_version 78990 (0.0007) +[2023-10-09 15:19:26,624][86122] Updated weights for policy 1, policy_version 79300 (0.0009) +[2023-10-09 15:19:26,981][86122] Updated weights for policy 1, policy_version 79310 (0.0009) +[2023-10-09 15:19:26,983][86121] Updated weights for policy 0, policy_version 79000 (0.0007) +[2023-10-09 15:19:27,350][86122] Updated weights for policy 1, policy_version 79320 (0.0009) +[2023-10-09 15:19:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 162136064. Throughput: 0: 1813.3, 1: 1809.3. Samples: 40534000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:19:30,670][86121] Updated weights for policy 0, policy_version 79010 (0.0008) +[2023-10-09 15:19:31,031][86121] Updated weights for policy 0, policy_version 79020 (0.0011) +[2023-10-09 15:19:31,165][86122] Updated weights for policy 1, policy_version 79330 (0.0008) +[2023-10-09 15:19:31,406][86121] Updated weights for policy 0, policy_version 79030 (0.0007) +[2023-10-09 15:19:31,521][86122] Updated weights for policy 1, policy_version 79340 (0.0007) +[2023-10-09 15:19:31,773][86121] Updated weights for policy 0, policy_version 79040 (0.0008) +[2023-10-09 15:19:31,889][86122] Updated weights for policy 1, policy_version 79350 (0.0009) +[2023-10-09 15:19:32,253][86122] Updated weights for policy 1, policy_version 79360 (0.0010) +[2023-10-09 15:19:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162201600. Throughput: 0: 1814.7, 1: 1820.2. Samples: 40554410. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:33,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 15:19:35,451][86121] Updated weights for policy 0, policy_version 79050 (0.0009) +[2023-10-09 15:19:35,814][86121] Updated weights for policy 0, policy_version 79060 (0.0009) +[2023-10-09 15:19:35,838][86122] Updated weights for policy 1, policy_version 79370 (0.0007) +[2023-10-09 15:19:36,177][86121] Updated weights for policy 0, policy_version 79070 (0.0007) +[2023-10-09 15:19:36,196][86122] Updated weights for policy 1, policy_version 79380 (0.0008) +[2023-10-09 15:19:36,557][86122] Updated weights for policy 1, policy_version 79390 (0.0009) +[2023-10-09 15:19:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 162267136. Throughput: 0: 1811.3, 1: 1817.9. Samples: 40576790. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:19:40,031][86121] Updated weights for policy 0, policy_version 79080 (0.0009) +[2023-10-09 15:19:40,229][86122] Updated weights for policy 1, policy_version 79400 (0.0007) +[2023-10-09 15:19:40,387][86121] Updated weights for policy 0, policy_version 79090 (0.0010) +[2023-10-09 15:19:40,584][86122] Updated weights for policy 1, policy_version 79410 (0.0007) +[2023-10-09 15:19:40,751][86121] Updated weights for policy 0, policy_version 79100 (0.0008) +[2023-10-09 15:19:40,939][86122] Updated weights for policy 1, policy_version 79420 (0.0007) +[2023-10-09 15:19:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162332672. Throughput: 0: 1809.0, 1: 1815.9. Samples: 40586998. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:19:44,439][86121] Updated weights for policy 0, policy_version 79110 (0.0007) +[2023-10-09 15:19:44,568][86122] Updated weights for policy 1, policy_version 79430 (0.0007) +[2023-10-09 15:19:44,808][86121] Updated weights for policy 0, policy_version 79120 (0.0007) +[2023-10-09 15:19:44,923][86122] Updated weights for policy 1, policy_version 79440 (0.0009) +[2023-10-09 15:19:45,169][86121] Updated weights for policy 0, policy_version 79130 (0.0007) +[2023-10-09 15:19:45,279][86122] Updated weights for policy 1, policy_version 79450 (0.0008) +[2023-10-09 15:19:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162398208. Throughput: 0: 1810.3, 1: 1826.8. Samples: 40609528. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:19:48,911][86121] Updated weights for policy 0, policy_version 79140 (0.0009) +[2023-10-09 15:19:49,159][86122] Updated weights for policy 1, policy_version 79460 (0.0009) +[2023-10-09 15:19:49,273][86121] Updated weights for policy 0, policy_version 79150 (0.0008) +[2023-10-09 15:19:49,521][86122] Updated weights for policy 1, policy_version 79470 (0.0008) +[2023-10-09 15:19:49,638][86121] Updated weights for policy 0, policy_version 79160 (0.0008) +[2023-10-09 15:19:49,875][86122] Updated weights for policy 1, policy_version 79480 (0.0007) +[2023-10-09 15:19:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162463744. Throughput: 0: 1806.1, 1: 1827.1. Samples: 40632224. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:19:53,436][86121] Updated weights for policy 0, policy_version 79170 (0.0007) +[2023-10-09 15:19:53,548][86122] Updated weights for policy 1, policy_version 79490 (0.0008) +[2023-10-09 15:19:53,804][86121] Updated weights for policy 0, policy_version 79180 (0.0008) +[2023-10-09 15:19:53,911][86122] Updated weights for policy 1, policy_version 79500 (0.0009) +[2023-10-09 15:19:54,180][86121] Updated weights for policy 0, policy_version 79190 (0.0009) +[2023-10-09 15:19:54,282][86122] Updated weights for policy 1, policy_version 79510 (0.0007) +[2023-10-09 15:19:54,547][86121] Updated weights for policy 0, policy_version 79200 (0.0007) +[2023-10-09 15:19:54,645][86122] Updated weights for policy 1, policy_version 79520 (0.0007) +[2023-10-09 15:19:58,218][86121] Updated weights for policy 0, policy_version 79210 (0.0007) +[2023-10-09 15:19:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 162529280. Throughput: 0: 1807.3, 1: 1824.2. Samples: 40642000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:19:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:19:58,479][86122] Updated weights for policy 1, policy_version 79530 (0.0009) +[2023-10-09 15:19:58,580][86121] Updated weights for policy 0, policy_version 79220 (0.0007) +[2023-10-09 15:19:58,837][86122] Updated weights for policy 1, policy_version 79540 (0.0007) +[2023-10-09 15:19:58,947][86121] Updated weights for policy 0, policy_version 79230 (0.0008) +[2023-10-09 15:19:59,195][86122] Updated weights for policy 1, policy_version 79550 (0.0009) +[2023-10-09 15:20:02,683][86121] Updated weights for policy 0, policy_version 79240 (0.0007) +[2023-10-09 15:20:02,911][86122] Updated weights for policy 1, policy_version 79560 (0.0007) +[2023-10-09 15:20:03,055][86121] Updated weights for policy 0, policy_version 79250 (0.0009) +[2023-10-09 15:20:03,266][86122] Updated weights for policy 1, policy_version 79570 (0.0007) +[2023-10-09 15:20:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 162594816. Throughput: 0: 1804.5, 1: 1821.0. Samples: 40664506. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:20:03,426][86121] Updated weights for policy 0, policy_version 79260 (0.0007) +[2023-10-09 15:20:03,622][86122] Updated weights for policy 1, policy_version 79580 (0.0007) +[2023-10-09 15:20:07,342][86122] Updated weights for policy 1, policy_version 79590 (0.0008) +[2023-10-09 15:20:07,359][86121] Updated weights for policy 0, policy_version 79270 (0.0008) +[2023-10-09 15:20:07,705][86122] Updated weights for policy 1, policy_version 79600 (0.0009) +[2023-10-09 15:20:07,718][86121] Updated weights for policy 0, policy_version 79280 (0.0008) +[2023-10-09 15:20:08,069][86122] Updated weights for policy 1, policy_version 79610 (0.0007) +[2023-10-09 15:20:08,085][86121] Updated weights for policy 0, policy_version 79290 (0.0008) +[2023-10-09 15:20:08,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 162725888. Throughput: 0: 1811.8, 1: 1827.6. Samples: 40685348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:08,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:20:11,705][86122] Updated weights for policy 1, policy_version 79620 (0.0007) +[2023-10-09 15:20:11,891][86121] Updated weights for policy 0, policy_version 79300 (0.0007) +[2023-10-09 15:20:12,073][86122] Updated weights for policy 1, policy_version 79630 (0.0008) +[2023-10-09 15:20:12,264][86121] Updated weights for policy 0, policy_version 79310 (0.0009) +[2023-10-09 15:20:12,441][86122] Updated weights for policy 1, policy_version 79640 (0.0008) +[2023-10-09 15:20:12,629][86121] Updated weights for policy 0, policy_version 79320 (0.0007) +[2023-10-09 15:20:13,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 162791424. Throughput: 0: 1801.9, 1: 1822.9. Samples: 40697114. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:13,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:20:16,108][86122] Updated weights for policy 1, policy_version 79650 (0.0011) +[2023-10-09 15:20:16,299][86121] Updated weights for policy 0, policy_version 79330 (0.0009) +[2023-10-09 15:20:16,475][86122] Updated weights for policy 1, policy_version 79660 (0.0008) +[2023-10-09 15:20:16,670][86121] Updated weights for policy 0, policy_version 79340 (0.0008) +[2023-10-09 15:20:16,833][86122] Updated weights for policy 1, policy_version 79670 (0.0008) +[2023-10-09 15:20:17,032][86121] Updated weights for policy 0, policy_version 79350 (0.0007) +[2023-10-09 15:20:17,187][86122] Updated weights for policy 1, policy_version 79680 (0.0007) +[2023-10-09 15:20:17,400][86121] Updated weights for policy 0, policy_version 79360 (0.0007) +[2023-10-09 15:20:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 162856960. Throughput: 0: 1816.6, 1: 1822.5. Samples: 40718172. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:18,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:20:20,943][86122] Updated weights for policy 1, policy_version 79690 (0.0008) +[2023-10-09 15:20:21,078][86121] Updated weights for policy 0, policy_version 79370 (0.0009) +[2023-10-09 15:20:21,310][86122] Updated weights for policy 1, policy_version 79700 (0.0007) +[2023-10-09 15:20:21,445][86121] Updated weights for policy 0, policy_version 79380 (0.0007) +[2023-10-09 15:20:21,668][86122] Updated weights for policy 1, policy_version 79710 (0.0007) +[2023-10-09 15:20:21,815][86121] Updated weights for policy 0, policy_version 79390 (0.0008) +[2023-10-09 15:20:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 162922496. Throughput: 0: 1799.8, 1: 1820.3. Samples: 40739692. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:20:25,280][86122] Updated weights for policy 1, policy_version 79720 (0.0009) +[2023-10-09 15:20:25,609][86121] Updated weights for policy 0, policy_version 79400 (0.0009) +[2023-10-09 15:20:25,635][86122] Updated weights for policy 1, policy_version 79730 (0.0010) +[2023-10-09 15:20:25,980][86121] Updated weights for policy 0, policy_version 79410 (0.0008) +[2023-10-09 15:20:25,992][86122] Updated weights for policy 1, policy_version 79740 (0.0009) +[2023-10-09 15:20:26,350][86121] Updated weights for policy 0, policy_version 79420 (0.0009) +[2023-10-09 15:20:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 162988032. Throughput: 0: 1815.7, 1: 1824.0. Samples: 40750784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:20:29,751][86122] Updated weights for policy 1, policy_version 79750 (0.0009) +[2023-10-09 15:20:30,037][86121] Updated weights for policy 0, policy_version 79430 (0.0007) +[2023-10-09 15:20:30,106][86122] Updated weights for policy 1, policy_version 79760 (0.0009) +[2023-10-09 15:20:30,398][86121] Updated weights for policy 0, policy_version 79440 (0.0009) +[2023-10-09 15:20:30,475][86122] Updated weights for policy 1, policy_version 79770 (0.0009) +[2023-10-09 15:20:30,758][86121] Updated weights for policy 0, policy_version 79450 (0.0009) +[2023-10-09 15:20:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163053568. Throughput: 0: 1801.2, 1: 1816.0. Samples: 40772302. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:20:34,174][86122] Updated weights for policy 1, policy_version 79780 (0.0008) +[2023-10-09 15:20:34,193][86121] Updated weights for policy 0, policy_version 79460 (0.0009) +[2023-10-09 15:20:34,548][86122] Updated weights for policy 1, policy_version 79790 (0.0008) +[2023-10-09 15:20:34,564][86121] Updated weights for policy 0, policy_version 79470 (0.0008) +[2023-10-09 15:20:34,912][86122] Updated weights for policy 1, policy_version 79800 (0.0009) +[2023-10-09 15:20:34,924][86121] Updated weights for policy 0, policy_version 79480 (0.0007) +[2023-10-09 15:20:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 163119104. Throughput: 0: 1815.3, 1: 1811.9. Samples: 40795446. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:20:38,543][86121] Updated weights for policy 0, policy_version 79490 (0.0010) +[2023-10-09 15:20:38,690][86122] Updated weights for policy 1, policy_version 79810 (0.0008) +[2023-10-09 15:20:38,904][86121] Updated weights for policy 0, policy_version 79500 (0.0008) +[2023-10-09 15:20:39,056][86122] Updated weights for policy 1, policy_version 79820 (0.0008) +[2023-10-09 15:20:39,271][86121] Updated weights for policy 0, policy_version 79510 (0.0008) +[2023-10-09 15:20:39,414][86122] Updated weights for policy 1, policy_version 79830 (0.0008) +[2023-10-09 15:20:39,631][86121] Updated weights for policy 0, policy_version 79520 (0.0008) +[2023-10-09 15:20:39,772][86122] Updated weights for policy 1, policy_version 79840 (0.0010) +[2023-10-09 15:20:43,394][86121] Updated weights for policy 0, policy_version 79530 (0.0009) +[2023-10-09 15:20:43,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163184640. Throughput: 0: 1819.6, 1: 1811.5. Samples: 40805402. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:20:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:20:43,631][86122] Updated weights for policy 1, policy_version 79850 (0.0007) +[2023-10-09 15:20:43,754][86121] Updated weights for policy 0, policy_version 79540 (0.0009) +[2023-10-09 15:20:44,000][86122] Updated weights for policy 1, policy_version 79860 (0.0007) +[2023-10-09 15:20:44,123][86121] Updated weights for policy 0, policy_version 79550 (0.0008) +[2023-10-09 15:20:44,354][86122] Updated weights for policy 1, policy_version 79870 (0.0007) +[2023-10-09 15:20:47,892][86121] Updated weights for policy 0, policy_version 79560 (0.0010) +[2023-10-09 15:20:47,916][86122] Updated weights for policy 1, policy_version 79880 (0.0008) +[2023-10-09 15:20:48,264][86121] Updated weights for policy 0, policy_version 79570 (0.0008) +[2023-10-09 15:20:48,273][86122] Updated weights for policy 1, policy_version 79890 (0.0008) +[2023-10-09 15:20:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163250176. Throughput: 0: 1813.0, 1: 1816.0. Samples: 40827810. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:20:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:20:48,620][86121] Updated weights for policy 0, policy_version 79580 (0.0008) +[2023-10-09 15:20:48,636][86122] Updated weights for policy 1, policy_version 79900 (0.0007) +[2023-10-09 15:20:52,313][86122] Updated weights for policy 1, policy_version 79910 (0.0008) +[2023-10-09 15:20:52,456][86121] Updated weights for policy 0, policy_version 79590 (0.0009) +[2023-10-09 15:20:52,689][86122] Updated weights for policy 1, policy_version 79920 (0.0009) +[2023-10-09 15:20:52,813][86121] Updated weights for policy 0, policy_version 79600 (0.0007) +[2023-10-09 15:20:53,054][86122] Updated weights for policy 1, policy_version 79930 (0.0009) +[2023-10-09 15:20:53,170][86121] Updated weights for policy 0, policy_version 79610 (0.0010) +[2023-10-09 15:20:53,397][85186] Fps is (10 sec: 19660.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 163381248. Throughput: 0: 1816.2, 1: 1812.6. Samples: 40848644. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:20:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:20:56,737][86122] Updated weights for policy 1, policy_version 79940 (0.0009) +[2023-10-09 15:20:56,908][86121] Updated weights for policy 0, policy_version 79620 (0.0009) +[2023-10-09 15:20:57,109][86122] Updated weights for policy 1, policy_version 79950 (0.0008) +[2023-10-09 15:20:57,264][86121] Updated weights for policy 0, policy_version 79630 (0.0007) +[2023-10-09 15:20:57,458][86122] Updated weights for policy 1, policy_version 79960 (0.0009) +[2023-10-09 15:20:57,627][86121] Updated weights for policy 0, policy_version 79640 (0.0008) +[2023-10-09 15:20:58,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 163446784. Throughput: 0: 1813.4, 1: 1813.7. Samples: 40860334. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:20:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:21:01,171][86122] Updated weights for policy 1, policy_version 79970 (0.0009) +[2023-10-09 15:21:01,368][86121] Updated weights for policy 0, policy_version 79650 (0.0009) +[2023-10-09 15:21:01,532][86122] Updated weights for policy 1, policy_version 79980 (0.0008) +[2023-10-09 15:21:01,725][86121] Updated weights for policy 0, policy_version 79660 (0.0009) +[2023-10-09 15:21:01,897][86122] Updated weights for policy 1, policy_version 79990 (0.0007) +[2023-10-09 15:21:02,089][86121] Updated weights for policy 0, policy_version 79670 (0.0008) +[2023-10-09 15:21:02,247][86122] Updated weights for policy 1, policy_version 80000 (0.0008) +[2023-10-09 15:21:02,455][86121] Updated weights for policy 0, policy_version 79680 (0.0008) +[2023-10-09 15:21:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 163512320. Throughput: 0: 1812.8, 1: 1814.3. Samples: 40881390. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:21:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:21:05,947][86122] Updated weights for policy 1, policy_version 80010 (0.0009) +[2023-10-09 15:21:06,180][86121] Updated weights for policy 0, policy_version 79690 (0.0007) +[2023-10-09 15:21:06,316][86122] Updated weights for policy 1, policy_version 80020 (0.0008) +[2023-10-09 15:21:06,534][86121] Updated weights for policy 0, policy_version 79700 (0.0008) +[2023-10-09 15:21:06,669][86122] Updated weights for policy 1, policy_version 80030 (0.0009) +[2023-10-09 15:21:06,900][86121] Updated weights for policy 0, policy_version 79710 (0.0008) +[2023-10-09 15:21:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 163577856. Throughput: 0: 1809.0, 1: 1819.5. Samples: 40902976. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:21:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:21:10,356][86122] Updated weights for policy 1, policy_version 80040 (0.0008) +[2023-10-09 15:21:10,716][86122] Updated weights for policy 1, policy_version 80050 (0.0009) +[2023-10-09 15:21:10,933][86121] Updated weights for policy 0, policy_version 79720 (0.0007) +[2023-10-09 15:21:11,073][86122] Updated weights for policy 1, policy_version 80060 (0.0008) +[2023-10-09 15:21:11,300][86121] Updated weights for policy 0, policy_version 79730 (0.0008) +[2023-10-09 15:21:11,671][86121] Updated weights for policy 0, policy_version 79740 (0.0010) +[2023-10-09 15:21:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 163643392. Throughput: 0: 1813.5, 1: 1818.3. Samples: 40914214. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:21:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:21:14,610][86122] Updated weights for policy 1, policy_version 80070 (0.0007) +[2023-10-09 15:21:14,968][86122] Updated weights for policy 1, policy_version 80080 (0.0007) +[2023-10-09 15:21:15,328][86122] Updated weights for policy 1, policy_version 80090 (0.0009) +[2023-10-09 15:21:15,454][86121] Updated weights for policy 0, policy_version 79750 (0.0008) +[2023-10-09 15:21:15,821][86121] Updated weights for policy 0, policy_version 79760 (0.0007) +[2023-10-09 15:21:16,193][86121] Updated weights for policy 0, policy_version 79770 (0.0007) +[2023-10-09 15:21:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163708928. Throughput: 0: 1804.3, 1: 1822.7. Samples: 40935518. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:21:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:21:19,017][86122] Updated weights for policy 1, policy_version 80100 (0.0008) +[2023-10-09 15:21:19,375][86122] Updated weights for policy 1, policy_version 80110 (0.0007) +[2023-10-09 15:21:19,692][86121] Updated weights for policy 0, policy_version 79780 (0.0008) +[2023-10-09 15:21:19,731][86122] Updated weights for policy 1, policy_version 80120 (0.0007) +[2023-10-09 15:21:20,056][86121] Updated weights for policy 0, policy_version 79790 (0.0007) +[2023-10-09 15:21:20,425][86121] Updated weights for policy 0, policy_version 79800 (0.0008) +[2023-10-09 15:21:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 163774464. Throughput: 0: 1791.9, 1: 1834.0. Samples: 40958610. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:21:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:21:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000079808_81723392.pth... +[2023-10-09 15:21:23,420][86122] Updated weights for policy 1, policy_version 80130 (0.0008) +[2023-10-09 15:21:23,442][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000078112_79986688.pth +[2023-10-09 15:21:23,785][86122] Updated weights for policy 1, policy_version 80140 (0.0008) +[2023-10-09 15:21:24,140][86122] Updated weights for policy 1, policy_version 80150 (0.0010) +[2023-10-09 15:21:24,242][86121] Updated weights for policy 0, policy_version 79810 (0.0010) +[2023-10-09 15:21:24,493][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000080160_82083840.pth... +[2023-10-09 15:21:24,497][86122] Updated weights for policy 1, policy_version 80160 (0.0007) +[2023-10-09 15:21:24,522][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000078432_80314368.pth +[2023-10-09 15:21:24,607][86121] Updated weights for policy 0, policy_version 79820 (0.0009) +[2023-10-09 15:21:24,967][86121] Updated weights for policy 0, policy_version 79830 (0.0007) +[2023-10-09 15:21:25,335][86121] Updated weights for policy 0, policy_version 79840 (0.0007) +[2023-10-09 15:21:28,121][86122] Updated weights for policy 1, policy_version 80170 (0.0008) +[2023-10-09 15:21:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 163840000. Throughput: 0: 1789.2, 1: 1839.6. Samples: 40968698. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:21:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:21:28,489][86122] Updated weights for policy 1, policy_version 80180 (0.0009) +[2023-10-09 15:21:28,851][86122] Updated weights for policy 1, policy_version 80190 (0.0009) +[2023-10-09 15:21:28,882][86121] Updated weights for policy 0, policy_version 79850 (0.0008) +[2023-10-09 15:21:29,244][86121] Updated weights for policy 0, policy_version 79860 (0.0009) +[2023-10-09 15:21:29,609][86121] Updated weights for policy 0, policy_version 79870 (0.0010) +[2023-10-09 15:21:32,505][86122] Updated weights for policy 1, policy_version 80200 (0.0009) +[2023-10-09 15:21:32,869][86122] Updated weights for policy 1, policy_version 80210 (0.0009) +[2023-10-09 15:21:33,244][86122] Updated weights for policy 1, policy_version 80220 (0.0010) +[2023-10-09 15:21:33,307][86121] Updated weights for policy 0, policy_version 79880 (0.0008) +[2023-10-09 15:21:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 163938304. Throughput: 0: 1801.2, 1: 1834.1. Samples: 40991400. Policy #0 lag: (min: 15.0, avg: 38.4, max: 40.0) +[2023-10-09 15:21:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:21:33,678][86121] Updated weights for policy 0, policy_version 79890 (0.0009) +[2023-10-09 15:21:34,052][86121] Updated weights for policy 0, policy_version 79900 (0.0009) +[2023-10-09 15:21:36,979][86122] Updated weights for policy 1, policy_version 80230 (0.0009) +[2023-10-09 15:21:37,338][86122] Updated weights for policy 1, policy_version 80240 (0.0011) +[2023-10-09 15:21:37,627][86121] Updated weights for policy 0, policy_version 79910 (0.0007) +[2023-10-09 15:21:37,699][86122] Updated weights for policy 1, policy_version 80250 (0.0007) +[2023-10-09 15:21:37,984][86121] Updated weights for policy 0, policy_version 79920 (0.0007) +[2023-10-09 15:21:38,349][86121] Updated weights for policy 0, policy_version 79930 (0.0010) +[2023-10-09 15:21:38,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 164003840. Throughput: 0: 1814.0, 1: 1822.5. Samples: 41012288. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:21:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:21:41,512][86122] Updated weights for policy 1, policy_version 80260 (0.0009) +[2023-10-09 15:21:41,874][86122] Updated weights for policy 1, policy_version 80270 (0.0009) +[2023-10-09 15:21:42,054][86121] Updated weights for policy 0, policy_version 79940 (0.0009) +[2023-10-09 15:21:42,232][86122] Updated weights for policy 1, policy_version 80280 (0.0008) +[2023-10-09 15:21:42,419][86121] Updated weights for policy 0, policy_version 79950 (0.0008) +[2023-10-09 15:21:42,785][86121] Updated weights for policy 0, policy_version 79960 (0.0011) +[2023-10-09 15:21:43,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 164102144. Throughput: 0: 1805.7, 1: 1833.2. Samples: 41024086. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:21:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:21:45,890][86122] Updated weights for policy 1, policy_version 80290 (0.0008) +[2023-10-09 15:21:46,247][86122] Updated weights for policy 1, policy_version 80300 (0.0009) +[2023-10-09 15:21:46,567][86121] Updated weights for policy 0, policy_version 79970 (0.0009) +[2023-10-09 15:21:46,599][86122] Updated weights for policy 1, policy_version 80310 (0.0007) +[2023-10-09 15:21:46,937][86121] Updated weights for policy 0, policy_version 79980 (0.0007) +[2023-10-09 15:21:46,967][86122] Updated weights for policy 1, policy_version 80320 (0.0008) +[2023-10-09 15:21:47,311][86121] Updated weights for policy 0, policy_version 79990 (0.0008) +[2023-10-09 15:21:47,673][86121] Updated weights for policy 0, policy_version 80000 (0.0010) +[2023-10-09 15:21:48,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 164167680. Throughput: 0: 1814.7, 1: 1822.1. Samples: 41045044. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:21:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:21:50,860][86122] Updated weights for policy 1, policy_version 80330 (0.0010) +[2023-10-09 15:21:51,224][86122] Updated weights for policy 1, policy_version 80340 (0.0009) +[2023-10-09 15:21:51,429][86121] Updated weights for policy 0, policy_version 80010 (0.0009) +[2023-10-09 15:21:51,590][86122] Updated weights for policy 1, policy_version 80350 (0.0008) +[2023-10-09 15:21:51,792][86121] Updated weights for policy 0, policy_version 80020 (0.0007) +[2023-10-09 15:21:52,162][86121] Updated weights for policy 0, policy_version 80030 (0.0011) +[2023-10-09 15:21:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 164233216. Throughput: 0: 1807.8, 1: 1828.0. Samples: 41066588. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:21:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:21:55,232][86122] Updated weights for policy 1, policy_version 80360 (0.0008) +[2023-10-09 15:21:55,591][86122] Updated weights for policy 1, policy_version 80370 (0.0008) +[2023-10-09 15:21:55,818][86121] Updated weights for policy 0, policy_version 80040 (0.0008) +[2023-10-09 15:21:55,950][86122] Updated weights for policy 1, policy_version 80380 (0.0009) +[2023-10-09 15:21:56,182][86121] Updated weights for policy 0, policy_version 80050 (0.0011) +[2023-10-09 15:21:56,547][86121] Updated weights for policy 0, policy_version 80060 (0.0008) +[2023-10-09 15:21:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 164298752. Throughput: 0: 1811.3, 1: 1827.8. Samples: 41077976. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:21:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:21:59,548][86122] Updated weights for policy 1, policy_version 80390 (0.0009) +[2023-10-09 15:21:59,903][86122] Updated weights for policy 1, policy_version 80400 (0.0008) +[2023-10-09 15:22:00,270][86122] Updated weights for policy 1, policy_version 80410 (0.0009) +[2023-10-09 15:22:00,314][86121] Updated weights for policy 0, policy_version 80070 (0.0008) +[2023-10-09 15:22:00,681][86121] Updated weights for policy 0, policy_version 80080 (0.0009) +[2023-10-09 15:22:01,045][86121] Updated weights for policy 0, policy_version 80090 (0.0009) +[2023-10-09 15:22:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 164364288. Throughput: 0: 1816.0, 1: 1831.6. Samples: 41099664. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:22:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:22:03,833][86122] Updated weights for policy 1, policy_version 80420 (0.0009) +[2023-10-09 15:22:04,191][86122] Updated weights for policy 1, policy_version 80430 (0.0009) +[2023-10-09 15:22:04,554][86122] Updated weights for policy 1, policy_version 80440 (0.0007) +[2023-10-09 15:22:04,684][86121] Updated weights for policy 0, policy_version 80100 (0.0009) +[2023-10-09 15:22:05,047][86121] Updated weights for policy 0, policy_version 80110 (0.0007) +[2023-10-09 15:22:05,404][86121] Updated weights for policy 0, policy_version 80120 (0.0010) +[2023-10-09 15:22:08,104][86122] Updated weights for policy 1, policy_version 80450 (0.0008) +[2023-10-09 15:22:08,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 164429824. Throughput: 0: 1815.5, 1: 1834.3. Samples: 41122850. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:22:08,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:22:08,478][86122] Updated weights for policy 1, policy_version 80460 (0.0008) +[2023-10-09 15:22:08,850][86122] Updated weights for policy 1, policy_version 80470 (0.0010) +[2023-10-09 15:22:09,149][86121] Updated weights for policy 0, policy_version 80130 (0.0010) +[2023-10-09 15:22:09,207][86122] Updated weights for policy 1, policy_version 80480 (0.0009) +[2023-10-09 15:22:09,501][86121] Updated weights for policy 0, policy_version 80140 (0.0010) +[2023-10-09 15:22:09,862][86121] Updated weights for policy 0, policy_version 80150 (0.0008) +[2023-10-09 15:22:10,225][86121] Updated weights for policy 0, policy_version 80160 (0.0008) +[2023-10-09 15:22:13,009][86122] Updated weights for policy 1, policy_version 80490 (0.0008) +[2023-10-09 15:22:13,379][86122] Updated weights for policy 1, policy_version 80500 (0.0009) +[2023-10-09 15:22:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 164495360. Throughput: 0: 1816.0, 1: 1829.5. Samples: 41132744. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:22:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:22:13,744][86122] Updated weights for policy 1, policy_version 80510 (0.0008) +[2023-10-09 15:22:14,045][86121] Updated weights for policy 0, policy_version 80170 (0.0007) +[2023-10-09 15:22:14,422][86121] Updated weights for policy 0, policy_version 80180 (0.0007) +[2023-10-09 15:22:14,775][86121] Updated weights for policy 0, policy_version 80190 (0.0007) +[2023-10-09 15:22:17,502][86122] Updated weights for policy 1, policy_version 80520 (0.0007) +[2023-10-09 15:22:17,874][86122] Updated weights for policy 1, policy_version 80530 (0.0007) +[2023-10-09 15:22:18,234][86122] Updated weights for policy 1, policy_version 80540 (0.0010) +[2023-10-09 15:22:18,363][86121] Updated weights for policy 0, policy_version 80200 (0.0008) +[2023-10-09 15:22:18,397][85186] Fps is (10 sec: 16384.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164593664. Throughput: 0: 1810.2, 1: 1835.8. Samples: 41155470. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:22:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:22:18,726][86121] Updated weights for policy 0, policy_version 80210 (0.0007) +[2023-10-09 15:22:19,090][86121] Updated weights for policy 0, policy_version 80220 (0.0007) +[2023-10-09 15:22:21,825][86122] Updated weights for policy 1, policy_version 80550 (0.0008) +[2023-10-09 15:22:22,198][86122] Updated weights for policy 1, policy_version 80560 (0.0009) +[2023-10-09 15:22:22,557][86122] Updated weights for policy 1, policy_version 80570 (0.0008) +[2023-10-09 15:22:22,753][86121] Updated weights for policy 0, policy_version 80230 (0.0007) +[2023-10-09 15:22:23,120][86121] Updated weights for policy 0, policy_version 80240 (0.0009) +[2023-10-09 15:22:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 164659200. Throughput: 0: 1817.1, 1: 1836.5. Samples: 41176702. Policy #0 lag: (min: 18.0, avg: 21.2, max: 50.0) +[2023-10-09 15:22:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:22:23,491][86121] Updated weights for policy 0, policy_version 80250 (0.0008) +[2023-10-09 15:22:26,255][86122] Updated weights for policy 1, policy_version 80580 (0.0009) +[2023-10-09 15:22:26,610][86122] Updated weights for policy 1, policy_version 80590 (0.0008) +[2023-10-09 15:22:26,975][86122] Updated weights for policy 1, policy_version 80600 (0.0007) +[2023-10-09 15:22:27,257][86121] Updated weights for policy 0, policy_version 80260 (0.0007) +[2023-10-09 15:22:27,625][86121] Updated weights for policy 0, policy_version 80270 (0.0008) +[2023-10-09 15:22:27,998][86121] Updated weights for policy 0, policy_version 80280 (0.0008) +[2023-10-09 15:22:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 164757504. Throughput: 0: 1809.2, 1: 1840.4. Samples: 41188322. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:22:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:22:30,645][86122] Updated weights for policy 1, policy_version 80610 (0.0008) +[2023-10-09 15:22:30,999][86122] Updated weights for policy 1, policy_version 80620 (0.0011) +[2023-10-09 15:22:31,363][86122] Updated weights for policy 1, policy_version 80630 (0.0008) +[2023-10-09 15:22:31,680][86121] Updated weights for policy 0, policy_version 80290 (0.0008) +[2023-10-09 15:22:31,724][86122] Updated weights for policy 1, policy_version 80640 (0.0009) +[2023-10-09 15:22:32,052][86121] Updated weights for policy 0, policy_version 80300 (0.0010) +[2023-10-09 15:22:32,416][86121] Updated weights for policy 0, policy_version 80310 (0.0008) +[2023-10-09 15:22:32,783][86121] Updated weights for policy 0, policy_version 80320 (0.0007) +[2023-10-09 15:22:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164823040. Throughput: 0: 1817.9, 1: 1832.8. Samples: 41209322. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:22:33,399][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:22:35,530][86122] Updated weights for policy 1, policy_version 80650 (0.0010) +[2023-10-09 15:22:35,901][86122] Updated weights for policy 1, policy_version 80660 (0.0010) +[2023-10-09 15:22:36,258][86122] Updated weights for policy 1, policy_version 80670 (0.0007) +[2023-10-09 15:22:36,466][86121] Updated weights for policy 0, policy_version 80330 (0.0008) +[2023-10-09 15:22:36,830][86121] Updated weights for policy 0, policy_version 80340 (0.0008) +[2023-10-09 15:22:37,196][86121] Updated weights for policy 0, policy_version 80350 (0.0008) +[2023-10-09 15:22:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 164888576. Throughput: 0: 1821.3, 1: 1833.7. Samples: 41231066. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:22:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:22:40,049][86122] Updated weights for policy 1, policy_version 80680 (0.0011) +[2023-10-09 15:22:40,413][86122] Updated weights for policy 1, policy_version 80690 (0.0011) +[2023-10-09 15:22:40,774][86122] Updated weights for policy 1, policy_version 80700 (0.0009) +[2023-10-09 15:22:41,017][86121] Updated weights for policy 0, policy_version 80360 (0.0009) +[2023-10-09 15:22:41,383][86121] Updated weights for policy 0, policy_version 80370 (0.0009) +[2023-10-09 15:22:41,754][86121] Updated weights for policy 0, policy_version 80380 (0.0007) +[2023-10-09 15:22:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 164954112. Throughput: 0: 1822.0, 1: 1827.0. Samples: 41242182. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:22:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:22:44,441][86122] Updated weights for policy 1, policy_version 80710 (0.0007) +[2023-10-09 15:22:44,801][86122] Updated weights for policy 1, policy_version 80720 (0.0007) +[2023-10-09 15:22:45,170][86122] Updated weights for policy 1, policy_version 80730 (0.0010) +[2023-10-09 15:22:45,531][86121] Updated weights for policy 0, policy_version 80390 (0.0008) +[2023-10-09 15:22:45,887][86121] Updated weights for policy 0, policy_version 80400 (0.0009) +[2023-10-09 15:22:46,265][86121] Updated weights for policy 0, policy_version 80410 (0.0010) +[2023-10-09 15:22:48,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165019648. Throughput: 0: 1814.9, 1: 1827.1. Samples: 41263558. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:22:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:22:48,838][86122] Updated weights for policy 1, policy_version 80740 (0.0008) +[2023-10-09 15:22:49,196][86122] Updated weights for policy 1, policy_version 80750 (0.0007) +[2023-10-09 15:22:49,564][86122] Updated weights for policy 1, policy_version 80760 (0.0009) +[2023-10-09 15:22:49,986][86121] Updated weights for policy 0, policy_version 80420 (0.0008) +[2023-10-09 15:22:50,360][86121] Updated weights for policy 0, policy_version 80430 (0.0009) +[2023-10-09 15:22:50,722][86121] Updated weights for policy 0, policy_version 80440 (0.0008) +[2023-10-09 15:22:53,212][86122] Updated weights for policy 1, policy_version 80770 (0.0008) +[2023-10-09 15:22:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165085184. Throughput: 0: 1816.3, 1: 1817.6. Samples: 41286374. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:22:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:22:53,568][86122] Updated weights for policy 1, policy_version 80780 (0.0008) +[2023-10-09 15:22:53,939][86122] Updated weights for policy 1, policy_version 80790 (0.0009) +[2023-10-09 15:22:54,308][86122] Updated weights for policy 1, policy_version 80800 (0.0010) +[2023-10-09 15:22:54,432][86121] Updated weights for policy 0, policy_version 80450 (0.0008) +[2023-10-09 15:22:54,807][86121] Updated weights for policy 0, policy_version 80460 (0.0007) +[2023-10-09 15:22:55,176][86121] Updated weights for policy 0, policy_version 80470 (0.0008) +[2023-10-09 15:22:55,546][86121] Updated weights for policy 0, policy_version 80480 (0.0010) +[2023-10-09 15:22:58,029][86122] Updated weights for policy 1, policy_version 80810 (0.0008) +[2023-10-09 15:22:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165150720. Throughput: 0: 1813.2, 1: 1819.5. Samples: 41296218. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:22:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:22:58,398][86122] Updated weights for policy 1, policy_version 80820 (0.0010) +[2023-10-09 15:22:58,756][86122] Updated weights for policy 1, policy_version 80830 (0.0007) +[2023-10-09 15:22:59,465][86121] Updated weights for policy 0, policy_version 80490 (0.0007) +[2023-10-09 15:22:59,831][86121] Updated weights for policy 0, policy_version 80500 (0.0008) +[2023-10-09 15:23:00,195][86121] Updated weights for policy 0, policy_version 80510 (0.0010) +[2023-10-09 15:23:02,550][86122] Updated weights for policy 1, policy_version 80840 (0.0008) +[2023-10-09 15:23:02,912][86122] Updated weights for policy 1, policy_version 80850 (0.0010) +[2023-10-09 15:23:03,277][86122] Updated weights for policy 1, policy_version 80860 (0.0010) +[2023-10-09 15:23:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165216256. Throughput: 0: 1805.6, 1: 1817.3. Samples: 41318500. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:23:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:03,958][86121] Updated weights for policy 0, policy_version 80520 (0.0009) +[2023-10-09 15:23:04,325][86121] Updated weights for policy 0, policy_version 80530 (0.0010) +[2023-10-09 15:23:04,702][86121] Updated weights for policy 0, policy_version 80540 (0.0007) +[2023-10-09 15:23:06,921][86122] Updated weights for policy 1, policy_version 80870 (0.0009) +[2023-10-09 15:23:07,284][86122] Updated weights for policy 1, policy_version 80880 (0.0008) +[2023-10-09 15:23:07,646][86122] Updated weights for policy 1, policy_version 80890 (0.0007) +[2023-10-09 15:23:08,266][86121] Updated weights for policy 0, policy_version 80550 (0.0010) +[2023-10-09 15:23:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 165314560. Throughput: 0: 1813.0, 1: 1818.6. Samples: 41340126. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:23:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:08,635][86121] Updated weights for policy 0, policy_version 80560 (0.0008) +[2023-10-09 15:23:09,006][86121] Updated weights for policy 0, policy_version 80570 (0.0010) +[2023-10-09 15:23:11,279][86122] Updated weights for policy 1, policy_version 80900 (0.0008) +[2023-10-09 15:23:11,643][86122] Updated weights for policy 1, policy_version 80910 (0.0008) +[2023-10-09 15:23:12,006][86122] Updated weights for policy 1, policy_version 80920 (0.0010) +[2023-10-09 15:23:12,611][86121] Updated weights for policy 0, policy_version 80580 (0.0011) +[2023-10-09 15:23:12,979][86121] Updated weights for policy 0, policy_version 80590 (0.0009) +[2023-10-09 15:23:13,350][86121] Updated weights for policy 0, policy_version 80600 (0.0007) +[2023-10-09 15:23:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165380096. Throughput: 0: 1808.1, 1: 1815.9. Samples: 41351402. Policy #0 lag: (min: 31.0, avg: 36.8, max: 63.0) +[2023-10-09 15:23:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:15,776][86122] Updated weights for policy 1, policy_version 80930 (0.0009) +[2023-10-09 15:23:16,134][86122] Updated weights for policy 1, policy_version 80940 (0.0007) +[2023-10-09 15:23:16,501][86122] Updated weights for policy 1, policy_version 80950 (0.0009) +[2023-10-09 15:23:16,868][86122] Updated weights for policy 1, policy_version 80960 (0.0008) +[2023-10-09 15:23:17,071][86121] Updated weights for policy 0, policy_version 80610 (0.0008) +[2023-10-09 15:23:17,436][86121] Updated weights for policy 0, policy_version 80620 (0.0008) +[2023-10-09 15:23:17,806][86121] Updated weights for policy 0, policy_version 80630 (0.0007) +[2023-10-09 15:23:18,165][86121] Updated weights for policy 0, policy_version 80640 (0.0009) +[2023-10-09 15:23:18,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165478400. Throughput: 0: 1815.2, 1: 1821.3. Samples: 41372964. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:20,526][86122] Updated weights for policy 1, policy_version 80970 (0.0008) +[2023-10-09 15:23:20,885][86122] Updated weights for policy 1, policy_version 80980 (0.0008) +[2023-10-09 15:23:21,244][86122] Updated weights for policy 1, policy_version 80990 (0.0008) +[2023-10-09 15:23:21,944][86121] Updated weights for policy 0, policy_version 80650 (0.0009) +[2023-10-09 15:23:22,315][86121] Updated weights for policy 0, policy_version 80660 (0.0007) +[2023-10-09 15:23:22,675][86121] Updated weights for policy 0, policy_version 80670 (0.0007) +[2023-10-09 15:23:23,398][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 165543936. Throughput: 0: 1800.3, 1: 1821.6. Samples: 41394050. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:23,399][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:23,412][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000080672_82608128.pth... +[2023-10-09 15:23:23,412][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000080992_82935808.pth... +[2023-10-09 15:23:23,450][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000078976_80871424.pth +[2023-10-09 15:23:23,450][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000079296_81199104.pth +[2023-10-09 15:23:24,875][86122] Updated weights for policy 1, policy_version 81000 (0.0008) +[2023-10-09 15:23:25,239][86122] Updated weights for policy 1, policy_version 81010 (0.0008) +[2023-10-09 15:23:25,606][86122] Updated weights for policy 1, policy_version 81020 (0.0008) +[2023-10-09 15:23:26,356][86121] Updated weights for policy 0, policy_version 80680 (0.0009) +[2023-10-09 15:23:26,727][86121] Updated weights for policy 0, policy_version 80690 (0.0009) +[2023-10-09 15:23:27,097][86121] Updated weights for policy 0, policy_version 80700 (0.0007) +[2023-10-09 15:23:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 165609472. Throughput: 0: 1815.0, 1: 1815.3. Samples: 41405548. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:29,221][86122] Updated weights for policy 1, policy_version 81030 (0.0008) +[2023-10-09 15:23:29,582][86122] Updated weights for policy 1, policy_version 81040 (0.0008) +[2023-10-09 15:23:29,941][86122] Updated weights for policy 1, policy_version 81050 (0.0009) +[2023-10-09 15:23:30,875][86121] Updated weights for policy 0, policy_version 80710 (0.0009) +[2023-10-09 15:23:31,236][86121] Updated weights for policy 0, policy_version 80720 (0.0008) +[2023-10-09 15:23:31,592][86121] Updated weights for policy 0, policy_version 80730 (0.0009) +[2023-10-09 15:23:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165675008. Throughput: 0: 1802.2, 1: 1822.8. Samples: 41426680. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:33,644][86122] Updated weights for policy 1, policy_version 81060 (0.0009) +[2023-10-09 15:23:34,006][86122] Updated weights for policy 1, policy_version 81070 (0.0008) +[2023-10-09 15:23:34,367][86122] Updated weights for policy 1, policy_version 81080 (0.0008) +[2023-10-09 15:23:35,282][86121] Updated weights for policy 0, policy_version 80740 (0.0007) +[2023-10-09 15:23:35,648][86121] Updated weights for policy 0, policy_version 80750 (0.0007) +[2023-10-09 15:23:36,015][86121] Updated weights for policy 0, policy_version 80760 (0.0011) +[2023-10-09 15:23:38,339][86122] Updated weights for policy 1, policy_version 81090 (0.0008) +[2023-10-09 15:23:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165740544. Throughput: 0: 1800.1, 1: 1819.0. Samples: 41449236. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:23:38,704][86122] Updated weights for policy 1, policy_version 81100 (0.0007) +[2023-10-09 15:23:39,057][86122] Updated weights for policy 1, policy_version 81110 (0.0008) +[2023-10-09 15:23:39,416][86122] Updated weights for policy 1, policy_version 81120 (0.0007) +[2023-10-09 15:23:39,769][86121] Updated weights for policy 0, policy_version 80770 (0.0011) +[2023-10-09 15:23:40,128][86121] Updated weights for policy 0, policy_version 80780 (0.0010) +[2023-10-09 15:23:40,498][86121] Updated weights for policy 0, policy_version 80790 (0.0010) +[2023-10-09 15:23:40,862][86121] Updated weights for policy 0, policy_version 80800 (0.0010) +[2023-10-09 15:23:43,080][86122] Updated weights for policy 1, policy_version 81130 (0.0008) +[2023-10-09 15:23:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 165806080. Throughput: 0: 1800.9, 1: 1818.4. Samples: 41459088. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:23:43,455][86122] Updated weights for policy 1, policy_version 81140 (0.0009) +[2023-10-09 15:23:43,809][86122] Updated weights for policy 1, policy_version 81150 (0.0009) +[2023-10-09 15:23:44,389][86121] Updated weights for policy 0, policy_version 80810 (0.0009) +[2023-10-09 15:23:44,760][86121] Updated weights for policy 0, policy_version 80820 (0.0007) +[2023-10-09 15:23:45,132][86121] Updated weights for policy 0, policy_version 80830 (0.0008) +[2023-10-09 15:23:47,508][86122] Updated weights for policy 1, policy_version 81160 (0.0009) +[2023-10-09 15:23:47,870][86122] Updated weights for policy 1, policy_version 81170 (0.0010) +[2023-10-09 15:23:48,235][86122] Updated weights for policy 1, policy_version 81180 (0.0010) +[2023-10-09 15:23:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 165904384. Throughput: 0: 1814.7, 1: 1816.1. Samples: 41481888. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:23:48,807][86121] Updated weights for policy 0, policy_version 80840 (0.0008) +[2023-10-09 15:23:49,164][86121] Updated weights for policy 0, policy_version 80850 (0.0009) +[2023-10-09 15:23:49,531][86121] Updated weights for policy 0, policy_version 80860 (0.0008) +[2023-10-09 15:23:52,061][86122] Updated weights for policy 1, policy_version 81190 (0.0009) +[2023-10-09 15:23:52,431][86122] Updated weights for policy 1, policy_version 81200 (0.0008) +[2023-10-09 15:23:52,785][86122] Updated weights for policy 1, policy_version 81210 (0.0009) +[2023-10-09 15:23:53,293][86121] Updated weights for policy 0, policy_version 80870 (0.0010) +[2023-10-09 15:23:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 165969920. Throughput: 0: 1812.7, 1: 1816.4. Samples: 41503434. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:23:53,652][86121] Updated weights for policy 0, policy_version 80880 (0.0009) +[2023-10-09 15:23:54,015][86121] Updated weights for policy 0, policy_version 80890 (0.0010) +[2023-10-09 15:23:56,235][86122] Updated weights for policy 1, policy_version 81220 (0.0009) +[2023-10-09 15:23:56,603][86122] Updated weights for policy 1, policy_version 81230 (0.0010) +[2023-10-09 15:23:56,975][86122] Updated weights for policy 1, policy_version 81240 (0.0008) +[2023-10-09 15:23:57,790][86121] Updated weights for policy 0, policy_version 80900 (0.0011) +[2023-10-09 15:23:58,156][86121] Updated weights for policy 0, policy_version 80910 (0.0008) +[2023-10-09 15:23:58,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166035456. Throughput: 0: 1810.2, 1: 1814.4. Samples: 41514510. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:23:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:23:58,534][86121] Updated weights for policy 0, policy_version 80920 (0.0008) +[2023-10-09 15:24:00,778][86122] Updated weights for policy 1, policy_version 81250 (0.0008) +[2023-10-09 15:24:01,139][86122] Updated weights for policy 1, policy_version 81260 (0.0010) +[2023-10-09 15:24:01,498][86122] Updated weights for policy 1, policy_version 81270 (0.0010) +[2023-10-09 15:24:01,861][86122] Updated weights for policy 1, policy_version 81280 (0.0010) +[2023-10-09 15:24:02,265][86121] Updated weights for policy 0, policy_version 80930 (0.0009) +[2023-10-09 15:24:02,626][86121] Updated weights for policy 0, policy_version 80940 (0.0008) +[2023-10-09 15:24:03,007][86121] Updated weights for policy 0, policy_version 80950 (0.0009) +[2023-10-09 15:24:03,373][86121] Updated weights for policy 0, policy_version 80960 (0.0009) +[2023-10-09 15:24:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 166133760. Throughput: 0: 1810.6, 1: 1817.3. Samples: 41536218. Policy #0 lag: (min: 31.0, avg: 31.5, max: 46.0) +[2023-10-09 15:24:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:24:05,405][86122] Updated weights for policy 1, policy_version 81290 (0.0008) +[2023-10-09 15:24:05,765][86122] Updated weights for policy 1, policy_version 81300 (0.0010) +[2023-10-09 15:24:06,136][86122] Updated weights for policy 1, policy_version 81310 (0.0009) +[2023-10-09 15:24:07,093][86121] Updated weights for policy 0, policy_version 80970 (0.0007) +[2023-10-09 15:24:07,461][86121] Updated weights for policy 0, policy_version 80980 (0.0009) +[2023-10-09 15:24:07,841][86121] Updated weights for policy 0, policy_version 80990 (0.0009) +[2023-10-09 15:24:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 166199296. Throughput: 0: 1813.6, 1: 1818.5. Samples: 41557492. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:09,945][86122] Updated weights for policy 1, policy_version 81320 (0.0009) +[2023-10-09 15:24:10,302][86122] Updated weights for policy 1, policy_version 81330 (0.0011) +[2023-10-09 15:24:10,664][86122] Updated weights for policy 1, policy_version 81340 (0.0009) +[2023-10-09 15:24:11,692][86121] Updated weights for policy 0, policy_version 81000 (0.0010) +[2023-10-09 15:24:12,063][86121] Updated weights for policy 0, policy_version 81010 (0.0007) +[2023-10-09 15:24:12,437][86121] Updated weights for policy 0, policy_version 81020 (0.0009) +[2023-10-09 15:24:13,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166264832. Throughput: 0: 1802.5, 1: 1817.6. Samples: 41568452. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:14,397][86122] Updated weights for policy 1, policy_version 81350 (0.0008) +[2023-10-09 15:24:14,753][86122] Updated weights for policy 1, policy_version 81360 (0.0009) +[2023-10-09 15:24:15,121][86122] Updated weights for policy 1, policy_version 81370 (0.0009) +[2023-10-09 15:24:16,178][86121] Updated weights for policy 0, policy_version 81030 (0.0011) +[2023-10-09 15:24:16,548][86121] Updated weights for policy 0, policy_version 81040 (0.0008) +[2023-10-09 15:24:16,911][86121] Updated weights for policy 0, policy_version 81050 (0.0008) +[2023-10-09 15:24:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 166330368. Throughput: 0: 1818.5, 1: 1810.5. Samples: 41589986. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:18,763][86122] Updated weights for policy 1, policy_version 81380 (0.0010) +[2023-10-09 15:24:19,131][86122] Updated weights for policy 1, policy_version 81390 (0.0008) +[2023-10-09 15:24:19,485][86122] Updated weights for policy 1, policy_version 81400 (0.0007) +[2023-10-09 15:24:20,451][86121] Updated weights for policy 0, policy_version 81060 (0.0008) +[2023-10-09 15:24:20,819][86121] Updated weights for policy 0, policy_version 81070 (0.0009) +[2023-10-09 15:24:21,184][86121] Updated weights for policy 0, policy_version 81080 (0.0008) +[2023-10-09 15:24:23,165][86122] Updated weights for policy 1, policy_version 81410 (0.0008) +[2023-10-09 15:24:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 166395904. Throughput: 0: 1812.0, 1: 1820.8. Samples: 41612712. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:23,532][86122] Updated weights for policy 1, policy_version 81420 (0.0007) +[2023-10-09 15:24:23,904][86122] Updated weights for policy 1, policy_version 81430 (0.0009) +[2023-10-09 15:24:24,260][86122] Updated weights for policy 1, policy_version 81440 (0.0009) +[2023-10-09 15:24:24,903][86121] Updated weights for policy 0, policy_version 81090 (0.0008) +[2023-10-09 15:24:25,270][86121] Updated weights for policy 0, policy_version 81100 (0.0007) +[2023-10-09 15:24:25,636][86121] Updated weights for policy 0, policy_version 81110 (0.0008) +[2023-10-09 15:24:25,991][86121] Updated weights for policy 0, policy_version 81120 (0.0008) +[2023-10-09 15:24:27,949][86122] Updated weights for policy 1, policy_version 81450 (0.0008) +[2023-10-09 15:24:28,310][86122] Updated weights for policy 1, policy_version 81460 (0.0009) +[2023-10-09 15:24:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 166461440. Throughput: 0: 1819.2, 1: 1824.0. Samples: 41623032. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:28,675][86122] Updated weights for policy 1, policy_version 81470 (0.0007) +[2023-10-09 15:24:29,682][86121] Updated weights for policy 0, policy_version 81130 (0.0008) +[2023-10-09 15:24:30,054][86121] Updated weights for policy 0, policy_version 81140 (0.0011) +[2023-10-09 15:24:30,427][86121] Updated weights for policy 0, policy_version 81150 (0.0008) +[2023-10-09 15:24:32,505][86122] Updated weights for policy 1, policy_version 81480 (0.0009) +[2023-10-09 15:24:32,877][86122] Updated weights for policy 1, policy_version 81490 (0.0007) +[2023-10-09 15:24:33,237][86122] Updated weights for policy 1, policy_version 81500 (0.0010) +[2023-10-09 15:24:33,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 166559744. Throughput: 0: 1811.0, 1: 1820.4. Samples: 41645304. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:34,077][86121] Updated weights for policy 0, policy_version 81160 (0.0007) +[2023-10-09 15:24:34,449][86121] Updated weights for policy 0, policy_version 81170 (0.0007) +[2023-10-09 15:24:34,814][86121] Updated weights for policy 0, policy_version 81180 (0.0009) +[2023-10-09 15:24:36,935][86122] Updated weights for policy 1, policy_version 81510 (0.0009) +[2023-10-09 15:24:37,287][86122] Updated weights for policy 1, policy_version 81520 (0.0009) +[2023-10-09 15:24:37,654][86122] Updated weights for policy 1, policy_version 81530 (0.0007) +[2023-10-09 15:24:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166625280. Throughput: 0: 1810.8, 1: 1818.7. Samples: 41666758. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:38,620][86121] Updated weights for policy 0, policy_version 81190 (0.0007) +[2023-10-09 15:24:38,988][86121] Updated weights for policy 0, policy_version 81200 (0.0010) +[2023-10-09 15:24:39,359][86121] Updated weights for policy 0, policy_version 81210 (0.0010) +[2023-10-09 15:24:41,325][86122] Updated weights for policy 1, policy_version 81540 (0.0007) +[2023-10-09 15:24:41,683][86122] Updated weights for policy 1, policy_version 81550 (0.0008) +[2023-10-09 15:24:42,044][86122] Updated weights for policy 1, policy_version 81560 (0.0007) +[2023-10-09 15:24:43,100][86121] Updated weights for policy 0, policy_version 81220 (0.0010) +[2023-10-09 15:24:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 166690816. Throughput: 0: 1810.7, 1: 1820.8. Samples: 41677926. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:43,465][86121] Updated weights for policy 0, policy_version 81230 (0.0009) +[2023-10-09 15:24:43,825][86121] Updated weights for policy 0, policy_version 81240 (0.0010) +[2023-10-09 15:24:45,622][86122] Updated weights for policy 1, policy_version 81570 (0.0008) +[2023-10-09 15:24:45,989][86122] Updated weights for policy 1, policy_version 81580 (0.0009) +[2023-10-09 15:24:46,353][86122] Updated weights for policy 1, policy_version 81590 (0.0010) +[2023-10-09 15:24:46,716][86122] Updated weights for policy 1, policy_version 81600 (0.0007) +[2023-10-09 15:24:47,512][86121] Updated weights for policy 0, policy_version 81250 (0.0007) +[2023-10-09 15:24:47,876][86121] Updated weights for policy 0, policy_version 81260 (0.0009) +[2023-10-09 15:24:48,252][86121] Updated weights for policy 0, policy_version 81270 (0.0008) +[2023-10-09 15:24:48,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 166756352. Throughput: 0: 1802.9, 1: 1823.0. Samples: 41699384. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:48,615][86121] Updated weights for policy 0, policy_version 81280 (0.0008) +[2023-10-09 15:24:50,451][86122] Updated weights for policy 1, policy_version 81610 (0.0008) +[2023-10-09 15:24:50,821][86122] Updated weights for policy 1, policy_version 81620 (0.0007) +[2023-10-09 15:24:51,176][86122] Updated weights for policy 1, policy_version 81630 (0.0007) +[2023-10-09 15:24:52,354][86121] Updated weights for policy 0, policy_version 81290 (0.0010) +[2023-10-09 15:24:52,722][86121] Updated weights for policy 0, policy_version 81300 (0.0007) +[2023-10-09 15:24:53,098][86121] Updated weights for policy 0, policy_version 81310 (0.0008) +[2023-10-09 15:24:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 166854656. Throughput: 0: 1812.5, 1: 1823.0. Samples: 41721092. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:24:54,827][86122] Updated weights for policy 1, policy_version 81640 (0.0010) +[2023-10-09 15:24:55,188][86122] Updated weights for policy 1, policy_version 81650 (0.0009) +[2023-10-09 15:24:55,552][86122] Updated weights for policy 1, policy_version 81660 (0.0008) +[2023-10-09 15:24:56,646][86121] Updated weights for policy 0, policy_version 81320 (0.0010) +[2023-10-09 15:24:57,010][86121] Updated weights for policy 0, policy_version 81330 (0.0008) +[2023-10-09 15:24:57,372][86121] Updated weights for policy 0, policy_version 81340 (0.0008) +[2023-10-09 15:24:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 166920192. Throughput: 0: 1816.5, 1: 1826.5. Samples: 41732386. Policy #0 lag: (min: 20.0, avg: 20.0, max: 20.0) +[2023-10-09 15:24:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:24:59,276][86122] Updated weights for policy 1, policy_version 81670 (0.0008) +[2023-10-09 15:24:59,632][86122] Updated weights for policy 1, policy_version 81680 (0.0007) +[2023-10-09 15:24:59,995][86122] Updated weights for policy 1, policy_version 81690 (0.0009) +[2023-10-09 15:25:01,020][86121] Updated weights for policy 0, policy_version 81350 (0.0008) +[2023-10-09 15:25:01,396][86121] Updated weights for policy 0, policy_version 81360 (0.0008) +[2023-10-09 15:25:01,762][86121] Updated weights for policy 0, policy_version 81370 (0.0008) +[2023-10-09 15:25:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 166985728. Throughput: 0: 1812.9, 1: 1830.7. Samples: 41753946. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:25:03,641][86122] Updated weights for policy 1, policy_version 81700 (0.0009) +[2023-10-09 15:25:03,998][86122] Updated weights for policy 1, policy_version 81710 (0.0009) +[2023-10-09 15:25:04,357][86122] Updated weights for policy 1, policy_version 81720 (0.0008) +[2023-10-09 15:25:05,396][86121] Updated weights for policy 0, policy_version 81380 (0.0009) +[2023-10-09 15:25:05,757][86121] Updated weights for policy 0, policy_version 81390 (0.0009) +[2023-10-09 15:25:06,128][86121] Updated weights for policy 0, policy_version 81400 (0.0010) +[2023-10-09 15:25:08,088][86122] Updated weights for policy 1, policy_version 81730 (0.0007) +[2023-10-09 15:25:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167051264. Throughput: 0: 1821.5, 1: 1825.7. Samples: 41776832. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:25:08,441][86122] Updated weights for policy 1, policy_version 81740 (0.0008) +[2023-10-09 15:25:08,811][86122] Updated weights for policy 1, policy_version 81750 (0.0008) +[2023-10-09 15:25:09,175][86122] Updated weights for policy 1, policy_version 81760 (0.0008) +[2023-10-09 15:25:09,823][86121] Updated weights for policy 0, policy_version 81410 (0.0007) +[2023-10-09 15:25:10,192][86121] Updated weights for policy 0, policy_version 81420 (0.0008) +[2023-10-09 15:25:10,561][86121] Updated weights for policy 0, policy_version 81430 (0.0011) +[2023-10-09 15:25:10,917][86121] Updated weights for policy 0, policy_version 81440 (0.0010) +[2023-10-09 15:25:13,001][86122] Updated weights for policy 1, policy_version 81770 (0.0007) +[2023-10-09 15:25:13,363][86122] Updated weights for policy 1, policy_version 81780 (0.0009) +[2023-10-09 15:25:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167116800. Throughput: 0: 1820.2, 1: 1820.1. Samples: 41786846. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:25:13,735][86122] Updated weights for policy 1, policy_version 81790 (0.0010) +[2023-10-09 15:25:14,683][86121] Updated weights for policy 0, policy_version 81450 (0.0007) +[2023-10-09 15:25:15,055][86121] Updated weights for policy 0, policy_version 81460 (0.0007) +[2023-10-09 15:25:15,415][86121] Updated weights for policy 0, policy_version 81470 (0.0008) +[2023-10-09 15:25:17,316][86122] Updated weights for policy 1, policy_version 81800 (0.0008) +[2023-10-09 15:25:17,682][86122] Updated weights for policy 1, policy_version 81810 (0.0007) +[2023-10-09 15:25:18,040][86122] Updated weights for policy 1, policy_version 81820 (0.0008) +[2023-10-09 15:25:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167215104. Throughput: 0: 1823.0, 1: 1829.5. Samples: 41809666. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:25:19,053][86121] Updated weights for policy 0, policy_version 81480 (0.0010) +[2023-10-09 15:25:19,432][86121] Updated weights for policy 0, policy_version 81490 (0.0011) +[2023-10-09 15:25:19,797][86121] Updated weights for policy 0, policy_version 81500 (0.0008) +[2023-10-09 15:25:21,768][86122] Updated weights for policy 1, policy_version 81830 (0.0009) +[2023-10-09 15:25:22,129][86122] Updated weights for policy 1, policy_version 81840 (0.0008) +[2023-10-09 15:25:22,490][86122] Updated weights for policy 1, policy_version 81850 (0.0008) +[2023-10-09 15:25:23,198][86121] Updated weights for policy 0, policy_version 81510 (0.0008) +[2023-10-09 15:25:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167280640. Throughput: 0: 1828.4, 1: 1823.4. Samples: 41831086. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:25:23,405][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000081856_83820544.pth... +[2023-10-09 15:25:23,443][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000080160_82083840.pth +[2023-10-09 15:25:23,567][86121] Updated weights for policy 0, policy_version 81520 (0.0007) +[2023-10-09 15:25:23,931][86121] Updated weights for policy 0, policy_version 81530 (0.0010) +[2023-10-09 15:25:24,147][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000081536_83492864.pth... +[2023-10-09 15:25:24,186][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000079808_81723392.pth +[2023-10-09 15:25:26,324][86122] Updated weights for policy 1, policy_version 81860 (0.0009) +[2023-10-09 15:25:26,684][86122] Updated weights for policy 1, policy_version 81870 (0.0010) +[2023-10-09 15:25:27,045][86122] Updated weights for policy 1, policy_version 81880 (0.0011) +[2023-10-09 15:25:27,675][86121] Updated weights for policy 0, policy_version 81540 (0.0009) +[2023-10-09 15:25:28,042][86121] Updated weights for policy 0, policy_version 81550 (0.0009) +[2023-10-09 15:25:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167346176. Throughput: 0: 1834.6, 1: 1822.0. Samples: 41842474. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:28,398][86121] Updated weights for policy 0, policy_version 81560 (0.0011) +[2023-10-09 15:25:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:25:30,720][86122] Updated weights for policy 1, policy_version 81890 (0.0008) +[2023-10-09 15:25:31,078][86122] Updated weights for policy 1, policy_version 81900 (0.0009) +[2023-10-09 15:25:31,436][86122] Updated weights for policy 1, policy_version 81910 (0.0010) +[2023-10-09 15:25:31,792][86122] Updated weights for policy 1, policy_version 81920 (0.0008) +[2023-10-09 15:25:32,227][86121] Updated weights for policy 0, policy_version 81570 (0.0009) +[2023-10-09 15:25:32,590][86121] Updated weights for policy 0, policy_version 81580 (0.0008) +[2023-10-09 15:25:32,960][86121] Updated weights for policy 0, policy_version 81590 (0.0011) +[2023-10-09 15:25:33,319][86121] Updated weights for policy 0, policy_version 81600 (0.0010) +[2023-10-09 15:25:33,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 167444480. Throughput: 0: 1839.5, 1: 1817.6. Samples: 41863954. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:25:35,390][86122] Updated weights for policy 1, policy_version 81930 (0.0011) +[2023-10-09 15:25:35,749][86122] Updated weights for policy 1, policy_version 81940 (0.0010) +[2023-10-09 15:25:36,107][86122] Updated weights for policy 1, policy_version 81950 (0.0007) +[2023-10-09 15:25:37,106][86121] Updated weights for policy 0, policy_version 81610 (0.0009) +[2023-10-09 15:25:37,469][86121] Updated weights for policy 0, policy_version 81620 (0.0007) +[2023-10-09 15:25:37,824][86121] Updated weights for policy 0, policy_version 81630 (0.0008) +[2023-10-09 15:25:38,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 167510016. Throughput: 0: 1828.7, 1: 1820.7. Samples: 41885316. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:25:39,922][86122] Updated weights for policy 1, policy_version 81960 (0.0008) +[2023-10-09 15:25:40,289][86122] Updated weights for policy 1, policy_version 81970 (0.0009) +[2023-10-09 15:25:40,665][86122] Updated weights for policy 1, policy_version 81980 (0.0008) +[2023-10-09 15:25:41,484][86121] Updated weights for policy 0, policy_version 81640 (0.0008) +[2023-10-09 15:25:41,864][86121] Updated weights for policy 0, policy_version 81650 (0.0009) +[2023-10-09 15:25:42,230][86121] Updated weights for policy 0, policy_version 81660 (0.0010) +[2023-10-09 15:25:43,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 167575552. Throughput: 0: 1831.9, 1: 1818.9. Samples: 41896672. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:25:44,457][86122] Updated weights for policy 1, policy_version 81990 (0.0010) +[2023-10-09 15:25:44,824][86122] Updated weights for policy 1, policy_version 82000 (0.0010) +[2023-10-09 15:25:45,186][86122] Updated weights for policy 1, policy_version 82010 (0.0008) +[2023-10-09 15:25:45,868][86121] Updated weights for policy 0, policy_version 81670 (0.0009) +[2023-10-09 15:25:46,226][86121] Updated weights for policy 0, policy_version 81680 (0.0009) +[2023-10-09 15:25:46,601][86121] Updated weights for policy 0, policy_version 81690 (0.0009) +[2023-10-09 15:25:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 167641088. Throughput: 0: 1825.7, 1: 1817.6. Samples: 41917892. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:25:48,821][86122] Updated weights for policy 1, policy_version 82020 (0.0011) +[2023-10-09 15:25:49,187][86122] Updated weights for policy 1, policy_version 82030 (0.0009) +[2023-10-09 15:25:49,550][86122] Updated weights for policy 1, policy_version 82040 (0.0008) +[2023-10-09 15:25:50,366][86121] Updated weights for policy 0, policy_version 81700 (0.0007) +[2023-10-09 15:25:50,729][86121] Updated weights for policy 0, policy_version 81710 (0.0008) +[2023-10-09 15:25:51,089][86121] Updated weights for policy 0, policy_version 81720 (0.0007) +[2023-10-09 15:25:53,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 167706624. Throughput: 0: 1825.4, 1: 1812.7. Samples: 41940544. Policy #0 lag: (min: 31.0, avg: 40.7, max: 63.0) +[2023-10-09 15:25:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:25:53,484][86122] Updated weights for policy 1, policy_version 82050 (0.0007) +[2023-10-09 15:25:53,849][86122] Updated weights for policy 1, policy_version 82060 (0.0010) +[2023-10-09 15:25:54,207][86122] Updated weights for policy 1, policy_version 82070 (0.0007) +[2023-10-09 15:25:54,580][86122] Updated weights for policy 1, policy_version 82080 (0.0008) +[2023-10-09 15:25:54,683][86121] Updated weights for policy 0, policy_version 81730 (0.0009) +[2023-10-09 15:25:55,047][86121] Updated weights for policy 0, policy_version 81740 (0.0008) +[2023-10-09 15:25:55,418][86121] Updated weights for policy 0, policy_version 81750 (0.0007) +[2023-10-09 15:25:55,788][86121] Updated weights for policy 0, policy_version 81760 (0.0010) +[2023-10-09 15:25:58,296][86122] Updated weights for policy 1, policy_version 82090 (0.0007) +[2023-10-09 15:25:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167772160. Throughput: 0: 1825.0, 1: 1813.7. Samples: 41950588. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:25:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:25:58,659][86122] Updated weights for policy 1, policy_version 82100 (0.0008) +[2023-10-09 15:25:59,029][86122] Updated weights for policy 1, policy_version 82110 (0.0008) +[2023-10-09 15:25:59,593][86121] Updated weights for policy 0, policy_version 81770 (0.0010) +[2023-10-09 15:25:59,967][86121] Updated weights for policy 0, policy_version 81780 (0.0010) +[2023-10-09 15:26:00,338][86121] Updated weights for policy 0, policy_version 81790 (0.0009) +[2023-10-09 15:26:02,584][86122] Updated weights for policy 1, policy_version 82120 (0.0009) +[2023-10-09 15:26:02,946][86122] Updated weights for policy 1, policy_version 82130 (0.0007) +[2023-10-09 15:26:03,301][86122] Updated weights for policy 1, policy_version 82140 (0.0008) +[2023-10-09 15:26:03,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 167837696. Throughput: 0: 1822.6, 1: 1810.1. Samples: 41973140. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:26:04,130][86121] Updated weights for policy 0, policy_version 81800 (0.0011) +[2023-10-09 15:26:04,494][86121] Updated weights for policy 0, policy_version 81810 (0.0009) +[2023-10-09 15:26:04,872][86121] Updated weights for policy 0, policy_version 81820 (0.0011) +[2023-10-09 15:26:06,904][86122] Updated weights for policy 1, policy_version 82150 (0.0007) +[2023-10-09 15:26:07,271][86122] Updated weights for policy 1, policy_version 82160 (0.0007) +[2023-10-09 15:26:07,625][86122] Updated weights for policy 1, policy_version 82170 (0.0007) +[2023-10-09 15:26:08,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 167936000. Throughput: 0: 1818.9, 1: 1817.2. Samples: 41994706. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:26:08,595][86121] Updated weights for policy 0, policy_version 81830 (0.0010) +[2023-10-09 15:26:08,954][86121] Updated weights for policy 0, policy_version 81840 (0.0009) +[2023-10-09 15:26:09,325][86121] Updated weights for policy 0, policy_version 81850 (0.0007) +[2023-10-09 15:26:11,313][86122] Updated weights for policy 1, policy_version 82180 (0.0007) +[2023-10-09 15:26:11,668][86122] Updated weights for policy 1, policy_version 82190 (0.0008) +[2023-10-09 15:26:12,030][86122] Updated weights for policy 1, policy_version 82200 (0.0009) +[2023-10-09 15:26:12,888][86121] Updated weights for policy 0, policy_version 81860 (0.0007) +[2023-10-09 15:26:13,260][86121] Updated weights for policy 0, policy_version 81870 (0.0007) +[2023-10-09 15:26:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168001536. Throughput: 0: 1818.3, 1: 1819.3. Samples: 42006164. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:26:13,626][86121] Updated weights for policy 0, policy_version 81880 (0.0007) +[2023-10-09 15:26:15,760][86122] Updated weights for policy 1, policy_version 82210 (0.0010) +[2023-10-09 15:26:16,130][86122] Updated weights for policy 1, policy_version 82220 (0.0008) +[2023-10-09 15:26:16,493][86122] Updated weights for policy 1, policy_version 82230 (0.0009) +[2023-10-09 15:26:16,860][86122] Updated weights for policy 1, policy_version 82240 (0.0008) +[2023-10-09 15:26:17,414][86121] Updated weights for policy 0, policy_version 81890 (0.0007) +[2023-10-09 15:26:17,788][86121] Updated weights for policy 0, policy_version 81900 (0.0007) +[2023-10-09 15:26:18,157][86121] Updated weights for policy 0, policy_version 81910 (0.0007) +[2023-10-09 15:26:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 168067072. Throughput: 0: 1815.4, 1: 1823.6. Samples: 42027712. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:26:18,516][86121] Updated weights for policy 0, policy_version 81920 (0.0007) +[2023-10-09 15:26:20,514][86122] Updated weights for policy 1, policy_version 82250 (0.0008) +[2023-10-09 15:26:20,885][86122] Updated weights for policy 1, policy_version 82260 (0.0009) +[2023-10-09 15:26:21,246][86122] Updated weights for policy 1, policy_version 82270 (0.0009) +[2023-10-09 15:26:21,987][86121] Updated weights for policy 0, policy_version 81930 (0.0008) +[2023-10-09 15:26:22,351][86121] Updated weights for policy 0, policy_version 81940 (0.0008) +[2023-10-09 15:26:22,720][86121] Updated weights for policy 0, policy_version 81950 (0.0008) +[2023-10-09 15:26:23,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 168165376. Throughput: 0: 1825.5, 1: 1818.4. Samples: 42049290. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:26:24,998][86122] Updated weights for policy 1, policy_version 82280 (0.0009) +[2023-10-09 15:26:25,369][86122] Updated weights for policy 1, policy_version 82290 (0.0010) +[2023-10-09 15:26:25,728][86122] Updated weights for policy 1, policy_version 82300 (0.0009) +[2023-10-09 15:26:26,363][86121] Updated weights for policy 0, policy_version 81960 (0.0008) +[2023-10-09 15:26:26,743][86121] Updated weights for policy 0, policy_version 81970 (0.0011) +[2023-10-09 15:26:27,111][86121] Updated weights for policy 0, policy_version 81980 (0.0008) +[2023-10-09 15:26:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168230912. Throughput: 0: 1828.3, 1: 1824.2. Samples: 42061034. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:26:29,274][86122] Updated weights for policy 1, policy_version 82310 (0.0007) +[2023-10-09 15:26:29,635][86122] Updated weights for policy 1, policy_version 82320 (0.0008) +[2023-10-09 15:26:29,995][86122] Updated weights for policy 1, policy_version 82330 (0.0008) +[2023-10-09 15:26:30,727][86121] Updated weights for policy 0, policy_version 81990 (0.0010) +[2023-10-09 15:26:31,091][86121] Updated weights for policy 0, policy_version 82000 (0.0010) +[2023-10-09 15:26:31,459][86121] Updated weights for policy 0, policy_version 82010 (0.0011) +[2023-10-09 15:26:33,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 168296448. Throughput: 0: 1827.6, 1: 1827.5. Samples: 42082368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:26:33,677][86122] Updated weights for policy 1, policy_version 82340 (0.0008) +[2023-10-09 15:26:34,045][86122] Updated weights for policy 1, policy_version 82350 (0.0009) +[2023-10-09 15:26:34,414][86122] Updated weights for policy 1, policy_version 82360 (0.0012) +[2023-10-09 15:26:35,117][86121] Updated weights for policy 0, policy_version 82020 (0.0010) +[2023-10-09 15:26:35,490][86121] Updated weights for policy 0, policy_version 82030 (0.0009) +[2023-10-09 15:26:35,851][86121] Updated weights for policy 0, policy_version 82040 (0.0009) +[2023-10-09 15:26:38,083][86122] Updated weights for policy 1, policy_version 82370 (0.0011) +[2023-10-09 15:26:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 168361984. Throughput: 0: 1830.9, 1: 1831.2. Samples: 42105336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:26:38,442][86122] Updated weights for policy 1, policy_version 82380 (0.0011) +[2023-10-09 15:26:38,809][86122] Updated weights for policy 1, policy_version 82390 (0.0008) +[2023-10-09 15:26:39,163][86122] Updated weights for policy 1, policy_version 82400 (0.0010) +[2023-10-09 15:26:39,469][86121] Updated weights for policy 0, policy_version 82050 (0.0009) +[2023-10-09 15:26:39,837][86121] Updated weights for policy 0, policy_version 82060 (0.0009) +[2023-10-09 15:26:40,207][86121] Updated weights for policy 0, policy_version 82070 (0.0010) +[2023-10-09 15:26:40,572][86121] Updated weights for policy 0, policy_version 82080 (0.0010) +[2023-10-09 15:26:42,748][86122] Updated weights for policy 1, policy_version 82410 (0.0007) +[2023-10-09 15:26:43,104][86122] Updated weights for policy 1, policy_version 82420 (0.0008) +[2023-10-09 15:26:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 168427520. Throughput: 0: 1825.0, 1: 1836.8. Samples: 42115368. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:26:43,468][86122] Updated weights for policy 1, policy_version 82430 (0.0007) +[2023-10-09 15:26:44,266][86121] Updated weights for policy 0, policy_version 82090 (0.0009) +[2023-10-09 15:26:44,641][86121] Updated weights for policy 0, policy_version 82100 (0.0009) +[2023-10-09 15:26:45,012][86121] Updated weights for policy 0, policy_version 82110 (0.0009) +[2023-10-09 15:26:47,037][86122] Updated weights for policy 1, policy_version 82440 (0.0007) +[2023-10-09 15:26:47,409][86122] Updated weights for policy 1, policy_version 82450 (0.0007) +[2023-10-09 15:26:47,765][86122] Updated weights for policy 1, policy_version 82460 (0.0008) +[2023-10-09 15:26:48,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168525824. Throughput: 0: 1837.5, 1: 1837.6. Samples: 42138516. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:26:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:26:48,786][86121] Updated weights for policy 0, policy_version 82120 (0.0008) +[2023-10-09 15:26:49,145][86121] Updated weights for policy 0, policy_version 82130 (0.0008) +[2023-10-09 15:26:49,513][86121] Updated weights for policy 0, policy_version 82140 (0.0009) +[2023-10-09 15:26:51,514][86122] Updated weights for policy 1, policy_version 82470 (0.0008) +[2023-10-09 15:26:51,877][86122] Updated weights for policy 1, policy_version 82480 (0.0011) +[2023-10-09 15:26:52,237][86122] Updated weights for policy 1, policy_version 82490 (0.0008) +[2023-10-09 15:26:53,144][86121] Updated weights for policy 0, policy_version 82150 (0.0010) +[2023-10-09 15:26:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 168591360. Throughput: 0: 1838.9, 1: 1835.6. Samples: 42160060. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:26:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:26:53,510][86121] Updated weights for policy 0, policy_version 82160 (0.0009) +[2023-10-09 15:26:53,872][86121] Updated weights for policy 0, policy_version 82170 (0.0009) +[2023-10-09 15:26:55,762][86122] Updated weights for policy 1, policy_version 82500 (0.0009) +[2023-10-09 15:26:56,133][86122] Updated weights for policy 1, policy_version 82510 (0.0008) +[2023-10-09 15:26:56,487][86122] Updated weights for policy 1, policy_version 82520 (0.0008) +[2023-10-09 15:26:57,513][86121] Updated weights for policy 0, policy_version 82180 (0.0008) +[2023-10-09 15:26:57,889][86121] Updated weights for policy 0, policy_version 82190 (0.0009) +[2023-10-09 15:26:58,261][86121] Updated weights for policy 0, policy_version 82200 (0.0011) +[2023-10-09 15:26:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168656896. Throughput: 0: 1838.9, 1: 1834.8. Samples: 42171478. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:26:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:27:00,315][86122] Updated weights for policy 1, policy_version 82530 (0.0008) +[2023-10-09 15:27:00,679][86122] Updated weights for policy 1, policy_version 82540 (0.0009) +[2023-10-09 15:27:01,043][86122] Updated weights for policy 1, policy_version 82550 (0.0008) +[2023-10-09 15:27:01,402][86122] Updated weights for policy 1, policy_version 82560 (0.0008) +[2023-10-09 15:27:01,822][86121] Updated weights for policy 0, policy_version 82210 (0.0008) +[2023-10-09 15:27:02,179][86121] Updated weights for policy 0, policy_version 82220 (0.0008) +[2023-10-09 15:27:02,548][86121] Updated weights for policy 0, policy_version 82230 (0.0010) +[2023-10-09 15:27:02,917][86121] Updated weights for policy 0, policy_version 82240 (0.0010) +[2023-10-09 15:27:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 168755200. Throughput: 0: 1839.4, 1: 1828.1. Samples: 42192748. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:27:04,919][86122] Updated weights for policy 1, policy_version 82570 (0.0008) +[2023-10-09 15:27:05,279][86122] Updated weights for policy 1, policy_version 82580 (0.0011) +[2023-10-09 15:27:05,641][86122] Updated weights for policy 1, policy_version 82590 (0.0009) +[2023-10-09 15:27:06,629][86121] Updated weights for policy 0, policy_version 82250 (0.0007) +[2023-10-09 15:27:06,993][86121] Updated weights for policy 0, policy_version 82260 (0.0010) +[2023-10-09 15:27:07,362][86121] Updated weights for policy 0, policy_version 82270 (0.0008) +[2023-10-09 15:27:08,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 168820736. Throughput: 0: 1834.6, 1: 1840.5. Samples: 42214670. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:27:09,342][86122] Updated weights for policy 1, policy_version 82600 (0.0008) +[2023-10-09 15:27:09,707][86122] Updated weights for policy 1, policy_version 82610 (0.0010) +[2023-10-09 15:27:10,071][86122] Updated weights for policy 1, policy_version 82620 (0.0008) +[2023-10-09 15:27:11,051][86121] Updated weights for policy 0, policy_version 82280 (0.0009) +[2023-10-09 15:27:11,419][86121] Updated weights for policy 0, policy_version 82290 (0.0008) +[2023-10-09 15:27:11,784][86121] Updated weights for policy 0, policy_version 82300 (0.0007) +[2023-10-09 15:27:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 168886272. Throughput: 0: 1828.0, 1: 1838.7. Samples: 42226038. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:27:13,647][86122] Updated weights for policy 1, policy_version 82630 (0.0008) +[2023-10-09 15:27:14,018][86122] Updated weights for policy 1, policy_version 82640 (0.0007) +[2023-10-09 15:27:14,372][86122] Updated weights for policy 1, policy_version 82650 (0.0007) +[2023-10-09 15:27:15,266][86121] Updated weights for policy 0, policy_version 82310 (0.0007) +[2023-10-09 15:27:15,634][86121] Updated weights for policy 0, policy_version 82320 (0.0008) +[2023-10-09 15:27:16,002][86121] Updated weights for policy 0, policy_version 82330 (0.0008) +[2023-10-09 15:27:18,171][86122] Updated weights for policy 1, policy_version 82660 (0.0009) +[2023-10-09 15:27:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 168951808. Throughput: 0: 1838.5, 1: 1837.8. Samples: 42247802. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:27:18,533][86122] Updated weights for policy 1, policy_version 82670 (0.0007) +[2023-10-09 15:27:18,896][86122] Updated weights for policy 1, policy_version 82680 (0.0008) +[2023-10-09 15:27:19,692][86121] Updated weights for policy 0, policy_version 82340 (0.0007) +[2023-10-09 15:27:20,068][86121] Updated weights for policy 0, policy_version 82350 (0.0007) +[2023-10-09 15:27:20,427][86121] Updated weights for policy 0, policy_version 82360 (0.0010) +[2023-10-09 15:27:22,632][86122] Updated weights for policy 1, policy_version 82690 (0.0011) +[2023-10-09 15:27:22,984][86122] Updated weights for policy 1, policy_version 82700 (0.0008) +[2023-10-09 15:27:23,344][86122] Updated weights for policy 1, policy_version 82710 (0.0007) +[2023-10-09 15:27:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 169017344. Throughput: 0: 1836.0, 1: 1823.9. Samples: 42270032. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:27:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000082368_84344832.pth... +[2023-10-09 15:27:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000080672_82608128.pth +[2023-10-09 15:27:23,704][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000082720_84705280.pth... +[2023-10-09 15:27:23,704][86122] Updated weights for policy 1, policy_version 82720 (0.0008) +[2023-10-09 15:27:23,744][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000080992_82935808.pth +[2023-10-09 15:27:24,013][86121] Updated weights for policy 0, policy_version 82370 (0.0010) +[2023-10-09 15:27:24,378][86121] Updated weights for policy 0, policy_version 82380 (0.0007) +[2023-10-09 15:27:24,740][86121] Updated weights for policy 0, policy_version 82390 (0.0007) +[2023-10-09 15:27:25,110][86121] Updated weights for policy 0, policy_version 82400 (0.0007) +[2023-10-09 15:27:27,667][86122] Updated weights for policy 1, policy_version 82730 (0.0011) +[2023-10-09 15:27:28,030][86122] Updated weights for policy 1, policy_version 82740 (0.0009) +[2023-10-09 15:27:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169082880. Throughput: 0: 1838.1, 1: 1823.8. Samples: 42280156. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:27:28,401][86122] Updated weights for policy 1, policy_version 82750 (0.0010) +[2023-10-09 15:27:29,263][86121] Updated weights for policy 0, policy_version 82410 (0.0009) +[2023-10-09 15:27:29,634][86121] Updated weights for policy 0, policy_version 82420 (0.0010) +[2023-10-09 15:27:29,998][86121] Updated weights for policy 0, policy_version 82430 (0.0008) +[2023-10-09 15:27:32,517][86122] Updated weights for policy 1, policy_version 82760 (0.0009) +[2023-10-09 15:27:32,873][86122] Updated weights for policy 1, policy_version 82770 (0.0010) +[2023-10-09 15:27:33,236][86122] Updated weights for policy 1, policy_version 82780 (0.0009) +[2023-10-09 15:27:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169181184. Throughput: 0: 1811.5, 1: 1803.6. Samples: 42301194. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:27:33,798][86121] Updated weights for policy 0, policy_version 82440 (0.0010) +[2023-10-09 15:27:34,165][86121] Updated weights for policy 0, policy_version 82450 (0.0009) +[2023-10-09 15:27:34,531][86121] Updated weights for policy 0, policy_version 82460 (0.0008) +[2023-10-09 15:27:37,160][86122] Updated weights for policy 1, policy_version 82790 (0.0010) +[2023-10-09 15:27:37,530][86122] Updated weights for policy 1, policy_version 82800 (0.0011) +[2023-10-09 15:27:37,885][86122] Updated weights for policy 1, policy_version 82810 (0.0011) +[2023-10-09 15:27:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169246720. Throughput: 0: 1798.7, 1: 1797.9. Samples: 42321912. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:27:38,447][86121] Updated weights for policy 0, policy_version 82470 (0.0010) +[2023-10-09 15:27:38,818][86121] Updated weights for policy 0, policy_version 82480 (0.0010) +[2023-10-09 15:27:39,181][86121] Updated weights for policy 0, policy_version 82490 (0.0009) +[2023-10-09 15:27:42,052][86122] Updated weights for policy 1, policy_version 82820 (0.0012) +[2023-10-09 15:27:42,408][86122] Updated weights for policy 1, policy_version 82830 (0.0011) +[2023-10-09 15:27:42,773][86122] Updated weights for policy 1, policy_version 82840 (0.0011) +[2023-10-09 15:27:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 169312256. Throughput: 0: 1785.0, 1: 1773.2. Samples: 42331598. Policy #0 lag: (min: 2.0, avg: 12.7, max: 34.0) +[2023-10-09 15:27:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:27:43,428][86121] Updated weights for policy 0, policy_version 82500 (0.0010) +[2023-10-09 15:27:43,783][86121] Updated weights for policy 0, policy_version 82510 (0.0010) +[2023-10-09 15:27:44,149][86121] Updated weights for policy 0, policy_version 82520 (0.0009) +[2023-10-09 15:27:46,781][86122] Updated weights for policy 1, policy_version 82850 (0.0009) +[2023-10-09 15:27:47,149][86122] Updated weights for policy 1, policy_version 82860 (0.0008) +[2023-10-09 15:27:47,512][86122] Updated weights for policy 1, policy_version 82870 (0.0008) +[2023-10-09 15:27:47,867][86122] Updated weights for policy 1, policy_version 82880 (0.0009) +[2023-10-09 15:27:48,345][86121] Updated weights for policy 0, policy_version 82530 (0.0011) +[2023-10-09 15:27:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169377792. Throughput: 0: 1757.2, 1: 1780.0. Samples: 42351922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:27:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:27:48,712][86121] Updated weights for policy 0, policy_version 82540 (0.0010) +[2023-10-09 15:27:49,082][86121] Updated weights for policy 0, policy_version 82550 (0.0009) +[2023-10-09 15:27:49,446][86121] Updated weights for policy 0, policy_version 82560 (0.0008) +[2023-10-09 15:27:51,819][86122] Updated weights for policy 1, policy_version 82890 (0.0010) +[2023-10-09 15:27:52,188][86122] Updated weights for policy 1, policy_version 82900 (0.0010) +[2023-10-09 15:27:52,544][86122] Updated weights for policy 1, policy_version 82910 (0.0010) +[2023-10-09 15:27:53,167][86121] Updated weights for policy 0, policy_version 82570 (0.0008) +[2023-10-09 15:27:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169443328. Throughput: 0: 1772.8, 1: 1734.1. Samples: 42372480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:27:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:27:53,541][86121] Updated weights for policy 0, policy_version 82580 (0.0007) +[2023-10-09 15:27:53,900][86121] Updated weights for policy 0, policy_version 82590 (0.0010) +[2023-10-09 15:27:56,504][86122] Updated weights for policy 1, policy_version 82920 (0.0010) +[2023-10-09 15:27:56,874][86122] Updated weights for policy 1, policy_version 82930 (0.0009) +[2023-10-09 15:27:57,241][86122] Updated weights for policy 1, policy_version 82940 (0.0008) +[2023-10-09 15:27:57,795][86121] Updated weights for policy 0, policy_version 82600 (0.0009) +[2023-10-09 15:27:58,156][86121] Updated weights for policy 0, policy_version 82610 (0.0009) +[2023-10-09 15:27:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 169508864. Throughput: 0: 1739.5, 1: 1756.3. Samples: 42383352. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:27:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 15:27:58,524][86121] Updated weights for policy 0, policy_version 82620 (0.0009) +[2023-10-09 15:28:01,133][86122] Updated weights for policy 1, policy_version 82950 (0.0009) +[2023-10-09 15:28:01,487][86122] Updated weights for policy 1, policy_version 82960 (0.0011) +[2023-10-09 15:28:01,850][86122] Updated weights for policy 1, policy_version 82970 (0.0010) +[2023-10-09 15:28:02,744][86121] Updated weights for policy 0, policy_version 82630 (0.0009) +[2023-10-09 15:28:03,114][86121] Updated weights for policy 0, policy_version 82640 (0.0009) +[2023-10-09 15:28:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 13653.3, 300 sec: 14440.1). Total num frames: 169574400. Throughput: 0: 1745.0, 1: 1717.9. Samples: 42403634. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 15:28:03,480][86121] Updated weights for policy 0, policy_version 82650 (0.0008) +[2023-10-09 15:28:05,586][86122] Updated weights for policy 1, policy_version 82980 (0.0010) +[2023-10-09 15:28:05,958][86122] Updated weights for policy 1, policy_version 82990 (0.0008) +[2023-10-09 15:28:06,321][86122] Updated weights for policy 1, policy_version 83000 (0.0008) +[2023-10-09 15:28:07,254][86121] Updated weights for policy 0, policy_version 82660 (0.0009) +[2023-10-09 15:28:07,622][86121] Updated weights for policy 0, policy_version 82670 (0.0009) +[2023-10-09 15:28:07,995][86121] Updated weights for policy 0, policy_version 82680 (0.0008) +[2023-10-09 15:28:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 169672704. Throughput: 0: 1715.5, 1: 1729.3. Samples: 42425048. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 15:28:09,861][86122] Updated weights for policy 1, policy_version 83010 (0.0009) +[2023-10-09 15:28:10,213][86122] Updated weights for policy 1, policy_version 83020 (0.0009) +[2023-10-09 15:28:10,580][86122] Updated weights for policy 1, policy_version 83030 (0.0009) +[2023-10-09 15:28:10,939][86122] Updated weights for policy 1, policy_version 83040 (0.0009) +[2023-10-09 15:28:11,620][86121] Updated weights for policy 0, policy_version 82690 (0.0007) +[2023-10-09 15:28:11,986][86121] Updated weights for policy 0, policy_version 82700 (0.0007) +[2023-10-09 15:28:12,358][86121] Updated weights for policy 0, policy_version 82710 (0.0010) +[2023-10-09 15:28:12,726][86121] Updated weights for policy 0, policy_version 82720 (0.0008) +[2023-10-09 15:28:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169738240. Throughput: 0: 1741.7, 1: 1730.8. Samples: 42436418. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:28:14,673][86122] Updated weights for policy 1, policy_version 83050 (0.0012) +[2023-10-09 15:28:15,023][86122] Updated weights for policy 1, policy_version 83060 (0.0009) +[2023-10-09 15:28:15,385][86122] Updated weights for policy 1, policy_version 83070 (0.0012) +[2023-10-09 15:28:16,380][86121] Updated weights for policy 0, policy_version 82730 (0.0009) +[2023-10-09 15:28:16,751][86121] Updated weights for policy 0, policy_version 82740 (0.0008) +[2023-10-09 15:28:17,120][86121] Updated weights for policy 0, policy_version 82750 (0.0010) +[2023-10-09 15:28:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.2). Total num frames: 169803776. Throughput: 0: 1738.8, 1: 1746.1. Samples: 42458014. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:28:19,064][86122] Updated weights for policy 1, policy_version 83080 (0.0008) +[2023-10-09 15:28:19,432][86122] Updated weights for policy 1, policy_version 83090 (0.0008) +[2023-10-09 15:28:19,795][86122] Updated weights for policy 1, policy_version 83100 (0.0007) +[2023-10-09 15:28:20,647][86121] Updated weights for policy 0, policy_version 82760 (0.0009) +[2023-10-09 15:28:21,010][86121] Updated weights for policy 0, policy_version 82770 (0.0009) +[2023-10-09 15:28:21,378][86121] Updated weights for policy 0, policy_version 82780 (0.0010) +[2023-10-09 15:28:23,368][86122] Updated weights for policy 1, policy_version 83110 (0.0009) +[2023-10-09 15:28:23,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169869312. Throughput: 0: 1746.7, 1: 1786.7. Samples: 42480914. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:28:23,736][86122] Updated weights for policy 1, policy_version 83120 (0.0008) +[2023-10-09 15:28:24,089][86122] Updated weights for policy 1, policy_version 83130 (0.0008) +[2023-10-09 15:28:25,018][86121] Updated weights for policy 0, policy_version 82790 (0.0008) +[2023-10-09 15:28:25,374][86121] Updated weights for policy 0, policy_version 82800 (0.0007) +[2023-10-09 15:28:25,738][86121] Updated weights for policy 0, policy_version 82810 (0.0008) +[2023-10-09 15:28:27,612][86122] Updated weights for policy 1, policy_version 83140 (0.0008) +[2023-10-09 15:28:27,972][86122] Updated weights for policy 1, policy_version 83150 (0.0008) +[2023-10-09 15:28:28,332][86122] Updated weights for policy 1, policy_version 83160 (0.0007) +[2023-10-09 15:28:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 169934848. Throughput: 0: 1764.3, 1: 1778.4. Samples: 42491020. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:28:29,327][86121] Updated weights for policy 0, policy_version 82820 (0.0009) +[2023-10-09 15:28:29,698][86121] Updated weights for policy 0, policy_version 82830 (0.0012) +[2023-10-09 15:28:30,053][86121] Updated weights for policy 0, policy_version 82840 (0.0010) +[2023-10-09 15:28:31,972][86122] Updated weights for policy 1, policy_version 83170 (0.0008) +[2023-10-09 15:28:32,335][86122] Updated weights for policy 1, policy_version 83180 (0.0008) +[2023-10-09 15:28:32,687][86122] Updated weights for policy 1, policy_version 83190 (0.0008) +[2023-10-09 15:28:33,044][86122] Updated weights for policy 1, policy_version 83200 (0.0007) +[2023-10-09 15:28:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 170033152. Throughput: 0: 1794.3, 1: 1812.3. Samples: 42514218. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:28:33,731][86121] Updated weights for policy 0, policy_version 82850 (0.0010) +[2023-10-09 15:28:34,095][86121] Updated weights for policy 0, policy_version 82860 (0.0008) +[2023-10-09 15:28:34,461][86121] Updated weights for policy 0, policy_version 82870 (0.0008) +[2023-10-09 15:28:34,830][86121] Updated weights for policy 0, policy_version 82880 (0.0009) +[2023-10-09 15:28:36,765][86122] Updated weights for policy 1, policy_version 83210 (0.0007) +[2023-10-09 15:28:37,127][86122] Updated weights for policy 1, policy_version 83220 (0.0007) +[2023-10-09 15:28:37,480][86122] Updated weights for policy 1, policy_version 83230 (0.0010) +[2023-10-09 15:28:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 170098688. Throughput: 0: 1816.8, 1: 1816.8. Samples: 42535992. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:28:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:28:38,467][86121] Updated weights for policy 0, policy_version 82890 (0.0008) +[2023-10-09 15:28:38,833][86121] Updated weights for policy 0, policy_version 82900 (0.0008) +[2023-10-09 15:28:39,210][86121] Updated weights for policy 0, policy_version 82910 (0.0009) +[2023-10-09 15:28:41,284][86122] Updated weights for policy 1, policy_version 83240 (0.0008) +[2023-10-09 15:28:41,647][86122] Updated weights for policy 1, policy_version 83250 (0.0008) +[2023-10-09 15:28:42,005][86122] Updated weights for policy 1, policy_version 83260 (0.0007) +[2023-10-09 15:28:42,890][86121] Updated weights for policy 0, policy_version 82920 (0.0009) +[2023-10-09 15:28:43,250][86121] Updated weights for policy 0, policy_version 82930 (0.0009) +[2023-10-09 15:28:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170164224. Throughput: 0: 1819.4, 1: 1825.4. Samples: 42547370. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:28:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:28:43,619][86121] Updated weights for policy 0, policy_version 82940 (0.0009) +[2023-10-09 15:28:45,633][86122] Updated weights for policy 1, policy_version 83270 (0.0010) +[2023-10-09 15:28:45,990][86122] Updated weights for policy 1, policy_version 83280 (0.0011) +[2023-10-09 15:28:46,355][86122] Updated weights for policy 1, policy_version 83290 (0.0010) +[2023-10-09 15:28:47,153][86121] Updated weights for policy 0, policy_version 82950 (0.0008) +[2023-10-09 15:28:47,522][86121] Updated weights for policy 0, policy_version 82960 (0.0010) +[2023-10-09 15:28:47,885][86121] Updated weights for policy 0, policy_version 82970 (0.0008) +[2023-10-09 15:28:48,397][85186] Fps is (10 sec: 16384.5, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 170262528. Throughput: 0: 1848.2, 1: 1830.1. Samples: 42569158. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:28:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:28:49,999][86122] Updated weights for policy 1, policy_version 83300 (0.0009) +[2023-10-09 15:28:50,355][86122] Updated weights for policy 1, policy_version 83310 (0.0009) +[2023-10-09 15:28:50,714][86122] Updated weights for policy 1, policy_version 83320 (0.0010) +[2023-10-09 15:28:51,454][86121] Updated weights for policy 0, policy_version 82980 (0.0011) +[2023-10-09 15:28:51,821][86121] Updated weights for policy 0, policy_version 82990 (0.0011) +[2023-10-09 15:28:52,188][86121] Updated weights for policy 0, policy_version 83000 (0.0010) +[2023-10-09 15:28:53,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170328064. Throughput: 0: 1847.4, 1: 1838.4. Samples: 42590910. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:28:53,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:28:54,454][86122] Updated weights for policy 1, policy_version 83330 (0.0011) +[2023-10-09 15:28:54,817][86122] Updated weights for policy 1, policy_version 83340 (0.0008) +[2023-10-09 15:28:55,172][86122] Updated weights for policy 1, policy_version 83350 (0.0008) +[2023-10-09 15:28:55,533][86122] Updated weights for policy 1, policy_version 83360 (0.0007) +[2023-10-09 15:28:55,801][86121] Updated weights for policy 0, policy_version 83010 (0.0010) +[2023-10-09 15:28:56,166][86121] Updated weights for policy 0, policy_version 83020 (0.0007) +[2023-10-09 15:28:56,528][86121] Updated weights for policy 0, policy_version 83030 (0.0007) +[2023-10-09 15:28:56,893][86121] Updated weights for policy 0, policy_version 83040 (0.0008) +[2023-10-09 15:28:58,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 170393600. Throughput: 0: 1854.1, 1: 1831.1. Samples: 42602252. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:28:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:28:59,253][86122] Updated weights for policy 1, policy_version 83370 (0.0007) +[2023-10-09 15:28:59,612][86122] Updated weights for policy 1, policy_version 83380 (0.0009) +[2023-10-09 15:28:59,968][86122] Updated weights for policy 1, policy_version 83390 (0.0009) +[2023-10-09 15:29:00,525][86121] Updated weights for policy 0, policy_version 83050 (0.0008) +[2023-10-09 15:29:00,886][86121] Updated weights for policy 0, policy_version 83060 (0.0010) +[2023-10-09 15:29:01,251][86121] Updated weights for policy 0, policy_version 83070 (0.0008) +[2023-10-09 15:29:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.5, 300 sec: 14440.1). Total num frames: 170459136. Throughput: 0: 1852.8, 1: 1837.1. Samples: 42624056. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:29:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:29:03,682][86122] Updated weights for policy 1, policy_version 83400 (0.0009) +[2023-10-09 15:29:04,046][86122] Updated weights for policy 1, policy_version 83410 (0.0008) +[2023-10-09 15:29:04,406][86122] Updated weights for policy 1, policy_version 83420 (0.0008) +[2023-10-09 15:29:04,873][86121] Updated weights for policy 0, policy_version 83080 (0.0010) +[2023-10-09 15:29:05,241][86121] Updated weights for policy 0, policy_version 83090 (0.0009) +[2023-10-09 15:29:05,597][86121] Updated weights for policy 0, policy_version 83100 (0.0010) +[2023-10-09 15:29:08,086][86122] Updated weights for policy 1, policy_version 83430 (0.0008) +[2023-10-09 15:29:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 170524672. Throughput: 0: 1852.6, 1: 1838.3. Samples: 42647002. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:29:08,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:29:08,459][86122] Updated weights for policy 1, policy_version 83440 (0.0009) +[2023-10-09 15:29:08,824][86122] Updated weights for policy 1, policy_version 83450 (0.0008) +[2023-10-09 15:29:09,360][86121] Updated weights for policy 0, policy_version 83110 (0.0008) +[2023-10-09 15:29:09,719][86121] Updated weights for policy 0, policy_version 83120 (0.0008) +[2023-10-09 15:29:10,086][86121] Updated weights for policy 0, policy_version 83130 (0.0007) +[2023-10-09 15:29:12,550][86122] Updated weights for policy 1, policy_version 83460 (0.0010) +[2023-10-09 15:29:12,915][86122] Updated weights for policy 1, policy_version 83470 (0.0009) +[2023-10-09 15:29:13,284][86122] Updated weights for policy 1, policy_version 83480 (0.0009) +[2023-10-09 15:29:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 170590208. Throughput: 0: 1849.6, 1: 1838.4. Samples: 42656984. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:29:13,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:29:13,793][86121] Updated weights for policy 0, policy_version 83140 (0.0008) +[2023-10-09 15:29:14,166][86121] Updated weights for policy 0, policy_version 83150 (0.0007) +[2023-10-09 15:29:14,542][86121] Updated weights for policy 0, policy_version 83160 (0.0010) +[2023-10-09 15:29:16,970][86122] Updated weights for policy 1, policy_version 83490 (0.0010) +[2023-10-09 15:29:17,330][86122] Updated weights for policy 1, policy_version 83500 (0.0008) +[2023-10-09 15:29:17,691][86122] Updated weights for policy 1, policy_version 83510 (0.0008) +[2023-10-09 15:29:18,048][86122] Updated weights for policy 1, policy_version 83520 (0.0008) +[2023-10-09 15:29:18,256][86121] Updated weights for policy 0, policy_version 83170 (0.0008) +[2023-10-09 15:29:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170688512. Throughput: 0: 1843.9, 1: 1831.3. Samples: 42679604. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:29:18,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:29:18,627][86121] Updated weights for policy 0, policy_version 83180 (0.0009) +[2023-10-09 15:29:18,993][86121] Updated weights for policy 0, policy_version 83190 (0.0011) +[2023-10-09 15:29:19,359][86121] Updated weights for policy 0, policy_version 83200 (0.0009) +[2023-10-09 15:29:21,733][86122] Updated weights for policy 1, policy_version 83530 (0.0008) +[2023-10-09 15:29:22,096][86122] Updated weights for policy 1, policy_version 83540 (0.0008) +[2023-10-09 15:29:22,456][86122] Updated weights for policy 1, policy_version 83550 (0.0008) +[2023-10-09 15:29:23,036][86121] Updated weights for policy 0, policy_version 83210 (0.0010) +[2023-10-09 15:29:23,396][86121] Updated weights for policy 0, policy_version 83220 (0.0009) +[2023-10-09 15:29:23,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 170754048. Throughput: 0: 1833.7, 1: 1828.0. Samples: 42700766. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:29:23,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:29:23,405][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000083552_85557248.pth... +[2023-10-09 15:29:23,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000081856_83820544.pth +[2023-10-09 15:29:23,766][86121] Updated weights for policy 0, policy_version 83230 (0.0008) +[2023-10-09 15:29:23,830][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000083232_85229568.pth... +[2023-10-09 15:29:23,858][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000081536_83492864.pth +[2023-10-09 15:29:26,127][86122] Updated weights for policy 1, policy_version 83560 (0.0009) +[2023-10-09 15:29:26,491][86122] Updated weights for policy 1, policy_version 83570 (0.0008) +[2023-10-09 15:29:26,862][86122] Updated weights for policy 1, policy_version 83580 (0.0009) +[2023-10-09 15:29:27,524][86121] Updated weights for policy 0, policy_version 83240 (0.0009) +[2023-10-09 15:29:27,904][86121] Updated weights for policy 0, policy_version 83250 (0.0009) +[2023-10-09 15:29:28,261][86121] Updated weights for policy 0, policy_version 83260 (0.0010) +[2023-10-09 15:29:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 170819584. Throughput: 0: 1840.4, 1: 1828.3. Samples: 42712462. Policy #0 lag: (min: 31.0, avg: 33.2, max: 62.0) +[2023-10-09 15:29:28,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:29:30,410][86122] Updated weights for policy 1, policy_version 83590 (0.0008) +[2023-10-09 15:29:30,767][86122] Updated weights for policy 1, policy_version 83600 (0.0007) +[2023-10-09 15:29:31,126][86122] Updated weights for policy 1, policy_version 83610 (0.0007) +[2023-10-09 15:29:31,842][86121] Updated weights for policy 0, policy_version 83270 (0.0008) +[2023-10-09 15:29:32,218][86121] Updated weights for policy 0, policy_version 83280 (0.0007) +[2023-10-09 15:29:32,577][86121] Updated weights for policy 0, policy_version 83290 (0.0007) +[2023-10-09 15:29:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 170917888. Throughput: 0: 1823.3, 1: 1831.6. Samples: 42733632. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:29:33,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:29:34,724][86122] Updated weights for policy 1, policy_version 83620 (0.0008) +[2023-10-09 15:29:35,089][86122] Updated weights for policy 1, policy_version 83630 (0.0007) +[2023-10-09 15:29:35,447][86122] Updated weights for policy 1, policy_version 83640 (0.0009) +[2023-10-09 15:29:36,157][86121] Updated weights for policy 0, policy_version 83300 (0.0007) +[2023-10-09 15:29:36,528][86121] Updated weights for policy 0, policy_version 83310 (0.0010) +[2023-10-09 15:29:36,902][86121] Updated weights for policy 0, policy_version 83320 (0.0008) +[2023-10-09 15:29:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 170983424. Throughput: 0: 1830.3, 1: 1832.5. Samples: 42755736. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:29:38,399][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:29:39,006][86122] Updated weights for policy 1, policy_version 83650 (0.0011) +[2023-10-09 15:29:39,362][86122] Updated weights for policy 1, policy_version 83660 (0.0008) +[2023-10-09 15:29:39,726][86122] Updated weights for policy 1, policy_version 83670 (0.0010) +[2023-10-09 15:29:40,088][86122] Updated weights for policy 1, policy_version 83680 (0.0010) +[2023-10-09 15:29:40,461][86121] Updated weights for policy 0, policy_version 83330 (0.0007) +[2023-10-09 15:29:40,826][86121] Updated weights for policy 0, policy_version 83340 (0.0008) +[2023-10-09 15:29:41,199][86121] Updated weights for policy 0, policy_version 83350 (0.0010) +[2023-10-09 15:29:41,564][86121] Updated weights for policy 0, policy_version 83360 (0.0011) +[2023-10-09 15:29:43,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171048960. Throughput: 0: 1819.2, 1: 1836.6. Samples: 42766762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:29:43,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:29:43,683][86122] Updated weights for policy 1, policy_version 83690 (0.0009) +[2023-10-09 15:29:44,048][86122] Updated weights for policy 1, policy_version 83700 (0.0010) +[2023-10-09 15:29:44,414][86122] Updated weights for policy 1, policy_version 83710 (0.0012) +[2023-10-09 15:29:45,298][86121] Updated weights for policy 0, policy_version 83370 (0.0008) +[2023-10-09 15:29:45,661][86121] Updated weights for policy 0, policy_version 83380 (0.0010) +[2023-10-09 15:29:46,032][86121] Updated weights for policy 0, policy_version 83390 (0.0010) +[2023-10-09 15:29:48,106][86122] Updated weights for policy 1, policy_version 83720 (0.0008) +[2023-10-09 15:29:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 171114496. Throughput: 0: 1819.3, 1: 1842.6. Samples: 42788842. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:29:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:29:48,473][86122] Updated weights for policy 1, policy_version 83730 (0.0009) +[2023-10-09 15:29:48,830][86122] Updated weights for policy 1, policy_version 83740 (0.0009) +[2023-10-09 15:29:49,817][86121] Updated weights for policy 0, policy_version 83400 (0.0008) +[2023-10-09 15:29:50,187][86121] Updated weights for policy 0, policy_version 83410 (0.0008) +[2023-10-09 15:29:50,546][86121] Updated weights for policy 0, policy_version 83420 (0.0008) +[2023-10-09 15:29:52,632][86122] Updated weights for policy 1, policy_version 83750 (0.0009) +[2023-10-09 15:29:52,997][86122] Updated weights for policy 1, policy_version 83760 (0.0011) +[2023-10-09 15:29:53,356][86122] Updated weights for policy 1, policy_version 83770 (0.0007) +[2023-10-09 15:29:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171180032. Throughput: 0: 1821.1, 1: 1828.8. Samples: 42811244. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:29:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:29:54,390][86121] Updated weights for policy 0, policy_version 83430 (0.0008) +[2023-10-09 15:29:54,755][86121] Updated weights for policy 0, policy_version 83440 (0.0008) +[2023-10-09 15:29:55,117][86121] Updated weights for policy 0, policy_version 83450 (0.0007) +[2023-10-09 15:29:56,972][86122] Updated weights for policy 1, policy_version 83780 (0.0007) +[2023-10-09 15:29:57,368][86122] Updated weights for policy 1, policy_version 83790 (0.0007) +[2023-10-09 15:29:57,725][86122] Updated weights for policy 1, policy_version 83800 (0.0009) +[2023-10-09 15:29:58,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 171278336. Throughput: 0: 1818.1, 1: 1844.9. Samples: 42821816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:29:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:29:58,709][86121] Updated weights for policy 0, policy_version 83460 (0.0009) +[2023-10-09 15:29:59,070][86121] Updated weights for policy 0, policy_version 83470 (0.0008) +[2023-10-09 15:29:59,450][86121] Updated weights for policy 0, policy_version 83480 (0.0007) +[2023-10-09 15:30:01,383][86122] Updated weights for policy 1, policy_version 83810 (0.0011) +[2023-10-09 15:30:01,746][86122] Updated weights for policy 1, policy_version 83820 (0.0008) +[2023-10-09 15:30:02,110][86122] Updated weights for policy 1, policy_version 83830 (0.0008) +[2023-10-09 15:30:02,476][86122] Updated weights for policy 1, policy_version 83840 (0.0009) +[2023-10-09 15:30:03,207][86121] Updated weights for policy 0, policy_version 83490 (0.0008) +[2023-10-09 15:30:03,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 171343872. Throughput: 0: 1824.8, 1: 1829.4. Samples: 42844042. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:30:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:03,581][86121] Updated weights for policy 0, policy_version 83500 (0.0007) +[2023-10-09 15:30:03,937][86121] Updated weights for policy 0, policy_version 83510 (0.0008) +[2023-10-09 15:30:04,304][86121] Updated weights for policy 0, policy_version 83520 (0.0009) +[2023-10-09 15:30:06,054][86122] Updated weights for policy 1, policy_version 83850 (0.0007) +[2023-10-09 15:30:06,413][86122] Updated weights for policy 1, policy_version 83860 (0.0008) +[2023-10-09 15:30:06,776][86122] Updated weights for policy 1, policy_version 83870 (0.0007) +[2023-10-09 15:30:08,031][86121] Updated weights for policy 0, policy_version 83530 (0.0007) +[2023-10-09 15:30:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171409408. Throughput: 0: 1814.6, 1: 1847.2. Samples: 42865548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:30:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 15:30:08,403][86121] Updated weights for policy 0, policy_version 83540 (0.0009) +[2023-10-09 15:30:08,762][86121] Updated weights for policy 0, policy_version 83550 (0.0009) +[2023-10-09 15:30:10,604][86122] Updated weights for policy 1, policy_version 83880 (0.0010) +[2023-10-09 15:30:10,964][86122] Updated weights for policy 1, policy_version 83890 (0.0007) +[2023-10-09 15:30:11,324][86122] Updated weights for policy 1, policy_version 83900 (0.0008) +[2023-10-09 15:30:12,448][86121] Updated weights for policy 0, policy_version 83560 (0.0008) +[2023-10-09 15:30:12,828][86121] Updated weights for policy 0, policy_version 83570 (0.0010) +[2023-10-09 15:30:13,194][86121] Updated weights for policy 0, policy_version 83580 (0.0011) +[2023-10-09 15:30:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 171507712. Throughput: 0: 1821.0, 1: 1830.7. Samples: 42876788. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:30:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:14,968][86122] Updated weights for policy 1, policy_version 83910 (0.0008) +[2023-10-09 15:30:15,325][86122] Updated weights for policy 1, policy_version 83920 (0.0008) +[2023-10-09 15:30:15,689][86122] Updated weights for policy 1, policy_version 83930 (0.0007) +[2023-10-09 15:30:16,767][86121] Updated weights for policy 0, policy_version 83590 (0.0009) +[2023-10-09 15:30:17,137][86121] Updated weights for policy 0, policy_version 83600 (0.0009) +[2023-10-09 15:30:17,505][86121] Updated weights for policy 0, policy_version 83610 (0.0008) +[2023-10-09 15:30:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 171573248. Throughput: 0: 1822.4, 1: 1844.0. Samples: 42898620. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:30:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:19,356][86122] Updated weights for policy 1, policy_version 83940 (0.0007) +[2023-10-09 15:30:19,720][86122] Updated weights for policy 1, policy_version 83950 (0.0007) +[2023-10-09 15:30:20,088][86122] Updated weights for policy 1, policy_version 83960 (0.0008) +[2023-10-09 15:30:20,979][86121] Updated weights for policy 0, policy_version 83620 (0.0009) +[2023-10-09 15:30:21,342][86121] Updated weights for policy 0, policy_version 83630 (0.0012) +[2023-10-09 15:30:21,711][86121] Updated weights for policy 0, policy_version 83640 (0.0008) +[2023-10-09 15:30:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171638784. Throughput: 0: 1825.5, 1: 1835.5. Samples: 42920480. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:30:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:23,780][86122] Updated weights for policy 1, policy_version 83970 (0.0010) +[2023-10-09 15:30:24,139][86122] Updated weights for policy 1, policy_version 83980 (0.0008) +[2023-10-09 15:30:24,507][86122] Updated weights for policy 1, policy_version 83990 (0.0007) +[2023-10-09 15:30:24,870][86122] Updated weights for policy 1, policy_version 84000 (0.0008) +[2023-10-09 15:30:25,565][86121] Updated weights for policy 0, policy_version 83650 (0.0010) +[2023-10-09 15:30:25,926][86121] Updated weights for policy 0, policy_version 83660 (0.0007) +[2023-10-09 15:30:26,296][86121] Updated weights for policy 0, policy_version 83670 (0.0008) +[2023-10-09 15:30:26,661][86121] Updated weights for policy 0, policy_version 83680 (0.0009) +[2023-10-09 15:30:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14440.1). Total num frames: 171704320. Throughput: 0: 1825.2, 1: 1832.9. Samples: 42931374. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:30:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:28,707][86122] Updated weights for policy 1, policy_version 84010 (0.0009) +[2023-10-09 15:30:29,066][86122] Updated weights for policy 1, policy_version 84020 (0.0010) +[2023-10-09 15:30:29,431][86122] Updated weights for policy 1, policy_version 84030 (0.0010) +[2023-10-09 15:30:30,310][86121] Updated weights for policy 0, policy_version 83690 (0.0009) +[2023-10-09 15:30:30,673][86121] Updated weights for policy 0, policy_version 83700 (0.0007) +[2023-10-09 15:30:31,034][86121] Updated weights for policy 0, policy_version 83710 (0.0008) +[2023-10-09 15:30:33,231][86122] Updated weights for policy 1, policy_version 84040 (0.0009) +[2023-10-09 15:30:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 171769856. Throughput: 0: 1825.7, 1: 1828.4. Samples: 42953274. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:30:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:33,586][86122] Updated weights for policy 1, policy_version 84050 (0.0010) +[2023-10-09 15:30:33,949][86122] Updated weights for policy 1, policy_version 84060 (0.0010) +[2023-10-09 15:30:34,831][86121] Updated weights for policy 0, policy_version 83720 (0.0008) +[2023-10-09 15:30:35,207][86121] Updated weights for policy 0, policy_version 83730 (0.0007) +[2023-10-09 15:30:35,564][86121] Updated weights for policy 0, policy_version 83740 (0.0008) +[2023-10-09 15:30:37,493][86122] Updated weights for policy 1, policy_version 84070 (0.0008) +[2023-10-09 15:30:37,851][86122] Updated weights for policy 1, policy_version 84080 (0.0008) +[2023-10-09 15:30:38,211][86122] Updated weights for policy 1, policy_version 84090 (0.0012) +[2023-10-09 15:30:38,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 171835392. Throughput: 0: 1823.6, 1: 1823.4. Samples: 42975358. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:30:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:39,396][86121] Updated weights for policy 0, policy_version 83750 (0.0009) +[2023-10-09 15:30:39,772][86121] Updated weights for policy 0, policy_version 83760 (0.0008) +[2023-10-09 15:30:40,130][86121] Updated weights for policy 0, policy_version 83770 (0.0007) +[2023-10-09 15:30:42,025][86122] Updated weights for policy 1, policy_version 84100 (0.0010) +[2023-10-09 15:30:42,401][86122] Updated weights for policy 1, policy_version 84110 (0.0007) +[2023-10-09 15:30:42,763][86122] Updated weights for policy 1, policy_version 84120 (0.0007) +[2023-10-09 15:30:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171933696. Throughput: 0: 1820.4, 1: 1821.1. Samples: 42985680. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:30:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:30:43,871][86121] Updated weights for policy 0, policy_version 83780 (0.0008) +[2023-10-09 15:30:44,245][86121] Updated weights for policy 0, policy_version 83790 (0.0008) +[2023-10-09 15:30:44,614][86121] Updated weights for policy 0, policy_version 83800 (0.0008) +[2023-10-09 15:30:46,353][86122] Updated weights for policy 1, policy_version 84130 (0.0007) +[2023-10-09 15:30:46,716][86122] Updated weights for policy 1, policy_version 84140 (0.0007) +[2023-10-09 15:30:47,063][86122] Updated weights for policy 1, policy_version 84150 (0.0009) +[2023-10-09 15:30:47,426][86122] Updated weights for policy 1, policy_version 84160 (0.0008) +[2023-10-09 15:30:48,151][86121] Updated weights for policy 0, policy_version 83810 (0.0008) +[2023-10-09 15:30:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 171999232. Throughput: 0: 1819.0, 1: 1825.6. Samples: 43008046. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:30:48,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:30:48,515][86121] Updated weights for policy 0, policy_version 83820 (0.0008) +[2023-10-09 15:30:48,880][86121] Updated weights for policy 0, policy_version 83830 (0.0010) +[2023-10-09 15:30:49,243][86121] Updated weights for policy 0, policy_version 83840 (0.0011) +[2023-10-09 15:30:51,075][86122] Updated weights for policy 1, policy_version 84170 (0.0011) +[2023-10-09 15:30:51,434][86122] Updated weights for policy 1, policy_version 84180 (0.0008) +[2023-10-09 15:30:51,794][86122] Updated weights for policy 1, policy_version 84190 (0.0011) +[2023-10-09 15:30:52,743][86121] Updated weights for policy 0, policy_version 83850 (0.0008) +[2023-10-09 15:30:53,109][86121] Updated weights for policy 0, policy_version 83860 (0.0009) +[2023-10-09 15:30:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172064768. Throughput: 0: 1823.3, 1: 1825.9. Samples: 43029762. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:30:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:53,479][86121] Updated weights for policy 0, policy_version 83870 (0.0008) +[2023-10-09 15:30:55,379][86122] Updated weights for policy 1, policy_version 84200 (0.0008) +[2023-10-09 15:30:55,740][86122] Updated weights for policy 1, policy_version 84210 (0.0009) +[2023-10-09 15:30:56,109][86122] Updated weights for policy 1, policy_version 84220 (0.0009) +[2023-10-09 15:30:57,330][86121] Updated weights for policy 0, policy_version 83880 (0.0008) +[2023-10-09 15:30:57,718][86121] Updated weights for policy 0, policy_version 83890 (0.0008) +[2023-10-09 15:30:58,080][86121] Updated weights for policy 0, policy_version 83900 (0.0011) +[2023-10-09 15:30:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 172163072. Throughput: 0: 1825.6, 1: 1824.4. Samples: 43041034. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:30:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:30:59,934][86122] Updated weights for policy 1, policy_version 84230 (0.0007) +[2023-10-09 15:31:00,288][86122] Updated weights for policy 1, policy_version 84240 (0.0009) +[2023-10-09 15:31:00,657][86122] Updated weights for policy 1, policy_version 84250 (0.0009) +[2023-10-09 15:31:01,831][86121] Updated weights for policy 0, policy_version 83910 (0.0010) +[2023-10-09 15:31:02,192][86121] Updated weights for policy 0, policy_version 83920 (0.0008) +[2023-10-09 15:31:02,564][86121] Updated weights for policy 0, policy_version 83930 (0.0008) +[2023-10-09 15:31:03,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 172228608. Throughput: 0: 1825.5, 1: 1822.3. Samples: 43062770. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:31:03,399][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:31:04,356][86122] Updated weights for policy 1, policy_version 84260 (0.0009) +[2023-10-09 15:31:04,712][86122] Updated weights for policy 1, policy_version 84270 (0.0007) +[2023-10-09 15:31:05,080][86122] Updated weights for policy 1, policy_version 84280 (0.0007) +[2023-10-09 15:31:06,180][86121] Updated weights for policy 0, policy_version 83940 (0.0009) +[2023-10-09 15:31:06,543][86121] Updated weights for policy 0, policy_version 83950 (0.0010) +[2023-10-09 15:31:06,921][86121] Updated weights for policy 0, policy_version 83960 (0.0007) +[2023-10-09 15:31:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 172294144. Throughput: 0: 1817.7, 1: 1823.6. Samples: 43084338. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:31:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:31:08,869][86122] Updated weights for policy 1, policy_version 84290 (0.0009) +[2023-10-09 15:31:09,225][86122] Updated weights for policy 1, policy_version 84300 (0.0008) +[2023-10-09 15:31:09,589][86122] Updated weights for policy 1, policy_version 84310 (0.0010) +[2023-10-09 15:31:09,953][86122] Updated weights for policy 1, policy_version 84320 (0.0008) +[2023-10-09 15:31:10,727][86121] Updated weights for policy 0, policy_version 83970 (0.0008) +[2023-10-09 15:31:11,080][86121] Updated weights for policy 0, policy_version 83980 (0.0011) +[2023-10-09 15:31:11,441][86121] Updated weights for policy 0, policy_version 83990 (0.0010) +[2023-10-09 15:31:11,806][86121] Updated weights for policy 0, policy_version 84000 (0.0010) +[2023-10-09 15:31:13,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 172359680. Throughput: 0: 1822.6, 1: 1821.5. Samples: 43095358. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:31:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:31:13,495][86122] Updated weights for policy 1, policy_version 84330 (0.0011) +[2023-10-09 15:31:13,867][86122] Updated weights for policy 1, policy_version 84340 (0.0009) +[2023-10-09 15:31:14,214][86122] Updated weights for policy 1, policy_version 84350 (0.0007) +[2023-10-09 15:31:15,381][86121] Updated weights for policy 0, policy_version 84010 (0.0009) +[2023-10-09 15:31:15,741][86121] Updated weights for policy 0, policy_version 84020 (0.0010) +[2023-10-09 15:31:16,105][86121] Updated weights for policy 0, policy_version 84030 (0.0007) +[2023-10-09 15:31:18,022][86122] Updated weights for policy 1, policy_version 84360 (0.0008) +[2023-10-09 15:31:18,382][86122] Updated weights for policy 1, policy_version 84370 (0.0011) +[2023-10-09 15:31:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 172425216. Throughput: 0: 1819.4, 1: 1822.0. Samples: 43117140. Policy #0 lag: (min: 15.0, avg: 15.1, max: 21.0) +[2023-10-09 15:31:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:31:18,740][86122] Updated weights for policy 1, policy_version 84380 (0.0011) +[2023-10-09 15:31:19,637][86121] Updated weights for policy 0, policy_version 84040 (0.0008) +[2023-10-09 15:31:19,999][86121] Updated weights for policy 0, policy_version 84050 (0.0007) +[2023-10-09 15:31:20,367][86121] Updated weights for policy 0, policy_version 84060 (0.0009) +[2023-10-09 15:31:22,405][86122] Updated weights for policy 1, policy_version 84390 (0.0008) +[2023-10-09 15:31:22,772][86122] Updated weights for policy 1, policy_version 84400 (0.0009) +[2023-10-09 15:31:23,132][86122] Updated weights for policy 1, policy_version 84410 (0.0008) +[2023-10-09 15:31:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172523520. Throughput: 0: 1826.4, 1: 1818.6. Samples: 43139384. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:31:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000084064_86081536.pth... +[2023-10-09 15:31:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000084416_86441984.pth... +[2023-10-09 15:31:23,438][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000082720_84705280.pth +[2023-10-09 15:31:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000082368_84344832.pth +[2023-10-09 15:31:23,980][86121] Updated weights for policy 0, policy_version 84070 (0.0008) +[2023-10-09 15:31:24,351][86121] Updated weights for policy 0, policy_version 84080 (0.0009) +[2023-10-09 15:31:24,717][86121] Updated weights for policy 0, policy_version 84090 (0.0008) +[2023-10-09 15:31:26,890][86122] Updated weights for policy 1, policy_version 84420 (0.0011) +[2023-10-09 15:31:27,278][86122] Updated weights for policy 1, policy_version 84430 (0.0008) +[2023-10-09 15:31:27,634][86122] Updated weights for policy 1, policy_version 84440 (0.0009) +[2023-10-09 15:31:28,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172589056. Throughput: 0: 1829.9, 1: 1822.0. Samples: 43150016. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:31:28,504][86121] Updated weights for policy 0, policy_version 84100 (0.0008) +[2023-10-09 15:31:28,875][86121] Updated weights for policy 0, policy_version 84110 (0.0008) +[2023-10-09 15:31:29,245][86121] Updated weights for policy 0, policy_version 84120 (0.0010) +[2023-10-09 15:31:31,359][86122] Updated weights for policy 1, policy_version 84450 (0.0011) +[2023-10-09 15:31:31,721][86122] Updated weights for policy 1, policy_version 84460 (0.0007) +[2023-10-09 15:31:32,073][86122] Updated weights for policy 1, policy_version 84470 (0.0007) +[2023-10-09 15:31:32,440][86122] Updated weights for policy 1, policy_version 84480 (0.0009) +[2023-10-09 15:31:32,888][86121] Updated weights for policy 0, policy_version 84130 (0.0008) +[2023-10-09 15:31:33,251][86121] Updated weights for policy 0, policy_version 84140 (0.0007) +[2023-10-09 15:31:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172654592. Throughput: 0: 1831.5, 1: 1818.3. Samples: 43172288. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:31:33,626][86121] Updated weights for policy 0, policy_version 84150 (0.0008) +[2023-10-09 15:31:33,985][86121] Updated weights for policy 0, policy_version 84160 (0.0007) +[2023-10-09 15:31:36,222][86122] Updated weights for policy 1, policy_version 84490 (0.0008) +[2023-10-09 15:31:36,570][86122] Updated weights for policy 1, policy_version 84500 (0.0007) +[2023-10-09 15:31:36,934][86122] Updated weights for policy 1, policy_version 84510 (0.0009) +[2023-10-09 15:31:37,707][86121] Updated weights for policy 0, policy_version 84170 (0.0008) +[2023-10-09 15:31:38,071][86121] Updated weights for policy 0, policy_version 84180 (0.0010) +[2023-10-09 15:31:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172720128. Throughput: 0: 1828.2, 1: 1816.3. Samples: 43193764. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:38,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:31:38,434][86121] Updated weights for policy 0, policy_version 84190 (0.0008) +[2023-10-09 15:31:40,509][86122] Updated weights for policy 1, policy_version 84520 (0.0012) +[2023-10-09 15:31:40,868][86122] Updated weights for policy 1, policy_version 84530 (0.0008) +[2023-10-09 15:31:41,228][86122] Updated weights for policy 1, policy_version 84540 (0.0008) +[2023-10-09 15:31:42,161][86121] Updated weights for policy 0, policy_version 84200 (0.0009) +[2023-10-09 15:31:42,526][86121] Updated weights for policy 0, policy_version 84210 (0.0007) +[2023-10-09 15:31:42,901][86121] Updated weights for policy 0, policy_version 84220 (0.0008) +[2023-10-09 15:31:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172818432. Throughput: 0: 1828.4, 1: 1817.5. Samples: 43205100. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:43,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:31:44,977][86122] Updated weights for policy 1, policy_version 84550 (0.0009) +[2023-10-09 15:31:45,344][86122] Updated weights for policy 1, policy_version 84560 (0.0008) +[2023-10-09 15:31:45,708][86122] Updated weights for policy 1, policy_version 84570 (0.0011) +[2023-10-09 15:31:46,625][86121] Updated weights for policy 0, policy_version 84230 (0.0009) +[2023-10-09 15:31:46,982][86121] Updated weights for policy 0, policy_version 84240 (0.0007) +[2023-10-09 15:31:47,343][86121] Updated weights for policy 0, policy_version 84250 (0.0008) +[2023-10-09 15:31:48,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172883968. Throughput: 0: 1826.0, 1: 1814.6. Samples: 43226598. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:48,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:31:49,350][86122] Updated weights for policy 1, policy_version 84580 (0.0010) +[2023-10-09 15:31:49,716][86122] Updated weights for policy 1, policy_version 84590 (0.0010) +[2023-10-09 15:31:50,070][86122] Updated weights for policy 1, policy_version 84600 (0.0009) +[2023-10-09 15:31:50,975][86121] Updated weights for policy 0, policy_version 84260 (0.0008) +[2023-10-09 15:31:51,346][86121] Updated weights for policy 0, policy_version 84270 (0.0009) +[2023-10-09 15:31:51,707][86121] Updated weights for policy 0, policy_version 84280 (0.0008) +[2023-10-09 15:31:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 172949504. Throughput: 0: 1839.9, 1: 1813.0. Samples: 43248720. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.990')] +[2023-10-09 15:31:53,904][86122] Updated weights for policy 1, policy_version 84610 (0.0008) +[2023-10-09 15:31:54,265][86122] Updated weights for policy 1, policy_version 84620 (0.0009) +[2023-10-09 15:31:54,623][86122] Updated weights for policy 1, policy_version 84630 (0.0008) +[2023-10-09 15:31:54,994][86122] Updated weights for policy 1, policy_version 84640 (0.0009) +[2023-10-09 15:31:55,229][86121] Updated weights for policy 0, policy_version 84290 (0.0008) +[2023-10-09 15:31:55,598][86121] Updated weights for policy 0, policy_version 84300 (0.0009) +[2023-10-09 15:31:55,958][86121] Updated weights for policy 0, policy_version 84310 (0.0009) +[2023-10-09 15:31:56,327][86121] Updated weights for policy 0, policy_version 84320 (0.0009) +[2023-10-09 15:31:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 173015040. Throughput: 0: 1832.8, 1: 1814.9. Samples: 43259504. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:31:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:31:58,598][86122] Updated weights for policy 1, policy_version 84650 (0.0011) +[2023-10-09 15:31:58,962][86122] Updated weights for policy 1, policy_version 84660 (0.0008) +[2023-10-09 15:31:59,321][86122] Updated weights for policy 1, policy_version 84670 (0.0008) +[2023-10-09 15:31:59,818][86121] Updated weights for policy 0, policy_version 84330 (0.0010) +[2023-10-09 15:32:00,185][86121] Updated weights for policy 0, policy_version 84340 (0.0009) +[2023-10-09 15:32:00,555][86121] Updated weights for policy 0, policy_version 84350 (0.0010) +[2023-10-09 15:32:03,185][86122] Updated weights for policy 1, policy_version 84680 (0.0007) +[2023-10-09 15:32:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 173080576. Throughput: 0: 1850.6, 1: 1811.8. Samples: 43281946. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:32:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:32:03,553][86122] Updated weights for policy 1, policy_version 84690 (0.0008) +[2023-10-09 15:32:03,920][86122] Updated weights for policy 1, policy_version 84700 (0.0009) +[2023-10-09 15:32:04,229][86121] Updated weights for policy 0, policy_version 84360 (0.0009) +[2023-10-09 15:32:04,593][86121] Updated weights for policy 0, policy_version 84370 (0.0007) +[2023-10-09 15:32:04,960][86121] Updated weights for policy 0, policy_version 84380 (0.0009) +[2023-10-09 15:32:07,570][86122] Updated weights for policy 1, policy_version 84710 (0.0007) +[2023-10-09 15:32:07,922][86122] Updated weights for policy 1, policy_version 84720 (0.0009) +[2023-10-09 15:32:08,283][86122] Updated weights for policy 1, policy_version 84730 (0.0009) +[2023-10-09 15:32:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 173146112. Throughput: 0: 1843.3, 1: 1823.7. Samples: 43304398. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:32:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:32:08,692][86121] Updated weights for policy 0, policy_version 84390 (0.0008) +[2023-10-09 15:32:09,058][86121] Updated weights for policy 0, policy_version 84400 (0.0010) +[2023-10-09 15:32:09,422][86121] Updated weights for policy 0, policy_version 84410 (0.0011) +[2023-10-09 15:32:11,957][86122] Updated weights for policy 1, policy_version 84740 (0.0009) +[2023-10-09 15:32:12,346][86122] Updated weights for policy 1, policy_version 84750 (0.0008) +[2023-10-09 15:32:12,706][86122] Updated weights for policy 1, policy_version 84760 (0.0007) +[2023-10-09 15:32:13,177][86121] Updated weights for policy 0, policy_version 84420 (0.0009) +[2023-10-09 15:32:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173244416. Throughput: 0: 1843.5, 1: 1821.0. Samples: 43314920. Policy #0 lag: (min: 31.0, avg: 35.6, max: 63.0) +[2023-10-09 15:32:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:32:13,548][86121] Updated weights for policy 0, policy_version 84430 (0.0010) +[2023-10-09 15:32:13,913][86121] Updated weights for policy 0, policy_version 84440 (0.0009) +[2023-10-09 15:32:16,366][86122] Updated weights for policy 1, policy_version 84770 (0.0009) +[2023-10-09 15:32:16,725][86122] Updated weights for policy 1, policy_version 84780 (0.0008) +[2023-10-09 15:32:17,089][86122] Updated weights for policy 1, policy_version 84790 (0.0008) +[2023-10-09 15:32:17,455][86122] Updated weights for policy 1, policy_version 84800 (0.0008) +[2023-10-09 15:32:17,705][86121] Updated weights for policy 0, policy_version 84450 (0.0008) +[2023-10-09 15:32:18,069][86121] Updated weights for policy 0, policy_version 84460 (0.0008) +[2023-10-09 15:32:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 173309952. Throughput: 0: 1835.2, 1: 1824.5. Samples: 43336972. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:32:18,436][86121] Updated weights for policy 0, policy_version 84470 (0.0009) +[2023-10-09 15:32:18,802][86121] Updated weights for policy 0, policy_version 84480 (0.0008) +[2023-10-09 15:32:21,073][86122] Updated weights for policy 1, policy_version 84810 (0.0009) +[2023-10-09 15:32:21,431][86122] Updated weights for policy 1, policy_version 84820 (0.0008) +[2023-10-09 15:32:21,791][86122] Updated weights for policy 1, policy_version 84830 (0.0008) +[2023-10-09 15:32:22,454][86121] Updated weights for policy 0, policy_version 84490 (0.0007) +[2023-10-09 15:32:22,821][86121] Updated weights for policy 0, policy_version 84500 (0.0008) +[2023-10-09 15:32:23,181][86121] Updated weights for policy 0, policy_version 84510 (0.0008) +[2023-10-09 15:32:23,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173408256. Throughput: 0: 1829.5, 1: 1824.9. Samples: 43358212. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:32:25,521][86122] Updated weights for policy 1, policy_version 84840 (0.0009) +[2023-10-09 15:32:25,885][86122] Updated weights for policy 1, policy_version 84850 (0.0011) +[2023-10-09 15:32:26,246][86122] Updated weights for policy 1, policy_version 84860 (0.0010) +[2023-10-09 15:32:26,759][86121] Updated weights for policy 0, policy_version 84520 (0.0008) +[2023-10-09 15:32:27,135][86121] Updated weights for policy 0, policy_version 84530 (0.0008) +[2023-10-09 15:32:27,496][86121] Updated weights for policy 0, policy_version 84540 (0.0009) +[2023-10-09 15:32:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173473792. Throughput: 0: 1839.7, 1: 1825.2. Samples: 43370024. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:32:29,933][86122] Updated weights for policy 1, policy_version 84870 (0.0009) +[2023-10-09 15:32:30,303][86122] Updated weights for policy 1, policy_version 84880 (0.0009) +[2023-10-09 15:32:30,664][86122] Updated weights for policy 1, policy_version 84890 (0.0007) +[2023-10-09 15:32:31,373][86121] Updated weights for policy 0, policy_version 84550 (0.0010) +[2023-10-09 15:32:31,754][86121] Updated weights for policy 0, policy_version 84560 (0.0010) +[2023-10-09 15:32:32,125][86121] Updated weights for policy 0, policy_version 84570 (0.0010) +[2023-10-09 15:32:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173539328. Throughput: 0: 1823.6, 1: 1832.5. Samples: 43391124. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:32:34,366][86122] Updated weights for policy 1, policy_version 84900 (0.0007) +[2023-10-09 15:32:34,722][86122] Updated weights for policy 1, policy_version 84910 (0.0011) +[2023-10-09 15:32:35,088][86122] Updated weights for policy 1, policy_version 84920 (0.0009) +[2023-10-09 15:32:35,749][86121] Updated weights for policy 0, policy_version 84580 (0.0009) +[2023-10-09 15:32:36,121][86121] Updated weights for policy 0, policy_version 84590 (0.0008) +[2023-10-09 15:32:36,487][86121] Updated weights for policy 0, policy_version 84600 (0.0008) +[2023-10-09 15:32:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 173604864. Throughput: 0: 1822.5, 1: 1837.9. Samples: 43413438. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:32:38,679][86122] Updated weights for policy 1, policy_version 84930 (0.0010) +[2023-10-09 15:32:39,026][86122] Updated weights for policy 1, policy_version 84940 (0.0008) +[2023-10-09 15:32:39,381][86122] Updated weights for policy 1, policy_version 84950 (0.0007) +[2023-10-09 15:32:39,739][86122] Updated weights for policy 1, policy_version 84960 (0.0007) +[2023-10-09 15:32:40,129][86121] Updated weights for policy 0, policy_version 84610 (0.0011) +[2023-10-09 15:32:40,499][86121] Updated weights for policy 0, policy_version 84620 (0.0009) +[2023-10-09 15:32:40,867][86121] Updated weights for policy 0, policy_version 84630 (0.0008) +[2023-10-09 15:32:41,230][86121] Updated weights for policy 0, policy_version 84640 (0.0008) +[2023-10-09 15:32:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173670400. Throughput: 0: 1816.8, 1: 1839.0. Samples: 43424016. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:32:43,456][86122] Updated weights for policy 1, policy_version 84970 (0.0008) +[2023-10-09 15:32:43,819][86122] Updated weights for policy 1, policy_version 84980 (0.0008) +[2023-10-09 15:32:44,172][86122] Updated weights for policy 1, policy_version 84990 (0.0007) +[2023-10-09 15:32:44,863][86121] Updated weights for policy 0, policy_version 84650 (0.0008) +[2023-10-09 15:32:45,235][86121] Updated weights for policy 0, policy_version 84660 (0.0010) +[2023-10-09 15:32:45,590][86121] Updated weights for policy 0, policy_version 84670 (0.0009) +[2023-10-09 15:32:47,720][86122] Updated weights for policy 1, policy_version 85000 (0.0008) +[2023-10-09 15:32:48,089][86122] Updated weights for policy 1, policy_version 85010 (0.0008) +[2023-10-09 15:32:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 173735936. Throughput: 0: 1815.5, 1: 1846.4. Samples: 43446730. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:32:48,454][86122] Updated weights for policy 1, policy_version 85020 (0.0007) +[2023-10-09 15:32:49,165][86121] Updated weights for policy 0, policy_version 84680 (0.0008) +[2023-10-09 15:32:49,530][86121] Updated weights for policy 0, policy_version 84690 (0.0008) +[2023-10-09 15:32:49,889][86121] Updated weights for policy 0, policy_version 84700 (0.0007) +[2023-10-09 15:32:52,299][86122] Updated weights for policy 1, policy_version 85030 (0.0007) +[2023-10-09 15:32:52,653][86122] Updated weights for policy 1, policy_version 85040 (0.0007) +[2023-10-09 15:32:53,013][86122] Updated weights for policy 1, policy_version 85050 (0.0010) +[2023-10-09 15:32:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173834240. Throughput: 0: 1826.4, 1: 1829.0. Samples: 43468890. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:32:53,508][86121] Updated weights for policy 0, policy_version 84710 (0.0007) +[2023-10-09 15:32:53,881][86121] Updated weights for policy 0, policy_version 84720 (0.0009) +[2023-10-09 15:32:54,243][86121] Updated weights for policy 0, policy_version 84730 (0.0011) +[2023-10-09 15:32:56,778][86122] Updated weights for policy 1, policy_version 85060 (0.0008) +[2023-10-09 15:32:57,162][86122] Updated weights for policy 1, policy_version 85070 (0.0007) +[2023-10-09 15:32:57,525][86122] Updated weights for policy 1, policy_version 85080 (0.0007) +[2023-10-09 15:32:58,026][86121] Updated weights for policy 0, policy_version 84740 (0.0009) +[2023-10-09 15:32:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 173899776. Throughput: 0: 1825.7, 1: 1834.1. Samples: 43479610. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:32:58,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:32:58,399][86121] Updated weights for policy 0, policy_version 84750 (0.0007) +[2023-10-09 15:32:58,756][86121] Updated weights for policy 0, policy_version 84760 (0.0007) +[2023-10-09 15:33:00,959][86122] Updated weights for policy 1, policy_version 85090 (0.0008) +[2023-10-09 15:33:01,315][86122] Updated weights for policy 1, policy_version 85100 (0.0008) +[2023-10-09 15:33:01,678][86122] Updated weights for policy 1, policy_version 85110 (0.0008) +[2023-10-09 15:33:02,029][86122] Updated weights for policy 1, policy_version 85120 (0.0007) +[2023-10-09 15:33:02,444][86121] Updated weights for policy 0, policy_version 84770 (0.0007) +[2023-10-09 15:33:02,813][86121] Updated weights for policy 0, policy_version 84780 (0.0007) +[2023-10-09 15:33:03,190][86121] Updated weights for policy 0, policy_version 84790 (0.0007) +[2023-10-09 15:33:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 173965312. Throughput: 0: 1833.0, 1: 1823.1. Samples: 43501494. Policy #0 lag: (min: 1.0, avg: 8.4, max: 33.0) +[2023-10-09 15:33:03,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:33:03,555][86121] Updated weights for policy 0, policy_version 84800 (0.0008) +[2023-10-09 15:33:05,647][86122] Updated weights for policy 1, policy_version 85130 (0.0008) +[2023-10-09 15:33:06,006][86122] Updated weights for policy 1, policy_version 85140 (0.0008) +[2023-10-09 15:33:06,373][86122] Updated weights for policy 1, policy_version 85150 (0.0011) +[2023-10-09 15:33:07,248][86121] Updated weights for policy 0, policy_version 84810 (0.0007) +[2023-10-09 15:33:07,609][86121] Updated weights for policy 0, policy_version 84820 (0.0007) +[2023-10-09 15:33:07,986][86121] Updated weights for policy 0, policy_version 84830 (0.0008) +[2023-10-09 15:33:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 174063616. Throughput: 0: 1818.8, 1: 1837.1. Samples: 43522724. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.990')] +[2023-10-09 15:33:10,018][86122] Updated weights for policy 1, policy_version 85160 (0.0008) +[2023-10-09 15:33:10,380][86122] Updated weights for policy 1, policy_version 85170 (0.0008) +[2023-10-09 15:33:10,742][86122] Updated weights for policy 1, policy_version 85180 (0.0007) +[2023-10-09 15:33:11,630][86121] Updated weights for policy 0, policy_version 84840 (0.0008) +[2023-10-09 15:33:11,999][86121] Updated weights for policy 0, policy_version 84850 (0.0010) +[2023-10-09 15:33:12,375][86121] Updated weights for policy 0, policy_version 84860 (0.0010) +[2023-10-09 15:33:13,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174129152. Throughput: 0: 1822.1, 1: 1825.6. Samples: 43534170. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:13,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.990')] +[2023-10-09 15:33:14,383][86122] Updated weights for policy 1, policy_version 85190 (0.0009) +[2023-10-09 15:33:14,749][86122] Updated weights for policy 1, policy_version 85200 (0.0010) +[2023-10-09 15:33:15,112][86122] Updated weights for policy 1, policy_version 85210 (0.0008) +[2023-10-09 15:33:15,941][86121] Updated weights for policy 0, policy_version 84870 (0.0008) +[2023-10-09 15:33:16,304][86121] Updated weights for policy 0, policy_version 84880 (0.0007) +[2023-10-09 15:33:16,677][86121] Updated weights for policy 0, policy_version 84890 (0.0008) +[2023-10-09 15:33:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174194688. Throughput: 0: 1822.4, 1: 1833.8. Samples: 43555652. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:33:18,870][86122] Updated weights for policy 1, policy_version 85220 (0.0007) +[2023-10-09 15:33:19,221][86122] Updated weights for policy 1, policy_version 85230 (0.0007) +[2023-10-09 15:33:19,585][86122] Updated weights for policy 1, policy_version 85240 (0.0007) +[2023-10-09 15:33:20,469][86121] Updated weights for policy 0, policy_version 84900 (0.0008) +[2023-10-09 15:33:20,838][86121] Updated weights for policy 0, policy_version 84910 (0.0008) +[2023-10-09 15:33:21,203][86121] Updated weights for policy 0, policy_version 84920 (0.0011) +[2023-10-09 15:33:23,326][86122] Updated weights for policy 1, policy_version 85250 (0.0008) +[2023-10-09 15:33:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 174260224. Throughput: 0: 1833.0, 1: 1834.8. Samples: 43578488. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:33:23,405][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000084928_86966272.pth... +[2023-10-09 15:33:23,441][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000083232_85229568.pth +[2023-10-09 15:33:23,445][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000084928_86966272.pth +[2023-10-09 15:33:23,686][86122] Updated weights for policy 1, policy_version 85260 (0.0008) +[2023-10-09 15:33:24,055][86122] Updated weights for policy 1, policy_version 85270 (0.0009) +[2023-10-09 15:33:24,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000085280_87326720.pth... +[2023-10-09 15:33:24,412][86122] Updated weights for policy 1, policy_version 85280 (0.0008) +[2023-10-09 15:33:24,444][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000083552_85557248.pth +[2023-10-09 15:33:24,450][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000085280_87326720.pth +[2023-10-09 15:33:24,762][86121] Updated weights for policy 0, policy_version 84930 (0.0008) +[2023-10-09 15:33:25,134][86121] Updated weights for policy 0, policy_version 84940 (0.0009) +[2023-10-09 15:33:25,511][86121] Updated weights for policy 0, policy_version 84950 (0.0010) +[2023-10-09 15:33:25,874][86121] Updated weights for policy 0, policy_version 84960 (0.0008) +[2023-10-09 15:33:28,187][86122] Updated weights for policy 1, policy_version 85290 (0.0007) +[2023-10-09 15:33:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 174325760. Throughput: 0: 1825.8, 1: 1833.1. Samples: 43588666. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:28,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.990')] +[2023-10-09 15:33:28,538][86122] Updated weights for policy 1, policy_version 85300 (0.0007) +[2023-10-09 15:33:28,901][86122] Updated weights for policy 1, policy_version 85310 (0.0007) +[2023-10-09 15:33:29,441][86121] Updated weights for policy 0, policy_version 84970 (0.0008) +[2023-10-09 15:33:29,804][86121] Updated weights for policy 0, policy_version 84980 (0.0007) +[2023-10-09 15:33:30,165][86121] Updated weights for policy 0, policy_version 84990 (0.0008) +[2023-10-09 15:33:32,639][86122] Updated weights for policy 1, policy_version 85320 (0.0007) +[2023-10-09 15:33:32,995][86122] Updated weights for policy 1, policy_version 85330 (0.0009) +[2023-10-09 15:33:33,364][86122] Updated weights for policy 1, policy_version 85340 (0.0009) +[2023-10-09 15:33:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 174391296. Throughput: 0: 1841.5, 1: 1830.3. Samples: 43611962. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:33,399][85186] Avg episode reward: [(0, '9.890'), (1, '9.990')] +[2023-10-09 15:33:33,736][86121] Updated weights for policy 0, policy_version 85000 (0.0010) +[2023-10-09 15:33:34,094][86121] Updated weights for policy 0, policy_version 85010 (0.0008) +[2023-10-09 15:33:34,459][86121] Updated weights for policy 0, policy_version 85020 (0.0008) +[2023-10-09 15:33:37,043][86122] Updated weights for policy 1, policy_version 85350 (0.0008) +[2023-10-09 15:33:37,404][86122] Updated weights for policy 1, policy_version 85360 (0.0007) +[2023-10-09 15:33:37,759][86122] Updated weights for policy 1, policy_version 85370 (0.0008) +[2023-10-09 15:33:38,151][86121] Updated weights for policy 0, policy_version 85030 (0.0007) +[2023-10-09 15:33:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174489600. Throughput: 0: 1837.3, 1: 1823.4. Samples: 43633620. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:38,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.990')] +[2023-10-09 15:33:38,507][86121] Updated weights for policy 0, policy_version 85040 (0.0009) +[2023-10-09 15:33:38,882][86121] Updated weights for policy 0, policy_version 85050 (0.0007) +[2023-10-09 15:33:41,450][86122] Updated weights for policy 1, policy_version 85380 (0.0010) +[2023-10-09 15:33:41,818][86122] Updated weights for policy 1, policy_version 85390 (0.0010) +[2023-10-09 15:33:42,170][86122] Updated weights for policy 1, policy_version 85400 (0.0008) +[2023-10-09 15:33:42,478][86121] Updated weights for policy 0, policy_version 85060 (0.0009) +[2023-10-09 15:33:42,850][86121] Updated weights for policy 0, policy_version 85070 (0.0011) +[2023-10-09 15:33:43,214][86121] Updated weights for policy 0, policy_version 85080 (0.0011) +[2023-10-09 15:33:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 174555136. Throughput: 0: 1835.8, 1: 1832.4. Samples: 43644680. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:43,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.980')] +[2023-10-09 15:33:45,935][86122] Updated weights for policy 1, policy_version 85410 (0.0009) +[2023-10-09 15:33:46,339][86122] Updated weights for policy 1, policy_version 85420 (0.0007) +[2023-10-09 15:33:46,701][86122] Updated weights for policy 1, policy_version 85430 (0.0008) +[2023-10-09 15:33:46,823][86121] Updated weights for policy 0, policy_version 85090 (0.0010) +[2023-10-09 15:33:47,056][86122] Updated weights for policy 1, policy_version 85440 (0.0009) +[2023-10-09 15:33:47,181][86121] Updated weights for policy 0, policy_version 85100 (0.0010) +[2023-10-09 15:33:47,556][86121] Updated weights for policy 0, policy_version 85110 (0.0009) +[2023-10-09 15:33:47,915][86121] Updated weights for policy 0, policy_version 85120 (0.0009) +[2023-10-09 15:33:48,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 174653440. Throughput: 0: 1832.4, 1: 1825.4. Samples: 43666096. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:48,398][85186] Avg episode reward: [(0, '9.880'), (1, '9.980')] +[2023-10-09 15:33:50,587][86122] Updated weights for policy 1, policy_version 85450 (0.0008) +[2023-10-09 15:33:50,940][86122] Updated weights for policy 1, policy_version 85460 (0.0008) +[2023-10-09 15:33:51,311][86122] Updated weights for policy 1, policy_version 85470 (0.0007) +[2023-10-09 15:33:51,829][86121] Updated weights for policy 0, policy_version 85130 (0.0007) +[2023-10-09 15:33:52,190][86121] Updated weights for policy 0, policy_version 85140 (0.0007) +[2023-10-09 15:33:52,566][86121] Updated weights for policy 0, policy_version 85150 (0.0008) +[2023-10-09 15:33:53,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174718976. Throughput: 0: 1829.4, 1: 1826.2. Samples: 43687228. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:53,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.980')] +[2023-10-09 15:33:55,005][86122] Updated weights for policy 1, policy_version 85480 (0.0008) +[2023-10-09 15:33:55,367][86122] Updated weights for policy 1, policy_version 85490 (0.0008) +[2023-10-09 15:33:55,739][86122] Updated weights for policy 1, policy_version 85500 (0.0009) +[2023-10-09 15:33:56,341][86121] Updated weights for policy 0, policy_version 85160 (0.0010) +[2023-10-09 15:33:56,704][86121] Updated weights for policy 0, policy_version 85170 (0.0010) +[2023-10-09 15:33:57,074][86121] Updated weights for policy 0, policy_version 85180 (0.0007) +[2023-10-09 15:33:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174784512. Throughput: 0: 1831.3, 1: 1824.2. Samples: 43698670. Policy #0 lag: (min: 31.0, avg: 38.3, max: 63.0) +[2023-10-09 15:33:58,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.980')] +[2023-10-09 15:33:59,515][86122] Updated weights for policy 1, policy_version 85510 (0.0008) +[2023-10-09 15:33:59,883][86122] Updated weights for policy 1, policy_version 85520 (0.0009) +[2023-10-09 15:34:00,242][86122] Updated weights for policy 1, policy_version 85530 (0.0008) +[2023-10-09 15:34:00,598][86121] Updated weights for policy 0, policy_version 85190 (0.0009) +[2023-10-09 15:34:00,967][86121] Updated weights for policy 0, policy_version 85200 (0.0009) +[2023-10-09 15:34:01,338][86121] Updated weights for policy 0, policy_version 85210 (0.0007) +[2023-10-09 15:34:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 174850048. Throughput: 0: 1825.6, 1: 1820.4. Samples: 43719722. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:03,398][85186] Avg episode reward: [(0, '9.870'), (1, '9.980')] +[2023-10-09 15:34:04,198][86122] Updated weights for policy 1, policy_version 85540 (0.0008) +[2023-10-09 15:34:04,560][86122] Updated weights for policy 1, policy_version 85550 (0.0010) +[2023-10-09 15:34:04,925][86122] Updated weights for policy 1, policy_version 85560 (0.0010) +[2023-10-09 15:34:05,040][86121] Updated weights for policy 0, policy_version 85220 (0.0008) +[2023-10-09 15:34:05,414][86121] Updated weights for policy 0, policy_version 85230 (0.0009) +[2023-10-09 15:34:05,771][86121] Updated weights for policy 0, policy_version 85240 (0.0009) +[2023-10-09 15:34:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 174915584. Throughput: 0: 1834.4, 1: 1815.8. Samples: 43742748. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:08,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.980')] +[2023-10-09 15:34:08,502][86122] Updated weights for policy 1, policy_version 85570 (0.0008) +[2023-10-09 15:34:08,865][86122] Updated weights for policy 1, policy_version 85580 (0.0008) +[2023-10-09 15:34:09,225][86122] Updated weights for policy 1, policy_version 85590 (0.0010) +[2023-10-09 15:34:09,532][86121] Updated weights for policy 0, policy_version 85250 (0.0010) +[2023-10-09 15:34:09,587][86122] Updated weights for policy 1, policy_version 85600 (0.0007) +[2023-10-09 15:34:09,906][86121] Updated weights for policy 0, policy_version 85260 (0.0007) +[2023-10-09 15:34:10,264][86121] Updated weights for policy 0, policy_version 85270 (0.0009) +[2023-10-09 15:34:10,631][86121] Updated weights for policy 0, policy_version 85280 (0.0009) +[2023-10-09 15:34:13,148][86122] Updated weights for policy 1, policy_version 85610 (0.0009) +[2023-10-09 15:34:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 174981120. Throughput: 0: 1827.9, 1: 1820.6. Samples: 43752848. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:13,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.980')] +[2023-10-09 15:34:13,513][86122] Updated weights for policy 1, policy_version 85620 (0.0009) +[2023-10-09 15:34:13,879][86122] Updated weights for policy 1, policy_version 85630 (0.0008) +[2023-10-09 15:34:14,375][86121] Updated weights for policy 0, policy_version 85290 (0.0010) +[2023-10-09 15:34:14,746][86121] Updated weights for policy 0, policy_version 85300 (0.0007) +[2023-10-09 15:34:15,114][86121] Updated weights for policy 0, policy_version 85310 (0.0010) +[2023-10-09 15:34:17,504][86122] Updated weights for policy 1, policy_version 85640 (0.0009) +[2023-10-09 15:34:17,862][86122] Updated weights for policy 1, policy_version 85650 (0.0010) +[2023-10-09 15:34:18,221][86122] Updated weights for policy 1, policy_version 85660 (0.0011) +[2023-10-09 15:34:18,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 175079424. Throughput: 0: 1824.8, 1: 1824.7. Samples: 43776188. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:18,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.980')] +[2023-10-09 15:34:18,751][86121] Updated weights for policy 0, policy_version 85320 (0.0010) +[2023-10-09 15:34:19,115][86121] Updated weights for policy 0, policy_version 85330 (0.0007) +[2023-10-09 15:34:19,487][86121] Updated weights for policy 0, policy_version 85340 (0.0009) +[2023-10-09 15:34:21,879][86122] Updated weights for policy 1, policy_version 85670 (0.0010) +[2023-10-09 15:34:22,248][86122] Updated weights for policy 1, policy_version 85680 (0.0011) +[2023-10-09 15:34:22,609][86122] Updated weights for policy 1, policy_version 85690 (0.0010) +[2023-10-09 15:34:23,271][86121] Updated weights for policy 0, policy_version 85350 (0.0007) +[2023-10-09 15:34:23,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175144960. Throughput: 0: 1820.4, 1: 1825.4. Samples: 43797682. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:23,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.980')] +[2023-10-09 15:34:23,641][86121] Updated weights for policy 0, policy_version 85360 (0.0008) +[2023-10-09 15:34:24,009][86121] Updated weights for policy 0, policy_version 85370 (0.0007) +[2023-10-09 15:34:26,268][86122] Updated weights for policy 1, policy_version 85700 (0.0009) +[2023-10-09 15:34:26,639][86122] Updated weights for policy 1, policy_version 85710 (0.0010) +[2023-10-09 15:34:27,008][86122] Updated weights for policy 1, policy_version 85720 (0.0010) +[2023-10-09 15:34:27,578][86121] Updated weights for policy 0, policy_version 85380 (0.0009) +[2023-10-09 15:34:27,951][86121] Updated weights for policy 0, policy_version 85390 (0.0009) +[2023-10-09 15:34:28,313][86121] Updated weights for policy 0, policy_version 85400 (0.0008) +[2023-10-09 15:34:28,398][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 175210496. Throughput: 0: 1818.9, 1: 1833.5. Samples: 43809036. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:28,399][85186] Avg episode reward: [(0, '9.850'), (1, '9.980')] +[2023-10-09 15:34:30,725][86122] Updated weights for policy 1, policy_version 85730 (0.0010) +[2023-10-09 15:34:31,092][86122] Updated weights for policy 1, policy_version 85740 (0.0007) +[2023-10-09 15:34:31,452][86122] Updated weights for policy 1, policy_version 85750 (0.0007) +[2023-10-09 15:34:31,818][86122] Updated weights for policy 1, policy_version 85760 (0.0007) +[2023-10-09 15:34:32,152][86121] Updated weights for policy 0, policy_version 85410 (0.0009) +[2023-10-09 15:34:32,515][86121] Updated weights for policy 0, policy_version 85420 (0.0009) +[2023-10-09 15:34:32,885][86121] Updated weights for policy 0, policy_version 85430 (0.0008) +[2023-10-09 15:34:33,253][86121] Updated weights for policy 0, policy_version 85440 (0.0008) +[2023-10-09 15:34:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 175308800. Throughput: 0: 1821.5, 1: 1833.8. Samples: 43830584. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:33,398][85186] Avg episode reward: [(0, '9.840'), (1, '9.980')] +[2023-10-09 15:34:35,657][86122] Updated weights for policy 1, policy_version 85770 (0.0009) +[2023-10-09 15:34:36,017][86122] Updated weights for policy 1, policy_version 85780 (0.0008) +[2023-10-09 15:34:36,379][86122] Updated weights for policy 1, policy_version 85790 (0.0010) +[2023-10-09 15:34:36,955][86121] Updated weights for policy 0, policy_version 85450 (0.0010) +[2023-10-09 15:34:37,330][86121] Updated weights for policy 0, policy_version 85460 (0.0007) +[2023-10-09 15:34:37,701][86121] Updated weights for policy 0, policy_version 85470 (0.0008) +[2023-10-09 15:34:38,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175374336. Throughput: 0: 1821.6, 1: 1831.4. Samples: 43851612. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:38,399][85186] Avg episode reward: [(0, '9.840'), (1, '9.990')] +[2023-10-09 15:34:39,962][86122] Updated weights for policy 1, policy_version 85800 (0.0008) +[2023-10-09 15:34:40,334][86122] Updated weights for policy 1, policy_version 85810 (0.0009) +[2023-10-09 15:34:40,688][86122] Updated weights for policy 1, policy_version 85820 (0.0009) +[2023-10-09 15:34:41,455][86121] Updated weights for policy 0, policy_version 85480 (0.0008) +[2023-10-09 15:34:41,820][86121] Updated weights for policy 0, policy_version 85490 (0.0009) +[2023-10-09 15:34:42,183][86121] Updated weights for policy 0, policy_version 85500 (0.0007) +[2023-10-09 15:34:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 175439872. Throughput: 0: 1824.8, 1: 1830.3. Samples: 43863148. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:43,398][85186] Avg episode reward: [(0, '9.850'), (1, '9.990')] +[2023-10-09 15:34:44,240][86122] Updated weights for policy 1, policy_version 85830 (0.0011) +[2023-10-09 15:34:44,600][86122] Updated weights for policy 1, policy_version 85840 (0.0008) +[2023-10-09 15:34:44,958][86122] Updated weights for policy 1, policy_version 85850 (0.0009) +[2023-10-09 15:34:45,857][86121] Updated weights for policy 0, policy_version 85510 (0.0011) +[2023-10-09 15:34:46,223][86121] Updated weights for policy 0, policy_version 85520 (0.0012) +[2023-10-09 15:34:46,583][86121] Updated weights for policy 0, policy_version 85530 (0.0011) +[2023-10-09 15:34:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 175505408. Throughput: 0: 1823.2, 1: 1839.6. Samples: 43884544. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:48,398][85186] Avg episode reward: [(0, '9.850'), (1, '9.990')] +[2023-10-09 15:34:48,783][86122] Updated weights for policy 1, policy_version 85860 (0.0010) +[2023-10-09 15:34:49,133][86122] Updated weights for policy 1, policy_version 85870 (0.0011) +[2023-10-09 15:34:49,502][86122] Updated weights for policy 1, policy_version 85880 (0.0008) +[2023-10-09 15:34:50,254][86121] Updated weights for policy 0, policy_version 85540 (0.0008) +[2023-10-09 15:34:50,655][86121] Updated weights for policy 0, policy_version 85550 (0.0011) +[2023-10-09 15:34:51,015][86121] Updated weights for policy 0, policy_version 85560 (0.0011) +[2023-10-09 15:34:53,077][86122] Updated weights for policy 1, policy_version 85890 (0.0008) +[2023-10-09 15:34:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175570944. Throughput: 0: 1814.2, 1: 1841.1. Samples: 43907234. Policy #0 lag: (min: 31.0, avg: 39.0, max: 63.0) +[2023-10-09 15:34:53,399][85186] Avg episode reward: [(0, '9.860'), (1, '9.990')] +[2023-10-09 15:34:53,439][86122] Updated weights for policy 1, policy_version 85900 (0.0009) +[2023-10-09 15:34:53,805][86122] Updated weights for policy 1, policy_version 85910 (0.0011) +[2023-10-09 15:34:54,158][86122] Updated weights for policy 1, policy_version 85920 (0.0010) +[2023-10-09 15:34:54,725][86121] Updated weights for policy 0, policy_version 85570 (0.0010) +[2023-10-09 15:34:55,093][86121] Updated weights for policy 0, policy_version 85580 (0.0009) +[2023-10-09 15:34:55,454][86121] Updated weights for policy 0, policy_version 85590 (0.0008) +[2023-10-09 15:34:55,833][86121] Updated weights for policy 0, policy_version 85600 (0.0008) +[2023-10-09 15:34:57,873][86122] Updated weights for policy 1, policy_version 85930 (0.0008) +[2023-10-09 15:34:58,231][86122] Updated weights for policy 1, policy_version 85940 (0.0010) +[2023-10-09 15:34:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175636480. Throughput: 0: 1814.4, 1: 1836.8. Samples: 43917152. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:34:58,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.990')] +[2023-10-09 15:34:58,602][86122] Updated weights for policy 1, policy_version 85950 (0.0008) +[2023-10-09 15:34:59,650][86121] Updated weights for policy 0, policy_version 85610 (0.0008) +[2023-10-09 15:35:00,021][86121] Updated weights for policy 0, policy_version 85620 (0.0007) +[2023-10-09 15:35:00,390][86121] Updated weights for policy 0, policy_version 85630 (0.0009) +[2023-10-09 15:35:02,218][86122] Updated weights for policy 1, policy_version 85960 (0.0010) +[2023-10-09 15:35:02,589][86122] Updated weights for policy 1, policy_version 85970 (0.0011) +[2023-10-09 15:35:02,959][86122] Updated weights for policy 1, policy_version 85980 (0.0009) +[2023-10-09 15:35:03,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 175734784. Throughput: 0: 1806.2, 1: 1828.7. Samples: 43939756. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:03,398][85186] Avg episode reward: [(0, '9.860'), (1, '9.990')] +[2023-10-09 15:35:04,068][86121] Updated weights for policy 0, policy_version 85640 (0.0009) +[2023-10-09 15:35:04,436][86121] Updated weights for policy 0, policy_version 85650 (0.0007) +[2023-10-09 15:35:04,804][86121] Updated weights for policy 0, policy_version 85660 (0.0008) +[2023-10-09 15:35:06,663][86122] Updated weights for policy 1, policy_version 85990 (0.0010) +[2023-10-09 15:35:07,027][86122] Updated weights for policy 1, policy_version 86000 (0.0011) +[2023-10-09 15:35:07,398][86122] Updated weights for policy 1, policy_version 86010 (0.0011) +[2023-10-09 15:35:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 175800320. Throughput: 0: 1814.5, 1: 1820.9. Samples: 43961276. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.990')] +[2023-10-09 15:35:08,418][86121] Updated weights for policy 0, policy_version 85670 (0.0007) +[2023-10-09 15:35:08,798][86121] Updated weights for policy 0, policy_version 85680 (0.0008) +[2023-10-09 15:35:09,170][86121] Updated weights for policy 0, policy_version 85690 (0.0009) +[2023-10-09 15:35:11,169][86122] Updated weights for policy 1, policy_version 86020 (0.0009) +[2023-10-09 15:35:11,525][86122] Updated weights for policy 1, policy_version 86030 (0.0007) +[2023-10-09 15:35:11,884][86122] Updated weights for policy 1, policy_version 86040 (0.0008) +[2023-10-09 15:35:12,917][86121] Updated weights for policy 0, policy_version 85700 (0.0007) +[2023-10-09 15:35:13,278][86121] Updated weights for policy 0, policy_version 85710 (0.0011) +[2023-10-09 15:35:13,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 175865856. Throughput: 0: 1814.1, 1: 1816.1. Samples: 43972396. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:35:13,645][86121] Updated weights for policy 0, policy_version 85720 (0.0010) +[2023-10-09 15:35:15,721][86122] Updated weights for policy 1, policy_version 86050 (0.0009) +[2023-10-09 15:35:16,085][86122] Updated weights for policy 1, policy_version 86060 (0.0007) +[2023-10-09 15:35:16,443][86122] Updated weights for policy 1, policy_version 86070 (0.0008) +[2023-10-09 15:35:16,809][86122] Updated weights for policy 1, policy_version 86080 (0.0009) +[2023-10-09 15:35:17,402][86121] Updated weights for policy 0, policy_version 85730 (0.0009) +[2023-10-09 15:35:17,764][86121] Updated weights for policy 0, policy_version 85740 (0.0007) +[2023-10-09 15:35:18,131][86121] Updated weights for policy 0, policy_version 85750 (0.0009) +[2023-10-09 15:35:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 175931392. Throughput: 0: 1811.9, 1: 1810.2. Samples: 43993576. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:35:18,499][86121] Updated weights for policy 0, policy_version 85760 (0.0010) +[2023-10-09 15:35:20,407][86122] Updated weights for policy 1, policy_version 86090 (0.0008) +[2023-10-09 15:35:20,766][86122] Updated weights for policy 1, policy_version 86100 (0.0008) +[2023-10-09 15:35:21,130][86122] Updated weights for policy 1, policy_version 86110 (0.0010) +[2023-10-09 15:35:22,248][86121] Updated weights for policy 0, policy_version 85770 (0.0009) +[2023-10-09 15:35:22,611][86121] Updated weights for policy 0, policy_version 85780 (0.0008) +[2023-10-09 15:35:22,978][86121] Updated weights for policy 0, policy_version 85790 (0.0008) +[2023-10-09 15:35:23,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 176029696. Throughput: 0: 1820.0, 1: 1817.9. Samples: 44015318. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:23,399][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:35:23,412][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000086112_88178688.pth... +[2023-10-09 15:35:23,412][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000085792_87851008.pth... +[2023-10-09 15:35:23,448][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000084064_86081536.pth +[2023-10-09 15:35:23,451][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000084416_86441984.pth +[2023-10-09 15:35:24,824][86122] Updated weights for policy 1, policy_version 86120 (0.0007) +[2023-10-09 15:35:25,188][86122] Updated weights for policy 1, policy_version 86130 (0.0009) +[2023-10-09 15:35:25,552][86122] Updated weights for policy 1, policy_version 86140 (0.0008) +[2023-10-09 15:35:26,632][86121] Updated weights for policy 0, policy_version 85800 (0.0011) +[2023-10-09 15:35:27,005][86121] Updated weights for policy 0, policy_version 85810 (0.0009) +[2023-10-09 15:35:27,384][86121] Updated weights for policy 0, policy_version 85820 (0.0009) +[2023-10-09 15:35:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 176095232. Throughput: 0: 1815.1, 1: 1817.1. Samples: 44026596. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:35:29,262][86122] Updated weights for policy 1, policy_version 86150 (0.0009) +[2023-10-09 15:35:29,614][86122] Updated weights for policy 1, policy_version 86160 (0.0008) +[2023-10-09 15:35:29,973][86122] Updated weights for policy 1, policy_version 86170 (0.0009) +[2023-10-09 15:35:31,124][86121] Updated weights for policy 0, policy_version 85830 (0.0009) +[2023-10-09 15:35:31,498][86121] Updated weights for policy 0, policy_version 85840 (0.0007) +[2023-10-09 15:35:31,867][86121] Updated weights for policy 0, policy_version 85850 (0.0010) +[2023-10-09 15:35:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 176160768. Throughput: 0: 1818.1, 1: 1813.2. Samples: 44047956. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:33,399][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:35:33,641][86122] Updated weights for policy 1, policy_version 86180 (0.0008) +[2023-10-09 15:35:34,003][86122] Updated weights for policy 1, policy_version 86190 (0.0010) +[2023-10-09 15:35:34,367][86122] Updated weights for policy 1, policy_version 86200 (0.0010) +[2023-10-09 15:35:35,485][86121] Updated weights for policy 0, policy_version 85860 (0.0011) +[2023-10-09 15:35:35,865][86121] Updated weights for policy 0, policy_version 85870 (0.0010) +[2023-10-09 15:35:36,225][86121] Updated weights for policy 0, policy_version 85880 (0.0009) +[2023-10-09 15:35:38,155][86122] Updated weights for policy 1, policy_version 86210 (0.0008) +[2023-10-09 15:35:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176226304. Throughput: 0: 1819.2, 1: 1812.7. Samples: 44070668. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:35:38,522][86122] Updated weights for policy 1, policy_version 86220 (0.0009) +[2023-10-09 15:35:38,882][86122] Updated weights for policy 1, policy_version 86230 (0.0008) +[2023-10-09 15:35:39,239][86122] Updated weights for policy 1, policy_version 86240 (0.0010) +[2023-10-09 15:35:39,876][86121] Updated weights for policy 0, policy_version 85890 (0.0009) +[2023-10-09 15:35:40,242][86121] Updated weights for policy 0, policy_version 85900 (0.0008) +[2023-10-09 15:35:40,616][86121] Updated weights for policy 0, policy_version 85910 (0.0009) +[2023-10-09 15:35:40,989][86121] Updated weights for policy 0, policy_version 85920 (0.0009) +[2023-10-09 15:35:42,948][86122] Updated weights for policy 1, policy_version 86250 (0.0008) +[2023-10-09 15:35:43,305][86122] Updated weights for policy 1, policy_version 86260 (0.0010) +[2023-10-09 15:35:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 176291840. Throughput: 0: 1823.2, 1: 1812.6. Samples: 44080766. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 15:35:43,674][86122] Updated weights for policy 1, policy_version 86270 (0.0011) +[2023-10-09 15:35:44,683][86121] Updated weights for policy 0, policy_version 85930 (0.0010) +[2023-10-09 15:35:45,050][86121] Updated weights for policy 0, policy_version 85940 (0.0009) +[2023-10-09 15:35:45,418][86121] Updated weights for policy 0, policy_version 85950 (0.0010) +[2023-10-09 15:35:47,227][86122] Updated weights for policy 1, policy_version 86280 (0.0008) +[2023-10-09 15:35:47,601][86122] Updated weights for policy 1, policy_version 86290 (0.0009) +[2023-10-09 15:35:47,956][86122] Updated weights for policy 1, policy_version 86300 (0.0008) +[2023-10-09 15:35:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176390144. Throughput: 0: 1820.7, 1: 1814.6. Samples: 44103344. Policy #0 lag: (min: 3.0, avg: 3.0, max: 3.0) +[2023-10-09 15:35:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 15:35:49,044][86121] Updated weights for policy 0, policy_version 85960 (0.0009) +[2023-10-09 15:35:49,417][86121] Updated weights for policy 0, policy_version 85970 (0.0008) +[2023-10-09 15:35:49,787][86121] Updated weights for policy 0, policy_version 85980 (0.0008) +[2023-10-09 15:35:51,689][86122] Updated weights for policy 1, policy_version 86310 (0.0008) +[2023-10-09 15:35:52,052][86122] Updated weights for policy 1, policy_version 86320 (0.0008) +[2023-10-09 15:35:52,414][86122] Updated weights for policy 1, policy_version 86330 (0.0007) +[2023-10-09 15:35:53,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176455680. Throughput: 0: 1812.9, 1: 1817.2. Samples: 44124630. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:35:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:35:53,522][86121] Updated weights for policy 0, policy_version 85990 (0.0007) +[2023-10-09 15:35:53,891][86121] Updated weights for policy 0, policy_version 86000 (0.0013) +[2023-10-09 15:35:54,255][86121] Updated weights for policy 0, policy_version 86010 (0.0008) +[2023-10-09 15:35:56,034][86122] Updated weights for policy 1, policy_version 86340 (0.0009) +[2023-10-09 15:35:56,389][86122] Updated weights for policy 1, policy_version 86350 (0.0010) +[2023-10-09 15:35:56,754][86122] Updated weights for policy 1, policy_version 86360 (0.0009) +[2023-10-09 15:35:57,903][86121] Updated weights for policy 0, policy_version 86020 (0.0009) +[2023-10-09 15:35:58,262][86121] Updated weights for policy 0, policy_version 86030 (0.0010) +[2023-10-09 15:35:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176521216. Throughput: 0: 1812.8, 1: 1823.5. Samples: 44136032. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:35:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:35:58,624][86121] Updated weights for policy 0, policy_version 86040 (0.0010) +[2023-10-09 15:36:00,322][86122] Updated weights for policy 1, policy_version 86370 (0.0009) +[2023-10-09 15:36:00,685][86122] Updated weights for policy 1, policy_version 86380 (0.0008) +[2023-10-09 15:36:01,051][86122] Updated weights for policy 1, policy_version 86390 (0.0008) +[2023-10-09 15:36:01,414][86122] Updated weights for policy 1, policy_version 86400 (0.0011) +[2023-10-09 15:36:02,360][86121] Updated weights for policy 0, policy_version 86050 (0.0009) +[2023-10-09 15:36:02,730][86121] Updated weights for policy 0, policy_version 86060 (0.0007) +[2023-10-09 15:36:03,098][86121] Updated weights for policy 0, policy_version 86070 (0.0007) +[2023-10-09 15:36:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176586752. Throughput: 0: 1817.6, 1: 1832.6. Samples: 44157834. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:36:03,469][86121] Updated weights for policy 0, policy_version 86080 (0.0008) +[2023-10-09 15:36:05,214][86122] Updated weights for policy 1, policy_version 86410 (0.0009) +[2023-10-09 15:36:05,584][86122] Updated weights for policy 1, policy_version 86420 (0.0008) +[2023-10-09 15:36:05,948][86122] Updated weights for policy 1, policy_version 86430 (0.0008) +[2023-10-09 15:36:07,168][86121] Updated weights for policy 0, policy_version 86090 (0.0009) +[2023-10-09 15:36:07,535][86121] Updated weights for policy 0, policy_version 86100 (0.0008) +[2023-10-09 15:36:07,910][86121] Updated weights for policy 0, policy_version 86110 (0.0008) +[2023-10-09 15:36:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176685056. Throughput: 0: 1812.6, 1: 1829.7. Samples: 44179220. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:36:09,484][86122] Updated weights for policy 1, policy_version 86440 (0.0008) +[2023-10-09 15:36:09,850][86122] Updated weights for policy 1, policy_version 86450 (0.0007) +[2023-10-09 15:36:10,205][86122] Updated weights for policy 1, policy_version 86460 (0.0008) +[2023-10-09 15:36:11,531][86121] Updated weights for policy 0, policy_version 86120 (0.0009) +[2023-10-09 15:36:11,891][86121] Updated weights for policy 0, policy_version 86130 (0.0008) +[2023-10-09 15:36:12,257][86121] Updated weights for policy 0, policy_version 86140 (0.0008) +[2023-10-09 15:36:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176750592. Throughput: 0: 1814.7, 1: 1826.3. Samples: 44190440. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:13,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:36:13,890][86122] Updated weights for policy 1, policy_version 86470 (0.0009) +[2023-10-09 15:36:14,240][86122] Updated weights for policy 1, policy_version 86480 (0.0009) +[2023-10-09 15:36:14,599][86122] Updated weights for policy 1, policy_version 86490 (0.0010) +[2023-10-09 15:36:15,878][86121] Updated weights for policy 0, policy_version 86150 (0.0007) +[2023-10-09 15:36:16,240][86121] Updated weights for policy 0, policy_version 86160 (0.0008) +[2023-10-09 15:36:16,614][86121] Updated weights for policy 0, policy_version 86170 (0.0009) +[2023-10-09 15:36:18,395][86122] Updated weights for policy 1, policy_version 86500 (0.0010) +[2023-10-09 15:36:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 176816128. Throughput: 0: 1813.5, 1: 1834.1. Samples: 44212096. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:36:18,746][86122] Updated weights for policy 1, policy_version 86510 (0.0008) +[2023-10-09 15:36:19,114][86122] Updated weights for policy 1, policy_version 86520 (0.0008) +[2023-10-09 15:36:20,477][86121] Updated weights for policy 0, policy_version 86180 (0.0009) +[2023-10-09 15:36:20,853][86121] Updated weights for policy 0, policy_version 86190 (0.0008) +[2023-10-09 15:36:21,220][86121] Updated weights for policy 0, policy_version 86200 (0.0008) +[2023-10-09 15:36:22,828][86122] Updated weights for policy 1, policy_version 86530 (0.0008) +[2023-10-09 15:36:23,193][86122] Updated weights for policy 1, policy_version 86540 (0.0007) +[2023-10-09 15:36:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 176881664. Throughput: 0: 1812.9, 1: 1826.7. Samples: 44234450. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:36:23,552][86122] Updated weights for policy 1, policy_version 86550 (0.0007) +[2023-10-09 15:36:23,910][86122] Updated weights for policy 1, policy_version 86560 (0.0009) +[2023-10-09 15:36:24,886][86121] Updated weights for policy 0, policy_version 86210 (0.0008) +[2023-10-09 15:36:25,245][86121] Updated weights for policy 0, policy_version 86220 (0.0008) +[2023-10-09 15:36:25,606][86121] Updated weights for policy 0, policy_version 86230 (0.0009) +[2023-10-09 15:36:25,972][86121] Updated weights for policy 0, policy_version 86240 (0.0010) +[2023-10-09 15:36:27,590][86122] Updated weights for policy 1, policy_version 86570 (0.0008) +[2023-10-09 15:36:27,954][86122] Updated weights for policy 1, policy_version 86580 (0.0007) +[2023-10-09 15:36:28,316][86122] Updated weights for policy 1, policy_version 86590 (0.0008) +[2023-10-09 15:36:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 176979968. Throughput: 0: 1812.1, 1: 1830.5. Samples: 44244684. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:36:29,720][86121] Updated weights for policy 0, policy_version 86250 (0.0007) +[2023-10-09 15:36:30,080][86121] Updated weights for policy 0, policy_version 86260 (0.0009) +[2023-10-09 15:36:30,448][86121] Updated weights for policy 0, policy_version 86270 (0.0008) +[2023-10-09 15:36:31,852][86122] Updated weights for policy 1, policy_version 86600 (0.0010) +[2023-10-09 15:36:32,215][86122] Updated weights for policy 1, policy_version 86610 (0.0008) +[2023-10-09 15:36:32,585][86122] Updated weights for policy 1, policy_version 86620 (0.0009) +[2023-10-09 15:36:33,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177045504. Throughput: 0: 1811.1, 1: 1828.2. Samples: 44267114. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:33,399][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:36:34,178][86121] Updated weights for policy 0, policy_version 86280 (0.0008) +[2023-10-09 15:36:34,552][86121] Updated weights for policy 0, policy_version 86290 (0.0007) +[2023-10-09 15:36:34,907][86121] Updated weights for policy 0, policy_version 86300 (0.0007) +[2023-10-09 15:36:36,370][86122] Updated weights for policy 1, policy_version 86630 (0.0008) +[2023-10-09 15:36:36,731][86122] Updated weights for policy 1, policy_version 86640 (0.0009) +[2023-10-09 15:36:37,097][86122] Updated weights for policy 1, policy_version 86650 (0.0008) +[2023-10-09 15:36:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177111040. Throughput: 0: 1811.9, 1: 1837.7. Samples: 44288864. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:36:38,634][86121] Updated weights for policy 0, policy_version 86310 (0.0009) +[2023-10-09 15:36:39,001][86121] Updated weights for policy 0, policy_version 86320 (0.0007) +[2023-10-09 15:36:39,370][86121] Updated weights for policy 0, policy_version 86330 (0.0008) +[2023-10-09 15:36:40,949][86122] Updated weights for policy 1, policy_version 86660 (0.0008) +[2023-10-09 15:36:41,314][86122] Updated weights for policy 1, policy_version 86670 (0.0008) +[2023-10-09 15:36:41,676][86122] Updated weights for policy 1, policy_version 86680 (0.0010) +[2023-10-09 15:36:43,064][86121] Updated weights for policy 0, policy_version 86340 (0.0008) +[2023-10-09 15:36:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177176576. Throughput: 0: 1816.9, 1: 1830.5. Samples: 44300164. Policy #0 lag: (min: 27.0, avg: 27.0, max: 27.0) +[2023-10-09 15:36:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:36:43,431][86121] Updated weights for policy 0, policy_version 86350 (0.0007) +[2023-10-09 15:36:43,804][86121] Updated weights for policy 0, policy_version 86360 (0.0008) +[2023-10-09 15:36:45,410][86122] Updated weights for policy 1, policy_version 86690 (0.0011) +[2023-10-09 15:36:45,777][86122] Updated weights for policy 1, policy_version 86700 (0.0010) +[2023-10-09 15:36:46,141][86122] Updated weights for policy 1, policy_version 86710 (0.0010) +[2023-10-09 15:36:46,500][86122] Updated weights for policy 1, policy_version 86720 (0.0009) +[2023-10-09 15:36:47,568][86121] Updated weights for policy 0, policy_version 86370 (0.0008) +[2023-10-09 15:36:47,928][86121] Updated weights for policy 0, policy_version 86380 (0.0008) +[2023-10-09 15:36:48,286][86121] Updated weights for policy 0, policy_version 86390 (0.0008) +[2023-10-09 15:36:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177242112. Throughput: 0: 1809.8, 1: 1826.0. Samples: 44321446. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:36:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:36:48,651][86121] Updated weights for policy 0, policy_version 86400 (0.0008) +[2023-10-09 15:36:50,408][86122] Updated weights for policy 1, policy_version 86730 (0.0009) +[2023-10-09 15:36:50,770][86122] Updated weights for policy 1, policy_version 86740 (0.0011) +[2023-10-09 15:36:51,138][86122] Updated weights for policy 1, policy_version 86750 (0.0011) +[2023-10-09 15:36:52,544][86121] Updated weights for policy 0, policy_version 86410 (0.0007) +[2023-10-09 15:36:52,906][86121] Updated weights for policy 0, policy_version 86420 (0.0007) +[2023-10-09 15:36:53,268][86121] Updated weights for policy 0, policy_version 86430 (0.0007) +[2023-10-09 15:36:53,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177340416. Throughput: 0: 1817.2, 1: 1823.7. Samples: 44343060. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:36:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:36:54,737][86122] Updated weights for policy 1, policy_version 86760 (0.0008) +[2023-10-09 15:36:55,104][86122] Updated weights for policy 1, policy_version 86770 (0.0008) +[2023-10-09 15:36:55,464][86122] Updated weights for policy 1, policy_version 86780 (0.0008) +[2023-10-09 15:36:56,935][86121] Updated weights for policy 0, policy_version 86440 (0.0008) +[2023-10-09 15:36:57,305][86121] Updated weights for policy 0, policy_version 86450 (0.0009) +[2023-10-09 15:36:57,670][86121] Updated weights for policy 0, policy_version 86460 (0.0011) +[2023-10-09 15:36:58,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177405952. Throughput: 0: 1809.7, 1: 1827.9. Samples: 44354130. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:36:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:36:58,984][86122] Updated weights for policy 1, policy_version 86790 (0.0008) +[2023-10-09 15:36:59,344][86122] Updated weights for policy 1, policy_version 86800 (0.0009) +[2023-10-09 15:36:59,711][86122] Updated weights for policy 1, policy_version 86810 (0.0010) +[2023-10-09 15:37:01,295][86121] Updated weights for policy 0, policy_version 86470 (0.0010) +[2023-10-09 15:37:01,657][86121] Updated weights for policy 0, policy_version 86480 (0.0009) +[2023-10-09 15:37:02,018][86121] Updated weights for policy 0, policy_version 86490 (0.0008) +[2023-10-09 15:37:03,319][86122] Updated weights for policy 1, policy_version 86820 (0.0009) +[2023-10-09 15:37:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177471488. Throughput: 0: 1814.2, 1: 1827.4. Samples: 44375968. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:37:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:37:03,677][86122] Updated weights for policy 1, policy_version 86830 (0.0010) +[2023-10-09 15:37:04,037][86122] Updated weights for policy 1, policy_version 86840 (0.0011) +[2023-10-09 15:37:05,876][86121] Updated weights for policy 0, policy_version 86500 (0.0010) +[2023-10-09 15:37:06,264][86121] Updated weights for policy 0, policy_version 86510 (0.0007) +[2023-10-09 15:37:06,633][86121] Updated weights for policy 0, policy_version 86520 (0.0007) +[2023-10-09 15:37:07,624][86122] Updated weights for policy 1, policy_version 86850 (0.0007) +[2023-10-09 15:37:07,979][86122] Updated weights for policy 1, policy_version 86860 (0.0008) +[2023-10-09 15:37:08,343][86122] Updated weights for policy 1, policy_version 86870 (0.0008) +[2023-10-09 15:37:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177537024. Throughput: 0: 1804.3, 1: 1832.2. Samples: 44398094. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:37:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:37:08,710][86122] Updated weights for policy 1, policy_version 86880 (0.0009) +[2023-10-09 15:37:10,224][86121] Updated weights for policy 0, policy_version 86530 (0.0008) +[2023-10-09 15:37:10,596][86121] Updated weights for policy 0, policy_version 86540 (0.0009) +[2023-10-09 15:37:10,950][86121] Updated weights for policy 0, policy_version 86550 (0.0010) +[2023-10-09 15:37:11,309][86121] Updated weights for policy 0, policy_version 86560 (0.0009) +[2023-10-09 15:37:12,426][86122] Updated weights for policy 1, policy_version 86890 (0.0009) +[2023-10-09 15:37:12,788][86122] Updated weights for policy 1, policy_version 86900 (0.0009) +[2023-10-09 15:37:13,161][86122] Updated weights for policy 1, policy_version 86910 (0.0009) +[2023-10-09 15:37:13,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177635328. Throughput: 0: 1813.8, 1: 1838.9. Samples: 44409058. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:37:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:37:15,054][86121] Updated weights for policy 0, policy_version 86570 (0.0007) +[2023-10-09 15:37:15,419][86121] Updated weights for policy 0, policy_version 86580 (0.0008) +[2023-10-09 15:37:15,780][86121] Updated weights for policy 0, policy_version 86590 (0.0008) +[2023-10-09 15:37:16,818][86122] Updated weights for policy 1, policy_version 86920 (0.0009) +[2023-10-09 15:37:17,178][86122] Updated weights for policy 1, policy_version 86930 (0.0007) +[2023-10-09 15:37:17,538][86122] Updated weights for policy 1, policy_version 86940 (0.0010) +[2023-10-09 15:37:18,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177700864. Throughput: 0: 1811.9, 1: 1832.3. Samples: 44431100. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:37:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:37:19,463][86121] Updated weights for policy 0, policy_version 86600 (0.0008) +[2023-10-09 15:37:19,829][86121] Updated weights for policy 0, policy_version 86610 (0.0008) +[2023-10-09 15:37:20,192][86121] Updated weights for policy 0, policy_version 86620 (0.0009) +[2023-10-09 15:37:21,184][86122] Updated weights for policy 1, policy_version 86950 (0.0008) +[2023-10-09 15:37:21,538][86122] Updated weights for policy 1, policy_version 86960 (0.0007) +[2023-10-09 15:37:21,897][86122] Updated weights for policy 1, policy_version 86970 (0.0008) +[2023-10-09 15:37:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 177766400. Throughput: 0: 1816.0, 1: 1832.0. Samples: 44453028. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:37:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:37:23,407][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000086624_88702976.pth... +[2023-10-09 15:37:23,407][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000086976_89063424.pth... +[2023-10-09 15:37:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000084928_86966272.pth +[2023-10-09 15:37:23,445][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000085280_87326720.pth +[2023-10-09 15:37:23,789][86121] Updated weights for policy 0, policy_version 86630 (0.0008) +[2023-10-09 15:37:24,150][86121] Updated weights for policy 0, policy_version 86640 (0.0009) +[2023-10-09 15:37:24,514][86121] Updated weights for policy 0, policy_version 86650 (0.0010) +[2023-10-09 15:37:25,780][86122] Updated weights for policy 1, policy_version 86980 (0.0008) +[2023-10-09 15:37:26,128][86122] Updated weights for policy 1, policy_version 86990 (0.0008) +[2023-10-09 15:37:26,494][86122] Updated weights for policy 1, policy_version 87000 (0.0009) +[2023-10-09 15:37:28,169][86121] Updated weights for policy 0, policy_version 86660 (0.0009) +[2023-10-09 15:37:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177831936. Throughput: 0: 1816.8, 1: 1829.4. Samples: 44464240. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:37:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:37:28,540][86121] Updated weights for policy 0, policy_version 86670 (0.0008) +[2023-10-09 15:37:28,901][86121] Updated weights for policy 0, policy_version 86680 (0.0007) +[2023-10-09 15:37:29,951][86122] Updated weights for policy 1, policy_version 87010 (0.0009) +[2023-10-09 15:37:30,312][86122] Updated weights for policy 1, policy_version 87020 (0.0009) +[2023-10-09 15:37:30,667][86122] Updated weights for policy 1, policy_version 87030 (0.0007) +[2023-10-09 15:37:31,026][86122] Updated weights for policy 1, policy_version 87040 (0.0008) +[2023-10-09 15:37:32,544][86121] Updated weights for policy 0, policy_version 86690 (0.0008) +[2023-10-09 15:37:32,904][86121] Updated weights for policy 0, policy_version 86700 (0.0007) +[2023-10-09 15:37:33,267][86121] Updated weights for policy 0, policy_version 86710 (0.0007) +[2023-10-09 15:37:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 177897472. Throughput: 0: 1820.6, 1: 1840.5. Samples: 44486198. Policy #0 lag: (min: 9.0, avg: 23.2, max: 41.0) +[2023-10-09 15:37:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:37:33,628][86121] Updated weights for policy 0, policy_version 86720 (0.0008) +[2023-10-09 15:37:34,672][86122] Updated weights for policy 1, policy_version 87050 (0.0011) +[2023-10-09 15:37:35,032][86122] Updated weights for policy 1, policy_version 87060 (0.0010) +[2023-10-09 15:37:35,398][86122] Updated weights for policy 1, policy_version 87070 (0.0011) +[2023-10-09 15:37:37,144][86121] Updated weights for policy 0, policy_version 86730 (0.0007) +[2023-10-09 15:37:37,517][86121] Updated weights for policy 0, policy_version 86740 (0.0010) +[2023-10-09 15:37:37,877][86121] Updated weights for policy 0, policy_version 86750 (0.0008) +[2023-10-09 15:37:38,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 177995776. Throughput: 0: 1817.8, 1: 1849.1. Samples: 44508070. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:37:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:37:39,116][86122] Updated weights for policy 1, policy_version 87080 (0.0010) +[2023-10-09 15:37:39,496][86122] Updated weights for policy 1, policy_version 87090 (0.0009) +[2023-10-09 15:37:39,859][86122] Updated weights for policy 1, policy_version 87100 (0.0008) +[2023-10-09 15:37:41,639][86121] Updated weights for policy 0, policy_version 86760 (0.0008) +[2023-10-09 15:37:41,999][86121] Updated weights for policy 0, policy_version 86770 (0.0010) +[2023-10-09 15:37:42,363][86121] Updated weights for policy 0, policy_version 86780 (0.0010) +[2023-10-09 15:37:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178061312. Throughput: 0: 1829.2, 1: 1840.8. Samples: 44519282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:37:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:37:43,509][86122] Updated weights for policy 1, policy_version 87110 (0.0010) +[2023-10-09 15:37:43,865][86122] Updated weights for policy 1, policy_version 87120 (0.0008) +[2023-10-09 15:37:44,236][86122] Updated weights for policy 1, policy_version 87130 (0.0008) +[2023-10-09 15:37:46,048][86121] Updated weights for policy 0, policy_version 86790 (0.0011) +[2023-10-09 15:37:46,407][86121] Updated weights for policy 0, policy_version 86800 (0.0010) +[2023-10-09 15:37:46,774][86121] Updated weights for policy 0, policy_version 86810 (0.0009) +[2023-10-09 15:37:47,798][86122] Updated weights for policy 1, policy_version 87140 (0.0008) +[2023-10-09 15:37:48,161][86122] Updated weights for policy 1, policy_version 87150 (0.0009) +[2023-10-09 15:37:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178126848. Throughput: 0: 1822.8, 1: 1842.7. Samples: 44540912. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:37:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:37:48,522][86122] Updated weights for policy 1, policy_version 87160 (0.0008) +[2023-10-09 15:37:50,578][86121] Updated weights for policy 0, policy_version 86820 (0.0011) +[2023-10-09 15:37:50,944][86121] Updated weights for policy 0, policy_version 86830 (0.0010) +[2023-10-09 15:37:51,317][86121] Updated weights for policy 0, policy_version 86840 (0.0009) +[2023-10-09 15:37:52,153][86122] Updated weights for policy 1, policy_version 87170 (0.0010) +[2023-10-09 15:37:52,517][86122] Updated weights for policy 1, policy_version 87180 (0.0008) +[2023-10-09 15:37:52,871][86122] Updated weights for policy 1, policy_version 87190 (0.0009) +[2023-10-09 15:37:53,233][86122] Updated weights for policy 1, policy_version 87200 (0.0007) +[2023-10-09 15:37:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178225152. Throughput: 0: 1830.1, 1: 1829.0. Samples: 44562752. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:37:53,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:37:55,002][86121] Updated weights for policy 0, policy_version 86850 (0.0010) +[2023-10-09 15:37:55,375][86121] Updated weights for policy 0, policy_version 86860 (0.0009) +[2023-10-09 15:37:55,744][86121] Updated weights for policy 0, policy_version 86870 (0.0011) +[2023-10-09 15:37:56,102][86121] Updated weights for policy 0, policy_version 86880 (0.0009) +[2023-10-09 15:37:56,749][86122] Updated weights for policy 1, policy_version 87210 (0.0008) +[2023-10-09 15:37:57,100][86122] Updated weights for policy 1, policy_version 87220 (0.0009) +[2023-10-09 15:37:57,459][86122] Updated weights for policy 1, policy_version 87230 (0.0012) +[2023-10-09 15:37:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178290688. Throughput: 0: 1823.3, 1: 1845.6. Samples: 44574160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:37:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:37:59,804][86121] Updated weights for policy 0, policy_version 86890 (0.0008) +[2023-10-09 15:38:00,171][86121] Updated weights for policy 0, policy_version 86900 (0.0008) +[2023-10-09 15:38:00,546][86121] Updated weights for policy 0, policy_version 86910 (0.0008) +[2023-10-09 15:38:00,992][86122] Updated weights for policy 1, policy_version 87240 (0.0008) +[2023-10-09 15:38:01,340][86122] Updated weights for policy 1, policy_version 87250 (0.0008) +[2023-10-09 15:38:01,700][86122] Updated weights for policy 1, policy_version 87260 (0.0008) +[2023-10-09 15:38:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178356224. Throughput: 0: 1828.5, 1: 1823.8. Samples: 44595452. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:38:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:38:04,392][86121] Updated weights for policy 0, policy_version 86920 (0.0007) +[2023-10-09 15:38:04,747][86121] Updated weights for policy 0, policy_version 86930 (0.0007) +[2023-10-09 15:38:05,120][86121] Updated weights for policy 0, policy_version 86940 (0.0007) +[2023-10-09 15:38:05,329][86122] Updated weights for policy 1, policy_version 87270 (0.0008) +[2023-10-09 15:38:05,694][86122] Updated weights for policy 1, policy_version 87280 (0.0007) +[2023-10-09 15:38:06,049][86122] Updated weights for policy 1, policy_version 87290 (0.0009) +[2023-10-09 15:38:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 178421760. Throughput: 0: 1819.2, 1: 1850.4. Samples: 44618160. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:38:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:38:08,861][86121] Updated weights for policy 0, policy_version 86950 (0.0008) +[2023-10-09 15:38:09,228][86121] Updated weights for policy 0, policy_version 86960 (0.0009) +[2023-10-09 15:38:09,580][86121] Updated weights for policy 0, policy_version 86970 (0.0008) +[2023-10-09 15:38:09,800][86122] Updated weights for policy 1, policy_version 87300 (0.0008) +[2023-10-09 15:38:10,157][86122] Updated weights for policy 1, policy_version 87310 (0.0011) +[2023-10-09 15:38:10,519][86122] Updated weights for policy 1, policy_version 87320 (0.0010) +[2023-10-09 15:38:13,022][86121] Updated weights for policy 0, policy_version 86980 (0.0007) +[2023-10-09 15:38:13,394][86121] Updated weights for policy 0, policy_version 86990 (0.0009) +[2023-10-09 15:38:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 178487296. Throughput: 0: 1819.7, 1: 1824.9. Samples: 44628250. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:38:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:38:13,759][86121] Updated weights for policy 0, policy_version 87000 (0.0009) +[2023-10-09 15:38:14,262][86122] Updated weights for policy 1, policy_version 87330 (0.0010) +[2023-10-09 15:38:14,625][86122] Updated weights for policy 1, policy_version 87340 (0.0010) +[2023-10-09 15:38:14,983][86122] Updated weights for policy 1, policy_version 87350 (0.0010) +[2023-10-09 15:38:15,347][86122] Updated weights for policy 1, policy_version 87360 (0.0008) +[2023-10-09 15:38:17,329][86121] Updated weights for policy 0, policy_version 87010 (0.0009) +[2023-10-09 15:38:17,699][86121] Updated weights for policy 0, policy_version 87020 (0.0010) +[2023-10-09 15:38:18,067][86121] Updated weights for policy 0, policy_version 87030 (0.0009) +[2023-10-09 15:38:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 178552832. Throughput: 0: 1824.2, 1: 1840.8. Samples: 44651124. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:38:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:38:18,433][86121] Updated weights for policy 0, policy_version 87040 (0.0009) +[2023-10-09 15:38:18,916][86122] Updated weights for policy 1, policy_version 87370 (0.0008) +[2023-10-09 15:38:19,276][86122] Updated weights for policy 1, policy_version 87380 (0.0010) +[2023-10-09 15:38:19,635][86122] Updated weights for policy 1, policy_version 87390 (0.0010) +[2023-10-09 15:38:22,176][86121] Updated weights for policy 0, policy_version 87050 (0.0009) +[2023-10-09 15:38:22,543][86121] Updated weights for policy 0, policy_version 87060 (0.0008) +[2023-10-09 15:38:22,918][86121] Updated weights for policy 0, policy_version 87070 (0.0008) +[2023-10-09 15:38:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178651136. Throughput: 0: 1821.5, 1: 1838.8. Samples: 44672786. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:38:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:38:23,454][86122] Updated weights for policy 1, policy_version 87400 (0.0009) +[2023-10-09 15:38:23,809][86122] Updated weights for policy 1, policy_version 87410 (0.0009) +[2023-10-09 15:38:24,172][86122] Updated weights for policy 1, policy_version 87420 (0.0007) +[2023-10-09 15:38:26,572][86121] Updated weights for policy 0, policy_version 87080 (0.0009) +[2023-10-09 15:38:26,935][86121] Updated weights for policy 0, policy_version 87090 (0.0009) +[2023-10-09 15:38:27,306][86121] Updated weights for policy 0, policy_version 87100 (0.0007) +[2023-10-09 15:38:27,955][86122] Updated weights for policy 1, policy_version 87430 (0.0008) +[2023-10-09 15:38:28,319][86122] Updated weights for policy 1, policy_version 87440 (0.0010) +[2023-10-09 15:38:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178716672. Throughput: 0: 1817.3, 1: 1842.0. Samples: 44683952. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:38:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:38:28,681][86122] Updated weights for policy 1, policy_version 87450 (0.0009) +[2023-10-09 15:38:31,037][86121] Updated weights for policy 0, policy_version 87110 (0.0008) +[2023-10-09 15:38:31,401][86121] Updated weights for policy 0, policy_version 87120 (0.0010) +[2023-10-09 15:38:31,778][86121] Updated weights for policy 0, policy_version 87130 (0.0009) +[2023-10-09 15:38:32,439][86122] Updated weights for policy 1, policy_version 87460 (0.0009) +[2023-10-09 15:38:32,802][86122] Updated weights for policy 1, policy_version 87470 (0.0010) +[2023-10-09 15:38:33,166][86122] Updated weights for policy 1, policy_version 87480 (0.0008) +[2023-10-09 15:38:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 178782208. Throughput: 0: 1825.5, 1: 1836.4. Samples: 44705700. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:38:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:38:35,382][86121] Updated weights for policy 0, policy_version 87140 (0.0008) +[2023-10-09 15:38:35,748][86121] Updated weights for policy 0, policy_version 87150 (0.0008) +[2023-10-09 15:38:36,111][86121] Updated weights for policy 0, policy_version 87160 (0.0008) +[2023-10-09 15:38:36,899][86122] Updated weights for policy 1, policy_version 87490 (0.0008) +[2023-10-09 15:38:37,266][86122] Updated weights for policy 1, policy_version 87500 (0.0011) +[2023-10-09 15:38:37,637][86122] Updated weights for policy 1, policy_version 87510 (0.0009) +[2023-10-09 15:38:37,996][86122] Updated weights for policy 1, policy_version 87520 (0.0011) +[2023-10-09 15:38:38,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 178880512. Throughput: 0: 1839.6, 1: 1823.5. Samples: 44727594. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:38:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:38:39,723][86121] Updated weights for policy 0, policy_version 87170 (0.0009) +[2023-10-09 15:38:40,131][86121] Updated weights for policy 0, policy_version 87180 (0.0009) +[2023-10-09 15:38:40,490][86121] Updated weights for policy 0, policy_version 87190 (0.0008) +[2023-10-09 15:38:40,853][86121] Updated weights for policy 0, policy_version 87200 (0.0008) +[2023-10-09 15:38:41,732][86122] Updated weights for policy 1, policy_version 87530 (0.0009) +[2023-10-09 15:38:42,100][86122] Updated weights for policy 1, policy_version 87540 (0.0008) +[2023-10-09 15:38:42,474][86122] Updated weights for policy 1, policy_version 87550 (0.0007) +[2023-10-09 15:38:43,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 178946048. Throughput: 0: 1830.2, 1: 1823.4. Samples: 44738574. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:38:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:38:44,794][86121] Updated weights for policy 0, policy_version 87210 (0.0009) +[2023-10-09 15:38:45,153][86121] Updated weights for policy 0, policy_version 87220 (0.0008) +[2023-10-09 15:38:45,520][86121] Updated weights for policy 0, policy_version 87230 (0.0009) +[2023-10-09 15:38:46,267][86122] Updated weights for policy 1, policy_version 87560 (0.0010) +[2023-10-09 15:38:46,620][86122] Updated weights for policy 1, policy_version 87570 (0.0010) +[2023-10-09 15:38:46,980][86122] Updated weights for policy 1, policy_version 87580 (0.0011) +[2023-10-09 15:38:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179011584. Throughput: 0: 1829.2, 1: 1827.2. Samples: 44759994. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:38:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 15:38:49,121][86121] Updated weights for policy 0, policy_version 87240 (0.0008) +[2023-10-09 15:38:49,482][86121] Updated weights for policy 0, policy_version 87250 (0.0007) +[2023-10-09 15:38:49,848][86121] Updated weights for policy 0, policy_version 87260 (0.0008) +[2023-10-09 15:38:50,693][86122] Updated weights for policy 1, policy_version 87590 (0.0009) +[2023-10-09 15:38:51,052][86122] Updated weights for policy 1, policy_version 87600 (0.0007) +[2023-10-09 15:38:51,413][86122] Updated weights for policy 1, policy_version 87610 (0.0007) +[2023-10-09 15:38:53,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 179077120. Throughput: 0: 1828.2, 1: 1820.6. Samples: 44782356. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:38:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 15:38:53,549][86121] Updated weights for policy 0, policy_version 87270 (0.0008) +[2023-10-09 15:38:53,915][86121] Updated weights for policy 0, policy_version 87280 (0.0010) +[2023-10-09 15:38:54,285][86121] Updated weights for policy 0, policy_version 87290 (0.0007) +[2023-10-09 15:38:55,057][86122] Updated weights for policy 1, policy_version 87620 (0.0010) +[2023-10-09 15:38:55,429][86122] Updated weights for policy 1, policy_version 87630 (0.0009) +[2023-10-09 15:38:55,781][86122] Updated weights for policy 1, policy_version 87640 (0.0009) +[2023-10-09 15:38:58,172][86121] Updated weights for policy 0, policy_version 87300 (0.0007) +[2023-10-09 15:38:58,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179142656. Throughput: 0: 1825.4, 1: 1826.7. Samples: 44792594. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:38:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 15:38:58,528][86121] Updated weights for policy 0, policy_version 87310 (0.0007) +[2023-10-09 15:38:58,895][86121] Updated weights for policy 0, policy_version 87320 (0.0007) +[2023-10-09 15:38:59,449][86122] Updated weights for policy 1, policy_version 87650 (0.0007) +[2023-10-09 15:38:59,812][86122] Updated weights for policy 1, policy_version 87660 (0.0011) +[2023-10-09 15:39:00,169][86122] Updated weights for policy 1, policy_version 87670 (0.0009) +[2023-10-09 15:39:00,532][86122] Updated weights for policy 1, policy_version 87680 (0.0010) +[2023-10-09 15:39:02,769][86121] Updated weights for policy 0, policy_version 87330 (0.0009) +[2023-10-09 15:39:03,132][86121] Updated weights for policy 0, policy_version 87340 (0.0010) +[2023-10-09 15:39:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 179208192. Throughput: 0: 1816.1, 1: 1823.1. Samples: 44814888. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:39:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:39:03,504][86121] Updated weights for policy 0, policy_version 87350 (0.0010) +[2023-10-09 15:39:03,876][86121] Updated weights for policy 0, policy_version 87360 (0.0008) +[2023-10-09 15:39:04,218][86122] Updated weights for policy 1, policy_version 87690 (0.0008) +[2023-10-09 15:39:04,580][86122] Updated weights for policy 1, policy_version 87700 (0.0011) +[2023-10-09 15:39:04,943][86122] Updated weights for policy 1, policy_version 87710 (0.0009) +[2023-10-09 15:39:07,571][86121] Updated weights for policy 0, policy_version 87370 (0.0007) +[2023-10-09 15:39:07,934][86121] Updated weights for policy 0, policy_version 87380 (0.0008) +[2023-10-09 15:39:08,311][86121] Updated weights for policy 0, policy_version 87390 (0.0010) +[2023-10-09 15:39:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 179306496. Throughput: 0: 1824.7, 1: 1821.7. Samples: 44836872. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:39:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:39:08,658][86122] Updated weights for policy 1, policy_version 87720 (0.0009) +[2023-10-09 15:39:09,022][86122] Updated weights for policy 1, policy_version 87730 (0.0009) +[2023-10-09 15:39:09,379][86122] Updated weights for policy 1, policy_version 87740 (0.0011) +[2023-10-09 15:39:11,931][86121] Updated weights for policy 0, policy_version 87400 (0.0010) +[2023-10-09 15:39:12,299][86121] Updated weights for policy 0, policy_version 87410 (0.0008) +[2023-10-09 15:39:12,672][86121] Updated weights for policy 0, policy_version 87420 (0.0009) +[2023-10-09 15:39:13,268][86122] Updated weights for policy 1, policy_version 87750 (0.0008) +[2023-10-09 15:39:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179372032. Throughput: 0: 1814.2, 1: 1823.5. Samples: 44847648. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:39:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:13,646][86122] Updated weights for policy 1, policy_version 87760 (0.0007) +[2023-10-09 15:39:14,016][86122] Updated weights for policy 1, policy_version 87770 (0.0008) +[2023-10-09 15:39:16,220][86121] Updated weights for policy 0, policy_version 87430 (0.0009) +[2023-10-09 15:39:16,585][86121] Updated weights for policy 0, policy_version 87440 (0.0008) +[2023-10-09 15:39:16,946][86121] Updated weights for policy 0, policy_version 87450 (0.0008) +[2023-10-09 15:39:17,663][86122] Updated weights for policy 1, policy_version 87780 (0.0010) +[2023-10-09 15:39:18,031][86122] Updated weights for policy 1, policy_version 87790 (0.0008) +[2023-10-09 15:39:18,382][86122] Updated weights for policy 1, policy_version 87800 (0.0008) +[2023-10-09 15:39:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179437568. Throughput: 0: 1815.9, 1: 1817.4. Samples: 44869198. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:39:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:20,709][86121] Updated weights for policy 0, policy_version 87460 (0.0009) +[2023-10-09 15:39:21,084][86121] Updated weights for policy 0, policy_version 87470 (0.0011) +[2023-10-09 15:39:21,460][86121] Updated weights for policy 0, policy_version 87480 (0.0007) +[2023-10-09 15:39:22,041][86122] Updated weights for policy 1, policy_version 87810 (0.0008) +[2023-10-09 15:39:22,412][86122] Updated weights for policy 1, policy_version 87820 (0.0008) +[2023-10-09 15:39:22,772][86122] Updated weights for policy 1, policy_version 87830 (0.0009) +[2023-10-09 15:39:23,138][86122] Updated weights for policy 1, policy_version 87840 (0.0009) +[2023-10-09 15:39:23,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179535872. Throughput: 0: 1803.7, 1: 1821.8. Samples: 44890742. Policy #0 lag: (min: 31.0, avg: 31.1, max: 37.0) +[2023-10-09 15:39:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:23,408][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth... +[2023-10-09 15:39:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000087840_89948160.pth... +[2023-10-09 15:39:23,454][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000085792_87851008.pth +[2023-10-09 15:39:23,455][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000086112_88178688.pth +[2023-10-09 15:39:25,086][86121] Updated weights for policy 0, policy_version 87490 (0.0008) +[2023-10-09 15:39:25,492][86121] Updated weights for policy 0, policy_version 87500 (0.0008) +[2023-10-09 15:39:25,856][86121] Updated weights for policy 0, policy_version 87510 (0.0009) +[2023-10-09 15:39:26,220][86121] Updated weights for policy 0, policy_version 87520 (0.0009) +[2023-10-09 15:39:26,603][86122] Updated weights for policy 1, policy_version 87850 (0.0009) +[2023-10-09 15:39:26,967][86122] Updated weights for policy 1, policy_version 87860 (0.0008) +[2023-10-09 15:39:27,317][86122] Updated weights for policy 1, policy_version 87870 (0.0008) +[2023-10-09 15:39:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179601408. Throughput: 0: 1819.9, 1: 1824.7. Samples: 44902580. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:39:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:29,757][86121] Updated weights for policy 0, policy_version 87530 (0.0008) +[2023-10-09 15:39:30,128][86121] Updated weights for policy 0, policy_version 87540 (0.0007) +[2023-10-09 15:39:30,486][86121] Updated weights for policy 0, policy_version 87550 (0.0007) +[2023-10-09 15:39:30,946][86122] Updated weights for policy 1, policy_version 87880 (0.0009) +[2023-10-09 15:39:31,302][86122] Updated weights for policy 1, policy_version 87890 (0.0009) +[2023-10-09 15:39:31,673][86122] Updated weights for policy 1, policy_version 87900 (0.0008) +[2023-10-09 15:39:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 179666944. Throughput: 0: 1817.7, 1: 1824.5. Samples: 44923892. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:39:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:34,262][86121] Updated weights for policy 0, policy_version 87560 (0.0008) +[2023-10-09 15:39:34,630][86121] Updated weights for policy 0, policy_version 87570 (0.0007) +[2023-10-09 15:39:34,996][86121] Updated weights for policy 0, policy_version 87580 (0.0007) +[2023-10-09 15:39:35,326][86122] Updated weights for policy 1, policy_version 87910 (0.0011) +[2023-10-09 15:39:35,687][86122] Updated weights for policy 1, policy_version 87920 (0.0012) +[2023-10-09 15:39:36,043][86122] Updated weights for policy 1, policy_version 87930 (0.0010) +[2023-10-09 15:39:38,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179732480. Throughput: 0: 1819.5, 1: 1832.2. Samples: 44946682. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:39:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:38,639][86121] Updated weights for policy 0, policy_version 87590 (0.0009) +[2023-10-09 15:39:38,998][86121] Updated weights for policy 0, policy_version 87600 (0.0010) +[2023-10-09 15:39:39,359][86121] Updated weights for policy 0, policy_version 87610 (0.0009) +[2023-10-09 15:39:39,643][86122] Updated weights for policy 1, policy_version 87940 (0.0010) +[2023-10-09 15:39:39,995][86122] Updated weights for policy 1, policy_version 87950 (0.0008) +[2023-10-09 15:39:40,360][86122] Updated weights for policy 1, policy_version 87960 (0.0010) +[2023-10-09 15:39:42,921][86121] Updated weights for policy 0, policy_version 87620 (0.0007) +[2023-10-09 15:39:43,292][86121] Updated weights for policy 0, policy_version 87630 (0.0007) +[2023-10-09 15:39:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179798016. Throughput: 0: 1822.2, 1: 1824.4. Samples: 44956690. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:39:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:43,662][86121] Updated weights for policy 0, policy_version 87640 (0.0008) +[2023-10-09 15:39:44,013][86122] Updated weights for policy 1, policy_version 87970 (0.0011) +[2023-10-09 15:39:44,376][86122] Updated weights for policy 1, policy_version 87980 (0.0009) +[2023-10-09 15:39:44,742][86122] Updated weights for policy 1, policy_version 87990 (0.0008) +[2023-10-09 15:39:45,108][86122] Updated weights for policy 1, policy_version 88000 (0.0009) +[2023-10-09 15:39:47,417][86121] Updated weights for policy 0, policy_version 87650 (0.0008) +[2023-10-09 15:39:47,784][86121] Updated weights for policy 0, policy_version 87660 (0.0011) +[2023-10-09 15:39:48,147][86121] Updated weights for policy 0, policy_version 87670 (0.0010) +[2023-10-09 15:39:48,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 179863552. Throughput: 0: 1830.7, 1: 1832.4. Samples: 44979726. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:39:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:39:48,510][86121] Updated weights for policy 0, policy_version 87680 (0.0010) +[2023-10-09 15:39:48,899][86122] Updated weights for policy 1, policy_version 88010 (0.0009) +[2023-10-09 15:39:49,264][86122] Updated weights for policy 1, policy_version 88020 (0.0007) +[2023-10-09 15:39:49,627][86122] Updated weights for policy 1, policy_version 88030 (0.0007) +[2023-10-09 15:39:52,391][86121] Updated weights for policy 0, policy_version 87690 (0.0008) +[2023-10-09 15:39:52,758][86121] Updated weights for policy 0, policy_version 87700 (0.0008) +[2023-10-09 15:39:53,126][86121] Updated weights for policy 0, policy_version 87710 (0.0007) +[2023-10-09 15:39:53,288][86122] Updated weights for policy 1, policy_version 88040 (0.0007) +[2023-10-09 15:39:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 179961856. Throughput: 0: 1821.4, 1: 1829.6. Samples: 45001164. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:39:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:39:53,657][86122] Updated weights for policy 1, policy_version 88050 (0.0008) +[2023-10-09 15:39:54,017][86122] Updated weights for policy 1, policy_version 88060 (0.0009) +[2023-10-09 15:39:56,784][86121] Updated weights for policy 0, policy_version 87720 (0.0008) +[2023-10-09 15:39:57,147][86121] Updated weights for policy 0, policy_version 87730 (0.0008) +[2023-10-09 15:39:57,518][86121] Updated weights for policy 0, policy_version 87740 (0.0007) +[2023-10-09 15:39:57,704][86122] Updated weights for policy 1, policy_version 88070 (0.0010) +[2023-10-09 15:39:58,065][86122] Updated weights for policy 1, policy_version 88080 (0.0009) +[2023-10-09 15:39:58,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180027392. Throughput: 0: 1825.1, 1: 1831.7. Samples: 45012202. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:39:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:39:58,427][86122] Updated weights for policy 1, policy_version 88090 (0.0009) +[2023-10-09 15:40:01,161][86121] Updated weights for policy 0, policy_version 87750 (0.0007) +[2023-10-09 15:40:01,516][86121] Updated weights for policy 0, policy_version 87760 (0.0009) +[2023-10-09 15:40:01,883][86121] Updated weights for policy 0, policy_version 87770 (0.0007) +[2023-10-09 15:40:02,211][86122] Updated weights for policy 1, policy_version 88100 (0.0008) +[2023-10-09 15:40:02,603][86122] Updated weights for policy 1, policy_version 88110 (0.0008) +[2023-10-09 15:40:02,963][86122] Updated weights for policy 1, policy_version 88120 (0.0010) +[2023-10-09 15:40:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 180125696. Throughput: 0: 1819.8, 1: 1835.6. Samples: 45033688. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:40:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:40:05,367][86121] Updated weights for policy 0, policy_version 87780 (0.0009) +[2023-10-09 15:40:05,734][86121] Updated weights for policy 0, policy_version 87790 (0.0009) +[2023-10-09 15:40:06,100][86121] Updated weights for policy 0, policy_version 87800 (0.0010) +[2023-10-09 15:40:06,647][86122] Updated weights for policy 1, policy_version 88130 (0.0010) +[2023-10-09 15:40:07,017][86122] Updated weights for policy 1, policy_version 88140 (0.0008) +[2023-10-09 15:40:07,384][86122] Updated weights for policy 1, policy_version 88150 (0.0009) +[2023-10-09 15:40:07,739][86122] Updated weights for policy 1, policy_version 88160 (0.0009) +[2023-10-09 15:40:08,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180191232. Throughput: 0: 1822.1, 1: 1825.3. Samples: 45054876. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:40:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:40:09,767][86121] Updated weights for policy 0, policy_version 87810 (0.0009) +[2023-10-09 15:40:10,174][86121] Updated weights for policy 0, policy_version 87820 (0.0008) +[2023-10-09 15:40:10,546][86121] Updated weights for policy 0, policy_version 87830 (0.0007) +[2023-10-09 15:40:10,904][86121] Updated weights for policy 0, policy_version 87840 (0.0008) +[2023-10-09 15:40:11,179][86122] Updated weights for policy 1, policy_version 88170 (0.0008) +[2023-10-09 15:40:11,541][86122] Updated weights for policy 1, policy_version 88180 (0.0010) +[2023-10-09 15:40:11,900][86122] Updated weights for policy 1, policy_version 88190 (0.0007) +[2023-10-09 15:40:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180256768. Throughput: 0: 1807.9, 1: 1835.9. Samples: 45066554. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:40:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:40:14,635][86121] Updated weights for policy 0, policy_version 87850 (0.0008) +[2023-10-09 15:40:14,998][86121] Updated weights for policy 0, policy_version 87860 (0.0010) +[2023-10-09 15:40:15,370][86121] Updated weights for policy 0, policy_version 87870 (0.0010) +[2023-10-09 15:40:15,666][86122] Updated weights for policy 1, policy_version 88200 (0.0009) +[2023-10-09 15:40:16,027][86122] Updated weights for policy 1, policy_version 88210 (0.0009) +[2023-10-09 15:40:16,391][86122] Updated weights for policy 1, policy_version 88220 (0.0007) +[2023-10-09 15:40:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180322304. Throughput: 0: 1819.0, 1: 1826.8. Samples: 45087954. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:40:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:40:19,122][86121] Updated weights for policy 0, policy_version 87880 (0.0009) +[2023-10-09 15:40:19,493][86121] Updated weights for policy 0, policy_version 87890 (0.0008) +[2023-10-09 15:40:19,848][86121] Updated weights for policy 0, policy_version 87900 (0.0008) +[2023-10-09 15:40:20,136][86122] Updated weights for policy 1, policy_version 88230 (0.0007) +[2023-10-09 15:40:20,498][86122] Updated weights for policy 1, policy_version 88240 (0.0009) +[2023-10-09 15:40:20,865][86122] Updated weights for policy 1, policy_version 88250 (0.0008) +[2023-10-09 15:40:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 180387840. Throughput: 0: 1819.4, 1: 1827.4. Samples: 45110788. Policy #0 lag: (min: 30.0, avg: 37.6, max: 62.0) +[2023-10-09 15:40:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:40:23,653][86121] Updated weights for policy 0, policy_version 87910 (0.0008) +[2023-10-09 15:40:24,023][86121] Updated weights for policy 0, policy_version 87920 (0.0011) +[2023-10-09 15:40:24,393][86121] Updated weights for policy 0, policy_version 87930 (0.0008) +[2023-10-09 15:40:24,597][86122] Updated weights for policy 1, policy_version 88260 (0.0007) +[2023-10-09 15:40:24,955][86122] Updated weights for policy 1, policy_version 88270 (0.0007) +[2023-10-09 15:40:25,317][86122] Updated weights for policy 1, policy_version 88280 (0.0008) +[2023-10-09 15:40:27,941][86121] Updated weights for policy 0, policy_version 87940 (0.0007) +[2023-10-09 15:40:28,294][86121] Updated weights for policy 0, policy_version 87950 (0.0009) +[2023-10-09 15:40:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180453376. Throughput: 0: 1819.2, 1: 1825.1. Samples: 45120686. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:40:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:40:28,654][86121] Updated weights for policy 0, policy_version 87960 (0.0007) +[2023-10-09 15:40:28,980][86122] Updated weights for policy 1, policy_version 88290 (0.0010) +[2023-10-09 15:40:29,341][86122] Updated weights for policy 1, policy_version 88300 (0.0007) +[2023-10-09 15:40:29,709][86122] Updated weights for policy 1, policy_version 88310 (0.0009) +[2023-10-09 15:40:30,062][86122] Updated weights for policy 1, policy_version 88320 (0.0008) +[2023-10-09 15:40:32,384][86121] Updated weights for policy 0, policy_version 87970 (0.0008) +[2023-10-09 15:40:32,753][86121] Updated weights for policy 0, policy_version 87980 (0.0009) +[2023-10-09 15:40:33,115][86121] Updated weights for policy 0, policy_version 87990 (0.0009) +[2023-10-09 15:40:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180518912. Throughput: 0: 1816.2, 1: 1830.4. Samples: 45143824. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:40:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:40:33,486][86121] Updated weights for policy 0, policy_version 88000 (0.0008) +[2023-10-09 15:40:33,715][86122] Updated weights for policy 1, policy_version 88330 (0.0007) +[2023-10-09 15:40:34,075][86122] Updated weights for policy 1, policy_version 88340 (0.0009) +[2023-10-09 15:40:34,440][86122] Updated weights for policy 1, policy_version 88350 (0.0007) +[2023-10-09 15:40:37,236][86121] Updated weights for policy 0, policy_version 88010 (0.0010) +[2023-10-09 15:40:37,612][86121] Updated weights for policy 0, policy_version 88020 (0.0010) +[2023-10-09 15:40:37,979][86121] Updated weights for policy 0, policy_version 88030 (0.0009) +[2023-10-09 15:40:38,268][86122] Updated weights for policy 1, policy_version 88360 (0.0010) +[2023-10-09 15:40:38,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180617216. Throughput: 0: 1815.9, 1: 1829.3. Samples: 45165198. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:40:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:40:38,625][86122] Updated weights for policy 1, policy_version 88370 (0.0008) +[2023-10-09 15:40:38,983][86122] Updated weights for policy 1, policy_version 88380 (0.0011) +[2023-10-09 15:40:41,810][86121] Updated weights for policy 0, policy_version 88040 (0.0009) +[2023-10-09 15:40:42,175][86121] Updated weights for policy 0, policy_version 88050 (0.0009) +[2023-10-09 15:40:42,472][86122] Updated weights for policy 1, policy_version 88390 (0.0008) +[2023-10-09 15:40:42,537][86121] Updated weights for policy 0, policy_version 88060 (0.0008) +[2023-10-09 15:40:42,825][86122] Updated weights for policy 1, policy_version 88400 (0.0010) +[2023-10-09 15:40:43,191][86122] Updated weights for policy 1, policy_version 88410 (0.0008) +[2023-10-09 15:40:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 180682752. Throughput: 0: 1818.0, 1: 1831.4. Samples: 45176424. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:40:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:40:46,288][86121] Updated weights for policy 0, policy_version 88070 (0.0010) +[2023-10-09 15:40:46,650][86121] Updated weights for policy 0, policy_version 88080 (0.0007) +[2023-10-09 15:40:46,820][86122] Updated weights for policy 1, policy_version 88420 (0.0008) +[2023-10-09 15:40:47,012][86121] Updated weights for policy 0, policy_version 88090 (0.0008) +[2023-10-09 15:40:47,194][86122] Updated weights for policy 1, policy_version 88430 (0.0007) +[2023-10-09 15:40:47,561][86122] Updated weights for policy 1, policy_version 88440 (0.0009) +[2023-10-09 15:40:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 180781056. Throughput: 0: 1823.7, 1: 1830.4. Samples: 45198120. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:40:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:40:50,491][86121] Updated weights for policy 0, policy_version 88100 (0.0009) +[2023-10-09 15:40:50,852][86121] Updated weights for policy 0, policy_version 88110 (0.0009) +[2023-10-09 15:40:51,215][86121] Updated weights for policy 0, policy_version 88120 (0.0007) +[2023-10-09 15:40:51,302][86122] Updated weights for policy 1, policy_version 88450 (0.0009) +[2023-10-09 15:40:51,657][86122] Updated weights for policy 1, policy_version 88460 (0.0007) +[2023-10-09 15:40:52,018][86122] Updated weights for policy 1, policy_version 88470 (0.0007) +[2023-10-09 15:40:52,381][86122] Updated weights for policy 1, policy_version 88480 (0.0007) +[2023-10-09 15:40:53,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180846592. Throughput: 0: 1817.0, 1: 1836.7. Samples: 45219294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:40:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:40:55,032][86121] Updated weights for policy 0, policy_version 88130 (0.0008) +[2023-10-09 15:40:55,422][86121] Updated weights for policy 0, policy_version 88140 (0.0010) +[2023-10-09 15:40:55,796][86121] Updated weights for policy 0, policy_version 88150 (0.0012) +[2023-10-09 15:40:56,033][86122] Updated weights for policy 1, policy_version 88490 (0.0007) +[2023-10-09 15:40:56,151][86121] Updated weights for policy 0, policy_version 88160 (0.0007) +[2023-10-09 15:40:56,402][86122] Updated weights for policy 1, policy_version 88500 (0.0008) +[2023-10-09 15:40:56,754][86122] Updated weights for policy 1, policy_version 88510 (0.0007) +[2023-10-09 15:40:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 180912128. Throughput: 0: 1824.0, 1: 1826.9. Samples: 45230848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:40:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:40:59,879][86121] Updated weights for policy 0, policy_version 88170 (0.0007) +[2023-10-09 15:41:00,246][86121] Updated weights for policy 0, policy_version 88180 (0.0008) +[2023-10-09 15:41:00,473][86122] Updated weights for policy 1, policy_version 88520 (0.0008) +[2023-10-09 15:41:00,617][86121] Updated weights for policy 0, policy_version 88190 (0.0010) +[2023-10-09 15:41:00,837][86122] Updated weights for policy 1, policy_version 88530 (0.0010) +[2023-10-09 15:41:01,195][86122] Updated weights for policy 1, policy_version 88540 (0.0009) +[2023-10-09 15:41:03,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 180977664. Throughput: 0: 1818.0, 1: 1835.5. Samples: 45252366. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:41:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:41:04,226][86121] Updated weights for policy 0, policy_version 88200 (0.0008) +[2023-10-09 15:41:04,601][86121] Updated weights for policy 0, policy_version 88210 (0.0008) +[2023-10-09 15:41:04,880][86122] Updated weights for policy 1, policy_version 88550 (0.0007) +[2023-10-09 15:41:04,967][86121] Updated weights for policy 0, policy_version 88220 (0.0008) +[2023-10-09 15:41:05,232][86122] Updated weights for policy 1, policy_version 88560 (0.0010) +[2023-10-09 15:41:05,598][86122] Updated weights for policy 1, policy_version 88570 (0.0011) +[2023-10-09 15:41:08,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 181043200. Throughput: 0: 1824.6, 1: 1834.8. Samples: 45275462. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:41:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:41:08,514][86121] Updated weights for policy 0, policy_version 88230 (0.0009) +[2023-10-09 15:41:08,885][86121] Updated weights for policy 0, policy_version 88240 (0.0010) +[2023-10-09 15:41:09,259][86121] Updated weights for policy 0, policy_version 88250 (0.0010) +[2023-10-09 15:41:09,391][86122] Updated weights for policy 1, policy_version 88580 (0.0010) +[2023-10-09 15:41:09,756][86122] Updated weights for policy 1, policy_version 88590 (0.0007) +[2023-10-09 15:41:10,113][86122] Updated weights for policy 1, policy_version 88600 (0.0010) +[2023-10-09 15:41:13,068][86121] Updated weights for policy 0, policy_version 88260 (0.0008) +[2023-10-09 15:41:13,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181108736. Throughput: 0: 1824.0, 1: 1835.6. Samples: 45285364. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:41:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:41:13,430][86121] Updated weights for policy 0, policy_version 88270 (0.0008) +[2023-10-09 15:41:13,763][86122] Updated weights for policy 1, policy_version 88610 (0.0008) +[2023-10-09 15:41:13,801][86121] Updated weights for policy 0, policy_version 88280 (0.0008) +[2023-10-09 15:41:14,124][86122] Updated weights for policy 1, policy_version 88620 (0.0008) +[2023-10-09 15:41:14,488][86122] Updated weights for policy 1, policy_version 88630 (0.0008) +[2023-10-09 15:41:14,848][86122] Updated weights for policy 1, policy_version 88640 (0.0007) +[2023-10-09 15:41:17,453][86121] Updated weights for policy 0, policy_version 88290 (0.0009) +[2023-10-09 15:41:17,827][86121] Updated weights for policy 0, policy_version 88300 (0.0009) +[2023-10-09 15:41:18,190][86121] Updated weights for policy 0, policy_version 88310 (0.0008) +[2023-10-09 15:41:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181174272. Throughput: 0: 1822.9, 1: 1834.1. Samples: 45308390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 32.0) +[2023-10-09 15:41:18,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 15:41:18,543][86122] Updated weights for policy 1, policy_version 88650 (0.0009) +[2023-10-09 15:41:18,555][86121] Updated weights for policy 0, policy_version 88320 (0.0007) +[2023-10-09 15:41:18,894][86122] Updated weights for policy 1, policy_version 88660 (0.0009) +[2023-10-09 15:41:19,255][86122] Updated weights for policy 1, policy_version 88670 (0.0008) +[2023-10-09 15:41:22,251][86121] Updated weights for policy 0, policy_version 88330 (0.0009) +[2023-10-09 15:41:22,616][86121] Updated weights for policy 0, policy_version 88340 (0.0007) +[2023-10-09 15:41:22,875][86122] Updated weights for policy 1, policy_version 88680 (0.0008) +[2023-10-09 15:41:22,981][86121] Updated weights for policy 0, policy_version 88350 (0.0007) +[2023-10-09 15:41:23,230][86122] Updated weights for policy 1, policy_version 88690 (0.0010) +[2023-10-09 15:41:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 181272576. Throughput: 0: 1825.6, 1: 1833.7. Samples: 45329868. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:23,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 15:41:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000088352_90472448.pth... +[2023-10-09 15:41:23,445][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000086624_88702976.pth +[2023-10-09 15:41:23,593][86122] Updated weights for policy 1, policy_version 88700 (0.0008) +[2023-10-09 15:41:23,737][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000088704_90832896.pth... +[2023-10-09 15:41:23,768][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000086976_89063424.pth +[2023-10-09 15:41:26,806][86121] Updated weights for policy 0, policy_version 88360 (0.0007) +[2023-10-09 15:41:27,172][86121] Updated weights for policy 0, policy_version 88370 (0.0007) +[2023-10-09 15:41:27,282][86122] Updated weights for policy 1, policy_version 88710 (0.0007) +[2023-10-09 15:41:27,541][86121] Updated weights for policy 0, policy_version 88380 (0.0008) +[2023-10-09 15:41:27,640][86122] Updated weights for policy 1, policy_version 88720 (0.0007) +[2023-10-09 15:41:28,005][86122] Updated weights for policy 1, policy_version 88730 (0.0009) +[2023-10-09 15:41:28,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 181370880. Throughput: 0: 1822.1, 1: 1834.2. Samples: 45340958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:28,398][85186] Avg episode reward: [(0, '10.000'), (1, '10.000')] +[2023-10-09 15:41:31,407][86121] Updated weights for policy 0, policy_version 88390 (0.0007) +[2023-10-09 15:41:31,775][86121] Updated weights for policy 0, policy_version 88400 (0.0008) +[2023-10-09 15:41:31,807][86122] Updated weights for policy 1, policy_version 88740 (0.0009) +[2023-10-09 15:41:32,138][86121] Updated weights for policy 0, policy_version 88410 (0.0007) +[2023-10-09 15:41:32,193][86122] Updated weights for policy 1, policy_version 88750 (0.0007) +[2023-10-09 15:41:32,544][86122] Updated weights for policy 1, policy_version 88760 (0.0008) +[2023-10-09 15:41:33,397][85186] Fps is (10 sec: 16384.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 181436416. Throughput: 0: 1819.2, 1: 1826.3. Samples: 45362170. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:41:35,777][86121] Updated weights for policy 0, policy_version 88420 (0.0009) +[2023-10-09 15:41:36,151][86121] Updated weights for policy 0, policy_version 88430 (0.0009) +[2023-10-09 15:41:36,294][86122] Updated weights for policy 1, policy_version 88770 (0.0008) +[2023-10-09 15:41:36,518][86121] Updated weights for policy 0, policy_version 88440 (0.0008) +[2023-10-09 15:41:36,648][86122] Updated weights for policy 1, policy_version 88780 (0.0008) +[2023-10-09 15:41:37,011][86122] Updated weights for policy 1, policy_version 88790 (0.0008) +[2023-10-09 15:41:37,374][86122] Updated weights for policy 1, policy_version 88800 (0.0009) +[2023-10-09 15:41:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181501952. Throughput: 0: 1815.2, 1: 1824.7. Samples: 45383090. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:41:40,245][86121] Updated weights for policy 0, policy_version 88450 (0.0009) +[2023-10-09 15:41:40,634][86121] Updated weights for policy 0, policy_version 88460 (0.0009) +[2023-10-09 15:41:40,990][86121] Updated weights for policy 0, policy_version 88470 (0.0009) +[2023-10-09 15:41:41,051][86122] Updated weights for policy 1, policy_version 88810 (0.0009) +[2023-10-09 15:41:41,354][86121] Updated weights for policy 0, policy_version 88480 (0.0008) +[2023-10-09 15:41:41,412][86122] Updated weights for policy 1, policy_version 88820 (0.0008) +[2023-10-09 15:41:41,782][86122] Updated weights for policy 1, policy_version 88830 (0.0008) +[2023-10-09 15:41:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 181567488. Throughput: 0: 1821.0, 1: 1823.3. Samples: 45394840. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:41:44,934][86121] Updated weights for policy 0, policy_version 88490 (0.0007) +[2023-10-09 15:41:45,294][86121] Updated weights for policy 0, policy_version 88500 (0.0008) +[2023-10-09 15:41:45,481][86122] Updated weights for policy 1, policy_version 88840 (0.0007) +[2023-10-09 15:41:45,659][86121] Updated weights for policy 0, policy_version 88510 (0.0008) +[2023-10-09 15:41:45,831][86122] Updated weights for policy 1, policy_version 88850 (0.0008) +[2023-10-09 15:41:46,193][86122] Updated weights for policy 1, policy_version 88860 (0.0011) +[2023-10-09 15:41:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 181633024. Throughput: 0: 1812.1, 1: 1817.2. Samples: 45415682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:41:49,511][86121] Updated weights for policy 0, policy_version 88520 (0.0009) +[2023-10-09 15:41:49,882][86121] Updated weights for policy 0, policy_version 88530 (0.0009) +[2023-10-09 15:41:49,995][86122] Updated weights for policy 1, policy_version 88870 (0.0010) +[2023-10-09 15:41:50,241][86121] Updated weights for policy 0, policy_version 88540 (0.0010) +[2023-10-09 15:41:50,352][86122] Updated weights for policy 1, policy_version 88880 (0.0009) +[2023-10-09 15:41:50,716][86122] Updated weights for policy 1, policy_version 88890 (0.0009) +[2023-10-09 15:41:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181698560. Throughput: 0: 1805.9, 1: 1812.6. Samples: 45438294. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:41:53,813][86121] Updated weights for policy 0, policy_version 88550 (0.0010) +[2023-10-09 15:41:54,186][86121] Updated weights for policy 0, policy_version 88560 (0.0009) +[2023-10-09 15:41:54,434][86122] Updated weights for policy 1, policy_version 88900 (0.0008) +[2023-10-09 15:41:54,543][86121] Updated weights for policy 0, policy_version 88570 (0.0009) +[2023-10-09 15:41:54,793][86122] Updated weights for policy 1, policy_version 88910 (0.0008) +[2023-10-09 15:41:55,147][86122] Updated weights for policy 1, policy_version 88920 (0.0007) +[2023-10-09 15:41:58,214][86121] Updated weights for policy 0, policy_version 88580 (0.0010) +[2023-10-09 15:41:58,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 181764096. Throughput: 0: 1803.8, 1: 1812.7. Samples: 45448108. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:41:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:41:58,583][86121] Updated weights for policy 0, policy_version 88590 (0.0008) +[2023-10-09 15:41:58,952][86121] Updated weights for policy 0, policy_version 88600 (0.0009) +[2023-10-09 15:41:58,994][86122] Updated weights for policy 1, policy_version 88930 (0.0008) +[2023-10-09 15:41:59,358][86122] Updated weights for policy 1, policy_version 88940 (0.0008) +[2023-10-09 15:41:59,723][86122] Updated weights for policy 1, policy_version 88950 (0.0008) +[2023-10-09 15:42:00,078][86122] Updated weights for policy 1, policy_version 88960 (0.0008) +[2023-10-09 15:42:02,453][86121] Updated weights for policy 0, policy_version 88610 (0.0007) +[2023-10-09 15:42:02,806][86121] Updated weights for policy 0, policy_version 88620 (0.0007) +[2023-10-09 15:42:03,178][86121] Updated weights for policy 0, policy_version 88630 (0.0008) +[2023-10-09 15:42:03,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 181829632. Throughput: 0: 1809.9, 1: 1813.1. Samples: 45471422. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:42:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:42:03,537][86121] Updated weights for policy 0, policy_version 88640 (0.0009) +[2023-10-09 15:42:03,607][86122] Updated weights for policy 1, policy_version 88970 (0.0008) +[2023-10-09 15:42:03,971][86122] Updated weights for policy 1, policy_version 88980 (0.0011) +[2023-10-09 15:42:04,333][86122] Updated weights for policy 1, policy_version 88990 (0.0011) +[2023-10-09 15:42:07,387][86121] Updated weights for policy 0, policy_version 88650 (0.0008) +[2023-10-09 15:42:07,755][86121] Updated weights for policy 0, policy_version 88660 (0.0008) +[2023-10-09 15:42:08,007][86122] Updated weights for policy 1, policy_version 89000 (0.0009) +[2023-10-09 15:42:08,111][86121] Updated weights for policy 0, policy_version 88670 (0.0007) +[2023-10-09 15:42:08,365][86122] Updated weights for policy 1, policy_version 89010 (0.0009) +[2023-10-09 15:42:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 181927936. Throughput: 0: 1810.1, 1: 1817.2. Samples: 45493100. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:42:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:42:08,729][86122] Updated weights for policy 1, policy_version 89020 (0.0009) +[2023-10-09 15:42:11,841][86121] Updated weights for policy 0, policy_version 88680 (0.0009) +[2023-10-09 15:42:12,192][86122] Updated weights for policy 1, policy_version 89030 (0.0008) +[2023-10-09 15:42:12,209][86121] Updated weights for policy 0, policy_version 88690 (0.0008) +[2023-10-09 15:42:12,555][86122] Updated weights for policy 1, policy_version 89040 (0.0008) +[2023-10-09 15:42:12,575][86121] Updated weights for policy 0, policy_version 88700 (0.0007) +[2023-10-09 15:42:12,919][86122] Updated weights for policy 1, policy_version 89050 (0.0007) +[2023-10-09 15:42:13,397][85186] Fps is (10 sec: 19661.2, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 182026240. Throughput: 0: 1808.8, 1: 1819.1. Samples: 45504210. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:42:13,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:42:16,281][86121] Updated weights for policy 0, policy_version 88710 (0.0007) +[2023-10-09 15:42:16,611][86122] Updated weights for policy 1, policy_version 89060 (0.0008) +[2023-10-09 15:42:16,654][86121] Updated weights for policy 0, policy_version 88720 (0.0007) +[2023-10-09 15:42:16,967][86122] Updated weights for policy 1, policy_version 89070 (0.0008) +[2023-10-09 15:42:17,021][86121] Updated weights for policy 0, policy_version 88730 (0.0008) +[2023-10-09 15:42:17,325][86122] Updated weights for policy 1, policy_version 89080 (0.0007) +[2023-10-09 15:42:18,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 182091776. Throughput: 0: 1816.0, 1: 1821.5. Samples: 45525856. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:42:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:42:20,648][86121] Updated weights for policy 0, policy_version 88740 (0.0009) +[2023-10-09 15:42:20,998][86121] Updated weights for policy 0, policy_version 88750 (0.0010) +[2023-10-09 15:42:21,008][86122] Updated weights for policy 1, policy_version 89090 (0.0011) +[2023-10-09 15:42:21,370][86121] Updated weights for policy 0, policy_version 88760 (0.0008) +[2023-10-09 15:42:21,399][86122] Updated weights for policy 1, policy_version 89100 (0.0008) +[2023-10-09 15:42:21,765][86122] Updated weights for policy 1, policy_version 89110 (0.0009) +[2023-10-09 15:42:22,116][86122] Updated weights for policy 1, policy_version 89120 (0.0009) +[2023-10-09 15:42:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 182157312. Throughput: 0: 1821.7, 1: 1826.0. Samples: 45547234. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:42:25,171][86121] Updated weights for policy 0, policy_version 88770 (0.0009) +[2023-10-09 15:42:25,584][86121] Updated weights for policy 0, policy_version 88780 (0.0009) +[2023-10-09 15:42:25,791][86122] Updated weights for policy 1, policy_version 89130 (0.0008) +[2023-10-09 15:42:25,941][86121] Updated weights for policy 0, policy_version 88790 (0.0008) +[2023-10-09 15:42:26,155][86122] Updated weights for policy 1, policy_version 89140 (0.0008) +[2023-10-09 15:42:26,308][86121] Updated weights for policy 0, policy_version 88800 (0.0009) +[2023-10-09 15:42:26,510][86122] Updated weights for policy 1, policy_version 89150 (0.0009) +[2023-10-09 15:42:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 182222848. Throughput: 0: 1816.0, 1: 1824.8. Samples: 45558680. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:42:29,995][86121] Updated weights for policy 0, policy_version 88810 (0.0008) +[2023-10-09 15:42:30,190][86122] Updated weights for policy 1, policy_version 89160 (0.0008) +[2023-10-09 15:42:30,353][86121] Updated weights for policy 0, policy_version 88820 (0.0008) +[2023-10-09 15:42:30,563][86122] Updated weights for policy 1, policy_version 89170 (0.0009) +[2023-10-09 15:42:30,728][86121] Updated weights for policy 0, policy_version 88830 (0.0007) +[2023-10-09 15:42:30,913][86122] Updated weights for policy 1, policy_version 89180 (0.0008) +[2023-10-09 15:42:33,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182288384. Throughput: 0: 1814.2, 1: 1833.3. Samples: 45579820. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:42:34,555][86121] Updated weights for policy 0, policy_version 88840 (0.0007) +[2023-10-09 15:42:34,633][86122] Updated weights for policy 1, policy_version 89190 (0.0009) +[2023-10-09 15:42:34,916][86121] Updated weights for policy 0, policy_version 88850 (0.0007) +[2023-10-09 15:42:34,987][86122] Updated weights for policy 1, policy_version 89200 (0.0008) +[2023-10-09 15:42:35,283][86121] Updated weights for policy 0, policy_version 88860 (0.0008) +[2023-10-09 15:42:35,349][86122] Updated weights for policy 1, policy_version 89210 (0.0009) +[2023-10-09 15:42:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182353920. Throughput: 0: 1808.4, 1: 1836.5. Samples: 45602314. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:42:39,074][86121] Updated weights for policy 0, policy_version 88870 (0.0008) +[2023-10-09 15:42:39,076][86122] Updated weights for policy 1, policy_version 89220 (0.0009) +[2023-10-09 15:42:39,443][86122] Updated weights for policy 1, policy_version 89230 (0.0010) +[2023-10-09 15:42:39,448][86121] Updated weights for policy 0, policy_version 88880 (0.0008) +[2023-10-09 15:42:39,804][86122] Updated weights for policy 1, policy_version 89240 (0.0007) +[2023-10-09 15:42:39,809][86121] Updated weights for policy 0, policy_version 88890 (0.0008) +[2023-10-09 15:42:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182419456. Throughput: 0: 1808.6, 1: 1836.0. Samples: 45612112. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:42:43,537][86121] Updated weights for policy 0, policy_version 88900 (0.0008) +[2023-10-09 15:42:43,555][86122] Updated weights for policy 1, policy_version 89250 (0.0007) +[2023-10-09 15:42:43,899][86121] Updated weights for policy 0, policy_version 88910 (0.0009) +[2023-10-09 15:42:43,930][86122] Updated weights for policy 1, policy_version 89260 (0.0008) +[2023-10-09 15:42:44,253][86121] Updated weights for policy 0, policy_version 88920 (0.0008) +[2023-10-09 15:42:44,283][86122] Updated weights for policy 1, policy_version 89270 (0.0008) +[2023-10-09 15:42:44,644][86122] Updated weights for policy 1, policy_version 89280 (0.0008) +[2023-10-09 15:42:48,112][86121] Updated weights for policy 0, policy_version 88930 (0.0008) +[2023-10-09 15:42:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182484992. Throughput: 0: 1802.7, 1: 1826.4. Samples: 45634732. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:42:48,414][86122] Updated weights for policy 1, policy_version 89290 (0.0008) +[2023-10-09 15:42:48,477][86121] Updated weights for policy 0, policy_version 88940 (0.0009) +[2023-10-09 15:42:48,778][86122] Updated weights for policy 1, policy_version 89300 (0.0007) +[2023-10-09 15:42:48,840][86121] Updated weights for policy 0, policy_version 88950 (0.0008) +[2023-10-09 15:42:49,144][86122] Updated weights for policy 1, policy_version 89310 (0.0009) +[2023-10-09 15:42:49,197][86121] Updated weights for policy 0, policy_version 88960 (0.0009) +[2023-10-09 15:42:53,000][86122] Updated weights for policy 1, policy_version 89320 (0.0009) +[2023-10-09 15:42:53,038][86121] Updated weights for policy 0, policy_version 88970 (0.0008) +[2023-10-09 15:42:53,365][86122] Updated weights for policy 1, policy_version 89330 (0.0007) +[2023-10-09 15:42:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 182550528. Throughput: 0: 1817.7, 1: 1810.3. Samples: 45656360. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:42:53,404][86121] Updated weights for policy 0, policy_version 88980 (0.0008) +[2023-10-09 15:42:53,724][86122] Updated weights for policy 1, policy_version 89340 (0.0008) +[2023-10-09 15:42:53,778][86121] Updated weights for policy 0, policy_version 88990 (0.0008) +[2023-10-09 15:42:57,448][86122] Updated weights for policy 1, policy_version 89350 (0.0008) +[2023-10-09 15:42:57,568][86121] Updated weights for policy 0, policy_version 89000 (0.0009) +[2023-10-09 15:42:57,806][86122] Updated weights for policy 1, policy_version 89360 (0.0008) +[2023-10-09 15:42:57,937][86121] Updated weights for policy 0, policy_version 89010 (0.0008) +[2023-10-09 15:42:58,169][86122] Updated weights for policy 1, policy_version 89370 (0.0008) +[2023-10-09 15:42:58,312][86121] Updated weights for policy 0, policy_version 89020 (0.0007) +[2023-10-09 15:42:58,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 182648832. Throughput: 0: 1804.3, 1: 1806.7. Samples: 45666704. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:42:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:43:01,894][86122] Updated weights for policy 1, policy_version 89380 (0.0009) +[2023-10-09 15:43:01,904][86121] Updated weights for policy 0, policy_version 89030 (0.0008) +[2023-10-09 15:43:02,257][86122] Updated weights for policy 1, policy_version 89390 (0.0007) +[2023-10-09 15:43:02,266][86121] Updated weights for policy 0, policy_version 89040 (0.0009) +[2023-10-09 15:43:02,623][86122] Updated weights for policy 1, policy_version 89400 (0.0008) +[2023-10-09 15:43:02,632][86121] Updated weights for policy 0, policy_version 89050 (0.0009) +[2023-10-09 15:43:03,397][85186] Fps is (10 sec: 19660.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 182747136. Throughput: 0: 1818.1, 1: 1810.2. Samples: 45689128. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:43:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:06,177][86121] Updated weights for policy 0, policy_version 89060 (0.0010) +[2023-10-09 15:43:06,447][86122] Updated weights for policy 1, policy_version 89410 (0.0008) +[2023-10-09 15:43:06,545][86121] Updated weights for policy 0, policy_version 89070 (0.0007) +[2023-10-09 15:43:06,817][86122] Updated weights for policy 1, policy_version 89420 (0.0007) +[2023-10-09 15:43:06,904][86121] Updated weights for policy 0, policy_version 89080 (0.0007) +[2023-10-09 15:43:07,174][86122] Updated weights for policy 1, policy_version 89430 (0.0007) +[2023-10-09 15:43:07,535][86122] Updated weights for policy 1, policy_version 89440 (0.0007) +[2023-10-09 15:43:08,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 182812672. Throughput: 0: 1803.6, 1: 1798.6. Samples: 45709334. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:43:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:10,586][86121] Updated weights for policy 0, policy_version 89090 (0.0008) +[2023-10-09 15:43:10,983][86121] Updated weights for policy 0, policy_version 89100 (0.0009) +[2023-10-09 15:43:11,282][86122] Updated weights for policy 1, policy_version 89450 (0.0007) +[2023-10-09 15:43:11,354][86121] Updated weights for policy 0, policy_version 89110 (0.0007) +[2023-10-09 15:43:11,638][86122] Updated weights for policy 1, policy_version 89460 (0.0007) +[2023-10-09 15:43:11,716][86121] Updated weights for policy 0, policy_version 89120 (0.0008) +[2023-10-09 15:43:12,000][86122] Updated weights for policy 1, policy_version 89470 (0.0007) +[2023-10-09 15:43:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 182878208. Throughput: 0: 1820.8, 1: 1798.8. Samples: 45721560. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:43:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:15,245][86121] Updated weights for policy 0, policy_version 89130 (0.0009) +[2023-10-09 15:43:15,607][86121] Updated weights for policy 0, policy_version 89140 (0.0008) +[2023-10-09 15:43:15,703][86122] Updated weights for policy 1, policy_version 89480 (0.0008) +[2023-10-09 15:43:15,963][86121] Updated weights for policy 0, policy_version 89150 (0.0008) +[2023-10-09 15:43:16,062][86122] Updated weights for policy 1, policy_version 89490 (0.0007) +[2023-10-09 15:43:16,425][86122] Updated weights for policy 1, policy_version 89500 (0.0008) +[2023-10-09 15:43:18,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 182943744. Throughput: 0: 1818.5, 1: 1789.7. Samples: 45742188. Policy #0 lag: (min: 31.0, avg: 34.0, max: 63.0) +[2023-10-09 15:43:18,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:19,543][86121] Updated weights for policy 0, policy_version 89160 (0.0008) +[2023-10-09 15:43:19,912][86121] Updated weights for policy 0, policy_version 89170 (0.0007) +[2023-10-09 15:43:20,101][86122] Updated weights for policy 1, policy_version 89510 (0.0008) +[2023-10-09 15:43:20,280][86121] Updated weights for policy 0, policy_version 89180 (0.0009) +[2023-10-09 15:43:20,455][86122] Updated weights for policy 1, policy_version 89520 (0.0009) +[2023-10-09 15:43:20,814][86122] Updated weights for policy 1, policy_version 89530 (0.0010) +[2023-10-09 15:43:23,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 183009280. Throughput: 0: 1832.5, 1: 1789.1. Samples: 45765288. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000089184_91324416.pth... +[2023-10-09 15:43:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000089536_91684864.pth... +[2023-10-09 15:43:23,452][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000087840_89948160.pth +[2023-10-09 15:43:23,455][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000087488_89587712.pth +[2023-10-09 15:43:23,874][86121] Updated weights for policy 0, policy_version 89190 (0.0010) +[2023-10-09 15:43:24,236][86121] Updated weights for policy 0, policy_version 89200 (0.0009) +[2023-10-09 15:43:24,517][86122] Updated weights for policy 1, policy_version 89540 (0.0008) +[2023-10-09 15:43:24,608][86121] Updated weights for policy 0, policy_version 89210 (0.0009) +[2023-10-09 15:43:24,870][86122] Updated weights for policy 1, policy_version 89550 (0.0009) +[2023-10-09 15:43:25,231][86122] Updated weights for policy 1, policy_version 89560 (0.0010) +[2023-10-09 15:43:28,398][86121] Updated weights for policy 0, policy_version 89220 (0.0007) +[2023-10-09 15:43:28,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183074816. Throughput: 0: 1831.1, 1: 1794.3. Samples: 45775254. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:28,765][86121] Updated weights for policy 0, policy_version 89230 (0.0008) +[2023-10-09 15:43:29,025][86122] Updated weights for policy 1, policy_version 89570 (0.0009) +[2023-10-09 15:43:29,127][86121] Updated weights for policy 0, policy_version 89240 (0.0007) +[2023-10-09 15:43:29,381][86122] Updated weights for policy 1, policy_version 89580 (0.0007) +[2023-10-09 15:43:29,743][86122] Updated weights for policy 1, policy_version 89590 (0.0008) +[2023-10-09 15:43:30,113][86122] Updated weights for policy 1, policy_version 89600 (0.0007) +[2023-10-09 15:43:32,693][86121] Updated weights for policy 0, policy_version 89250 (0.0008) +[2023-10-09 15:43:33,064][86121] Updated weights for policy 0, policy_version 89260 (0.0008) +[2023-10-09 15:43:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183140352. Throughput: 0: 1831.1, 1: 1796.7. Samples: 45797984. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:33,433][86121] Updated weights for policy 0, policy_version 89270 (0.0009) +[2023-10-09 15:43:33,792][86122] Updated weights for policy 1, policy_version 89610 (0.0008) +[2023-10-09 15:43:33,802][86121] Updated weights for policy 0, policy_version 89280 (0.0009) +[2023-10-09 15:43:34,148][86122] Updated weights for policy 1, policy_version 89620 (0.0008) +[2023-10-09 15:43:34,508][86122] Updated weights for policy 1, policy_version 89630 (0.0008) +[2023-10-09 15:43:37,635][86121] Updated weights for policy 0, policy_version 89290 (0.0008) +[2023-10-09 15:43:37,999][86121] Updated weights for policy 0, policy_version 89300 (0.0007) +[2023-10-09 15:43:38,121][86122] Updated weights for policy 1, policy_version 89640 (0.0008) +[2023-10-09 15:43:38,360][86121] Updated weights for policy 0, policy_version 89310 (0.0008) +[2023-10-09 15:43:38,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183205888. Throughput: 0: 1822.1, 1: 1812.6. Samples: 45819922. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:38,481][86122] Updated weights for policy 1, policy_version 89650 (0.0007) +[2023-10-09 15:43:38,853][86122] Updated weights for policy 1, policy_version 89660 (0.0009) +[2023-10-09 15:43:42,041][86121] Updated weights for policy 0, policy_version 89320 (0.0008) +[2023-10-09 15:43:42,412][86121] Updated weights for policy 0, policy_version 89330 (0.0008) +[2023-10-09 15:43:42,574][86122] Updated weights for policy 1, policy_version 89670 (0.0007) +[2023-10-09 15:43:42,774][86121] Updated weights for policy 0, policy_version 89340 (0.0007) +[2023-10-09 15:43:42,927][86122] Updated weights for policy 1, policy_version 89680 (0.0008) +[2023-10-09 15:43:43,283][86122] Updated weights for policy 1, policy_version 89690 (0.0008) +[2023-10-09 15:43:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183304192. Throughput: 0: 1835.5, 1: 1809.9. Samples: 45830750. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.970')] +[2023-10-09 15:43:46,455][86121] Updated weights for policy 0, policy_version 89350 (0.0007) +[2023-10-09 15:43:46,821][86121] Updated weights for policy 0, policy_version 89360 (0.0007) +[2023-10-09 15:43:47,173][86122] Updated weights for policy 1, policy_version 89700 (0.0009) +[2023-10-09 15:43:47,187][86121] Updated weights for policy 0, policy_version 89370 (0.0008) +[2023-10-09 15:43:47,524][86122] Updated weights for policy 1, policy_version 89710 (0.0010) +[2023-10-09 15:43:47,883][86122] Updated weights for policy 1, policy_version 89720 (0.0008) +[2023-10-09 15:43:48,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 183402496. Throughput: 0: 1825.4, 1: 1812.6. Samples: 45852838. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.970')] +[2023-10-09 15:43:50,830][86121] Updated weights for policy 0, policy_version 89380 (0.0008) +[2023-10-09 15:43:51,192][86121] Updated weights for policy 0, policy_version 89390 (0.0008) +[2023-10-09 15:43:51,566][86121] Updated weights for policy 0, policy_version 89400 (0.0008) +[2023-10-09 15:43:51,822][86122] Updated weights for policy 1, policy_version 89730 (0.0009) +[2023-10-09 15:43:52,226][86122] Updated weights for policy 1, policy_version 89740 (0.0008) +[2023-10-09 15:43:52,595][86122] Updated weights for policy 1, policy_version 89750 (0.0010) +[2023-10-09 15:43:52,950][86122] Updated weights for policy 1, policy_version 89760 (0.0009) +[2023-10-09 15:43:53,397][85186] Fps is (10 sec: 16383.7, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 183468032. Throughput: 0: 1834.4, 1: 1811.4. Samples: 45873396. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:53,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:43:55,179][86121] Updated weights for policy 0, policy_version 89410 (0.0008) +[2023-10-09 15:43:55,551][86121] Updated weights for policy 0, policy_version 89420 (0.0008) +[2023-10-09 15:43:55,913][86121] Updated weights for policy 0, policy_version 89430 (0.0007) +[2023-10-09 15:43:56,272][86121] Updated weights for policy 0, policy_version 89440 (0.0007) +[2023-10-09 15:43:56,555][86122] Updated weights for policy 1, policy_version 89770 (0.0008) +[2023-10-09 15:43:56,915][86122] Updated weights for policy 1, policy_version 89780 (0.0008) +[2023-10-09 15:43:57,287][86122] Updated weights for policy 1, policy_version 89790 (0.0008) +[2023-10-09 15:43:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 183533568. Throughput: 0: 1825.0, 1: 1816.1. Samples: 45885410. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:43:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:43:59,900][86121] Updated weights for policy 0, policy_version 89450 (0.0008) +[2023-10-09 15:44:00,267][86121] Updated weights for policy 0, policy_version 89460 (0.0009) +[2023-10-09 15:44:00,638][86121] Updated weights for policy 0, policy_version 89470 (0.0008) +[2023-10-09 15:44:01,050][86122] Updated weights for policy 1, policy_version 89800 (0.0010) +[2023-10-09 15:44:01,416][86122] Updated weights for policy 1, policy_version 89810 (0.0007) +[2023-10-09 15:44:01,772][86122] Updated weights for policy 1, policy_version 89820 (0.0009) +[2023-10-09 15:44:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183599104. Throughput: 0: 1832.3, 1: 1819.7. Samples: 45906528. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:44:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:44:04,263][86121] Updated weights for policy 0, policy_version 89480 (0.0008) +[2023-10-09 15:44:04,640][86121] Updated weights for policy 0, policy_version 89490 (0.0009) +[2023-10-09 15:44:04,998][86121] Updated weights for policy 0, policy_version 89500 (0.0009) +[2023-10-09 15:44:05,483][86122] Updated weights for policy 1, policy_version 89830 (0.0011) +[2023-10-09 15:44:05,846][86122] Updated weights for policy 1, policy_version 89840 (0.0009) +[2023-10-09 15:44:06,202][86122] Updated weights for policy 1, policy_version 89850 (0.0010) +[2023-10-09 15:44:08,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 183664640. Throughput: 0: 1832.3, 1: 1819.7. Samples: 45929628. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:44:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:44:08,635][86121] Updated weights for policy 0, policy_version 89510 (0.0011) +[2023-10-09 15:44:08,994][86121] Updated weights for policy 0, policy_version 89520 (0.0009) +[2023-10-09 15:44:09,356][86121] Updated weights for policy 0, policy_version 89530 (0.0009) +[2023-10-09 15:44:09,835][86122] Updated weights for policy 1, policy_version 89860 (0.0009) +[2023-10-09 15:44:10,196][86122] Updated weights for policy 1, policy_version 89870 (0.0009) +[2023-10-09 15:44:10,558][86122] Updated weights for policy 1, policy_version 89880 (0.0009) +[2023-10-09 15:44:13,012][86121] Updated weights for policy 0, policy_version 89540 (0.0009) +[2023-10-09 15:44:13,393][86121] Updated weights for policy 0, policy_version 89550 (0.0007) +[2023-10-09 15:44:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 183730176. Throughput: 0: 1837.6, 1: 1818.5. Samples: 45939780. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:44:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:44:13,756][86121] Updated weights for policy 0, policy_version 89560 (0.0009) +[2023-10-09 15:44:14,022][86122] Updated weights for policy 1, policy_version 89890 (0.0009) +[2023-10-09 15:44:14,376][86122] Updated weights for policy 1, policy_version 89900 (0.0007) +[2023-10-09 15:44:14,727][86122] Updated weights for policy 1, policy_version 89910 (0.0007) +[2023-10-09 15:44:15,089][86122] Updated weights for policy 1, policy_version 89920 (0.0009) +[2023-10-09 15:44:17,405][86121] Updated weights for policy 0, policy_version 89570 (0.0009) +[2023-10-09 15:44:17,770][86121] Updated weights for policy 0, policy_version 89580 (0.0010) +[2023-10-09 15:44:18,132][86121] Updated weights for policy 0, policy_version 89590 (0.0011) +[2023-10-09 15:44:18,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 183795712. Throughput: 0: 1835.9, 1: 1824.5. Samples: 45962704. Policy #0 lag: (min: 18.0, avg: 22.2, max: 50.0) +[2023-10-09 15:44:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:44:18,493][86121] Updated weights for policy 0, policy_version 89600 (0.0009) +[2023-10-09 15:44:18,711][86122] Updated weights for policy 1, policy_version 89930 (0.0009) +[2023-10-09 15:44:19,074][86122] Updated weights for policy 1, policy_version 89940 (0.0011) +[2023-10-09 15:44:19,432][86122] Updated weights for policy 1, policy_version 89950 (0.0010) +[2023-10-09 15:44:22,160][86121] Updated weights for policy 0, policy_version 89610 (0.0010) +[2023-10-09 15:44:22,530][86121] Updated weights for policy 0, policy_version 89620 (0.0007) +[2023-10-09 15:44:22,893][86121] Updated weights for policy 0, policy_version 89630 (0.0007) +[2023-10-09 15:44:23,079][86122] Updated weights for policy 1, policy_version 89960 (0.0008) +[2023-10-09 15:44:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 183894016. Throughput: 0: 1826.3, 1: 1828.6. Samples: 45984390. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:23,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:44:23,437][86122] Updated weights for policy 1, policy_version 89970 (0.0008) +[2023-10-09 15:44:23,797][86122] Updated weights for policy 1, policy_version 89980 (0.0007) +[2023-10-09 15:44:26,720][86121] Updated weights for policy 0, policy_version 89640 (0.0008) +[2023-10-09 15:44:27,085][86121] Updated weights for policy 0, policy_version 89650 (0.0008) +[2023-10-09 15:44:27,409][86122] Updated weights for policy 1, policy_version 89990 (0.0009) +[2023-10-09 15:44:27,453][86121] Updated weights for policy 0, policy_version 89660 (0.0009) +[2023-10-09 15:44:27,779][86122] Updated weights for policy 1, policy_version 90000 (0.0010) +[2023-10-09 15:44:28,134][86122] Updated weights for policy 1, policy_version 90010 (0.0008) +[2023-10-09 15:44:28,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 183992320. Throughput: 0: 1835.6, 1: 1831.3. Samples: 45995756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:44:31,216][86121] Updated weights for policy 0, policy_version 89670 (0.0007) +[2023-10-09 15:44:31,589][86121] Updated weights for policy 0, policy_version 89680 (0.0011) +[2023-10-09 15:44:31,949][86121] Updated weights for policy 0, policy_version 89690 (0.0008) +[2023-10-09 15:44:32,111][86122] Updated weights for policy 1, policy_version 90020 (0.0008) +[2023-10-09 15:44:32,481][86122] Updated weights for policy 1, policy_version 90030 (0.0008) +[2023-10-09 15:44:32,835][86122] Updated weights for policy 1, policy_version 90040 (0.0008) +[2023-10-09 15:44:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 184057856. Throughput: 0: 1820.4, 1: 1830.8. Samples: 46017142. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:44:35,695][86121] Updated weights for policy 0, policy_version 89700 (0.0008) +[2023-10-09 15:44:36,057][86121] Updated weights for policy 0, policy_version 89710 (0.0010) +[2023-10-09 15:44:36,428][86121] Updated weights for policy 0, policy_version 89720 (0.0009) +[2023-10-09 15:44:36,629][86122] Updated weights for policy 1, policy_version 90050 (0.0010) +[2023-10-09 15:44:36,999][86122] Updated weights for policy 1, policy_version 90060 (0.0009) +[2023-10-09 15:44:37,348][86122] Updated weights for policy 1, policy_version 90070 (0.0008) +[2023-10-09 15:44:37,704][86122] Updated weights for policy 1, policy_version 90080 (0.0009) +[2023-10-09 15:44:38,397][85186] Fps is (10 sec: 13106.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 184123392. Throughput: 0: 1821.6, 1: 1834.6. Samples: 46037922. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:44:40,171][86121] Updated weights for policy 0, policy_version 89730 (0.0009) +[2023-10-09 15:44:40,568][86121] Updated weights for policy 0, policy_version 89740 (0.0010) +[2023-10-09 15:44:40,931][86121] Updated weights for policy 0, policy_version 89750 (0.0008) +[2023-10-09 15:44:41,301][86121] Updated weights for policy 0, policy_version 89760 (0.0008) +[2023-10-09 15:44:41,600][86122] Updated weights for policy 1, policy_version 90090 (0.0008) +[2023-10-09 15:44:41,971][86122] Updated weights for policy 1, policy_version 90100 (0.0010) +[2023-10-09 15:44:42,336][86122] Updated weights for policy 1, policy_version 90110 (0.0010) +[2023-10-09 15:44:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184188928. Throughput: 0: 1818.4, 1: 1829.6. Samples: 46049572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:44:44,886][86121] Updated weights for policy 0, policy_version 89770 (0.0008) +[2023-10-09 15:44:45,262][86121] Updated weights for policy 0, policy_version 89780 (0.0008) +[2023-10-09 15:44:45,621][86121] Updated weights for policy 0, policy_version 89790 (0.0009) +[2023-10-09 15:44:45,895][86122] Updated weights for policy 1, policy_version 90120 (0.0007) +[2023-10-09 15:44:46,252][86122] Updated weights for policy 1, policy_version 90130 (0.0010) +[2023-10-09 15:44:46,621][86122] Updated weights for policy 1, policy_version 90140 (0.0008) +[2023-10-09 15:44:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 184254464. Throughput: 0: 1817.3, 1: 1827.6. Samples: 46070546. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:44:49,362][86121] Updated weights for policy 0, policy_version 89800 (0.0007) +[2023-10-09 15:44:49,731][86121] Updated weights for policy 0, policy_version 89810 (0.0008) +[2023-10-09 15:44:50,095][86121] Updated weights for policy 0, policy_version 89820 (0.0008) +[2023-10-09 15:44:50,277][86122] Updated weights for policy 1, policy_version 90150 (0.0008) +[2023-10-09 15:44:50,639][86122] Updated weights for policy 1, policy_version 90160 (0.0008) +[2023-10-09 15:44:51,005][86122] Updated weights for policy 1, policy_version 90170 (0.0010) +[2023-10-09 15:44:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 184320000. Throughput: 0: 1806.3, 1: 1830.7. Samples: 46093292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:44:53,926][86121] Updated weights for policy 0, policy_version 89830 (0.0008) +[2023-10-09 15:44:54,295][86121] Updated weights for policy 0, policy_version 89840 (0.0007) +[2023-10-09 15:44:54,539][86122] Updated weights for policy 1, policy_version 90180 (0.0009) +[2023-10-09 15:44:54,653][86121] Updated weights for policy 0, policy_version 89850 (0.0007) +[2023-10-09 15:44:54,891][86122] Updated weights for policy 1, policy_version 90190 (0.0007) +[2023-10-09 15:44:55,256][86122] Updated weights for policy 1, policy_version 90200 (0.0007) +[2023-10-09 15:44:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184385536. Throughput: 0: 1803.6, 1: 1832.1. Samples: 46103382. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:44:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:44:58,431][86121] Updated weights for policy 0, policy_version 89860 (0.0007) +[2023-10-09 15:44:58,791][86121] Updated weights for policy 0, policy_version 89870 (0.0011) +[2023-10-09 15:44:59,037][86122] Updated weights for policy 1, policy_version 90210 (0.0010) +[2023-10-09 15:44:59,157][86121] Updated weights for policy 0, policy_version 89880 (0.0009) +[2023-10-09 15:44:59,393][86122] Updated weights for policy 1, policy_version 90220 (0.0008) +[2023-10-09 15:44:59,753][86122] Updated weights for policy 1, policy_version 90230 (0.0009) +[2023-10-09 15:45:00,117][86122] Updated weights for policy 1, policy_version 90240 (0.0010) +[2023-10-09 15:45:02,965][86121] Updated weights for policy 0, policy_version 89890 (0.0008) +[2023-10-09 15:45:03,333][86121] Updated weights for policy 0, policy_version 89900 (0.0007) +[2023-10-09 15:45:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184451072. Throughput: 0: 1798.3, 1: 1825.1. Samples: 46125756. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:45:03,704][86121] Updated weights for policy 0, policy_version 89910 (0.0008) +[2023-10-09 15:45:03,804][86122] Updated weights for policy 1, policy_version 90250 (0.0008) +[2023-10-09 15:45:04,065][86121] Updated weights for policy 0, policy_version 89920 (0.0010) +[2023-10-09 15:45:04,172][86122] Updated weights for policy 1, policy_version 90260 (0.0009) +[2023-10-09 15:45:04,541][86122] Updated weights for policy 1, policy_version 90270 (0.0008) +[2023-10-09 15:45:07,790][86121] Updated weights for policy 0, policy_version 89930 (0.0007) +[2023-10-09 15:45:08,155][86121] Updated weights for policy 0, policy_version 89940 (0.0007) +[2023-10-09 15:45:08,226][86122] Updated weights for policy 1, policy_version 90280 (0.0008) +[2023-10-09 15:45:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 184516608. Throughput: 0: 1815.1, 1: 1822.0. Samples: 46148058. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:45:08,521][86121] Updated weights for policy 0, policy_version 89950 (0.0007) +[2023-10-09 15:45:08,585][86122] Updated weights for policy 1, policy_version 90290 (0.0008) +[2023-10-09 15:45:08,939][86122] Updated weights for policy 1, policy_version 90300 (0.0009) +[2023-10-09 15:45:12,166][86121] Updated weights for policy 0, policy_version 89960 (0.0010) +[2023-10-09 15:45:12,538][86121] Updated weights for policy 0, policy_version 89970 (0.0010) +[2023-10-09 15:45:12,698][86122] Updated weights for policy 1, policy_version 90310 (0.0008) +[2023-10-09 15:45:12,902][86121] Updated weights for policy 0, policy_version 89980 (0.0009) +[2023-10-09 15:45:13,056][86122] Updated weights for policy 1, policy_version 90320 (0.0007) +[2023-10-09 15:45:13,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 184614912. Throughput: 0: 1798.3, 1: 1819.8. Samples: 46158570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:13,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:45:13,414][86122] Updated weights for policy 1, policy_version 90330 (0.0009) +[2023-10-09 15:45:16,656][86121] Updated weights for policy 0, policy_version 89990 (0.0008) +[2023-10-09 15:45:17,028][86121] Updated weights for policy 0, policy_version 90000 (0.0007) +[2023-10-09 15:45:17,062][86122] Updated weights for policy 1, policy_version 90340 (0.0009) +[2023-10-09 15:45:17,393][86121] Updated weights for policy 0, policy_version 90010 (0.0007) +[2023-10-09 15:45:17,416][86122] Updated weights for policy 1, policy_version 90350 (0.0007) +[2023-10-09 15:45:17,771][86122] Updated weights for policy 1, policy_version 90360 (0.0009) +[2023-10-09 15:45:18,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 184713216. Throughput: 0: 1816.6, 1: 1820.8. Samples: 46180826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:45:21,036][86121] Updated weights for policy 0, policy_version 90020 (0.0008) +[2023-10-09 15:45:21,391][86121] Updated weights for policy 0, policy_version 90030 (0.0011) +[2023-10-09 15:45:21,405][86122] Updated weights for policy 1, policy_version 90370 (0.0009) +[2023-10-09 15:45:21,763][86121] Updated weights for policy 0, policy_version 90040 (0.0009) +[2023-10-09 15:45:21,771][86122] Updated weights for policy 1, policy_version 90380 (0.0007) +[2023-10-09 15:45:22,125][86122] Updated weights for policy 1, policy_version 90390 (0.0007) +[2023-10-09 15:45:22,486][86122] Updated weights for policy 1, policy_version 90400 (0.0007) +[2023-10-09 15:45:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 184778752. Throughput: 0: 1813.7, 1: 1825.4. Samples: 46201682. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:45:23,411][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000090048_92209152.pth... +[2023-10-09 15:45:23,412][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000090400_92569600.pth... +[2023-10-09 15:45:23,449][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000088704_90832896.pth +[2023-10-09 15:45:23,452][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000088352_90472448.pth +[2023-10-09 15:45:25,430][86121] Updated weights for policy 0, policy_version 90050 (0.0008) +[2023-10-09 15:45:25,826][86121] Updated weights for policy 0, policy_version 90060 (0.0010) +[2023-10-09 15:45:26,181][86121] Updated weights for policy 0, policy_version 90070 (0.0010) +[2023-10-09 15:45:26,260][86122] Updated weights for policy 1, policy_version 90410 (0.0007) +[2023-10-09 15:45:26,548][86121] Updated weights for policy 0, policy_version 90080 (0.0009) +[2023-10-09 15:45:26,622][86122] Updated weights for policy 1, policy_version 90420 (0.0009) +[2023-10-09 15:45:26,977][86122] Updated weights for policy 1, policy_version 90430 (0.0009) +[2023-10-09 15:45:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 184844288. Throughput: 0: 1818.3, 1: 1828.6. Samples: 46213680. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.990')] +[2023-10-09 15:45:30,151][86121] Updated weights for policy 0, policy_version 90090 (0.0007) +[2023-10-09 15:45:30,524][86121] Updated weights for policy 0, policy_version 90100 (0.0009) +[2023-10-09 15:45:30,722][86122] Updated weights for policy 1, policy_version 90440 (0.0009) +[2023-10-09 15:45:30,887][86121] Updated weights for policy 0, policy_version 90110 (0.0008) +[2023-10-09 15:45:31,080][86122] Updated weights for policy 1, policy_version 90450 (0.0010) +[2023-10-09 15:45:31,447][86122] Updated weights for policy 1, policy_version 90460 (0.0007) +[2023-10-09 15:45:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 184909824. Throughput: 0: 1805.7, 1: 1824.2. Samples: 46233894. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.990')] +[2023-10-09 15:45:34,702][86121] Updated weights for policy 0, policy_version 90120 (0.0007) +[2023-10-09 15:45:35,038][86122] Updated weights for policy 1, policy_version 90470 (0.0008) +[2023-10-09 15:45:35,082][86121] Updated weights for policy 0, policy_version 90130 (0.0007) +[2023-10-09 15:45:35,406][86122] Updated weights for policy 1, policy_version 90480 (0.0008) +[2023-10-09 15:45:35,437][86121] Updated weights for policy 0, policy_version 90140 (0.0009) +[2023-10-09 15:45:35,764][86122] Updated weights for policy 1, policy_version 90490 (0.0009) +[2023-10-09 15:45:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 184975360. Throughput: 0: 1809.2, 1: 1826.7. Samples: 46256908. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:45:39,222][86121] Updated weights for policy 0, policy_version 90150 (0.0008) +[2023-10-09 15:45:39,438][86122] Updated weights for policy 1, policy_version 90500 (0.0008) +[2023-10-09 15:45:39,590][86121] Updated weights for policy 0, policy_version 90160 (0.0008) +[2023-10-09 15:45:39,804][86122] Updated weights for policy 1, policy_version 90510 (0.0008) +[2023-10-09 15:45:39,956][86121] Updated weights for policy 0, policy_version 90170 (0.0008) +[2023-10-09 15:45:40,162][86122] Updated weights for policy 1, policy_version 90520 (0.0007) +[2023-10-09 15:45:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185040896. Throughput: 0: 1804.6, 1: 1821.8. Samples: 46266572. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:45:43,627][86121] Updated weights for policy 0, policy_version 90180 (0.0009) +[2023-10-09 15:45:43,839][86122] Updated weights for policy 1, policy_version 90530 (0.0011) +[2023-10-09 15:45:43,992][86121] Updated weights for policy 0, policy_version 90190 (0.0010) +[2023-10-09 15:45:44,209][86122] Updated weights for policy 1, policy_version 90540 (0.0007) +[2023-10-09 15:45:44,364][86121] Updated weights for policy 0, policy_version 90200 (0.0009) +[2023-10-09 15:45:44,573][86122] Updated weights for policy 1, policy_version 90550 (0.0007) +[2023-10-09 15:45:44,941][86122] Updated weights for policy 1, policy_version 90560 (0.0007) +[2023-10-09 15:45:48,135][86121] Updated weights for policy 0, policy_version 90210 (0.0008) +[2023-10-09 15:45:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185106432. Throughput: 0: 1808.0, 1: 1825.3. Samples: 46289256. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:48,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:45:48,492][86121] Updated weights for policy 0, policy_version 90220 (0.0008) +[2023-10-09 15:45:48,681][86122] Updated weights for policy 1, policy_version 90570 (0.0008) +[2023-10-09 15:45:48,858][86121] Updated weights for policy 0, policy_version 90230 (0.0007) +[2023-10-09 15:45:49,038][86122] Updated weights for policy 1, policy_version 90580 (0.0007) +[2023-10-09 15:45:49,210][86121] Updated weights for policy 0, policy_version 90240 (0.0008) +[2023-10-09 15:45:49,402][86122] Updated weights for policy 1, policy_version 90590 (0.0007) +[2023-10-09 15:45:52,870][86121] Updated weights for policy 0, policy_version 90250 (0.0008) +[2023-10-09 15:45:53,024][86122] Updated weights for policy 1, policy_version 90600 (0.0007) +[2023-10-09 15:45:53,238][86121] Updated weights for policy 0, policy_version 90260 (0.0007) +[2023-10-09 15:45:53,383][86122] Updated weights for policy 1, policy_version 90610 (0.0007) +[2023-10-09 15:45:53,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 185171968. Throughput: 0: 1813.0, 1: 1826.3. Samples: 46311826. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:45:53,603][86121] Updated weights for policy 0, policy_version 90270 (0.0007) +[2023-10-09 15:45:53,735][86122] Updated weights for policy 1, policy_version 90620 (0.0008) +[2023-10-09 15:45:57,364][86121] Updated weights for policy 0, policy_version 90280 (0.0009) +[2023-10-09 15:45:57,516][86122] Updated weights for policy 1, policy_version 90630 (0.0009) +[2023-10-09 15:45:57,718][86121] Updated weights for policy 0, policy_version 90290 (0.0007) +[2023-10-09 15:45:57,881][86122] Updated weights for policy 1, policy_version 90640 (0.0007) +[2023-10-09 15:45:58,077][86121] Updated weights for policy 0, policy_version 90300 (0.0007) +[2023-10-09 15:45:58,241][86122] Updated weights for policy 1, policy_version 90650 (0.0007) +[2023-10-09 15:45:58,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185270272. Throughput: 0: 1811.7, 1: 1826.7. Samples: 46322296. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:45:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:01,816][86121] Updated weights for policy 0, policy_version 90310 (0.0008) +[2023-10-09 15:46:01,975][86122] Updated weights for policy 1, policy_version 90660 (0.0007) +[2023-10-09 15:46:02,170][86121] Updated weights for policy 0, policy_version 90320 (0.0007) +[2023-10-09 15:46:02,339][86122] Updated weights for policy 1, policy_version 90670 (0.0007) +[2023-10-09 15:46:02,530][86121] Updated weights for policy 0, policy_version 90330 (0.0008) +[2023-10-09 15:46:02,699][86122] Updated weights for policy 1, policy_version 90680 (0.0008) +[2023-10-09 15:46:03,397][85186] Fps is (10 sec: 19661.2, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 185368576. Throughput: 0: 1812.8, 1: 1826.0. Samples: 46344574. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:46:03,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:06,156][86121] Updated weights for policy 0, policy_version 90340 (0.0010) +[2023-10-09 15:46:06,307][86122] Updated weights for policy 1, policy_version 90690 (0.0009) +[2023-10-09 15:46:06,525][86121] Updated weights for policy 0, policy_version 90350 (0.0008) +[2023-10-09 15:46:06,673][86122] Updated weights for policy 1, policy_version 90700 (0.0008) +[2023-10-09 15:46:06,902][86121] Updated weights for policy 0, policy_version 90360 (0.0007) +[2023-10-09 15:46:07,032][86122] Updated weights for policy 1, policy_version 90710 (0.0008) +[2023-10-09 15:46:07,395][86122] Updated weights for policy 1, policy_version 90720 (0.0007) +[2023-10-09 15:46:08,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 185434112. Throughput: 0: 1801.8, 1: 1827.0. Samples: 46364978. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:46:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:10,693][86121] Updated weights for policy 0, policy_version 90370 (0.0009) +[2023-10-09 15:46:11,103][86121] Updated weights for policy 0, policy_version 90380 (0.0007) +[2023-10-09 15:46:11,178][86122] Updated weights for policy 1, policy_version 90730 (0.0009) +[2023-10-09 15:46:11,462][86121] Updated weights for policy 0, policy_version 90390 (0.0007) +[2023-10-09 15:46:11,544][86122] Updated weights for policy 1, policy_version 90740 (0.0010) +[2023-10-09 15:46:11,827][86121] Updated weights for policy 0, policy_version 90400 (0.0007) +[2023-10-09 15:46:11,905][86122] Updated weights for policy 1, policy_version 90750 (0.0009) +[2023-10-09 15:46:13,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 185499648. Throughput: 0: 1809.2, 1: 1826.4. Samples: 46377282. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:46:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:15,455][86121] Updated weights for policy 0, policy_version 90410 (0.0008) +[2023-10-09 15:46:15,536][86122] Updated weights for policy 1, policy_version 90760 (0.0008) +[2023-10-09 15:46:15,815][86121] Updated weights for policy 0, policy_version 90420 (0.0009) +[2023-10-09 15:46:15,889][86122] Updated weights for policy 1, policy_version 90770 (0.0008) +[2023-10-09 15:46:16,174][86121] Updated weights for policy 0, policy_version 90430 (0.0008) +[2023-10-09 15:46:16,248][86122] Updated weights for policy 1, policy_version 90780 (0.0008) +[2023-10-09 15:46:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 185565184. Throughput: 0: 1807.9, 1: 1826.2. Samples: 46397430. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:19,856][86121] Updated weights for policy 0, policy_version 90440 (0.0009) +[2023-10-09 15:46:19,949][86122] Updated weights for policy 1, policy_version 90790 (0.0008) +[2023-10-09 15:46:20,215][86121] Updated weights for policy 0, policy_version 90450 (0.0007) +[2023-10-09 15:46:20,316][86122] Updated weights for policy 1, policy_version 90800 (0.0007) +[2023-10-09 15:46:20,593][86121] Updated weights for policy 0, policy_version 90460 (0.0008) +[2023-10-09 15:46:20,669][86122] Updated weights for policy 1, policy_version 90810 (0.0009) +[2023-10-09 15:46:23,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185630720. Throughput: 0: 1803.0, 1: 1825.7. Samples: 46420198. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:23,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:24,261][86122] Updated weights for policy 1, policy_version 90820 (0.0009) +[2023-10-09 15:46:24,351][86121] Updated weights for policy 0, policy_version 90470 (0.0009) +[2023-10-09 15:46:24,622][86122] Updated weights for policy 1, policy_version 90830 (0.0007) +[2023-10-09 15:46:24,725][86121] Updated weights for policy 0, policy_version 90480 (0.0008) +[2023-10-09 15:46:24,981][86122] Updated weights for policy 1, policy_version 90840 (0.0007) +[2023-10-09 15:46:25,086][86121] Updated weights for policy 0, policy_version 90490 (0.0008) +[2023-10-09 15:46:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185696256. Throughput: 0: 1804.0, 1: 1828.9. Samples: 46430050. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:28,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:28,415][86122] Updated weights for policy 1, policy_version 90850 (0.0009) +[2023-10-09 15:46:28,782][86122] Updated weights for policy 1, policy_version 90860 (0.0008) +[2023-10-09 15:46:28,841][86121] Updated weights for policy 0, policy_version 90500 (0.0009) +[2023-10-09 15:46:29,139][86122] Updated weights for policy 1, policy_version 90870 (0.0009) +[2023-10-09 15:46:29,213][86121] Updated weights for policy 0, policy_version 90510 (0.0007) +[2023-10-09 15:46:29,493][86122] Updated weights for policy 1, policy_version 90880 (0.0007) +[2023-10-09 15:46:29,575][86121] Updated weights for policy 0, policy_version 90520 (0.0008) +[2023-10-09 15:46:33,310][86121] Updated weights for policy 0, policy_version 90530 (0.0010) +[2023-10-09 15:46:33,363][86122] Updated weights for policy 1, policy_version 90890 (0.0008) +[2023-10-09 15:46:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185761792. Throughput: 0: 1807.5, 1: 1833.0. Samples: 46453080. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:33,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:33,678][86121] Updated weights for policy 0, policy_version 90540 (0.0008) +[2023-10-09 15:46:33,718][86122] Updated weights for policy 1, policy_version 90900 (0.0008) +[2023-10-09 15:46:34,043][86121] Updated weights for policy 0, policy_version 90550 (0.0007) +[2023-10-09 15:46:34,070][86122] Updated weights for policy 1, policy_version 90910 (0.0007) +[2023-10-09 15:46:34,410][86121] Updated weights for policy 0, policy_version 90560 (0.0007) +[2023-10-09 15:46:37,855][86122] Updated weights for policy 1, policy_version 90920 (0.0010) +[2023-10-09 15:46:38,214][86122] Updated weights for policy 1, policy_version 90930 (0.0008) +[2023-10-09 15:46:38,299][86121] Updated weights for policy 0, policy_version 90570 (0.0010) +[2023-10-09 15:46:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 185827328. Throughput: 0: 1813.1, 1: 1817.1. Samples: 46475182. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:38,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:38,584][86122] Updated weights for policy 1, policy_version 90940 (0.0007) +[2023-10-09 15:46:38,667][86121] Updated weights for policy 0, policy_version 90580 (0.0008) +[2023-10-09 15:46:39,029][86121] Updated weights for policy 0, policy_version 90590 (0.0007) +[2023-10-09 15:46:42,309][86122] Updated weights for policy 1, policy_version 90950 (0.0008) +[2023-10-09 15:46:42,676][86122] Updated weights for policy 1, policy_version 90960 (0.0008) +[2023-10-09 15:46:42,889][86121] Updated weights for policy 0, policy_version 90600 (0.0008) +[2023-10-09 15:46:43,037][86122] Updated weights for policy 1, policy_version 90970 (0.0007) +[2023-10-09 15:46:43,254][86121] Updated weights for policy 0, policy_version 90610 (0.0008) +[2023-10-09 15:46:43,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 185925632. Throughput: 0: 1796.9, 1: 1825.7. Samples: 46485314. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:43,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:43,624][86121] Updated weights for policy 0, policy_version 90620 (0.0008) +[2023-10-09 15:46:46,766][86122] Updated weights for policy 1, policy_version 90980 (0.0008) +[2023-10-09 15:46:47,141][86122] Updated weights for policy 1, policy_version 90990 (0.0007) +[2023-10-09 15:46:47,344][86121] Updated weights for policy 0, policy_version 90630 (0.0007) +[2023-10-09 15:46:47,503][86122] Updated weights for policy 1, policy_version 91000 (0.0008) +[2023-10-09 15:46:47,701][86121] Updated weights for policy 0, policy_version 90640 (0.0008) +[2023-10-09 15:46:48,067][86121] Updated weights for policy 0, policy_version 90650 (0.0010) +[2023-10-09 15:46:48,397][85186] Fps is (10 sec: 19660.4, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 186023936. Throughput: 0: 1811.5, 1: 1820.7. Samples: 46508022. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:51,166][86122] Updated weights for policy 1, policy_version 91010 (0.0007) +[2023-10-09 15:46:51,530][86122] Updated weights for policy 1, policy_version 91020 (0.0007) +[2023-10-09 15:46:51,641][86121] Updated weights for policy 0, policy_version 90660 (0.0009) +[2023-10-09 15:46:51,887][86122] Updated weights for policy 1, policy_version 91030 (0.0007) +[2023-10-09 15:46:51,999][86121] Updated weights for policy 0, policy_version 90670 (0.0008) +[2023-10-09 15:46:52,242][86122] Updated weights for policy 1, policy_version 91040 (0.0008) +[2023-10-09 15:46:52,363][86121] Updated weights for policy 0, policy_version 90680 (0.0009) +[2023-10-09 15:46:53,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 186089472. Throughput: 0: 1803.2, 1: 1824.9. Samples: 46528244. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:53,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:46:56,071][86122] Updated weights for policy 1, policy_version 91050 (0.0009) +[2023-10-09 15:46:56,098][86121] Updated weights for policy 0, policy_version 90690 (0.0008) +[2023-10-09 15:46:56,431][86122] Updated weights for policy 1, policy_version 91060 (0.0009) +[2023-10-09 15:46:56,508][86121] Updated weights for policy 0, policy_version 90700 (0.0007) +[2023-10-09 15:46:56,790][86122] Updated weights for policy 1, policy_version 91070 (0.0007) +[2023-10-09 15:46:56,876][86121] Updated weights for policy 0, policy_version 90710 (0.0007) +[2023-10-09 15:46:57,239][86121] Updated weights for policy 0, policy_version 90720 (0.0008) +[2023-10-09 15:46:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186155008. Throughput: 0: 1813.7, 1: 1822.6. Samples: 46540916. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:46:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:47:00,573][86122] Updated weights for policy 1, policy_version 91080 (0.0008) +[2023-10-09 15:47:00,912][86121] Updated weights for policy 0, policy_version 90730 (0.0010) +[2023-10-09 15:47:00,932][86122] Updated weights for policy 1, policy_version 91090 (0.0008) +[2023-10-09 15:47:01,273][86121] Updated weights for policy 0, policy_version 90740 (0.0007) +[2023-10-09 15:47:01,301][86122] Updated weights for policy 1, policy_version 91100 (0.0008) +[2023-10-09 15:47:01,634][86121] Updated weights for policy 0, policy_version 90750 (0.0008) +[2023-10-09 15:47:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186220544. Throughput: 0: 1802.3, 1: 1823.5. Samples: 46560590. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:47:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 15:47:04,911][86122] Updated weights for policy 1, policy_version 91110 (0.0007) +[2023-10-09 15:47:05,270][86122] Updated weights for policy 1, policy_version 91120 (0.0009) +[2023-10-09 15:47:05,380][86121] Updated weights for policy 0, policy_version 90760 (0.0009) +[2023-10-09 15:47:05,633][86122] Updated weights for policy 1, policy_version 91130 (0.0010) +[2023-10-09 15:47:05,742][86121] Updated weights for policy 0, policy_version 90770 (0.0008) +[2023-10-09 15:47:06,109][86121] Updated weights for policy 0, policy_version 90780 (0.0007) +[2023-10-09 15:47:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186286080. Throughput: 0: 1802.1, 1: 1819.1. Samples: 46583150. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:47:08,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 15:47:09,305][86122] Updated weights for policy 1, policy_version 91140 (0.0009) +[2023-10-09 15:47:09,669][86122] Updated weights for policy 1, policy_version 91150 (0.0008) +[2023-10-09 15:47:09,966][86121] Updated weights for policy 0, policy_version 90790 (0.0008) +[2023-10-09 15:47:10,021][86122] Updated weights for policy 1, policy_version 91160 (0.0007) +[2023-10-09 15:47:10,329][86121] Updated weights for policy 0, policy_version 90800 (0.0008) +[2023-10-09 15:47:10,705][86121] Updated weights for policy 0, policy_version 90810 (0.0009) +[2023-10-09 15:47:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186351616. Throughput: 0: 1805.2, 1: 1816.8. Samples: 46593042. Policy #0 lag: (min: 5.0, avg: 5.0, max: 5.0) +[2023-10-09 15:47:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:47:13,777][86122] Updated weights for policy 1, policy_version 91170 (0.0008) +[2023-10-09 15:47:14,143][86122] Updated weights for policy 1, policy_version 91180 (0.0009) +[2023-10-09 15:47:14,454][86121] Updated weights for policy 0, policy_version 90820 (0.0008) +[2023-10-09 15:47:14,502][86122] Updated weights for policy 1, policy_version 91190 (0.0008) +[2023-10-09 15:47:14,820][86121] Updated weights for policy 0, policy_version 90830 (0.0008) +[2023-10-09 15:47:14,863][86122] Updated weights for policy 1, policy_version 91200 (0.0007) +[2023-10-09 15:47:15,193][86121] Updated weights for policy 0, policy_version 90840 (0.0008) +[2023-10-09 15:47:18,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186417152. Throughput: 0: 1800.8, 1: 1808.3. Samples: 46615488. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:47:18,699][86122] Updated weights for policy 1, policy_version 91210 (0.0009) +[2023-10-09 15:47:18,860][86121] Updated weights for policy 0, policy_version 90850 (0.0008) +[2023-10-09 15:47:19,057][86122] Updated weights for policy 1, policy_version 91220 (0.0008) +[2023-10-09 15:47:19,221][86121] Updated weights for policy 0, policy_version 90860 (0.0008) +[2023-10-09 15:47:19,409][86122] Updated weights for policy 1, policy_version 91230 (0.0008) +[2023-10-09 15:47:19,586][86121] Updated weights for policy 0, policy_version 90870 (0.0008) +[2023-10-09 15:47:19,953][86121] Updated weights for policy 0, policy_version 90880 (0.0008) +[2023-10-09 15:47:22,977][86122] Updated weights for policy 1, policy_version 91240 (0.0008) +[2023-10-09 15:47:23,335][86122] Updated weights for policy 1, policy_version 91250 (0.0008) +[2023-10-09 15:47:23,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 186482688. Throughput: 0: 1810.4, 1: 1821.9. Samples: 46638632. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:23,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:47:23,624][86121] Updated weights for policy 0, policy_version 90890 (0.0008) +[2023-10-09 15:47:23,692][86122] Updated weights for policy 1, policy_version 91260 (0.0007) +[2023-10-09 15:47:23,838][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000091264_93454336.pth... +[2023-10-09 15:47:23,875][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000089536_91684864.pth +[2023-10-09 15:47:24,000][86121] Updated weights for policy 0, policy_version 90900 (0.0011) +[2023-10-09 15:47:24,364][86121] Updated weights for policy 0, policy_version 90910 (0.0008) +[2023-10-09 15:47:24,437][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000090912_93093888.pth... +[2023-10-09 15:47:24,473][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000089184_91324416.pth +[2023-10-09 15:47:27,451][86122] Updated weights for policy 1, policy_version 91270 (0.0010) +[2023-10-09 15:47:27,805][86122] Updated weights for policy 1, policy_version 91280 (0.0009) +[2023-10-09 15:47:28,154][86121] Updated weights for policy 0, policy_version 90920 (0.0008) +[2023-10-09 15:47:28,168][86122] Updated weights for policy 1, policy_version 91290 (0.0008) +[2023-10-09 15:47:28,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186580992. Throughput: 0: 1808.8, 1: 1816.3. Samples: 46648444. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:28,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:47:28,510][86121] Updated weights for policy 0, policy_version 90930 (0.0008) +[2023-10-09 15:47:28,879][86121] Updated weights for policy 0, policy_version 90940 (0.0007) +[2023-10-09 15:47:31,803][86122] Updated weights for policy 1, policy_version 91300 (0.0007) +[2023-10-09 15:47:32,166][86122] Updated weights for policy 1, policy_version 91310 (0.0007) +[2023-10-09 15:47:32,526][86122] Updated weights for policy 1, policy_version 91320 (0.0008) +[2023-10-09 15:47:32,549][86121] Updated weights for policy 0, policy_version 90950 (0.0008) +[2023-10-09 15:47:32,914][86121] Updated weights for policy 0, policy_version 90960 (0.0008) +[2023-10-09 15:47:33,286][86121] Updated weights for policy 0, policy_version 90970 (0.0008) +[2023-10-09 15:47:33,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 186646528. Throughput: 0: 1805.2, 1: 1820.5. Samples: 46671178. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:33,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:47:36,270][86122] Updated weights for policy 1, policy_version 91330 (0.0009) +[2023-10-09 15:47:36,630][86122] Updated weights for policy 1, policy_version 91340 (0.0007) +[2023-10-09 15:47:36,990][86122] Updated weights for policy 1, policy_version 91350 (0.0008) +[2023-10-09 15:47:37,029][86121] Updated weights for policy 0, policy_version 90980 (0.0008) +[2023-10-09 15:47:37,355][86122] Updated weights for policy 1, policy_version 91360 (0.0008) +[2023-10-09 15:47:37,390][86121] Updated weights for policy 0, policy_version 90990 (0.0008) +[2023-10-09 15:47:37,760][86121] Updated weights for policy 0, policy_version 91000 (0.0008) +[2023-10-09 15:47:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 186744832. Throughput: 0: 1808.9, 1: 1820.7. Samples: 46691576. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:38,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:47:40,948][86122] Updated weights for policy 1, policy_version 91370 (0.0009) +[2023-10-09 15:47:41,323][86122] Updated weights for policy 1, policy_version 91380 (0.0009) +[2023-10-09 15:47:41,512][86121] Updated weights for policy 0, policy_version 91010 (0.0009) +[2023-10-09 15:47:41,675][86122] Updated weights for policy 1, policy_version 91390 (0.0008) +[2023-10-09 15:47:41,905][86121] Updated weights for policy 0, policy_version 91020 (0.0007) +[2023-10-09 15:47:42,264][86121] Updated weights for policy 0, policy_version 91030 (0.0007) +[2023-10-09 15:47:42,627][86121] Updated weights for policy 0, policy_version 91040 (0.0008) +[2023-10-09 15:47:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 186810368. Throughput: 0: 1801.9, 1: 1821.6. Samples: 46703974. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:43,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:47:45,328][86122] Updated weights for policy 1, policy_version 91400 (0.0008) +[2023-10-09 15:47:45,693][86122] Updated weights for policy 1, policy_version 91410 (0.0010) +[2023-10-09 15:47:46,052][86122] Updated weights for policy 1, policy_version 91420 (0.0010) +[2023-10-09 15:47:46,402][86121] Updated weights for policy 0, policy_version 91050 (0.0008) +[2023-10-09 15:47:46,777][86121] Updated weights for policy 0, policy_version 91060 (0.0009) +[2023-10-09 15:47:47,150][86121] Updated weights for policy 0, policy_version 91070 (0.0009) +[2023-10-09 15:47:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 186875904. Throughput: 0: 1812.1, 1: 1831.9. Samples: 46724570. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:47:49,666][86122] Updated weights for policy 1, policy_version 91430 (0.0008) +[2023-10-09 15:47:50,034][86122] Updated weights for policy 1, policy_version 91440 (0.0007) +[2023-10-09 15:47:50,388][86122] Updated weights for policy 1, policy_version 91450 (0.0008) +[2023-10-09 15:47:50,815][86121] Updated weights for policy 0, policy_version 91080 (0.0008) +[2023-10-09 15:47:51,173][86121] Updated weights for policy 0, policy_version 91090 (0.0008) +[2023-10-09 15:47:51,538][86121] Updated weights for policy 0, policy_version 91100 (0.0008) +[2023-10-09 15:47:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 186941440. Throughput: 0: 1810.0, 1: 1834.7. Samples: 46747164. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.990')] +[2023-10-09 15:47:54,167][86122] Updated weights for policy 1, policy_version 91460 (0.0008) +[2023-10-09 15:47:54,537][86122] Updated weights for policy 1, policy_version 91470 (0.0008) +[2023-10-09 15:47:54,891][86122] Updated weights for policy 1, policy_version 91480 (0.0009) +[2023-10-09 15:47:55,323][86121] Updated weights for policy 0, policy_version 91110 (0.0009) +[2023-10-09 15:47:55,694][86121] Updated weights for policy 0, policy_version 91120 (0.0011) +[2023-10-09 15:47:56,059][86121] Updated weights for policy 0, policy_version 91130 (0.0008) +[2023-10-09 15:47:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 187006976. Throughput: 0: 1820.6, 1: 1834.0. Samples: 46757498. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:47:58,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:47:58,580][86122] Updated weights for policy 1, policy_version 91490 (0.0007) +[2023-10-09 15:47:58,954][86122] Updated weights for policy 1, policy_version 91500 (0.0007) +[2023-10-09 15:47:59,312][86122] Updated weights for policy 1, policy_version 91510 (0.0008) +[2023-10-09 15:47:59,609][86121] Updated weights for policy 0, policy_version 91140 (0.0008) +[2023-10-09 15:47:59,674][86122] Updated weights for policy 1, policy_version 91520 (0.0009) +[2023-10-09 15:47:59,981][86121] Updated weights for policy 0, policy_version 91150 (0.0009) +[2023-10-09 15:48:00,354][86121] Updated weights for policy 0, policy_version 91160 (0.0009) +[2023-10-09 15:48:03,174][86122] Updated weights for policy 1, policy_version 91530 (0.0007) +[2023-10-09 15:48:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 187072512. Throughput: 0: 1810.6, 1: 1838.0. Samples: 46779672. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:48:03,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:48:03,530][86122] Updated weights for policy 1, policy_version 91540 (0.0007) +[2023-10-09 15:48:03,886][86122] Updated weights for policy 1, policy_version 91550 (0.0008) +[2023-10-09 15:48:03,979][86121] Updated weights for policy 0, policy_version 91170 (0.0009) +[2023-10-09 15:48:04,346][86121] Updated weights for policy 0, policy_version 91180 (0.0008) +[2023-10-09 15:48:04,710][86121] Updated weights for policy 0, policy_version 91190 (0.0007) +[2023-10-09 15:48:05,069][86121] Updated weights for policy 0, policy_version 91200 (0.0007) +[2023-10-09 15:48:07,578][86122] Updated weights for policy 1, policy_version 91560 (0.0009) +[2023-10-09 15:48:07,954][86122] Updated weights for policy 1, policy_version 91570 (0.0010) +[2023-10-09 15:48:08,313][86122] Updated weights for policy 1, policy_version 91580 (0.0008) +[2023-10-09 15:48:08,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 187138048. Throughput: 0: 1810.5, 1: 1825.3. Samples: 46802242. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:48:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:48:08,800][86121] Updated weights for policy 0, policy_version 91210 (0.0007) +[2023-10-09 15:48:09,165][86121] Updated weights for policy 0, policy_version 91220 (0.0010) +[2023-10-09 15:48:09,526][86121] Updated weights for policy 0, policy_version 91230 (0.0010) +[2023-10-09 15:48:11,919][86122] Updated weights for policy 1, policy_version 91590 (0.0009) +[2023-10-09 15:48:12,280][86122] Updated weights for policy 1, policy_version 91600 (0.0008) +[2023-10-09 15:48:12,640][86122] Updated weights for policy 1, policy_version 91610 (0.0007) +[2023-10-09 15:48:13,145][86121] Updated weights for policy 0, policy_version 91240 (0.0008) +[2023-10-09 15:48:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187236352. Throughput: 0: 1816.4, 1: 1840.9. Samples: 46813020. Policy #0 lag: (min: 16.0, avg: 34.7, max: 48.0) +[2023-10-09 15:48:13,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:48:13,505][86121] Updated weights for policy 0, policy_version 91250 (0.0008) +[2023-10-09 15:48:13,871][86121] Updated weights for policy 0, policy_version 91260 (0.0008) +[2023-10-09 15:48:16,279][86122] Updated weights for policy 1, policy_version 91620 (0.0009) +[2023-10-09 15:48:16,639][86122] Updated weights for policy 1, policy_version 91630 (0.0008) +[2023-10-09 15:48:17,001][86122] Updated weights for policy 1, policy_version 91640 (0.0008) +[2023-10-09 15:48:17,549][86121] Updated weights for policy 0, policy_version 91270 (0.0008) +[2023-10-09 15:48:17,917][86121] Updated weights for policy 0, policy_version 91280 (0.0009) +[2023-10-09 15:48:18,283][86121] Updated weights for policy 0, policy_version 91290 (0.0008) +[2023-10-09 15:48:18,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187301888. Throughput: 0: 1818.4, 1: 1827.4. Samples: 46835240. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:18,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.990')] +[2023-10-09 15:48:20,670][86122] Updated weights for policy 1, policy_version 91650 (0.0009) +[2023-10-09 15:48:21,029][86122] Updated weights for policy 1, policy_version 91660 (0.0009) +[2023-10-09 15:48:21,395][86122] Updated weights for policy 1, policy_version 91670 (0.0009) +[2023-10-09 15:48:21,754][86122] Updated weights for policy 1, policy_version 91680 (0.0009) +[2023-10-09 15:48:22,055][86121] Updated weights for policy 0, policy_version 91300 (0.0008) +[2023-10-09 15:48:22,429][86121] Updated weights for policy 0, policy_version 91310 (0.0008) +[2023-10-09 15:48:22,794][86121] Updated weights for policy 0, policy_version 91320 (0.0008) +[2023-10-09 15:48:23,397][85186] Fps is (10 sec: 16384.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 187400192. Throughput: 0: 1823.9, 1: 1841.3. Samples: 46856510. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:23,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.990')] +[2023-10-09 15:48:25,384][86122] Updated weights for policy 1, policy_version 91690 (0.0009) +[2023-10-09 15:48:25,751][86122] Updated weights for policy 1, policy_version 91700 (0.0007) +[2023-10-09 15:48:26,100][86122] Updated weights for policy 1, policy_version 91710 (0.0008) +[2023-10-09 15:48:26,636][86121] Updated weights for policy 0, policy_version 91330 (0.0008) +[2023-10-09 15:48:27,048][86121] Updated weights for policy 0, policy_version 91340 (0.0007) +[2023-10-09 15:48:27,419][86121] Updated weights for policy 0, policy_version 91350 (0.0008) +[2023-10-09 15:48:27,774][86121] Updated weights for policy 0, policy_version 91360 (0.0007) +[2023-10-09 15:48:28,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187465728. Throughput: 0: 1823.6, 1: 1825.2. Samples: 46868170. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:28,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:48:29,691][86122] Updated weights for policy 1, policy_version 91720 (0.0009) +[2023-10-09 15:48:30,055][86122] Updated weights for policy 1, policy_version 91730 (0.0010) +[2023-10-09 15:48:30,423][86122] Updated weights for policy 1, policy_version 91740 (0.0010) +[2023-10-09 15:48:31,372][86121] Updated weights for policy 0, policy_version 91370 (0.0010) +[2023-10-09 15:48:31,738][86121] Updated weights for policy 0, policy_version 91380 (0.0007) +[2023-10-09 15:48:32,109][86121] Updated weights for policy 0, policy_version 91390 (0.0007) +[2023-10-09 15:48:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 187531264. Throughput: 0: 1820.3, 1: 1846.5. Samples: 46889576. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:33,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:48:34,100][86122] Updated weights for policy 1, policy_version 91750 (0.0009) +[2023-10-09 15:48:34,482][86122] Updated weights for policy 1, policy_version 91760 (0.0010) +[2023-10-09 15:48:34,834][86122] Updated weights for policy 1, policy_version 91770 (0.0010) +[2023-10-09 15:48:35,745][86121] Updated weights for policy 0, policy_version 91400 (0.0008) +[2023-10-09 15:48:36,115][86121] Updated weights for policy 0, policy_version 91410 (0.0007) +[2023-10-09 15:48:36,478][86121] Updated weights for policy 0, policy_version 91420 (0.0009) +[2023-10-09 15:48:38,354][86122] Updated weights for policy 1, policy_version 91780 (0.0009) +[2023-10-09 15:48:38,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 187596800. Throughput: 0: 1820.0, 1: 1847.1. Samples: 46912182. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:38,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:48:38,719][86122] Updated weights for policy 1, policy_version 91790 (0.0008) +[2023-10-09 15:48:39,073][86122] Updated weights for policy 1, policy_version 91800 (0.0011) +[2023-10-09 15:48:40,251][86121] Updated weights for policy 0, policy_version 91430 (0.0011) +[2023-10-09 15:48:40,616][86121] Updated weights for policy 0, policy_version 91440 (0.0008) +[2023-10-09 15:48:40,982][86121] Updated weights for policy 0, policy_version 91450 (0.0009) +[2023-10-09 15:48:42,774][86122] Updated weights for policy 1, policy_version 91810 (0.0009) +[2023-10-09 15:48:43,136][86122] Updated weights for policy 1, policy_version 91820 (0.0007) +[2023-10-09 15:48:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 187662336. Throughput: 0: 1820.7, 1: 1850.2. Samples: 46922688. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:43,398][85186] Avg episode reward: [(0, '9.950'), (1, '9.980')] +[2023-10-09 15:48:43,496][86122] Updated weights for policy 1, policy_version 91830 (0.0010) +[2023-10-09 15:48:43,849][86122] Updated weights for policy 1, policy_version 91840 (0.0008) +[2023-10-09 15:48:44,555][86121] Updated weights for policy 0, policy_version 91460 (0.0008) +[2023-10-09 15:48:44,921][86121] Updated weights for policy 0, policy_version 91470 (0.0008) +[2023-10-09 15:48:45,297][86121] Updated weights for policy 0, policy_version 91480 (0.0009) +[2023-10-09 15:48:47,526][86122] Updated weights for policy 1, policy_version 91850 (0.0008) +[2023-10-09 15:48:47,880][86122] Updated weights for policy 1, policy_version 91860 (0.0008) +[2023-10-09 15:48:48,239][86122] Updated weights for policy 1, policy_version 91870 (0.0009) +[2023-10-09 15:48:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187760640. Throughput: 0: 1827.7, 1: 1854.4. Samples: 46945364. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:48,398][85186] Avg episode reward: [(0, '9.940'), (1, '9.980')] +[2023-10-09 15:48:48,923][86121] Updated weights for policy 0, policy_version 91490 (0.0011) +[2023-10-09 15:48:49,283][86121] Updated weights for policy 0, policy_version 91500 (0.0011) +[2023-10-09 15:48:49,655][86121] Updated weights for policy 0, policy_version 91510 (0.0008) +[2023-10-09 15:48:50,017][86121] Updated weights for policy 0, policy_version 91520 (0.0008) +[2023-10-09 15:48:51,907][86122] Updated weights for policy 1, policy_version 91880 (0.0008) +[2023-10-09 15:48:52,274][86122] Updated weights for policy 1, policy_version 91890 (0.0008) +[2023-10-09 15:48:52,639][86122] Updated weights for policy 1, policy_version 91900 (0.0008) +[2023-10-09 15:48:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187826176. Throughput: 0: 1822.4, 1: 1836.4. Samples: 46966892. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:53,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 15:48:53,706][86121] Updated weights for policy 0, policy_version 91530 (0.0009) +[2023-10-09 15:48:54,081][86121] Updated weights for policy 0, policy_version 91540 (0.0010) +[2023-10-09 15:48:54,437][86121] Updated weights for policy 0, policy_version 91550 (0.0009) +[2023-10-09 15:48:56,449][86122] Updated weights for policy 1, policy_version 91910 (0.0008) +[2023-10-09 15:48:56,806][86122] Updated weights for policy 1, policy_version 91920 (0.0007) +[2023-10-09 15:48:57,159][86122] Updated weights for policy 1, policy_version 91930 (0.0010) +[2023-10-09 15:48:58,082][86121] Updated weights for policy 0, policy_version 91560 (0.0009) +[2023-10-09 15:48:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187891712. Throughput: 0: 1819.6, 1: 1847.4. Samples: 46978036. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:48:58,398][85186] Avg episode reward: [(0, '9.930'), (1, '9.980')] +[2023-10-09 15:48:58,454][86121] Updated weights for policy 0, policy_version 91570 (0.0008) +[2023-10-09 15:48:58,817][86121] Updated weights for policy 0, policy_version 91580 (0.0009) +[2023-10-09 15:49:01,060][86122] Updated weights for policy 1, policy_version 91940 (0.0008) +[2023-10-09 15:49:01,415][86122] Updated weights for policy 1, policy_version 91950 (0.0010) +[2023-10-09 15:49:01,772][86122] Updated weights for policy 1, policy_version 91960 (0.0008) +[2023-10-09 15:49:02,481][86121] Updated weights for policy 0, policy_version 91590 (0.0009) +[2023-10-09 15:49:02,841][86121] Updated weights for policy 0, policy_version 91600 (0.0009) +[2023-10-09 15:49:03,221][86121] Updated weights for policy 0, policy_version 91610 (0.0009) +[2023-10-09 15:49:03,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 187957248. Throughput: 0: 1821.5, 1: 1838.2. Samples: 46999926. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:49:03,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 15:49:05,419][86122] Updated weights for policy 1, policy_version 91970 (0.0008) +[2023-10-09 15:49:05,784][86122] Updated weights for policy 1, policy_version 91980 (0.0009) +[2023-10-09 15:49:06,144][86122] Updated weights for policy 1, policy_version 91990 (0.0008) +[2023-10-09 15:49:06,496][86122] Updated weights for policy 1, policy_version 92000 (0.0011) +[2023-10-09 15:49:06,915][86121] Updated weights for policy 0, policy_version 91620 (0.0008) +[2023-10-09 15:49:07,287][86121] Updated weights for policy 0, policy_version 91630 (0.0007) +[2023-10-09 15:49:07,647][86121] Updated weights for policy 0, policy_version 91640 (0.0007) +[2023-10-09 15:49:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 188055552. Throughput: 0: 1816.8, 1: 1847.5. Samples: 47021402. Policy #0 lag: (min: 6.0, avg: 14.0, max: 38.0) +[2023-10-09 15:49:08,398][85186] Avg episode reward: [(0, '9.920'), (1, '9.980')] +[2023-10-09 15:49:10,099][86122] Updated weights for policy 1, policy_version 92010 (0.0009) +[2023-10-09 15:49:10,452][86122] Updated weights for policy 1, policy_version 92020 (0.0009) +[2023-10-09 15:49:10,817][86122] Updated weights for policy 1, policy_version 92030 (0.0007) +[2023-10-09 15:49:11,510][86121] Updated weights for policy 0, policy_version 91650 (0.0008) +[2023-10-09 15:49:11,922][86121] Updated weights for policy 0, policy_version 91660 (0.0008) +[2023-10-09 15:49:12,288][86121] Updated weights for policy 0, policy_version 91670 (0.0007) +[2023-10-09 15:49:12,652][86121] Updated weights for policy 0, policy_version 91680 (0.0007) +[2023-10-09 15:49:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 188121088. Throughput: 0: 1822.1, 1: 1838.8. Samples: 47032908. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:13,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:49:14,540][86122] Updated weights for policy 1, policy_version 92040 (0.0009) +[2023-10-09 15:49:14,906][86122] Updated weights for policy 1, policy_version 92050 (0.0009) +[2023-10-09 15:49:15,274][86122] Updated weights for policy 1, policy_version 92060 (0.0008) +[2023-10-09 15:49:16,244][86121] Updated weights for policy 0, policy_version 91690 (0.0007) +[2023-10-09 15:49:16,618][86121] Updated weights for policy 0, policy_version 91700 (0.0010) +[2023-10-09 15:49:16,986][86121] Updated weights for policy 0, policy_version 91710 (0.0007) +[2023-10-09 15:49:18,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188186624. Throughput: 0: 1824.4, 1: 1839.9. Samples: 47054472. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:18,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:49:19,041][86122] Updated weights for policy 1, policy_version 92070 (0.0007) +[2023-10-09 15:49:19,424][86122] Updated weights for policy 1, policy_version 92080 (0.0009) +[2023-10-09 15:49:19,785][86122] Updated weights for policy 1, policy_version 92090 (0.0007) +[2023-10-09 15:49:20,707][86121] Updated weights for policy 0, policy_version 91720 (0.0008) +[2023-10-09 15:49:21,081][86121] Updated weights for policy 0, policy_version 91730 (0.0007) +[2023-10-09 15:49:21,445][86121] Updated weights for policy 0, policy_version 91740 (0.0008) +[2023-10-09 15:49:23,358][86122] Updated weights for policy 1, policy_version 92100 (0.0008) +[2023-10-09 15:49:23,398][85186] Fps is (10 sec: 13106.7, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 188252160. Throughput: 0: 1830.1, 1: 1836.7. Samples: 47077186. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:23,399][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 15:49:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000091744_93945856.pth... +[2023-10-09 15:49:23,446][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000090048_92209152.pth +[2023-10-09 15:49:23,711][86122] Updated weights for policy 1, policy_version 92110 (0.0010) +[2023-10-09 15:49:24,067][86122] Updated weights for policy 1, policy_version 92120 (0.0009) +[2023-10-09 15:49:24,355][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000092128_94339072.pth... +[2023-10-09 15:49:24,383][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000090400_92569600.pth +[2023-10-09 15:49:25,031][86121] Updated weights for policy 0, policy_version 91750 (0.0009) +[2023-10-09 15:49:25,388][86121] Updated weights for policy 0, policy_version 91760 (0.0010) +[2023-10-09 15:49:25,753][86121] Updated weights for policy 0, policy_version 91770 (0.0008) +[2023-10-09 15:49:27,584][86122] Updated weights for policy 1, policy_version 92130 (0.0007) +[2023-10-09 15:49:27,947][86122] Updated weights for policy 1, policy_version 92140 (0.0007) +[2023-10-09 15:49:28,313][86122] Updated weights for policy 1, policy_version 92150 (0.0007) +[2023-10-09 15:49:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188317696. Throughput: 0: 1828.5, 1: 1839.7. Samples: 47087758. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.980')] +[2023-10-09 15:49:28,676][86122] Updated weights for policy 1, policy_version 92160 (0.0009) +[2023-10-09 15:49:29,417][86121] Updated weights for policy 0, policy_version 91780 (0.0008) +[2023-10-09 15:49:29,780][86121] Updated weights for policy 0, policy_version 91790 (0.0008) +[2023-10-09 15:49:30,142][86121] Updated weights for policy 0, policy_version 91800 (0.0007) +[2023-10-09 15:49:32,319][86122] Updated weights for policy 1, policy_version 92170 (0.0009) +[2023-10-09 15:49:32,681][86122] Updated weights for policy 1, policy_version 92180 (0.0009) +[2023-10-09 15:49:33,043][86122] Updated weights for policy 1, policy_version 92190 (0.0008) +[2023-10-09 15:49:33,397][85186] Fps is (10 sec: 16384.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188416000. Throughput: 0: 1824.6, 1: 1840.3. Samples: 47110286. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:33,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.980')] +[2023-10-09 15:49:33,838][86121] Updated weights for policy 0, policy_version 91810 (0.0009) +[2023-10-09 15:49:34,202][86121] Updated weights for policy 0, policy_version 91820 (0.0010) +[2023-10-09 15:49:34,570][86121] Updated weights for policy 0, policy_version 91830 (0.0008) +[2023-10-09 15:49:34,941][86121] Updated weights for policy 0, policy_version 91840 (0.0008) +[2023-10-09 15:49:36,568][86122] Updated weights for policy 1, policy_version 92200 (0.0008) +[2023-10-09 15:49:36,926][86122] Updated weights for policy 1, policy_version 92210 (0.0007) +[2023-10-09 15:49:37,290][86122] Updated weights for policy 1, policy_version 92220 (0.0007) +[2023-10-09 15:49:38,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188481536. Throughput: 0: 1825.7, 1: 1840.6. Samples: 47131878. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:38,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.990')] +[2023-10-09 15:49:38,668][86121] Updated weights for policy 0, policy_version 91850 (0.0010) +[2023-10-09 15:49:39,024][86121] Updated weights for policy 0, policy_version 91860 (0.0011) +[2023-10-09 15:49:39,388][86121] Updated weights for policy 0, policy_version 91870 (0.0009) +[2023-10-09 15:49:40,894][86122] Updated weights for policy 1, policy_version 92230 (0.0008) +[2023-10-09 15:49:41,267][86122] Updated weights for policy 1, policy_version 92240 (0.0009) +[2023-10-09 15:49:41,624][86122] Updated weights for policy 1, policy_version 92250 (0.0008) +[2023-10-09 15:49:43,028][86121] Updated weights for policy 0, policy_version 91880 (0.0010) +[2023-10-09 15:49:43,391][86121] Updated weights for policy 0, policy_version 91890 (0.0011) +[2023-10-09 15:49:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 188547072. Throughput: 0: 1828.2, 1: 1844.0. Samples: 47143286. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:43,398][85186] Avg episode reward: [(0, '9.890'), (1, '9.990')] +[2023-10-09 15:49:43,764][86121] Updated weights for policy 0, policy_version 91900 (0.0011) +[2023-10-09 15:49:45,278][86122] Updated weights for policy 1, policy_version 92260 (0.0010) +[2023-10-09 15:49:45,640][86122] Updated weights for policy 1, policy_version 92270 (0.0009) +[2023-10-09 15:49:46,006][86122] Updated weights for policy 1, policy_version 92280 (0.0009) +[2023-10-09 15:49:47,445][86121] Updated weights for policy 0, policy_version 91910 (0.0010) +[2023-10-09 15:49:47,811][86121] Updated weights for policy 0, policy_version 91920 (0.0011) +[2023-10-09 15:49:48,177][86121] Updated weights for policy 0, policy_version 91930 (0.0010) +[2023-10-09 15:49:48,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 188612608. Throughput: 0: 1825.0, 1: 1840.0. Samples: 47164850. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:48,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.990')] +[2023-10-09 15:49:49,679][86122] Updated weights for policy 1, policy_version 92290 (0.0011) +[2023-10-09 15:49:50,037][86122] Updated weights for policy 1, policy_version 92300 (0.0007) +[2023-10-09 15:49:50,398][86122] Updated weights for policy 1, policy_version 92310 (0.0010) +[2023-10-09 15:49:50,759][86122] Updated weights for policy 1, policy_version 92320 (0.0010) +[2023-10-09 15:49:51,952][86121] Updated weights for policy 0, policy_version 91940 (0.0010) +[2023-10-09 15:49:52,318][86121] Updated weights for policy 0, policy_version 91950 (0.0010) +[2023-10-09 15:49:52,690][86121] Updated weights for policy 0, policy_version 91960 (0.0008) +[2023-10-09 15:49:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188710912. Throughput: 0: 1824.4, 1: 1837.9. Samples: 47186208. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:53,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:49:54,550][86122] Updated weights for policy 1, policy_version 92330 (0.0007) +[2023-10-09 15:49:54,911][86122] Updated weights for policy 1, policy_version 92340 (0.0007) +[2023-10-09 15:49:55,273][86122] Updated weights for policy 1, policy_version 92350 (0.0007) +[2023-10-09 15:49:56,388][86121] Updated weights for policy 0, policy_version 91970 (0.0007) +[2023-10-09 15:49:56,791][86121] Updated weights for policy 0, policy_version 91980 (0.0008) +[2023-10-09 15:49:57,159][86121] Updated weights for policy 0, policy_version 91990 (0.0010) +[2023-10-09 15:49:57,519][86121] Updated weights for policy 0, policy_version 92000 (0.0009) +[2023-10-09 15:49:58,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188776448. Throughput: 0: 1825.2, 1: 1832.0. Samples: 47197482. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:49:58,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:49:59,082][86122] Updated weights for policy 1, policy_version 92360 (0.0008) +[2023-10-09 15:49:59,440][86122] Updated weights for policy 1, policy_version 92370 (0.0008) +[2023-10-09 15:49:59,807][86122] Updated weights for policy 1, policy_version 92380 (0.0008) +[2023-10-09 15:50:01,184][86121] Updated weights for policy 0, policy_version 92010 (0.0009) +[2023-10-09 15:50:01,555][86121] Updated weights for policy 0, policy_version 92020 (0.0007) +[2023-10-09 15:50:01,915][86121] Updated weights for policy 0, policy_version 92030 (0.0007) +[2023-10-09 15:50:03,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 188841984. Throughput: 0: 1818.5, 1: 1837.8. Samples: 47219006. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:50:03,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.990')] +[2023-10-09 15:50:03,517][86122] Updated weights for policy 1, policy_version 92390 (0.0009) +[2023-10-09 15:50:03,869][86122] Updated weights for policy 1, policy_version 92400 (0.0010) +[2023-10-09 15:50:04,226][86122] Updated weights for policy 1, policy_version 92410 (0.0007) +[2023-10-09 15:50:05,361][86121] Updated weights for policy 0, policy_version 92040 (0.0008) +[2023-10-09 15:50:05,723][86121] Updated weights for policy 0, policy_version 92050 (0.0007) +[2023-10-09 15:50:06,087][86121] Updated weights for policy 0, policy_version 92060 (0.0008) +[2023-10-09 15:50:08,005][86122] Updated weights for policy 1, policy_version 92420 (0.0008) +[2023-10-09 15:50:08,393][86122] Updated weights for policy 1, policy_version 92430 (0.0009) +[2023-10-09 15:50:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 188907520. Throughput: 0: 1825.7, 1: 1837.6. Samples: 47242030. Policy #0 lag: (min: 13.0, avg: 13.0, max: 13.0) +[2023-10-09 15:50:08,398][85186] Avg episode reward: [(0, '9.900'), (1, '9.990')] +[2023-10-09 15:50:08,751][86122] Updated weights for policy 1, policy_version 92440 (0.0007) +[2023-10-09 15:50:09,684][86121] Updated weights for policy 0, policy_version 92070 (0.0008) +[2023-10-09 15:50:10,044][86121] Updated weights for policy 0, policy_version 92080 (0.0009) +[2023-10-09 15:50:10,423][86121] Updated weights for policy 0, policy_version 92090 (0.0009) +[2023-10-09 15:50:12,397][86122] Updated weights for policy 1, policy_version 92450 (0.0009) +[2023-10-09 15:50:12,762][86122] Updated weights for policy 1, policy_version 92460 (0.0009) +[2023-10-09 15:50:13,125][86122] Updated weights for policy 1, policy_version 92470 (0.0009) +[2023-10-09 15:50:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 188973056. Throughput: 0: 1817.7, 1: 1828.9. Samples: 47251854. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:13,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:50:13,488][86122] Updated weights for policy 1, policy_version 92480 (0.0007) +[2023-10-09 15:50:14,120][86121] Updated weights for policy 0, policy_version 92100 (0.0008) +[2023-10-09 15:50:14,488][86121] Updated weights for policy 0, policy_version 92110 (0.0008) +[2023-10-09 15:50:14,854][86121] Updated weights for policy 0, policy_version 92120 (0.0008) +[2023-10-09 15:50:17,066][86122] Updated weights for policy 1, policy_version 92490 (0.0009) +[2023-10-09 15:50:17,432][86122] Updated weights for policy 1, policy_version 92500 (0.0009) +[2023-10-09 15:50:17,778][86122] Updated weights for policy 1, policy_version 92510 (0.0010) +[2023-10-09 15:50:18,379][86121] Updated weights for policy 0, policy_version 92130 (0.0008) +[2023-10-09 15:50:18,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189071360. Throughput: 0: 1832.7, 1: 1823.6. Samples: 47274820. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:18,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:50:18,752][86121] Updated weights for policy 0, policy_version 92140 (0.0010) +[2023-10-09 15:50:19,113][86121] Updated weights for policy 0, policy_version 92150 (0.0011) +[2023-10-09 15:50:19,472][86121] Updated weights for policy 0, policy_version 92160 (0.0011) +[2023-10-09 15:50:21,466][86122] Updated weights for policy 1, policy_version 92520 (0.0007) +[2023-10-09 15:50:21,827][86122] Updated weights for policy 1, policy_version 92530 (0.0009) +[2023-10-09 15:50:22,177][86122] Updated weights for policy 1, policy_version 92540 (0.0011) +[2023-10-09 15:50:23,216][86121] Updated weights for policy 0, policy_version 92170 (0.0008) +[2023-10-09 15:50:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189136896. Throughput: 0: 1832.8, 1: 1825.9. Samples: 47296518. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:23,398][85186] Avg episode reward: [(0, '9.910'), (1, '9.990')] +[2023-10-09 15:50:23,584][86121] Updated weights for policy 0, policy_version 92180 (0.0009) +[2023-10-09 15:50:23,957][86121] Updated weights for policy 0, policy_version 92190 (0.0010) +[2023-10-09 15:50:25,774][86122] Updated weights for policy 1, policy_version 92550 (0.0011) +[2023-10-09 15:50:26,143][86122] Updated weights for policy 1, policy_version 92560 (0.0008) +[2023-10-09 15:50:26,504][86122] Updated weights for policy 1, policy_version 92570 (0.0009) +[2023-10-09 15:50:27,632][86121] Updated weights for policy 0, policy_version 92200 (0.0007) +[2023-10-09 15:50:27,994][86121] Updated weights for policy 0, policy_version 92210 (0.0007) +[2023-10-09 15:50:28,358][86121] Updated weights for policy 0, policy_version 92220 (0.0008) +[2023-10-09 15:50:28,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 189202432. Throughput: 0: 1834.1, 1: 1823.1. Samples: 47307858. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:28,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 15:50:30,024][86122] Updated weights for policy 1, policy_version 92580 (0.0009) +[2023-10-09 15:50:30,372][86122] Updated weights for policy 1, policy_version 92590 (0.0009) +[2023-10-09 15:50:30,736][86122] Updated weights for policy 1, policy_version 92600 (0.0011) +[2023-10-09 15:50:32,009][86121] Updated weights for policy 0, policy_version 92230 (0.0010) +[2023-10-09 15:50:32,370][86121] Updated weights for policy 0, policy_version 92240 (0.0011) +[2023-10-09 15:50:32,744][86121] Updated weights for policy 0, policy_version 92250 (0.0011) +[2023-10-09 15:50:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 189300736. Throughput: 0: 1829.0, 1: 1826.8. Samples: 47329362. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:33,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 15:50:34,401][86122] Updated weights for policy 1, policy_version 92610 (0.0009) +[2023-10-09 15:50:34,761][86122] Updated weights for policy 1, policy_version 92620 (0.0007) +[2023-10-09 15:50:35,127][86122] Updated weights for policy 1, policy_version 92630 (0.0008) +[2023-10-09 15:50:35,482][86122] Updated weights for policy 1, policy_version 92640 (0.0009) +[2023-10-09 15:50:36,569][86121] Updated weights for policy 0, policy_version 92260 (0.0009) +[2023-10-09 15:50:36,939][86121] Updated weights for policy 0, policy_version 92270 (0.0010) +[2023-10-09 15:50:37,305][86121] Updated weights for policy 0, policy_version 92280 (0.0010) +[2023-10-09 15:50:38,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189366272. Throughput: 0: 1827.8, 1: 1839.1. Samples: 47351218. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:38,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 15:50:39,170][86122] Updated weights for policy 1, policy_version 92650 (0.0011) +[2023-10-09 15:50:39,537][86122] Updated weights for policy 1, policy_version 92660 (0.0008) +[2023-10-09 15:50:39,894][86122] Updated weights for policy 1, policy_version 92670 (0.0009) +[2023-10-09 15:50:41,106][86121] Updated weights for policy 0, policy_version 92290 (0.0010) +[2023-10-09 15:50:41,512][86121] Updated weights for policy 0, policy_version 92300 (0.0008) +[2023-10-09 15:50:41,879][86121] Updated weights for policy 0, policy_version 92310 (0.0008) +[2023-10-09 15:50:42,236][86121] Updated weights for policy 0, policy_version 92320 (0.0009) +[2023-10-09 15:50:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 189431808. Throughput: 0: 1830.3, 1: 1840.7. Samples: 47362674. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:43,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 15:50:43,608][86122] Updated weights for policy 1, policy_version 92680 (0.0009) +[2023-10-09 15:50:43,965][86122] Updated weights for policy 1, policy_version 92690 (0.0008) +[2023-10-09 15:50:44,324][86122] Updated weights for policy 1, policy_version 92700 (0.0010) +[2023-10-09 15:50:45,781][86121] Updated weights for policy 0, policy_version 92330 (0.0008) +[2023-10-09 15:50:46,148][86121] Updated weights for policy 0, policy_version 92340 (0.0009) +[2023-10-09 15:50:46,524][86121] Updated weights for policy 0, policy_version 92350 (0.0008) +[2023-10-09 15:50:47,992][86122] Updated weights for policy 1, policy_version 92710 (0.0009) +[2023-10-09 15:50:48,355][86122] Updated weights for policy 1, policy_version 92720 (0.0007) +[2023-10-09 15:50:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 189497344. Throughput: 0: 1831.9, 1: 1839.9. Samples: 47384242. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:48,398][85186] Avg episode reward: [(0, '9.910'), (1, '10.000')] +[2023-10-09 15:50:48,723][86122] Updated weights for policy 1, policy_version 92730 (0.0008) +[2023-10-09 15:50:50,074][86121] Updated weights for policy 0, policy_version 92360 (0.0007) +[2023-10-09 15:50:50,435][86121] Updated weights for policy 0, policy_version 92370 (0.0009) +[2023-10-09 15:50:50,804][86121] Updated weights for policy 0, policy_version 92380 (0.0007) +[2023-10-09 15:50:52,606][86122] Updated weights for policy 1, policy_version 92740 (0.0009) +[2023-10-09 15:50:52,958][86122] Updated weights for policy 1, policy_version 92750 (0.0010) +[2023-10-09 15:50:53,315][86122] Updated weights for policy 1, policy_version 92760 (0.0009) +[2023-10-09 15:50:53,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 189562880. Throughput: 0: 1837.3, 1: 1820.7. Samples: 47406638. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:53,398][85186] Avg episode reward: [(0, '9.920'), (1, '10.000')] +[2023-10-09 15:50:54,571][86121] Updated weights for policy 0, policy_version 92390 (0.0008) +[2023-10-09 15:50:54,944][86121] Updated weights for policy 0, policy_version 92400 (0.0008) +[2023-10-09 15:50:55,310][86121] Updated weights for policy 0, policy_version 92410 (0.0008) +[2023-10-09 15:50:56,868][86122] Updated weights for policy 1, policy_version 92770 (0.0007) +[2023-10-09 15:50:57,268][86122] Updated weights for policy 1, policy_version 92780 (0.0007) +[2023-10-09 15:50:57,625][86122] Updated weights for policy 1, policy_version 92790 (0.0008) +[2023-10-09 15:50:57,983][86122] Updated weights for policy 1, policy_version 92800 (0.0007) +[2023-10-09 15:50:58,398][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189661184. Throughput: 0: 1836.5, 1: 1838.7. Samples: 47417240. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:50:58,399][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:50:59,109][86121] Updated weights for policy 0, policy_version 92420 (0.0010) +[2023-10-09 15:50:59,478][86121] Updated weights for policy 0, policy_version 92430 (0.0008) +[2023-10-09 15:50:59,837][86121] Updated weights for policy 0, policy_version 92440 (0.0008) +[2023-10-09 15:51:01,627][86122] Updated weights for policy 1, policy_version 92810 (0.0010) +[2023-10-09 15:51:01,988][86122] Updated weights for policy 1, policy_version 92820 (0.0009) +[2023-10-09 15:51:02,350][86122] Updated weights for policy 1, policy_version 92830 (0.0007) +[2023-10-09 15:51:03,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189726720. Throughput: 0: 1832.8, 1: 1828.0. Samples: 47439558. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:51:03,398][85186] Avg episode reward: [(0, '9.930'), (1, '10.000')] +[2023-10-09 15:51:03,411][86121] Updated weights for policy 0, policy_version 92450 (0.0010) +[2023-10-09 15:51:03,785][86121] Updated weights for policy 0, policy_version 92460 (0.0008) +[2023-10-09 15:51:04,144][86121] Updated weights for policy 0, policy_version 92470 (0.0009) +[2023-10-09 15:51:04,503][86121] Updated weights for policy 0, policy_version 92480 (0.0009) +[2023-10-09 15:51:05,990][86122] Updated weights for policy 1, policy_version 92840 (0.0007) +[2023-10-09 15:51:06,348][86122] Updated weights for policy 1, policy_version 92850 (0.0008) +[2023-10-09 15:51:06,705][86122] Updated weights for policy 1, policy_version 92860 (0.0007) +[2023-10-09 15:51:08,141][86121] Updated weights for policy 0, policy_version 92490 (0.0008) +[2023-10-09 15:51:08,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14745.7, 300 sec: 14551.2). Total num frames: 189792256. Throughput: 0: 1833.5, 1: 1839.9. Samples: 47461818. Policy #0 lag: (min: 19.0, avg: 27.0, max: 51.0) +[2023-10-09 15:51:08,398][85186] Avg episode reward: [(0, '9.940'), (1, '10.000')] +[2023-10-09 15:51:08,500][86121] Updated weights for policy 0, policy_version 92500 (0.0008) +[2023-10-09 15:51:08,875][86121] Updated weights for policy 0, policy_version 92510 (0.0010) +[2023-10-09 15:51:10,535][86122] Updated weights for policy 1, policy_version 92870 (0.0007) +[2023-10-09 15:51:10,891][86122] Updated weights for policy 1, policy_version 92880 (0.0007) +[2023-10-09 15:51:11,257][86122] Updated weights for policy 1, policy_version 92890 (0.0007) +[2023-10-09 15:51:12,531][86121] Updated weights for policy 0, policy_version 92520 (0.0010) +[2023-10-09 15:51:12,898][86121] Updated weights for policy 0, policy_version 92530 (0.0011) +[2023-10-09 15:51:13,259][86121] Updated weights for policy 0, policy_version 92540 (0.0010) +[2023-10-09 15:51:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 189857792. Throughput: 0: 1835.4, 1: 1829.3. Samples: 47472768. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:13,398][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 15:51:14,736][86122] Updated weights for policy 1, policy_version 92900 (0.0010) +[2023-10-09 15:51:15,107][86122] Updated weights for policy 1, policy_version 92910 (0.0010) +[2023-10-09 15:51:15,471][86122] Updated weights for policy 1, policy_version 92920 (0.0011) +[2023-10-09 15:51:17,030][86121] Updated weights for policy 0, policy_version 92550 (0.0008) +[2023-10-09 15:51:17,386][86121] Updated weights for policy 0, policy_version 92560 (0.0007) +[2023-10-09 15:51:17,756][86121] Updated weights for policy 0, policy_version 92570 (0.0008) +[2023-10-09 15:51:18,398][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 189956096. Throughput: 0: 1838.6, 1: 1837.0. Samples: 47494764. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:18,399][85186] Avg episode reward: [(0, '9.950'), (1, '10.000')] +[2023-10-09 15:51:19,202][86122] Updated weights for policy 1, policy_version 92930 (0.0007) +[2023-10-09 15:51:19,565][86122] Updated weights for policy 1, policy_version 92940 (0.0009) +[2023-10-09 15:51:19,926][86122] Updated weights for policy 1, policy_version 92950 (0.0008) +[2023-10-09 15:51:20,290][86122] Updated weights for policy 1, policy_version 92960 (0.0008) +[2023-10-09 15:51:21,261][86121] Updated weights for policy 0, policy_version 92580 (0.0008) +[2023-10-09 15:51:21,627][86121] Updated weights for policy 0, policy_version 92590 (0.0009) +[2023-10-09 15:51:21,984][86121] Updated weights for policy 0, policy_version 92600 (0.0011) +[2023-10-09 15:51:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190021632. Throughput: 0: 1840.0, 1: 1832.4. Samples: 47516474. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:23,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:51:23,405][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000092960_95191040.pth... +[2023-10-09 15:51:23,406][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000092608_94830592.pth... +[2023-10-09 15:51:23,440][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000091264_93454336.pth +[2023-10-09 15:51:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000090912_93093888.pth +[2023-10-09 15:51:23,974][86122] Updated weights for policy 1, policy_version 92970 (0.0007) +[2023-10-09 15:51:24,338][86122] Updated weights for policy 1, policy_version 92980 (0.0007) +[2023-10-09 15:51:24,695][86122] Updated weights for policy 1, policy_version 92990 (0.0008) +[2023-10-09 15:51:25,504][86121] Updated weights for policy 0, policy_version 92610 (0.0008) +[2023-10-09 15:51:25,874][86121] Updated weights for policy 0, policy_version 92620 (0.0007) +[2023-10-09 15:51:26,237][86121] Updated weights for policy 0, policy_version 92630 (0.0007) +[2023-10-09 15:51:26,603][86121] Updated weights for policy 0, policy_version 92640 (0.0011) +[2023-10-09 15:51:28,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190087168. Throughput: 0: 1832.9, 1: 1829.4. Samples: 47527478. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 15:51:28,422][86122] Updated weights for policy 1, policy_version 93000 (0.0008) +[2023-10-09 15:51:28,788][86122] Updated weights for policy 1, policy_version 93010 (0.0007) +[2023-10-09 15:51:29,142][86122] Updated weights for policy 1, policy_version 93020 (0.0009) +[2023-10-09 15:51:30,208][86121] Updated weights for policy 0, policy_version 92650 (0.0008) +[2023-10-09 15:51:30,576][86121] Updated weights for policy 0, policy_version 92660 (0.0007) +[2023-10-09 15:51:30,940][86121] Updated weights for policy 0, policy_version 92670 (0.0010) +[2023-10-09 15:51:32,716][86122] Updated weights for policy 1, policy_version 93030 (0.0010) +[2023-10-09 15:51:33,069][86122] Updated weights for policy 1, policy_version 93040 (0.0011) +[2023-10-09 15:51:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190152704. Throughput: 0: 1842.1, 1: 1829.8. Samples: 47549478. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:51:33,426][86122] Updated weights for policy 1, policy_version 93050 (0.0011) +[2023-10-09 15:51:34,619][86121] Updated weights for policy 0, policy_version 92680 (0.0008) +[2023-10-09 15:51:34,994][86121] Updated weights for policy 0, policy_version 92690 (0.0007) +[2023-10-09 15:51:35,366][86121] Updated weights for policy 0, policy_version 92700 (0.0011) +[2023-10-09 15:51:37,109][86122] Updated weights for policy 1, policy_version 93060 (0.0008) +[2023-10-09 15:51:37,477][86122] Updated weights for policy 1, policy_version 93070 (0.0007) +[2023-10-09 15:51:37,837][86122] Updated weights for policy 1, policy_version 93080 (0.0008) +[2023-10-09 15:51:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190251008. Throughput: 0: 1839.6, 1: 1824.7. Samples: 47571532. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:51:39,058][86121] Updated weights for policy 0, policy_version 92710 (0.0008) +[2023-10-09 15:51:39,421][86121] Updated weights for policy 0, policy_version 92720 (0.0008) +[2023-10-09 15:51:39,786][86121] Updated weights for policy 0, policy_version 92730 (0.0008) +[2023-10-09 15:51:41,624][86122] Updated weights for policy 1, policy_version 93090 (0.0008) +[2023-10-09 15:51:41,982][86122] Updated weights for policy 1, policy_version 93100 (0.0009) +[2023-10-09 15:51:42,343][86122] Updated weights for policy 1, policy_version 93110 (0.0009) +[2023-10-09 15:51:42,703][86122] Updated weights for policy 1, policy_version 93120 (0.0010) +[2023-10-09 15:51:43,351][86121] Updated weights for policy 0, policy_version 92740 (0.0007) +[2023-10-09 15:51:43,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190316544. Throughput: 0: 1840.0, 1: 1833.8. Samples: 47582558. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 15:51:43,715][86121] Updated weights for policy 0, policy_version 92750 (0.0009) +[2023-10-09 15:51:44,082][86121] Updated weights for policy 0, policy_version 92760 (0.0011) +[2023-10-09 15:51:46,496][86122] Updated weights for policy 1, policy_version 93130 (0.0008) +[2023-10-09 15:51:46,872][86122] Updated weights for policy 1, policy_version 93140 (0.0009) +[2023-10-09 15:51:47,231][86122] Updated weights for policy 1, policy_version 93150 (0.0009) +[2023-10-09 15:51:47,775][86121] Updated weights for policy 0, policy_version 92770 (0.0009) +[2023-10-09 15:51:48,148][86121] Updated weights for policy 0, policy_version 92780 (0.0008) +[2023-10-09 15:51:48,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 190382080. Throughput: 0: 1842.1, 1: 1823.2. Samples: 47604498. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:51:48,518][86121] Updated weights for policy 0, policy_version 92790 (0.0007) +[2023-10-09 15:51:48,884][86121] Updated weights for policy 0, policy_version 92800 (0.0008) +[2023-10-09 15:51:50,907][86122] Updated weights for policy 1, policy_version 93160 (0.0008) +[2023-10-09 15:51:51,266][86122] Updated weights for policy 1, policy_version 93170 (0.0007) +[2023-10-09 15:51:51,639][86122] Updated weights for policy 1, policy_version 93180 (0.0008) +[2023-10-09 15:51:52,569][86121] Updated weights for policy 0, policy_version 92810 (0.0008) +[2023-10-09 15:51:52,942][86121] Updated weights for policy 0, policy_version 92820 (0.0007) +[2023-10-09 15:51:53,314][86121] Updated weights for policy 0, policy_version 92830 (0.0008) +[2023-10-09 15:51:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 190480384. Throughput: 0: 1819.3, 1: 1824.7. Samples: 47625800. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:51:55,396][86122] Updated weights for policy 1, policy_version 93190 (0.0008) +[2023-10-09 15:51:55,753][86122] Updated weights for policy 1, policy_version 93200 (0.0008) +[2023-10-09 15:51:56,109][86122] Updated weights for policy 1, policy_version 93210 (0.0008) +[2023-10-09 15:51:57,016][86121] Updated weights for policy 0, policy_version 92840 (0.0009) +[2023-10-09 15:51:57,386][86121] Updated weights for policy 0, policy_version 92850 (0.0008) +[2023-10-09 15:51:57,754][86121] Updated weights for policy 0, policy_version 92860 (0.0008) +[2023-10-09 15:51:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190545920. Throughput: 0: 1837.5, 1: 1820.0. Samples: 47637358. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:51:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:51:59,824][86122] Updated weights for policy 1, policy_version 93220 (0.0007) +[2023-10-09 15:52:00,176][86122] Updated weights for policy 1, policy_version 93230 (0.0011) +[2023-10-09 15:52:00,536][86122] Updated weights for policy 1, policy_version 93240 (0.0012) +[2023-10-09 15:52:01,564][86121] Updated weights for policy 0, policy_version 92870 (0.0009) +[2023-10-09 15:52:01,925][86121] Updated weights for policy 0, policy_version 92880 (0.0008) +[2023-10-09 15:52:02,302][86121] Updated weights for policy 0, policy_version 92890 (0.0008) +[2023-10-09 15:52:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190611456. Throughput: 0: 1822.1, 1: 1821.4. Samples: 47658724. Policy #0 lag: (min: 24.0, avg: 47.2, max: 56.0) +[2023-10-09 15:52:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:52:04,142][86122] Updated weights for policy 1, policy_version 93250 (0.0010) +[2023-10-09 15:52:04,506][86122] Updated weights for policy 1, policy_version 93260 (0.0007) +[2023-10-09 15:52:04,871][86122] Updated weights for policy 1, policy_version 93270 (0.0007) +[2023-10-09 15:52:05,229][86122] Updated weights for policy 1, policy_version 93280 (0.0007) +[2023-10-09 15:52:05,747][86121] Updated weights for policy 0, policy_version 92900 (0.0008) +[2023-10-09 15:52:06,121][86121] Updated weights for policy 0, policy_version 92910 (0.0009) +[2023-10-09 15:52:06,487][86121] Updated weights for policy 0, policy_version 92920 (0.0011) +[2023-10-09 15:52:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190676992. Throughput: 0: 1839.6, 1: 1825.6. Samples: 47681404. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.980')] +[2023-10-09 15:52:08,950][86122] Updated weights for policy 1, policy_version 93290 (0.0008) +[2023-10-09 15:52:09,313][86122] Updated weights for policy 1, policy_version 93300 (0.0009) +[2023-10-09 15:52:09,678][86122] Updated weights for policy 1, policy_version 93310 (0.0008) +[2023-10-09 15:52:10,302][86121] Updated weights for policy 0, policy_version 92930 (0.0008) +[2023-10-09 15:52:10,663][86121] Updated weights for policy 0, policy_version 92940 (0.0008) +[2023-10-09 15:52:11,031][86121] Updated weights for policy 0, policy_version 92950 (0.0008) +[2023-10-09 15:52:11,396][86121] Updated weights for policy 0, policy_version 92960 (0.0009) +[2023-10-09 15:52:13,266][86122] Updated weights for policy 1, policy_version 93320 (0.0008) +[2023-10-09 15:52:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190742528. Throughput: 0: 1826.0, 1: 1826.8. Samples: 47691854. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:52:13,634][86122] Updated weights for policy 1, policy_version 93330 (0.0007) +[2023-10-09 15:52:13,992][86122] Updated weights for policy 1, policy_version 93340 (0.0008) +[2023-10-09 15:52:15,242][86121] Updated weights for policy 0, policy_version 92970 (0.0010) +[2023-10-09 15:52:15,609][86121] Updated weights for policy 0, policy_version 92980 (0.0008) +[2023-10-09 15:52:15,978][86121] Updated weights for policy 0, policy_version 92990 (0.0009) +[2023-10-09 15:52:17,589][86122] Updated weights for policy 1, policy_version 93350 (0.0009) +[2023-10-09 15:52:17,958][86122] Updated weights for policy 1, policy_version 93360 (0.0009) +[2023-10-09 15:52:18,312][86122] Updated weights for policy 1, policy_version 93370 (0.0011) +[2023-10-09 15:52:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 190808064. Throughput: 0: 1832.0, 1: 1829.2. Samples: 47714230. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:19,554][86121] Updated weights for policy 0, policy_version 93000 (0.0011) +[2023-10-09 15:52:19,923][86121] Updated weights for policy 0, policy_version 93010 (0.0009) +[2023-10-09 15:52:20,289][86121] Updated weights for policy 0, policy_version 93020 (0.0010) +[2023-10-09 15:52:22,243][86122] Updated weights for policy 1, policy_version 93380 (0.0010) +[2023-10-09 15:52:22,607][86122] Updated weights for policy 1, policy_version 93390 (0.0010) +[2023-10-09 15:52:22,977][86122] Updated weights for policy 1, policy_version 93400 (0.0009) +[2023-10-09 15:52:23,397][85186] Fps is (10 sec: 16383.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190906368. Throughput: 0: 1824.1, 1: 1830.7. Samples: 47736000. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:23,953][86121] Updated weights for policy 0, policy_version 93030 (0.0010) +[2023-10-09 15:52:24,320][86121] Updated weights for policy 0, policy_version 93040 (0.0007) +[2023-10-09 15:52:24,691][86121] Updated weights for policy 0, policy_version 93050 (0.0008) +[2023-10-09 15:52:26,570][86122] Updated weights for policy 1, policy_version 93410 (0.0008) +[2023-10-09 15:52:26,926][86122] Updated weights for policy 1, policy_version 93420 (0.0007) +[2023-10-09 15:52:27,290][86122] Updated weights for policy 1, policy_version 93430 (0.0007) +[2023-10-09 15:52:27,651][86122] Updated weights for policy 1, policy_version 93440 (0.0007) +[2023-10-09 15:52:28,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 190971904. Throughput: 0: 1823.0, 1: 1827.2. Samples: 47746816. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:28,524][86121] Updated weights for policy 0, policy_version 93060 (0.0008) +[2023-10-09 15:52:28,888][86121] Updated weights for policy 0, policy_version 93070 (0.0008) +[2023-10-09 15:52:29,256][86121] Updated weights for policy 0, policy_version 93080 (0.0010) +[2023-10-09 15:52:31,453][86122] Updated weights for policy 1, policy_version 93450 (0.0007) +[2023-10-09 15:52:31,817][86122] Updated weights for policy 1, policy_version 93460 (0.0010) +[2023-10-09 15:52:32,171][86122] Updated weights for policy 1, policy_version 93470 (0.0008) +[2023-10-09 15:52:32,902][86121] Updated weights for policy 0, policy_version 93090 (0.0008) +[2023-10-09 15:52:33,265][86121] Updated weights for policy 0, policy_version 93100 (0.0007) +[2023-10-09 15:52:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 191037440. Throughput: 0: 1816.4, 1: 1824.4. Samples: 47768336. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:33,634][86121] Updated weights for policy 0, policy_version 93110 (0.0009) +[2023-10-09 15:52:33,995][86121] Updated weights for policy 0, policy_version 93120 (0.0008) +[2023-10-09 15:52:35,714][86122] Updated weights for policy 1, policy_version 93480 (0.0007) +[2023-10-09 15:52:36,081][86122] Updated weights for policy 1, policy_version 93490 (0.0007) +[2023-10-09 15:52:36,436][86122] Updated weights for policy 1, policy_version 93500 (0.0007) +[2023-10-09 15:52:37,836][86121] Updated weights for policy 0, policy_version 93130 (0.0007) +[2023-10-09 15:52:38,198][86121] Updated weights for policy 0, policy_version 93140 (0.0008) +[2023-10-09 15:52:38,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191102976. Throughput: 0: 1826.4, 1: 1834.5. Samples: 47790538. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:38,557][86121] Updated weights for policy 0, policy_version 93150 (0.0009) +[2023-10-09 15:52:40,038][86122] Updated weights for policy 1, policy_version 93510 (0.0008) +[2023-10-09 15:52:40,402][86122] Updated weights for policy 1, policy_version 93520 (0.0008) +[2023-10-09 15:52:40,757][86122] Updated weights for policy 1, policy_version 93530 (0.0008) +[2023-10-09 15:52:42,008][86121] Updated weights for policy 0, policy_version 93160 (0.0008) +[2023-10-09 15:52:42,371][86121] Updated weights for policy 0, policy_version 93170 (0.0010) +[2023-10-09 15:52:42,739][86121] Updated weights for policy 0, policy_version 93180 (0.0007) +[2023-10-09 15:52:43,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191201280. Throughput: 0: 1821.5, 1: 1828.7. Samples: 47801618. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:44,412][86122] Updated weights for policy 1, policy_version 93540 (0.0008) +[2023-10-09 15:52:44,773][86122] Updated weights for policy 1, policy_version 93550 (0.0007) +[2023-10-09 15:52:45,144][86122] Updated weights for policy 1, policy_version 93560 (0.0008) +[2023-10-09 15:52:46,491][86121] Updated weights for policy 0, policy_version 93190 (0.0009) +[2023-10-09 15:52:46,868][86121] Updated weights for policy 0, policy_version 93200 (0.0008) +[2023-10-09 15:52:47,228][86121] Updated weights for policy 0, policy_version 93210 (0.0010) +[2023-10-09 15:52:48,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191266816. Throughput: 0: 1818.4, 1: 1840.9. Samples: 47823392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:48,665][86122] Updated weights for policy 1, policy_version 93570 (0.0009) +[2023-10-09 15:52:49,026][86122] Updated weights for policy 1, policy_version 93580 (0.0009) +[2023-10-09 15:52:49,387][86122] Updated weights for policy 1, policy_version 93590 (0.0007) +[2023-10-09 15:52:49,749][86122] Updated weights for policy 1, policy_version 93600 (0.0007) +[2023-10-09 15:52:50,980][86121] Updated weights for policy 0, policy_version 93220 (0.0009) +[2023-10-09 15:52:51,335][86121] Updated weights for policy 0, policy_version 93230 (0.0010) +[2023-10-09 15:52:51,702][86121] Updated weights for policy 0, policy_version 93240 (0.0009) +[2023-10-09 15:52:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 191332352. Throughput: 0: 1815.9, 1: 1836.5. Samples: 47845762. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:53,466][86122] Updated weights for policy 1, policy_version 93610 (0.0010) +[2023-10-09 15:52:53,825][86122] Updated weights for policy 1, policy_version 93620 (0.0011) +[2023-10-09 15:52:54,194][86122] Updated weights for policy 1, policy_version 93630 (0.0007) +[2023-10-09 15:52:55,470][86121] Updated weights for policy 0, policy_version 93250 (0.0009) +[2023-10-09 15:52:55,836][86121] Updated weights for policy 0, policy_version 93260 (0.0009) +[2023-10-09 15:52:56,200][86121] Updated weights for policy 0, policy_version 93270 (0.0007) +[2023-10-09 15:52:56,570][86121] Updated weights for policy 0, policy_version 93280 (0.0007) +[2023-10-09 15:52:57,887][86122] Updated weights for policy 1, policy_version 93640 (0.0008) +[2023-10-09 15:52:58,243][86122] Updated weights for policy 1, policy_version 93650 (0.0011) +[2023-10-09 15:52:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 191397888. Throughput: 0: 1820.8, 1: 1839.5. Samples: 47856570. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:52:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:52:58,619][86122] Updated weights for policy 1, policy_version 93660 (0.0009) +[2023-10-09 15:53:00,231][86121] Updated weights for policy 0, policy_version 93290 (0.0010) +[2023-10-09 15:53:00,605][86121] Updated weights for policy 0, policy_version 93300 (0.0011) +[2023-10-09 15:53:00,967][86121] Updated weights for policy 0, policy_version 93310 (0.0009) +[2023-10-09 15:53:02,279][86122] Updated weights for policy 1, policy_version 93670 (0.0009) +[2023-10-09 15:53:02,645][86122] Updated weights for policy 1, policy_version 93680 (0.0008) +[2023-10-09 15:53:03,009][86122] Updated weights for policy 1, policy_version 93690 (0.0007) +[2023-10-09 15:53:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14773.4). Total num frames: 191496192. Throughput: 0: 1816.0, 1: 1838.3. Samples: 47878670. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:53:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:04,648][86121] Updated weights for policy 0, policy_version 93320 (0.0009) +[2023-10-09 15:53:05,024][86121] Updated weights for policy 0, policy_version 93330 (0.0009) +[2023-10-09 15:53:05,392][86121] Updated weights for policy 0, policy_version 93340 (0.0007) +[2023-10-09 15:53:06,719][86122] Updated weights for policy 1, policy_version 93700 (0.0008) +[2023-10-09 15:53:07,072][86122] Updated weights for policy 1, policy_version 93710 (0.0007) +[2023-10-09 15:53:07,442][86122] Updated weights for policy 1, policy_version 93720 (0.0008) +[2023-10-09 15:53:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191561728. Throughput: 0: 1823.3, 1: 1828.1. Samples: 47900314. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:09,073][86121] Updated weights for policy 0, policy_version 93350 (0.0007) +[2023-10-09 15:53:09,437][86121] Updated weights for policy 0, policy_version 93360 (0.0008) +[2023-10-09 15:53:09,812][86121] Updated weights for policy 0, policy_version 93370 (0.0008) +[2023-10-09 15:53:11,027][86122] Updated weights for policy 1, policy_version 93730 (0.0008) +[2023-10-09 15:53:11,386][86122] Updated weights for policy 1, policy_version 93740 (0.0010) +[2023-10-09 15:53:11,749][86122] Updated weights for policy 1, policy_version 93750 (0.0010) +[2023-10-09 15:53:12,108][86122] Updated weights for policy 1, policy_version 93760 (0.0011) +[2023-10-09 15:53:13,349][86121] Updated weights for policy 0, policy_version 93380 (0.0008) +[2023-10-09 15:53:13,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191627264. Throughput: 0: 1821.8, 1: 1845.6. Samples: 47911848. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:13,712][86121] Updated weights for policy 0, policy_version 93390 (0.0009) +[2023-10-09 15:53:14,088][86121] Updated weights for policy 0, policy_version 93400 (0.0010) +[2023-10-09 15:53:15,829][86122] Updated weights for policy 1, policy_version 93770 (0.0007) +[2023-10-09 15:53:16,190][86122] Updated weights for policy 1, policy_version 93780 (0.0007) +[2023-10-09 15:53:16,549][86122] Updated weights for policy 1, policy_version 93790 (0.0008) +[2023-10-09 15:53:17,845][86121] Updated weights for policy 0, policy_version 93410 (0.0008) +[2023-10-09 15:53:18,212][86121] Updated weights for policy 0, policy_version 93420 (0.0009) +[2023-10-09 15:53:18,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 191692800. Throughput: 0: 1826.8, 1: 1836.5. Samples: 47933188. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:18,573][86121] Updated weights for policy 0, policy_version 93430 (0.0010) +[2023-10-09 15:53:18,936][86121] Updated weights for policy 0, policy_version 93440 (0.0012) +[2023-10-09 15:53:20,432][86122] Updated weights for policy 1, policy_version 93800 (0.0007) +[2023-10-09 15:53:20,792][86122] Updated weights for policy 1, policy_version 93810 (0.0008) +[2023-10-09 15:53:21,166][86122] Updated weights for policy 1, policy_version 93820 (0.0009) +[2023-10-09 15:53:22,642][86121] Updated weights for policy 0, policy_version 93450 (0.0010) +[2023-10-09 15:53:23,010][86121] Updated weights for policy 0, policy_version 93460 (0.0010) +[2023-10-09 15:53:23,374][86121] Updated weights for policy 0, policy_version 93470 (0.0008) +[2023-10-09 15:53:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 191758336. Throughput: 0: 1821.7, 1: 1833.4. Samples: 47955016. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:23,404][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000093824_96075776.pth... +[2023-10-09 15:53:23,437][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000092128_94339072.pth +[2023-10-09 15:53:23,443][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000093472_95715328.pth... +[2023-10-09 15:53:23,443][85963] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p1/milestones/checkpoint_000093824_96075776.pth +[2023-10-09 15:53:23,482][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000091744_93945856.pth +[2023-10-09 15:53:23,486][85763] Saving a milestone ./train_atari/atari_bowling_APPO/checkpoint_p0/milestones/checkpoint_000093472_95715328.pth +[2023-10-09 15:53:24,822][86122] Updated weights for policy 1, policy_version 93830 (0.0008) +[2023-10-09 15:53:25,188][86122] Updated weights for policy 1, policy_version 93840 (0.0007) +[2023-10-09 15:53:25,548][86122] Updated weights for policy 1, policy_version 93850 (0.0007) +[2023-10-09 15:53:27,053][86121] Updated weights for policy 0, policy_version 93480 (0.0007) +[2023-10-09 15:53:27,413][86121] Updated weights for policy 0, policy_version 93490 (0.0008) +[2023-10-09 15:53:27,774][86121] Updated weights for policy 0, policy_version 93500 (0.0008) +[2023-10-09 15:53:28,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191856640. Throughput: 0: 1823.8, 1: 1825.3. Samples: 47965828. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:29,307][86122] Updated weights for policy 1, policy_version 93860 (0.0009) +[2023-10-09 15:53:29,667][86122] Updated weights for policy 1, policy_version 93870 (0.0009) +[2023-10-09 15:53:30,020][86122] Updated weights for policy 1, policy_version 93880 (0.0007) +[2023-10-09 15:53:31,458][86121] Updated weights for policy 0, policy_version 93510 (0.0008) +[2023-10-09 15:53:31,821][86121] Updated weights for policy 0, policy_version 93520 (0.0008) +[2023-10-09 15:53:32,183][86121] Updated weights for policy 0, policy_version 93530 (0.0011) +[2023-10-09 15:53:33,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 191922176. Throughput: 0: 1826.5, 1: 1827.7. Samples: 47987834. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 15:53:33,578][86122] Updated weights for policy 1, policy_version 93890 (0.0008) +[2023-10-09 15:53:33,941][86122] Updated weights for policy 1, policy_version 93900 (0.0010) +[2023-10-09 15:53:34,301][86122] Updated weights for policy 1, policy_version 93910 (0.0007) +[2023-10-09 15:53:34,655][86122] Updated weights for policy 1, policy_version 93920 (0.0010) +[2023-10-09 15:53:35,863][86121] Updated weights for policy 0, policy_version 93540 (0.0008) +[2023-10-09 15:53:36,226][86121] Updated weights for policy 0, policy_version 93550 (0.0010) +[2023-10-09 15:53:36,597][86121] Updated weights for policy 0, policy_version 93560 (0.0010) +[2023-10-09 15:53:38,316][86122] Updated weights for policy 1, policy_version 93930 (0.0010) +[2023-10-09 15:53:38,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 191987712. Throughput: 0: 1826.2, 1: 1826.1. Samples: 48010116. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 15:53:38,673][86122] Updated weights for policy 1, policy_version 93940 (0.0011) +[2023-10-09 15:53:39,030][86122] Updated weights for policy 1, policy_version 93950 (0.0011) +[2023-10-09 15:53:40,331][86121] Updated weights for policy 0, policy_version 93570 (0.0007) +[2023-10-09 15:53:40,690][86121] Updated weights for policy 0, policy_version 93580 (0.0009) +[2023-10-09 15:53:41,057][86121] Updated weights for policy 0, policy_version 93590 (0.0008) +[2023-10-09 15:53:41,413][86121] Updated weights for policy 0, policy_version 93600 (0.0008) +[2023-10-09 15:53:42,758][86122] Updated weights for policy 1, policy_version 93960 (0.0009) +[2023-10-09 15:53:43,117][86122] Updated weights for policy 1, policy_version 93970 (0.0008) +[2023-10-09 15:53:43,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 192053248. Throughput: 0: 1822.7, 1: 1824.0. Samples: 48020674. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 15:53:43,476][86122] Updated weights for policy 1, policy_version 93980 (0.0007) +[2023-10-09 15:53:44,938][86121] Updated weights for policy 0, policy_version 93610 (0.0010) +[2023-10-09 15:53:45,303][86121] Updated weights for policy 0, policy_version 93620 (0.0010) +[2023-10-09 15:53:45,663][86121] Updated weights for policy 0, policy_version 93630 (0.0010) +[2023-10-09 15:53:47,162][86122] Updated weights for policy 1, policy_version 93990 (0.0009) +[2023-10-09 15:53:47,527][86122] Updated weights for policy 1, policy_version 94000 (0.0009) +[2023-10-09 15:53:47,885][86122] Updated weights for policy 1, policy_version 94010 (0.0007) +[2023-10-09 15:53:48,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192151552. Throughput: 0: 1826.2, 1: 1821.4. Samples: 48042812. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.970')] +[2023-10-09 15:53:49,518][86121] Updated weights for policy 0, policy_version 93640 (0.0008) +[2023-10-09 15:53:49,896][86121] Updated weights for policy 0, policy_version 93650 (0.0011) +[2023-10-09 15:53:50,272][86121] Updated weights for policy 0, policy_version 93660 (0.0009) +[2023-10-09 15:53:51,472][86122] Updated weights for policy 1, policy_version 94020 (0.0009) +[2023-10-09 15:53:51,830][86122] Updated weights for policy 1, policy_version 94030 (0.0008) +[2023-10-09 15:53:52,193][86122] Updated weights for policy 1, policy_version 94040 (0.0007) +[2023-10-09 15:53:53,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192217088. Throughput: 0: 1825.0, 1: 1820.9. Samples: 48064382. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:53,817][86121] Updated weights for policy 0, policy_version 93670 (0.0008) +[2023-10-09 15:53:54,185][86121] Updated weights for policy 0, policy_version 93680 (0.0009) +[2023-10-09 15:53:54,552][86121] Updated weights for policy 0, policy_version 93690 (0.0009) +[2023-10-09 15:53:55,928][86122] Updated weights for policy 1, policy_version 94050 (0.0008) +[2023-10-09 15:53:56,281][86122] Updated weights for policy 1, policy_version 94060 (0.0009) +[2023-10-09 15:53:56,648][86122] Updated weights for policy 1, policy_version 94070 (0.0009) +[2023-10-09 15:53:57,014][86122] Updated weights for policy 1, policy_version 94080 (0.0009) +[2023-10-09 15:53:58,302][86121] Updated weights for policy 0, policy_version 93700 (0.0007) +[2023-10-09 15:53:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192282624. Throughput: 0: 1825.6, 1: 1817.2. Samples: 48075776. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:53:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:53:58,666][86121] Updated weights for policy 0, policy_version 93710 (0.0007) +[2023-10-09 15:53:59,026][86121] Updated weights for policy 0, policy_version 93720 (0.0010) +[2023-10-09 15:54:00,567][86122] Updated weights for policy 1, policy_version 94090 (0.0010) +[2023-10-09 15:54:00,927][86122] Updated weights for policy 1, policy_version 94100 (0.0009) +[2023-10-09 15:54:01,282][86122] Updated weights for policy 1, policy_version 94110 (0.0010) +[2023-10-09 15:54:02,596][86121] Updated weights for policy 0, policy_version 93730 (0.0008) +[2023-10-09 15:54:02,961][86121] Updated weights for policy 0, policy_version 93740 (0.0010) +[2023-10-09 15:54:03,320][86121] Updated weights for policy 0, policy_version 93750 (0.0011) +[2023-10-09 15:54:03,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192348160. Throughput: 0: 1828.3, 1: 1823.3. Samples: 48097506. Policy #0 lag: (min: 26.0, avg: 26.0, max: 26.0) +[2023-10-09 15:54:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.970')] +[2023-10-09 15:54:03,684][86121] Updated weights for policy 0, policy_version 93760 (0.0010) +[2023-10-09 15:54:05,271][86122] Updated weights for policy 1, policy_version 94120 (0.0010) +[2023-10-09 15:54:05,642][86122] Updated weights for policy 1, policy_version 94130 (0.0008) +[2023-10-09 15:54:06,005][86122] Updated weights for policy 1, policy_version 94140 (0.0008) +[2023-10-09 15:54:07,472][86121] Updated weights for policy 0, policy_version 93770 (0.0009) +[2023-10-09 15:54:07,844][86121] Updated weights for policy 0, policy_version 93780 (0.0008) +[2023-10-09 15:54:08,203][86121] Updated weights for policy 0, policy_version 93790 (0.0008) +[2023-10-09 15:54:08,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192446464. Throughput: 0: 1820.0, 1: 1825.2. Samples: 48119050. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:54:09,705][86122] Updated weights for policy 1, policy_version 94150 (0.0008) +[2023-10-09 15:54:10,074][86122] Updated weights for policy 1, policy_version 94160 (0.0008) +[2023-10-09 15:54:10,431][86122] Updated weights for policy 1, policy_version 94170 (0.0010) +[2023-10-09 15:54:11,991][86121] Updated weights for policy 0, policy_version 93800 (0.0009) +[2023-10-09 15:54:12,363][86121] Updated weights for policy 0, policy_version 93810 (0.0008) +[2023-10-09 15:54:12,729][86121] Updated weights for policy 0, policy_version 93820 (0.0008) +[2023-10-09 15:54:13,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192512000. Throughput: 0: 1822.4, 1: 1822.1. Samples: 48129832. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:54:14,052][86122] Updated weights for policy 1, policy_version 94180 (0.0008) +[2023-10-09 15:54:14,409][86122] Updated weights for policy 1, policy_version 94190 (0.0009) +[2023-10-09 15:54:14,764][86122] Updated weights for policy 1, policy_version 94200 (0.0008) +[2023-10-09 15:54:16,262][86121] Updated weights for policy 0, policy_version 93830 (0.0008) +[2023-10-09 15:54:16,633][86121] Updated weights for policy 0, policy_version 93840 (0.0011) +[2023-10-09 15:54:16,994][86121] Updated weights for policy 0, policy_version 93850 (0.0010) +[2023-10-09 15:54:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 192577536. Throughput: 0: 1818.6, 1: 1827.7. Samples: 48151914. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:18,476][86122] Updated weights for policy 1, policy_version 94210 (0.0010) +[2023-10-09 15:54:18,836][86122] Updated weights for policy 1, policy_version 94220 (0.0007) +[2023-10-09 15:54:19,199][86122] Updated weights for policy 1, policy_version 94230 (0.0008) +[2023-10-09 15:54:19,561][86122] Updated weights for policy 1, policy_version 94240 (0.0008) +[2023-10-09 15:54:20,751][86121] Updated weights for policy 0, policy_version 93860 (0.0011) +[2023-10-09 15:54:21,121][86121] Updated weights for policy 0, policy_version 93870 (0.0009) +[2023-10-09 15:54:21,483][86121] Updated weights for policy 0, policy_version 93880 (0.0009) +[2023-10-09 15:54:23,315][86122] Updated weights for policy 1, policy_version 94250 (0.0008) +[2023-10-09 15:54:23,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192643072. Throughput: 0: 1820.7, 1: 1827.6. Samples: 48174292. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:23,673][86122] Updated weights for policy 1, policy_version 94260 (0.0009) +[2023-10-09 15:54:24,038][86122] Updated weights for policy 1, policy_version 94270 (0.0009) +[2023-10-09 15:54:25,272][86121] Updated weights for policy 0, policy_version 93890 (0.0008) +[2023-10-09 15:54:25,631][86121] Updated weights for policy 0, policy_version 93900 (0.0010) +[2023-10-09 15:54:25,998][86121] Updated weights for policy 0, policy_version 93910 (0.0011) +[2023-10-09 15:54:26,369][86121] Updated weights for policy 0, policy_version 93920 (0.0011) +[2023-10-09 15:54:27,755][86122] Updated weights for policy 1, policy_version 94280 (0.0009) +[2023-10-09 15:54:28,116][86122] Updated weights for policy 1, policy_version 94290 (0.0009) +[2023-10-09 15:54:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 192708608. Throughput: 0: 1821.7, 1: 1827.9. Samples: 48184908. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:28,477][86122] Updated weights for policy 1, policy_version 94300 (0.0007) +[2023-10-09 15:54:30,129][86121] Updated weights for policy 0, policy_version 93930 (0.0010) +[2023-10-09 15:54:30,493][86121] Updated weights for policy 0, policy_version 93940 (0.0009) +[2023-10-09 15:54:30,852][86121] Updated weights for policy 0, policy_version 93950 (0.0010) +[2023-10-09 15:54:32,117][86122] Updated weights for policy 1, policy_version 94310 (0.0010) +[2023-10-09 15:54:32,471][86122] Updated weights for policy 1, policy_version 94320 (0.0009) +[2023-10-09 15:54:32,840][86122] Updated weights for policy 1, policy_version 94330 (0.0008) +[2023-10-09 15:54:33,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.7, 300 sec: 14662.3). Total num frames: 192806912. Throughput: 0: 1818.0, 1: 1833.4. Samples: 48207126. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:34,576][86121] Updated weights for policy 0, policy_version 93960 (0.0008) +[2023-10-09 15:54:34,955][86121] Updated weights for policy 0, policy_version 93970 (0.0008) +[2023-10-09 15:54:35,315][86121] Updated weights for policy 0, policy_version 93980 (0.0007) +[2023-10-09 15:54:36,457][86122] Updated weights for policy 1, policy_version 94340 (0.0008) +[2023-10-09 15:54:36,827][86122] Updated weights for policy 1, policy_version 94350 (0.0008) +[2023-10-09 15:54:37,188][86122] Updated weights for policy 1, policy_version 94360 (0.0007) +[2023-10-09 15:54:38,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192872448. Throughput: 0: 1817.3, 1: 1835.3. Samples: 48228752. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:39,082][86121] Updated weights for policy 0, policy_version 93990 (0.0010) +[2023-10-09 15:54:39,442][86121] Updated weights for policy 0, policy_version 94000 (0.0009) +[2023-10-09 15:54:39,805][86121] Updated weights for policy 0, policy_version 94010 (0.0009) +[2023-10-09 15:54:40,750][86122] Updated weights for policy 1, policy_version 94370 (0.0008) +[2023-10-09 15:54:41,108][86122] Updated weights for policy 1, policy_version 94380 (0.0009) +[2023-10-09 15:54:41,473][86122] Updated weights for policy 1, policy_version 94390 (0.0011) +[2023-10-09 15:54:41,832][86122] Updated weights for policy 1, policy_version 94400 (0.0011) +[2023-10-09 15:54:43,368][86121] Updated weights for policy 0, policy_version 94020 (0.0011) +[2023-10-09 15:54:43,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 192937984. Throughput: 0: 1815.1, 1: 1834.0. Samples: 48239984. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:43,735][86121] Updated weights for policy 0, policy_version 94030 (0.0007) +[2023-10-09 15:54:44,098][86121] Updated weights for policy 0, policy_version 94040 (0.0008) +[2023-10-09 15:54:45,495][86122] Updated weights for policy 1, policy_version 94410 (0.0010) +[2023-10-09 15:54:45,859][86122] Updated weights for policy 1, policy_version 94420 (0.0009) +[2023-10-09 15:54:46,218][86122] Updated weights for policy 1, policy_version 94430 (0.0008) +[2023-10-09 15:54:47,712][86121] Updated weights for policy 0, policy_version 94050 (0.0007) +[2023-10-09 15:54:48,066][86121] Updated weights for policy 0, policy_version 94060 (0.0009) +[2023-10-09 15:54:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 193003520. Throughput: 0: 1816.1, 1: 1834.9. Samples: 48261802. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:48,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:48,437][86121] Updated weights for policy 0, policy_version 94070 (0.0007) +[2023-10-09 15:54:48,808][86121] Updated weights for policy 0, policy_version 94080 (0.0008) +[2023-10-09 15:54:49,869][86122] Updated weights for policy 1, policy_version 94440 (0.0009) +[2023-10-09 15:54:50,239][86122] Updated weights for policy 1, policy_version 94450 (0.0010) +[2023-10-09 15:54:50,607][86122] Updated weights for policy 1, policy_version 94460 (0.0010) +[2023-10-09 15:54:52,430][86121] Updated weights for policy 0, policy_version 94090 (0.0008) +[2023-10-09 15:54:52,800][86121] Updated weights for policy 0, policy_version 94100 (0.0007) +[2023-10-09 15:54:53,158][86121] Updated weights for policy 0, policy_version 94110 (0.0008) +[2023-10-09 15:54:53,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193101824. Throughput: 0: 1819.4, 1: 1841.7. Samples: 48283800. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:53,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:54,309][86122] Updated weights for policy 1, policy_version 94470 (0.0009) +[2023-10-09 15:54:54,668][86122] Updated weights for policy 1, policy_version 94480 (0.0008) +[2023-10-09 15:54:55,033][86122] Updated weights for policy 1, policy_version 94490 (0.0009) +[2023-10-09 15:54:56,774][86121] Updated weights for policy 0, policy_version 94120 (0.0007) +[2023-10-09 15:54:57,140][86121] Updated weights for policy 0, policy_version 94130 (0.0011) +[2023-10-09 15:54:57,508][86121] Updated weights for policy 0, policy_version 94140 (0.0011) +[2023-10-09 15:54:58,397][85186] Fps is (10 sec: 16384.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193167360. Throughput: 0: 1823.7, 1: 1840.8. Samples: 48294734. Policy #0 lag: (min: 31.0, avg: 31.2, max: 41.0) +[2023-10-09 15:54:58,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:54:58,808][86122] Updated weights for policy 1, policy_version 94500 (0.0009) +[2023-10-09 15:54:59,161][86122] Updated weights for policy 1, policy_version 94510 (0.0008) +[2023-10-09 15:54:59,521][86122] Updated weights for policy 1, policy_version 94520 (0.0007) +[2023-10-09 15:55:01,235][86121] Updated weights for policy 0, policy_version 94150 (0.0008) +[2023-10-09 15:55:01,591][86121] Updated weights for policy 0, policy_version 94160 (0.0007) +[2023-10-09 15:55:01,952][86121] Updated weights for policy 0, policy_version 94170 (0.0010) +[2023-10-09 15:55:03,259][86122] Updated weights for policy 1, policy_version 94530 (0.0008) +[2023-10-09 15:55:03,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 193232896. Throughput: 0: 1816.8, 1: 1834.3. Samples: 48316216. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:03,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:55:03,628][86122] Updated weights for policy 1, policy_version 94540 (0.0008) +[2023-10-09 15:55:03,980][86122] Updated weights for policy 1, policy_version 94550 (0.0008) +[2023-10-09 15:55:04,340][86122] Updated weights for policy 1, policy_version 94560 (0.0011) +[2023-10-09 15:55:05,763][86121] Updated weights for policy 0, policy_version 94180 (0.0010) +[2023-10-09 15:55:06,142][86121] Updated weights for policy 0, policy_version 94190 (0.0009) +[2023-10-09 15:55:06,507][86121] Updated weights for policy 0, policy_version 94200 (0.0009) +[2023-10-09 15:55:07,942][86122] Updated weights for policy 1, policy_version 94570 (0.0007) +[2023-10-09 15:55:08,307][86122] Updated weights for policy 1, policy_version 94580 (0.0007) +[2023-10-09 15:55:08,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 193298432. Throughput: 0: 1824.0, 1: 1826.2. Samples: 48338548. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:55:08,660][86122] Updated weights for policy 1, policy_version 94590 (0.0008) +[2023-10-09 15:55:10,262][86121] Updated weights for policy 0, policy_version 94210 (0.0009) +[2023-10-09 15:55:10,638][86121] Updated weights for policy 0, policy_version 94220 (0.0008) +[2023-10-09 15:55:11,003][86121] Updated weights for policy 0, policy_version 94230 (0.0007) +[2023-10-09 15:55:11,373][86121] Updated weights for policy 0, policy_version 94240 (0.0008) +[2023-10-09 15:55:12,408][86122] Updated weights for policy 1, policy_version 94600 (0.0008) +[2023-10-09 15:55:12,764][86122] Updated weights for policy 1, policy_version 94610 (0.0008) +[2023-10-09 15:55:13,126][86122] Updated weights for policy 1, policy_version 94620 (0.0010) +[2023-10-09 15:55:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193396736. Throughput: 0: 1820.9, 1: 1832.2. Samples: 48349298. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:13,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:55:15,123][86121] Updated weights for policy 0, policy_version 94250 (0.0009) +[2023-10-09 15:55:15,480][86121] Updated weights for policy 0, policy_version 94260 (0.0008) +[2023-10-09 15:55:15,842][86121] Updated weights for policy 0, policy_version 94270 (0.0009) +[2023-10-09 15:55:16,882][86122] Updated weights for policy 1, policy_version 94630 (0.0008) +[2023-10-09 15:55:17,244][86122] Updated weights for policy 1, policy_version 94640 (0.0009) +[2023-10-09 15:55:17,609][86122] Updated weights for policy 1, policy_version 94650 (0.0007) +[2023-10-09 15:55:18,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193462272. Throughput: 0: 1828.7, 1: 1820.7. Samples: 48371348. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:18,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:55:19,634][86121] Updated weights for policy 0, policy_version 94280 (0.0008) +[2023-10-09 15:55:20,014][86121] Updated weights for policy 0, policy_version 94290 (0.0009) +[2023-10-09 15:55:20,378][86121] Updated weights for policy 0, policy_version 94300 (0.0008) +[2023-10-09 15:55:21,137][86122] Updated weights for policy 1, policy_version 94660 (0.0009) +[2023-10-09 15:55:21,499][86122] Updated weights for policy 1, policy_version 94670 (0.0008) +[2023-10-09 15:55:21,861][86122] Updated weights for policy 1, policy_version 94680 (0.0007) +[2023-10-09 15:55:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 193527808. Throughput: 0: 1820.6, 1: 1825.8. Samples: 48392838. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:23,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:55:23,410][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000094304_96567296.pth... +[2023-10-09 15:55:23,410][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000094688_96960512.pth... +[2023-10-09 15:55:23,447][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000092608_94830592.pth +[2023-10-09 15:55:23,447][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000092960_95191040.pth +[2023-10-09 15:55:24,010][86121] Updated weights for policy 0, policy_version 94310 (0.0010) +[2023-10-09 15:55:24,378][86121] Updated weights for policy 0, policy_version 94320 (0.0007) +[2023-10-09 15:55:24,747][86121] Updated weights for policy 0, policy_version 94330 (0.0007) +[2023-10-09 15:55:25,477][86122] Updated weights for policy 1, policy_version 94690 (0.0008) +[2023-10-09 15:55:25,836][86122] Updated weights for policy 1, policy_version 94700 (0.0009) +[2023-10-09 15:55:26,197][86122] Updated weights for policy 1, policy_version 94710 (0.0007) +[2023-10-09 15:55:26,561][86122] Updated weights for policy 1, policy_version 94720 (0.0008) +[2023-10-09 15:55:28,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 193593344. Throughput: 0: 1824.7, 1: 1818.5. Samples: 48403930. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:55:28,505][86121] Updated weights for policy 0, policy_version 94340 (0.0009) +[2023-10-09 15:55:28,880][86121] Updated weights for policy 0, policy_version 94350 (0.0008) +[2023-10-09 15:55:29,245][86121] Updated weights for policy 0, policy_version 94360 (0.0007) +[2023-10-09 15:55:30,233][86122] Updated weights for policy 1, policy_version 94730 (0.0007) +[2023-10-09 15:55:30,589][86122] Updated weights for policy 1, policy_version 94740 (0.0007) +[2023-10-09 15:55:30,958][86122] Updated weights for policy 1, policy_version 94750 (0.0008) +[2023-10-09 15:55:32,980][86121] Updated weights for policy 0, policy_version 94370 (0.0008) +[2023-10-09 15:55:33,346][86121] Updated weights for policy 0, policy_version 94380 (0.0009) +[2023-10-09 15:55:33,397][85186] Fps is (10 sec: 13107.6, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193658880. Throughput: 0: 1814.1, 1: 1824.5. Samples: 48425540. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:55:33,712][86121] Updated weights for policy 0, policy_version 94390 (0.0010) +[2023-10-09 15:55:34,081][86121] Updated weights for policy 0, policy_version 94400 (0.0011) +[2023-10-09 15:55:34,793][86122] Updated weights for policy 1, policy_version 94760 (0.0011) +[2023-10-09 15:55:35,166][86122] Updated weights for policy 1, policy_version 94770 (0.0007) +[2023-10-09 15:55:35,519][86122] Updated weights for policy 1, policy_version 94780 (0.0009) +[2023-10-09 15:55:37,691][86121] Updated weights for policy 0, policy_version 94410 (0.0008) +[2023-10-09 15:55:38,054][86121] Updated weights for policy 0, policy_version 94420 (0.0010) +[2023-10-09 15:55:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 193724416. Throughput: 0: 1823.9, 1: 1821.5. Samples: 48447848. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:38,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:55:38,420][86121] Updated weights for policy 0, policy_version 94430 (0.0007) +[2023-10-09 15:55:39,189][86122] Updated weights for policy 1, policy_version 94790 (0.0008) +[2023-10-09 15:55:39,559][86122] Updated weights for policy 1, policy_version 94800 (0.0008) +[2023-10-09 15:55:39,909][86122] Updated weights for policy 1, policy_version 94810 (0.0008) +[2023-10-09 15:55:41,881][86121] Updated weights for policy 0, policy_version 94440 (0.0009) +[2023-10-09 15:55:42,252][86121] Updated weights for policy 0, policy_version 94450 (0.0008) +[2023-10-09 15:55:42,609][86121] Updated weights for policy 0, policy_version 94460 (0.0008) +[2023-10-09 15:55:43,397][85186] Fps is (10 sec: 16383.7, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 193822720. Throughput: 0: 1812.4, 1: 1824.4. Samples: 48458392. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:43,398][85186] Avg episode reward: [(0, '9.990'), (1, '9.990')] +[2023-10-09 15:55:43,665][86122] Updated weights for policy 1, policy_version 94820 (0.0008) +[2023-10-09 15:55:44,031][86122] Updated weights for policy 1, policy_version 94830 (0.0009) +[2023-10-09 15:55:44,390][86122] Updated weights for policy 1, policy_version 94840 (0.0008) +[2023-10-09 15:55:46,274][86121] Updated weights for policy 0, policy_version 94470 (0.0007) +[2023-10-09 15:55:46,642][86121] Updated weights for policy 0, policy_version 94480 (0.0007) +[2023-10-09 15:55:47,011][86121] Updated weights for policy 0, policy_version 94490 (0.0008) +[2023-10-09 15:55:48,161][86122] Updated weights for policy 1, policy_version 94850 (0.0008) +[2023-10-09 15:55:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 193888256. Throughput: 0: 1819.4, 1: 1822.2. Samples: 48480088. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:55:48,524][86122] Updated weights for policy 1, policy_version 94860 (0.0009) +[2023-10-09 15:55:48,889][86122] Updated weights for policy 1, policy_version 94870 (0.0008) +[2023-10-09 15:55:49,255][86122] Updated weights for policy 1, policy_version 94880 (0.0008) +[2023-10-09 15:55:50,717][86121] Updated weights for policy 0, policy_version 94500 (0.0009) +[2023-10-09 15:55:51,093][86121] Updated weights for policy 0, policy_version 94510 (0.0010) +[2023-10-09 15:55:51,455][86121] Updated weights for policy 0, policy_version 94520 (0.0008) +[2023-10-09 15:55:52,979][86122] Updated weights for policy 1, policy_version 94890 (0.0008) +[2023-10-09 15:55:53,342][86122] Updated weights for policy 1, policy_version 94900 (0.0009) +[2023-10-09 15:55:53,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 193953792. Throughput: 0: 1817.9, 1: 1820.8. Samples: 48502292. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:55:53,716][86122] Updated weights for policy 1, policy_version 94910 (0.0010) +[2023-10-09 15:55:55,152][86121] Updated weights for policy 0, policy_version 94530 (0.0007) +[2023-10-09 15:55:55,514][86121] Updated weights for policy 0, policy_version 94540 (0.0010) +[2023-10-09 15:55:55,884][86121] Updated weights for policy 0, policy_version 94550 (0.0010) +[2023-10-09 15:55:56,253][86121] Updated weights for policy 0, policy_version 94560 (0.0008) +[2023-10-09 15:55:57,198][86122] Updated weights for policy 1, policy_version 94920 (0.0009) +[2023-10-09 15:55:57,565][86122] Updated weights for policy 1, policy_version 94930 (0.0009) +[2023-10-09 15:55:57,928][86122] Updated weights for policy 1, policy_version 94940 (0.0008) +[2023-10-09 15:55:58,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194052096. Throughput: 0: 1817.3, 1: 1822.7. Samples: 48513096. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:55:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:55:59,944][86121] Updated weights for policy 0, policy_version 94570 (0.0011) +[2023-10-09 15:56:00,301][86121] Updated weights for policy 0, policy_version 94580 (0.0007) +[2023-10-09 15:56:00,677][86121] Updated weights for policy 0, policy_version 94590 (0.0009) +[2023-10-09 15:56:01,621][86122] Updated weights for policy 1, policy_version 94950 (0.0008) +[2023-10-09 15:56:01,979][86122] Updated weights for policy 1, policy_version 94960 (0.0008) +[2023-10-09 15:56:02,343][86122] Updated weights for policy 1, policy_version 94970 (0.0007) +[2023-10-09 15:56:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194117632. Throughput: 0: 1821.1, 1: 1820.3. Samples: 48535210. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:56:04,500][86121] Updated weights for policy 0, policy_version 94600 (0.0009) +[2023-10-09 15:56:04,870][86121] Updated weights for policy 0, policy_version 94610 (0.0009) +[2023-10-09 15:56:05,237][86121] Updated weights for policy 0, policy_version 94620 (0.0009) +[2023-10-09 15:56:05,976][86122] Updated weights for policy 1, policy_version 94980 (0.0008) +[2023-10-09 15:56:06,346][86122] Updated weights for policy 1, policy_version 94990 (0.0007) +[2023-10-09 15:56:06,703][86122] Updated weights for policy 1, policy_version 95000 (0.0008) +[2023-10-09 15:56:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194183168. Throughput: 0: 1823.6, 1: 1828.4. Samples: 48557178. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:56:08,822][86121] Updated weights for policy 0, policy_version 94630 (0.0009) +[2023-10-09 15:56:09,183][86121] Updated weights for policy 0, policy_version 94640 (0.0010) +[2023-10-09 15:56:09,562][86121] Updated weights for policy 0, policy_version 94650 (0.0008) +[2023-10-09 15:56:10,244][86122] Updated weights for policy 1, policy_version 95010 (0.0009) +[2023-10-09 15:56:10,597][86122] Updated weights for policy 1, policy_version 95020 (0.0008) +[2023-10-09 15:56:10,955][86122] Updated weights for policy 1, policy_version 95030 (0.0008) +[2023-10-09 15:56:11,317][86122] Updated weights for policy 1, policy_version 95040 (0.0008) +[2023-10-09 15:56:13,211][86121] Updated weights for policy 0, policy_version 94660 (0.0009) +[2023-10-09 15:56:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194248704. Throughput: 0: 1823.2, 1: 1822.4. Samples: 48567982. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:56:13,583][86121] Updated weights for policy 0, policy_version 94670 (0.0008) +[2023-10-09 15:56:13,945][86121] Updated weights for policy 0, policy_version 94680 (0.0007) +[2023-10-09 15:56:14,829][86122] Updated weights for policy 1, policy_version 95050 (0.0008) +[2023-10-09 15:56:15,190][86122] Updated weights for policy 1, policy_version 95060 (0.0010) +[2023-10-09 15:56:15,547][86122] Updated weights for policy 1, policy_version 95070 (0.0011) +[2023-10-09 15:56:17,558][86121] Updated weights for policy 0, policy_version 94690 (0.0007) +[2023-10-09 15:56:17,921][86121] Updated weights for policy 0, policy_version 94700 (0.0011) +[2023-10-09 15:56:18,283][86121] Updated weights for policy 0, policy_version 94710 (0.0009) +[2023-10-09 15:56:18,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 194314240. Throughput: 0: 1830.8, 1: 1836.6. Samples: 48590574. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:56:18,650][86121] Updated weights for policy 0, policy_version 94720 (0.0009) +[2023-10-09 15:56:19,335][86122] Updated weights for policy 1, policy_version 95080 (0.0010) +[2023-10-09 15:56:19,695][86122] Updated weights for policy 1, policy_version 95090 (0.0008) +[2023-10-09 15:56:20,050][86122] Updated weights for policy 1, policy_version 95100 (0.0007) +[2023-10-09 15:56:22,285][86121] Updated weights for policy 0, policy_version 94730 (0.0007) +[2023-10-09 15:56:22,665][86121] Updated weights for policy 0, policy_version 94740 (0.0009) +[2023-10-09 15:56:23,028][86121] Updated weights for policy 0, policy_version 94750 (0.0007) +[2023-10-09 15:56:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194412544. Throughput: 0: 1815.9, 1: 1838.6. Samples: 48612298. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:56:23,795][86122] Updated weights for policy 1, policy_version 95110 (0.0011) +[2023-10-09 15:56:24,161][86122] Updated weights for policy 1, policy_version 95120 (0.0008) +[2023-10-09 15:56:24,526][86122] Updated weights for policy 1, policy_version 95130 (0.0008) +[2023-10-09 15:56:26,721][86121] Updated weights for policy 0, policy_version 94760 (0.0009) +[2023-10-09 15:56:27,080][86121] Updated weights for policy 0, policy_version 94770 (0.0010) +[2023-10-09 15:56:27,445][86121] Updated weights for policy 0, policy_version 94780 (0.0008) +[2023-10-09 15:56:28,350][86122] Updated weights for policy 1, policy_version 95140 (0.0008) +[2023-10-09 15:56:28,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194478080. Throughput: 0: 1829.4, 1: 1841.1. Samples: 48623562. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:56:28,737][86122] Updated weights for policy 1, policy_version 95150 (0.0009) +[2023-10-09 15:56:29,097][86122] Updated weights for policy 1, policy_version 95160 (0.0010) +[2023-10-09 15:56:31,196][86121] Updated weights for policy 0, policy_version 94790 (0.0008) +[2023-10-09 15:56:31,561][86121] Updated weights for policy 0, policy_version 94800 (0.0007) +[2023-10-09 15:56:31,926][86121] Updated weights for policy 0, policy_version 94810 (0.0009) +[2023-10-09 15:56:32,929][86122] Updated weights for policy 1, policy_version 95170 (0.0011) +[2023-10-09 15:56:33,281][86122] Updated weights for policy 1, policy_version 95180 (0.0008) +[2023-10-09 15:56:33,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194543616. Throughput: 0: 1821.5, 1: 1837.2. Samples: 48644732. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:56:33,644][86122] Updated weights for policy 1, policy_version 95190 (0.0007) +[2023-10-09 15:56:34,011][86122] Updated weights for policy 1, policy_version 95200 (0.0007) +[2023-10-09 15:56:35,652][86121] Updated weights for policy 0, policy_version 94820 (0.0009) +[2023-10-09 15:56:36,025][86121] Updated weights for policy 0, policy_version 94830 (0.0008) +[2023-10-09 15:56:36,388][86121] Updated weights for policy 0, policy_version 94840 (0.0008) +[2023-10-09 15:56:37,772][86122] Updated weights for policy 1, policy_version 95210 (0.0009) +[2023-10-09 15:56:38,131][86122] Updated weights for policy 1, policy_version 95220 (0.0008) +[2023-10-09 15:56:38,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194609152. Throughput: 0: 1822.8, 1: 1833.1. Samples: 48666810. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:56:38,501][86122] Updated weights for policy 1, policy_version 95230 (0.0007) +[2023-10-09 15:56:39,960][86121] Updated weights for policy 0, policy_version 94850 (0.0007) +[2023-10-09 15:56:40,332][86121] Updated weights for policy 0, policy_version 94860 (0.0009) +[2023-10-09 15:56:40,701][86121] Updated weights for policy 0, policy_version 94870 (0.0008) +[2023-10-09 15:56:41,068][86121] Updated weights for policy 0, policy_version 94880 (0.0009) +[2023-10-09 15:56:42,283][86122] Updated weights for policy 1, policy_version 95240 (0.0008) +[2023-10-09 15:56:42,641][86122] Updated weights for policy 1, policy_version 95250 (0.0007) +[2023-10-09 15:56:43,015][86122] Updated weights for policy 1, policy_version 95260 (0.0009) +[2023-10-09 15:56:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 194707456. Throughput: 0: 1820.2, 1: 1834.2. Samples: 48677544. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:56:44,872][86121] Updated weights for policy 0, policy_version 94890 (0.0009) +[2023-10-09 15:56:45,242][86121] Updated weights for policy 0, policy_version 94900 (0.0009) +[2023-10-09 15:56:45,606][86121] Updated weights for policy 0, policy_version 94910 (0.0009) +[2023-10-09 15:56:46,533][86122] Updated weights for policy 1, policy_version 95270 (0.0008) +[2023-10-09 15:56:46,887][86122] Updated weights for policy 1, policy_version 95280 (0.0008) +[2023-10-09 15:56:47,256][86122] Updated weights for policy 1, policy_version 95290 (0.0011) +[2023-10-09 15:56:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 194772992. Throughput: 0: 1830.0, 1: 1835.4. Samples: 48700150. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:56:49,236][86121] Updated weights for policy 0, policy_version 94920 (0.0008) +[2023-10-09 15:56:49,607][86121] Updated weights for policy 0, policy_version 94930 (0.0007) +[2023-10-09 15:56:49,973][86121] Updated weights for policy 0, policy_version 94940 (0.0007) +[2023-10-09 15:56:50,995][86122] Updated weights for policy 1, policy_version 95300 (0.0009) +[2023-10-09 15:56:51,353][86122] Updated weights for policy 1, policy_version 95310 (0.0009) +[2023-10-09 15:56:51,720][86122] Updated weights for policy 1, policy_version 95320 (0.0008) +[2023-10-09 15:56:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.5, 300 sec: 14551.2). Total num frames: 194838528. Throughput: 0: 1831.7, 1: 1830.2. Samples: 48721964. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:53,399][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:56:53,641][86121] Updated weights for policy 0, policy_version 94950 (0.0009) +[2023-10-09 15:56:54,001][86121] Updated weights for policy 0, policy_version 94960 (0.0007) +[2023-10-09 15:56:54,373][86121] Updated weights for policy 0, policy_version 94970 (0.0007) +[2023-10-09 15:56:55,163][86122] Updated weights for policy 1, policy_version 95330 (0.0008) +[2023-10-09 15:56:55,530][86122] Updated weights for policy 1, policy_version 95340 (0.0010) +[2023-10-09 15:56:55,890][86122] Updated weights for policy 1, policy_version 95350 (0.0007) +[2023-10-09 15:56:56,252][86122] Updated weights for policy 1, policy_version 95360 (0.0008) +[2023-10-09 15:56:57,863][86121] Updated weights for policy 0, policy_version 94980 (0.0008) +[2023-10-09 15:56:58,235][86121] Updated weights for policy 0, policy_version 94990 (0.0008) +[2023-10-09 15:56:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 194904064. Throughput: 0: 1835.6, 1: 1832.5. Samples: 48733044. Policy #0 lag: (min: 20.0, avg: 25.2, max: 52.0) +[2023-10-09 15:56:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:56:58,598][86121] Updated weights for policy 0, policy_version 95000 (0.0007) +[2023-10-09 15:56:59,902][86122] Updated weights for policy 1, policy_version 95370 (0.0008) +[2023-10-09 15:57:00,264][86122] Updated weights for policy 1, policy_version 95380 (0.0007) +[2023-10-09 15:57:00,614][86122] Updated weights for policy 1, policy_version 95390 (0.0008) +[2023-10-09 15:57:02,386][86121] Updated weights for policy 0, policy_version 95010 (0.0008) +[2023-10-09 15:57:02,745][86121] Updated weights for policy 0, policy_version 95020 (0.0008) +[2023-10-09 15:57:03,119][86121] Updated weights for policy 0, policy_version 95030 (0.0010) +[2023-10-09 15:57:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 194969600. Throughput: 0: 1838.4, 1: 1825.3. Samples: 48755440. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:57:03,482][86121] Updated weights for policy 0, policy_version 95040 (0.0008) +[2023-10-09 15:57:04,358][86122] Updated weights for policy 1, policy_version 95400 (0.0008) +[2023-10-09 15:57:04,712][86122] Updated weights for policy 1, policy_version 95410 (0.0007) +[2023-10-09 15:57:05,076][86122] Updated weights for policy 1, policy_version 95420 (0.0007) +[2023-10-09 15:57:07,111][86121] Updated weights for policy 0, policy_version 95050 (0.0008) +[2023-10-09 15:57:07,485][86121] Updated weights for policy 0, policy_version 95060 (0.0008) +[2023-10-09 15:57:07,858][86121] Updated weights for policy 0, policy_version 95070 (0.0009) +[2023-10-09 15:57:08,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195067904. Throughput: 0: 1832.3, 1: 1832.0. Samples: 48777190. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:08,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:57:08,569][86122] Updated weights for policy 1, policy_version 95430 (0.0008) +[2023-10-09 15:57:08,925][86122] Updated weights for policy 1, policy_version 95440 (0.0009) +[2023-10-09 15:57:09,283][86122] Updated weights for policy 1, policy_version 95450 (0.0010) +[2023-10-09 15:57:11,599][86121] Updated weights for policy 0, policy_version 95080 (0.0008) +[2023-10-09 15:57:11,964][86121] Updated weights for policy 0, policy_version 95090 (0.0007) +[2023-10-09 15:57:12,327][86121] Updated weights for policy 0, policy_version 95100 (0.0009) +[2023-10-09 15:57:12,970][86122] Updated weights for policy 1, policy_version 95460 (0.0009) +[2023-10-09 15:57:13,363][86122] Updated weights for policy 1, policy_version 95470 (0.0007) +[2023-10-09 15:57:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195133440. Throughput: 0: 1837.2, 1: 1832.5. Samples: 48788700. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:57:13,722][86122] Updated weights for policy 1, policy_version 95480 (0.0009) +[2023-10-09 15:57:16,117][86121] Updated weights for policy 0, policy_version 95110 (0.0009) +[2023-10-09 15:57:16,482][86121] Updated weights for policy 0, policy_version 95120 (0.0008) +[2023-10-09 15:57:16,860][86121] Updated weights for policy 0, policy_version 95130 (0.0010) +[2023-10-09 15:57:17,197][86122] Updated weights for policy 1, policy_version 95490 (0.0011) +[2023-10-09 15:57:17,560][86122] Updated weights for policy 1, policy_version 95500 (0.0008) +[2023-10-09 15:57:17,918][86122] Updated weights for policy 1, policy_version 95510 (0.0010) +[2023-10-09 15:57:18,278][86122] Updated weights for policy 1, policy_version 95520 (0.0009) +[2023-10-09 15:57:18,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195231744. Throughput: 0: 1835.7, 1: 1847.8. Samples: 48810492. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:18,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:57:20,628][86121] Updated weights for policy 0, policy_version 95140 (0.0010) +[2023-10-09 15:57:20,988][86121] Updated weights for policy 0, policy_version 95150 (0.0008) +[2023-10-09 15:57:21,361][86121] Updated weights for policy 0, policy_version 95160 (0.0010) +[2023-10-09 15:57:21,830][86122] Updated weights for policy 1, policy_version 95530 (0.0010) +[2023-10-09 15:57:22,195][86122] Updated weights for policy 1, policy_version 95540 (0.0007) +[2023-10-09 15:57:22,552][86122] Updated weights for policy 1, policy_version 95550 (0.0007) +[2023-10-09 15:57:23,397][85186] Fps is (10 sec: 16383.8, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 195297280. Throughput: 0: 1832.7, 1: 1832.5. Samples: 48831744. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:23,399][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:57:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000095552_97845248.pth... +[2023-10-09 15:57:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000095168_97452032.pth... +[2023-10-09 15:57:23,449][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000093472_95715328.pth +[2023-10-09 15:57:23,449][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000093824_96075776.pth +[2023-10-09 15:57:24,993][86121] Updated weights for policy 0, policy_version 95170 (0.0008) +[2023-10-09 15:57:25,358][86121] Updated weights for policy 0, policy_version 95180 (0.0009) +[2023-10-09 15:57:25,725][86121] Updated weights for policy 0, policy_version 95190 (0.0009) +[2023-10-09 15:57:26,100][86121] Updated weights for policy 0, policy_version 95200 (0.0008) +[2023-10-09 15:57:26,248][86122] Updated weights for policy 1, policy_version 95560 (0.0007) +[2023-10-09 15:57:26,609][86122] Updated weights for policy 1, policy_version 95570 (0.0008) +[2023-10-09 15:57:26,960][86122] Updated weights for policy 1, policy_version 95580 (0.0007) +[2023-10-09 15:57:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195362816. Throughput: 0: 1833.7, 1: 1854.9. Samples: 48843532. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:28,398][85186] Avg episode reward: [(0, '9.960'), (1, '9.980')] +[2023-10-09 15:57:29,558][86121] Updated weights for policy 0, policy_version 95210 (0.0007) +[2023-10-09 15:57:29,920][86121] Updated weights for policy 0, policy_version 95220 (0.0009) +[2023-10-09 15:57:30,281][86121] Updated weights for policy 0, policy_version 95230 (0.0009) +[2023-10-09 15:57:30,646][86122] Updated weights for policy 1, policy_version 95590 (0.0009) +[2023-10-09 15:57:31,012][86122] Updated weights for policy 1, policy_version 95600 (0.0009) +[2023-10-09 15:57:31,376][86122] Updated weights for policy 1, policy_version 95610 (0.0009) +[2023-10-09 15:57:33,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195428352. Throughput: 0: 1826.8, 1: 1829.3. Samples: 48864672. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:33,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:57:33,983][86121] Updated weights for policy 0, policy_version 95240 (0.0009) +[2023-10-09 15:57:34,354][86121] Updated weights for policy 0, policy_version 95250 (0.0007) +[2023-10-09 15:57:34,719][86121] Updated weights for policy 0, policy_version 95260 (0.0007) +[2023-10-09 15:57:35,132][86122] Updated weights for policy 1, policy_version 95620 (0.0010) +[2023-10-09 15:57:35,490][86122] Updated weights for policy 1, policy_version 95630 (0.0011) +[2023-10-09 15:57:35,841][86122] Updated weights for policy 1, policy_version 95640 (0.0010) +[2023-10-09 15:57:38,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195493888. Throughput: 0: 1831.6, 1: 1851.6. Samples: 48887710. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:57:38,467][86121] Updated weights for policy 0, policy_version 95270 (0.0008) +[2023-10-09 15:57:38,840][86121] Updated weights for policy 0, policy_version 95280 (0.0009) +[2023-10-09 15:57:39,204][86121] Updated weights for policy 0, policy_version 95290 (0.0009) +[2023-10-09 15:57:39,639][86122] Updated weights for policy 1, policy_version 95650 (0.0010) +[2023-10-09 15:57:40,000][86122] Updated weights for policy 1, policy_version 95660 (0.0009) +[2023-10-09 15:57:40,364][86122] Updated weights for policy 1, policy_version 95670 (0.0011) +[2023-10-09 15:57:40,722][86122] Updated weights for policy 1, policy_version 95680 (0.0009) +[2023-10-09 15:57:42,864][86121] Updated weights for policy 0, policy_version 95300 (0.0011) +[2023-10-09 15:57:43,237][86121] Updated weights for policy 0, policy_version 95310 (0.0009) +[2023-10-09 15:57:43,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 195559424. Throughput: 0: 1828.2, 1: 1827.0. Samples: 48897526. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:57:43,601][86121] Updated weights for policy 0, policy_version 95320 (0.0009) +[2023-10-09 15:57:44,372][86122] Updated weights for policy 1, policy_version 95690 (0.0009) +[2023-10-09 15:57:44,734][86122] Updated weights for policy 1, policy_version 95700 (0.0008) +[2023-10-09 15:57:45,101][86122] Updated weights for policy 1, policy_version 95710 (0.0007) +[2023-10-09 15:57:47,261][86121] Updated weights for policy 0, policy_version 95330 (0.0008) +[2023-10-09 15:57:47,624][86121] Updated weights for policy 0, policy_version 95340 (0.0008) +[2023-10-09 15:57:47,987][86121] Updated weights for policy 0, policy_version 95350 (0.0009) +[2023-10-09 15:57:48,348][86121] Updated weights for policy 0, policy_version 95360 (0.0009) +[2023-10-09 15:57:48,397][85186] Fps is (10 sec: 16384.3, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195657728. Throughput: 0: 1821.3, 1: 1844.0. Samples: 48920376. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:48,398][85186] Avg episode reward: [(0, '9.970'), (1, '9.980')] +[2023-10-09 15:57:48,733][86122] Updated weights for policy 1, policy_version 95720 (0.0008) +[2023-10-09 15:57:49,096][86122] Updated weights for policy 1, policy_version 95730 (0.0010) +[2023-10-09 15:57:49,468][86122] Updated weights for policy 1, policy_version 95740 (0.0010) +[2023-10-09 15:57:52,166][86121] Updated weights for policy 0, policy_version 95370 (0.0007) +[2023-10-09 15:57:52,520][86121] Updated weights for policy 0, policy_version 95380 (0.0009) +[2023-10-09 15:57:52,886][86121] Updated weights for policy 0, policy_version 95390 (0.0007) +[2023-10-09 15:57:53,231][86122] Updated weights for policy 1, policy_version 95750 (0.0008) +[2023-10-09 15:57:53,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195723264. Throughput: 0: 1820.7, 1: 1836.4. Samples: 48941764. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 15:57:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:57:53,587][86122] Updated weights for policy 1, policy_version 95760 (0.0007) +[2023-10-09 15:57:53,948][86122] Updated weights for policy 1, policy_version 95770 (0.0009) +[2023-10-09 15:57:56,612][86121] Updated weights for policy 0, policy_version 95400 (0.0010) +[2023-10-09 15:57:56,980][86121] Updated weights for policy 0, policy_version 95410 (0.0009) +[2023-10-09 15:57:57,347][86121] Updated weights for policy 0, policy_version 95420 (0.0010) +[2023-10-09 15:57:57,613][86122] Updated weights for policy 1, policy_version 95780 (0.0009) +[2023-10-09 15:57:57,975][86122] Updated weights for policy 1, policy_version 95790 (0.0010) +[2023-10-09 15:57:58,336][86122] Updated weights for policy 1, policy_version 95800 (0.0010) +[2023-10-09 15:57:58,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 195788800. Throughput: 0: 1818.6, 1: 1832.9. Samples: 48953016. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:57:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:58:00,956][86121] Updated weights for policy 0, policy_version 95430 (0.0008) +[2023-10-09 15:58:01,321][86121] Updated weights for policy 0, policy_version 95440 (0.0010) +[2023-10-09 15:58:01,678][86121] Updated weights for policy 0, policy_version 95450 (0.0010) +[2023-10-09 15:58:02,138][86122] Updated weights for policy 1, policy_version 95810 (0.0008) +[2023-10-09 15:58:02,523][86122] Updated weights for policy 1, policy_version 95820 (0.0009) +[2023-10-09 15:58:02,879][86122] Updated weights for policy 1, policy_version 95830 (0.0008) +[2023-10-09 15:58:03,242][86122] Updated weights for policy 1, policy_version 95840 (0.0007) +[2023-10-09 15:58:03,397][85186] Fps is (10 sec: 16384.1, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 195887104. Throughput: 0: 1810.9, 1: 1830.9. Samples: 48974372. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:58:05,459][86121] Updated weights for policy 0, policy_version 95460 (0.0011) +[2023-10-09 15:58:05,812][86121] Updated weights for policy 0, policy_version 95470 (0.0011) +[2023-10-09 15:58:06,174][86121] Updated weights for policy 0, policy_version 95480 (0.0010) +[2023-10-09 15:58:07,043][86122] Updated weights for policy 1, policy_version 95850 (0.0007) +[2023-10-09 15:58:07,405][86122] Updated weights for policy 1, policy_version 95860 (0.0009) +[2023-10-09 15:58:07,766][86122] Updated weights for policy 1, policy_version 95870 (0.0009) +[2023-10-09 15:58:08,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 195952640. Throughput: 0: 1823.6, 1: 1822.6. Samples: 48995822. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:58:09,831][86121] Updated weights for policy 0, policy_version 95490 (0.0007) +[2023-10-09 15:58:10,209][86121] Updated weights for policy 0, policy_version 95500 (0.0008) +[2023-10-09 15:58:10,575][86121] Updated weights for policy 0, policy_version 95510 (0.0008) +[2023-10-09 15:58:10,943][86121] Updated weights for policy 0, policy_version 95520 (0.0009) +[2023-10-09 15:58:11,532][86122] Updated weights for policy 1, policy_version 95880 (0.0010) +[2023-10-09 15:58:11,891][86122] Updated weights for policy 1, policy_version 95890 (0.0010) +[2023-10-09 15:58:12,256][86122] Updated weights for policy 1, policy_version 95900 (0.0009) +[2023-10-09 15:58:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196018176. Throughput: 0: 1815.0, 1: 1821.2. Samples: 49007164. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:58:14,759][86121] Updated weights for policy 0, policy_version 95530 (0.0007) +[2023-10-09 15:58:15,117][86121] Updated weights for policy 0, policy_version 95540 (0.0007) +[2023-10-09 15:58:15,478][86121] Updated weights for policy 0, policy_version 95550 (0.0008) +[2023-10-09 15:58:15,908][86122] Updated weights for policy 1, policy_version 95910 (0.0009) +[2023-10-09 15:58:16,269][86122] Updated weights for policy 1, policy_version 95920 (0.0010) +[2023-10-09 15:58:16,634][86122] Updated weights for policy 1, policy_version 95930 (0.0007) +[2023-10-09 15:58:18,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 196083712. Throughput: 0: 1821.4, 1: 1823.5. Samples: 49028690. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:58:19,190][86121] Updated weights for policy 0, policy_version 95560 (0.0008) +[2023-10-09 15:58:19,562][86121] Updated weights for policy 0, policy_version 95570 (0.0010) +[2023-10-09 15:58:19,931][86121] Updated weights for policy 0, policy_version 95580 (0.0011) +[2023-10-09 15:58:20,350][86122] Updated weights for policy 1, policy_version 95940 (0.0008) +[2023-10-09 15:58:20,720][86122] Updated weights for policy 1, policy_version 95950 (0.0009) +[2023-10-09 15:58:21,084][86122] Updated weights for policy 1, policy_version 95960 (0.0009) +[2023-10-09 15:58:23,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196149248. Throughput: 0: 1812.0, 1: 1824.3. Samples: 49051342. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:58:23,682][86121] Updated weights for policy 0, policy_version 95590 (0.0010) +[2023-10-09 15:58:24,050][86121] Updated weights for policy 0, policy_version 95600 (0.0010) +[2023-10-09 15:58:24,422][86121] Updated weights for policy 0, policy_version 95610 (0.0009) +[2023-10-09 15:58:24,634][86122] Updated weights for policy 1, policy_version 95970 (0.0009) +[2023-10-09 15:58:25,006][86122] Updated weights for policy 1, policy_version 95980 (0.0008) +[2023-10-09 15:58:25,358][86122] Updated weights for policy 1, policy_version 95990 (0.0009) +[2023-10-09 15:58:25,724][86122] Updated weights for policy 1, policy_version 96000 (0.0010) +[2023-10-09 15:58:28,050][86121] Updated weights for policy 0, policy_version 95620 (0.0009) +[2023-10-09 15:58:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196214784. Throughput: 0: 1807.7, 1: 1831.3. Samples: 49061282. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.980')] +[2023-10-09 15:58:28,427][86121] Updated weights for policy 0, policy_version 95630 (0.0009) +[2023-10-09 15:58:28,788][86121] Updated weights for policy 0, policy_version 95640 (0.0008) +[2023-10-09 15:58:29,235][86122] Updated weights for policy 1, policy_version 96010 (0.0007) +[2023-10-09 15:58:29,598][86122] Updated weights for policy 1, policy_version 96020 (0.0010) +[2023-10-09 15:58:29,962][86122] Updated weights for policy 1, policy_version 96030 (0.0007) +[2023-10-09 15:58:32,477][86121] Updated weights for policy 0, policy_version 95650 (0.0008) +[2023-10-09 15:58:32,837][86121] Updated weights for policy 0, policy_version 95660 (0.0008) +[2023-10-09 15:58:33,201][86121] Updated weights for policy 0, policy_version 95670 (0.0009) +[2023-10-09 15:58:33,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196280320. Throughput: 0: 1813.7, 1: 1830.6. Samples: 49084372. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:58:33,572][86121] Updated weights for policy 0, policy_version 95680 (0.0008) +[2023-10-09 15:58:33,639][86122] Updated weights for policy 1, policy_version 96040 (0.0009) +[2023-10-09 15:58:34,007][86122] Updated weights for policy 1, policy_version 96050 (0.0010) +[2023-10-09 15:58:34,368][86122] Updated weights for policy 1, policy_version 96060 (0.0009) +[2023-10-09 15:58:37,286][86121] Updated weights for policy 0, policy_version 95690 (0.0010) +[2023-10-09 15:58:37,639][86121] Updated weights for policy 0, policy_version 95700 (0.0010) +[2023-10-09 15:58:38,001][86121] Updated weights for policy 0, policy_version 95710 (0.0009) +[2023-10-09 15:58:38,019][86122] Updated weights for policy 1, policy_version 96070 (0.0008) +[2023-10-09 15:58:38,386][86122] Updated weights for policy 1, policy_version 96080 (0.0010) +[2023-10-09 15:58:38,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196378624. Throughput: 0: 1813.6, 1: 1833.6. Samples: 49105888. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:38,399][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:58:38,735][86122] Updated weights for policy 1, policy_version 96090 (0.0008) +[2023-10-09 15:58:41,949][86121] Updated weights for policy 0, policy_version 95720 (0.0010) +[2023-10-09 15:58:42,319][86121] Updated weights for policy 0, policy_version 95730 (0.0008) +[2023-10-09 15:58:42,399][86122] Updated weights for policy 1, policy_version 96100 (0.0008) +[2023-10-09 15:58:42,684][86121] Updated weights for policy 0, policy_version 95740 (0.0007) +[2023-10-09 15:58:42,750][86122] Updated weights for policy 1, policy_version 96110 (0.0010) +[2023-10-09 15:58:43,114][86122] Updated weights for policy 1, policy_version 96120 (0.0007) +[2023-10-09 15:58:43,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 196476928. Throughput: 0: 1807.7, 1: 1837.4. Samples: 49117044. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '9.990')] +[2023-10-09 15:58:46,465][86121] Updated weights for policy 0, policy_version 95750 (0.0009) +[2023-10-09 15:58:46,661][86122] Updated weights for policy 1, policy_version 96130 (0.0007) +[2023-10-09 15:58:46,837][86121] Updated weights for policy 0, policy_version 95760 (0.0008) +[2023-10-09 15:58:47,018][86122] Updated weights for policy 1, policy_version 96140 (0.0008) +[2023-10-09 15:58:47,195][86121] Updated weights for policy 0, policy_version 95770 (0.0007) +[2023-10-09 15:58:47,371][86122] Updated weights for policy 1, policy_version 96150 (0.0008) +[2023-10-09 15:58:47,725][86122] Updated weights for policy 1, policy_version 96160 (0.0009) +[2023-10-09 15:58:48,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196542464. Throughput: 0: 1821.9, 1: 1829.6. Samples: 49138690. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:58:50,864][86121] Updated weights for policy 0, policy_version 95780 (0.0008) +[2023-10-09 15:58:51,241][86121] Updated weights for policy 0, policy_version 95790 (0.0008) +[2023-10-09 15:58:51,588][86122] Updated weights for policy 1, policy_version 96170 (0.0008) +[2023-10-09 15:58:51,605][86121] Updated weights for policy 0, policy_version 95800 (0.0008) +[2023-10-09 15:58:51,948][86122] Updated weights for policy 1, policy_version 96180 (0.0008) +[2023-10-09 15:58:52,310][86122] Updated weights for policy 1, policy_version 96190 (0.0008) +[2023-10-09 15:58:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196608000. Throughput: 0: 1802.0, 1: 1836.4. Samples: 49159550. Policy #0 lag: (min: 15.0, avg: 15.0, max: 15.0) +[2023-10-09 15:58:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:58:55,216][86121] Updated weights for policy 0, policy_version 95810 (0.0007) +[2023-10-09 15:58:55,579][86121] Updated weights for policy 0, policy_version 95820 (0.0008) +[2023-10-09 15:58:55,942][86121] Updated weights for policy 0, policy_version 95830 (0.0008) +[2023-10-09 15:58:56,168][86122] Updated weights for policy 1, policy_version 96200 (0.0007) +[2023-10-09 15:58:56,306][86121] Updated weights for policy 0, policy_version 95840 (0.0008) +[2023-10-09 15:58:56,533][86122] Updated weights for policy 1, policy_version 96210 (0.0008) +[2023-10-09 15:58:56,901][86122] Updated weights for policy 1, policy_version 96220 (0.0009) +[2023-10-09 15:58:58,397][85186] Fps is (10 sec: 13107.0, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 196673536. Throughput: 0: 1812.6, 1: 1834.3. Samples: 49171276. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:58:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:00,087][86121] Updated weights for policy 0, policy_version 95850 (0.0010) +[2023-10-09 15:59:00,446][86121] Updated weights for policy 0, policy_version 95860 (0.0011) +[2023-10-09 15:59:00,760][86122] Updated weights for policy 1, policy_version 96230 (0.0008) +[2023-10-09 15:59:00,814][86121] Updated weights for policy 0, policy_version 95870 (0.0008) +[2023-10-09 15:59:01,123][86122] Updated weights for policy 1, policy_version 96240 (0.0007) +[2023-10-09 15:59:01,484][86122] Updated weights for policy 1, policy_version 96250 (0.0007) +[2023-10-09 15:59:03,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196739072. Throughput: 0: 1799.2, 1: 1821.4. Samples: 49191620. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:04,241][86121] Updated weights for policy 0, policy_version 95880 (0.0008) +[2023-10-09 15:59:04,599][86121] Updated weights for policy 0, policy_version 95890 (0.0010) +[2023-10-09 15:59:04,965][86121] Updated weights for policy 0, policy_version 95900 (0.0010) +[2023-10-09 15:59:05,103][86122] Updated weights for policy 1, policy_version 96260 (0.0008) +[2023-10-09 15:59:05,463][86122] Updated weights for policy 1, policy_version 96270 (0.0009) +[2023-10-09 15:59:05,823][86122] Updated weights for policy 1, policy_version 96280 (0.0008) +[2023-10-09 15:59:08,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196804608. Throughput: 0: 1811.5, 1: 1822.9. Samples: 49214890. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:08,700][86121] Updated weights for policy 0, policy_version 95910 (0.0008) +[2023-10-09 15:59:09,075][86121] Updated weights for policy 0, policy_version 95920 (0.0008) +[2023-10-09 15:59:09,452][86121] Updated weights for policy 0, policy_version 95930 (0.0008) +[2023-10-09 15:59:09,479][86122] Updated weights for policy 1, policy_version 96290 (0.0010) +[2023-10-09 15:59:09,839][86122] Updated weights for policy 1, policy_version 96300 (0.0007) +[2023-10-09 15:59:10,200][86122] Updated weights for policy 1, policy_version 96310 (0.0009) +[2023-10-09 15:59:10,552][86122] Updated weights for policy 1, policy_version 96320 (0.0007) +[2023-10-09 15:59:13,178][86121] Updated weights for policy 0, policy_version 95940 (0.0008) +[2023-10-09 15:59:13,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196870144. Throughput: 0: 1809.5, 1: 1821.0. Samples: 49224654. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:13,539][86121] Updated weights for policy 0, policy_version 95950 (0.0010) +[2023-10-09 15:59:13,907][86121] Updated weights for policy 0, policy_version 95960 (0.0009) +[2023-10-09 15:59:14,295][86122] Updated weights for policy 1, policy_version 96330 (0.0007) +[2023-10-09 15:59:14,658][86122] Updated weights for policy 1, policy_version 96340 (0.0010) +[2023-10-09 15:59:15,028][86122] Updated weights for policy 1, policy_version 96350 (0.0008) +[2023-10-09 15:59:17,554][86121] Updated weights for policy 0, policy_version 95970 (0.0007) +[2023-10-09 15:59:17,933][86121] Updated weights for policy 0, policy_version 95980 (0.0008) +[2023-10-09 15:59:18,294][86121] Updated weights for policy 0, policy_version 95990 (0.0008) +[2023-10-09 15:59:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 196935680. Throughput: 0: 1802.4, 1: 1822.7. Samples: 49247502. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:18,504][86122] Updated weights for policy 1, policy_version 96360 (0.0008) +[2023-10-09 15:59:18,657][86121] Updated weights for policy 0, policy_version 96000 (0.0008) +[2023-10-09 15:59:18,869][86122] Updated weights for policy 1, policy_version 96370 (0.0008) +[2023-10-09 15:59:19,224][86122] Updated weights for policy 1, policy_version 96380 (0.0010) +[2023-10-09 15:59:22,541][86121] Updated weights for policy 0, policy_version 96010 (0.0008) +[2023-10-09 15:59:22,863][86122] Updated weights for policy 1, policy_version 96390 (0.0009) +[2023-10-09 15:59:22,908][86121] Updated weights for policy 0, policy_version 96020 (0.0007) +[2023-10-09 15:59:23,227][86122] Updated weights for policy 1, policy_version 96400 (0.0008) +[2023-10-09 15:59:23,267][86121] Updated weights for policy 0, policy_version 96030 (0.0007) +[2023-10-09 15:59:23,397][85186] Fps is (10 sec: 16383.6, 60 sec: 14745.5, 300 sec: 14662.3). Total num frames: 197033984. Throughput: 0: 1815.2, 1: 1819.2. Samples: 49269436. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:23,399][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:59:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000096032_98336768.pth... +[2023-10-09 15:59:23,440][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000094304_96567296.pth +[2023-10-09 15:59:23,585][86122] Updated weights for policy 1, policy_version 96410 (0.0007) +[2023-10-09 15:59:23,793][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000096416_98729984.pth... +[2023-10-09 15:59:23,834][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000094688_96960512.pth +[2023-10-09 15:59:26,961][86121] Updated weights for policy 0, policy_version 96040 (0.0008) +[2023-10-09 15:59:27,278][86122] Updated weights for policy 1, policy_version 96420 (0.0008) +[2023-10-09 15:59:27,325][86121] Updated weights for policy 0, policy_version 96050 (0.0008) +[2023-10-09 15:59:27,634][86122] Updated weights for policy 1, policy_version 96430 (0.0008) +[2023-10-09 15:59:27,694][86121] Updated weights for policy 0, policy_version 96060 (0.0007) +[2023-10-09 15:59:27,999][86122] Updated weights for policy 1, policy_version 96440 (0.0010) +[2023-10-09 15:59:28,397][85186] Fps is (10 sec: 19660.9, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 197132288. Throughput: 0: 1809.4, 1: 1822.5. Samples: 49280480. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:28,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:59:31,588][86121] Updated weights for policy 0, policy_version 96070 (0.0007) +[2023-10-09 15:59:31,811][86122] Updated weights for policy 1, policy_version 96450 (0.0010) +[2023-10-09 15:59:31,944][86121] Updated weights for policy 0, policy_version 96080 (0.0007) +[2023-10-09 15:59:32,215][86122] Updated weights for policy 1, policy_version 96460 (0.0009) +[2023-10-09 15:59:32,317][86121] Updated weights for policy 0, policy_version 96090 (0.0008) +[2023-10-09 15:59:32,586][86122] Updated weights for policy 1, policy_version 96470 (0.0008) +[2023-10-09 15:59:32,947][86122] Updated weights for policy 1, policy_version 96480 (0.0008) +[2023-10-09 15:59:33,397][85186] Fps is (10 sec: 16384.5, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 197197824. Throughput: 0: 1812.4, 1: 1823.2. Samples: 49302294. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:33,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 15:59:35,825][86121] Updated weights for policy 0, policy_version 96100 (0.0008) +[2023-10-09 15:59:36,192][86121] Updated weights for policy 0, policy_version 96110 (0.0010) +[2023-10-09 15:59:36,555][86121] Updated weights for policy 0, policy_version 96120 (0.0008) +[2023-10-09 15:59:36,634][86122] Updated weights for policy 1, policy_version 96490 (0.0008) +[2023-10-09 15:59:37,000][86122] Updated weights for policy 1, policy_version 96500 (0.0009) +[2023-10-09 15:59:37,358][86122] Updated weights for policy 1, policy_version 96510 (0.0007) +[2023-10-09 15:59:38,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197263360. Throughput: 0: 1809.9, 1: 1815.8. Samples: 49322708. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:38,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:40,544][86121] Updated weights for policy 0, policy_version 96130 (0.0008) +[2023-10-09 15:59:40,907][86121] Updated weights for policy 0, policy_version 96140 (0.0007) +[2023-10-09 15:59:41,069][86122] Updated weights for policy 1, policy_version 96520 (0.0010) +[2023-10-09 15:59:41,273][86121] Updated weights for policy 0, policy_version 96150 (0.0008) +[2023-10-09 15:59:41,424][86122] Updated weights for policy 1, policy_version 96530 (0.0007) +[2023-10-09 15:59:41,641][86121] Updated weights for policy 0, policy_version 96160 (0.0008) +[2023-10-09 15:59:41,784][86122] Updated weights for policy 1, policy_version 96540 (0.0008) +[2023-10-09 15:59:43,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14662.3). Total num frames: 197328896. Throughput: 0: 1814.3, 1: 1820.3. Samples: 49334834. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:45,231][86121] Updated weights for policy 0, policy_version 96170 (0.0008) +[2023-10-09 15:59:45,456][86122] Updated weights for policy 1, policy_version 96550 (0.0008) +[2023-10-09 15:59:45,593][86121] Updated weights for policy 0, policy_version 96180 (0.0009) +[2023-10-09 15:59:45,813][86122] Updated weights for policy 1, policy_version 96560 (0.0011) +[2023-10-09 15:59:45,965][86121] Updated weights for policy 0, policy_version 96190 (0.0008) +[2023-10-09 15:59:46,176][86122] Updated weights for policy 1, policy_version 96570 (0.0010) +[2023-10-09 15:59:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 197394432. Throughput: 0: 1810.3, 1: 1833.7. Samples: 49355598. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:49,680][86121] Updated weights for policy 0, policy_version 96200 (0.0008) +[2023-10-09 15:59:49,785][86122] Updated weights for policy 1, policy_version 96580 (0.0009) +[2023-10-09 15:59:50,052][86121] Updated weights for policy 0, policy_version 96210 (0.0008) +[2023-10-09 15:59:50,145][86122] Updated weights for policy 1, policy_version 96590 (0.0009) +[2023-10-09 15:59:50,409][86121] Updated weights for policy 0, policy_version 96220 (0.0008) +[2023-10-09 15:59:50,498][86122] Updated weights for policy 1, policy_version 96600 (0.0008) +[2023-10-09 15:59:53,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197459968. Throughput: 0: 1794.9, 1: 1833.9. Samples: 49378184. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 15:59:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:54,204][86122] Updated weights for policy 1, policy_version 96610 (0.0008) +[2023-10-09 15:59:54,314][86121] Updated weights for policy 0, policy_version 96230 (0.0007) +[2023-10-09 15:59:54,564][86122] Updated weights for policy 1, policy_version 96620 (0.0007) +[2023-10-09 15:59:54,698][86121] Updated weights for policy 0, policy_version 96240 (0.0007) +[2023-10-09 15:59:54,919][86122] Updated weights for policy 1, policy_version 96630 (0.0007) +[2023-10-09 15:59:55,062][86121] Updated weights for policy 0, policy_version 96250 (0.0008) +[2023-10-09 15:59:55,283][86122] Updated weights for policy 1, policy_version 96640 (0.0007) +[2023-10-09 15:59:58,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 197525504. Throughput: 0: 1796.9, 1: 1833.9. Samples: 49388040. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 15:59:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 15:59:58,689][86121] Updated weights for policy 0, policy_version 96260 (0.0007) +[2023-10-09 15:59:58,946][86122] Updated weights for policy 1, policy_version 96650 (0.0008) +[2023-10-09 15:59:59,055][86121] Updated weights for policy 0, policy_version 96270 (0.0009) +[2023-10-09 15:59:59,305][86122] Updated weights for policy 1, policy_version 96660 (0.0008) +[2023-10-09 15:59:59,422][86121] Updated weights for policy 0, policy_version 96280 (0.0008) +[2023-10-09 15:59:59,665][86122] Updated weights for policy 1, policy_version 96670 (0.0007) +[2023-10-09 16:00:03,309][86121] Updated weights for policy 0, policy_version 96290 (0.0008) +[2023-10-09 16:00:03,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 197591040. Throughput: 0: 1795.0, 1: 1831.9. Samples: 49410712. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:03,427][86122] Updated weights for policy 1, policy_version 96680 (0.0009) +[2023-10-09 16:00:03,679][86121] Updated weights for policy 0, policy_version 96300 (0.0008) +[2023-10-09 16:00:03,793][86122] Updated weights for policy 1, policy_version 96690 (0.0008) +[2023-10-09 16:00:04,038][86121] Updated weights for policy 0, policy_version 96310 (0.0007) +[2023-10-09 16:00:04,148][86122] Updated weights for policy 1, policy_version 96700 (0.0007) +[2023-10-09 16:00:04,405][86121] Updated weights for policy 0, policy_version 96320 (0.0008) +[2023-10-09 16:00:07,665][86122] Updated weights for policy 1, policy_version 96710 (0.0007) +[2023-10-09 16:00:08,023][86122] Updated weights for policy 1, policy_version 96720 (0.0008) +[2023-10-09 16:00:08,202][86121] Updated weights for policy 0, policy_version 96330 (0.0008) +[2023-10-09 16:00:08,382][86122] Updated weights for policy 1, policy_version 96730 (0.0007) +[2023-10-09 16:00:08,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 197656576. Throughput: 0: 1808.2, 1: 1821.9. Samples: 49432790. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:08,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:08,563][86121] Updated weights for policy 0, policy_version 96340 (0.0009) +[2023-10-09 16:00:08,923][86121] Updated weights for policy 0, policy_version 96350 (0.0011) +[2023-10-09 16:00:12,096][86122] Updated weights for policy 1, policy_version 96740 (0.0008) +[2023-10-09 16:00:12,459][86122] Updated weights for policy 1, policy_version 96750 (0.0008) +[2023-10-09 16:00:12,647][86121] Updated weights for policy 0, policy_version 96360 (0.0009) +[2023-10-09 16:00:12,825][86122] Updated weights for policy 1, policy_version 96760 (0.0008) +[2023-10-09 16:00:13,022][86121] Updated weights for policy 0, policy_version 96370 (0.0008) +[2023-10-09 16:00:13,396][86121] Updated weights for policy 0, policy_version 96380 (0.0009) +[2023-10-09 16:00:13,397][85186] Fps is (10 sec: 16384.0, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 197754880. Throughput: 0: 1791.4, 1: 1824.8. Samples: 49443210. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:16,693][86122] Updated weights for policy 1, policy_version 96770 (0.0008) +[2023-10-09 16:00:17,046][86122] Updated weights for policy 1, policy_version 96780 (0.0007) +[2023-10-09 16:00:17,154][86121] Updated weights for policy 0, policy_version 96390 (0.0008) +[2023-10-09 16:00:17,403][86122] Updated weights for policy 1, policy_version 96790 (0.0009) +[2023-10-09 16:00:17,520][86121] Updated weights for policy 0, policy_version 96400 (0.0008) +[2023-10-09 16:00:17,759][86122] Updated weights for policy 1, policy_version 96800 (0.0008) +[2023-10-09 16:00:17,888][86121] Updated weights for policy 0, policy_version 96410 (0.0007) +[2023-10-09 16:00:18,397][85186] Fps is (10 sec: 19660.6, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 197853184. Throughput: 0: 1811.8, 1: 1816.7. Samples: 49465578. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:21,584][86122] Updated weights for policy 1, policy_version 96810 (0.0010) +[2023-10-09 16:00:21,695][86121] Updated weights for policy 0, policy_version 96420 (0.0009) +[2023-10-09 16:00:21,948][86122] Updated weights for policy 1, policy_version 96820 (0.0008) +[2023-10-09 16:00:22,059][86121] Updated weights for policy 0, policy_version 96430 (0.0008) +[2023-10-09 16:00:22,302][86122] Updated weights for policy 1, policy_version 96830 (0.0007) +[2023-10-09 16:00:22,425][86121] Updated weights for policy 0, policy_version 96440 (0.0009) +[2023-10-09 16:00:23,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 197918720. Throughput: 0: 1791.3, 1: 1824.5. Samples: 49485420. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:26,068][86122] Updated weights for policy 1, policy_version 96840 (0.0009) +[2023-10-09 16:00:26,182][86121] Updated weights for policy 0, policy_version 96450 (0.0008) +[2023-10-09 16:00:26,422][86122] Updated weights for policy 1, policy_version 96850 (0.0008) +[2023-10-09 16:00:26,548][86121] Updated weights for policy 0, policy_version 96460 (0.0007) +[2023-10-09 16:00:26,779][86122] Updated weights for policy 1, policy_version 96860 (0.0007) +[2023-10-09 16:00:26,914][86121] Updated weights for policy 0, policy_version 96470 (0.0009) +[2023-10-09 16:00:27,280][86121] Updated weights for policy 0, policy_version 96480 (0.0008) +[2023-10-09 16:00:28,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 197984256. Throughput: 0: 1806.7, 1: 1819.1. Samples: 49497994. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:30,684][86122] Updated weights for policy 1, policy_version 96870 (0.0009) +[2023-10-09 16:00:30,940][86121] Updated weights for policy 0, policy_version 96490 (0.0008) +[2023-10-09 16:00:31,044][86122] Updated weights for policy 1, policy_version 96880 (0.0008) +[2023-10-09 16:00:31,311][86121] Updated weights for policy 0, policy_version 96500 (0.0007) +[2023-10-09 16:00:31,396][86122] Updated weights for policy 1, policy_version 96890 (0.0007) +[2023-10-09 16:00:31,670][86121] Updated weights for policy 0, policy_version 96510 (0.0007) +[2023-10-09 16:00:33,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14662.3). Total num frames: 198049792. Throughput: 0: 1793.8, 1: 1808.7. Samples: 49517710. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:35,158][86122] Updated weights for policy 1, policy_version 96900 (0.0007) +[2023-10-09 16:00:35,289][86121] Updated weights for policy 0, policy_version 96520 (0.0008) +[2023-10-09 16:00:35,516][86122] Updated weights for policy 1, policy_version 96910 (0.0008) +[2023-10-09 16:00:35,648][86121] Updated weights for policy 0, policy_version 96530 (0.0007) +[2023-10-09 16:00:35,877][86122] Updated weights for policy 1, policy_version 96920 (0.0008) +[2023-10-09 16:00:36,014][86121] Updated weights for policy 0, policy_version 96540 (0.0008) +[2023-10-09 16:00:38,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198115328. Throughput: 0: 1798.5, 1: 1802.7. Samples: 49540238. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:38,399][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:39,789][86122] Updated weights for policy 1, policy_version 96930 (0.0007) +[2023-10-09 16:00:39,823][86121] Updated weights for policy 0, policy_version 96550 (0.0009) +[2023-10-09 16:00:40,140][86122] Updated weights for policy 1, policy_version 96940 (0.0007) +[2023-10-09 16:00:40,192][86121] Updated weights for policy 0, policy_version 96560 (0.0009) +[2023-10-09 16:00:40,503][86122] Updated weights for policy 1, policy_version 96950 (0.0009) +[2023-10-09 16:00:40,552][86121] Updated weights for policy 0, policy_version 96570 (0.0008) +[2023-10-09 16:00:40,854][86122] Updated weights for policy 1, policy_version 96960 (0.0009) +[2023-10-09 16:00:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 198180864. Throughput: 0: 1797.3, 1: 1800.2. Samples: 49549930. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:43,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:44,112][86121] Updated weights for policy 0, policy_version 96580 (0.0008) +[2023-10-09 16:00:44,477][86121] Updated weights for policy 0, policy_version 96590 (0.0008) +[2023-10-09 16:00:44,721][86122] Updated weights for policy 1, policy_version 96970 (0.0008) +[2023-10-09 16:00:44,844][86121] Updated weights for policy 0, policy_version 96600 (0.0008) +[2023-10-09 16:00:45,078][86122] Updated weights for policy 1, policy_version 96980 (0.0010) +[2023-10-09 16:00:45,439][86122] Updated weights for policy 1, policy_version 96990 (0.0008) +[2023-10-09 16:00:48,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198246400. Throughput: 0: 1804.6, 1: 1787.6. Samples: 49572360. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:48,593][86121] Updated weights for policy 0, policy_version 96610 (0.0008) +[2023-10-09 16:00:48,962][86121] Updated weights for policy 0, policy_version 96620 (0.0007) +[2023-10-09 16:00:49,098][86122] Updated weights for policy 1, policy_version 97000 (0.0009) +[2023-10-09 16:00:49,332][86121] Updated weights for policy 0, policy_version 96630 (0.0007) +[2023-10-09 16:00:49,455][86122] Updated weights for policy 1, policy_version 97010 (0.0009) +[2023-10-09 16:00:49,695][86121] Updated weights for policy 0, policy_version 96640 (0.0008) +[2023-10-09 16:00:49,826][86122] Updated weights for policy 1, policy_version 97020 (0.0010) +[2023-10-09 16:00:53,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198311936. Throughput: 0: 1808.7, 1: 1797.0. Samples: 49595044. Policy #0 lag: (min: 22.0, avg: 22.0, max: 22.0) +[2023-10-09 16:00:53,398][86122] Updated weights for policy 1, policy_version 97030 (0.0008) +[2023-10-09 16:00:53,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:53,567][86121] Updated weights for policy 0, policy_version 96650 (0.0009) +[2023-10-09 16:00:53,750][86122] Updated weights for policy 1, policy_version 97040 (0.0008) +[2023-10-09 16:00:53,924][86121] Updated weights for policy 0, policy_version 96660 (0.0007) +[2023-10-09 16:00:54,111][86122] Updated weights for policy 1, policy_version 97050 (0.0008) +[2023-10-09 16:00:54,291][86121] Updated weights for policy 0, policy_version 96670 (0.0008) +[2023-10-09 16:00:57,839][86122] Updated weights for policy 1, policy_version 97060 (0.0007) +[2023-10-09 16:00:58,074][86121] Updated weights for policy 0, policy_version 96680 (0.0010) +[2023-10-09 16:00:58,207][86122] Updated weights for policy 1, policy_version 97070 (0.0009) +[2023-10-09 16:00:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198377472. Throughput: 0: 1807.3, 1: 1787.7. Samples: 49604988. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:00:58,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:00:58,443][86121] Updated weights for policy 0, policy_version 96690 (0.0008) +[2023-10-09 16:00:58,569][86122] Updated weights for policy 1, policy_version 97080 (0.0008) +[2023-10-09 16:00:58,811][86121] Updated weights for policy 0, policy_version 96700 (0.0008) +[2023-10-09 16:01:02,181][86122] Updated weights for policy 1, policy_version 97090 (0.0007) +[2023-10-09 16:01:02,514][86121] Updated weights for policy 0, policy_version 96710 (0.0009) +[2023-10-09 16:01:02,538][86122] Updated weights for policy 1, policy_version 97100 (0.0008) +[2023-10-09 16:01:02,879][86121] Updated weights for policy 0, policy_version 96720 (0.0008) +[2023-10-09 16:01:02,898][86122] Updated weights for policy 1, policy_version 97110 (0.0007) +[2023-10-09 16:01:03,246][86121] Updated weights for policy 0, policy_version 96730 (0.0007) +[2023-10-09 16:01:03,259][86122] Updated weights for policy 1, policy_version 97120 (0.0008) +[2023-10-09 16:01:03,397][85186] Fps is (10 sec: 16383.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 198475776. Throughput: 0: 1805.6, 1: 1796.5. Samples: 49627674. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:03,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:01:06,896][86121] Updated weights for policy 0, policy_version 96740 (0.0008) +[2023-10-09 16:01:07,224][86122] Updated weights for policy 1, policy_version 97130 (0.0007) +[2023-10-09 16:01:07,266][86121] Updated weights for policy 0, policy_version 96750 (0.0008) +[2023-10-09 16:01:07,591][86122] Updated weights for policy 1, policy_version 97140 (0.0008) +[2023-10-09 16:01:07,625][86121] Updated weights for policy 0, policy_version 96760 (0.0007) +[2023-10-09 16:01:07,963][86122] Updated weights for policy 1, policy_version 97150 (0.0008) +[2023-10-09 16:01:08,397][85186] Fps is (10 sec: 19660.3, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 198574080. Throughput: 0: 1813.9, 1: 1793.6. Samples: 49647758. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:08,398][85186] Avg episode reward: [(0, '9.990'), (1, '10.000')] +[2023-10-09 16:01:11,276][86121] Updated weights for policy 0, policy_version 96770 (0.0008) +[2023-10-09 16:01:11,642][86121] Updated weights for policy 0, policy_version 96780 (0.0008) +[2023-10-09 16:01:11,695][86122] Updated weights for policy 1, policy_version 97160 (0.0009) +[2023-10-09 16:01:12,008][86121] Updated weights for policy 0, policy_version 96790 (0.0009) +[2023-10-09 16:01:12,047][86122] Updated weights for policy 1, policy_version 97170 (0.0008) +[2023-10-09 16:01:12,371][86121] Updated weights for policy 0, policy_version 96800 (0.0008) +[2023-10-09 16:01:12,412][86122] Updated weights for policy 1, policy_version 97180 (0.0008) +[2023-10-09 16:01:13,397][85186] Fps is (10 sec: 16384.1, 60 sec: 14745.6, 300 sec: 14662.3). Total num frames: 198639616. Throughput: 0: 1813.0, 1: 1788.2. Samples: 49660048. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:13,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:01:16,073][86121] Updated weights for policy 0, policy_version 96810 (0.0009) +[2023-10-09 16:01:16,196][86122] Updated weights for policy 1, policy_version 97190 (0.0008) +[2023-10-09 16:01:16,442][86121] Updated weights for policy 0, policy_version 96820 (0.0009) +[2023-10-09 16:01:16,557][86122] Updated weights for policy 1, policy_version 97200 (0.0007) +[2023-10-09 16:01:16,808][86121] Updated weights for policy 0, policy_version 96830 (0.0007) +[2023-10-09 16:01:16,913][86122] Updated weights for policy 1, policy_version 97210 (0.0008) +[2023-10-09 16:01:18,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198705152. Throughput: 0: 1812.5, 1: 1801.2. Samples: 49680328. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:18,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:01:20,514][86121] Updated weights for policy 0, policy_version 96840 (0.0008) +[2023-10-09 16:01:20,562][86122] Updated weights for policy 1, policy_version 97220 (0.0010) +[2023-10-09 16:01:20,878][86121] Updated weights for policy 0, policy_version 96850 (0.0008) +[2023-10-09 16:01:20,924][86122] Updated weights for policy 1, policy_version 97230 (0.0008) +[2023-10-09 16:01:21,239][86121] Updated weights for policy 0, policy_version 96860 (0.0008) +[2023-10-09 16:01:21,289][86122] Updated weights for policy 1, policy_version 97240 (0.0009) +[2023-10-09 16:01:23,397][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198770688. Throughput: 0: 1811.5, 1: 1799.1. Samples: 49702714. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:23,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:01:23,409][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000097248_99581952.pth... +[2023-10-09 16:01:23,409][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000096864_99188736.pth... +[2023-10-09 16:01:23,438][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000095552_97845248.pth +[2023-10-09 16:01:23,441][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000095168_97452032.pth +[2023-10-09 16:01:24,852][86122] Updated weights for policy 1, policy_version 97250 (0.0009) +[2023-10-09 16:01:25,023][86121] Updated weights for policy 0, policy_version 96870 (0.0007) +[2023-10-09 16:01:25,217][86122] Updated weights for policy 1, policy_version 97260 (0.0009) +[2023-10-09 16:01:25,401][86121] Updated weights for policy 0, policy_version 96880 (0.0009) +[2023-10-09 16:01:25,585][86122] Updated weights for policy 1, policy_version 97270 (0.0009) +[2023-10-09 16:01:25,767][86121] Updated weights for policy 0, policy_version 96890 (0.0009) +[2023-10-09 16:01:25,957][86122] Updated weights for policy 1, policy_version 97280 (0.0009) +[2023-10-09 16:01:28,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198836224. Throughput: 0: 1818.6, 1: 1805.5. Samples: 49713012. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:28,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:01:29,378][86121] Updated weights for policy 0, policy_version 96900 (0.0009) +[2023-10-09 16:01:29,735][86122] Updated weights for policy 1, policy_version 97290 (0.0008) +[2023-10-09 16:01:29,746][86121] Updated weights for policy 0, policy_version 96910 (0.0009) +[2023-10-09 16:01:30,099][86122] Updated weights for policy 1, policy_version 97300 (0.0008) +[2023-10-09 16:01:30,109][86121] Updated weights for policy 0, policy_version 96920 (0.0007) +[2023-10-09 16:01:30,450][86122] Updated weights for policy 1, policy_version 97310 (0.0010) +[2023-10-09 16:01:33,397][85186] Fps is (10 sec: 13107.5, 60 sec: 14199.5, 300 sec: 14551.2). Total num frames: 198901760. Throughput: 0: 1810.7, 1: 1806.5. Samples: 49735132. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:33,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:01:33,818][86121] Updated weights for policy 0, policy_version 96930 (0.0009) +[2023-10-09 16:01:34,183][86122] Updated weights for policy 1, policy_version 97320 (0.0007) +[2023-10-09 16:01:34,186][86121] Updated weights for policy 0, policy_version 96940 (0.0007) +[2023-10-09 16:01:34,545][86121] Updated weights for policy 0, policy_version 96950 (0.0008) +[2023-10-09 16:01:34,546][86122] Updated weights for policy 1, policy_version 97330 (0.0008) +[2023-10-09 16:01:34,901][86122] Updated weights for policy 1, policy_version 97340 (0.0007) +[2023-10-09 16:01:34,906][86121] Updated weights for policy 0, policy_version 96960 (0.0009) +[2023-10-09 16:01:38,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 198967296. Throughput: 0: 1810.9, 1: 1813.9. Samples: 49758158. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:38,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:01:38,545][86121] Updated weights for policy 0, policy_version 96970 (0.0009) +[2023-10-09 16:01:38,557][86122] Updated weights for policy 1, policy_version 97350 (0.0009) +[2023-10-09 16:01:38,910][86121] Updated weights for policy 0, policy_version 96980 (0.0009) +[2023-10-09 16:01:38,910][86122] Updated weights for policy 1, policy_version 97360 (0.0008) +[2023-10-09 16:01:39,267][86122] Updated weights for policy 1, policy_version 97370 (0.0007) +[2023-10-09 16:01:39,279][86121] Updated weights for policy 0, policy_version 96990 (0.0007) +[2023-10-09 16:01:42,979][86121] Updated weights for policy 0, policy_version 97000 (0.0007) +[2023-10-09 16:01:42,981][86122] Updated weights for policy 1, policy_version 97380 (0.0008) +[2023-10-09 16:01:43,333][86122] Updated weights for policy 1, policy_version 97390 (0.0010) +[2023-10-09 16:01:43,345][86121] Updated weights for policy 0, policy_version 97010 (0.0007) +[2023-10-09 16:01:43,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199032832. Throughput: 0: 1808.1, 1: 1814.5. Samples: 49768006. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:43,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:01:43,695][86122] Updated weights for policy 1, policy_version 97400 (0.0008) +[2023-10-09 16:01:43,723][86121] Updated weights for policy 0, policy_version 97020 (0.0011) +[2023-10-09 16:01:47,333][86122] Updated weights for policy 1, policy_version 97410 (0.0008) +[2023-10-09 16:01:47,511][86121] Updated weights for policy 0, policy_version 97030 (0.0008) +[2023-10-09 16:01:47,698][86122] Updated weights for policy 1, policy_version 97420 (0.0008) +[2023-10-09 16:01:47,880][86121] Updated weights for policy 0, policy_version 97040 (0.0008) +[2023-10-09 16:01:48,060][86122] Updated weights for policy 1, policy_version 97430 (0.0008) +[2023-10-09 16:01:48,245][86121] Updated weights for policy 0, policy_version 97050 (0.0007) +[2023-10-09 16:01:48,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199098368. Throughput: 0: 1812.9, 1: 1817.4. Samples: 49791038. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:48,398][85186] Avg episode reward: [(0, '9.980'), (1, '10.000')] +[2023-10-09 16:01:48,421][86122] Updated weights for policy 1, policy_version 97440 (0.0007) +[2023-10-09 16:01:51,833][86121] Updated weights for policy 0, policy_version 97060 (0.0009) +[2023-10-09 16:01:52,167][86122] Updated weights for policy 1, policy_version 97450 (0.0008) +[2023-10-09 16:01:52,191][86121] Updated weights for policy 0, policy_version 97070 (0.0008) +[2023-10-09 16:01:52,530][86122] Updated weights for policy 1, policy_version 97460 (0.0007) +[2023-10-09 16:01:52,557][86121] Updated weights for policy 0, policy_version 97080 (0.0008) +[2023-10-09 16:01:52,889][86122] Updated weights for policy 1, policy_version 97470 (0.0007) +[2023-10-09 16:01:53,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 199229440. Throughput: 0: 1810.0, 1: 1818.6. Samples: 49811044. Policy #0 lag: (min: 11.0, avg: 11.0, max: 11.0) +[2023-10-09 16:01:53,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:01:56,425][86121] Updated weights for policy 0, policy_version 97090 (0.0007) +[2023-10-09 16:01:56,620][86122] Updated weights for policy 1, policy_version 97480 (0.0009) +[2023-10-09 16:01:56,793][86121] Updated weights for policy 0, policy_version 97100 (0.0007) +[2023-10-09 16:01:56,986][86122] Updated weights for policy 1, policy_version 97490 (0.0008) +[2023-10-09 16:01:57,155][86121] Updated weights for policy 0, policy_version 97110 (0.0007) +[2023-10-09 16:01:57,346][86122] Updated weights for policy 1, policy_version 97500 (0.0010) +[2023-10-09 16:01:57,515][86121] Updated weights for policy 0, policy_version 97120 (0.0008) +[2023-10-09 16:01:58,397][85186] Fps is (10 sec: 19661.0, 60 sec: 15291.7, 300 sec: 14662.3). Total num frames: 199294976. Throughput: 0: 1808.5, 1: 1824.4. Samples: 49823528. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:01:58,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:02:01,129][86122] Updated weights for policy 1, policy_version 97510 (0.0008) +[2023-10-09 16:02:01,372][86121] Updated weights for policy 0, policy_version 97130 (0.0007) +[2023-10-09 16:02:01,488][86122] Updated weights for policy 1, policy_version 97520 (0.0008) +[2023-10-09 16:02:01,736][86121] Updated weights for policy 0, policy_version 97140 (0.0007) +[2023-10-09 16:02:01,849][86122] Updated weights for policy 1, policy_version 97530 (0.0009) +[2023-10-09 16:02:02,104][86121] Updated weights for policy 0, policy_version 97150 (0.0007) +[2023-10-09 16:02:03,397][85186] Fps is (10 sec: 13106.9, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199360512. Throughput: 0: 1808.6, 1: 1828.3. Samples: 49843988. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:03,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:02:05,695][86122] Updated weights for policy 1, policy_version 97540 (0.0009) +[2023-10-09 16:02:05,758][86121] Updated weights for policy 0, policy_version 97160 (0.0007) +[2023-10-09 16:02:06,059][86122] Updated weights for policy 1, policy_version 97550 (0.0007) +[2023-10-09 16:02:06,123][86121] Updated weights for policy 0, policy_version 97170 (0.0009) +[2023-10-09 16:02:06,418][86122] Updated weights for policy 1, policy_version 97560 (0.0007) +[2023-10-09 16:02:06,488][86121] Updated weights for policy 0, policy_version 97180 (0.0008) +[2023-10-09 16:02:08,398][85186] Fps is (10 sec: 13106.8, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 199426048. Throughput: 0: 1810.7, 1: 1826.5. Samples: 49866388. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:08,399][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:02:09,927][86122] Updated weights for policy 1, policy_version 97570 (0.0009) +[2023-10-09 16:02:10,110][86121] Updated weights for policy 0, policy_version 97190 (0.0009) +[2023-10-09 16:02:10,292][86122] Updated weights for policy 1, policy_version 97580 (0.0007) +[2023-10-09 16:02:10,471][86121] Updated weights for policy 0, policy_version 97200 (0.0009) +[2023-10-09 16:02:10,648][86122] Updated weights for policy 1, policy_version 97590 (0.0009) +[2023-10-09 16:02:10,846][86121] Updated weights for policy 0, policy_version 97210 (0.0009) +[2023-10-09 16:02:11,003][86122] Updated weights for policy 1, policy_version 97600 (0.0009) +[2023-10-09 16:02:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14440.1). Total num frames: 199491584. Throughput: 0: 1811.7, 1: 1831.1. Samples: 49876940. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:02:14,432][86121] Updated weights for policy 0, policy_version 97220 (0.0009) +[2023-10-09 16:02:14,691][86122] Updated weights for policy 1, policy_version 97610 (0.0009) +[2023-10-09 16:02:14,804][86121] Updated weights for policy 0, policy_version 97230 (0.0007) +[2023-10-09 16:02:15,044][86122] Updated weights for policy 1, policy_version 97620 (0.0008) +[2023-10-09 16:02:15,160][86121] Updated weights for policy 0, policy_version 97240 (0.0007) +[2023-10-09 16:02:15,414][86122] Updated weights for policy 1, policy_version 97630 (0.0009) +[2023-10-09 16:02:18,397][85186] Fps is (10 sec: 13107.7, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199557120. Throughput: 0: 1815.1, 1: 1835.4. Samples: 49899406. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:18,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:02:18,723][86121] Updated weights for policy 0, policy_version 97250 (0.0009) +[2023-10-09 16:02:19,090][86121] Updated weights for policy 0, policy_version 97260 (0.0008) +[2023-10-09 16:02:19,106][86122] Updated weights for policy 1, policy_version 97640 (0.0008) +[2023-10-09 16:02:19,455][86121] Updated weights for policy 0, policy_version 97270 (0.0007) +[2023-10-09 16:02:19,470][86122] Updated weights for policy 1, policy_version 97650 (0.0008) +[2023-10-09 16:02:19,808][86121] Updated weights for policy 0, policy_version 97280 (0.0007) +[2023-10-09 16:02:19,823][86122] Updated weights for policy 1, policy_version 97660 (0.0009) +[2023-10-09 16:02:23,397][85186] Fps is (10 sec: 13107.1, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199622656. Throughput: 0: 1824.4, 1: 1826.1. Samples: 49922430. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:23,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:02:23,442][86121] Updated weights for policy 0, policy_version 97290 (0.0008) +[2023-10-09 16:02:23,471][86122] Updated weights for policy 1, policy_version 97670 (0.0009) +[2023-10-09 16:02:23,811][86121] Updated weights for policy 0, policy_version 97300 (0.0007) +[2023-10-09 16:02:23,831][86122] Updated weights for policy 1, policy_version 97680 (0.0008) +[2023-10-09 16:02:24,165][86121] Updated weights for policy 0, policy_version 97310 (0.0008) +[2023-10-09 16:02:24,191][86122] Updated weights for policy 1, policy_version 97690 (0.0008) +[2023-10-09 16:02:27,775][86121] Updated weights for policy 0, policy_version 97320 (0.0008) +[2023-10-09 16:02:28,148][86121] Updated weights for policy 0, policy_version 97330 (0.0008) +[2023-10-09 16:02:28,169][86122] Updated weights for policy 1, policy_version 97700 (0.0009) +[2023-10-09 16:02:28,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 199688192. Throughput: 0: 1828.5, 1: 1822.2. Samples: 49932288. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:28,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:02:28,506][86121] Updated weights for policy 0, policy_version 97340 (0.0008) +[2023-10-09 16:02:28,529][86122] Updated weights for policy 1, policy_version 97710 (0.0009) +[2023-10-09 16:02:28,894][86122] Updated weights for policy 1, policy_version 97720 (0.0010) +[2023-10-09 16:02:32,239][86121] Updated weights for policy 0, policy_version 97350 (0.0008) +[2023-10-09 16:02:32,586][86122] Updated weights for policy 1, policy_version 97730 (0.0009) +[2023-10-09 16:02:32,611][86121] Updated weights for policy 0, policy_version 97360 (0.0009) +[2023-10-09 16:02:32,939][86122] Updated weights for policy 1, policy_version 97740 (0.0007) +[2023-10-09 16:02:32,974][86121] Updated weights for policy 0, policy_version 97370 (0.0008) +[2023-10-09 16:02:33,298][86122] Updated weights for policy 1, policy_version 97750 (0.0009) +[2023-10-09 16:02:33,397][85186] Fps is (10 sec: 16384.4, 60 sec: 14745.6, 300 sec: 14551.2). Total num frames: 199786496. Throughput: 0: 1826.0, 1: 1810.9. Samples: 49954700. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:33,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:02:33,662][86122] Updated weights for policy 1, policy_version 97760 (0.0009) +[2023-10-09 16:02:36,714][86121] Updated weights for policy 0, policy_version 97380 (0.0008) +[2023-10-09 16:02:37,074][86121] Updated weights for policy 0, policy_version 97390 (0.0007) +[2023-10-09 16:02:37,340][86122] Updated weights for policy 1, policy_version 97770 (0.0009) +[2023-10-09 16:02:37,430][86121] Updated weights for policy 0, policy_version 97400 (0.0007) +[2023-10-09 16:02:37,687][86122] Updated weights for policy 1, policy_version 97780 (0.0007) +[2023-10-09 16:02:38,053][86122] Updated weights for policy 1, policy_version 97790 (0.0008) +[2023-10-09 16:02:38,397][85186] Fps is (10 sec: 19660.8, 60 sec: 15291.8, 300 sec: 14662.3). Total num frames: 199884800. Throughput: 0: 1824.0, 1: 1818.1. Samples: 49974942. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:38,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:02:41,221][86121] Updated weights for policy 0, policy_version 97410 (0.0007) +[2023-10-09 16:02:41,586][86121] Updated weights for policy 0, policy_version 97420 (0.0007) +[2023-10-09 16:02:41,650][86122] Updated weights for policy 1, policy_version 97800 (0.0008) +[2023-10-09 16:02:41,944][86121] Updated weights for policy 0, policy_version 97430 (0.0008) +[2023-10-09 16:02:42,015][86122] Updated weights for policy 1, policy_version 97810 (0.0009) +[2023-10-09 16:02:42,302][86121] Updated weights for policy 0, policy_version 97440 (0.0007) +[2023-10-09 16:02:42,369][86122] Updated weights for policy 1, policy_version 97820 (0.0008) +[2023-10-09 16:02:43,397][85186] Fps is (10 sec: 16383.8, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 199950336. Throughput: 0: 1828.7, 1: 1816.2. Samples: 49987548. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:43,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:02:46,145][86121] Updated weights for policy 0, policy_version 97450 (0.0007) +[2023-10-09 16:02:46,245][86122] Updated weights for policy 1, policy_version 97830 (0.0007) +[2023-10-09 16:02:46,514][86121] Updated weights for policy 0, policy_version 97460 (0.0007) +[2023-10-09 16:02:46,598][86122] Updated weights for policy 1, policy_version 97840 (0.0007) +[2023-10-09 16:02:46,870][86121] Updated weights for policy 0, policy_version 97470 (0.0007) +[2023-10-09 16:02:46,955][86122] Updated weights for policy 1, policy_version 97850 (0.0009) +[2023-10-09 16:02:48,397][85186] Fps is (10 sec: 13106.9, 60 sec: 15291.7, 300 sec: 14551.2). Total num frames: 200015872. Throughput: 0: 1825.9, 1: 1820.3. Samples: 50008066. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:48,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:02:50,504][86121] Updated weights for policy 0, policy_version 97480 (0.0010) +[2023-10-09 16:02:50,579][86122] Updated weights for policy 1, policy_version 97860 (0.0009) +[2023-10-09 16:02:50,862][86121] Updated weights for policy 0, policy_version 97490 (0.0008) +[2023-10-09 16:02:50,938][86122] Updated weights for policy 1, policy_version 97870 (0.0007) +[2023-10-09 16:02:51,231][86121] Updated weights for policy 0, policy_version 97500 (0.0008) +[2023-10-09 16:02:51,295][86122] Updated weights for policy 1, policy_version 97880 (0.0007) +[2023-10-09 16:02:53,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.4, 300 sec: 14551.2). Total num frames: 200081408. Throughput: 0: 1824.1, 1: 1815.4. Samples: 50030168. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:53,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:02:55,092][86121] Updated weights for policy 0, policy_version 97510 (0.0008) +[2023-10-09 16:02:55,146][86122] Updated weights for policy 1, policy_version 97890 (0.0007) +[2023-10-09 16:02:55,466][86121] Updated weights for policy 0, policy_version 97520 (0.0008) +[2023-10-09 16:02:55,504][86122] Updated weights for policy 1, policy_version 97900 (0.0008) +[2023-10-09 16:02:55,837][86121] Updated weights for policy 0, policy_version 97530 (0.0009) +[2023-10-09 16:02:55,856][86122] Updated weights for policy 1, policy_version 97910 (0.0007) +[2023-10-09 16:02:56,217][86122] Updated weights for policy 1, policy_version 97920 (0.0007) +[2023-10-09 16:02:58,397][85186] Fps is (10 sec: 13107.4, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 200146944. Throughput: 0: 1820.5, 1: 1818.8. Samples: 50040706. Policy #0 lag: (min: 29.0, avg: 29.0, max: 29.0) +[2023-10-09 16:02:58,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:02:59,564][86121] Updated weights for policy 0, policy_version 97540 (0.0007) +[2023-10-09 16:02:59,925][86122] Updated weights for policy 1, policy_version 97930 (0.0008) +[2023-10-09 16:02:59,929][86121] Updated weights for policy 0, policy_version 97550 (0.0008) +[2023-10-09 16:03:00,284][86122] Updated weights for policy 1, policy_version 97940 (0.0008) +[2023-10-09 16:03:00,297][86121] Updated weights for policy 0, policy_version 97560 (0.0009) +[2023-10-09 16:03:00,647][86122] Updated weights for policy 1, policy_version 97950 (0.0007) +[2023-10-09 16:03:03,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 200212480. Throughput: 0: 1820.9, 1: 1808.2. Samples: 50062718. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 16:03:03,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:03:03,895][86121] Updated weights for policy 0, policy_version 97570 (0.0010) +[2023-10-09 16:03:04,254][86121] Updated weights for policy 0, policy_version 97580 (0.0009) +[2023-10-09 16:03:04,390][86122] Updated weights for policy 1, policy_version 97960 (0.0008) +[2023-10-09 16:03:04,618][86121] Updated weights for policy 0, policy_version 97590 (0.0007) +[2023-10-09 16:03:04,748][86122] Updated weights for policy 1, policy_version 97970 (0.0008) +[2023-10-09 16:03:04,975][86121] Updated weights for policy 0, policy_version 97600 (0.0007) +[2023-10-09 16:03:05,114][86122] Updated weights for policy 1, policy_version 97980 (0.0010) +[2023-10-09 16:03:08,397][85186] Fps is (10 sec: 13107.3, 60 sec: 14199.6, 300 sec: 14440.1). Total num frames: 200278016. Throughput: 0: 1821.0, 1: 1813.0. Samples: 50085958. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 16:03:08,398][85186] Avg episode reward: [(0, '9.960'), (1, '10.000')] +[2023-10-09 16:03:08,680][86121] Updated weights for policy 0, policy_version 97610 (0.0008) +[2023-10-09 16:03:08,726][86122] Updated weights for policy 1, policy_version 97990 (0.0008) +[2023-10-09 16:03:09,049][86121] Updated weights for policy 0, policy_version 97620 (0.0009) +[2023-10-09 16:03:09,084][86122] Updated weights for policy 1, policy_version 98000 (0.0008) +[2023-10-09 16:03:09,427][86121] Updated weights for policy 0, policy_version 97630 (0.0008) +[2023-10-09 16:03:09,451][86122] Updated weights for policy 1, policy_version 98010 (0.0008) +[2023-10-09 16:03:13,235][86121] Updated weights for policy 0, policy_version 97640 (0.0008) +[2023-10-09 16:03:13,247][86122] Updated weights for policy 1, policy_version 98020 (0.0008) +[2023-10-09 16:03:13,397][85186] Fps is (10 sec: 13107.2, 60 sec: 14199.5, 300 sec: 14440.1). Total num frames: 200343552. Throughput: 0: 1816.4, 1: 1816.8. Samples: 50095784. Policy #0 lag: (min: 31.0, avg: 31.0, max: 31.0) +[2023-10-09 16:03:13,398][85186] Avg episode reward: [(0, '9.970'), (1, '10.000')] +[2023-10-09 16:03:13,609][86122] Updated weights for policy 1, policy_version 98030 (0.0009) +[2023-10-09 16:03:13,613][86121] Updated weights for policy 0, policy_version 97650 (0.0008) +[2023-10-09 16:03:13,970][86122] Updated weights for policy 1, policy_version 98040 (0.0009) +[2023-10-09 16:03:13,980][86121] Updated weights for policy 0, policy_version 97660 (0.0009) +[2023-10-09 16:03:14,255][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000098048_100401152.pth... +[2023-10-09 16:03:14,255][86157] Stopping RolloutWorker_w2... +[2023-10-09 16:03:14,256][86157] Loop rollout_proc2_evt_loop terminating... +[2023-10-09 16:03:14,255][85186] Component RolloutWorker_w2 stopped! +[2023-10-09 16:03:14,256][85763] Stopping Batcher_0... +[2023-10-09 16:03:14,256][85186] Component Batcher_1 stopped! +[2023-10-09 16:03:14,256][85763] Loop batcher_evt_loop terminating... +[2023-10-09 16:03:14,256][85186] Component Batcher_0 stopped! +[2023-10-09 16:03:14,256][86159] Stopping RolloutWorker_w4... +[2023-10-09 16:03:14,257][85186] Component RolloutWorker_w4 stopped! +[2023-10-09 16:03:14,257][86159] Loop rollout_proc4_evt_loop terminating... +[2023-10-09 16:03:14,257][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... +[2023-10-09 16:03:14,258][86160] Stopping RolloutWorker_w6... +[2023-10-09 16:03:14,258][86713] Stopping RolloutWorker_w14... +[2023-10-09 16:03:14,258][85186] Component RolloutWorker_w6 stopped! +[2023-10-09 16:03:14,258][86160] Loop rollout_proc6_evt_loop terminating... +[2023-10-09 16:03:14,258][86713] Loop rollout_proc14_evt_loop terminating... +[2023-10-09 16:03:14,258][85186] Component RolloutWorker_w14 stopped! +[2023-10-09 16:03:14,259][86165] Stopping RolloutWorker_w10... +[2023-10-09 16:03:14,259][86154] Stopping RolloutWorker_w1... +[2023-10-09 16:03:14,259][85186] Component RolloutWorker_w10 stopped! +[2023-10-09 16:03:14,259][86165] Loop rollout_proc10_evt_loop terminating... +[2023-10-09 16:03:14,259][86154] Loop rollout_proc1_evt_loop terminating... +[2023-10-09 16:03:14,259][86167] Stopping RolloutWorker_w8... +[2023-10-09 16:03:14,259][85186] Component RolloutWorker_w1 stopped! +[2023-10-09 16:03:14,260][86155] Stopping RolloutWorker_w0... +[2023-10-09 16:03:14,260][86167] Loop rollout_proc8_evt_loop terminating... +[2023-10-09 16:03:14,260][86155] Loop rollout_proc0_evt_loop terminating... +[2023-10-09 16:03:14,260][85186] Component RolloutWorker_w8 stopped! +[2023-10-09 16:03:14,260][85186] Component RolloutWorker_w0 stopped! +[2023-10-09 16:03:14,260][86158] Stopping RolloutWorker_w3... +[2023-10-09 16:03:14,261][85186] Component RolloutWorker_w3 stopped! +[2023-10-09 16:03:14,261][86158] Loop rollout_proc3_evt_loop terminating... +[2023-10-09 16:03:14,262][85186] Component RolloutWorker_w7 stopped! +[2023-10-09 16:03:14,261][86161] Stopping RolloutWorker_w7... +[2023-10-09 16:03:14,262][86163] Stopping RolloutWorker_w9... +[2023-10-09 16:03:14,262][86745] Stopping RolloutWorker_w15... +[2023-10-09 16:03:14,262][86168] Stopping RolloutWorker_w13... +[2023-10-09 16:03:14,262][86162] Stopping RolloutWorker_w5... +[2023-10-09 16:03:14,262][86166] Stopping RolloutWorker_w12... +[2023-10-09 16:03:14,262][85186] Component RolloutWorker_w9 stopped! +[2023-10-09 16:03:14,262][86163] Loop rollout_proc9_evt_loop terminating... +[2023-10-09 16:03:14,262][86161] Loop rollout_proc7_evt_loop terminating... +[2023-10-09 16:03:14,262][86745] Loop rollout_proc15_evt_loop terminating... +[2023-10-09 16:03:14,262][86168] Loop rollout_proc13_evt_loop terminating... +[2023-10-09 16:03:14,262][86162] Loop rollout_proc5_evt_loop terminating... +[2023-10-09 16:03:14,262][86166] Loop rollout_proc12_evt_loop terminating... +[2023-10-09 16:03:14,262][85186] Component RolloutWorker_w15 stopped! +[2023-10-09 16:03:14,263][85186] Component RolloutWorker_w13 stopped! +[2023-10-09 16:03:14,263][85186] Component RolloutWorker_w5 stopped! +[2023-10-09 16:03:14,263][85186] Component RolloutWorker_w12 stopped! +[2023-10-09 16:03:14,264][85186] Component RolloutWorker_w11 stopped! +[2023-10-09 16:03:14,264][86164] Stopping RolloutWorker_w11... +[2023-10-09 16:03:14,264][86164] Loop rollout_proc11_evt_loop terminating... +[2023-10-09 16:03:14,255][85963] Stopping Batcher_1... +[2023-10-09 16:03:14,279][86121] Weights refcount: 2 0 +[2023-10-09 16:03:14,280][86121] Stopping InferenceWorker_p0-w0... +[2023-10-09 16:03:14,281][86121] Loop inference_proc0-0_evt_loop terminating... +[2023-10-09 16:03:14,280][85186] Component InferenceWorker_p0-w0 stopped! +[2023-10-09 16:03:14,282][86122] Weights refcount: 2 0 +[2023-10-09 16:03:14,284][86122] Stopping InferenceWorker_p1-w0... +[2023-10-09 16:03:14,284][86122] Loop inference_proc1-0_evt_loop terminating... +[2023-10-09 16:03:14,284][85186] Component InferenceWorker_p1-w0 stopped! +[2023-10-09 16:03:14,278][85963] Loop batcher_evt_loop terminating... +[2023-10-09 16:03:14,288][85963] Removing ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000096416_98729984.pth +[2023-10-09 16:03:14,292][85763] Removing ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000096032_98336768.pth +[2023-10-09 16:03:14,293][85963] Saving ./train_atari/atari_bowling_APPO/checkpoint_p1/checkpoint_000098048_100401152.pth... +[2023-10-09 16:03:14,296][85763] Saving ./train_atari/atari_bowling_APPO/checkpoint_p0/checkpoint_000097664_100007936.pth... +[2023-10-09 16:03:14,330][85963] Stopping LearnerWorker_p1... +[2023-10-09 16:03:14,331][85963] Loop learner_proc1_evt_loop terminating... +[2023-10-09 16:03:14,331][85186] Component LearnerWorker_p1 stopped! +[2023-10-09 16:03:14,334][85763] Stopping LearnerWorker_p0... +[2023-10-09 16:03:14,334][85763] Loop learner_proc0_evt_loop terminating... +[2023-10-09 16:03:14,334][85186] Component LearnerWorker_p0 stopped! +[2023-10-09 16:03:14,335][85186] Waiting for process learner_proc0 to stop... +[2023-10-09 16:03:15,151][85186] Waiting for process learner_proc1 to stop... +[2023-10-09 16:03:15,153][85186] Waiting for process inference_proc0-0 to join... +[2023-10-09 16:03:15,153][85186] Waiting for process inference_proc1-0 to join... +[2023-10-09 16:03:15,154][85186] Waiting for process rollout_proc0 to join... +[2023-10-09 16:03:15,155][85186] Waiting for process rollout_proc1 to join... +[2023-10-09 16:03:15,155][85186] Waiting for process rollout_proc2 to join... +[2023-10-09 16:03:15,156][85186] Waiting for process rollout_proc3 to join... +[2023-10-09 16:03:15,157][85186] Waiting for process rollout_proc4 to join... +[2023-10-09 16:03:15,157][85186] Waiting for process rollout_proc5 to join... +[2023-10-09 16:03:15,158][85186] Waiting for process rollout_proc6 to join... +[2023-10-09 16:03:15,158][85186] Waiting for process rollout_proc7 to join... +[2023-10-09 16:03:15,159][85186] Waiting for process rollout_proc8 to join... +[2023-10-09 16:03:15,160][85186] Waiting for process rollout_proc9 to join... +[2023-10-09 16:03:15,160][85186] Waiting for process rollout_proc10 to join... +[2023-10-09 16:03:15,161][85186] Waiting for process rollout_proc11 to join... +[2023-10-09 16:03:15,162][85186] Waiting for process rollout_proc12 to join... +[2023-10-09 16:03:15,162][85186] Waiting for process rollout_proc13 to join... +[2023-10-09 16:03:15,163][85186] Waiting for process rollout_proc14 to join... +[2023-10-09 16:03:15,163][85186] Waiting for process rollout_proc15 to join... +[2023-10-09 16:03:15,163][85186] Batcher 0 profile tree view: +batching: 169.2941, releasing_batches: 0.0907 +[2023-10-09 16:03:15,163][85186] Batcher 1 profile tree view: +batching: 169.5836, releasing_batches: 0.0887 +[2023-10-09 16:03:15,164][85186] InferenceWorker_p0-w0 profile tree view: +wait_policy: 0.0000 + wait_policy_total: 1845.7884 +update_model: 196.4145 + weight_update: 0.0009 +one_step: 0.0025 + handle_policy_step: 11070.7990 + deserialize: 61.7450, stack: 188.6516, obs_to_device_normalize: 2475.9746, forward: 4984.3434, prepare_outputs: 2406.2789, send_messages: 469.1716 +[2023-10-09 16:03:15,164][85186] InferenceWorker_p1-w0 profile tree view: +wait_policy: 0.0001 + wait_policy_total: 1764.9336 +update_model: 203.0864 + weight_update: 0.0008 +one_step: 0.0024 + handle_policy_step: 11152.4231 + deserialize: 61.9450, stack: 187.7959, obs_to_device_normalize: 2494.2503, forward: 5041.5993, prepare_outputs: 2432.5217, send_messages: 458.5282 +[2023-10-09 16:03:15,164][85186] Learner 0 profile tree view: +misc: 0.0172, prepare_batch: 267.7274 +train: 3595.3363 + epoch_init: 0.1878, minibatch_init: 12.9673, losses_postprocess: 885.8009, kl_divergence: 30.5989, update: 384.5836, after_optimizer: 2095.3812 + calculate_losses: 169.3130 + losses_init: 0.3956, forward_head: 59.5113, bptt_initial: 1.4220, bptt: 2.0273, tail: 37.9455, advantages_returns: 11.0147, losses: 43.4708 +[2023-10-09 16:03:15,164][85186] Learner 1 profile tree view: +misc: 0.0187, prepare_batch: 268.4289 +train: 3570.5814 + epoch_init: 0.1889, minibatch_init: 13.3102, losses_postprocess: 875.2231, kl_divergence: 30.5304, update: 389.3589, after_optimizer: 2079.1550 + calculate_losses: 166.2555 + losses_init: 0.3770, forward_head: 55.8252, bptt_initial: 1.4366, bptt: 2.0625, tail: 38.0068, advantages_returns: 11.0320, losses: 43.9279 +[2023-10-09 16:03:15,165][85186] RolloutWorker_w0 profile tree view: +wait_for_trajectories: 1.2416, enqueue_policy_requests: 402.1637, process_policy_outputs: 189.0630, env_step: 6572.1368, finalize_trajectories: 3.4923, complete_rollouts: 2.8625 +post_env_step: 369.1748 + process_env_step: 82.4450 +[2023-10-09 16:03:15,165][85186] RolloutWorker_w15 profile tree view: +wait_for_trajectories: 1.2318, enqueue_policy_requests: 404.5108, process_policy_outputs: 191.7585, env_step: 6414.4386, finalize_trajectories: 3.5254, complete_rollouts: 2.9150 +post_env_step: 372.2549 + process_env_step: 82.8069 +[2023-10-09 16:03:15,165][85186] Loop Runner_EvtLoop terminating... +[2023-10-09 16:03:15,166][85186] Runner profile tree view: +main_loop: 13787.2366 +[2023-10-09 16:03:15,166][85186] Collected {0: 100007936, 1: 100401152}, FPS: 14535.8